Multi-class API + Endpoints Proto Datastore - google-app-engine

When separating the API classes into multiple files, the API explorer shows the same request definition for all resources.
So based on the structure shown below (my apologies if it's too long), in the API explorer, both my_api.api_a.test and my_api.api_b.test show the same attribute, attr_b, which is the last in the api_server list definition. If I change it and put ApiA last, then both methods show attr_a.
Any idea what am I doing wrong
# model/model_a.py
class A(EndpointsModel):
attr_a = ndb.StringProperty()
# model/model_b.py
class B(EndpointsModel):
attr_b = ndb.StringProperty()
# api/__init__.py
my_api = endpoints.api(name='my_api', version='v1')
# api/api_a.py
#my_api.api_class(resource_name='api_a')
class ApiA(remote.Service):
#A.method(name='test', ...)
...
# api/api_b.py
#my_api.api_class(resource_name='api_b')
class ApiB(remote.Service):
#B.method(name='test', ...)
...
# services.py
from api import my_api
application = endpoints.api_server([ApiA, ApiB])
Also tried to define the api_server as shown below, but didn't work at all.
application = endpoints.api_server([my_api])

I've noticed similar issues (which might be a bug in the endpoints-proto-datastore libary) when the actual method names (not the name in the decorator) are the same in different api classes.
Doesn't work:
class ApiA(remote.Service):
#A.method(...)
def test(self, model):
...
class ApiB(remote.Service):
#B.method(...)
def test(self, model):
...
Works:
class ApiA(remote.Service):
#A.method(...)
def test_a(self, model):
...
class ApiB(remote.Service):
#B.method(...)
def test_b(self, model):
...
You skipped those lines in your sample, but the behaviour you state matches what I encountered in this scenario.

Related

Django 1.11.23 - calling superclass methods in a script

I'm exporting data from a legacy Django application and have written a script that imports the old models.py, which are something like this:
class GeneralType(models.Model):
...
publications = models.ManyToManyField(Publication, blank=True)
class Publication(models.Model):
...
authors = models.TextField()
url = models.URLField(blank=True)
class Record(GeneralType):
...
name = models.CharField(max_length=128)
So, this should work:
for r in Record.objects.all():
for p in r.publications.all(): # Exception thrown here
print(p.url)
But instead it gives:
django.core.exceptions.FieldError: Cannot resolve keyword 'generaltype' into field. Choices are: authors, url
So, there's some issue with calling GeneralType methods on a Record instance, presumably related to having abstracted this out to a script, as everything is fine if similar code is run in the Django app.
There's probably something obvious I've missed here - would happen to know what it is?

Django TypeError when trying to use custom base ModelForm or custom error_class

I wanted to have for error_class rendering in forms. I saw this definition and put it into a file in my app directory:
from django.forms.util import ErrorList
class DivErrorList(ErrorList):
def __unicode__(self):
return self.as_divs()
def as_divs(self):
if not self: return u''
return u'<div class="errorlist">%s</div>' % \
''.join([u'<div class="error">%s</div>' % e for e in self])
But, when I try to use it in my view:
from sporty import DivErrorList
...
form = LocationForm(request.POST or None, error_class=DivErrorList)
if form.is_valid():
I get this error, when submitting the form with an error:
TypeError: 'module' object is not callable
/usr/local/lib/python2.7/dist-packages/django/forms/forms.py in _clean_fields, line 293.
This is at the form.is_valid() line. If I don't use the error_class, it works fine (only without the desired .
Next, I tried to instead, create a base ModelForm class that uses the DivErrorList in my app directory:
from django.forms import ModelForm
from sporty import DivErrorList
class MyModelForm(ModelForm):
def __init__(self, *args, **kwargs):
kwargs_new = {'error_class': DivErrorList}
kwargs_new.update(kwargs)
super(MyModelForm, self).__init__(*args, **kwargs_new)
and then I defined my ModelForm based on that class and no longer used the error_class argument on the form creation:
from sporty import MyModelForm
from sporty.models import Location
class LocationForm(MyModelForm):
class Meta:
model = Location
Now, when I try to even view the form (not submitting it with any data), I get this error:
TypeError: Error when calling the metaclass bases module.init() takes at most 2 arguments (3 given)
/home/pcm/workspace/sportscaster/sporty/forms.py in , line 5
I'm at a loss on both of these. Any ideas? I'd prefer the latter, as all my forms will want to use for error reporting (I'd like to actually render the form as divs too, as some point.
Googling around, I found a discussion on type errors and metaclass bases. The issue was that I had a class, MyModelForm, in a file MyModelForm.py, and was then importing the module attempting to use it like a class:
from sporty import MyModelForm
The solution was to place MyModelForm class into a file modelforms.py and do:
from sporty.modelforms import MyModelForm
I did the same with DivErrorList, placing the class in the modelforms.py file.

App Engine multiple namespaces

Recently there's been some data structure changes in our app, and we decided to use namespaces to separate different versions of of the data, and a mapreduce task that converts old entities to the new format.
Now that's all fine, but we don't want to always isolate the entire data set we have. The biggest part of our data is stored in a kind that's pretty simple and doesn't need to change often. So we decided to use per-kind namespaces.
Something like:
class Author(ndb.model.Model):
ns = '2'
class Book(ndb.model.Model):
ns = '1'
So, when migrating to version 2, we don't need to convert all our data (and copy all 'Book' kinds to the other namespace), only entities of the 'Author' kind. Then, instead of defining the appengine_config.namespace_manager_default_namespace_for_request, we just the 'namespace' keyword arguments to our queries:
Author.query(namespace=Author.ns).get()
Question: how to store (i.e. put()) the different kinds using these different namespaces? Something like:
# Not an API
Author().put(namespace=Author.ns)
Of course, the above doesn't work. (Yes, I could ask the datastore for an avaliable key in that namespace, and then use that key to store the instance with, but it's an extra API call that I'd like to avoid.)
To solve a problem like this I wrote a decorator as follows:
MY_NS = 'abc'
def in_my_namespace(fn):
"""Decorator: Run the given function in the MY_NS namespace"""
from google.appengine.api import namespace_manager
#functools.wraps(fn)
def wrapper(*args, **kwargs):
orig_ns = namespace_manager.get_namespace()
namespace_manager.set_namespace(MY_NS)
try:
res = fn(*args, **kwargs)
finally: # always drop out of the NS on the way up.
namespace_manager.set_namespace(orig_ns)
return res
return wrapper
So I can simply write, for functions that ought to occur in a separate namespace:
#in_my_namespace
def foo():
Author().put() # put into `my` namespace
Of course, applying this to a system to get the results you desire is a bit beyond the scope of this, but I thought it might be helpful.
EDIT: Using a with context
Here's how to accomplish the above using a with context:
class namespace_of(object):
def __init__(self, namespace):
self.ns = namespace
def __enter__(self):
self.orig_ns = namespace_manager.get_namespace()
namespace_manager.set_namespace(self.ns)
def __exit__(self, type, value, traceback):
namespace_manager.set_namespace(self.orig_ns)
Then elsewhere:
with namespace_of("Hello World"):
Author().put() # put into the `Hello World` namespace
A Model instance will use the namespace you set with the namespace_manager[1] as you can see here: python/google/appengine/ext/db/init.py
What you could do is create a child class of Model which expects a class-level 'ns' attribute to be defined. This sub class then overrides put() and sets the namespace before calling original put and resets the namespace afterwards. Something like this:
'''
class MyModel(db.Model):
ns = None
def put(*args, **kwargs):
if self.ns == None:
raise ValueError('"ns" is not defined for this class.')
original_namespace = namespace_manager.get_namespace()
try:
super(MyModelClass, self).put(*args, **kwargs)
finally:
namespace_manager.set_namespace(original_namespace)
'''
[1] http://code.google.com/appengine/docs/python/multitenancy/multitenancy.html
I don't think that it is possible to avoid the extra API call. Namespaces are encoded into the entity's Key, so in order to change the namespace within which a entity is stored, you need to create a new entity (that has a Key with the new namespace) and copy the old entity's data into it.

Django-nonrel form field for ListField

I'm experimenting with django-nonrel on appengine and trying to use a djangotoolbox.fields.ListField to implement a many-to-many relation. As I read in the documentation a ListField is something that you can use to make a workaround for djamgo-nonrel not supporting many-to-many relations.
This is an excerpt from my model:
class MyClass(models.Model):
field = ListField(models.ForeignKey(AnotherClass))
So if I am getting this right I am creating a list of foreign keys to another class to show a relationship with multiple instances of another class
With this approach everything works fine ... No Exceptions. I can create `MyClass' objects in code and views. But when I try to use the admin interface I get the following error
No form field implemented for <class 'djangotoolbox.fields.ListField'>
So I though I would try something that I haven't done before. Create my own field. Well actually my own form for editing MyClass instances in the admin interface. Here is what I did:
class MyClassForm(ModelForm):
field = fields.MultipleChoiceField(choices=AnotherClass.objects.all(), widget=FilteredSelectMultiple("verbose_name", is_stacked=False))
class Meta:
model = MyClass
then I pass MyClassForm as the form to use to the admin interface
class MyClassAdmin(admin.ModelAdmin):
form = MyClassForm
admin.site.register(MyClass, MyClassAdmin)
I though that this would work but It doesn't. When I go to the admin interface I get the same error as before. Can anyone tell what I am doing wrong here ... or if you have any other suggestions or success stories of using the ListField, SetField, etc. from djangotoolbox.fields in the admin interface it would be very much appreciated.
OK, here is what I did to get this all working ...
I'll start from the beginning
This is what what my model looked like
class MyClass(models.Model):
field = ListField(models.ForeignKey(AnotherClass))
I wanted to be able to use the admin interface to create/edit instances of this model using a multiple select widget for the list field. Therefore, I created some custom classes as follows
class ModelListField(ListField):
def formfield(self, **kwargs):
return FormListField(**kwargs)
class ListFieldWidget(SelectMultiple):
pass
class FormListField(MultipleChoiceField):
"""
This is a custom form field that can display a ModelListField as a Multiple Select GUI element.
"""
widget = ListFieldWidget
def clean(self, value):
#TODO: clean your data in whatever way is correct in your case and return cleaned data instead of just the value
return value
These classes allow the listfield to be used in the admin. Then I created a form to use in the admin site
class MyClassForm(ModelForm):
def __init__(self, *args, **kwargs):
super(MyClasstForm,self).__init__(*args, **kwargs)
self.fields['field'].widget.choices = [(i.pk, i) for i in AnotherClass.objects.all()]
if self.instance.pk:
self.fields['field'].initial = self.instance.field
class Meta:
model = MyClass
After having done this I created a admin model and registered it with the admin site
class MyClassAdmin(admin.ModelAdmin):
form = MyClassForm
def __init__(self, model, admin_site):
super(MyClassAdmin,self).__init__(model, admin_site)
admin.site.register(MyClass, MyClassAdmin)
This is now working in my code. Keep in mind that this approach might not at all be well suited for google_appengine as I am not very adept at how it works and it might create inefficient queries an such.
As far as I understand, you're trying to have a M2M relationship in django-nonrel, which is not an out-of-the-box functionality. For starters, if you want a quick hack, you can go with this simple class and use a CharField to enter foreign keys manually:
class ListFormField(forms.Field):
""" A form field for being able to display a djangotoolbox.fields.ListField. """
widget = ListWidget
def clean(self, value):
return [v.strip() for v in value.split(',') if len(v.strip()) > 0]
But if you want to have a multiple selection from a list of models normally you'd have to use ModelMultipleChoiceField, which is also not functional in django-nonrel. Here's what I've done to emulate a M2M relationship using a MultipleSelectField:
Let's say you have a M2M relationship between 2 classes, SomeClass and AnotherClass respectively. You want to select the relationship on the form for SomeClass. Also I assume you want to hold the references as a ListField in SomeClass. (Naturally you want to create M2M relationships as they're explained here, to prevent exploding indexes if you're working on App Engine).
So you have your models like:
class SomeClass(models.Model):
another_class_ids = ListField(models.PositiveIntegerField(), null=True, blank=True)
#fields go here
class AnotherClass(models.Model):
#fields go here
And in your form:
class SomeClassForm(forms.ModelForm):
#Empty field, will be populated after form is initialized
#Otherwise selection list is not refreshed after new entities are created.
another_class = forms.MultipleChoiceField(required=False)
def __init__(self, *args, **kwargs):
super(SomeClassForm,self).__init__(*args, **kwargs)
self.fields['another_class'].choices = [(item.pk,item) for item in AnotherClass.objects.all()]
if self.instance.pk: #If class is saved, highlight the instances that are related
self.fields['another_class'].initial = self.instance.another_class_ids
def save(self, *args, **kwargs):
self.instance.another_class_ids = self.cleaned_data['another_class']
return super(SomeClassForm, self).save()
class Meta:
model = SomeClass
Hopefully this should get you going for the start, I implemented this functionality for normal forms, adjust it for admin panel shouldn't be that hard.
This could be unrelated but for the admin interface, be sure you have djangotoolbox listed after django.contrib.admin in the settings.. INSTALLED_APPS
You could avoid a custom form class for such usage by inquiring for the model object
class ModelListField(ListField):
def __init__(self, embedded_model=None, *args, **kwargs):
super(ModelListField, self).__init__(*args, **kwargs)
self._model = embedded_model.embedded_model
def formfield(self, **kwargs):
return FormListField(model=self._model, **kwargs)
class ListFieldWidget(SelectMultiple):
pass
class FormListField(MultipleChoiceField):
widget = ListFieldWidget
def __init__(self, model=None, *args, **kwargs):
self._model = model
super(FormListField, self).__init__(*args, **kwargs)
self.widget.choices = [(unicode(i.pk), i) for i in self._model.objects.all()]
def to_python(self, value):
return [self._model.objects.get(pk=key) for key in value]
def clean(self, value):
return value

Django-nonrel in Google App Engine ListField

I am trying to build an example app in Google App Engine using django-nonrel. and am having problems implementing ListField attribute into a model.
I have created an app test_model and have included it as an installed app in my settings. The model.py is:
from django.db import models
from djangotoolbox import *
from dbindexer import *
# Create your models here.
class Example(models.Model):
some_choices = models.ListField('Choice_examples')
notes = models.CharField(max_length='20')
updated_at = models.DateTimeField(auto_now=True)
def __unicode__(self):
return u'%s' % (self.notes)
class Choice_examples(models.Model):
name = models.CharField(max_length='30')
def __unicode__(self):
return u'%s' % (self.name)
The above example gives me:
AttributeError:'module' object has no attribute 'Model'
If I comment out the djangotoolbox import, I get the following :
AttributeError: 'module' object has no attribute 'ListField'
What am I doing wrong here? I can't seem to find any documention as to how to go about using ListField in django-nonrel. Is that because it is supposed to really obvious?
Your imports are smashing each other:
from django.db import models
from djangotoolbox import *
The second import will replace the django.db models with djangotoolbox' empty models module. Using from X import * is a terrible idea in general in Python and produces confusing results like these.
If you're looking to use ListField from djangotoolbox, use:
from djangotoolbox import fields
and refer to the ListField class as fields.ListField.
OK, here is what I did to be able to use ListFields. MyClass the equivalent to your Example class and AnotherClass is the same as your Choice_examples. What I describe will allow you to use ListFields in the admin interface and your self implemented views.
I'll start from the beginning
This is what what my model looks like
class MyClass(models.Model):
field = ListField(models.ForeignKey(AnotherClass))
I wanted to be able to use the admin interface to create/edit instances of this model using a multiple select widget for the list field. Therefore, I created some custom classes as follows
class ModelListField(ListField):
def formfield(self, **kwargs):
return FormListField(**kwargs)
class ListFieldWidget(SelectMultiple):
pass
class FormListField(MultipleChoiceField):
"""
This is a custom form field that can display a ModelListField as a Multiple Select GUI element.
"""
widget = ListFieldWidget
def clean(self, value):
#TODO: clean your data in whatever way is correct in your case and return cleaned data instead of just the value
return value
These classes allow the listfield to be used in the admin. Then I created a form to use in the admin site
class MyClassForm(ModelForm):
def __init__(self, *args, **kwargs):
super(MyClasstForm,self).__init__(*args, **kwargs)
self.fields['field'].widget.choices = [(i.pk, i) for i in AnotherClass.objects.all()]
if self.instance.pk:
self.fields['field'].initial = self.instance.field
class Meta:
model = MyClass
After having done this I created a admin model and registered it with the admin site
class MyClassAdmin(admin.ModelAdmin):
form = MyClassForm
def __init__(self, model, admin_site):
super(MyClassAdmin,self).__init__(model, admin_site)
admin.site.register(MyClass, MyClassAdmin)
This is now working in my code. Keep in mind that this approach might not at all be well suited for google_appengine as I am not very adept at how it works and it might create inefficient queries an such.
I don't know, but try with:
class Choice_examples(models.Model):
name = models.CharField(max_length='30')
def __unicode__(self):
return u'%s' % (self.name)
class Example(models.Model):
some_choices = models.ListField(Choice_examples)
notes = models.CharField(max_length='20')
updated_at = models.DateTimeField(auto_now=True)
def __unicode__(self):
return u'%s' % (self.notes)
Looks like the answer is that you cannot pass an object into fields.ListField.
I have ditched trying to work with ListField as documentation is limited and my coding skills aren't at a level for me to work it out.
Anyone else coming across a similar problem, you should consider create a new model to map the ManyToMany relationships. And if the admin view is important, you should look into the following to display the ManyToMany table inline with any given admin view:
http://docs.djangoproject.com/en/1.2/ref/contrib/admin/#s-working-with-many-to-many-models

Resources