Django, autogenerate one-to-many tables, and database structure - database

I am using django for a website where I have a database with users, people, locations, items and so on. Know i find that I need some extra information that requires one-to-many relations like Aliases for most of these tables.
Should I (1) create a common alias table for all of these by using the content type framework (will probably end up with billions of rows), or should I (2) create a alias table for each of these. If the latter one, how do I auto-create one-to-many table like this by just adding a single line like this
"alias = Ailias()"
in each model. I`m sure I saw an app doing something like that way a while ago, I think is was a reversion app of some kind. Even if the second method is not suited i would love tho understand how to do it. I do not know what to search after to find an explanation of this.
I plan to add Haystack with Solr to this, so method 2 might add much extra work there. But I do not have much experience with it jet, so I might be wrong.
PS: ended up wih method one.

Manage to do what I wanted in method 2, easily generate one-to-many fields. Not sure if this is the easiest way, or the best way. If someone has a better way of doing it, I would love to learn it. I am a long way from a django expert, so I might have meddled with some unnecessary complex stuff to do what I wanted.
This example creates an easy way of adding a one-to-many alias relationship.
Alias Managers
class AliasManagerDescriptor(object):
def __init__(self, model,fkName):
self.model = model
self.fkName = fkName
def __get__(self, instance, owner):
if instance is None:
return AliasManager(self.model,self.fkName)
return AliasManager(self.model, self.fkName, instance)
class AliasManager(models.Manager):
def __init__(self, model,fkName, instance=None):
super(AliasManager, self).__init__()
self.model = model
self.instance = instance
#Name of FK linking this model to linked model
self.fkName=fkName
def get_query_set(self):
"""
Get query set, or only get instances from this model that is linked
to the chosen instance from the linked model if one is chosen
"""
if self.instance is None:
return super(AliasManager, self).get_query_set()
if isinstance(self.instance._meta.pk, models.OneToOneField):
#TODO: Checkif this part works, not checked
filter = {self.instance._meta.pk.name+"_id":self.instance.pk}
else:
filter = {self.fkName: self.instance.pk}
return super(AliasManager, self).get_query_set().filter(**filter)
def create(self,**kwargs):
"""
Create alias instances. If FK is not given then it is automatically set
to the chosen instance from the linked model
"""
if self.fkName not in kwargs:
kwargs[self.fkName]=self.instance
print kwargs
super(AliasManager, self).create(**kwargs)
Alias Models
class Alias(object):
def contribute_to_class(self, cls, name):
self.manager_name = name
aliasModel = self.create_alias_model(cls)
descriptor = AliasManagerDescriptor(aliasModel,cls._meta.object_name.lower())
setattr(cls, self.manager_name, descriptor)
def create_alias_model(self, model):
"""
Creates a alias model to associate with the model provided.
"""
attrs = {
#'id': models.AutoField(primary_key=True),
"name": models.CharField(max_length=255),
#Not sure which to use of the two next methods
model._meta.object_name.lower(): models.ForeignKey(model),
#model._meta.object_name.lower(): AliasObjectDescriptor(model),
'__unicode__': lambda self: u'%s' % self.name,
'__module__': model.__module__
}
attrs.update(Meta=type('Meta', (), self.get_meta_options(model)))
name = '%s_alias' % model._meta.object_name
return type(name, (models.Model,), attrs)
def get_meta_options(self, model):
"""
Returns a dictionary of fields that will be added to
the Meta inner class.
"""
return {
}
"""class AliasObjectDescriptor(object):
def __init__(self, model):
self.model = model
def __get__(self, instance, owner):
values = (getattr(instance, f.attname) for f in self.model._meta.fields)
return self.model(*values)"""
Person Model - Only need to add "alias = Alias()" to a model to add a one-to-many alias field.
class Person(models.Model):
name = models.CharField(max_length=30,blank=True,null=True)
age = models.IntegerField(blank=True,null=True)
alias = Alias()
Now you I can do something like this:
per = Person(name="Per",age=99)
per.save()
per.alias.create(name="Mr.P")
per_alias = per.alias.all().values_list("name",flat=True)

Related

How to read old property values in a _pre_put_hook

I am trying to implement an ndb model audit so that all changes to properties are stored within each model instance. Here is the code of the _pre_put_hook I chose to implement that.
def _pre_put_hook(self):
# save a history record for updates
if not (self.key is None or self.key.id() is None):
old_object = self.key.get(use_cache=True)
for attr in dir(self):
if not callable(getattr(self, attr)) and not attr.startswith("_"):
if getattr(self, attr) != getattr(old_object, attr):
logging.debug('UPDATE: {0}'.format(attr))
logging.debug('OLD: {0} NEW: {1}'.format(getattr(old_object, attr), getattr(self, attr)))
The problem is old_object is always populated with the same values of the self (object) being updated. How can I access the property values of the old object BEFORE the put() being actually made (_pre_put)?
EDIT:
I realized over time I was doing a bunch of work that didn't need to be done (alot of CPU/memory used copying entire entities and passing them around when may not be needed). Here's the updated version which stores a reference to the original protobuf and only deserializes it if you need it
__original = None # a shadow-copy of this object so we can see what changed... lazily inflated
_original_pb = None # the original encoded Protobuf representation of this entity
#property
def _original(self):
"""
Singleton to deserialize the protobuf into a new entity that looks like the original from database
"""
if not self.__original and self._original_pb:
self.__original = self.__class__._from_pb(self._original_pb)
return self.__original
#classmethod
def _from_pb(cls, pb, set_key=True, ent=None, key=None):
"""
save copy of original pb so we can track if anything changes between puts
"""
entity = super(ChangesetMixin, cls)._from_pb(pb, set_key=set_key, ent=ent, key=key)
if entity._original_pb is None and not entity._projection:
# _from_pb will get called if we unpickle a new object (like when passing through deferred library)
# so if we are being materialized from pb and we don't have a key, then we don't have _original
entity.__original = None
entity._original_pb = pb
return entity
Make a clone of the entity when you first read it:
Copy an entity in Google App Engine datastore in Python without knowing property names at 'compile' time
and put it on the entity itself so it can be referenced later when desired. That way you don't have to do a second datastore read just to make the comparison
We override two different Model methods to make this happen:
#classmethod
def _post_get_hook(cls, key, future):
"""
clone this entity so we can track if anything changes between puts
NOTE: this only gets called after a ndb.Key.get() ... NOT when loaded from a Query
see _from_pb override below to understand the full picture
also note: this gets called after EVERY key.get()... regardless if NDB had cached it already
so that's why we're only doing the clone() if _original is not set...
"""
entity = future.get_result()
if entity is not None and entity._original is None:
entity._original = clone(entity)
#classmethod
def _from_pb(cls, pb, set_key=True, ent=None, key=None):
"""
clone this entity so we can track if anything changes between puts
this is one way to know when an object loads from a datastore QUERY
_post_get_hook only gets called on direct Key.get()
none of the documented hooks are called after query results
SEE: https://code.google.com/p/appengine-ndb-experiment/issues/detail?id=211
"""
entity = super(BaseModel, cls)._from_pb(pb, set_key=set_key, ent=ent, key=key)
if entity.key and entity._original is None:
# _from_pb will get called if we unpickle a new object (like when passing through deferred library)
# so if we are being materialized from pb and we don't have a key, then we don't have _original
entity._original = clone(entity)
return entity

Simple datastore entity with 2 fields that are also unique

All I am trying to produce is an entity that holds a unique username, and a unique device ID, and the ability to return an error if either of these conditions are not met on submission.
The only way I can see is to perform a query within a transaction, then filter the results. This however requires an ancestor (which seems unnecessary for a single simple entity).
What is the best method to go about doing this?
Here is an example that does what you want.
I put 2 entities to show you also how to make relationships
class Person(ndb.Expando):
registration_date = ndb.DateTimeProperty(auto_now_add=True)
#property
def info(self):
info = PersonInfo.query(ancestor=self.key).get()
return info
class PersonInfo(ndb.Expando):
email = ndb.StringProperty()
nick_name = ndb.StringProperty()
edit_date = ndb.DateTimeProperty(auto_now=True)
Later in the controller for register:
class RegisterPersonHandler(webapp2.RequestHandler):
def get(self):
user = users.get_current_user() #Stub here
if not user:
self.redirect(users.create_login_url(self.request.uri), abort=True)
return
person = Person.get_or_insert(user.user_id())
if not self._register(person, user):
# more logging is needed
logging.warning('Warning registration failed')
return
#ndb.transactional()
def _register(self, person, user):
''' Registration process happens here
'''
# check if the person has info and if not create it
info = PersonInfo.query(ancestor=person.key).get()
if not info:
info = PersonInfo(id=user.user_id(), parent=person.key)
info.nick_name = user.nickname()
info.email = user.email()
info.put()
return True
To answer also the comment question:
How can you programatically tell whether the returned entity is a new
or existing one though?
Try checking against a property that is default. Eg creation_date etc.
Though you can also check on something you need or on another entity's existence like I do because I expect the data to be consistent, and if not then create the bond.

Django: Single model for multiple tables

I have a main table
Slideshow
then a site specific table that captures a few extra details for that site.
Site1_Slideshow
In a web app (specific to a site) i want a single model i.e. Slideshow that combines the 2 tables above.
Currently i have the code below, but i dont think this is correct. I cant do things like
s = Slideshow.objects.get(slideshowId=1) as Slideshows only has the properties featurecategory and slideshow. So how can i have an model called Slideshow that is composed of these 2 tables but looks like it was a single db table.
class SlideshowAbstract(models.Model):
slideshowid = models.IntegerField(primary_key=True, db_column=u'SlideshowId') # Field name made lowercase.
headline = models.TextField(db_column=u'Headline') # Field name made lowercase.
class Meta:
db_table = u'Slideshow'
class Slideshow(models.Model):
slideshow = models.OneToOneField(SlideshowAbstract, primary_key=True,db_column=u'SlideshowId')
def __unicode__(self):
return self.slideshow.headline
class Meta:
db_table = u'Site1_Slideshow'
Think i found the solution.
On the Site1_Slideshow you need to add a column for django to use, that i presume is always the same as primary key value.
Its name is SlideshowAbstract_ptr_id
Once that is added you can change the Slideshow model to be
class Slideshow(SlideshowAbstract):
featureCategory = models.ForeignKey(Featurecategory,db_column=u'FeatureCategoryId')
def __unicode__(self):
return self.headline
class Meta:
db_table = u'Site1_Slideshow'
So doable but not the nicest if you are not doing "model first" and already have the schema. Would be good to be able to override the name of the _ptr_id column.
I did try adding the following to Slideshow too see if i could map this ptr col to the primary key
slideshowabstract_ptr_id = models.IntegerField(primary_key=True, db_column=u'SlideshowId')
but no cigar.
I havent tested inserts either but ...objects.all() works

Django-nonrel form field for ListField

I'm experimenting with django-nonrel on appengine and trying to use a djangotoolbox.fields.ListField to implement a many-to-many relation. As I read in the documentation a ListField is something that you can use to make a workaround for djamgo-nonrel not supporting many-to-many relations.
This is an excerpt from my model:
class MyClass(models.Model):
field = ListField(models.ForeignKey(AnotherClass))
So if I am getting this right I am creating a list of foreign keys to another class to show a relationship with multiple instances of another class
With this approach everything works fine ... No Exceptions. I can create `MyClass' objects in code and views. But when I try to use the admin interface I get the following error
No form field implemented for <class 'djangotoolbox.fields.ListField'>
So I though I would try something that I haven't done before. Create my own field. Well actually my own form for editing MyClass instances in the admin interface. Here is what I did:
class MyClassForm(ModelForm):
field = fields.MultipleChoiceField(choices=AnotherClass.objects.all(), widget=FilteredSelectMultiple("verbose_name", is_stacked=False))
class Meta:
model = MyClass
then I pass MyClassForm as the form to use to the admin interface
class MyClassAdmin(admin.ModelAdmin):
form = MyClassForm
admin.site.register(MyClass, MyClassAdmin)
I though that this would work but It doesn't. When I go to the admin interface I get the same error as before. Can anyone tell what I am doing wrong here ... or if you have any other suggestions or success stories of using the ListField, SetField, etc. from djangotoolbox.fields in the admin interface it would be very much appreciated.
OK, here is what I did to get this all working ...
I'll start from the beginning
This is what what my model looked like
class MyClass(models.Model):
field = ListField(models.ForeignKey(AnotherClass))
I wanted to be able to use the admin interface to create/edit instances of this model using a multiple select widget for the list field. Therefore, I created some custom classes as follows
class ModelListField(ListField):
def formfield(self, **kwargs):
return FormListField(**kwargs)
class ListFieldWidget(SelectMultiple):
pass
class FormListField(MultipleChoiceField):
"""
This is a custom form field that can display a ModelListField as a Multiple Select GUI element.
"""
widget = ListFieldWidget
def clean(self, value):
#TODO: clean your data in whatever way is correct in your case and return cleaned data instead of just the value
return value
These classes allow the listfield to be used in the admin. Then I created a form to use in the admin site
class MyClassForm(ModelForm):
def __init__(self, *args, **kwargs):
super(MyClasstForm,self).__init__(*args, **kwargs)
self.fields['field'].widget.choices = [(i.pk, i) for i in AnotherClass.objects.all()]
if self.instance.pk:
self.fields['field'].initial = self.instance.field
class Meta:
model = MyClass
After having done this I created a admin model and registered it with the admin site
class MyClassAdmin(admin.ModelAdmin):
form = MyClassForm
def __init__(self, model, admin_site):
super(MyClassAdmin,self).__init__(model, admin_site)
admin.site.register(MyClass, MyClassAdmin)
This is now working in my code. Keep in mind that this approach might not at all be well suited for google_appengine as I am not very adept at how it works and it might create inefficient queries an such.
As far as I understand, you're trying to have a M2M relationship in django-nonrel, which is not an out-of-the-box functionality. For starters, if you want a quick hack, you can go with this simple class and use a CharField to enter foreign keys manually:
class ListFormField(forms.Field):
""" A form field for being able to display a djangotoolbox.fields.ListField. """
widget = ListWidget
def clean(self, value):
return [v.strip() for v in value.split(',') if len(v.strip()) > 0]
But if you want to have a multiple selection from a list of models normally you'd have to use ModelMultipleChoiceField, which is also not functional in django-nonrel. Here's what I've done to emulate a M2M relationship using a MultipleSelectField:
Let's say you have a M2M relationship between 2 classes, SomeClass and AnotherClass respectively. You want to select the relationship on the form for SomeClass. Also I assume you want to hold the references as a ListField in SomeClass. (Naturally you want to create M2M relationships as they're explained here, to prevent exploding indexes if you're working on App Engine).
So you have your models like:
class SomeClass(models.Model):
another_class_ids = ListField(models.PositiveIntegerField(), null=True, blank=True)
#fields go here
class AnotherClass(models.Model):
#fields go here
And in your form:
class SomeClassForm(forms.ModelForm):
#Empty field, will be populated after form is initialized
#Otherwise selection list is not refreshed after new entities are created.
another_class = forms.MultipleChoiceField(required=False)
def __init__(self, *args, **kwargs):
super(SomeClassForm,self).__init__(*args, **kwargs)
self.fields['another_class'].choices = [(item.pk,item) for item in AnotherClass.objects.all()]
if self.instance.pk: #If class is saved, highlight the instances that are related
self.fields['another_class'].initial = self.instance.another_class_ids
def save(self, *args, **kwargs):
self.instance.another_class_ids = self.cleaned_data['another_class']
return super(SomeClassForm, self).save()
class Meta:
model = SomeClass
Hopefully this should get you going for the start, I implemented this functionality for normal forms, adjust it for admin panel shouldn't be that hard.
This could be unrelated but for the admin interface, be sure you have djangotoolbox listed after django.contrib.admin in the settings.. INSTALLED_APPS
You could avoid a custom form class for such usage by inquiring for the model object
class ModelListField(ListField):
def __init__(self, embedded_model=None, *args, **kwargs):
super(ModelListField, self).__init__(*args, **kwargs)
self._model = embedded_model.embedded_model
def formfield(self, **kwargs):
return FormListField(model=self._model, **kwargs)
class ListFieldWidget(SelectMultiple):
pass
class FormListField(MultipleChoiceField):
widget = ListFieldWidget
def __init__(self, model=None, *args, **kwargs):
self._model = model
super(FormListField, self).__init__(*args, **kwargs)
self.widget.choices = [(unicode(i.pk), i) for i in self._model.objects.all()]
def to_python(self, value):
return [self._model.objects.get(pk=key) for key in value]
def clean(self, value):
return value

Django models, custom functions

I am creating simple application with django. Also, I realized that I am doing some kind of operations very often. For example I often need to get all Article objects which have isPublick = True. So I am thinking is that possible to define get_published function in model?
if models looks like this (simplified)
class Article(models.Model):
title = models.CharField(...)
isPublished = models.BooleandField()
def get_active(self):
return Article.objects.filter(isPublicshed = 1)
But it doesn't work this way
Can you suggest a way of implementing the function?
What you probably want is a custom manager
From the django docs:
# An example of a custom manager called "objects".
class PersonManager(models.Manager):
def get_fun_people(self):
return self.filter(fun=True)
class Person(models.Model):
first_name = models.CharField(max_length=30)
last_name = models.CharField(max_length=30)
fun = models.BooleanField()
objects = PersonManager()
def __unicode__(self):
return u"%s %s" % (self.first_name, self.last_name)
which then allows you to do something like:
>>> p1 = Person(first_name='Bugs', last_name='Bunny', fun=True)
>>> p1.save()
>>> p2 = Person(first_name='Droopy', last_name='Dog', fun=False)
>>> p2.save()
>>> Person.objects.get_fun_people()
[<Person: Bugs Bunny>]
As is stated in the docs here, if you need to add custom row-level functionality to your objects, you need to define custom methods on your models. However, if what you are after is custom table-wide functionality (such as getting all Article objects that qualify certain conditions), you have to define custom methods on model Managers (much as aciniglio above points out in their answer).
You can use the staticmethod decorator.
class Article(models.Model):
title = models.CharField(...)
isPublished = models.BooleandField()
#staticmethod
def get_active():
return Article.objects.filter(isPublished = 1)

Resources