Django Custom field's attributes making database queries - database

Question

I am facing a very weird problem in one of my django projects. In my project I have a custom field class that handles foreign keys, one to one and many 2 many model fields. The class is some thing like the following.
from django import forms
class CustomRelatedField(forms.Field):
def __init__(self, model, limit=None, multiple=False, create_objects=True, *args, *kwargs):
self.model = model
self.limit = limit
self.multiple = multiple
self.create_objects = create_objects
super(CustomRelatedField, self).__init__(*args, **kwargs)
def clean(self, value):
""" Calls self.get_objects to get the actual model object instance(s)
from the given unicode value.
"""
# Do some value processing here
return self.get_objects(value)
def get_objects(self, values):
""" Returns the model object instances for the given unicode values.
"""
results = []
for value in values:
try:
obj = self.model.object.get_or_create(name=value)[0]
results.append(obj)
except Exception, err:
# Log the error here.
return results
def prepare_value(self, value):
""" Returns the value to be sent to the UI. The value
passed to this method is generally the object id or
a list of object id's (in case it is a many to many object).
So we need to make a database query to get the object and
then return the name attribute.
"""
if self.multiple:
result = [obj.name for obj in self.model.filter(pk__in=value)]
else:
result = self.model.object.get(pk=value)
return result
Recently while I was playing with the django-toolbar, I found out one of the pages that has a form with the above mentioned fields was ridiculously making multiple queries for the same objects again and again.
While debugging, I found out the prepare_value method was being called again and again. After some more debugging, I realized the culprit was the template. I have a generic template that I use for forms, It looks something like the following:
{% for field in form %}
{% if field.is_hidden %}
<!-- Do something here -->
{% endif %}
{% if field.field.required %}
<!-- Do something here -->
{% endif %}
<label>{{ field.label }}</label>
<div class="form-field">{{ field }}</div>
{% if field.field.widget.attrs.help_text %}
<!-- Do something here -->
{% elif field.errors %}
<!-- Do something here -->
{% endif %}
{% endfor %}
In the above code, each if statement calls the field class which calls the prepare_value method which then makes the database queries. Each of the following listed is making a database query, I am totally lost to why this is happening and have no clue about any solutions. Any help, suggestions would be really appreciated. Thanks.
field.is_hidden
field.field.required
field.label
field.label_tag
field
field.field.widget.attrs.help_text
field.errors
Also, why does this happen with my custom field class only, other fields (FKs, O2Os, M2M's) in the application and the application admin, just make one query, even though they are using a similar template.

Answer 1

Problem is with your prepare_value() method which does explicit queries. .get() does not get cached and always hits the db while iterating on .filter() queryset will evaluate that.
This might be causing you multiple queries.
This is not seen in default fields because they do not do any queries in prepare_value().
To resolve this, you can try to cache the value and result. If value hasn't changed, return cached result. Something like:
class CustomRelatedField(forms.Field):
def __init__(self, model, limit=None, multiple=False, create_objects=True, *args, *kwargs):
self.cached_result = None
self.cached_value = None
...
def prepare_value(self, value):
#check we have cached result
if value == self.cached_value:
return self.cached_result
if self.multiple:
result = [obj.name for obj in self.model.filter(pk__in=value)]
else:
result = self.model.object.get(pk=value)
#cache the result and value
self.cached_result = result
self.cached_value = value
return result
Not sure how good/bad this work around though!