i'm building a django app from an existing database. for better or worse, i have a couple of views that i would like to base my models off. they look something like this:
class Device(Model):
id = models.IntegerField( primary_key=True, db_column='node_id' )
name = models.CharField(max_length=127, db_column='node' )
class Meta:
db_table = 'node' # db view
managed = False
class Entity(Model):
id = models.IntegerField( primary_key=True, db_column='_id' )
device = models.ForeignKey(Device, db_column='node_id' )
class Meta:
db_table = 'entity' # db view
managed = Fase
so things work okay. however, when i try to use a template that utilises the ForeignKey it is very slow:
{% for e in entities %}
{{ e.device.name }}
{% endfor %}
looking at the logs, it appears to be repeating queries for each 'node_id', and ultimately timeout out.
(of course, if i do not include e.device.name is all quick)
is there a way i can optimise this?
to be fair, the 'entity' view already has the name of the device as another field (node), so i could use this instead but i would like the relation to exist.
have you try .select_related() in your view?
entities = Entity.objects.select_related('device').filter(...)
in the other hand, if it is and old database, and no autogenerated by django, probably node_id may not be an index in the database, this will slow down a lot any JOIN.
Related
I am trying to use filters with a Wagtail Page model and a Orderable model. But I get duplicates in my filter now. How can I solve something like this?
My code:
class FieldPosition(Orderable):
page = ParentalKey('PlayerDetailPage', on_delete=models.CASCADE, related_name='field_position_relationship')
field_position = models.CharField(max_length=3, choices=FIELD_POSITION_CHOICES, null=True)
panels = [
FieldPanel('field_position')
]
def __str__(self):
return self.get_field_position_display()
class PlayerDetailPage(Page):
content_panels = Page.content_panels + [
InlinePanel('field_position_relationship', label="Field position", max_num=3),
]
class PlayerDetailPageFilter(FilterSet):
field_position_relationship = filters.ModelChoiceFilter(queryset=FieldPosition.objects.all())
class Meta:
model = PlayerDetailPage
fields = []
So what I am trying to do is create a filter which uses the entries from FIELD_POSITION_CHOICES to filter out any page that has this position declared in the inline panel in Wagtail.
As you can see in the picture down below, the filters are coming through and the page is being rendered. (These are 2 pages with a list of 3 field positions).
So Page 1 and Page 2 both have a "Left Winger" entry, so this is double in the dropdown. The filtering works perfectly fine.
What can I do to prevent this?
The solution should be something like this (Credits to Harris for this):
I basically have one FieldPosition object per page-field position, so it's listing all of the objects correctly. I suspect I should not use the model chooser there, but a list of the hard coded values in FIELD_POSITION_CHOICES and then a filter to execute a query that looks something like PlayerDetailPage.objects.filter(field_position_relationship__field_position=str_field_position_choice). But what is the Django Filter way of doing this?
Raf
In my limited simplistic view it looks like
class PlayerDetailPageFilter(FilterSet):
field_position_relationship = filters.ModelChoiceFilter(queryset=FieldPosition.objects.all())
is going to return all the objects from FieldPosition and if you have 2 entries for 'left wing' in here (one for page 1 and one for page 2) then it would make sense that this is duplicating in your list. So have you tried to filter this list queryset with a .distinct? Perhaps something like
class PlayerDetailPageFilter(FilterSet):
field_position_relationship = filters.ModelChoiceFilter(queryset=FieldPosition.objects.values('field_position').distinct())
I found the solution after some trial and error:
The filter:
class PlayerDetailPageFilter(FilterSet):
field_position_relationship__field_position = filters.ChoiceFilter(choices=FIELD_POSITION_CHOICES)
class Meta:
model = PlayerDetailPage
fields = []
And then the view just like this:
context['filter_page'] = PlayerDetailPageFilter(request.GET, queryset=PlayerDetailPage.objects.all()
By accessing the field_position through the related name of the ParentalKey field_position_relationship with __.
Then using the Django Filter ChoiceFilter I get all the hard-coded entries now from the choice list and compare them against the entries inside the PlayerDetailPage query set.
In the template I can get the list using the Django Filter method and then just looping through the query set:
<form action="" method="get">
{{ filter_page.form.as_p }}
<input type="submit" />
</form>
{% for obj in filter_page.qs %}
{{ obj }} >
{% endfor %}
I am facing a very weird problem in one of my django projects. In my project I have a custom field class that handles foreign keys, one to one and many 2 many model fields. The class is some thing like the following.
from django import forms
class CustomRelatedField(forms.Field):
def __init__(self, model, limit=None, multiple=False, create_objects=True, *args, *kwargs):
self.model = model
self.limit = limit
self.multiple = multiple
self.create_objects = create_objects
super(CustomRelatedField, self).__init__(*args, **kwargs)
def clean(self, value):
""" Calls self.get_objects to get the actual model object instance(s)
from the given unicode value.
"""
# Do some value processing here
return self.get_objects(value)
def get_objects(self, values):
""" Returns the model object instances for the given unicode values.
"""
results = []
for value in values:
try:
obj = self.model.object.get_or_create(name=value)[0]
results.append(obj)
except Exception, err:
# Log the error here.
return results
def prepare_value(self, value):
""" Returns the value to be sent to the UI. The value
passed to this method is generally the object id or
a list of object id's (in case it is a many to many object).
So we need to make a database query to get the object and
then return the name attribute.
"""
if self.multiple:
result = [obj.name for obj in self.model.filter(pk__in=value)]
else:
result = self.model.object.get(pk=value)
return result
Recently while I was playing with the django-toolbar, I found out one of the pages that has a form with the above mentioned fields was ridiculously making multiple queries for the same objects again and again.
While debugging, I found out the prepare_value method was being called again and again. After some more debugging, I realized the culprit was the template. I have a generic template that I use for forms, It looks something like the following:
{% for field in form %}
{% if field.is_hidden %}
<!-- Do something here -->
{% endif %}
{% if field.field.required %}
<!-- Do something here -->
{% endif %}
<label>{{ field.label }}</label>
<div class="form-field">{{ field }}</div>
{% if field.field.widget.attrs.help_text %}
<!-- Do something here -->
{% elif field.errors %}
<!-- Do something here -->
{% endif %}
{% endfor %}
In the above code, each if statement calls the field class which calls the prepare_value method which then makes the database queries. Each of the following listed is making a database query, I am totally lost to why this is happening and have no clue about any solutions. Any help, suggestions would be really appreciated. Thanks.
field.is_hidden
field.field.required
field.label
field.label_tag
field
field.field.widget.attrs.help_text
field.errors
Also, why does this happen with my custom field class only, other fields (FKs, O2Os, M2M's) in the application and the application admin, just make one query, even though they are using a similar template.
Problem is with your prepare_value() method which does explicit queries. .get() does not get cached and always hits the db while iterating on .filter() queryset will evaluate that.
This might be causing you multiple queries.
This is not seen in default fields because they do not do any queries in prepare_value().
To resolve this, you can try to cache the value and result. If value hasn't changed, return cached result. Something like:
class CustomRelatedField(forms.Field):
def __init__(self, model, limit=None, multiple=False, create_objects=True, *args, *kwargs):
self.cached_result = None
self.cached_value = None
...
def prepare_value(self, value):
#check we have cached result
if value == self.cached_value:
return self.cached_result
if self.multiple:
result = [obj.name for obj in self.model.filter(pk__in=value)]
else:
result = self.model.object.get(pk=value)
#cache the result and value
self.cached_result = result
self.cached_value = value
return result
Not sure how good/bad this work around though!
I've looked into the answers posted to similar questions but I couldn't get anything to work for my situation, so I will post my question. The way I have it, I always get the result "None" when I try to get the first name of the user who created an event... So I have multiple classes of users and I want to allow one type of user to be able to schedule events, with the template showing the user who created the event along with the various other fields that are in the class relating to the event. So for example:
#models.py
class BaseProfile(models.Model):
user = models.ForeignKey(User, unique=True)
first_name = models.CharField(max_length=20, verbose_name=_('First Name'))
last_name = models.CharField(max_length=20, verbose_name=_('Last Name'))
...
#More common fields
#Used for when a new broadcaster registers; fields specific to broadcasters
classBroadcasterProfile(models.Model):
user_type = models.CharField(max_length=20, default="broadcaster")
...
#Other fields specific to broadcasters
#Used when a broadcaster creates a profile for themselves after they have registered
class BroadcasterAccountProfile(BroadcasterProfile):
...
#A broadcaster wants to schedule an event
class UpcomingEvents(models.Model):
title = models.CharField(max_length=50, verbose_name=_('Title'))
description = models.TextField(max_length=100, verbose_name=_('Event Description'))
instructor = models.ForeignKey(BroadcasterAccountProfile, null=True, blank=True, related_name='instructor')
event_date = models.DateField(auto_now=False, auto_now_add=False, verbose_name=_('Date of Event'))
...
#Additional info to be included with the scheduling of event
So after a broadcaster has registered, created a profile, and scheduled an event (which all works successfully), the view I am trying to use for displaying the upcoming scheduled events is as follows:
#views.py
def upcoming_event_list(request):
today = datetime.date.today()
tomorrow = datetime.date.today() + datetime.timedelta(days=1)
todays_events = UpcomingEvents.objects.filter(event_date=today)
tomorrows_events = UpcomingEvents.objects.filter(event_date=tomorrow)
return render_to_response('events/preview.html', {'today': today, 'tomorrow': tomorrow,
...
I can have my template display anything related to the upcoming events except for anything related to the broadcaster who created the event. I have tried adding values to my views:
def upcoming_event_list(request):
...
values = UpcomingEvents.objects.values('instructor__first_name', 'instructor__last_name', 'instructor__avatar')
#template.html
{% for v in values %}
Broadcaster Name:{{ v.instructor__first_name }}
{% endfor %}
This would show "Broadcaster Name: None". Am I completely heading in the wrong direction, or what can I do to try to make this work? Sorry if this was confusing. I tried to make it as clear as possible, but seeing as how I am confused myself, I don't know how successful I was in trying to describe my problem. Thanks for any help. Please let me know if I need to clarify anything.
Have you tried {{ v.instructor.first_name }} in your template?
Note the .(full-stop/dot/period/whatever-you-call-it) instead of a __(double underscore).
Same code, in production don't work while in development yes.
models:
class Comput(ndb.Model):
#staticmethod
def membership(user):
q = ndb.gql("SELECT * FROM Members WHERE member = :1", user)
result = q.fetch()
return [m.comput.get() for m in result]
#return [m.comput.get() for m in q] #same issue (dev yes, prod no)
class Members(ndb.Model):
comput = ndb.KeyProperty(kind=Comput)
member = ndb.UserProperty()
handler:
comput_list = Comput.membership(users.get_current_user())
template:
{% for comput in comput_list %}
<tr onclick="location.href='/comput?id={$ comput.key.id() $}'">
Traceback (only in production):
UndefinedError: 'None' has no attribute 'key'
This appears even when comput_list have entities.
update: No autogenerate index for Members, I tried to add manually but nothing changes
Looks like one of your Members objects has None for its comput property. I would use the production data viewer (or add a log statements) to find which member it is.
I'm trying to change some database entries from one legal state to another, but the intermediate (partially updated) state is not legal. As an example, suppose I'm modeling lectures, each of which is made up of several short topics in some order:
class Lecture(models.Model):
slug = models.TextField(
help_text='Abbreviated name of lecture.'
)
class Topic(models.Model):
slug = models.TextField(
help_text='Abbreviated name of topic.'
)
lecture = models.ForeignKey(
Lecture,
help_text='The lecture this topic is part of.'
)
order = models.IntegerField(
help_text='Impose ordering on topics.'
)
class Meta:
unique_together = (('lecture', 'order'),)
My test case is:
class TestTopicOrder(TestCase):
def test_reordering_topics(self):
# The lecture these topics are part of.
lecture = Lecture(title='Test Lecture', slug='lec')
lecture.save()
# Two topics 'zero' and 'one' in that order.
Topic(lecture=lecture, slug='zero', order=0).save()
Topic(lecture=lecture, slug='one, order=1).save()
# Try to invert the order.
t0 = Topic.objects.get(slug='zero')
t1 = Topic.objects.get(slug='one')
t0.order = 1
t1.order = 0
t0.save()
t1.save()
Essentially, I'm trying to do:
t0.order, t1.order = t1.order, t0.order
and then save, but whichever modified object I save first will have the same 'order' value as the other entry. I could delete and re-make, but when it comes time to re-order a dozen topics at once, that'll be a pain. What's the cleanest way to do this?
A dirty dirty solution... you could drop and recreate the restriction on the database using south api:
from south.db import db
db.delete_unique('app_lecture', ['lecture', 'order'])
# do your stuff
# then reenable the unique constraint...
db.create_unique('app_lecture', ['lecture', 'order'])