Debugging GQL Queries - google-app-engine

I have been trying to run this query:
my_post = db.GqlQuery("SELECT * FROM Post where __key__ = KEY('Post', 1)")
self.render("showpost.html", my_post = my_post)
and my template showpost.html looks like this:
<div class="post">
<h3 class="post-subject">{{ my_post.subject }}</h3>
<div class="post-content">{{ my_post.content }}</div>
</div
Instead of getting the details of the Post of id=1 I'm getting a blank page. Although there exists a post with id=1 i'm not sure how to confirm if the GQLquery is actually returning something. Is there a way to check?
On trying the following:
my_post = db.GqlQuery("SELECT * FROM Post where subject = 'hello_world')
self.render("showpost.html", my_post = my_post)
I get the same issue (blank page), even though there is post who's subject is hello_world
Thanks a lot

GqlQuery is a class.
Your code has created an instance of the GqlQuery class.
To return something from the datastore you need to use either of the following instance methods:
.fetch() # requires number of entities as first argument
.get()
Example:
my_query = db.GqlQuery("SELECT * FROM Post WHERE subject = :subject",
subject="hello_world")
my_post = my_query.get() # the magic happens here
self.render("showpost.html", my_post = my_post)

Bernie is right in that you need to use .fetch or .get to actually retrieve results from the GqlQuery that you created, but for the first version, where you have a key or an id, you don't want to create a query at all. Rather, use the methods that are provided for using them directly.
If you have an id, use:
my_post = Post.get_by_id(id)
this will be much faster than using a query object.
If you have an entire key, use:
my_post = Post.get(key)
That is the fastest and most efficient way to get an entity out of the database.

Related

possible to update a property after new model is created in sailsjs?

I am trying to create a new record for a model and there are fields I need to save later after the model is created.
I have the code like this
let new_model = await Model.create({ name, type}).fetch();
new_model.content = 'abc';
new_model.save();
TypeError: new_model.save is not a function
I googled, it doesn't seem like sailsjs can be done this way so I guess I have to use the update or updateOne by sails. But the thing is, name,type fields are not unique and will not be so using update will actually turn into update for ALL records. updateOne will give me errors saying there's more than one record though
Does anyone has any suggestions on what can be done here?
One other option I can think of is doing something like this with updateOne
let new_model = await Model.create({ name, type}).fetch();
// getting your content here or do something else
const content = 'abc';
new_model = await Model.updateOne({ id:new_model.id}).set({content});

Ordering the solr search results based on the indexed fields

I have to order the search results from solr based on some fields which are already indexed.
My current api request is like this without sorting.
http://127.0.0.1:8000/api/v1/search/facets/?page=1&gender=Male&age__gte=19
And it gives the search results based on the indexed order. But I have to reorder this results based on the filed 'last_login' which is already indexed DateTimeField.
Here is my viewset
class ProfileSearchView(FacetMixin, HaystackViewSet):
index_models = [Profile]
serializer_class = ProfileSearchSerializer
pagination_class = PageNumberPagination
facet_serializer_class = ProfileFacetSerializer
filter_backends = [HaystackFilter]
facet_filter_backends = [HaystackFilter, HaystackFacetFilter]
def get_queryset(self, index_models=None):
if not index_models:
index_models = []
queryset = super(ProfileSearchView, self).get_queryset(index_models)
queryset = queryset.order_by('-created_at')
return queryset`
Here I have changed the default search order by 'created_at' value. But for the next request I have order based on the 'last_login' value. I have added a new parameter in my request like this
http://127.0.0.1:8000/api/v1/search/facets/?page=1&gender=Male&age__gte=19&sort='last_login'
but it gives me an error
SolrError: Solr responded with an error (HTTP 400): [Reason: undefined field sort]
How can I achieve this ordering possible? Please help me with a solution.
The URL you provided http://127.0.0.1:8000/api/v1/search/facets/ is not direct SOLR URL. It must be your middle-ware. Since you have tried the query directly against Solr and it works, the problem must be somewhere in middle-ware layer.
Try to print or monitor or check logs to see what URL the midde-ware actually generates and compare it to the valid URL you know works.

How to edit information submitted using Django forms in Google App Engine

Thank you for the tips. Maybe I can edit this to explain it a little better...
I'm writing an application for Google App Engine where I would like to do a GQL query to get all the evaluations of each type that have already been submitted. I want to allow the user to click on a link to edit each individual evaluation and then submit the change. This is the best way I've found to do it...
class EvaluationApproval(webapp.RequestHandler):
def get(self):
#search for unapproved general evaluations
query = db.GqlQuery("SELECT * FROM GeneralAssessmentReport WHERE Approved = False")
if query.count() != 0:
for item in query:
#create a link to edit that item
self.response.out.write('%s' % (item.key(), item.Name))
#do this for each type of evaluation...
query = db.GqlQuery("SELECT * FROM HeadNeck WHERE Approved = False")
if query.count() != 0:
for item in query:
self.response.out.write('%s' % (item.key(), item.Name))
query = db.GqlQuery("SELECT * FROM lowerExtremity WHERE Approved = False")
if query.count() != 0:
for item in query:
self.response.out.write('%s' % (item.key(), item.Name))
So, when the user clicks on the link they are directed to...
class GeneralFormApprove(webapp.RequestHandler):
def get(self):
# get the correct evaluation to approve
key = self.request.get('key')
item = GeneralAssessmentReport.get(key)
#write out the form with key in the form action!
self.response.out.write('<form method="POST" action="/generalFormApprove?key=%s">' % key)
self.response.out.write(GeneralAssessmentReportForm(instance=item))
def post(self):
# get the key once more
key = self.request.get('key')
data = GeneralAssessmentReportForm(data=self.request.POST,instance=GeneralAssessmentReport.get(key))
if data.is_valid():
# save the edited evaluation
entity = data.save(commit=False)
entity.put()
and then have a WhateverFormApprove class for each type of evaluation. Does this make sense and do you have any other ideas to accomplish this?
for editing existing entities, a good way is to use a url pattern that includes the id of the entity. Upon post, you can use the id to load the entity and then use it in the instance argument for the form.
the url pattern is something like:
r'^evaluationapproval/(?P<item_id>[\d]+)/$'
def post(self,item_id=None):
item = None
if item_id:
item = db.get(db.Key.from_path('MyModelKind',int(item_id)))
#...

Getting error message (if value is not None and not value.has_key() )

I am trying to save Event. But it does not work. Will you please help? thanks alot
query = []
query = Identity.all().filter('name =', 'k').fetch(1)
if query:
for q in query:
event_id = q.key().id()
Event(description=description, identity=event_id)
Event Model
class Event(search.SearchableModel):
description = db.TextProperty(required=True)
identity = db.ReferenceProperty(Identity)
Getting error message >
if value is not None and not value.has_key():
AttributeError: 'long' object has no attribute 'has_key'
Your assigning the id of the object into a ReferenceProeprty, this is wrong.
Your code should look something like this:
query = Identity.all().filter('name =', 'k').get()
if query:
Event(description=description, identity=q)
Also instead of having your own name attribute you use use the buitin key_name attribute that each Model has, its faster and cheaper.
query = Identity.get_by_key_name(k)
if query:
Event(description=description, identity=q)

How can I mimic 'select_related' using google-appengine and django-nonrel?

django nonrel's documentation states: "you have to manually write code for merging the results of multiple queries (JOINs, select_related(), etc.)".
Can someone point me to any snippets that manually add the related data? #nickjohnson has an excellent post showing how to do this with the straight AppEngine models, but I'm using django-nonrel.
For my particular use I'm trying to get the UserProfiles with their related User models. This should be just two simple queries, then match the data.
However, using django-nonrel, a new query gets fired off for each result in the queryset. How can I get access to the related items in a 'select_related' sort of way?
I've tried this, but it doesn't seem to work as I'd expect. Looking at the rpc stats, it still seems to be firing a query for each item displayed.
all_profiles = UserProfile.objects.all()
user_pks = set()
for profile in all_profiles:
user_pks.add(profile.user_id) # a way to access the pk without triggering the query
users = User.objects.filter(pk__in=user_pks)
for profile in all_profiles:
profile.user = get_matching_model(profile.user_id, users)
def get_matching_model(key, queryset):
"""Generator expression to get the next match for a given key"""
try:
return (model for model in queryset if model.pk == key).next()
except StopIteration:
return None
UPDATE:
Ick... I figured out what my issue was.
I was trying to improve the efficiency of the changelist_view in the django admin. It seemed that the select_related logic above was still producing additional queries for each row in the results set when a foreign key was in my 'display_list'. However, I traced it down to something different. The above logic does not produce multiple queries (but if you more closely mimic Nick Johnson's way it will look a lot prettier).
The issue is that in django.contrib.admin.views.main on line 117 inside the ChangeList method there is the following code: result_list = self.query_set._clone(). So, even though I was properly overriding the queryset in the admin and selecting the related stuff, this method was triggering a clone of the queryset which does NOT keep the attributes on the model that I had added for my 'select related', resulting in an even more inefficient page load than when I started.
Not sure what to do about it yet, but the code that selects related stuff is just fine.
I don't like answering my own question, but the answer might help others.
Here is my solution that will get related items on a queryset based entirely on Nick Johnson's solution linked above.
from collections import defaultdict
def get_with_related(queryset, *attrs):
"""
Adds related attributes to a queryset in a more efficient way
than simply triggering the new query on access at runtime.
attrs must be valid either foreign keys or one to one fields on the queryset model
"""
# Makes a list of the entity and related attribute to grab for all possibilities
fields = [(model, attr) for model in queryset for attr in attrs]
# we'll need to make one query for each related attribute because
# I don't know how to get everything at once. So, we make a list
# of the attribute to fetch and pks to fetch.
ref_keys = defaultdict(list)
for model, attr in fields:
ref_keys[attr].append(get_value_for_datastore(model, attr))
# now make the actual queries for each attribute and store the results
# in a dict of {pk: model} for easy matching later
ref_models = {}
for attr, pk_vals in ref_keys.items():
related_queryset = queryset.model._meta.get_field(attr).rel.to.objects.filter(pk__in=set(pk_vals))
ref_models[attr] = dict((x.pk, x) for x in related_queryset)
# Finally put related items on their models
for model, attr in fields:
setattr(model, attr, ref_models[attr].get(get_value_for_datastore(model, attr)))
return queryset
def get_value_for_datastore(model, attr):
"""
Django's foreign key fields all have attributes 'field_id' where
you can access the pk of the related field without grabbing the
actual value.
"""
return getattr(model, attr + '_id')
To be able to modify the queryset on the admin to make use of the select related we have to jump through a couple hoops. Here is what I've done. The only thing changed on the 'get_results' method of the 'AppEngineRelatedChangeList' is that I removed the self.query_set._clone() and just used self.query_set instead.
class UserProfileAdmin(admin.ModelAdmin):
list_display = ('username', 'user', 'paid')
select_related_fields = ['user']
def get_changelist(self, request, **kwargs):
return AppEngineRelatedChangeList
class AppEngineRelatedChangeList(ChangeList):
def get_query_set(self):
qs = super(AppEngineRelatedChangeList, self).get_query_set()
related_fields = getattr(self.model_admin, 'select_related_fields', [])
return get_with_related(qs, *related_fields)
def get_results(self, request):
paginator = self.model_admin.get_paginator(request, self.query_set, self.list_per_page)
# Get the number of objects, with admin filters applied.
result_count = paginator.count
# Get the total number of objects, with no admin filters applied.
# Perform a slight optimization: Check to see whether any filters were
# given. If not, use paginator.hits to calculate the number of objects,
# because we've already done paginator.hits and the value is cached.
if not self.query_set.query.where:
full_result_count = result_count
else:
full_result_count = self.root_query_set.count()
can_show_all = result_count self.list_per_page
# Get the list of objects to display on this page.
if (self.show_all and can_show_all) or not multi_page:
result_list = self.query_set
else:
try:
result_list = paginator.page(self.page_num+1).object_list
except InvalidPage:
raise IncorrectLookupParameters
self.result_count = result_count
self.full_result_count = full_result_count
self.result_list = result_list
self.can_show_all = can_show_all
self.multi_page = multi_page
self.paginator = paginator

Resources