I have the following models:
class Author(db.Model):
name = db.StringProperty()
class Story(db.Model):
author = db.ReferenceProperty(Author)
What's the raw GQL to find all Stories by a certain author. In regular SQL, I will use joins but I don't think that's available in GQL.
Edit:
I'm looking for the raw GQL way, I know how to do it the Pythonic way. For instance something like(the following is probably totally wrong):
"SELECT * FROM Story WHERE author = :1", "Shakespeare"
I want to run the above from the GAE admin Data > Data Viewer > Query the Datastore. I want the raw SQL that someone could run from a typical mysql or psql shell.
Edit2: Ah, the raw-GQL for use in the data-viewer...
Here's one way:
1) Run this and get the ID number:
SELECT * FROM Author where name = 'shakespeare'
2) Using ID number from previous query, run this:
SELECT * FROM Story where author = key('Author', 12345)
Edit: at long last, the raw GQL:
(Easiest way: Use the implicit backreference property name; in the form "modelname_set".)
qry = GqlQuery("SELECT * FROM Author WHERE name = :1", "shakespeare")
shakespeare = qry.get()
shakespeare.story_set # this property now contains all Shakespeare's stories
or
qry0 = GqlQuery("SELECT * FROM Author WHERE name = :1", "shakespeare")
shakespeare = qry0.get()
qry1 = GqlQuery("SELECT * FROM Story WHERE author = :1", shakespeare.key())
shakespeare_stories = qry1.fetch(10) # probably good to have some limit here
I prefer this way:
qry = Author.all()
qry.filter('name = ', 'shakespeare')
shakespeare = qry.get()
shakespeare.story_set # this property now contains all Shakespeare's stories
The more involved way may sometimes be necessary:
qry0 = Author.all()
qry0.filter('name = ', 'shakespeare')
shakespeare = qry0.get()
qry1 = Story.all()
qry1.filter('author = ', shakespeare.key())
shakespeare_stories = qry1.fetch(10) # probably good to have some limit here
Related
I have a couple ndb models looks like these:
class Product(ndb.Model):
manufacturer = ndb.StringProperty()
category = ndb.StringProperty()
price = ndb.FloatProperty()
class Customer(ndb.Model):
customerId = ndb.StringProperty()
name = ndb.StringProperty()
products = ndb.StructuredProperty(Product, repeated=True)
And I'd like to query based on 'manufacturer' and 'category' of the product he/she owns. So this query works as expected.
query = Customer.query(Customer.products == Product(manufacturer=data_json["product"]["manufacturer"],
category=data_json["product"]["category"]))
results= query.fetch()
However, I cannot get the "projection" to work along with this query. The following query simply returned nothing.
query = Customer.query(Customer.products == Product(manufacturer=data_json["product"]["manufacturer"],
category=data_json["product"]["category"]))
results= query.fetch(projection=[Customer.products.price])
But if I use the projection without the filter, the projection part works fine. The following query will return all entities but only the 'price' property
results= Customer.query().fetch(projection=[Customer.products.price])
Any thoughts? Thanks.
BTW, my queries were developed based on this article.
https://cloud.google.com/appengine/docs/standard/python/ndb/queries#filtering_structured_properties
The correct way of combining AND and OR operations in the ndb library is documented in NDB Client Library's documentation.
With the query below, you are performing an AND operation in the filter, so instead of this one, you should use the one I propose below, using ndb.AND().
# Your query
query = Customer.query(Customer.products == Product(manufacturer=data_json["product"]["manufacturer"], category=data_json["product"]["category"]))
# Query using ndb.AND
query = Customer.query(ndb.AND(Customer.products == Product(manufacturer=data_json["product"]["manufacturer"]), Customer.products == Product(category=data_json["product"]["category"])))
Also, it turns out that if you perform the filtering in multiple steps, the query also works:
# Your request
query = Customer.query(Customer.products == Product(manufacturer=data_json["product"]["manufacturer"], category=data_json["product"]["category"]))
results = query.fetch(projection=[Customer.products.price])
# Request performing filter in multiple steps
query = Customer.query(Customer.products == Product(category=data_json["product"]["category"]))
query1 = query.filter(Customer.products == Product(manufacturer=data_json["product"]["manufacturer"]))
results = query1.fetch(projection=[Customer.products.price])
You can use either of the proposed alternatives, although I would suggest using ndb.AND() as it minimizes the code and is also the best way to combine AND operations.
UPDATE with some code:
app.yaml
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /.*
script: main.app
main.py
import webapp2
from google.appengine.ext import ndb
# Datastore Models
class Product(ndb.Model):
manufacturer = ndb.StringProperty()
category = ndb.StringProperty()
price = ndb.FloatProperty()
class Customer(ndb.Model):
customerId = ndb.StringProperty()
name = ndb.StringProperty()
products = ndb.StructuredProperty(Product, repeated=True)
# Create entities for testing purposes
class CreateEntities(webapp2.RequestHandler):
def get(self):
prod1 = Product(manufacturer="Google", category="GCP", price=105.55)
prod2 = Product(manufacturer="Google", category="GCP", price=123.45)
prod3 = Product(manufacturer="Google", category="Drive", price=10.38)
prod1.put()
prod2.put()
prod3.put()
cust1 = Customer(customerId="Customer1", name="Someone", products=[prod1,prod2,prod3])
cust2 = Customer(customerId="Customer2", name="Someone else", products=[prod1])
cust3 = Customer(customerId="Customer3", name="Noone", products=[prod3])
cust1.put()
cust2.put()
cust3.put()
# Response text
self.response.headers['Content-Type'] = 'text/plain'
self.response.write('Done creating entities')
class GetEntities(webapp2.RequestHandler):
def get(self):
# This will not work
#query = Customer.query(Customer.products == Product(category="GCP", manufacturer="Google"))
#results = query.fetch(projection=[Customer.products.price])
# Alternative 1 - WORKS
#query = Customer.query(Customer.products == Product(category="GCP"))
#query1 = query.filter(Customer.products == Product(manufacturer="Google"))
#results = query1.fetch(projection=[Customer.products.price])
# Alternative 2 - WORKS
query = Customer.query(ndb.AND(Customer.products == Product(manufacturer="Google"), Customer.products == Product(category="GCP")))
results = query.fetch(projection=[Customer.products.price])
self.response.out.write('<html><body>')
for result in results:
self.response.out.write("%s<br><br>" % result)
self.response.out.write('</body></html>')
app = webapp2.WSGIApplication([
('/createEntities', CreateEntities),
('/getEntities', GetEntities),
], debug=True)
My model:
class User(ndb.Model):
name = ndb.StringProperty()
Is there any difference in terms of efficiency/cost/speed between the following two queries?
u = User.query(User.name==name).get()
u = User.query().filter(User.name==name).get()
Should I use one of them over the other? I assume the 2nd one is worse because it firsts get the entire User class queryset and then applies the filter?
There is no difference in functionality between the two so you can choose whatever you like best. On the google documentation, they show these two examples:
query = Account.query(Account.userid >= 40, Account.userid < 50)
and
query1 = Account.query() # Retrieve all Account entitites
query2 = query1.filter(Account.userid >= 40) # Filter on userid >= 40
query3 = query2.filter(Account.userid < 50) # Filter on userid < 50 too
and state:
query3 is equivalent to the query variable from the previous example.
I'm trying to develop a blog using webpy.
def getThread(self,num):
myvar = dict(numero=num)
print myvar
que = self.datab.select('contenidos',vars=myvar,what='contentTitle,content,update',where="category LIKE %%s%" %numero)
return que
I've used some of the tips you answer in this web but I only get a
<type 'exceptions.NameError'> at /
global name 'numero' is not defined
Python C:\xampp\htdocs\webpy\functions.py in getThread, line 42
Web GET http://:8080/
...
I'm trying to make a selection of some categorized posts. There is a table with category name and id. There is a column in the content table which takes a string which will be formatted '1,2,3,5'.
Then the way I think I can select the correct entries with the LIKE statement and some %something% magic. But I have this problem.
I call the code from the .py file which builds the web, the import statement works properly
getThread is defined inside this class:
class categoria(object):
def __init__(self,datab,nombre):
self.nombre = nombre
self.datab = datab
self.n = str(self.getCat()) #making the integer to be a string
self.thread = self.getThread(self.n)
return self.thread
def getCat(self):
'''
returns the id of the categorie (integer)
'''
return self.datab.select('categorias',what='catId', where='catName = %r' %(self.nombre), limit=1)
Please check the correct syntax for db.select (http://webpy.org/cookbook/select), you should not format query with "%" because it makes code vulnerable to sql injections. Instead, put vars in dict and refer to them with $ in your query.
myvars = dict(category=1)
db.select('contenidos', what='contentTitle,content,`update`', where="category LIKE '%'+$category+'%'", vars=myvars)
Will produce this query:
SELECT contentTitle,content,`update` FROM contenidos WHERE category LIKE '%'+1+'%'
Note that I backquoted update because it is reserved word in SQL.
I created a content type and it's have many nodes. And write custom module for show on frontpage.
My nodes order:
$themeurl = '/drupal/'.path_to_theme().'/images';
$sql = "SELECT * FROM `node`
LEFT JOIN `field_data_field_anasayfa_catwalk` ON `field_data_field_anasayfa_catwalk`.`entity_id` = `node`.`nid`
WHERE type = 'designers_albums'
AND `node`.`status` = 1
AND `node`.`language` = '".$language->language."'
AND `field_data_field_anasayfa_catwalk`.`field_anasayfa_catwalk_value` = 1
ORDER BY `node`.`created` ASC";
But now, installed Weight module and want sort by weight. How can i change this option? Thank you.
Entities(nodes) weights are stored in weight_weights database table.
You can try that query
$sql = "SELECT * FROM `node`
LEFT JOIN `field_data_field_anasayfa_catwalk` ON `field_data_field_anasayfa_catwalk`.`entity_id` = `node`.`nid`
LEFT JOIN `weight_weights` AS w ON w.entity_id=`node`.`nid`
WHERE type = 'designers_albums'
AND `node`.`status` = 1
AND `node`.`language` = '".$language->language."'
AND `field_data_field_anasayfa_catwalk`.`field_anasayfa_catwalk_value` = 1
ORDER BY w.weight, `node`.`created` ASC";
Btw, in future you should consider learning Drupal database abstraction layer or EntityFieldQuery.
I have a model containing ranges of IP addresses, similar to this:
class Country(db.Model):
begin_ipnum = db.IntegerProperty()
end_ipnum = db.IntegerProperty()
On a SQL database, I would be able to find rows which contained an IP in a certain range like this:
SELECT * FROM Country WHERE ipnum BETWEEN begin_ipnum AND end_ipnum
or this:
SELECT * FROM Country WHERE begin_ipnum < ipnum AND end_ipnum > ipnum
Sadly, GQL only allows inequality filters on one property, and doesn't support the BETWEEN syntax. How can I work around this and construct a query equivalent to these on App Engine?
Also, can a ListProperty be 'live' or does it have to be computed when the record is created?
question updated with a first stab at a solution:
So based on David's answer below and articles such as these:
http://appengine-cookbook.appspot.com/recipe/custom-model-properties-are-cute/
I'm trying to add a custom field to my model like so:
class IpRangeProperty(db.Property):
def __init__(self, begin=None, end=None, **kwargs):
if not isinstance(begin, db.IntegerProperty) or not isinstance(end, db.IntegerProperty):
raise TypeError('Begin and End must be Integers.')
self.begin = begin
self.end = end
super(IpRangeProperty, self).__init__(self.begin, self.end, **kwargs)
def get_value_for_datastore(self, model_instance):
begin = self.begin.get_value_for_datastore(model_instance)
end = self.end.get_value_for_datastore(model_instance)
if begin is not None and end is not None:
return range(begin, end)
class Country(db.Model):
begin_ipnum = db.IntegerProperty()
end_ipnum = db.IntegerProperty()
ip_range = IpRangeProperty(begin=begin_ipnum, end=end_ipnum)
The thinking is that after i add the custom property i can just import my dataset as is and then run queries on based on the ListProperty like so:
q = Country.gql('WHERE ip_range = :1', my_num_ipaddress)
When i try to insert new Country objects this fails though, complaning about not being able to create the name:
...
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/db/__init__.py", line 619, in _attr_name
return '_' + self.name
TypeError: cannot concatenate 'str' and 'IntegerProperty' objects
I tried defining an attr_name method for the new property or just setting self.name but that does not seem to help. Hopelessly stuck or heading in the right direction?
Short answer: Between queries aren't really supported at the moment. However, if you know a priori that your range is going to be relatively small, then you can fake it: just store a list on the entity with every number in the range. Then you can use a simple equality filter to get entities whose ranges contain a particular value. Obviously this won't work if your range is large. But here's how it would work:
class M(db.Model):
r = db.ListProperty(int)
# create an instance of M which has a range from `begin` to `end` (inclusive)
M(r=range(begin, end+1)).put()
# query to find instances of M which contain a value `v`
q = M.gql('WHERE r = :1', v)
The better solution (eventually - for now the following only works on the development server due to a bug (see issue 798). In theory, you can work around the limitations you mentioned and perform a range query by taking advantage of how db.ListProperty is queried. The idea is to store both the start and end of your range in a list (in your case, integers representing IP addresses). Then to get entities whose ranges contain some value v (i.e., between the two values in your list), you simply perform a query with two inequality filters on the list - one to ensure that v is at least as big as the smallest element in the list, and one to ensure that v is at least as small as the biggest element in the list.
Here's a simple example of how to implement this technique:
class M(db.Model):
r = db.ListProperty(int)
# create an instance of M which has a rnage from `begin` to `end` (inclusive)
M(r=[begin, end]).put()
# query to find instances of M which contain a value `v`
q = M.gql('WHERE r >= :1 AND r <= :1', v)
My solution doesn't follow the pattern you have requested, but I think it would work well on app engine. I'm using a list of strings of CIDR ranges to define the IP blocks instead of specific begin and end numbers.
from google.appengine.ext import db
class Country(db.Model):
subnets = db.StringListProperty()
country_code = db.StringProperty()
c = Country()
c.subnets = ['1.2.3.0/24', '1.2.0.0/16', '1.3.4.0/24']
c.country_code = 'US'
c.put()
c = Country()
c.subnets = ['2.2.3.0/24', '2.2.0.0/16', '2.3.4.0/24']
c.country_code = 'CA'
c.put()
# Search for 1.2.4.5 starting with most specific block and then expanding until found
result = Country.all().filter('subnets =', '1.2.4.5/32').fetch(1)
result = Country.all().filter('subnets =', '1.2.4.4/31').fetch(1)
result = Country.all().filter('subnets =', '1.2.4.4/30').fetch(1)
result = Country.all().filter('subnets =', '1.2.4.0/29').fetch(1)
# ... repeat until found
# optimize by starting with the largest routing prefix actually found in your data (probably not 32)