I write these codes
Query q = pm.newQuery(User.class);
q.setFilter("textPosts != null"); // textPosts is a Text
List<User> users = (List<User>) q.execute();
resp.getWriter().println("user num: " + users.size());
And I get
user num: 0
I'm sure that the number should greater than 0.
I probably miss something important.
Thanks in advance!
Querying Unindexed properties:
A query with a filter or sort order on a property will never match an entity with that property unindexed
So your entity is not found because Text properties are never indexed.
This is also relevant (but in your case Text being unindexed prevails) - entities with non-existing properties are not found with queries:
The App Engine datastore distinguishes between an entity without a given property and an entity with a null value for a property. JDO does not support this distinction: every field of an object has a value, possibly null.
So if you have an Entity without a property, then it will not be found with a query, since a query requires that the property exists.
Maybe because Text will not be indexed for query purposes, as it specified in javadoc: http://code.google.com/intl/en/appengine/docs/java/javadoc/com/google/appengine/api/datastore/Text.html
Btw, i'm sure that it must throw exception at this case.
Related
My TwAccount is
class TwAccount(ndb.Model):
100 different properties here
error = ndb.IntegerProperty(repeated=True)
I try:
twaccount_dbs = model.TwAccount.query().filter(ndb.GenericProperty('followuserfollowme') == True)
it returns 1 entity
But I only want to query 1 property.
twaccount_dbs = model.TwAccount.query().filter(ndb.GenericProperty('followuserfollowme') == True).fetch(projection=["error"])
then it returns 0 entity.
I try
twaccount_dbs = model.TwAccount.query().filter(ndb.GenericProperty('followuserfollowme') == True).fetch(projection=[model.TwAccount.error])
but it also returns 0 entity
I expect it returns 1 entity.
Update 1:
I figure out that if error is an emtpy (so that does not exist), then the projection query will return 0
My objective is to query all entities in TwAccount. If error is empty, then do deferred.defer(function,entity_key).
I want to use projection query to save the cost of reading. Is it impossible?
Unfortunately you cannot filter query results by an unset/empty property:
From Index definition and structure:
An entity is included in the index only if it has an indexed value set
for every property used in the index; if the index definition refers
to a property for which the entity has no value, that entity will not
appear in the index and hence will never be returned as a result for
any query based on the index.
See also related AppEngine: Query datastore for records with <missing> value
Here's what I have:
class A{
Ref<foo> b;
Ref<foo> c;
int code;
Date timestamp
}
The pseudo "where" clause of the SQL statement would like like this:
where b = object or (c = object and code = 1) order by timestamp
In plain English, give me all the records of A if b equals the specific object or if c equals the specified object when code equals 1. Order the result w/ timestamp.
Is the composite query part even possible w/ datastore (Objectify)? I really don't want to do two queries and merge the results, because I have to sort by timestamp.
Any help is appreciated.
P.S. I already tried
new FilterPredicate(b, EQUAL, object)
This didn't work, because the entity type is not a support type.
Thanks!
Pass a native datastore Key object to the FilterPredicate. The Google SDK Key, not the generic Objectify Key<?>.
Normally when filtering on properties, Objectify translates Ref<?> and Key<?> objects to native datastore keys for you. With the google-supplied FilterPredicate, that isn't an option. So you have to do the translation manually.
Objectify stores all Key<?> and Ref<?> fields and native datastore Keys, so you can freely interchange them (or even change the type of fields if you want).
I want to get an entity key knowing entity ID and an ancestor.
ID is unique within entity group defined by the ancestor.
It seems to me that it's not possible using ndb interface. As I understand datastore it may be caused by the fact that this operation requires full index scan to perform.
The workaround I used is to create a computed property in the model, which will contain the id part of the key. I'm able now to do an ancestor query and get the key
class SomeModel(ndb.Model):
ID = ndb.ComputedProperty( lambda self: self.key.id() )
#classmethod
def id_to_key(cls, identifier, ancestor):
return cls.query(cls.ID == identifier,
ancestor = ancestor.key ).get( keys_only = True)
It seems to work, but are there any better solutions to this problem?
Update
It seems that for datastore the natural solution is to use full paths instead of identifiers. Initially I thought it'd be too burdensome. After reading dragonx answer I redesigned my application. To my suprise everything looks much simpler now. Additional benefits are that my entities will use less space and I won't need additional indexes.
I ran into this problem too. I think you do have the solution.
The better solution would be to stop using IDs to reference entities, and store either the actual key or a full path.
Internally, I use keys instead of IDs.
On my rest API, I used to do http://url/kind/id (where id looked like "123") to fetch an entity. I modified that to provide the complete ancestor path to the entity: http://url/kind/ancestor-ancestor-id (789-456-123), I'd then parse that string, generate a key, and then get by key.
Since you have full information about your ancestor and you know your id, you could directly create your key and get the entity, as follows:
my_key = ndb.Key(Ancestor, ancestor.key.id(), SomeModel, id)
entity = my_key.get()
This way you avoid making a query that costs more than a get operation both in terms of money and speed.
Hope this helps.
I want to make a little addition to dargonx's answer.
In my application on front-end I use string representation of keys:
str(instance.key())
When I need to make some changes with instence even if it is a descendant I use only string representation of its key. For example I have key_str -- argument from request to delete instance':
instance = Kind.get(key_str)
instance.delete()
My solution is using urlsafe to get item without worry about parent id:
pk = ndb.Key(Product, 1234)
usafe = LocationItem.get_by_id(5678, parent=pk).key.urlsafe()
# now can get by urlsafe
item = ndb.Key(urlsafe=usafe)
print item
in an app i have an entity that contains a list of other entities (let's say an event holding a list of assigned employees)
using objectify - i need to find all the events a particular employee is assigned to.
is there a basic way to filter a query if it contains the parameter - kind of the opposite of the query in
... quick pseudocode
findAll(Employee employee) {
...
return ofy.query(Event.class).filter("employees.contains", employee).list();
}
any help would be greatly appreciated
i tried just doing filter("employees", employee) after seeing this http://groups.google.com/group/objectify-appengine/browse_thread/thread/77ba676192c08e20 - but unfortunately this returns me an empty list
currently i'm doing something really inefficient - going through each event, iterating through the employees and adding them to a new list if it contains the given employee just to have something that works - i know this is not right though
let me add one thing,
the above query is not actually what it is, i was just using that because i did not think this would make a difference.
The Employee and Events are in the same entity group with Business as a parent
the actual query i am using is the following
ofy.query(Event.class).ancestor(businessKey).filter("employees", employee).list();
unfortunately this is still returning an empty list - does having the ancestor(key) in there mess up the filter?
solution, the employees field was not indexed correctly.
I added the datastore-indexes file to create a composite index, but was testing originally on a value that I added before the employees field was indexed, this was something stupid i was doing - simply having an index on the "business" field and the "employees" field fixed everything. the datastore-indexes file did not appear to be necessary, after deleting it and trying again everything worked fine.
Generally, you do this one of two ways:
Put a property of Set<Key<Employee>> on the Event
or
Put a property of Set<Key<Event>> on the Employee
You could also create a relationship entity, but if you're just doing filtering on values with relatively low counts, usually it's easier to just put the set property on one entity or the other.
Then filter as you describe:
ofy.query(Event.class).filter("employees", employee).list()
or
ofy.query(Employee.class).filter("events", event).list()
The list property should hold a Keys to the target entity. If you pass in an entity to the filter() method, Objectify will understand that you want to filter by the key instead.
Example :
/***************************************************/
#Entity
#Cache
public class News {
#Id Long id;
String news ;
#Index List<Long> friend_list = new ArrayList<Long>();
// My friends who can see my news , exemele : friend_list.add(id_f1); friend_list.add(id_f2); friend_list.add(id_f3);
//To make an operation on "friend_list", it is obligatory to index it
}
/*************************************************/
public News(Long id_f){
List<Long> friend_id = new ArrayList<Long>();
friend_id.add(id_f);
Query<Nesw> query = ofy().load().type(News.class).filter("friend_list in",friend_id).limit(limit);
//To filter a list, just after the name of the field you want to filter, add "IN".
//here ==> .filter("friend_list in",friend_id);
// if friend_list contains "id_friend" ==> the query return value
.........
}
I have a Model UnitPattern, which reference another Model UnitPatternSet
e.g.
class UnitPattern(db.Model):
unit_pattern_set = db.ReferenceProperty(UnitPatternSet)
in my view I want to display all UnitPatterns having unit_pattern_set refrences as None, but query UnitPattern.all().filter("unit_pattern_set =", None) returns nothing, though I have total 5 UnitPatterns, out of which 2 have 'unit_pattern_set' set and 3 doesn't have
e.g.
print 'Total',UnitPattern.all().count()
print 'ref set',UnitPattern.all().filter("unit_pattern_set !=", None).count()
print 'ref not set',UnitPattern.all().filter("unit_pattern_set =", None).count()
outputs:
Total 5
ref set 2
ref not set 0
Shouldn't sum of query 2 and 3 be equal to query 1 ?
Reason seems to be that I added reference property unit_pattern_set later on, and these UnitPattern objects existed before that, but then how can I filter such entities?
This is described succinctly in the docs:
An index only contains entities that
have every property referred to by the
index. If an entity does not have a
property referred to by an index, the
entity will not appear in the index,
and will never be a result for the
query that uses the index.
Note that
the App Engine datastore makes a
distinction between an entity that
does not possess a property and an
entity that possesses the property
with a null value (None). If you want
every entity of a kind to be a
potential result for a query, you can
use a data model that assigns a
default value (such as None) to
properties used by query filters.
In your case, you have 3 entities that don't have the unit_pattern_set property set at all (because that property wasn't defined in the Model at the time those entities were created) - therefore those properties doesn't exist in the database representation of that entity, therefore that entity does not appear in the index of that property for that kind of entity.
Dan Sanderson's book Programming Google App Engine explains this in great detail on ~page 150 (unfortunately not available in the Google Books preview)
To fix the models you already have, you'll have to iterate over a query on UnitPattern (I've not tested the following code, please check it before you run it on your live data):
patterns = UnitPattern.all()
for pattern in patterns:
if not pattern.unit_pattern_set:
pattern.unit_pattern_set = None
pattern.put()
Edit: Also, the Updating you model's schema article discuss strategies you can use to handle schema changes such as this in future. However, that article is quite old and its method requires a web browser to keep hitting a url to trigger the next job to update more records - now that Task Queues exist, you could use a series of Tasks to make the change. The article on using deferred.defer has a framework you could utilise - it does a small amount of work, catches the DeadlineExceededError, and uses the handler to queue a new task which picks up where the current task left off.