I have two organisation in my datastore inside their own namespace. Lets say organisation1 present inside namespace1 and organisation2 present inside namespace2. I am retrieving organisation by its web-safe-key. lets say that web-safe-key of organisation1 is orgWebSafeKey1 and web-safe-key of organisation2 is orgWebSafeKey2. I am using following code to get an organisation:
NamespaceManager.set("namespace1");
Organisation organisation = (Organisation) ofy().load().key(Key.create(orgWebSafeKey1)).now();
above code works as I expected because organisation1 is present inside namespace1 and I am trying get that organisation in its namespace.
But if I just change the websafekey of the organisation then according to my expectaion below query should result "null" organisation because there is no organisation with key orgWebSafeKey2 inside namespace1. But practically it is giving me organisation2.
NamespaceManager.set("namespace1");
Organisation organisation = (Organisation) ofy().load().key(Key.create(orgWebSafeKey2)).now();
If the above query result is correct and expected according to objectify and datastore then can I assume that query by key works globally , across all the namespaces?
I also want confirmation that in this case Key.create(orgWebSafeKey2) will not change the namespace of the key? and query is running according to the namespace of the key not by NamespaceManager.set("namespace1")?
A Datastore Key contains the following components:
Project/App ID
Namespace
Entity Path (Ancestor Kind + ID/Name(zero or more), Final Entity Kind + ID/Name)
Since namespace is part of the key, lookup of an entity by Key always finds the right entity regardless of the namespace set by the NamespaceManager. In other words, a Key is a GUID that uniquely identifies an entity across all apps/projects.
Refer to the below link for more details/answers for your questions:
https://cloud.google.com/appengine/docs/java/multitenancy/multitenancy#Java_Using_namespaces_with_the_Datastore
Related
Here's what I have:
class A{
Ref<foo> b;
Ref<foo> c;
int code;
Date timestamp
}
The pseudo "where" clause of the SQL statement would like like this:
where b = object or (c = object and code = 1) order by timestamp
In plain English, give me all the records of A if b equals the specific object or if c equals the specified object when code equals 1. Order the result w/ timestamp.
Is the composite query part even possible w/ datastore (Objectify)? I really don't want to do two queries and merge the results, because I have to sort by timestamp.
Any help is appreciated.
P.S. I already tried
new FilterPredicate(b, EQUAL, object)
This didn't work, because the entity type is not a support type.
Thanks!
Pass a native datastore Key object to the FilterPredicate. The Google SDK Key, not the generic Objectify Key<?>.
Normally when filtering on properties, Objectify translates Ref<?> and Key<?> objects to native datastore keys for you. With the google-supplied FilterPredicate, that isn't an option. So you have to do the translation manually.
Objectify stores all Key<?> and Ref<?> fields and native datastore Keys, so you can freely interchange them (or even change the type of fields if you want).
I want to get an entity key knowing entity ID and an ancestor.
ID is unique within entity group defined by the ancestor.
It seems to me that it's not possible using ndb interface. As I understand datastore it may be caused by the fact that this operation requires full index scan to perform.
The workaround I used is to create a computed property in the model, which will contain the id part of the key. I'm able now to do an ancestor query and get the key
class SomeModel(ndb.Model):
ID = ndb.ComputedProperty( lambda self: self.key.id() )
#classmethod
def id_to_key(cls, identifier, ancestor):
return cls.query(cls.ID == identifier,
ancestor = ancestor.key ).get( keys_only = True)
It seems to work, but are there any better solutions to this problem?
Update
It seems that for datastore the natural solution is to use full paths instead of identifiers. Initially I thought it'd be too burdensome. After reading dragonx answer I redesigned my application. To my suprise everything looks much simpler now. Additional benefits are that my entities will use less space and I won't need additional indexes.
I ran into this problem too. I think you do have the solution.
The better solution would be to stop using IDs to reference entities, and store either the actual key or a full path.
Internally, I use keys instead of IDs.
On my rest API, I used to do http://url/kind/id (where id looked like "123") to fetch an entity. I modified that to provide the complete ancestor path to the entity: http://url/kind/ancestor-ancestor-id (789-456-123), I'd then parse that string, generate a key, and then get by key.
Since you have full information about your ancestor and you know your id, you could directly create your key and get the entity, as follows:
my_key = ndb.Key(Ancestor, ancestor.key.id(), SomeModel, id)
entity = my_key.get()
This way you avoid making a query that costs more than a get operation both in terms of money and speed.
Hope this helps.
I want to make a little addition to dargonx's answer.
In my application on front-end I use string representation of keys:
str(instance.key())
When I need to make some changes with instence even if it is a descendant I use only string representation of its key. For example I have key_str -- argument from request to delete instance':
instance = Kind.get(key_str)
instance.delete()
My solution is using urlsafe to get item without worry about parent id:
pk = ndb.Key(Product, 1234)
usafe = LocationItem.get_by_id(5678, parent=pk).key.urlsafe()
# now can get by urlsafe
item = ndb.Key(urlsafe=usafe)
print item
Quick Question. In the below code, you can see that the for loop (which takes all of the records in newTimecards and puts them as a variable called timecard) and adds the Resource_c to the resourceIds set. I'm confused about how this object is considered an ID data type. When an object is made in Salesforce does it automatically have an ID made, so that it knows Resource_c ID can be added to a set? Note that within the Resource_c Object there is also a field called Resource_ID_c. Resource_c within Timecard_c is a Master-Detail data type. Resource_c is the parent of Timecard_c.
Now that I think about it, resourceIds.add(timecard.Resource_c), does that reference the relationship between the two objects and then searches through Resource_c and adds the ID field Resource_ID_c automactically since it's a unique field?
Thanks for your help.
public class TimecardManager {
public class TimecardException extends Exception {}
public static void handleTimecardChange(List<Timecard__c> oldTimecards,
List<Timecard__c> newTimecards) {
Set<ID> resourceIds = new Set<ID>();
for (Timecard__c timecard : newTimecards) {
resourceIds.add(timecard.Resource__c);
}
Every object instance (and that means EVERY, including factory ones) has a unique organization level ID, whose field name is always Id, is covered by Apex type ID and is a case-sensitive string of 15 characters that also has an 18 character case-insensitive representation. The first three characters are object prefix code (e.g. 500 for a Case) so all instances of the same object share the same prefix. You see these values all across SF (for example in https://na1.salesforce.com/02s7000000BW59L the 02s7000000BW59L in the URL is the ID). When an instance of the object is created using INSERT DML operation, the salesforce automatically assigns unique value based on the prefix and the next available transactional sub ID, it all happens transparently to you.
This is not to be confused with object Name field which is a field you define when you create an object and which can be auto-incremented and so on (e.g. MYOBJ-{00000}) and which can have more meaning to a user than a cryptic ID
When you create a lookup or master-detail relationship it is ID that is being used to link the two instances, not the Name. In the above example Resource__c seems to be that lookup field and it contains Id value of row's master.
What the code does is it enumerates all resources used in timelines and builds a set of their IDs, the purpose of which is most probably to be used via WHERE Id IN :resourceIds clause to load resource details from master table.
mmix's answer is a great overview to what an ID is and where it comes from. To answer what I think is your specific question:
Any time there is a reference from one object to another (like here, between Timecard_c and Resource_c), the field representing the reference will be an ID. So, the for loop that calls resourceIds.add(timecard.Resource__c) is just building up your set of ID's (those 15-character strings). The timecard.Resource__c doesn't look through the Resource__c table to find the ID, timecard.Resource__c is the ID.
I have a Model called Version that looks like this:
from google.appengine.ext import db
import piece
class Version(db.Model):
"A particular version of a piece of writing."
parent_piece = db.ReferenceProperty(piece.Piece, collection_name='versions')
"The Piece to which this version belongs."
note = db.TextProperty()
"A note from the Author about this version."
content = db.TextProperty()
"The actual content of this version of the Piece."
published_on = db.DateProperty(auto_now_add=True)
"The date on which the version was published."
I would like to access instances of Version via their IDs, using Version.get_by_id(), but this call always returns None. I can see in the Datastore Viewer that they have ID values, and in the debugger, I can query for them but not use them:
>>> for each_ver in version.Version.all():
... print each_ver.key().id()
...
34
35
36
31
32
>>> a = version.Version.get_by_id(34)
>>> type(a)
<type 'NoneType'>
I see that there are plenty of questions here where people are able to use get_by_id() effectively just as I wish, and they do not see the results that I am seeing.
Could the problem be that each Version instance is a child in an Entity Group rather than a root of an Entity Group? Each Version lives in an Entity Group that looks like Member->Piece->Version. If that is the problem, is there a way that I can refer to Version entity without using its entire key? If that is not the problem, can anyone tell me what I can do to make get_by_id() work as expected?
Could the problem be that each Version
instance is a child in an Entity Group
rather than a root of an Entity Group?
Yes. An entity's key includes the keys of any parent entities.
If that is the problem, is there a
way that I can refer to Version entity
without using its entire key?
No. An entity is uniquely identified only by its entire key, which includes the keys of all the parent entities. If you know the kinds of its parent entities, though, you can use db.Key.from_path to construct the key from the chain of IDs or key names.
I had your same problem but in ndb.Model and I found that I need to convert the ID to an int. So maybe using version.Version.get_by_id(int(34)) can solve your problem.
I have a Model UnitPattern, which reference another Model UnitPatternSet
e.g.
class UnitPattern(db.Model):
unit_pattern_set = db.ReferenceProperty(UnitPatternSet)
in my view I want to display all UnitPatterns having unit_pattern_set refrences as None, but query UnitPattern.all().filter("unit_pattern_set =", None) returns nothing, though I have total 5 UnitPatterns, out of which 2 have 'unit_pattern_set' set and 3 doesn't have
e.g.
print 'Total',UnitPattern.all().count()
print 'ref set',UnitPattern.all().filter("unit_pattern_set !=", None).count()
print 'ref not set',UnitPattern.all().filter("unit_pattern_set =", None).count()
outputs:
Total 5
ref set 2
ref not set 0
Shouldn't sum of query 2 and 3 be equal to query 1 ?
Reason seems to be that I added reference property unit_pattern_set later on, and these UnitPattern objects existed before that, but then how can I filter such entities?
This is described succinctly in the docs:
An index only contains entities that
have every property referred to by the
index. If an entity does not have a
property referred to by an index, the
entity will not appear in the index,
and will never be a result for the
query that uses the index.
Note that
the App Engine datastore makes a
distinction between an entity that
does not possess a property and an
entity that possesses the property
with a null value (None). If you want
every entity of a kind to be a
potential result for a query, you can
use a data model that assigns a
default value (such as None) to
properties used by query filters.
In your case, you have 3 entities that don't have the unit_pattern_set property set at all (because that property wasn't defined in the Model at the time those entities were created) - therefore those properties doesn't exist in the database representation of that entity, therefore that entity does not appear in the index of that property for that kind of entity.
Dan Sanderson's book Programming Google App Engine explains this in great detail on ~page 150 (unfortunately not available in the Google Books preview)
To fix the models you already have, you'll have to iterate over a query on UnitPattern (I've not tested the following code, please check it before you run it on your live data):
patterns = UnitPattern.all()
for pattern in patterns:
if not pattern.unit_pattern_set:
pattern.unit_pattern_set = None
pattern.put()
Edit: Also, the Updating you model's schema article discuss strategies you can use to handle schema changes such as this in future. However, that article is quite old and its method requires a web browser to keep hitting a url to trigger the next job to update more records - now that Task Queues exist, you could use a series of Tasks to make the change. The article on using deferred.defer has a framework you could utilise - it does a small amount of work, catches the DeadlineExceededError, and uses the handler to queue a new task which picks up where the current task left off.