I'm trying to get entity Key by its int Id. (not the entity itself, but it's Key) (in the long run I do it to find entities parent)
Data from DatastoreViewer:
Entity Kind
File
Entity Key
ag9kZXZ-dHJhc2hib3hhcHByIgsSBEZpbGUYgICAgICAwAoMCxIERmlsZRiAgICAgIDACww
ID
6473924464345088
Parent
ag9kZXZ-dHJhc2hib3hhcHByEQsSBEZpbGUYgICAgICAwAoM
File: id=5910974510923776
I do it like this:
k := datastore.NewKey(c, "File", "", 6473924464345088, nil)
currentDirQuery := datastore.NewQuery("File").Filter("__key__ =", k).KeysOnly()
keys, err := currentDirQuery.GetAll(c, nil)
The length if keys is 0. What do I do wrong?
If you have the key already why are doing a keys only query matching the key ? Why don't you just do a datastore.Get() with the key?
As to why your keys_only query is not working, you are not including the ancestor in the key you are constructing , the key in your example has a parent you showed ag9kZXZ-dHJhc2hib3hhcHByEQsSBEZpbGUYgICAgICAwAoM this urlsafe version of the key must have a parent if you are specifying an ancestor.
Using python we can decode this key
> ndb.Key(urlsafe="ag9kZXZ-dHJhc2hib3hhcHByIgsSBEZpbGUYgICAgICAwAoMCxIERmlsZRiAgICAgIDACww")
Key('File', 5910974510923776, 'File', 6473924464345088, app='dev~trashboxapp')
See the parent Key is Key('File', 5910974510923776)
You can not perform a partial match on a child with the key you created for the query. You can only perform ancestor queries which will return the ancestor and all of it's children irrespective of the depth of the heirarchy.
This also means a datastore.Get() will fail with the key you have created in your example code.
So construct you key so that it includes the ancestor - see the docs https://developers.google.com/appengine/docs/go/datastore/entities#Go_Retrieving_an_entity
But to be honest what you are doing is completely redundant unless it's just an excercise in understanding whats going on and your trying to roundtrip a key -> query -> key
Related
If you're dealing with a hierarchy of records, where most of the keys have ancestors, will you have to create a chain of all of the keys before you can retrieve a leaf?
Example (in Go):
rootKey = datastore.NewKey(ctx, "EntityType", "", id1, nil)
secondGenKey = datastore.NewKey(ctx, "EntityType", "", id2, rootKey)
thirdGenKey = datastore.NewKey(ctx, "EntityType", "", id3, rootKey)
How do you get the record described by thirdGenKey without having to declare the keys for all of the levels of the hierarchy above it?
In order to get an individual entity, its key must be globally unique - this is enforced through each entity key being unique within its entity group. The ancestor path forms an intrinsic part of the entity key for this reason.
So, the only way to get a single entity with strong consistency is to specify its ancestor path. This can be done either with a get-by-key or with an ancestor query.
If you don't know the full ancestor path, your only option is to query on a property of the entity, but bear in mind that:
this may not be unique within your application
you will be subject to eventual consistency.
To supplement tx802's answer:
If you want to load an entity by key, you need its key. If the key is such a key that has a parent, in order to form / create the key, you also need the parent key to be created prior. The parent key is part of the key, just like the numeric ID or the string name.
Looking from the implementation perspective: datastore.Key is a struct:
type Key struct {
kind string
stringID string
intID int64
parent *Key
appID string
namespace string
}
In order to construct a Key which has a parent, you must construct the parent key too, recursively. If you're finding it too verbose to always create the key hierarchy, you can create a helper function for it.
For the sake of simplicity, let's assume all keys use the same entity name, and we only use numeric IDs. It may look like this:
func createKey(ctx context.Context, entity string, ids ...int) (k *datastore.Key) {
for _, id := range ids {
k = datastore.NewKey(ctx, entity, "", id, k)
}
return
}
With this helper function, your example is reduced to this:
k2 := createKey(ctx, "EntityType", id1, id2)
k3 := createKey(ctx, "EntityType", id1, id3)
I have the key id of the entity I want to delete, but when trying to delete using get_by_id, it is not deleting the entity (it's not doing anything).
The numerical id (I assume its a number, not a string) is stored in the 'key' get data:
d = car_database.Car()
ident = self.request.get('key')
d.get_by_id(ident).delete()
The id in this instance is: 5659313586569216
Thanks for the help
This is incorrect. Please look at the documentation on deleting entities - https://cloud.google.com/appengine/docs/python/ndb/entities#deleting_entities - it's pretty clear what you should be doing.
In your case you should be
ident = self.request.get('key')
d = car_database.Car.get_by_id(ident)
if d:
d.key.delete()
Note that get_by_id is in fact a classmethod so you don't need to instantiate in instance of Car to use it.
And delete is a method of the Key, not of Model.
I need to check the existence of a key (i.e. an username). It seems that KEY_RESERVED_PROPERTY is a special key available for the java api that you can use to achieve the best performance and strong consistency so I'm wondering if there is any equivalent in Go.
Currently I'm considering using a query with the username as ancestor + KeysOnly().
If you look at the docs, KEY_RESERVED_PROPERTY is nothing but a property to refer to the key:
A reserved property name used to refer to the key of the entity. This string can be used for filtering and sorting by the entity key itself.
So this is nothing magical, you could do the same thing in Go with the __key__ property, as stated in the docs:
Key filters
To filter on the value of an entity's key, use the special property __key__:
q := datastore.NewQuery("Person").Filter("__key__ >", lastSeenKey)
I need to check the existence of a key (i.e. an username).
You can also do that by attempting to load the entity by key using the datastore.Get() function. A return value of ErrNoSuchEntity means no entity exists with the specified key:
if err := datastore.Get(c, key, dst); err == datastore.ErrNoSuchEntity {
// Key doesn't exist
}
Ok so I have an application that uses GAE and consequently the datastore.
Say I have multiple companies A, B and C and I have within each company Employees X,Y and Z. The relationship between a company and employee will be OneToMany, with the company being the owner. This results in the Company Key being of the form
long id = 4504699138998272; // Random Example
Key CompanyKey = KeyFactory.createKey(Company.class.getSimpleName(), id);
and the employee key would be of the form
long id2 = 5630599045840896;
Key EmployeeKey = KeyFactory.createKey(CompanyKey,Employee.class.getSimpleName(),id2);
all fine and well and there is no problem, until in the front end, during jsp representation. Sometimes I would need to generate a report, or open an Employees profile, in which case the div containing his information would get an id as follows
<div class="employeeInfo" id="<%=employee.getKey().getId()%>" > .....</div>
and this div has an onclick / submit event, that will ajax the new modifications to the employee profile to servelet, at which point I have to specify the primary key of the employee, (which I thought I could easily get from the div id), but it didnt work server side.
The problem is I know the Employees String portion of the Key and the long portion, but not the Parent key. To save time I tried this and it didnt work
Key key = KeyFactory.creatKey(Employee.class.getSimpleName(); id);
Employee X = em.find(Employee.class,key);
X is always returned null.
I would really appreciate any idea of how to find or "query" Entities by keys without knowing their parents key (as I would hate having to re-adjust Entity classes)
Thanks alot !!
An Entity key and its parents cannot be separated. It's called ancestor path, a chain composed of entity kinds and ids.
So, in your example ancestor paths will look like this:
CompanyKey: ("Company", 4504699138998272)
EmployeeKey: ("Company", 4504699138998272, "Employee", 5630599045840896)
A key composed only of ("Employee", 5630599045840896) is a completely different one comparing to the EmployeeKey even though both keys end with the same values. Think of concatenating elements into a single "string" and comparing final values, they will never match.
One thing you can do is use encoded keys instead of their id values:
String encodedKey = KeyFactory.keyToString(EmployeeKey);
Key decodedKey = KeyFactory.stringToKey(encodedKey);
decodedKey.equals(EmployeeKey); // true
More about Ancestor Paths:
https://developers.google.com/appengine/docs/java/datastore/entities#Java_Ancestor_paths
KeyFactory Java doc:
https://developers.google.com/appengine/docs/java/javadoc/com/google/appengine/api/datastore/KeyFactory#keyToString(com.google.appengine.api.datastore.Key)
I want to get an entity key knowing entity ID and an ancestor.
ID is unique within entity group defined by the ancestor.
It seems to me that it's not possible using ndb interface. As I understand datastore it may be caused by the fact that this operation requires full index scan to perform.
The workaround I used is to create a computed property in the model, which will contain the id part of the key. I'm able now to do an ancestor query and get the key
class SomeModel(ndb.Model):
ID = ndb.ComputedProperty( lambda self: self.key.id() )
#classmethod
def id_to_key(cls, identifier, ancestor):
return cls.query(cls.ID == identifier,
ancestor = ancestor.key ).get( keys_only = True)
It seems to work, but are there any better solutions to this problem?
Update
It seems that for datastore the natural solution is to use full paths instead of identifiers. Initially I thought it'd be too burdensome. After reading dragonx answer I redesigned my application. To my suprise everything looks much simpler now. Additional benefits are that my entities will use less space and I won't need additional indexes.
I ran into this problem too. I think you do have the solution.
The better solution would be to stop using IDs to reference entities, and store either the actual key or a full path.
Internally, I use keys instead of IDs.
On my rest API, I used to do http://url/kind/id (where id looked like "123") to fetch an entity. I modified that to provide the complete ancestor path to the entity: http://url/kind/ancestor-ancestor-id (789-456-123), I'd then parse that string, generate a key, and then get by key.
Since you have full information about your ancestor and you know your id, you could directly create your key and get the entity, as follows:
my_key = ndb.Key(Ancestor, ancestor.key.id(), SomeModel, id)
entity = my_key.get()
This way you avoid making a query that costs more than a get operation both in terms of money and speed.
Hope this helps.
I want to make a little addition to dargonx's answer.
In my application on front-end I use string representation of keys:
str(instance.key())
When I need to make some changes with instence even if it is a descendant I use only string representation of its key. For example I have key_str -- argument from request to delete instance':
instance = Kind.get(key_str)
instance.delete()
My solution is using urlsafe to get item without worry about parent id:
pk = ndb.Key(Product, 1234)
usafe = LocationItem.get_by_id(5678, parent=pk).key.urlsafe()
# now can get by urlsafe
item = ndb.Key(urlsafe=usafe)
print item