I want to store hierarchical entities in the datastore.
The children entities would have different kind, to represent something like this :
type EntityA struct {
Id string
LeafA *EntityA
LeafB *EntityB
SomeValue string
}
type EntityB struct {
Id string
OtherValue string
}
I planned to use ancestors, but it seems impossible to retrieve the children of a common ancestor that have different kind.
To retrieve the whole parent, is it possible to query all children of a common ancestors without specifying the kind ?
Or is there another possibility to solve this problem ?
From Using ancestor paths:
The complete key identifying the entity consists of a sequence of
kind-identifier pairs specifying its ancestor path and terminating
with those of the entity itself:
[Person:GreatGrandpa, Person:Grandpa, Person:Dad, Person:Me]
For a root entity, the ancestor path is empty and the key consists
solely of the entity's own kind and identifier:
[Person:GreatGrandpa]
In other words the key of any of the entities in an entity group (including the root entity - the common ancestor) is a list of keys for the entire ancestry path, which starts with the root entity's key.
So, to obtain the root entity key from any of the descendants just get the 1st element in that entity's key list.
Not familiar with go, but in python a particular item in the ancestry path is a tuple (pair) and to obtain the root entity key from any entity key it'd be something like this:
root_entity_key = ndb.Key(pairs=tuple(list(entity_key.pairs())[0])
Now, with the root entity (common ancestor) key you can perform ancestor queries for any descendant kind you desire.
Related
I don't have so much experience with MongoDB so for that I think it's better to try to explain my request through an example.
I have those two entities. The Tree has a list of at least one Leaf and a Leaf can't exists without a Tree.
data class Tree(
val id: UUID,
val name: String,
val leaves: List<Leaf>,
)
data class Leaf(
val id: UUID,
val names: List<String>,
)
I would like the id of the Leaf to be unique per Tree.
For example I can have the first Tree document with a Leaf which has the id: 250bb131-2134-5667-0000-000000000000 and another Tree can has a Leaf with the same id: 250bb131-2134-5667-0000-000000000000.
I am assuming you want to store the Trees and Leafs in a MongoDB given the title of your question.
First, know that MongoDB has no joining, unlike a relational (SQL) database, so you have two choices:
One collection. You have one Collection with all the Trees in it.
Tree: If you change the type of Tree.id to be BigInteger Mongo and leave the value null, Mongo will take care of allocating a unique key*.
You need to allocate unique keys yourself for the Leaf objects perhaps by using a Set to ensure you get a unique one
Two collections. You have two collections one for Leaf and one for Tree.
Tree: If you change the type of Tree.id to be BigInteger Mongo and leave the value null, Mongo will take care of allocating a unique key*.
Leaf: If you change the type of Leaf.id to be BigInteger Mongo and leave the value null, Mongo will take care of allocating a unique key*.
then you have to handle the reference from Tree.leaves. This is all manual - you need change the leaves List to contain references to the Leaf object keys and handle the fetch yourself, e.g. val leaves: List<BigInteger>,. (Mongo does have a DBRef which is no more than an formal reference to database/collection/id)
*Mongo's default key is an ObjectId that will serialize into BigInteger or you could (pollute?) your code with the org.bson.types.ObjectId type. This class has some useful properties, such as sorting in datetime ascending order. But more than that you should read how such keys are generated. Unlike a central RDBMS which is responsible for allocating a lock, issuing a key, and releasing the lock - Mongo's strategy is create the primary key on the client. How could this work safely? Well: it uses a combination of date time, a client (random) key and an increment (see spec). You could this idea yourself in option 1, where you need to guarantee a unique Leaf identifier by using the ObjectId constructor
I have the AdminFiles entity which holds files for all project entities (news, pages, events etc.). It is structured (abridged) like this:
id int(11) Auto Increment
entity varchar(255) // this holds the entity name, e.g. Page or News
entity_id int(11) NULL // this holds the entity ID
filename varchar(1000) // this holds the path to the file
I would love to access the files in the entities (Page, News etc.) with something like $entity->getFiles(). But I'm having trouble creating the relation, since it is not only constrained by the ID, but also the entity name. Is there any way to join this inside the Doctrine2 entity, or do I have to do this in the service?
This can be handled by Doctrine by using discriminator map feature. In your case there will be multiple entities inherited from base AdminFiles entity, each one will be specific for single type of files (e.g. PageFiles, NewsFiles and so on). You will need to establish mapping between entity column and these entities into #DiscriminatorMap annotation, then you will be able to fetch just one particular type of file by fetching (or using in association mapping) entity that represents some specific type of file.
You will also be able to even have entity properties that are specific to some particular type of file by using this feature.
Let's say I have a model:
class Pet(ndb.Model):
age = ndb.IntegerProperty(indexed=False)
name = ndb.StringProperty(indexed=True)
owner = ndb.KeyProperty(indexed=True)
And I have a list of keys named owners. To do a query for Pets I would do:
pets = Pets.query(Pets.owner.IN(owners)).fetch()
The problem is that this query returns the whole entity.
How can I do a projected query and get just the owner and the name?
Or how should I structure the data to just get the name and the owner.
I can do a projection for the name but I loose reference from the pet to the owner. And owner can't be in the projection.
As you have noticed, you can't do that with the exact context you mentioned, because you hit one of the Limitations on projections:
Properties referenced in an equality (=) or membership (IN) filter cannot be projected.
Since owner is used in a IN filter it can't be projected. Since you need the owner and you can't project it you'll have to drop the projection and thus you'll always get the entire entity.
One alternative would be to split your entity into 2 peer entities, always into a 1:1 relationship, using the same entity IDs:
class PetA(ndb.Model):
name = ndb.StringProperty(indexed=True)
owner = ndb.KeyProperty(indexed=True)
class PetB(ndb.Model):
age = ndb.IntegerProperty(indexed=False)
This way you can do the same query, except on PetA kind instead of the original Pet and the result you'd get would be the equivalent of the original projection query you were seeking.
Unfortunately this will only work with one or a very few such projection queries for the same entity, otherwise you'd need to split the entity in too many pieces. So you may have to compromise.
You can find more details about the entity splitting in re-using an entity's ID for other entities of different kinds - sane idea?
What is the clear difference of a Ancestor Key and a Parent Key?
Is it basically the same? How does it differently affect a get() method and a Query method?
From the GAE KeyFactory class, we can only see the Key parent and no such ancestor Key.
public static Key createKey(Key parent, String kind, String name) {
return createKey(parent, kind, name, (AppIdNamespace)null);
}
Then use the key like this:
Entity e = new Entity(key); // key may include a parent Key
In a typical query, like in a guestbook app. We can make a Key as an ancestor, so we can have different guestbooks to which Entities could be saved like:
Key guestbookKey = KeyFactory.createKey("Guestbook", guestbookName);
Query q = new Query(kind, guestbookKey); // guestbookKey = ancestor
So what is the difference of a Parent and Ancestor, are they basically the same thing without any difference, the same thing with just different notation?
A Parent means an immediate parent. An Ancestor is a more general term, encompassing immediate parents, parents of parents, parents of parents of parents, and so forth -- exactly the same use as in English.
An "entity group", the kind on which you can ask for an atomic transaction, need to be made up of entities with a common ancestor -- they don't need to all have the same (immediate) parent, but their lines of descent do need to meet at one common ancestor some way "up the line" (not necessarily the same number of steps for all entities involved).
When you createKey based on a parent key, that parent key may in turn have been made with its own parent -- and if that is so, then the parent's parent is not itself the parent of what you're creating now, but, it is an ancestor thereof.
Assumptions:
1) Google AppEngine has the concept of Entity Groups.
2) Entities in an entity group form a tree. However, as far as I understood, every put() to any entity in that tree will lock the whole tree (not just the immediate parent) for a while.
3) Users are allowed to write ca. 5 times per seconds to the tree.
4) There is no way to have a non-locking behaviour (i.e. by making them unindexed properties)
Would it be a smart idea to come up with my own parent-child model that does not use the built-in Key-functions (as those would create entity groups) but instead use some snytax convetions that I made up? This would allow me to retrieve a "child" entity via a query an compute the parent key.
The ancestor relationship used by entity groups can be modeled in your own code by using a list of references/keys to parent entities. Root entities will have none, children of roots will have just the root entity in their list, their children will have the root and their immediate parent, and so forth. This is how ancestors are implemented in App Engine for indexing purposes, and it'll permit you to make the same sorts of queries.
You can use a reference property:
class Parent(db.Model):
x = db.IntegerProperty()
class Child(db.Model):
parent = db.ReferenceProperty(
reference_class = Parent,
collection_name = 'children')
y = db.IntegerProperty()