The Gaelyk tutorial provides some nice low level wrappers to the datastore and this Gaelyk google groups article describes an easy way to model relationships by simply storing the keys in a collection on the entity.
My question is how can I perform queries on the values within the collections? Here is some example code to clarify...
def b1 = new Entity("Book")
b1.title = "Book1"
b1.save()
def b2 = new Entity("Book")
b2.title = "Book2"
b2.save()
def author1 = new Entity("Author")
author1.books = [b1.key, b2.key]
author1.name = "Chris"
author1.save()
// It is easy to simply query the Author entity for a standard string property
def query = new Query("Author")
query.addFilter("name", Query.FilterOperator.EQUAL, "Chris")
PreparedQuery preparedQuery = datastore.prepare(query)
def authors = preparedQuery.asList(withLimit(1))
assert authors[0].name == "Chris"
// But I can't find out how to query a collection property eg books on the Author entity
def query2 = new Query("Author")
// I would like to do something to return the entity if the value is in the collection property
// eg if there could be something like 'CONTAINS' criteria ...
// Unfortunately Query.FilterOperator.CONTAINS doesn't exist...
query2.addFilter("books", Query.FilterOperator.CONTAINS, b2.key)
PreparedQuery preparedQuery2 = datastore.prepare(query2)
def authors2 = preparedQuery2.asList(withLimit(1))
assert authors2[0].name == "Chris"
How can I create a query that that searches for matches within a collection property of an Entity? ie how to recreate the functionality of the mythical 'FilterOperator.CONTAINS' query above?
Answering just for the sake of users landing to this page in future:
Query.FilterOperator.EQUAL will find inside the list of keys as well (it works as CONTAINS, in case of lists). So now your second case looks like:
def query2 = new Query("Author")
query2.addFilter("books", Query.FilterOperator.EQUAL, b2.key)
PreparedQuery preparedQuery2 = datastore.prepare(query2)
def authors2 = preparedQuery2.asList(withLimit(1))
assert authors2[0].name == "Chris"
and the assertion passes :)
This may look strange at first sight but it is indeed a great feature of DataStore.
One way to do this would be to have the author key as one of the fields in your entity "Book". Therefore, you can query the book first and then get the authors for it.
def author1 = new Entity("Author")
author1.name = "Chris"
author1.save()
def b1 = new Entity("Book")
b1.title = "Book1"
b1.authors = [author1.key]
b1.save()
author1.books = [ b1.key ]
author1.save()
def book = datastore.get(b1.key)
def chrisAuthors = book.authors.findAll { authorKey -> datastore.get(authorKey).name == 'Chris' }
assert chrisAuthors.size() == 1
By the way, a little update on this topic: you might want to have a look at the new Query DSL feature of Gaelyk 1.0, which was released recently.
http://gaelyk.appspot.com/tutorial/app-engine-shortcuts#query
This should simplify a lot of the queries one can make against the datastore.
Related
how can I do a query by ancestor according to the user_id that I get from the token ?
This is my code:
class MyUserClass(EndPointModel):
user_object = ndb.UserProperty(required=True, indexed=False)
class MyModel(EndPointModel):
...
#MyModel.method(user_required=True, name="model.add", path="model")
def add_mymodel(self, my_model):
gplus_user_id = auth_util.get_google_plus_user_id()
if gplus_user_id is None:
raise endpoints.ForbiddenException(NidAppUser.NO_GPLUS_ID)
user = MyUserClass.get_or_insert(gplus_user_id, user_object=endpoints.get_current_user())
my_model.parent = user.key.string_id()
my_model.put()
return my_model
#MyModel.query_method(user_required=True, name="list.mymodel", path="models",
query_fields=('order','attr1',))
def list_models(self, query):
gplus_user_id = auth_util.get_google_plus_user_id()
if gplus_user_id is None:
raise endpoints.ForbiddenException(NidAppUser.NO_GPLUS_ID)
## Here or in thsi decorator function I want to do something like
## query.ancestor.(ndb.Key(MyClassUser, gplus_user_id)) (such as the DB Datastore)
## to return only the query of MyModels that belong to the current user
return query
and I need to create a query ancestor by the key of MyUserClass key. I don't want to use the solution to add a field owner in my model because query ancestor is more fast than filters, and it really help me to organize my db.
Thank you
You can construct a query based on ancestor like this:
myancestorkey = ndb.Key(MyUserClass, gplus_user_id)
myquery = MyModel.query(ancestor=myancestorkey)
With endpoints_proto_datastore you can do something like this:
#MyModel.query_method(user_required=True, name="list.mymodel",
path="models", query_fields=('order','attr1',))
def list_models(self, query):
# Returns only MyModels that belong to the current g+ user
return query.filter(MyModel.parent == auth_util.
get_google_plus_user_id())
I am a beginner in Java and the Google App Engine (java).
I am trying to make a linked list type of model to link up entities together.
For example, I would like Class Fruit to be able to hold or take in a list of fruits, like Apple, Oranges...etc.
I was thinking of using the entity key as pointers between Class Fruit, and class Apple or class Orange.
Is there a way to grab an entity's key, and place it into another entity?
Better yet, grab multiple entity keys and place it into one entity?
Take a look at Objectify - https://code.google.com/p/objectify-appengine/ they have a really nice wrapper on top of the datastore that handles doing this for you.
You can also do this with Mungo-Appengine which has a MongoDB-like syntax with zero-configuration:
Mungo mungo = new Mungo();
DB fruitsDB = mungo.getDB("fruitsDB");
DBCollection fruits = fruitsDB.createCollection("fruits");
DBObject apple = new BasicDBObject("name", "apple")
.put("color", "red");
DBObject orange = new BasicDBObject("name", "orange")
.put("color", "orange")
.put("weight", 1);
DBObject myBasket = new BasicDBObject("name", "myBasket")
.put("fruits", Lists.newArrayList(apple, orange)); // Google Guava
fruits.insert(myBasket); // Done!
DBObject theBasket = fruits.findOne("{'name' : 'myBasket'}"); // Get it
List<DBObject> theFruits = theBasket.get("fruits");
Hope this helps.
I´m developing a grails app, and I already have a domain class "ExtendedUser" wich has info about users like: "name", "bio", "birthDate". Now I´m planning to do statistics about user´s age so I have created another controller "StatisticsController" and the idea is to store all the birthDates in a local array so I can manage multiple calculations with it
class StatisticsController {
// #Secured(["ROLE_COMPANY"])
def teststat(){
def user = ExtendedUser.findAll() //A list with all of the users
def emptyList = [] //AN empty list to store all the birthdates
def k = 0
while (k<=user.size()){
emptyList.add(user[k].birthDate) //Add a new birthdate to the emptyList (The Error)
k++
}
[age: user]
}
}
When I test, it shows me this error message: Cannot get property 'birthDate' on null object
So my question is how is the best way to store all the birthdates in an single array or list, so I can make calculations with it. Thank you
I prefer to .each() in groovy as much as possible. Read about groovy looping here.
For this try something like:
user.each() {
emptylist.push(it.birthdate) //'it' is the name of the default iterator created by the .each()
}
I don't have a grails environment set up on this computer so that is right off the top of my head without being tested but give it a shot.
I would use this approach:
def birthDates = ExtendedUser.findAll().collect { it.birthDate }
The collect method transforms each element of the collection and returns the transformed collection. In this case, users are being transformed into their birth dates.
Can you try:
List dates = ExtendedUser.findAll().birthDate
I want to create two objects and link them via a parent child relationship in C# using the Metadata API.
I can create objects and 'custom' fields for the objects via the metadata, but the service just ignores the field def for the relationship.
By snipet for the fields are as follows:
CustomField[] fields = new CustomField[] { new CustomField()
{
type = FieldType.Text,
label = "FirstName",
length = 50,
lengthSpecified = true,
fullName = "LJUTestObject__c.FirstName__c"
},
new CustomField()
{
type = FieldType.Text,
label = "LastName",
length = 50,
lengthSpecified = true,
fullName = "LJUTestObject__c.Lastname__c"
},
new CustomField()
{
type = FieldType.Text,
label = "Postcode",
length = 50,
lengthSpecified = true,
fullName = "LJUTestChildObject__c.Postcode__c"
},
new CustomField()
{
type = FieldType.MasterDetail,
relationshipLabel = "PostcodeLookup",
relationshipName = "LJUTestObject__c.LJUTestObject_Id__c",
relationshipOrder = 0,
relationshipOrderSpecified = true,
fullName = "LJUTestChildObject__c.Lookup__r"
}
};
The parent object looks like:
LJUTestObject
ID,
FirstName, Text(50)
LastName, Text(50)
The child objext looks like:
LJUTestChildObject
ID,
Postcode, Text(50)
I want to link the parent to the child so one "LJUTestObject", can have many "LJUTestChildObjects".
What values do I need for FieldType, RelationshipName, and RelationshipOrder to make this happen?
TL;DR:
Use this as a template for accomplishing what you want:
var cf = new CustomField();
cf.fullName = "ChildCustomObject__c.ParentCustomField__c";
cf.type = FieldType.MasterDetail;
cf.typeSpecified = true;
cf.label = "Parent Or Whatever You Want This To Be Called In The UI";
cf.referenceTo = "ParentCustomObject__c";
cf.relationshipName = "ParentOrWhateverYouWantThisToBeCalledInternally";
cf.relationshipLabel = "This is an optional label";
var aUpsertResponse = smc.upsertMetadata(metadataSession, null, null, new Metadata[] { cf });
The key difference:
The natural temptation is to put the CustomField instances into the fields array of a CustomObject, and pass that CustomObject to the Salesforce Metadata API. And this does work for most data fields, but it seems that it does not work for relationship fields.
Instead, pass the CustomField directly to the Salesforce Metadata API, not wrapped in a CustomObject.
Those muted errors:
Turns out that errors are occurring, and the Salesforce Metadata API knows about them, but doesn't bother telling you about them when they occur for CustomFields nested inside a CustomObject.
By passing the CustomField directly to the Metadata API (not wrapped in a CustomObject), the call to upsertMetadata will still return without an exception being thrown (as it was already doing for you), but this time, if something goes wrong, upsertResponse[0].success will be false instead of true, and upsertResponse[0].errors will give you more information.
Other gotchas
Must specify referenceTo, and if it doesn't match the name of an existing built-in or custom object, the error message will be the same as if you had not specified referenceTo at all.
fullName should end in __c not __r. __r is for relationship names, but remember that fullName is specifying the field name, not the relationship name.
relationshipName - I got it working by not including __r on the end, and not including the custom object name at the start. I haven't tested to be sure other ways don't work, but be aware that at the very least, you don't need to have those extra components in the relationshipName.
Remember generally that anything with label in its name is probably for display to users in the UI, and thus can have spaces in it to be nicely formatted the way users expect.
Salesforce... really???
(mini rant warning)
The Salesforce Metadata API is unintuitive and poorly documented. That's why you got stuck on such a simple thing. That's why no-one knew the answer to your question. That's why, four years later, I got stuck on the same thing. Creating relationships is one of the main things you would want to do with the Salesforce Metadata API, and yet it has been this difficult to figure out, for this long. C'mon Salesforce, we know you're a sales company more than a tech company, but you earn trazillions of dollars and are happy to show it off - invest a little more in a better API experience for the developers who invest in learning your platform.
I've not created these through the meta data API like this myself, but I'd suggest that:
relationshipName = "LJUTestObject__c.LJUTestObject_Id__c
Should be:
relationshipName = "LJUTestObject__c.Id
as Id is a standard field, the __c suffix is only used for custom fields (not standard fields on custom objects). Also, it may be that the relationship full name should end in __c not __r, but try the change above first and see how you go.
SELECT
Id,
OwnerId,
WhatId,
Reminder_Date_Time__c,
WhoId,
Record_Type_Name__c,
Task_Type__c,
Assigned_Date__c,
Task_Status__c,
ActivityDate,
Subject,
Attended_By__c,
Is_Assigned__c
FROM Task
WHERE
(NOT Task_Status__c LIKE 'Open') AND
ActivityDate >= 2017-12-13 AND
(NOT Service__r.Service_State__c LIKE 'Karnataka')
django nonrel's documentation states: "you have to manually write code for merging the results of multiple queries (JOINs, select_related(), etc.)".
Can someone point me to any snippets that manually add the related data? #nickjohnson has an excellent post showing how to do this with the straight AppEngine models, but I'm using django-nonrel.
For my particular use I'm trying to get the UserProfiles with their related User models. This should be just two simple queries, then match the data.
However, using django-nonrel, a new query gets fired off for each result in the queryset. How can I get access to the related items in a 'select_related' sort of way?
I've tried this, but it doesn't seem to work as I'd expect. Looking at the rpc stats, it still seems to be firing a query for each item displayed.
all_profiles = UserProfile.objects.all()
user_pks = set()
for profile in all_profiles:
user_pks.add(profile.user_id) # a way to access the pk without triggering the query
users = User.objects.filter(pk__in=user_pks)
for profile in all_profiles:
profile.user = get_matching_model(profile.user_id, users)
def get_matching_model(key, queryset):
"""Generator expression to get the next match for a given key"""
try:
return (model for model in queryset if model.pk == key).next()
except StopIteration:
return None
UPDATE:
Ick... I figured out what my issue was.
I was trying to improve the efficiency of the changelist_view in the django admin. It seemed that the select_related logic above was still producing additional queries for each row in the results set when a foreign key was in my 'display_list'. However, I traced it down to something different. The above logic does not produce multiple queries (but if you more closely mimic Nick Johnson's way it will look a lot prettier).
The issue is that in django.contrib.admin.views.main on line 117 inside the ChangeList method there is the following code: result_list = self.query_set._clone(). So, even though I was properly overriding the queryset in the admin and selecting the related stuff, this method was triggering a clone of the queryset which does NOT keep the attributes on the model that I had added for my 'select related', resulting in an even more inefficient page load than when I started.
Not sure what to do about it yet, but the code that selects related stuff is just fine.
I don't like answering my own question, but the answer might help others.
Here is my solution that will get related items on a queryset based entirely on Nick Johnson's solution linked above.
from collections import defaultdict
def get_with_related(queryset, *attrs):
"""
Adds related attributes to a queryset in a more efficient way
than simply triggering the new query on access at runtime.
attrs must be valid either foreign keys or one to one fields on the queryset model
"""
# Makes a list of the entity and related attribute to grab for all possibilities
fields = [(model, attr) for model in queryset for attr in attrs]
# we'll need to make one query for each related attribute because
# I don't know how to get everything at once. So, we make a list
# of the attribute to fetch and pks to fetch.
ref_keys = defaultdict(list)
for model, attr in fields:
ref_keys[attr].append(get_value_for_datastore(model, attr))
# now make the actual queries for each attribute and store the results
# in a dict of {pk: model} for easy matching later
ref_models = {}
for attr, pk_vals in ref_keys.items():
related_queryset = queryset.model._meta.get_field(attr).rel.to.objects.filter(pk__in=set(pk_vals))
ref_models[attr] = dict((x.pk, x) for x in related_queryset)
# Finally put related items on their models
for model, attr in fields:
setattr(model, attr, ref_models[attr].get(get_value_for_datastore(model, attr)))
return queryset
def get_value_for_datastore(model, attr):
"""
Django's foreign key fields all have attributes 'field_id' where
you can access the pk of the related field without grabbing the
actual value.
"""
return getattr(model, attr + '_id')
To be able to modify the queryset on the admin to make use of the select related we have to jump through a couple hoops. Here is what I've done. The only thing changed on the 'get_results' method of the 'AppEngineRelatedChangeList' is that I removed the self.query_set._clone() and just used self.query_set instead.
class UserProfileAdmin(admin.ModelAdmin):
list_display = ('username', 'user', 'paid')
select_related_fields = ['user']
def get_changelist(self, request, **kwargs):
return AppEngineRelatedChangeList
class AppEngineRelatedChangeList(ChangeList):
def get_query_set(self):
qs = super(AppEngineRelatedChangeList, self).get_query_set()
related_fields = getattr(self.model_admin, 'select_related_fields', [])
return get_with_related(qs, *related_fields)
def get_results(self, request):
paginator = self.model_admin.get_paginator(request, self.query_set, self.list_per_page)
# Get the number of objects, with admin filters applied.
result_count = paginator.count
# Get the total number of objects, with no admin filters applied.
# Perform a slight optimization: Check to see whether any filters were
# given. If not, use paginator.hits to calculate the number of objects,
# because we've already done paginator.hits and the value is cached.
if not self.query_set.query.where:
full_result_count = result_count
else:
full_result_count = self.root_query_set.count()
can_show_all = result_count self.list_per_page
# Get the list of objects to display on this page.
if (self.show_all and can_show_all) or not multi_page:
result_list = self.query_set
else:
try:
result_list = paginator.page(self.page_num+1).object_list
except InvalidPage:
raise IncorrectLookupParameters
self.result_count = result_count
self.full_result_count = full_result_count
self.result_list = result_list
self.can_show_all = can_show_all
self.multi_page = multi_page
self.paginator = paginator