I'm playing with Google App Engine using the Go APIs and despite everything I've tried I can't get the Queries to return data that is in the datastore. I can see that Put() works as expected, and the entities are listable/accessible from inspection of the Admin Console available from dev_appserver.py
The struct I'm storing is defined this way:
type TweetData struct {
Id int64 `datastore:",noindex" json:"id"`
Text string `datastore:",noindex" json:"text"`
Screen_name string `json:"screen_name"`
}
And my calls to query it are such:
func getDatastoreTweets(c appengine.Context, twitterUser string) []*TweetData {
q := datastore.NewQuery("TweetData").Filter("Screen_name =", twitterUser).Order("-Id").Limit(10)
var oldTweets []*TweetData
if _, err := q.GetAll(c, &oldTweets); err != nil {
fmt.Printf("Getall had non-nil error! %v\n", err)
}
return oldTweets
}
For this query, err is never non-nil, but always returns 0 results, even when the Console tells me there are many. I've tried it without the Filter call as well, after the guestbook example the SDK provides, to no avail.
In case it's an issue with keys (unlikely if I understand correctly, because I'm querying on a property), the call to Put is as follows:
// tweetData passed in via parameter...
key := datastore.NewIncompleteKey(c, "TweetData", nil)
_, err := datastore.Put(c, key, &tweetData)
Any help would be appreciated, thanks! ^_^
The query ask for ordering by Id desc, while the index field is un-indexed, you should either:
rewrite the TweetData annotation to index Id field:
Id int64 `json:"id"`
Remove the Order clause of your query:
q := datastore.NewQuery("TweetData").Filter("Screen_name =", twitterUser).Limit(10)
Related
A continuation from this question:
Doing a "IN Array" query on google app engine datastore with golang
Right now, I am following the suggestion from the previous question on querying with an array of keys/ids ids []int64. These IDs may or may not actually exist (they have been deleted, but the reference on other instances have not been removed).
My method of trying to obtain these instances looks like so:
var keys []*datastore.Key
for _, id := range ids {
keys = append(keys, datastore.NewKey(c, "Category", "", id, nil))
}
categories := make([]Category, len(keys))
err := datastore.GetMulti(c, keys, categories)
if err != nil {
return nil, err
}
for i := 0; i < len(categories); i++ {
categories[i].Id = keys[i].IntID()
}
However, it errors out throwing me:
datastore: no such entity
I could on the other hand grab each one individually, but is there a more efficient way to approach this?
You need to type assert the error to an appengine.MultiError. This way you can get access to the errors for an individual entity.
if me, ok := err.(appengine.MultiError); ok {
for i, e := range me {
// e != nil if entity i failed
}
} else {
// something else went wrong (timeout, etc).
}
See the docs for MultiError here
Is there a way to do a query with ids []int64 on datastore? I've tried the following with no avail.
Errors out
q := datastore.NewQuery("Category").Filter("Id IN", ids)
Just gets me all the the categories in the datastore
for _, id := range ids {
q.Filter("Id =", id)
}
After icza's answer
var keys []*datastore.Key
for _, id := range ids {
keys = append(keys, datastore.NewKey(c, "Category", "", id, nil))
}
categories := make([]Category, len(keys))
err := datastore.GetMulti(c, keys, categories)
if err != nil {
return nil, err
}
Generally "IN" filters are not supported by the Datastore. The documentation of Query.Filter() lists the allowed operators:
">", "<", ">=", "<=", or "="
What you can do is execute a separate query for each of the elements in the array you want to filter by. Also if the elements are in a continous range, you can substitute the IN with id>=min and id<=max. E.g.:
ids := []int64{1,2,3,4}
q := datastore.NewQuery("Category").Filter("Id>=", 1).Filter("Id<=", 4)
Also note that while the IN is not supported in general, if the property is the entity key itself, you can get a list of entities specified by an array of their keys using the datastore.GetMulti() function:
func GetMulti(c appengine.Context, key []*Key, dst interface{}) error
Note:
Your 2nd attempt returns all entities because you call Filter() on your query, but you don't store the return value, so the query you end up executing will have no filters at all. Query.Filter() returns a derivative query which contains the filter you just specified, you have to use the returned Query ongoing. So it should be:
q = q.Filter("Id=", id)
But even this won't work either: if multiple filters are specified, they will be in logical AND connection so it will most likely give you 0 results as I suspect no category will exists where Id is a list and which would contain all the ids you want to filter by.
I am using gorilla mux for my routes and I pass an id.
Using that Id how can I get a entity from datastore.
param := mux.Vars(r)
c := appengine.NewContext(r)
item := []Item{}
pr, err := datastore.NewQuery("Item").Filter("ID = ", param["id"]).GetAll(c, &item)
And here I'm stuck, I tried using filter but it doesn't work.
What do I have to do next?
If you have an id, you don't need to use a query. A faster and cheaper way is to create a key and retrieve this entity directly from the Datastore.
https://cloud.google.com/appengine/docs/go/datastore/entities#Go_Retrieving_an_entity
You don't say if you are using integer or string ids for entities. I'll assume integer because it requires a little more code. First, create a key:
n, err := strconv.ParseInt(param["id"], 10, 64)
if err != nil {
// handle error
}
key := datastore.NewKey(c, "Item", "", n, nil)
Now that you have key, you can fetch the entity:
var item Item
if err := datastore.Get(c, key, &item); err != nil {
// handle error
}
It's more efficient to get the entity than to query for the entity. If you do want to query by id, then see key filters in the App Engine docs.
In Python it's
q = db.Query()
q.ancestor(ancestor_key)
I tried:
q := datastore.NewQuery("")
q.Ancestor(ancestor_key)
I get the error "datastore: empty kind" when running GetAll
I also tried:
q := &datastore.Query{}
q.Ancestor(ancestor_key)
I get the error "datastore: empty query kind"
Thanks in advance for any help with this matter.
func NewQuery
func NewQuery(kind string) *Query
NewQuery creates a new Query for a specific entity kind. The kind must
be non-empty.
In your code,
q := datastore.NewQuery("")
the kind is empty.
Rich Churcher's comment seems to be right, at least at this point in time.
I don't think the Python kindless ancestor query is supported in Go.
For a moment there I thought you could use the ancestor key's Kind()
method, then I had some more coffee and came to my senses.
GetAll doesn't seem to work, but you can do kindless queries.
ctx := appengine.NewContext(r)
q := datastore.NewQuery("")
for it := q.Run(ctx); ; {
key, err := t.Next(nil)
if err == datastore.Done {
break
}
if err != nil {
break
}
fmt.Printf("%v\n", key)
}
Here is an Example of the app. The essential code is in: golang-code/handler/handler.go (After the subject should appear an ID!)
Im trying to build a little blog system in Golang on Google Appengine and use Mustache as template engine.
So, i have a struct:
type Blogposts struct {
PostTitle string
PostPreview string
Content string
Creator string
Date time.Time
}
The data is passed to GAE via
datastore.Put(c, datastore.NewIncompleteKey(c, "Blogposts", nil), &blogposts)
So, GAE assigns automatically a intID (int64).
Now I tried to get the latest blogposts
// Get the latest blogposts
c := appengine.NewContext(r)
q := datastore.NewQuery("Blogposts").Order("-Date").Limit(10)
var blogposts []Blogposts
_, err := q.GetAll(c, &blogposts)
Until there all things works fine, but when I try to access intID (or stringID, whatever) I dont have access to this :-(
<h3>{{{PostTitle}}}</h3>
(PostTitle works, intID not, i've tried thousand of things, nothing worked :-( )
Anyone an idea? This would be great!
Edit:
I use mustache.
http://mustache.github.com/
In the code I use:
x["Blogposts"] = blogposts
data := mustache.RenderFile("templates/about.mustache", x)
sendData(w, data) // Equivalent to fmt.Fprintf
And then the data can be accessed in the .mustache template with {{{Content}}} or {{{PostTitle}}} etc.
As hyperslug pointed out, the id field of an entity is on the key, not the struct it gets read into.
Another way around this is to add an id field to your struct and tell datastore to ignore it, eg:
type Blogposts struct {
PostTitle string
PostPreview string
Content string
Creator string
Date time.Time
Id int64 `datastore:"-"`
}
You can then populate the Id field manually after a call to GetAll() like so
keys, err := q.GetAll(c, &blogposts)
if err != nil {
// handle the error
return
}
for i, key := range keys {
blogposts[i].Id = key.IntID()
}
This has the benefit of not introducing an extra type.
intID is an internal property of a Key not the struct, and is accessible through a getter:
id := key.IntID()
GetAll returns []*Key, which you're not using:
_, err := q.GetAll(c, &blogposts)
One way to get around this is to create a viewmodel struct that has both your post and key info (untested, but this is the gist of it):
//... handler code ...
keys, err := q.GetAll(c, &blogposts)
if err != nil {
http.Error(w, "Problem fetching posts.", http.StatusInternalServerError)
return
}
models := make([]BlogPostVM, len(blogposts))
for i := 0; i < len(blogposts); i++ {
models[i].Id = keys[i].IntID()
models[i].Title = blogposts[i].Title
models[i].Content = blogposts[i].Content
}
//... render with mustache ...
}
type BlogPostVM struct {
Id int
Title string
Content string
}
I know this question is a couple years old, but the following article was very helpful to me in this regard: Golang basics with Google Datastore.
In the article, the author provides a nice example of how you can run a query that gets an entity by its ID...
func GetCategory(c appengine.Context, id int64) (*Category, error) {
var category Category
category.Id = id
k := category.key(c)
err := datastore.Get(c, k, &category)
if err != nil {
return nil, err
}
category.Id = k.IntID()
return &category, nil
}
...as well as getting a list/collection of entities with their associated ID:
func GetCategories(c appengine.Context) ([]Category, error) {
q := datastore.NewQuery("Category").Order("Name")
var categories []Category
keys, err := q.GetAll(c, &categories)
if err != nil {
return nil, err
}
// you'll see this a lot because instances
// do not have this by default
for i := 0; i < len(categories); i++ {
categories[i].Id = keys[i].IntID()
}
return categories, nil
}
The snippet above is very close to the helpful answer by #koz.
AFAICS, the Blogposts struct has no field intID, but it has a field PostTitle. I guess that could be the reason why the former doesn't and the later does get rendered, though I've never used Mustache...