I'm trying to use Python and NDB to access the datastore, which contains one entity:
I've defined by NDB model with the following code:
class Test(ndb.Model):
name = ndb.StringProperty()
val = ndb.IntegerProperty()
Then I run a query for the entity:
query = Test.get_by_id("Testing")
This returns a NoneType with no val field. I tried setting the argument to name=Testing instead of Testing, but that doesn't help.
What can I do to access my entity in Python? Do I need to identify the project's ID somewhere?
Also, I've been using Flask to serve as the microframework. But all the NDB example code I've seen uses webapp2. Should I use webapp2 instead?
Capitalization matters in Python. Instead of "Test", you need to query from the model "test".
Related
I am trying to make a python program that uses the google cloud data store in python and i am having some trouble with the DataStore input\output system. this is my first time working with google cloud, and i am also somewhat new to python.
I am trying to build a very simple database, with only 1 type of entity model- 'Command', which has 2 variables- 'name' which i want to be the key, and 'value'. all the entities will have one parent, because the google cloud guide said this will put all the entities in the same entity group, which helps sort them? (i am not sure about this, so explanation will be nice)
class Command(ndb.Model):
value = ndb.IntegerProperty()
# no 'name' variable, since it's the key.
def parent_key():
return ndb.Key(Command, DEFAULT_PARENT_NAME)
when the user uses a 'set' command, the code will either insert a new entity with the given name and value, or if the name exists already, it will change the existing value to the given value.
(assume 'variable_name' is the name and 'variable_value' is the value)
this is the code for the 'set' command:
variable_name = self.request.get('name')
variable_value = self.request.get('value')
newcommand = Command(id=variable_name, value=int(variable_value), parent=parent_key()) # create a new command model
newcommand.put()
this inserts a new command, but doesn't check if it is already in the datastore.
i want the 'get' command to extract the value of an existing name in the database (or return an error, if it doesn't exist), given the name (in a string)
in the online manual i found how to extract stuff from the database given a key, but here i dont have a key, i have a string.
I don't know how to complete the 'set' and 'get' commands and would appreciate some help with this.
Thanks in advance!
I am using the google cloud datastore python client to write an entity into the datastore which contains an embedded entity. An example entity might look like:
data_type: 1
raw_bytes: <unindexed blob>
values: <indexed embedded entity>
I checked the data from the console and the data is getting saved correctly and the values are present.
Next, I need to run a query from a python app engine application. I have represented the above as the following entity in my app engine code:
class DataValues(ndb.Model):
param1 = ndb.BooleanProperty()
param2 = ndb.IntegerProperty()
param3 = ndb.IntegerProperty()
class MyEntity(ndb.Expando):
data_type = ndb.IntegerProperty(required=True)
raw_bytes = ndb.BlobProperty()
values = ndb.StructuredProperty(DataValues)
One of the filters in the query depends on a property in values. Sample query code is as below:
MyEntity.query().filter(MyEntity.data_type == 1).filter(MyEntity.values.param1 == True).get()
I have created the corresponding composite index in my index.yaml
The query runs successfully but the resulting entity contains the embedded entity values as None. All other property values are present.
What can be the issue here ?
Add properties of DataValues entity as properties of the MyEntity.
This is a bit of a guess, but since datastore attributes are kind of keyed by both their name (in this case values) and the name of the "field type/class" (i.e. StructuredProperty), this might fix your problem:
class EmbeddedProperty(ndb.StructuredProperty):
pass
class MyEntity(ndb.Expando):
data_type = ndb.IntegerProperty(required=True)
raw_bytes = ndb.BlobProperty()
values = EmbeddedProperty(DataValues)
Give it a shot and let me know if values starts coming back non-null.
I struggled with the same problem, wanting to convert the embedded entity into a Python dictionary. One possible solution, although not a very elegant one, is to use a GenericProperty:
class MyEntity(ndb.Model):
data_type = ndb.IntegerProperty(required=True)
raw_bytes = ndb.BlobProperty()
values = ndb.GenericProperty()
values will then be read as an "Expando" object: Expando(param1=False,...). You can access the individual values with values.param1, values.param2 etc. I would prefer having a custom model class, but this should do the job.
The index.yaml file of my GAE app is no longer updated by the development server.
I have recently added a new kind to my app and a handler that queries this kind like so:
from google.appengine.ext import ndb
class MyKind(ndb.Model):
thing = ndb.TextProperty()
timestamp = ndb.DateTimeProperty(auto_now_add=True)
and in the handler I have a query
query = MyKind.query()
query.order(-MyKind.timestamp)
logging.info(query.iter().index_list())
entities = query.fetch(100)
for entity in entities:
# do something
AFAIK, the development server should create an index for this query and update index.yaml accordingly. However, it doesn't. It just looks like this:
indexes:
# AUTOGENERATED
The logging.info(query.iter().index_list()) should output the index used for the query, it just says 'None'. Also, the SDK console says 'Datastore contains no indexes.'
Running the query returns the entities unsorted. I have two questions:
is there some syntax error in my code causes the query results be unsorted or is it the missing index?
if it's the missing index, is there a way to manually force the dev server to update index.yaml? Other suggestions?
Thank you
your call to order returns the new query..
query = MyKind.query()
query = query.order(-MyKind.timestamp)
..to clarify..
query.order(-MyKind.timestamp) does not change the query, it returns a new one, so you need to use the query returned by that method. As it is query.order(-MyKind.timestamp) in your code does nothing.
I've created two different Entities, one a User and one a Message they can create. I assign each user an ID and then want to assign this ID to each message which that user creates. How can I go about this? Do I have to do it in a query?
Thanks
Assuming that you are using Python NDB, you can having something like the following:
class User(ndb.Model):
# put your fileds here
class Message(ndb.Model):
owner = ndb.KeyProperty()
# other fields
Create and save a User:
user = User(field1=value1, ....)
user.put()
Create and save a Message:
message = Message(owner=user.key, ...)
message.put()
Query a message based on user:
messages = Message.query().filter(Message.owner==user.key).fetch() # returns a list of messages that have this owner
For more information about NDB, take a look at Python NDB API.
Also, you should take a look at Python Datastore in order to get a better understanding of data modeling in App Engine.
class MyEntity(db.Model):
timestamp = db.DateTimeProperty()
title = db.StringProperty()
number = db.FloatProperty()
db.GqlQuery("SELECT * FROM MyEntity WHERE title = 'mystring' AND timestamp >= date('2012-01-01') AND timestamp <= date('2012-12-31') ORDER BY timestamp DESC").fetch(1000)
This should fetch ~600 entities on app engine. On my dev server it behaves as expected, builds the index.yaml, I upload it, test on server but on app engine it does not return anything.
Index:
- kind: MyEntity
properties:
- name: title
- name: timestamp
direction: desc
I try splitting the query down on datastore viewer to see where the issue is and the timestamp constraints work as expected. The query returns nothing on WHERE title = 'mystring' when it should be returning a bunch of entities.
I vaguely remember fussy filtering where you had to call .filter("prop =",propValue) with the space between property and operator, but this is a GqlQuery so it's not that (and I tried that format with the GQL too).
Anyone know what my issue is?
One thing I can think of:
I added the list of MyEntity entities into the app via BulkLoader.py prior to the new index being created on my devserver & uploaded. Would that make a difference?
The last line you wrote is probably the problem.
Your entities in the actual real datastore are missing the index required for the query.
As far as I know, when you add a new index, App Engine is supposed to rebuild your indexes for you. This may take some time. You can check your admin page to check the state of your indexes and see if it's still building.
Turns out there's a slight bug in the bulkloader supplied with App Engine SDK - basically autogenerated config transforms strings as db.Text, which is no good if you want these fields indexed. The correct import_transform directive should be:
transform.none_if_empty(str)
This will instruct App Engine to index the uploaded field as a db.StringProperty().