Using examples from the GAE documentation I have successfully put and object to the datastore as I can view it in the admin console. Retrieving has been difficult, here is my code.
import webapp2
from google.appengine.ext import ndb
user_key = ndb.Key('Info_model', 'Bill')
class Info_model(ndb.Model):
username = ndb.StringProperty()
phone = ndb.IntegerProperty()
active = ndb.BooleanProperty()
class Create_entity(webapp2.RequestHandler):
def get(self):
user1 = Info_model(username = 'Bill',
phone = 1231231234,
active = False)
user1.put()
self.response.write('<!doctype html><html><body>Entity created.<pre>')
self.response.write('</pre></body></html>')
class MainPage(webapp2.RequestHandler):
def get(self):
self.response.out.write('<html><body>')
#get object from datastore using example from GAE Documentation.
user_key = ndb.Key('Info_model', 'Bill')
user1 = user_key.get()
self.response.write(user1.username)
self.response.write(user1.phone)
self.response.write(user1.active)
self.response.write('</body></html>')
application = webapp2.WSGIApplication([
('/', MainPage),
('/create', Create_entity)
], debug=True)
From datastore documentation:
It says:
Retrieving Entities from Keys
Given an entity's key, you can retrieve the entity from the Datastore:
sandy = sandy_key.get()
I believe this example assumes we have set a variable named sandy_key as an ndb.Key() but it doesn't really say.
I have successfully run all the ndb tutorial examples but they create a new object for each entry. I want to have only one object, call it up, edit it and put() it again. I suspect I have made the key incorrectly or called it incorrectly. I have also tried:
user1 = ndb.get('agxkZXZ-aWZnYWxlcnRyFwsSCkluZm9fbW9kZWwYgICAgICAgAkM')
Having copied the key from the admin console. This does not work, Eclipse code editor says, "Undefined variable frome import:get". I have tried a different example from the GAE documentation:
# Create an entity and write it to the Datastore.
ent = MyModel(name='booh', xyz=[10**100, 6**666])
assert ent.abc == 0
key = ent.put()
# Read an entity back from the Datastore and update it.
ent = key.get()
ent.abc += 1
ent.xyz.append(ent.abc//3)
ent.put()
But this seems to be made for all being in one scope. If I create an object in one class then try to retrieve it in another class, the variable ent in ent=key.get() is undefined.
As well I have tried many other examples in the documentation but many are incomplete and assume the reader is not a novice.
Given I have an object in the datastore, how can I retrieve that object specifically and print it out like the following:
class MainPage(webapp2.RequestHandler):
def get(self):
self.response.out.write('<html><body>')
#get object from datastore using example from GAE Documentation.
user_key = ndb.Key('Info_model', 'Bill')
user1 = user_key.get()
self.response.write(user1.username)
self.response.write(user1.phone)
self.response.write(user1.active)
self.response.write('</body></html>')
Sorry for the noob question, if there is a more appropriate forum for beginner GAE programmers please let me know.
The problem is, that you have created a key here ...
user_key = ndb.Key('Info_model', 'Bill')
... but this is never put() to the datastore.
Later (in your Create_entity get method) you are using ...
user1 = Info_model(username = 'Bill',
phone = 1231231234,
active = False)
user1.put()
... and this is correctly 'putting' the Info_model entity into the datastore.
However, this code ...
user_key = ndb.Key('Info_model', 'Bill')
user1 = user_key.get()
... attempts to get an entity of kind Info_model from the datastore with key_name "Bill", but this is not what you put into the datastore in the first place.
Maybe what you are trying to achieve, is to create an entity of kind Info_model, with key_name 'Bill', which you can then later get by keyname? If that is the case, try this code ...
user1 = Info_model.get_or_insert('Bill',
username='Bill',
phone=1231231234,
active=False)
user2_key = ndb.Key('Info_model', 'Bill')
user2 = user2_key.get()
assert user1.key == user2.key
Do be aware, however, that an entities key_name is final - you cannot change this later. You may want to consider how you would handle the scenario where Bill wanted to change his username.
As a side note, I would recommend using an ndb.StringProperty for the phone property, as often phone numbers start with a 0, or contain spaces or characters such as +
Related
All I am trying to produce is an entity that holds a unique username, and a unique device ID, and the ability to return an error if either of these conditions are not met on submission.
The only way I can see is to perform a query within a transaction, then filter the results. This however requires an ancestor (which seems unnecessary for a single simple entity).
What is the best method to go about doing this?
Here is an example that does what you want.
I put 2 entities to show you also how to make relationships
class Person(ndb.Expando):
registration_date = ndb.DateTimeProperty(auto_now_add=True)
#property
def info(self):
info = PersonInfo.query(ancestor=self.key).get()
return info
class PersonInfo(ndb.Expando):
email = ndb.StringProperty()
nick_name = ndb.StringProperty()
edit_date = ndb.DateTimeProperty(auto_now=True)
Later in the controller for register:
class RegisterPersonHandler(webapp2.RequestHandler):
def get(self):
user = users.get_current_user() #Stub here
if not user:
self.redirect(users.create_login_url(self.request.uri), abort=True)
return
person = Person.get_or_insert(user.user_id())
if not self._register(person, user):
# more logging is needed
logging.warning('Warning registration failed')
return
#ndb.transactional()
def _register(self, person, user):
''' Registration process happens here
'''
# check if the person has info and if not create it
info = PersonInfo.query(ancestor=person.key).get()
if not info:
info = PersonInfo(id=user.user_id(), parent=person.key)
info.nick_name = user.nickname()
info.email = user.email()
info.put()
return True
To answer also the comment question:
How can you programatically tell whether the returned entity is a new
or existing one though?
Try checking against a property that is default. Eg creation_date etc.
Though you can also check on something you need or on another entity's existence like I do because I expect the data to be consistent, and if not then create the bond.
I am using Endpoints-proto-datastore written by Danny Hermes for Google App Engine and need help figuring out how to update an entity.. My model for what I need to update is the following
class Topic(EndpointsModel):
#_message_fields_schema = ('id','topic_name','topic_author')
topic_name = ndb.StringProperty(required=True)
topic_date = ndb.DateTimeProperty(auto_now_add=True)
topic_author = ndb.KeyProperty(required=True)
topic_num_views = ndb.IntegerProperty(default=0)
topic_num_replies = ndb.IntegerProperty(default=0)
topic_flagged = ndb.BooleanProperty(default=False)
topic_followers = ndb.KeyProperty(repeated=True)
topic_avg_rating = ndb.FloatProperty(default=0.0)
topic_total_rating = ndb.FloatProperty(default=0.0)
topic_num_ratings = ndb.IntegerProperty(default=0)
topic_raters = ndb.KeyProperty(repeated=True)
And as you can see, the rating properties have a default of 0. So each time a topic is rated, I need to update each of the rating properties. However, none of my properties is the actual rating being provided by the user. How can i pass in the value the user rated the topic to be able to update the properties in the model? Thanks!
You can do this by having an "alias" property called rating associated with your UserModel:
from endpoints_proto_datastore.ndb import EndpointsAliasProperty
class UserModel(EndpointsModel):
...
def rating_set(self, value):
# Do some validation
self._rating = value
#EndpointsAliasProperty(setter=rating_set)
def rating(self):
return self._rating
This will allow ratings to be sent with UserModels in requests but won't require those ratings to be stored.
You're better off using the OAuth 2.0 token for the user and calling endpoints.get_current_user() to determine who the user is in the request.
Something like a dedicated model for ratings could be much easier:
from endpoints_proto_datastore.ndb import EndpointsUserProperty
class Rating(EndpointsModel):
rater = EndpointsUserProperty(raise_unauthorized=True)
rating = ndb.IntegerProperty()
topic = ndb.KeyProperty(kind=Topic)
and then transactionally retrieving the Topic from the datastore and updating it in a request method decorated by #Rating.method.
On the Google App Engine, I have a webapp2 model like this:
def Person(db.model):
first_name = db.StringProperty()
surname = db.StringProperty()
languages_spoken = db. ListProperty()
However, I'd like to have the languages_spoken list constrained to a list of options, say English, French, and Spanish, that can change as time goes on. With SQL this would be easy with person, language, and person_language tables (or similar) but I can't get my head round how to do it in this case, or even if it's possible. Any suggestions anyone?
I you are starting an appengine project, use the new NDB instead of the datastore. You can use properties with arguments like: choices and validator: https://developers.google.com/appengine/docs/python/ndb/properties
By using the NDB, you can have the following:
def Person(ndb.model):
first_name = db.StringProperty()
surname = db.StringProperty()
languages_spoken = db. ListProperty(choices=['English', 'French', 'Spanish'])
For more information about it, take a look at NDB Property Options.
However, notice that these options cannot change "on-the-fly". When a new option is supported by your application, you will need to add it in your model manually.
In the end I went with this, thanks to the pointers from the two other answers:
from google.appengine.ext import ndb
languages = ['English', 'French', 'Spanish']
def Person(ndb.model):
first_name = ndb.StringProperty()
surname = ndb.StringProperty()
languages_spoken = ndb.StringProperty(repeated=True, choices=languages)
With old datastore I've used keys. Now I have to use ids (taken from NDB Cheat Sheet doc):
user = User.get_by_id(user_id)
if user is None:
user = User(id=user_id)
But looks like this code doesn't work - the record is added several times.
(user_id is String in my case)
I found the reason, my code above is correct, but my class contained id property:
class User(ndb.Model):
id = ndb.StringProperty()
and it was the problem.
Sounds like you want a transactional operation:
user = User.get_or_insert(user_id)
See the section 'Building a Key' of the NDB Cheat Sheet.
user = ndb.Key('User', user_id).get()
if user is None:
user = User(id=user_id)
or try this and let me know if it works:
user = User.get_by_id(user_id)
if user is None:
user = User(id=user_id)
BTW even if user_id is a string you should use '%s' % user_id
Database migrations are a popular pattern, particularly with Ruby on Rails. Since migrations specify how to mold old data to fit a new schema, they can be helpful when you have production data that must be converted quickly and reliably.
But migrating models in App Engine is difficult since processing all entities sequentially is difficult, and there is no offline operation to migrate everything effectively in one big transaction.
What are your techniques for modifying a db.Model "schema" and migrating the data to fit the new schema?
Here is what I do.
I have a MigratingModel class, which all of my models inherit from. Here is migrating_model.py:
"""Models which know how to migrate themselves"""
import logging
from google.appengine.ext import db
from google.appengine.api import memcache
class MigrationError(Exception):
"""Error migrating"""
class MigratingModel(db.Model):
"""A model which knows how to migrate itself.
Subclasses must define a class-level migration_version integer attribute.
"""
current_migration_version = db.IntegerProperty(required=True, default=0)
def __init__(self, *args, **kw):
if not kw.get('_from_entity'):
# Assume newly-created entities needn't migrate.
try:
kw.setdefault('current_migration_version',
self.__class__.migration_version)
except AttributeError:
msg = ('migration_version required for %s'
% self.__class__.__name__)
logging.critical(msg)
raise MigrationError, msg
super(MigratingModel, self).__init__(*args, **kw)
#classmethod
def from_entity(cls, *args, **kw):
# From_entity() calls __init__() with _from_entity=True
obj = super(MigratingModel, cls).from_entity(*args, **kw)
return obj.migrate()
def migrate(self):
target_version = self.__class__.migration_version
if self.current_migration_version < target_version:
migrations = range(self.current_migration_version+1, target_version+1)
for self.current_migration_version in migrations:
method_name = 'migrate_%d' % self.current_migration_version
logging.debug('%s migrating to %d: %s'
% (self.__class__.__name__,
self.current_migration_version, method_name))
getattr(self, method_name)()
db.put(self)
return self
MigratingModel intercepts the conversion from the raw datastore entity to the full db.Model instance. If current_migration_version has fallen behind the class's latest migration_version, then it runs a series of migrate_N() methods which do the heavy lifting.
For example:
"""Migrating model example"""
# ...imports...
class User(MigratingModel):
migration_version = 3
name = db.StringProperty() # deprecated: use first_name and last_name
first_name = db.StringProperty()
last_name = db.StringProperty()
age = db.IntegerProperty()
invalid = db.BooleanProperty() # to search for bad users
def migrate_1(self):
"""Convert the unified name to dedicated first/last properties."""
self.first_name, self.last_name = self.name.split()
def migrate_2(self):
"""Ensure the users' names are capitalized."""
self.first_name = self.first_name.capitalize()
self.last_name = self.last_name.capitalize()
def migrate_3(self):
"""Detect invalid accounts"""
if self.age < 0 or self.age > 85:
self.invalid = True
On a busy site, the migrate() method should retry if db.put() fails, and possibly log a critical error if the migration didn't work.
I haven't gotten there yet, but at some point I would probably mix-in my migrations from a separate file.
Final thoughts
It is hard to test on App Engine. It's hard to get access to your production data in a test environment, and at this time it is difficult-to-impossible to make a coherent snapshot backup. Therefore, for major changes, consider making a new version that uses a completely different model name which imports from the old model and migrates as it needs. (For example, User2 instead of User). That way, if you need to fall back to the previous version, you have an effective backup of the data.