In my application I have a model like so:
class MyModel(ndb.Model):
entity_key_list = ndb.KeyProperty('k', repeated=True, indexed=False)
entity_key_num = ndb.ComputedProperty('n', lambda self: len(self.entity_key_list))
verified = ndb.BooleanProperty('v')
Is it possible to have the entity_key_num property when verified is false?
You can return None if not verified like this:
entity_key_num = ndb.ComputedProperty('n', lambda self: len(self.entity_key_list) if not self.verified else None)
If you don't want to have the value None at all and dynamically delete or create this property then you will have to use the ndb.Expando class where you can do all these fancy stuff. Note that you won't be able to delete the ComputedProperty so you will have to keep track of that value on your own.
Related
I currently have a model in NDB and I'd like to add a new property to it. Let's say I have the following:
class User(Model, BaseModel):
name = ndb.StringProperty(required=False)
email = ndb.StringProperty(required=False)
#property
def user_roles(self):
return UserRole.query(ancestor=self.key).fetch()
#property
def roles(self):
return [user_role.role for user_role in UserRole.query(ancestor=self.key).fetch()]
Now, let's say, I've added one additional property called market_id. For example,
class User(Model, BaseModel):
name = ndb.StringProperty(required=False)
email = ndb.StringProperty(required=False)
#property
def user_roles(self):
return UserRole.query(ancestor=self.key).fetch()
#property
def roles(self):
return [user_role.role for user_role in UserRole.query(ancestor=self.key).fetch()]
#property
def market_id(self):
""" fetches `id` for the resource `market` associated with `user` """
for each_role in UserRole.query(ancestor=self.key):
resource = each_role.role.get().resource
if resource.kind() == 'Market':
return resource.id()
return None
The problem here is, roles are fetched properly as expected for all the existing entities (since that property had been there since the beginning and also, an extra column can be observed in datastore called roles).
Since, I'm dealing with Python class property, I assume that migration is not required. But, how does column called roles already exist? And why newly added property called market_id does not? Does it require migration?
The change you're suggesting is not an actual ndb model change as you're not adding/deleting/modifying any of the model's datastore properties. Only ndb.Property class children are real ndb model properties that are stored when the entity is put() into the datastore.
The property you're adding is a Python class #property - nothing to do with what's in the datastore.
So for this particular case no migration is needed.
The update to the question makes this even more clear, I believe. The market_id #property is not a User datastore entity property. To get values for it you don't need to update the User entity, but you have to create/edit corresponding UserRole entities with their resource property point to a Market entity.
I'm trying to add property options to my model. I have a StringProperty and I added required=True but I'm still able to create an object with the required field being empty.
I tried it in the admin and also in my update form for the specific model so not sure what I'm doing wrong?
You can create it, but can you put it?
class x(ndb.Model):
author = ndb.StringProperty(required=True)
a = x()
a.put()
Fails with: BadValueError: Entity has uninitialized properties: **author**
Setting a value on the required property author allows you to save it:
a.author = "some_value"
The put can now succeed.
key = a.put()
and key is now:
Key('x', 5707702298738688)
or even key.urlsafe()
ahNzfnNoYXJlZC1wbGF5Z3JvdW5kcg4LEgF4GICAgICArpkKDKIBEDYwNTM4Njc2MzY2NTQwODA
Read more about storing data here.
In my Google App Engine app I have model objects that need to be stored. These objects are parameterized by various policy objects. For example, my Event class has a Privacy policy object which determines who can see, update, etc. There are various subclasses of PrivacyPolicy that behave differently. The Event consults its PrivacyPolicy object at various points.
class PrivacyPolicy(db.Model):
def can_see(self, event, user):
pass
class OwnerOnlyPolicy(PrivacyPolicy):
def can_see(self, event, user):
return user == event.owner
class GroupOnlyPolicy(PrivacyPolicy):
def can_see(self, event, user):
for grp in event.owner.groups()
if grp.is_member(user):
return True
return False
class OnlyCertainUsersPolicy(PrivacyPolicy):
def __init__(self, others):
self.others = others
def can_see(self, event, user):
return user in others
I could make my Event class use a ReferenceProperty to the PrivacyPolicy:
class Event(db.Model):
privacy: db.ReferenceProperty(PrivacyPolicy)
#…
The reason I don’t like this is that the one-to-one relationship means that nobody every queries for the policy object, there is no need to maintain the back-reference from the policy to its Event object, and in no other way is PrivacyPolicy an independent db-level object. It is functionally equivalent to an IntegerProperty, in that it is part of the Event object’s state, it’s just an object instead of a number — specifically it’s an object that can have zero state or lots of state, unknown to the Event type.
I can’t find anyone talking about how to approach such a situation. Is there a tool/approach I don’t know about? Do I just suck it up and use a reference property and the hell with the overhead?
If the only other way to handle this is a custom Property type, any advice about how to approach it would be welcome. My first thought is to use a TextProperty to store the string rep of the policy object (policy), decode it when needed, caching the result, and having any change to the policy object invalidate the cache and update the string rep.
You're overcomplicating by trying to store this in the datastore. This belongs in code rather than in the datastore.
The least complicated way would be:
class Event(db.Model):
privacy = db.IntegerProperty()
def can_see(self, user):
if self.privacy == PRIVACY_OWNER_ONLY:
return user == event.owner
else if self.privacy == PRIVACY_GROUP:
for grp in self.owner.groups()
if grp.is_member(user):
return True
return False
Sometimes all it takes is to think of the right approach. The solution is to introduce a new kind of property that uses pickle to store and retrieve values, such as that described in https://groups.google.com/forum/?fromgroups#!topic/google-appengine/bwMD0ZfRnJg
I wanted something slightly more sophisticated, because pickle isn’t always the answer, and anyway documentation is nice, so here is my ObjectReference type:
import pickle
from google.appengine.ext import db
class ObjectProperty(db.Property):
def __init__(self, object_type=None, verbose_name=None, to_store=pickle.dumps, from_store=pickle.loads, **kwds):
"""Initializes this Property with all the given options
All args are passed to the superclass. The ones used specifically by this class are described here. For
all other args, see base class method documentation for details.
Args:
object_type: If not None, all values assigned to the property must be either instances of this type or None
to_store: A function to use to convert a property value to a storable str representation. The default is
to use pickle.dumps()
from_store: A function to use to convert a storable str representation to a property value. The default is
to use pickle.loads()
"""
if object_type and not isinstance(object_type, type):
raise TypeError('object_type should be a type object')
kwds['indexed'] = False # It never makes sense to index pickled data
super(ObjectProperty, self).__init__(verbose_name, **kwds)
self.to_store = to_store
self.from_store = from_store
self.object_type = object_type
def get_value_for_datastore(self, model_instance):
"""Get value from property to send to datastore.
We retrieve the value of the attribute and return the result of invoking the to_store function on it
See base class method documentation for details.
"""
value = getattr(model_instance, self.name, None)
return self.to_store(value)
def make_value_from_datastore(self, rep):
"""Get value from datastore to assign to the property.
We take the value passed, convert it to str() and return the result of invoking the from_store function
on it. The Property class assigns this returned value to the property.
See base class method documentation for details.
"""
# It passes us a unicode, even though I returned a str, so this is required
rep = str(rep)
return self.from_store(rep)
def validate(self, value):
"""Validate reference.
Returns:
A valid value.
Raises:
BadValueError for the following reasons:
- Object not of correct type.
"""
value = super(ObjectProperty, self).validate(value)
if value is not None and not isinstance(value, self.object_type):
raise db.KindError('Property %s must be of type %s' % (self.name, self.object_type))
return value
Recently there's been some data structure changes in our app, and we decided to use namespaces to separate different versions of of the data, and a mapreduce task that converts old entities to the new format.
Now that's all fine, but we don't want to always isolate the entire data set we have. The biggest part of our data is stored in a kind that's pretty simple and doesn't need to change often. So we decided to use per-kind namespaces.
Something like:
class Author(ndb.model.Model):
ns = '2'
class Book(ndb.model.Model):
ns = '1'
So, when migrating to version 2, we don't need to convert all our data (and copy all 'Book' kinds to the other namespace), only entities of the 'Author' kind. Then, instead of defining the appengine_config.namespace_manager_default_namespace_for_request, we just the 'namespace' keyword arguments to our queries:
Author.query(namespace=Author.ns).get()
Question: how to store (i.e. put()) the different kinds using these different namespaces? Something like:
# Not an API
Author().put(namespace=Author.ns)
Of course, the above doesn't work. (Yes, I could ask the datastore for an avaliable key in that namespace, and then use that key to store the instance with, but it's an extra API call that I'd like to avoid.)
To solve a problem like this I wrote a decorator as follows:
MY_NS = 'abc'
def in_my_namespace(fn):
"""Decorator: Run the given function in the MY_NS namespace"""
from google.appengine.api import namespace_manager
#functools.wraps(fn)
def wrapper(*args, **kwargs):
orig_ns = namespace_manager.get_namespace()
namespace_manager.set_namespace(MY_NS)
try:
res = fn(*args, **kwargs)
finally: # always drop out of the NS on the way up.
namespace_manager.set_namespace(orig_ns)
return res
return wrapper
So I can simply write, for functions that ought to occur in a separate namespace:
#in_my_namespace
def foo():
Author().put() # put into `my` namespace
Of course, applying this to a system to get the results you desire is a bit beyond the scope of this, but I thought it might be helpful.
EDIT: Using a with context
Here's how to accomplish the above using a with context:
class namespace_of(object):
def __init__(self, namespace):
self.ns = namespace
def __enter__(self):
self.orig_ns = namespace_manager.get_namespace()
namespace_manager.set_namespace(self.ns)
def __exit__(self, type, value, traceback):
namespace_manager.set_namespace(self.orig_ns)
Then elsewhere:
with namespace_of("Hello World"):
Author().put() # put into the `Hello World` namespace
A Model instance will use the namespace you set with the namespace_manager[1] as you can see here: python/google/appengine/ext/db/init.py
What you could do is create a child class of Model which expects a class-level 'ns' attribute to be defined. This sub class then overrides put() and sets the namespace before calling original put and resets the namespace afterwards. Something like this:
'''
class MyModel(db.Model):
ns = None
def put(*args, **kwargs):
if self.ns == None:
raise ValueError('"ns" is not defined for this class.')
original_namespace = namespace_manager.get_namespace()
try:
super(MyModelClass, self).put(*args, **kwargs)
finally:
namespace_manager.set_namespace(original_namespace)
'''
[1] http://code.google.com/appengine/docs/python/multitenancy/multitenancy.html
I don't think that it is possible to avoid the extra API call. Namespaces are encoded into the entity's Key, so in order to change the namespace within which a entity is stored, you need to create a new entity (that has a Key with the new namespace) and copy the old entity's data into it.
I have my own User model in app engine, which should have a property of his gravatar url. However, since this can be very quickly computed using his email address, it doesn't make sense to store it. Is there a way to just automatically initialize this property when it s loaded from the datastore?
I could just add a method called get_avatar_url(), but you can't call an object's methods (as far as I know), from within a jinja2 template, and I don't want to post all these values individually to the template.
You can define a method, as you describe, or you can define a property, like this:
class MyModel(db.Model):
email = db.StringProperty(required=True)
#property
def avatar_url(self):
return "http://gravatar.com/avatar/%s" % (hashlib.md5(self.email).hexdigest(),)
You can then refer to this as instance.avatar_url (or in a template, {{instance.avatar_url}}).
Either will work fine in a jinja2 template, but using a property is slightly neater if you need to request it elsewhere. Since only datastore property instances result in storing data in the datastore, your property will not be stored in the datastore.
It's ok to call them from a template. All you need to do is to declare this model's method as classmethod or property
Here's a quick example:
# sample model
class UserProfile(db.Model):
...
email = db.EmailProperty()
...
#property
def id(self):
return self.key().id()
#classmethod
def get_avatar_url(self):
# whatever you need to call gravatar url
return self.email
# sample view
def show_user(user_id):
user = User.all().filter("user = ", user_id).get()
flowers = Flower.all().filter("user = ", user)
return render_template('index.html', u=user, f=flowers)
# sample template
<div class="user">user id: {{ u.id }}, and gravatar: {{ u.get_gravatar_url() }}<div>
HTH.
You most certainly can call methods within a template. That is the best way to do it.