I have a google app engine app where I would like to extend one of my Entity definitions. How would I ensure existent entity objects get the new fields properly initialized? Would the existent objects, the next time I query them, simply have default values? I'd like to add a StringListProperty.
If you add a new property to your model, existing entities will have the default value for it when you load them, if you supplied a default. They won't show up in queries for that value until you fetch them and store them again, though.
You will have to add the property to all of your existing entities, one by one.
You don't mention which language or API you are using. The exact details of the procedure will vary with your situation.
In general, the safest way to do this is to load up each entity with a API that doesn't validate your entities. In python, you can use Expando models. In java, you can use the low level datastore API. (trying this with JDO or JPA may not work) You now need to iterate through all existing entities. (try the new Mapper API to do this with relatively little fuss). For each entity, you will load it, add your new property, then put/save it back to the datastore. Now you can safely go back to a framework that validates your entities, like JDO or non-expando models.
This method applies to modifying the type of a property or deleting a property as well.
Related
Our app logic has multiple layers. Each time a save is invoked, the entity on the domain layer is mapped to a database entity.
For example:
class Sample(); // the domain entity
#Entity("Sample")
#Cache
class DatabaseSample; // the database entity
Let's assume the domain entity is modified and save is invoked, which will map all properties to a new database entity, which is then saved deferred.
Let's assume the same domain entity is modified again and saved again, which will again map all properties to a new database entity and invoke deferred save.
Will the two separate Google Cloud Datastore entities compete with each other, e.g. the newer save overwrite the older save completely, or will objectify collect modified Key-Value pairs during the request and save a consolidated entity at the end of the request?
I don't quite understand the question, but I'll try to give you some context that might help.
If you defer save of an entity POJO, it just marks it as "save this thing" in the session. When the session closes (end of transaction) anything marked for deferred save (or delete) gets saved.
Only at the very moment of saving to the datastore does the POJO get mapped to a low-level Entity. You can defer save the same object a zillion times with no practical effect. In fact, this is the core use case - you might have a method that sets changes some data and wants a save, and some other method that changes some data and wants a save... by using deferred saves, you avoid making separate real saves to the datastore.
If you're asking about doing something really crazy like changing the #Id after deferring a save... don't. I really have no idea what that will do, but you probably won't like it :-)
Let's say we have a User entity. Should I have two smaller services (User and Users)? Or one larger service that deals with both a collection of Users and an individual User? If it is the latter, is it best practice to name the service User or Users?
I use one service per entity that houses the collection, methods used by the entity collection controller, and methods used by the individual entity's controller. This works for my team as we follow the repository pattern in our server code. I save the collection in the service because it is accessed often, and parts of the collection are nice to share in other area's like to keep a count in the menu, or create a relational list in another controller. The individual entity is typically only accessed by the controller for the view, and can be disposed of when the route is changed as long as the list item in the controller was updated if the entity was changed.
The only time I used separate services was one edge case where the customer wanted an entity to save state if the route was changed without persisting the entity to the server or cache. The entity needed to be saved somewhere so that was reason enough to create a service for the individual entity.
I do use a separate service per entity to manage http requests. So each entity does have two services, one to manage the collection and all crud+ functionality, and the other for http for separation of concerns.
I understand that the Google AppEngine Data Store changed their default policy on how ID's are auto-generated.
We have application code that expects all ID's to be less than the maximum value for an Integer. In trying to create sample data using the dashboard ("Datastore Viewer"), there is a way to create Entities manually. However when I do this, there appears to be no place to manually set the ID, and the auto-generated ID is larger than the maximum Integer value.
Setting <auto-id-policy>legacy</auto-id-policy> in appengine-web.xml and re-deploying did not seem to help.
I understand when you create Entities programmatically, you can specify your own ID number. Is there any way to do this from the Dashboard, or at least use "legacy" auto-id generation?
No, the Datastore Viewer only allows auto-generated IDs. :(
In my app, I'd like to have an entity defined like this:
class MyModel(db.Model):
title = db.StringProperty()
myBlob = db.BlobProperty()
Say this blob holds around 1 megabyte. Will this slow down any queries I make on the MyModel type? Is it fetching the entire 1mb per entity, or just references until I actually try to access the blob?
The minute you retrieve the entity, the blob is loaded from the datastore unless you do a projection query.
You have a few options to avoid loading the BlobProperty until you need it.
do a projection query and then only fetch the full entity when you need it.
stick the BlobProperty in a Child entity (make the top level one the ancestor) and only fetch the property with a get, when you need it.
Don't use a BlobProperty but stick it in GCS (Google CLoud Storage) and serve it from there.
The last has the benefit that if you do no processing on the blob your appengine instance doesn't need to get involved with serving it (depending on what your requirements are of course)
I am currently exporting data from App Engine (to csv files) so that a few graduate students can analyze and build models based on the data. The data that I am exporting consists of non-confidential properties and the entity keys for a few models that DO contain confidential data (email addresses, names, etc.). My goal is to produce a dataset that can be shared and analyzed publicly without exposing any private data in the other entities that I’m sharing the keys for.
I started off planning to encrpyt all the entity keys to annonymize the data, but then I realized that the unencrypted keys might not be of any value to anyone as long as my app never exposes any data based on entity keys. My app only exposes personally indentifyable data based on the id for each entity.
Is there any reason to be concerned about sharing App Engine keys for entities? Is there anything that anonymous users can do with entity keys if there aren’t any urls that return any information based on the keys?
The entitiy keys contain the name of your app, the name of the entity (and any parents) and the numeric id of your entity.
If your app does not expose the data of the referenced entity you are okay.
Here`s the link in the google docs:
http://code.google.com/intl/de-DE/appengine/docs/python/datastore/entities.html
And more concrete docs on db.Key(): http://code.google.com/intl/de-DE/appengine/docs/python/datastore/keyclass.html
"Note: A string-encoded key can be converted back to the raw key data. This makes it easy to guess other keys when one is known. While string-encoded key values are safe to include in URLs, an application should only do so if key guessability is not an issue."