I'm just starting to learn about NoSQL/Document storage this morning. I am used to EntityFramework/SQLServer.
My questions is the following: If I have a bunch of "documents" stored and somewhere down the line I add a property to my class that is needed by my app, how do I back-populate the already existing records?
If you change the model after the fact then you have a few options.
If you have a default value for the additional field and can wait until the next time that entity is saved for the database then you can simply add the new property and set the value to the defaultv value in the constructor.
You can use a IDocumentConversionListener (http://ayende.com/blog/66563/ravendb-migrations-rolling-updates)
You can also use https://github.com/khalidabuhakmeh/RavenMigrations which I have never used but it seems like it would do what you want.
Related
I have been trying to add data to sections in my firestorm database, I have a collection -> document -> data fields. Whenever I use setData({'key': 'value'}) is always overwrites the data already in the document. Is there anyway around this?
That is because what setData(object) is used for
To create or overwrite a single document
While update(object)
To update some fields of a document without overwriting the entire document
So what you need is to use update()
Using merge: true in your setData() statement prevents overwrite.
This seems similar to the update() method but you can use it even when document does not exist (ie is being created).
I’m trying to update my Drupal application with an external script writing directly in the MySQL DB, I only need to modify one field of one specific datatype. I see it in a table named field_data_field_FIELDNAME but when I update this the application doesn’t change. I need to modify something else? Thanks
Assuming you are trying to change the field's value, try modifying both field_data_field_FIELDNAME and field_revision_field_FIELDNAME, then clear all cache.
Note that you do not actually need to clear all cache if you know what you need to clear, e.g. to clear the field's value for a particular node, you can use cache_clear_all('field:node:[your nid here]', 'cache_field');
If your are trying to change more than the field's value, I suggest you do it through the Field API.
I'd like to know whether default values are applied at the time of instance creation, instance update, or only when attempting to read the value. I'd like to understand how this applies to queries.
For example, suppose I initially create a model w/o default values
class Foo (db.Model):
bar = db.BooleanProperty()
Then I create and put a few instances of Foo.
Then later I update the model
class Foo (db.Model):
bar = db.BooleanProperty(default=False)
And then I have a query
foos = Foo.all().filter('bar =', False)
Will the result include the instances of Foo that were created prior to adding the default value to the model definition?
And if I instead query as
foos = Foo.all().filter('bar !=', True)
Does this include the instances of Foo that were created prior to adding the default value to the model definition? Is it (in the case of a Boolean Property) and wrt default values any different to the previous query?
I looked for docs on this, but found none, sorry. Hopefully I'm just missing them. If possible, please provide links to GAE docs.
Update:
I found this in the docs
Entities lacking a property named in the query are ignored
Entities of the same kind need not have the same properties. To be eligible as a query result, an entity must possess a value (possibly null) for every property named in the query's filters and sort orders. If not, the entity is omitted from the indexes used to execute the query and consequently will not be included in the query's results.
Note: It is not possible to query for entities that are specifically lacking a given property. One alternative is to define a fixed (modeled) property with a default value of None, then filter for entities with None as the value of that property.
https://cloud.google.com/appengine/docs/python/datastore/queries
Assuming that default values are applied at time of entity creation or update, I believe that means that neither of the above queries would return entities that were last put prior to the default value being added to the model. Is that right?
thanks
It really depends. If you actually put a "bar" in your foo prior to having the default value or not.
1- In the case you never set bar and just "put" foos, then yes, anything you put prior to adding the default value will have nothing as their "bar".
2- In the case you actually programaticaly setup bars for your foos as you create them, then your query will return everything.
The way you phrased your question makes me believe you had it setup with # 1, so no, anything you put in your datastore before the default value wouldn't be returned. You will need to go through your whole datastore and update your foos.
While not in the official documentation, this has been validated in comments by Alex Martelli (a Googler)
The default values are assigned only on when left untouched on entity creation. Updates will keep the original value.
This default value doesn't affect queries in any way, indexes are written when you save the entity, so you would need to write the entity again with the proper value.
I have a persistent class stored in a GAE datastore. I have removed one of the attributes from the class. The new records in this table show a value <none> for the removed attribute. But is there a way I can completely drop this column off the table?
Thanks.
Added the following 'migration' code according to moraes' suggestion, but it isn't achieving the desired result:
PersistenceManager pm = PMF.get().getPersistenceManager();
try {
Query q = pm.newQuery(UserLogin.class);
Collection<UserLogin> list = (Collection<UserLogin>) q.execute();
Iterator<UserLogin> iter = list.iterator();
while (iter.hasNext()) {
UserLogin obj = (UserLogin) iter.next();
obj.setLoginDate(obj.getLoginDate());
}
pm.makePersistentAll(list);
} finally {
pm.close();
}
I found the answer to this problem in this Article:
http://code.google.com/appengine/articles/update_schema.html
"Removing Deleted Properties from the Datastore
If you remove a property from your model, you will find that existing entities still have the property. It will still be shown in the admin console and will still be present in the datastore. To really clean out the old data, you need to cycle through your entities and remove the data from each one.
Make sure you have removed the properties from the model definition.
If your model class inherits from db.Model, temporarily switch it to inherit from db.Expando. (db.Model instances can't be modified dynamically, which is what we need to do in the next step.)
Cycle through existing entities (like described above). For each entity, use delattr to delete the obsolete property and then save the entity.
If your model originally inherited from db.Model, don't forget to change it back after updating all the data."
And here is an example with code:
http://sandrylogan.wordpress.com/2010/12/08/delattr/
If you are using ndb (and you probably should), you can easily delete properties by deleting them from entity._properties:
for entity in MyModel.query():
if 'old_property' in entity._values:
del entity._properties['old_property']
del entity._values['old_property']
entity.put()
Or you could make it faster by using an asynchronous query map:
#ndb.tasklet
def cleanup(entity):
if 'old_property' in entity._values:
del entity._properties['old_property']
del entity._values['old_property']
yield entity.put_async()
MyModel.query().map(cleanup)
There is no concept of "table" in datastore. Each entity can have arbitrary properties that don't follow a common schema. The only "schema" is in your model code, and existing records don't change automatically when you change your models.
So, to delete the property from existing records, you need to iterate over all records and re-save them without the property.
The datastore viewer gets its list of columns from the datastore stats, which are updated on a regular basis. If you've removed that column from every entity that had it, wait a day or two and the datastore viewer will stop showing it.
I'm having a pretty hard time trying to create a write-once field in a Django model. Ideally I'd want it to work like a final variable, although I can settle for simply preventing it from being edited through the admin.
I know there is a solution for read-only fields, but it also affects the add form, and I don't want the field to be read-only there.
Use get_readonly_fields(), and return a tuple with the write-once field name if obj exists, or an empty tuple if obj is None.
http://docs.djangoproject.com/en/dev/ref/contrib/admin/#django.contrib.admin.ModelAdmin.get_readonly_fields