I've changed my object to have a new required property in v2. When I attempt to fetch a v1 object from the datastore, I get BadValueError because v1 doesn't have the required property. What's the best way to introduce new required properties on existing data
I would resolve this problem using the mapreduce library.
First, register the mapper in mapreduce.yaml:
mapreduce:
- name: fixing required property
mapper:
input_reader: mapreduce.input_readers.DatastoreInputReader
handler: your handler
params:
- name: entity_kind
default: main.ModelV2
then define a process function to modify the entities:
from mapreduce import operation as op
def process(entity):
if not entity.newproperty :
entity.newproperty = None
yield op.db.Put(entity)
If you are dealing with a relative small number of entities, you could avoid mapreduce modifying directly your entities with something like this:
entities = ModelV2.all()
for entity in entities :
if not entity.newproperty :
entity.newproperty = None
entity.put()
You'll need to add it as an optional property to your model, fetch each existing entity, add the property to it (generating a reasonable value somehow), then put() the entity. Once all of your existing entities have been "upgraded", you can make the property required.
The AppEngine mapreduce API should make this fairly easy.
Related
I recently encountered a situation where one might want to run a datastore query which includes a kind, but the class of the corresponding model is not available (e.g. if it's defined in a module that hasn't been imported yet).
I couldn't find any out-of-the-box way to do this using the google.appengine.ext.db package, so I ended up using the google.appengine.api.datastore.Query class from the low-level datastore API.
This worked fine for my needs (my query only needed to count the number of results, without returning any model instances), but I was wondering if anyone knows of a better solution.
Another approach I've tried (which also worked) was subclassing db.GqlQuery to bypass its constructor. This might not be the cleanest solution, but if anyone is interested, here is the code:
import logging
from google.appengine.ext import db, gql
class ClasslessGqlQuery(db.GqlQuery):
"""
This subclass of :class:`db.GqlQuery` uses a modified version of ``db.GqlQuery``'s constructor to suppress any
:class:`db.KindError` that might be raised by ``db.class_for_kind(kindName)``.
This allows using the functionality :class:`db.GqlQuery` without requiring that a Model class for the query's kind
be available in the local environment, which could happen if a module defining that class hasn't been imported yet.
In that case, no validation of the Model's properties will be performed (will not check whether they're not indexed),
but otherwise, this class should work the same as :class:`db.GqlQuery`.
"""
def __init__(self, query_string, *args, **kwds):
"""
**NOTE**: this is a modified version of :class:`db.GqlQuery`'s constructor, suppressing any :class:`db.KindError`s
that might be raised by ``db.class_for_kind(kindName)``.
In that case, no validation of the Model's properties will be performed (will not check whether they're not indexed),
but otherwise, this class should work the same as :class:`db.GqlQuery`.
Args:
query_string: Properly formatted GQL query string.
*args: Positional arguments used to bind numeric references in the query.
**kwds: Dictionary-based arguments for named references.
Raises:
PropertyError if the query filters or sorts on a property that's not indexed.
"""
from google.appengine.ext import gql
app = kwds.pop('_app', None)
namespace = None
if isinstance(app, tuple):
if len(app) != 2:
raise db.BadArgumentError('_app must have 2 values if type is tuple.')
app, namespace = app
self._proto_query = gql.GQL(query_string, _app=app, namespace=namespace)
kind = self._proto_query._kind
model_class = None
try:
if kind is not None:
model_class = db.class_for_kind(kind)
except db.KindError, e:
logging.warning("%s on %s without a model class", self.__class__.__name__, kind, exc_info=True)
super(db.GqlQuery, self).__init__(model_class)
if model_class is not None:
for property, unused in (self._proto_query.filters().keys() +
self._proto_query.orderings()):
if property in model_class._unindexed_properties:
raise db.PropertyError('Property \'%s\' is not indexed' % property)
self.bind(*args, **kwds)
(also available as a gist)
You could create a temporary class just to do the query. If you use an Expando model, the properties of the class don't need to match what is actually in the datastore.
class KindName(ndb.Expando):
pass
You could then do:
KindName.query()
If you need to filter on specific properties, then I suspect you'll have to add them to the temporary class.
Hiy!
I want all objects(rows in Test Type) with ModelService
So I could iterate through collection and update a Single row (object)'s attribute with new value
I see getModelService.create(TestModel.class) and getModelService.save()
but will they not create a new object/row rather than update a existing object?right
I don't want to create a new one rather selecting one of the existing matching my criteria and update one attribute of that
can somebody help with List<TestModel> testModels = getModelService.get(TestModel.class) will that return me all rows (collection) of Test Type/Table?
unfortunately I can't test it so need help
Actually I am in validateInterceptor ... and on the basis of this intercepted model changed attribute value I have to update another model attribute value...
thanks
ModelService.create(new TestModel.class) will create a single instance of the specified type and attach it to the modelservice's context.
But it will only be saved to the persistence store when you call modelService.save(newInstance)
ModelService.get() returns a model object but expects a Jalo object as input, (Jalo being the legacy persistence layer of hybris) so that won't work for you.
To retrieve objects you can either write your own queries using the FlexibleSearchService or you can have a look at the DefaultGenericDao which has a bunch of simple find() type of methods.
Typically you would inject the dao like e.g.:
private GenericDao<TestModel> dao;
[...]
public void myMethod()
{
List<TestModel> allTestModels = dao.find();
[...]
}
There are a lot more methods with which you can create WHERE type of statements to restrict your result.
Regarding ValidateInterceptor:
Have a look at the wiki page for the lifecycle of interceptors:
https://wiki.hybris.com/display/release5/Interceptors
It's not a good idea to modify 'all' objects of a type while being an interceptor of that type.
So if you're in an interceptor declared for the Test item type, then don't try to modify the items there.
If you happen to be in a different interceptor and want to modify items of a different type:
E.g. you have Type1 which has a list of Type2 objects in it and in the interceptor for Type1 you want to modify all Type2 objects.
For those scenarios you would have to add the instances of Type2 that you modify to the interceptor context so that those changes will be persisted.
That would be something like:
void onValidate(Test1 model, InterceptorContext ctx) throws InterceptorException
{
...
List<Type2> type2s = dao.find();
for (Type2 type2 : type2s)
{
// do something with it
// then make sure to persist that change
ctx.registerElementFor(type2, PersistenceOperation.SAVE);
[...]
}
}
First of all - i think it's not a good idea, to create/update models in any interceptor, especially in 'validation' one.
Regarding your question:
ModelService in most of the cases works with single model, and
designed for create/update/delete operations.
To retreive all models of certain type, you have to use FlexibleSearchService
Then to update each retrieved TestType model, you can use ModelService's save method.
A query to retreive all TestType models will look like:
SELECT PK FROM {TestType}
You could simply use the Flexible Search Service search by example method, and the model service to save them all. Here is an example using Groovy script, with all products :
import java.util.List
import de.hybris.platform.core.model.product.ProductModel
import de.hybris.platform.servicelayer.search.FlexibleSearchService
import de.hybris.platform.servicelayer.model.ModelService
FlexibleSearchService fsq = spring.getBean("flexibleSearchService")
ModelService ms = spring.getBean("modelService")
ProductModel prd = ms.create(ProductModel.class)
List<ProductModel> products = fsq.getModelsByExample(prd)
//Do Whatever you want with the objects in the List
ms.saveAll(products)
My goal is to keep session size as small as possible. (Why?.. it's other topic).
What I have is Phase listener declared in faces-config.xml
<lifecycle>
<phase-listener>mypackage.listener.PhaseListener</phase-listener>
</lifecycle>
I want to save all other views, except the last one(maximum two) , in some memcache. Getting the session map:
Map<String, Object> sessionMap = event.getFacesContext().getExternalContext().getSessionMap();
in beforePhase(PhaseEvent event) method is giving me access to all views. So here I could save all views to the memcache and delete them from the session. The question is where in jsf these views that are still loaded in the browser are requested so that I can refill with this view if it's needed. Is it possible at all? Thank you.
To address the core of your question, implement a ViewHandler, within which you can take control of the RESTORE_VIEW and RENDER_RESPONSE phases/processes. You'll save the view during the RENDER_RESPONSE and selectively restore, during the RESTORE_VIEW phase. Your view handler could look something like the following
public class CustomViewHandlerImpl extends ViewHandlerWrapper{
#Inject ViewStore viewStore; //hypothetical storage for the views. Could be anything, like a ConcurrentHashMap
ViewHandler wrapped;
public CustomViewHandlerImpl(ViewHandler toWrap){
this.wrapped = toWrap;
}
public UIViewRoot restoreView(FacesContext context, String viewId) throws IOException{
//this assumes you've previously saved the view, using the viewId
UIViewRoot theView = viewStore.get(viewId);
if(theView == null){
theView = getWrapped().restoreView(context, viewId);
}
return theView;
}
public void renderView(FacesContext context, UIViewRoot viewToRender) throws IOException, FacesException{
viewStore.put(viewToRender.getId(),viewToRender);
getWrapped().renderView(context, viewToRender);
}
}
Simply plug in your custom viewhandler, using
<view-handler>com.you.customs.CustomViewHandlerImpl</view-handler>
Of course, you probably don't want to give this treatment to all your views; you're free to add any conditions to the logic above, to implement conditional view-saving and restoration.
You should also consider other options. It appears that you're conflating issues here. If your true concern is limit the overhead associated with view processing, you should consider
Stateless Views, new with JSF-2.2. The stateless view option allows you to exclude specific pages from the JSF view-saving mechanism, simply by specifying transient="true" on the f:view. Much cleaner than mangling the UIViewRoot by hand. The caveat here is that a stateless view cannot be backed by scopes that depend on state-saving, i.e. #ViewScoped. In a stateless view, the #ViewScoped bean is going to be recreated for every postback. Ajax functionality also suffers in this scenario, because state saving is the backbone of ajax-operations.
Selectively set mark components as transient The transient property is available for all UIComponents, which means, on a per-view basis, you can mark specific components with transient="true", effectively giving you the same benefits as 1) but on a much smaller scope. Without the downside of no ViewScoped
EDIT: For some reason, UIViewRoot#getViewId() is not returning the name of the current view (this might be a bug). Alternatively, you can use
ExternalContext extCtxt = FacesContext.getCurrentInstance().getExternalContext();
String viewName = ((HttpServletRequest)extCtxt.getRequest()).getRequestURI(); //use this id as the key to store your views instead
I'm using XmlSerializer. I've had no problems with it until now. I updated Silverlight from 4 to 5 and at the same time also updated the WCF RIA Services from v1 SP1 to v1 SP2. Now the following line gives me an error.
XmlSerializer s = new XmlSerializer(typeof(MyCustomObject));
The error is:
System.InvalidOperationException: System.ServiceModel.DomainServices.Client.EntityConflict cannot be serialized because it does not have a parameterless constructor.
The object I'm using (MyCustomObject in the sample) has not changed in any way so I'm starting to think it's either SL5 or the new RIA Services that is breaking my code. I didn't find any breaking changes document or mentions that this could happen. I don't know why it has a problem with EntityConflict since I'm not using any entities within my object.
Has anyone seen an error like this and/or know how to solve it?
UPDATE!
The final property that the error message says before EntityConflict is an Entity. I think that makes a difference but it has been working before. I'd also like to know why the serializer already tries to serialize the object in the constructor?
public static XmlSerializer GetEntityXmlSerializer<TEntity>()
where TEntity : Entity
{
XmlAttributes ignoreAttribute = new XmlAttributes()
{
XmlIgnore = true,
};
// use base class of Entity,
// if you use type of implementation
// you will get the error.
Type entityType = typeof(Entity);
var xmlAttributeOverrides = new XmlAttributeOverrides();
xmlAttributeOverrides.Add(entityType, "EntityConflict", ignoreAttribute);
xmlAttributeOverrides.Add(entityType, "EntityState", ignoreAttribute);
return new XmlSerializer(typeof(TEntity), xmlAttributeOverrides);
}
I am not sure why this would be happening, RIA Services entities are not XmlSerializable objects and the entities themselves are not decorated with the [Serializable] attribute. Have you added partial classes on the client side which decorate the entities with [Serializable] or modified the code generation in some way?
I got around this problem by using intermediary serializable POCO objects which were copies of my custom objects (which were inherited from Entity). The POCO objects did not inherit from Entity. I just updated their values from the original Entity objects. They then serialized quite nicely. Of course, when you de-serialize you need to update your Entity objects from the POCO objects.
When you change data models on the app engine to add new properties those entries without a certain property are listed with the value <missing> in the online data viewer.
What I'm wondering is how can I write a query to find those entries?
There is no direct way to query for older entities with missing attribute, but you can design data model upfront to support this. Add a version attribute to each model class. Version should have a default value, which is increased every time model class is changed and deployed. This way you will be able to query entities by version number.
There's no way to query the datastore for entities that don't have a given property. You need to iterate over all the entities and check each one - possibly by using the mapreduce API.
Or you could create a script to stick null in there for all current items that don't have that property using the low level datastore API, so then you can query on null.
I had this issue and that's how I solved it. The rough code would look like:
DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
Query query = new Query("JDOObjectType");
List<Entity> results = datastore.prepare(query).asList(FetchOptions.Builder.withLimit(9999));
for (Entity lObject : results) {
Object lProperty = lObject.getProperty("YOUR_PROPERTY");
if (lProperty == null) {
lObject.setProperty("YOUR_PROPERTY", null);
datastore.put(lProperty);
}
}
}