I have an entity in my app engine datastore. There's actually only one instance of this entity. I can see it in my admin console. Is it possible to add a new attribute to the entity via the admin console (using gql perhaps)?
Right now it looks something like:
Entity: Foo
Attributes: mName, mAge, mScore
and I'd like to add a new boolean attribute to this entity like "mGraduated" or something like that.
In the worst case I can write some code to delete the entity then save a new one, but yeah was just wondering.
Thanks
-------- Update ---------
Tried adding the new attribute to my class (using java) and upon loading from the datastore I get the following:
java.lang.NullPointerException:
Datastore entity with kind Foo and key Foo(\"Foo\") has a null property named mGraduated.
This property is mapped to com.me.types.Foo.mGraduated, which cannot accept null values.
This is what my entity class looks like, I just added the new attribute (mGraduated), then deployed, then tried loading the single entity from the datastore (which produced the above exception):
#PersistenceCapable
public class Foo
{
#PrimaryKey
private String k;
/** Some old attributes, look like the following. */
#Persistent
#Extension(vendorName = "datanucleus", key = "gae.unindexed", value="true")
private String mName;
...
/** Tried adding the new one. */
#Persistent
#Extension(vendorName = "datanucleus", key = "gae.unindexed", value="true")
private boolean mGraduated;
The only way to implement this is to use Boolean as the type for the new property..
Than in set method you can accept boolean value, that's no issue.
If you want the get method to also return boolean.. you also can, but be sure to check if the value is null and if so.. return default value (e.g. true)
so
private Boolean newProp = null; // can also assing default value .. e.g. true;
public void setNewProp(boolean val)
{
this.newProp = val;
}
public boolean getNewProp()
{
if(this.newProp == null)
return true; // Default value if not set
return this.newProp.booleanValue();
}
I recommend you not to migrate your data in this case - it can be very costly and can deplete your quota easily (read old data, create new, delete old = 3 operations for every entry in you data store)
You can't do this through the admin console, but you shouldn't have to delete the entity. Instead just update it- the Datastore does not enforce schemas for Kinds.
E.g., if Foo is a subclass of db.Model (Python), change your model subclass to include the new property; fetch the model instance (e.g., by its key), update the instance, including setting the value of the new field; and save the modified instance. Since you just have one instance this is easy. With many such instances to update you'd probably want to do this via task queue tasks or via a mapreduce job.
You have declared the new mGraduated field using the primitive type boolean, which cannot be null. The existing entity can't be loaded into the model class because it doesn't have this property. One option is to declare this property using the Boolean class, which can accept a null value.
The Admin Console only knows about properties in existing entities. You cannot use the Admin Console directly to create a new property with a name not used by any existing entities. (This is just a limitation of the Console. App code can do this easily.)
Related
I wanted to know if the {save} method in CrudRepository do an update if it finds already the entry in the database like :
#Repository
public interface ProjectDAO extends CrudRepository<Project, Integer> {}
#Service
public class ProjectServiceImpl {
#Autowired private ProjectDAO pDAO;
public void save(Project p) { pDAO.save(p); } }
So if I call that method on an already registred entry, it'll update it if it finds a changed attribute ?
Thanks.
I wanted to know if the {save} method in CrudRepository do an update
if it finds already the entry in the database
The Spring documentation about it is not precise :
Saves a given entity. Use the returned instance for further operations
as the save operation might have changed the entity instance
completely.
But as the CrudRepository interface doesn't propose another method with an explicit naming for updating an entity, we may suppose that yes since CRUD is expected to do all CRUD operations (CREATE, READ, UPDATE, DELETE).
This supposition is confirmed by the implementation of the SimpleJpaRepository
class which is the default implementation of CrudRepository which shows that both cases are handled by the method :
#Transactional
public <S extends T> S save(S entity) {
if (entityInformation.isNew(entity)) {
em.persist(entity);
return entity;
} else {
return em.merge(entity);
}
}
So if I call that method on an already registered entry, it'll update
it if it finds a changed attribute?
It will do a merge operation in this case. So all fields are updated according to how the merging cascade and read-only option are set.
Looking at the default implemantation of CrudRepository interface
/*
* (non-Javadoc)
* #see org.springframework.data.repository.CrudRepository#save(java.lang.Object)
*/
#Transactional
public <S extends T> S save(S entity) {
if (entityInformation.isNew(entity)) {
em.persist(entity);
return entity;
} else {
return em.merge(entity);
}
}
Save method manage two situations:
-If the person Id is null (a new entity is created) then save will call persist method => insert query will be executed.
-If the person id is not null then save will call merge: fetch the existing entity from entityManagerFactory(from the 2 level cache if it doesn't exist then it will be fetched from the database) and comparing the detached entity with the managed and finally propagate the changes to the database by calling update query.
To be precise, the save(obj) method will treat obj as a new record if the id is empty (therefore will do an insert) and will treat obj as an existing record if the id is filled in (therefore will do the merge).
Why is this important?
Let's say the Project object contains an auto-generated id and also a person_id which must be unique. You make a Project object and fill in the person_id but not the id and then try to save. Hibernate will try to insert this record, since the id is empty, but if that person exists in the database already, you will get a duplicate key exception.
How to handle
Either do a findByPersonId(id) to check if the obj is in the db already, and get the id from that if it is found,
Or just try the save and catch the exception in which case you know it's in the db already and you need to get and set the id before saving.
I wanted to know if the {save} method in CrudRepository do an update if it finds already the entry in the database:
The Answer is Yes, It will update if it finds an entry:
From Spring Documentation: Herehttps://docs.spring.io/spring-data/jpa/docs/1.5.0.RELEASE/reference/html/jpa.repositories.html?
Saving an entity can be performed via the CrudRepository.save(…)-Method. It will persist or merge the given entity using the underlying JPA EntityManager. If the entity has not been persisted yet Spring Data JPA will save the entity via a call to the entityManager.persist(…)-Method, otherwise the entityManager.merge(…)-Method will be called.
In my case I had to add the id property to the Entity, and put the annotation #Id like this.
#Id
private String id;
This way when you get the object has the Id of the entity in the database, and does the Update operation instead of the Create.
I am using google cloud SQL with JDO. When I try to use the JDO PersistenceManager to store new objects with a new key it works fine, however I get an error when I try to update entities already inserted in the db:
com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry '503062001-43661003' for key 'PRIMARY'
The key is indeed duplicated, but I want to update that object.
Is it possible to do this with the PersistentManager.makePersistentAll() method or in an another way that avoids to write the UPDATE query manually?
More details:
The object I am trying to persist is defined like this:
PersistenceCapable(table = "xxx")
public class XXX {
#PrimaryKey
#Index(name = "xxx_id")
private Long userId;
#PrimaryKey
#Index(name = "xxx_idx")
#Column(length = 128)
private String otherId;
...
}
If you want to update the object then you retrieve the object first, and update it (either in the same txn, or detach it and update it whilst detached). All of that would be described in the JDO spec
For example https://db.apache.org/jdo/pm.html and page down to "Update Objects"
Let's say I have a very easy, classic setup: GAE(1.7.4) + GWT(2.5.0) Application, running on local Jetty (Development Server), using JDO for persistence.
Let's also say I have just 2 #PersistenceCapable classes: Person and Color. Every Person has exactly one favourite Color, but it does not mean that this Person owns this Color - many different Persons can have the same favourite Color. There is a limited number of well-known Colors and a Color may exist even if it is not anyone's favourite.
To model this I should use #Unowned relationship - please correct me if I am wrong:
#PersistenceCapable
public class Color { // just the most regular Entity class
#PrimaryKey
#Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Key key;
#Persistent
String rgb;
// getter, setter, no constructor
}
#PersistenceCapable
public class Person {
#PrimaryKey
#Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Key key;
#Persistent
String surname;
#Persistent
#Unowned // here is the tricky part
Color color;
// getters, setters, no constructor
}
With some simple, well-known, PersistentManager-based code, I am able to successfully create and persist an instance of a Color class. I see it in GAE Development Console -> Datastore Viewer, having nice generated Key and ID/Name of (13), and my assigned RGB.
With very similar code, I am able to create an instance of Person class (in another request), assign a pre-existing Color as his favourite color (it pre-existed, I obtained it by pm.getObjectById()) and persist it. I see it in Datastore Viewer, with my nice generated Key and ID/Name of (15) and my assigned surname, and color_key_OID of (13). This looks very promising.
But then, when I fetch the Person(15) back from the DB (simple pm.getObjectById(), no transactions), it has my assigned surname correctly, but has null instead of Color(13)! Right - the Datastore Viewer gets it ok, but my code does not.
Oh, the problematic code? "Person p = pm.getObjectById(Person.class, key);".
(side notes: I am also having the same problem with #Unowned collections (nice list of values in Datastore Viewer, but null Collection field in my code.) My JDO jars on classpath are "datanucleus-api-jdo-3.1.1.jar" and "jdo-api-3.0.1.jar" so I assume they support #Unowned. There is no problem with not-#Unowned fields. I get no exceptions upon persisting or fetching, just plain nulls as field values.)
Either mark the color to be "eagerly fetched"
#Persistent(defaultFetchGroup="true")
#Unowned
Color color
or define your own fetchgroup like this:
#FetchGroup(name="eager", members={#Persistent(name="color")})
#PersistenceCapable
public class Person {
and use it if required by specifying the group to be fetched:
PersistenceManager pm = pmf.getPersistenceManager();
pm.getFetchPlan().addGroup("eager");
I was facing the same issue in one of my #Unowned Lists. I had more other two, which the Array is fetched perfectly.
What solved this issue for me was to change the name of property for a bigger one. In your case is like change the property name from "color" to something bigger, like "myfavoritecolor".
I have the same issue what you describe. How DataNucleus said you need to describe the whole lifecycle of the objects. In my case the problem was solved forcing getting the color, from the person object, before closing the PersistenceManager with the close() function.
Remember JDO uses the lazy-load technique to get objects.
I was able to solve this problem by adding fetch groups to the query and not to persistent manager.
PersistenceManager pm = PMF.get().getPersistenceManager();
logger.info("EVENTS FETCH GROUPS : " + pm.getFetchPlan().getGroups());
/*pm.getFetchPlan().addGroup("eventFetchGroup");
pm.getFetchPlan().setMaxFetchDepth(2);*/
Query q = pm.newQuery(Event.class);
q.getFetchPlan().addGroup("eventFetchGroup");
logger.info("EVENTS FETCH GROUPS : " +q.getFetchPlan().getGroups());
q.setFilter("date >= fromDate && date <= toDate");
q.declareParameters("java.util.Date fromDate, java.util.Date toDate");
I'm currently developing a mobile application who uses a Google App Engine-hosted web service.
But i'm facing an issue. I just want to add a field in one my database's table.
App Engine doesn't use classic SQL syntax, but GQL. So i cannot use the ALTER TABLE statement. How can i do this with GQL ? I looked for a solution on the web, but there's not a lot of help.
public MyEntity() {
}
#Id
#GeneratedValue(strategy=GenerationType.IDENTITY)
private Key idStation;
private String name;
private double longitude;
private double latitude;
private java.util.Date dateRefresh = new Date(); //the field i want to add in DB
So, now when i create a "MyEntity" object, it should add the "dateRefresh" field into the database... I create my object like this:
MyEntity station = new MyEntity();
station.setName("test");
station.setLatitude(0);
station.setLongitude(0);
station.setDateRefresh(new Date("01/01/1980"));
DaoFactory.getStationDao().addStation(station);
addStation method:
#Override
public MyEntity addStation(MyEntity station) {
EntityManager em = PersistenceManager.getEntityManagerFactory().createEntityManager();
try {
em.getTransaction().begin();
em.persist(station);
em.getTransaction().commit();
} finally {
if(em.getTransaction().isActive()) em.getTransaction().rollback();
em.close();
}
return station;
}
The field "dateRefresh" is never created into my DB...
Someone to help me please ?
Thanks in advance
Just add another field to your data structure, maybe providing a default clause, and that's all. For example, if you have a UserAccount:
class UserAccount(db.Model):
user = db.UserProperty()
user_id = db.StringProperty()
you may easily add:
class UserAccount(db.Model):
user = db.UserProperty()
user_id = db.StringProperty()
extra_info = db.IntegerProperty(default=0)
timezone = db.StringProperty(default="UTC")
and let it go.
While the datastore kinda mimics tables, data is stored on a per entity basis. There is no schema or table.
All you need to do is update your model class, and new entities will be saved with the structure (fields) of the new entity.
Old entities and indexes, however, are not automatically updated. They still have the same fields as they had when they were originally written to the datastore.
There's two ways to do this. One is to make sure your code can handle situations where your new properties are missing, ie make sure no exceptions are thrown, or handle the exceptions properly when you're missing the properties.
The second way is to write a little function (usu a mapreduce function) to update every entity with appropriate or null values for your new properties.
Note that indexes are not updated unless the entity is written. So if you add a new indexed property, old entities won't show up when you query for the new property. In this case, you must use the second method and update all the entities in the datastore so that they are indexed.
In the python app engine docs, I see something called dbReferenceProperty. I can't understand what it is, or how it's used. I'm using the java interface to app engine, so I'm not sure if there's an equivalent.
I'm interested in it because it sounds like some sort of pseudo-join, where we can point a property of a class to some other object's value - something like if we had:
class User {
private String mPhotoUrl;
private String mPhone;
private String mState;
private String mCountry;
.. etc ..
}
class UserLite {
#ReferenceProperty User.mPhotoUrl;
private String mPhotoUrl;
}
then if we had to update a User object's mPhotoUrl value, the change would somehow propagate out to all UserLite instances referencing it, rather than having to update every UserLite object instance manually,
Thanks
A db.ReferenceProperty simply holds the key of another datastore entity, which is automatically fetched from the datastore when the property is used.
There's some additional magic where the entity that is referenced has access to a query for entities of type Foo that reference it in the special attribute foo_set.
The Java datastore API instead has owned relationships, which serve the same purpose.