I'm working with Objectify to access DataStore in a Google App Engine application.
I have 2 Entities:
#Entity
public class Client {
#Id String id;
}
#Entity
public class Queue {
#Index #Id String name;
#Load LinkedHashMap<String,Ref<Client>> clientsInQueue;
}
In a transaction, i do something like this:
Client newClient = new Client(...);
ofy().save().entity(newClient);
Queue selectedQueue = ofy().load().type(Queue.class).id(queueName).now();
selectedQueue.getClientsInQueue().put(newClient.getId(), Ref.create(newClient));
But when i try to print all the elements in the LinkedHashMap, i noticed the elements is in the exactly reverse order.
For example if i add 2 Clients in the LinkedHashMap and save it, like this:
Queue selectedQueue = new LinkedHashMap<String,Ref<Client>>();
String id1 = "id1";
Client c1 = new Client(id1);
ofy().save().entity(c1);
String id2 = "id2"
Client c2 = new Client(id2);
ofy().save().entity(c2);
selectedQueue.getClientsInQueue().put(c1.getId(), Ref.create(c1));
selectedQueue.getClientsInQueue().put(c2.getId(), Ref.create(c2));
ofy().save().entity(selectedQueue);
Set<String> keys = selectedQueue.getClientsInQueue().keySet();
for(String key : keys){
Client c= selectedQueue.getClientsInQueue().get(key).get();
log.info("id: "+c.getId());
}
The result is:
id: id2
id: id1
Why i obtain this behavior ? I know that LinkedHashMap has to maintain the order of the keys! Is there any problem to use LinkedHashMap with GAE DataStore?
Thanks in advance.
Cheers,
Alessandro.
Fields of type Map<String, ?> are stored in the low level api as type EmbeddedEntity, the only map-like structure available as a property. This EmbeddedEntity is implemented as a non-linked HashMap. Consequently, there is no way for Objectify to preserve order.
I'll make a note of this in Objectify's documentation. If you would like this behavior to change, open up an issue in GAE's issue tracker requesting "Entity and EmbeddedEntity should use a LinkedHashMap to preserve order".
Related
I have a Course entity that contains the following field
#Index
private #Load
Ref<Student> student;
The student entity then has the field
#Index
private String matric;
I want to load all the Course entities sorted using the students matric number.
I have tried using the "." operator to get the sub field like this
ofy().load().type(Course.class).filter("course", course).order("student.matric").list();
but this return no result.
Is it possible to do this? how?
I don't think that is possible with objectify. I would let Course implement Comparable:
#Entity
public class Course implements Comparable<Course> {
.
.
.
#Override
public int compareTo(Course otherCourse) {
return this.getStudent().getMatric().compareTo(otherCourse.getStudent().getMatric());
}
}
Remove the "order" part of the Objectify load and use Collections.sort() instead:
List<Course> courses = ofy().load().type(Course.class).filter("course", course).list();
Collections.sort(courses);
There are no joins in the datastore. If you want to query your Courses by Student properties, you probably will need to denormalize the data into the Course and index it. This means changing the Student data will also require changing Courses.
As an aside: This data model is weird. Are you sure what you're calling Course isn't really an Enrollment?
I am using Objectify to store data on Google App Engine's datastore. I have been trying to implement a one-to-many relationship between two classes, but by storing a list of parameterised keys. The method below works perfectly some of the time, but returns an empty array other times - does anyone know why this may be?
It will either return the correct list of CourseYears, or
{
"items": [
]
}
Here is the method:
#ApiMethod(name = "getCourseYears") #ApiResourceProperty(ignored = AnnotationBoolean.TRUE)
public ArrayList<CourseYear> getCourseYears(#Named("name") String name){
Course course = ofy().load().type(Course.class).filter("name", name).first().now();
System.out.println(course.getName());
ArrayList<CourseYear> courseYears = new ArrayList<CourseYear>();
for(Key<CourseYear> courseYearKey: course.getCourseYears()){
courseYears.add(ofy().load().type(CourseYear.class).id(courseYearKey.getId()).now());
}
return courseYears;
}
The Course class which stores many CourseYear keys
#Entity
public class Course {
#Id
#Index
private Long courseId;
private String code;
#Index
private String name;
#ApiResourceProperty(ignored = AnnotationBoolean.TRUE)
public List<Key<CourseYear>> getCourseYears() {
return courseYears;
}
#ApiResourceProperty(ignored = AnnotationBoolean.TRUE)
public void setCourseYears(List<Key<CourseYear>> courseYears) {
this.courseYears = courseYears;
}
#ApiResourceProperty(ignored = AnnotationBoolean.TRUE)
public void addCourseYear(Key<CourseYear> courseYearRef){
courseYears.add(courseYearRef);
}
#Load
#ApiResourceProperty(ignored = AnnotationBoolean.TRUE)
List<Key<CourseYear>> courseYears = new ArrayList<Key<CourseYear>>();
...
}
I am debugging this on the debug server using the API explorer. I have found that it will generally work at the start for a few times but if I leave and return to the API and try and run it again, it will not start working again after that.
Does anyone have any idea what might be going wrong?
Many thanks.
You might want to reduce the amount of queries you send to the datastore. Try something like this:
Course course = ofy().load().type(Course.class).filter("name", name).first().now();
ArrayList<CourseYear> courseYears = new ArrayList<CourseYear>();
List<Long> courseIds = new List<>();
for(Key<CourseYear> courseYearKey: course.getCourseYears()){
courseIds.add(courseYearKey.getId());
}
Map<Long, Course> courses = ofy().load().type(CourseYear.class).ids(courseIds).list();
// add all courses from map to you courseYears list
I also strongly recommend a change in your data structure / entities:
In your CourseYears add a property Ref<Course> courseRef with the parent Course and make it indexed (#Index). Then query by
ofy().load().type(CourseYear.class).filter("courseRef", yourCourseRef).list();
This way you'll only require a single query.
The two most likely candidates are:
Eventual consistency behavior of the high replication datastore. Queries (ie your filter() operation) always run a little behind because indexes propagate through GAE asynchronously. See the GAE docs.
You haven't installed the ObjectifyFilter. Read the setup guide. Recent versions of Objectify throws an error if you haven't installed it, so if you're on the latest version, this isn't it.
I'm getting a ClassCastException (java.lang.Double cannot be cast to java.lang.Long) when retrieving multiple instances of a class in the datastore. The class has many values that are doubles and many that are longs. I'd like to view what is in the datastore and compare with the class properties to see if there is a mismatch. I tried the representationsOfProperty method found near the bottom of https://cloud.google.com/appengine/docs/java/datastore/metadataqueries#Java_Kind_queries, but my queries return null.
I have a class defined similar to the following:
#PersistenceCapable(identityType = IdentityType.APPLICATION)
public class Container
{
#PrimaryKey
#Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Long containerID = null;
#Persistent
private Long extensionID = null;
#Persistent
private String homeUrl;
#Persistent
private Double containerScore;
}
I copied the code from the GAE page linked above. The only change I made was to convert 'key' to "key" since a string is requested and what is shown in the example isn't a character.
Collection<String> representationsOfProperty(DatastoreService ds,
String kind,
String property)
{
// Start with unrestricted non-keys-only property query
Query q = new Query(Entities.PROPERTY_METADATA_KIND);
// Limit to specified kind and property
q.setFilter(new FilterPredicate("__key__", Query.FilterOperator.EQUAL, Entities.createPropertyKey(kind, property)));
// Get query result
PreparedQuery pq = ds.prepare(q);
Entity propInfo = pq.asSingleEntity();
if( null == propInfo )
{
Collection<String> strs = new ArrayList<String>();
strs.add( "[ERROR: Invalid Query: " + pq.toString() + "]" );
return strs;
}
// Return collection of property representations
return (Collection<String>) propInfo.getProperty("property_representation");
}
I call that method with the following code:
DatastoreService datastore = DatastoreServiceFactory.getDatastoreService();
String prop = "containerID";
Collection<String> reps = representationsOfProperty( datastore, Container.class.toString(), prop );
Unfortunately, propInfo is always null. Any ideas on what I could try? Am I doing something wrong?
I just created a sample project to reproduce your issue. JDO is saving the entities in the Datastore, but not naming the primary key property according to the member variables of the PersistenceCapable class. Check your datastore viewer and grab a random Container entity to see what's happening.
If you need to do metadata queries on the id property in this manner, you should be aware that the ID/Name property is part of the key. Check this interesting talk to learn more about how the Datastore internals work (it's based on BigTable).
I am using google cloud SQL with JDO. When I try to use the JDO PersistenceManager to store new objects with a new key it works fine, however I get an error when I try to update entities already inserted in the db:
com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry '503062001-43661003' for key 'PRIMARY'
The key is indeed duplicated, but I want to update that object.
Is it possible to do this with the PersistentManager.makePersistentAll() method or in an another way that avoids to write the UPDATE query manually?
More details:
The object I am trying to persist is defined like this:
PersistenceCapable(table = "xxx")
public class XXX {
#PrimaryKey
#Index(name = "xxx_id")
private Long userId;
#PrimaryKey
#Index(name = "xxx_idx")
#Column(length = 128)
private String otherId;
...
}
If you want to update the object then you retrieve the object first, and update it (either in the same txn, or detach it and update it whilst detached). All of that would be described in the JDO spec
For example https://db.apache.org/jdo/pm.html and page down to "Update Objects"
I'm currently developing a mobile application who uses a Google App Engine-hosted web service.
But i'm facing an issue. I just want to add a field in one my database's table.
App Engine doesn't use classic SQL syntax, but GQL. So i cannot use the ALTER TABLE statement. How can i do this with GQL ? I looked for a solution on the web, but there's not a lot of help.
public MyEntity() {
}
#Id
#GeneratedValue(strategy=GenerationType.IDENTITY)
private Key idStation;
private String name;
private double longitude;
private double latitude;
private java.util.Date dateRefresh = new Date(); //the field i want to add in DB
So, now when i create a "MyEntity" object, it should add the "dateRefresh" field into the database... I create my object like this:
MyEntity station = new MyEntity();
station.setName("test");
station.setLatitude(0);
station.setLongitude(0);
station.setDateRefresh(new Date("01/01/1980"));
DaoFactory.getStationDao().addStation(station);
addStation method:
#Override
public MyEntity addStation(MyEntity station) {
EntityManager em = PersistenceManager.getEntityManagerFactory().createEntityManager();
try {
em.getTransaction().begin();
em.persist(station);
em.getTransaction().commit();
} finally {
if(em.getTransaction().isActive()) em.getTransaction().rollback();
em.close();
}
return station;
}
The field "dateRefresh" is never created into my DB...
Someone to help me please ?
Thanks in advance
Just add another field to your data structure, maybe providing a default clause, and that's all. For example, if you have a UserAccount:
class UserAccount(db.Model):
user = db.UserProperty()
user_id = db.StringProperty()
you may easily add:
class UserAccount(db.Model):
user = db.UserProperty()
user_id = db.StringProperty()
extra_info = db.IntegerProperty(default=0)
timezone = db.StringProperty(default="UTC")
and let it go.
While the datastore kinda mimics tables, data is stored on a per entity basis. There is no schema or table.
All you need to do is update your model class, and new entities will be saved with the structure (fields) of the new entity.
Old entities and indexes, however, are not automatically updated. They still have the same fields as they had when they were originally written to the datastore.
There's two ways to do this. One is to make sure your code can handle situations where your new properties are missing, ie make sure no exceptions are thrown, or handle the exceptions properly when you're missing the properties.
The second way is to write a little function (usu a mapreduce function) to update every entity with appropriate or null values for your new properties.
Note that indexes are not updated unless the entity is written. So if you add a new indexed property, old entities won't show up when you query for the new property. In this case, you must use the second method and update all the entities in the datastore so that they are indexed.