Why is it not possible to unindex a list of an objectify entity?
To demonstrate the problem I made a simple example project.
I used the entity
#Entity
public class Car {
#Id String id;
#Unindex List<Passenger> passengers;
}
an the object
public class Passenger {
String name;
}
and saved it using this simple method.
public class CarFactory {
public void writeCarEntity() {
Car car = new Car();
car.setId("myCar");
List<Passenger> passengers = new LinkedList<Passenger>();
Passenger carl = new Passenger();
carl.setName("Carl");
Passenger pete = new Passenger();
pete.setName("Pete");
Passenger jeff = new Passenger();
jeff.setName("Jeff");
passengers.add(carl);
passengers.add(pete);
passengers.add(jeff);
car.setPassengers(passengers);
ObjectifyService.register(car.getClass());
ObjectifyService.ofy().save().entity(car).now();
}
}
Looking up the entity in the datastore you get this information:
Although the passengers field has the annotation #Unindex it will be indexed, as one can see in the google "Datastore". Why does the annotation #Unindex has no effect in this example???
This is unrelated to Objectify and appears to be some new quirk of the datastore. It might just be a display glitch in the UI. Is it causing problems?
With the code you posted, Objectify will call Entity.saveUnindexedProperty() on the passengers field (even without the #Unindex annotation). But even if Objectify tried to index it, historically you can't index embedded objects, so it's unclear what it means to index a list of them. Maybe Google is rolling out some new behavior and they haven't got the GUI working correctly yet? Or maybe there is a bug in their save behavior?
If you want to be a good citizen, create a simple test case with the low level API (an Entity that contains a property of type List<EmbeddedEntity>), verify that this same behavior occurs, and file a bug in the GAE issue tracker.
Related
I have a Course entity that contains the following field
#Index
private #Load
Ref<Student> student;
The student entity then has the field
#Index
private String matric;
I want to load all the Course entities sorted using the students matric number.
I have tried using the "." operator to get the sub field like this
ofy().load().type(Course.class).filter("course", course).order("student.matric").list();
but this return no result.
Is it possible to do this? how?
I don't think that is possible with objectify. I would let Course implement Comparable:
#Entity
public class Course implements Comparable<Course> {
.
.
.
#Override
public int compareTo(Course otherCourse) {
return this.getStudent().getMatric().compareTo(otherCourse.getStudent().getMatric());
}
}
Remove the "order" part of the Objectify load and use Collections.sort() instead:
List<Course> courses = ofy().load().type(Course.class).filter("course", course).list();
Collections.sort(courses);
There are no joins in the datastore. If you want to query your Courses by Student properties, you probably will need to denormalize the data into the Course and index it. This means changing the Student data will also require changing Courses.
As an aside: This data model is weird. Are you sure what you're calling Course isn't really an Enrollment?
Let's say I have a very easy, classic setup: GAE(1.7.4) + GWT(2.5.0) Application, running on local Jetty (Development Server), using JDO for persistence.
Let's also say I have just 2 #PersistenceCapable classes: Person and Color. Every Person has exactly one favourite Color, but it does not mean that this Person owns this Color - many different Persons can have the same favourite Color. There is a limited number of well-known Colors and a Color may exist even if it is not anyone's favourite.
To model this I should use #Unowned relationship - please correct me if I am wrong:
#PersistenceCapable
public class Color { // just the most regular Entity class
#PrimaryKey
#Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Key key;
#Persistent
String rgb;
// getter, setter, no constructor
}
#PersistenceCapable
public class Person {
#PrimaryKey
#Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Key key;
#Persistent
String surname;
#Persistent
#Unowned // here is the tricky part
Color color;
// getters, setters, no constructor
}
With some simple, well-known, PersistentManager-based code, I am able to successfully create and persist an instance of a Color class. I see it in GAE Development Console -> Datastore Viewer, having nice generated Key and ID/Name of (13), and my assigned RGB.
With very similar code, I am able to create an instance of Person class (in another request), assign a pre-existing Color as his favourite color (it pre-existed, I obtained it by pm.getObjectById()) and persist it. I see it in Datastore Viewer, with my nice generated Key and ID/Name of (15) and my assigned surname, and color_key_OID of (13). This looks very promising.
But then, when I fetch the Person(15) back from the DB (simple pm.getObjectById(), no transactions), it has my assigned surname correctly, but has null instead of Color(13)! Right - the Datastore Viewer gets it ok, but my code does not.
Oh, the problematic code? "Person p = pm.getObjectById(Person.class, key);".
(side notes: I am also having the same problem with #Unowned collections (nice list of values in Datastore Viewer, but null Collection field in my code.) My JDO jars on classpath are "datanucleus-api-jdo-3.1.1.jar" and "jdo-api-3.0.1.jar" so I assume they support #Unowned. There is no problem with not-#Unowned fields. I get no exceptions upon persisting or fetching, just plain nulls as field values.)
Either mark the color to be "eagerly fetched"
#Persistent(defaultFetchGroup="true")
#Unowned
Color color
or define your own fetchgroup like this:
#FetchGroup(name="eager", members={#Persistent(name="color")})
#PersistenceCapable
public class Person {
and use it if required by specifying the group to be fetched:
PersistenceManager pm = pmf.getPersistenceManager();
pm.getFetchPlan().addGroup("eager");
I was facing the same issue in one of my #Unowned Lists. I had more other two, which the Array is fetched perfectly.
What solved this issue for me was to change the name of property for a bigger one. In your case is like change the property name from "color" to something bigger, like "myfavoritecolor".
I have the same issue what you describe. How DataNucleus said you need to describe the whole lifecycle of the objects. In my case the problem was solved forcing getting the color, from the person object, before closing the PersistenceManager with the close() function.
Remember JDO uses the lazy-load technique to get objects.
I was able to solve this problem by adding fetch groups to the query and not to persistent manager.
PersistenceManager pm = PMF.get().getPersistenceManager();
logger.info("EVENTS FETCH GROUPS : " + pm.getFetchPlan().getGroups());
/*pm.getFetchPlan().addGroup("eventFetchGroup");
pm.getFetchPlan().setMaxFetchDepth(2);*/
Query q = pm.newQuery(Event.class);
q.getFetchPlan().addGroup("eventFetchGroup");
logger.info("EVENTS FETCH GROUPS : " +q.getFetchPlan().getGroups());
q.setFilter("date >= fromDate && date <= toDate");
q.declareParameters("java.util.Date fromDate, java.util.Date toDate");
This is my first question on any of these websites so pardon my unprofessionalism.
I use playframework with SIENA module (with GAE) and I came accross the following problem:
Given 3 entities:
public class Meeting extends Model{
#Id
public Long id;
public String place;
#Owned
Many<MeetingUser> users;
.
.
.
}
public class User extends Model{
#Id
public Long id;
public String firstName;
public String lastName;
#Owned
Many<MeetingUser> meetings;
.
.
.
}
public class MeetingUser extends Model{
#Id
public Long id;
public Meeting meeting;
public User user;
.
.
.
public User getUser(){
return Model.all(User.class).filter("id", user).get();
}
public Meeting getMeeting(){
return Model.all(Meeting.class).filter("id", meeting).get();
}
}
For instance I am listing a meeting and all their users:
public static void meetingInfo(Long meetingId){
Meeting meeting = Models.all(Meeting.class).filter("id",meetingId);
List<MeetingUser> meetingusers = meeting.asList();
List<User> users = new ArrayList<User>();
for(MeetingUser mu: meetingusers){
users.add(mu.getUser());
}
render(users);
}
This is done(is there any better way here?) however when it comes to filtering (especially dynamic filtering for many many fields) I can not use the Query's filter method on the MeetingUser as I need to filter on a MeetingUser's field's field (firstName). The same problem arise for ordering. I need the solution for both problems.
I hope my problem is clear and I appreciate any kind of help here.
Remember that you are in GAE which is a NoSQL DB.
So you can't do Join request as in RDBMS.
Yet, this is not really the pb you have so this was just to be sure you are aware of it ;)
So if you want to find the person having given firstname in a given meeting, can you try the following:
List<MeetingUser> meetingusers = meeting.users.asQuery().filter("firstname", "XXX");
(you can also order)
Nevertheless, knowing that you can't join, remember that you can't write a query searching for a meeting in which there are users whose firstname is XXX as it would require some joins and it doesn't exist in GAE. In this case, you need to change your model following NoSQL philosophy but this is another subject
regards
Let's try to give a way to do what you want...
Your relation is a Many-to-Many which is always the worst case :)
You want to filter Meeting by User's firstname.
It requires a join request which is not possible in GAE. In this case, you must change your model by denormalizing it (sometimes use redundancy also) and manage the join by yourself. Actually, you must do the job of the RDBMS by yourself. It seems overkill but in fact, it's quite easy. The only drawback is that you must perform several requests to the DB. NoSQL means No Schema (& No Join) so there are a few drawbacks but it allows to scale and to manage huge data load... it depends on your needs :)
The choice you did to create the MeetingUser which is a "joined" table and a kind of denormalization is good in GAE because it allows to manage the join yourself.
Solution:
// fetch users by firstname
List<User> users = users.all().filter("firstName", "John").fetch();
// fetch meetingusers associated to these users (verify the "IN" operator works because I didn't use that for a long time and don't remember if it works with this syntax)
List<MeetingUser> meetingusers = MeetingUser.all().filter("user IN", users);
// now you must fetch the whole meeting because in MeetingUser, only the Meeting ID is stored (other fields are Null or O)
List<Meeting> meetings = new ArrayList<Meeting>()
for(MeetingUsers mu:meetingusers) {
meetings.add(meetingusers.meeting);
}
// use the batch feature to fetch all objects
Meeting.batch(Meeting.class).get(meetings);
// you have your meetings
Hope this helps!
Some Background
I have a game database with a table called Games that has multiple attributes and one called Genres. The Genres attribute is defined as an integer[] in PostgreSQL. For the sake of simplicity, I'm not using any foreign key constraints, but essentially each integer in this array is a foreign key constraint on the id attribute in the Genres table. First time working with the NetBeans Master/Detail Sample Form and Java persistence and it's been working great so far except for 1 thing. I get this error when the program tries to display a column that has a 1-dimensional integer array. In this example, the value is {1, 11}.
Exception Description: The object [{1,11}], of class [class org.postgresql.jdbc3.Jdbc3Array], from mapping [oracle.toplink.essentials.mappings.DirectToFieldMapping[genres-->final.public.games.genres]] with descriptor [RelationalDescriptor(finalproject.Games --> [DatabaseTable(final.public.games)])], could not be converted to [class [B].
Exception [TOPLINK-3002] (Oracle TopLink Essentials - 2.0.1 (Build b09d-fcs (12/06/2007))): oracle.toplink.essentials.exceptions.ConversionException
My Research
From what I've been able to read, it looks like PostgreSQL arrays need something special done to them before you can display and edit them in this template. By default, the sample form uses TopLink Essentials (JPA 1.0) as its persistence library, but I can also use Hibernate (JPA 1.0).
Here is the code that needs to be changed in some way. From the Games.java file:
#Entity
#Table(name = "games", catalog = "final", schema = "public")
#NamedQueries({
// omitting named queries
#NamedQuery(name = "Games.findByGenres", query = "SELECT g FROM Games g WHERE g.genres = :genres")
})
public class Games implements Serializable {
#Transient
private PropertyChangeSupport changeSupport = new PropertyChangeSupport(this);
private static final long serialVersionUID = 1L;
// omitting other attributes
#Column(name = "genres")
private Serializable genres;
// omitting constructors and other getters/setters
public Serializable getGenres() {
return genres;
}
public void setGenres(Serializable genres) {
Serializable oldGenres = this.genres;
this.genres = genres;
changeSupport.firePropertyChange("genres", oldGenres, genres);
}
} // end class Games
Here are also some of the sites that might have the solution that I'm just not understanding:
https://forum.hibernate.org/viewtopic.php?t=946973
http://blog.xebia.com/2009/11/09/understanding-and-writing-hibernate-user-types/
// omitted hyperlink due to user restriction
Attempted Solutions
I'm able to get the data to display if I change the type of genres to String, but it is immutable and I cannot edit it. This is what I changed to do this:
#Column(name = "genres")
private String genres;
public String getGenres() {
return genres;
}
public void setGenres(String genres) {
String oldGenres = this.genres;
this.genres = genres;
changeSupport.firePropertyChange("genres", oldGenres, genres);
}
I also attempted to create a UserType file for use with Hibernate (JPA 1.0), but had no idea what was going wrong there.
I also attempted to use the #OneToMany and other tags, but these aren't working probably because I'm not using them properly.
What I'm Looking For
There has to be a simple way to get this data to display and make it editable, but since I'm completely new to persistence, I have no idea what to do.
The effort put into your question shows. Unfortunately JPA does not currently support PostgreSQL arrays. The fundamental problem is that arrays are not frequently used in many other databases frequently and so heavy reliance on them is somewhat PostgreSQL specific. Thus you can expect that general cross-db persistence API's are not generally going to support them well if at all. JPA is no exception, having currently no support for PostgreSQL arrays.
I have been looking at writing my own persistence API in Java that would support arrays, but it hasn't happened yet, would be PostgreSQL-only when written, and would be based on a very different principle than JPA and friends.
I'm pretty new to JPA/JDO and the whole objectdb world.
I have an entity with a set of strings, looks a bit like:
#Entity
public class Foo{
#Id
#GeneratedValue(strategy=GenerationType.IDENTITY)
private Key id;
private Set<String> bars;
public void setBars(Set<String> newBars){
if(this.bars == null)
this.bars = new HashSet<String>;
this.bars = newBars;
}
public Set<String> getBars(){
return this.bars;
}
public void addBar(String bar){
if(this.bars == null)
this.bars = new HashSet<String>;
this.bars.add(bar);
}
}
Now, in another part of the code, I'm trying to do something like this:
EntityManager em = EMF.get().createEntityManager();
Foo myFoo = em.find(Foo.class, fooKey);
em.getTransaction().begin();
myFoo.addBar(newBar);
em.merge(myFoo);
em.getTransaction().commit();
When, of course, newBar is a String.
But, what I get is:
javax.jdo.JDODetachedFieldAccessException: You have just attempted to access field "bars" yet this field was not detached when you detached the object. Either dont access this field, or detach it when detaching the object.
I've searched for an answer, but I couldn't find one.
I've seen someone ask about a Set of strings, and he was told to add an #ElementCollection notation.
I tried that, but I got an error about the String class Metadata (I don't really understand what it means.)
I would really appreciate some help on this thing, even a good reference to someone explaining this (in simple English).
OK,
So I found the answer in some blog.
So for anyone who's interested:
In order to use a Collection of simple data types (in JPA), a
#Basic
notation should be added to the collection. So from my example at the top, It should've been written:
#Basic
private Set<String> bars;
So you are using JPA, right? (I see EntityManager rather than JDO's PersistenceManager.) Since you are getting a JDO error, I suspect that your app isn't configured properly for JPA.
JPA docs: http://code.google.com/appengine/docs/java/datastore/jpa/overview.html
JDO docs: http://code.google.com/appengine/docs/java/datastore/jdo/overview.html
You need to pick one datastore wrapper and stick with it. The default new app with the Eclipse tools is configured for JDO, and it is a reasonable choice, but you'll have to change your annotations around a little bit.