I am using Objectify to store data on Google App Engine's datastore. I have been trying to implement a one-to-many relationship between two classes, but by storing a list of parameterised keys. The method below works perfectly some of the time, but returns an empty array other times - does anyone know why this may be?
It will either return the correct list of CourseYears, or
{
"items": [
]
}
Here is the method:
#ApiMethod(name = "getCourseYears") #ApiResourceProperty(ignored = AnnotationBoolean.TRUE)
public ArrayList<CourseYear> getCourseYears(#Named("name") String name){
Course course = ofy().load().type(Course.class).filter("name", name).first().now();
System.out.println(course.getName());
ArrayList<CourseYear> courseYears = new ArrayList<CourseYear>();
for(Key<CourseYear> courseYearKey: course.getCourseYears()){
courseYears.add(ofy().load().type(CourseYear.class).id(courseYearKey.getId()).now());
}
return courseYears;
}
The Course class which stores many CourseYear keys
#Entity
public class Course {
#Id
#Index
private Long courseId;
private String code;
#Index
private String name;
#ApiResourceProperty(ignored = AnnotationBoolean.TRUE)
public List<Key<CourseYear>> getCourseYears() {
return courseYears;
}
#ApiResourceProperty(ignored = AnnotationBoolean.TRUE)
public void setCourseYears(List<Key<CourseYear>> courseYears) {
this.courseYears = courseYears;
}
#ApiResourceProperty(ignored = AnnotationBoolean.TRUE)
public void addCourseYear(Key<CourseYear> courseYearRef){
courseYears.add(courseYearRef);
}
#Load
#ApiResourceProperty(ignored = AnnotationBoolean.TRUE)
List<Key<CourseYear>> courseYears = new ArrayList<Key<CourseYear>>();
...
}
I am debugging this on the debug server using the API explorer. I have found that it will generally work at the start for a few times but if I leave and return to the API and try and run it again, it will not start working again after that.
Does anyone have any idea what might be going wrong?
Many thanks.
You might want to reduce the amount of queries you send to the datastore. Try something like this:
Course course = ofy().load().type(Course.class).filter("name", name).first().now();
ArrayList<CourseYear> courseYears = new ArrayList<CourseYear>();
List<Long> courseIds = new List<>();
for(Key<CourseYear> courseYearKey: course.getCourseYears()){
courseIds.add(courseYearKey.getId());
}
Map<Long, Course> courses = ofy().load().type(CourseYear.class).ids(courseIds).list();
// add all courses from map to you courseYears list
I also strongly recommend a change in your data structure / entities:
In your CourseYears add a property Ref<Course> courseRef with the parent Course and make it indexed (#Index). Then query by
ofy().load().type(CourseYear.class).filter("courseRef", yourCourseRef).list();
This way you'll only require a single query.
The two most likely candidates are:
Eventual consistency behavior of the high replication datastore. Queries (ie your filter() operation) always run a little behind because indexes propagate through GAE asynchronously. See the GAE docs.
You haven't installed the ObjectifyFilter. Read the setup guide. Recent versions of Objectify throws an error if you haven't installed it, so if you're on the latest version, this isn't it.
Related
I'm currently working in project that needs Cassandra database to have search ability. We've got DataStax cluster and we want to use Spring Data to simplify database operations. However, when we made entity that got both - #Table (for cassandra) and #SolrDocument (for Solr) it happened to be broken. The only error we got is the one below. Anyone have encountered such a problem?
Caused by: org.springframework.data.mapping.PropertyReferenceException: No property findAll found for type ENTITYNAME!
I know that this is probably Spring issue, but hope to find someone who have fought this type of problem.
Greetings!
Some sample entity causing problems:
#SolrDocument(solrCoreName = "sample_entity")
#Table("sample_entity")
#Getter
#Setter
#AllArgsConstructor
#NoArgsConstructor
public final class SampleEntity {
#PrimaryKey
#Indexed(name = "id")
private UUID id;
private LocalDateTime created;
private UUID foreignId;
#Indexed(name = "name")
private String name;
private boolean someflag = true;
}
You're mixing up things - if you're using DSE Search, then it's better to perform search via CQL, by querying in the solr_query column. In your example, the #SolrDocument will force using of the Solr's HTTP API, while #Table will force use of CQL.
You can use Object Mapper from DataStax to map classes to tables, like this:
// STest.java
#Table(keyspace = "test",name = "stest")
public class STest {
#PartitionKey
private int id;
private String t;
}
// STestAccessor.java
#Accessor
public interface STestAccessor {
#Query("SELECT * FROM test.stest WHERE solr_query = :solr")
Result<STest> getViaSolr(#Param("solr") String solr);
}
// STestMain.java
MappingManager manager = new MappingManager(session);
STestAccessor sa = manager.createAccessor(STestAccessor.class);
Result<STest> rs = sa.getViaSolr("*:*");
for (STest sTest : rs) {
System.out.println("id=" + sTest.getId() + ", text=" + sTest.getT());
}
Here is the full code.
I have Question, Like and Hashtag entities. Also there is one to many relationship between Like and Question entities. I am using google cloud endpoints and my problem begins here. In my list method, I return 20 question as json. But for each question object in query I have to check if user is already liked the question and also fetch related hashtags that belongs to the question. How can I do the same operation by key only batch query. Otherwise, I do
ofy().load().type(Like.class)
.filter("questionRef =", questionKey)
.filter("accountRef =", accountKey).first().now();
for each object.
Like entity
#Entity
#Cache
public class Like {
#Id
#Getter
protected Long id;
#Index
#Load
#ApiResourceProperty(ignored = AnnotationBoolean.TRUE)
private Ref<Account> accountRef;
#Index
#Load
#ApiResourceProperty(ignored = AnnotationBoolean.TRUE)
private Ref<Question> questionRef;
#Index
#Getter
protected Date createdAt;
Like() {
}
Like(Key<Account> accountKey) {
this.accountRef = Ref.create(accountKey);
this.createdAt = new Date();
}
}
Hashtag entity
#Entity
#Cache
public class Hashtag implements Model<Hashtag> {
#Id
#Getter
#ApiResourceProperty(ignored = AnnotationBoolean.TRUE)
private Long id;
#Index
#Load
#ApiResourceProperty(ignored = AnnotationBoolean.TRUE)
private Ref<Question> questionRef;
#Index
#Getter
#Setter
private String text;
private Hashtag() {
}
private Hashtag(Builder builder) {
this.questionRef = builder.questionRef;
this.text = builder.text;
}
}
There are several parts to this question.
First, hashtags: Just store hashtags in the Question as an indexed list property. Easy.
Second, likes: There are a couple ways to do this efficiently.
One is to create a Like entity with a natural key of "account:question" (use the stringified websafe key). This way you can do a batch get by key for all the {user,question} tuples. Some will be absent, some will be present. Reasonably efficient if you're only concerned about 20 questions, especially if you #Cache the Like.
Another is to create a separate Relation Index Entity that tracks all the likes of a user and just load those up each time. You can put 5k items in any list property, which means you'll need to juggle multiple entities when a user likes more than 5k things. But it's easy to load them all up with a single ancestor query. The RIE will need to be #Parented by the User.
On a separate note - don't call fields thingRef. It's just a thing. The data in the database is just a key. You can interchange Ref<?>, Key<?>, and the native low-level Key. Type information doesn't belong in database names.
I am not sure if you can change the structure of your entities. If the answer is no, then there is no option other than the approach you have taken.
If yes, I would suggest structuring your Question to include the Like and Hashtag information as well.
#Entity
public class Question {
#Id
private long id;
private Set<Key<Account>> likedBy;
private List<String> hashtags;
}
For a question, you can retrieve all the information in one single query. Then collect all the Account keys and make another datastore query to retrieve all the people who have liked the question using keys as below:
Map<Key<Account>, Account> likedByAccounts = ofy().load().keys(accountKeys);
I'm using Objectify on Google's AppEngine.
I have the following Entity-Model:
#Entity
public class ChallengeEntity {
#Id
private Long id;
#Index
public List<ChallengeParticipant> participants;
}
The Participant (not an entity... should it be one?)
public class ChallengeParticipant {
#Load
public Ref<UserEntity> user;
// ... participant-specific attributes
}
And the User-Entity:
#Entity
public class UserEntity {
#Id
Long id;
#Index
public String email = "";
}
Now how would I find all challenges for a given user-email?
Something along:
ofy().load().type(ChallengeEntity.class).filter("participants.user.email", "test#local.foo")
I am willing to adapt my entity-model to GAE's needs... how may I support this query efficiently and keep a nice model?
Thanks alot
Assuming your list of ChallengeParticipant is reasonably bounded (a few hundred at most) and you aren't at risk of hitting the 1M per-entity size limit, you're probably best leaving it as embedded.
To perform your query, first lookup the person by email, then filter by person:
UserEntity user = // load user (or get the key) by email
ofy().load().type(ChallengeEntity.class).filter("participants.user", user);
Note that you need to #Index the ChallengeParticipant.user field, not the ChallengeEntity.participants list.
Assuming that email is unique for a user, I'd keep ChallengeParticipant as a separate entity and maintain 2 way relationship with ChallangeEntity:
public class ChallengeParticipant {
#Id
String email; // must be able to uniquely identify a user.
List<Ref<ChallengeEntity>> challenges;
// ... participant-specific attributes
}
ChallengeEntity will exist as is but without any #Index
#Entity
public class ChallengeEntity {
#Id
private Long id;
public List<Ref<ChallengeParticipant>> participants;
}
When you want to add a new participant to a challenge, update both entities (Participant & Challenge) in one transaction. As there are no indexes involved, you'll always get consistent results.
So I have the follow entity with the following code to create and update the object. The resulting data after calling the method is that the field Value has the value of 1 in the Datastore running at my local computer. Why? Isn't the object JPA-managed (or attached as described in this article) and supposed to update itself upon em2.close()?
Entity:
#Entity
public class MyObj {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
public Long Id;
public int Value = 0;
}
Code:
public void createAndUpdate() {
Long id = null;
// Line below calls Persistence.createEntityManagerFactory("transactions-optional").createEntityManager()
EntityManager em1 = Database.getEntityManager();
MyObj obj = new MyObj();
obj.Value = 1;
em1.persist(obj);
em1.close();
id = obj.Id;
EntityManager em2 = Database.getEntityManager();
obj = (MyObj) em2.find(MyObj.class, id);
obj.Value = 2;
// NOTE1
em2.close();
}
On the line marked NOTE1 I have tried inserting em2.persist(obj) and em2.merge(obj) (even though from what I've read, that should not be neccessary since find() returns an attached object). None of them helped.
I'm using JPA v1.
As the JPA spec says, you cannot directly update fields of Entity objects, yet in your posted code you do just that (so the persistence provider has no way of knowing that something has changed, and so can't flush those changes to the datastore).
DataNucleus JPA does actually allow you to update the field directly if you enhance the class that does it to be "PersistenceAware".
Solution : use setters to update fields
I'm pretty new to JPA/JDO and the whole objectdb world.
I have an entity with a set of strings, looks a bit like:
#Entity
public class Foo{
#Id
#GeneratedValue(strategy=GenerationType.IDENTITY)
private Key id;
private Set<String> bars;
public void setBars(Set<String> newBars){
if(this.bars == null)
this.bars = new HashSet<String>;
this.bars = newBars;
}
public Set<String> getBars(){
return this.bars;
}
public void addBar(String bar){
if(this.bars == null)
this.bars = new HashSet<String>;
this.bars.add(bar);
}
}
Now, in another part of the code, I'm trying to do something like this:
EntityManager em = EMF.get().createEntityManager();
Foo myFoo = em.find(Foo.class, fooKey);
em.getTransaction().begin();
myFoo.addBar(newBar);
em.merge(myFoo);
em.getTransaction().commit();
When, of course, newBar is a String.
But, what I get is:
javax.jdo.JDODetachedFieldAccessException: You have just attempted to access field "bars" yet this field was not detached when you detached the object. Either dont access this field, or detach it when detaching the object.
I've searched for an answer, but I couldn't find one.
I've seen someone ask about a Set of strings, and he was told to add an #ElementCollection notation.
I tried that, but I got an error about the String class Metadata (I don't really understand what it means.)
I would really appreciate some help on this thing, even a good reference to someone explaining this (in simple English).
OK,
So I found the answer in some blog.
So for anyone who's interested:
In order to use a Collection of simple data types (in JPA), a
#Basic
notation should be added to the collection. So from my example at the top, It should've been written:
#Basic
private Set<String> bars;
So you are using JPA, right? (I see EntityManager rather than JDO's PersistenceManager.) Since you are getting a JDO error, I suspect that your app isn't configured properly for JPA.
JPA docs: http://code.google.com/appengine/docs/java/datastore/jpa/overview.html
JDO docs: http://code.google.com/appengine/docs/java/datastore/jdo/overview.html
You need to pick one datastore wrapper and stick with it. The default new app with the Eclipse tools is configured for JDO, and it is a reasonable choice, but you'll have to change your annotations around a little bit.