Load list of items in objectify - google-app-engine

I have Question, Like and Hashtag entities. Also there is one to many relationship between Like and Question entities. I am using google cloud endpoints and my problem begins here. In my list method, I return 20 question as json. But for each question object in query I have to check if user is already liked the question and also fetch related hashtags that belongs to the question. How can I do the same operation by key only batch query. Otherwise, I do
ofy().load().type(Like.class)
.filter("questionRef =", questionKey)
.filter("accountRef =", accountKey).first().now();
for each object.
Like entity
#Entity
#Cache
public class Like {
#Id
#Getter
protected Long id;
#Index
#Load
#ApiResourceProperty(ignored = AnnotationBoolean.TRUE)
private Ref<Account> accountRef;
#Index
#Load
#ApiResourceProperty(ignored = AnnotationBoolean.TRUE)
private Ref<Question> questionRef;
#Index
#Getter
protected Date createdAt;
Like() {
}
Like(Key<Account> accountKey) {
this.accountRef = Ref.create(accountKey);
this.createdAt = new Date();
}
}
Hashtag entity
#Entity
#Cache
public class Hashtag implements Model<Hashtag> {
#Id
#Getter
#ApiResourceProperty(ignored = AnnotationBoolean.TRUE)
private Long id;
#Index
#Load
#ApiResourceProperty(ignored = AnnotationBoolean.TRUE)
private Ref<Question> questionRef;
#Index
#Getter
#Setter
private String text;
private Hashtag() {
}
private Hashtag(Builder builder) {
this.questionRef = builder.questionRef;
this.text = builder.text;
}
}

There are several parts to this question.
First, hashtags: Just store hashtags in the Question as an indexed list property. Easy.
Second, likes: There are a couple ways to do this efficiently.
One is to create a Like entity with a natural key of "account:question" (use the stringified websafe key). This way you can do a batch get by key for all the {user,question} tuples. Some will be absent, some will be present. Reasonably efficient if you're only concerned about 20 questions, especially if you #Cache the Like.
Another is to create a separate Relation Index Entity that tracks all the likes of a user and just load those up each time. You can put 5k items in any list property, which means you'll need to juggle multiple entities when a user likes more than 5k things. But it's easy to load them all up with a single ancestor query. The RIE will need to be #Parented by the User.
On a separate note - don't call fields thingRef. It's just a thing. The data in the database is just a key. You can interchange Ref<?>, Key<?>, and the native low-level Key. Type information doesn't belong in database names.

I am not sure if you can change the structure of your entities. If the answer is no, then there is no option other than the approach you have taken.
If yes, I would suggest structuring your Question to include the Like and Hashtag information as well.
#Entity
public class Question {
#Id
private long id;
private Set<Key<Account>> likedBy;
private List<String> hashtags;
}
For a question, you can retrieve all the information in one single query. Then collect all the Account keys and make another datastore query to retrieve all the people who have liked the question using keys as below:
Map<Key<Account>, Account> likedByAccounts = ofy().load().keys(accountKeys);

Related

Spring data JPA inserting null into column error even though I POST a value?

I want to save both child and parent entities whenever a POST call is made. I have an Item entity with a one to one mapping to a parent table Attribute:
#Entity
#Table(name="Item")
#JsonIgnoreProperties(ignoreUnknown = true)
public class Item
{
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name="id")
private Long id;
#OneToOne(fetch=FetchType.LAZY)
#JoinColumn(name="attr_id")
private Attribute attribute;
#OneToMany(mappedBy = "item", cascade = CascadeType.ALL, orphanRemoval=true)
private List<ValueSet> valueSets = new ArrayList<>();
// Other fields, getters, setters, overriding equals and hashcode using Objects.equals() and Objects.hashCode() for all the fields, helper for adding and removing ValueSet
}
The Attribute entity looks like this:
#Entity
#Table(name="Attribute")
#JsonIgnoreProperties(ignoreUnknown = true)
public class Attribute
{
#Id
#Column(name="id")
private Long id;
// Other fields, getters, setters, NOT overriding equals hashCode
}
Whenever an Item gets saved I need the Attribute to get saved as well. I've my postman sending JSON POST data as follows:
{
"attribute":{
"id":"6"
},
"valueSets":[
{
"value":"basic"
}
]
}
My handler looks like this:
#PostMapping("/item")
public void postItems(#RequestBody Item item)
{
itemRepository.save(item);
}
ItemRepository is just a one liner with #Repository annotation:
public interface ItemRepository extends CrudRepository<Item, Long>
When I try to save the Item I run into - Cannot insert the value NULL into column 'attr_id', table 'Item'; column does not allow nulls. INSERT fails.
I can't figure out why is it unable to take the id value of 6 that I am supplying as part of my POST invocation. The id value 6 already exists on the Attribute table. I have also tried making the relationship bi-directional using mappedBy and CASCADE.ALL but still get the same error.
Any thoughts/suggestions on what I'm messing/missing? Also, is there a better approach to handle nested entities? Should I try to save each of them individually? In that case can the #RequestBody still be the parent entity?
I have built an example project, and try to replicate your situation, not successful. I am able to insert the "item" without any issue.
I placed the project under this repository https://github.com/hepoiko/user-5483731-1
Hope this help you to troubleshooting further or let me know If I miss anything in there.

Objectify: Filter by an attribute of collection entries?

I'm using Objectify on Google's AppEngine.
I have the following Entity-Model:
#Entity
public class ChallengeEntity {
#Id
private Long id;
#Index
public List<ChallengeParticipant> participants;
}
The Participant (not an entity... should it be one?)
public class ChallengeParticipant {
#Load
public Ref<UserEntity> user;
// ... participant-specific attributes
}
And the User-Entity:
#Entity
public class UserEntity {
#Id
Long id;
#Index
public String email = "";
}
Now how would I find all challenges for a given user-email?
Something along:
ofy().load().type(ChallengeEntity.class).filter("participants.user.email", "test#local.foo")
I am willing to adapt my entity-model to GAE's needs... how may I support this query efficiently and keep a nice model?
Thanks alot
Assuming your list of ChallengeParticipant is reasonably bounded (a few hundred at most) and you aren't at risk of hitting the 1M per-entity size limit, you're probably best leaving it as embedded.
To perform your query, first lookup the person by email, then filter by person:
UserEntity user = // load user (or get the key) by email
ofy().load().type(ChallengeEntity.class).filter("participants.user", user);
Note that you need to #Index the ChallengeParticipant.user field, not the ChallengeEntity.participants list.
Assuming that email is unique for a user, I'd keep ChallengeParticipant as a separate entity and maintain 2 way relationship with ChallangeEntity:
public class ChallengeParticipant {
#Id
String email; // must be able to uniquely identify a user.
List<Ref<ChallengeEntity>> challenges;
// ... participant-specific attributes
}
ChallengeEntity will exist as is but without any #Index
#Entity
public class ChallengeEntity {
#Id
private Long id;
public List<Ref<ChallengeParticipant>> participants;
}
When you want to add a new participant to a challenge, update both entities (Participant & Challenge) in one transaction. As there are no indexes involved, you'll always get consistent results.

Objectify doesn't always return results

I am using Objectify to store data on Google App Engine's datastore. I have been trying to implement a one-to-many relationship between two classes, but by storing a list of parameterised keys. The method below works perfectly some of the time, but returns an empty array other times - does anyone know why this may be?
It will either return the correct list of CourseYears, or
{
"items": [
]
}
Here is the method:
#ApiMethod(name = "getCourseYears") #ApiResourceProperty(ignored = AnnotationBoolean.TRUE)
public ArrayList<CourseYear> getCourseYears(#Named("name") String name){
Course course = ofy().load().type(Course.class).filter("name", name).first().now();
System.out.println(course.getName());
ArrayList<CourseYear> courseYears = new ArrayList<CourseYear>();
for(Key<CourseYear> courseYearKey: course.getCourseYears()){
courseYears.add(ofy().load().type(CourseYear.class).id(courseYearKey.getId()).now());
}
return courseYears;
}
The Course class which stores many CourseYear keys
#Entity
public class Course {
#Id
#Index
private Long courseId;
private String code;
#Index
private String name;
#ApiResourceProperty(ignored = AnnotationBoolean.TRUE)
public List<Key<CourseYear>> getCourseYears() {
return courseYears;
}
#ApiResourceProperty(ignored = AnnotationBoolean.TRUE)
public void setCourseYears(List<Key<CourseYear>> courseYears) {
this.courseYears = courseYears;
}
#ApiResourceProperty(ignored = AnnotationBoolean.TRUE)
public void addCourseYear(Key<CourseYear> courseYearRef){
courseYears.add(courseYearRef);
}
#Load
#ApiResourceProperty(ignored = AnnotationBoolean.TRUE)
List<Key<CourseYear>> courseYears = new ArrayList<Key<CourseYear>>();
...
}
I am debugging this on the debug server using the API explorer. I have found that it will generally work at the start for a few times but if I leave and return to the API and try and run it again, it will not start working again after that.
Does anyone have any idea what might be going wrong?
Many thanks.
You might want to reduce the amount of queries you send to the datastore. Try something like this:
Course course = ofy().load().type(Course.class).filter("name", name).first().now();
ArrayList<CourseYear> courseYears = new ArrayList<CourseYear>();
List<Long> courseIds = new List<>();
for(Key<CourseYear> courseYearKey: course.getCourseYears()){
courseIds.add(courseYearKey.getId());
}
Map<Long, Course> courses = ofy().load().type(CourseYear.class).ids(courseIds).list();
// add all courses from map to you courseYears list
I also strongly recommend a change in your data structure / entities:
In your CourseYears add a property Ref<Course> courseRef with the parent Course and make it indexed (#Index). Then query by
ofy().load().type(CourseYear.class).filter("courseRef", yourCourseRef).list();
This way you'll only require a single query.
The two most likely candidates are:
Eventual consistency behavior of the high replication datastore. Queries (ie your filter() operation) always run a little behind because indexes propagate through GAE asynchronously. See the GAE docs.
You haven't installed the ObjectifyFilter. Read the setup guide. Recent versions of Objectify throws an error if you haven't installed it, so if you're on the latest version, this isn't it.

GAE jpa database model example

I am totally new at this, I am sorry if it is stupid question.
I am trying to design database model for Google App Engine in JPA, but I am unable to get it right. When I find the way I can't get annotations right or I am getting error about M:N not supported in Google App Engine.
I need entity user to have multiple groups and groups have multiple users and there are users who are also group admins.
My basic model was User -> usergroup(user; group; (bool)isAdmin) <-Group
Can somebody give a clean and simple example of how to define relationships?
Please try this.
#Entity
public class User {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Key id;
private String name;
#ManyToOne(fetch = FetchType.LAZY)
private UserGroup usergroup;
}
class userGroup
#Entity
public class UserGroup {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Key id;
private String name;
private boolean admin;
#OneToMany(mappedBy = "usergroup", cascade = CascadeType.ALL)
private List<User> users = new ArrayList<User>();
}
please be noticed GAE have limitation on JPA you can read more here
I don't know anything about Google App Engine, but I can help with JPA though.
The problem here is the "isAdmin" column, which prevents the data model to be a simple #ManyToMany relationship with a joiner table.
With the introduction of this field, in the data model you need a Map on the User entity with key=Group and value=isAdmin, similarly you need a corresponding Map in the Group entity in order to know if each User is an admin.
This is modeled with #ElementCollection in the following way:
#Entity
#Table(name="User")
public class User
{
#Id
#GeneratedValue(strategy= GenerationType.TABLE)
private int id;
private String name;
#ElementCollection
#CollectionTable(name="Users_Groups", joinColumns={#JoinColumn(name="userId")})
#MapKeyJoinColumn(name="groupId")
#Column(name="isAdmin")
private Map<Group, Boolean> groups;
}
#Entity
#Table(name="Group")
public class Group
{
#Id
#GeneratedValue(strategy= GenerationType.TABLE)
private int id;
private String name;
#ElementCollection
#CollectionTable(name="Users_Groups", joinColumns={#JoinColumn(name="groupId")})
#MapKeyJoinColumn(name="userId", insertable=false, updatable=false)
#Column(name="isAdmin", insertable=false, updatable=false)
private Map<User, Boolean> users;
}
The important annotation is #ElementCollection, the other annotations are just to name the specific columns of the collection table and make sure they match from both entities: #CollectionTable gives the name of the table and the name of the column representing the id in the current entity. #MapKeyJoinColumn gives the name of the column representing the id of the "key" element in the Map, and #Column gives the name of the "value" element in the map.
I'm not sure if the insertable=false and updatable=false are needed in one of the entities, might avoid adding duplicate rows due to the cyclic dependency between User and Group.
Also you need to manually create the collection table, because at least EclipseLink tries to create it with two "groupId" and "isAdmin" columns. You might consider reviewing the design if it is absolutely needed a cyclic dependency between User and Group.

Ancestor Query with Objectify not returning results

I am using objectify 3.1 on appengine and attempting to do a ancestor query. I want to fetch all the children of an object that are of a certain kind. Below is my code:
#Cached
#Entity
public class Car {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
private String make;
private String model;
}
#Cached
#Entity
public class Tire {
#Id
public String keyName;
#Parent Key<Car> car;
private String brand;
private String size;
}
My query is this code:
List<Tire> list= ofy.query(Tire.class).ancestor(new Key<Car(Car.class,carID))).list();
When I create the tire objects I use this code to set the relationship:
newTire.setCar(new Key<Car>(Car.class,Car.getID()));
I know the Parent relationship is there because I can query for it in the datastore admin and it shows the parent in the decoded entity key:
Decoded entity key: Car: id=135172 > Tire: name=myCarUniqueID
This query always returns 0 results, and it seems like I have followed all the best practices on the objectify website. Any help would be appreciated!
Your code looks ok, so the only thing that might be wrong is a non-existing carID.
The Objectify javadoc listed on the site is actually for trunk version which is a forthcoming Objectify 4. What you need to look at is Objectify 3.1 javadoc: this version has fetch() on the query.
Also, #GeneratedValue is not an Objectify annotation.

Resources