Appengine query returning no matching index - Objectify - google-app-engine

I have some entities that I wish to do an ancestor query on and filter a parameter by the ">" operator.
The entity in question inherits another object(I don't think this should matter). Below are my entity classes:
#Indexed
public class ValidatedObject {
public Long timeCreated=System.currentTimeMillis();
public Long timeUpdated=System.currentTimeMillis();
public Long getTimeUpdated() {
return timeUpdated;
}
public void setTimeUpdated(Long timeUpdated) {
this.timeUpdated = timeUpdated;
}
public Boolean validated=false;
#Unindexed
public String validatedID;
#Unindexed
private Long validatedTime;
#Unindexed
private String creatorID;
public Long getTimeCreated() {
return timeCreated;
}
public void setTimeCreated(Long timeCreated) {
this.timeCreated = timeCreated;
}
public boolean isValidated() {
return validated;
}
public void setValidated(boolean validated) {
this.validated = validated;
}
public String getValidatedID() {
return validatedID;
}
public void setValidatedID(String validatedID) {
this.validatedID = validatedID;
}
public Long getValidatedTime() {
return validatedTime;
}
public void setValidatedTime(Long validatedTime) {
this.validatedTime = validatedTime;
}
public String getCreatorID() {
return creatorID;
}
public void setCreatorID(String creatorID) {
this.creatorID = creatorID;
}
}
#Cached
#Entity
public class PersonnelInfo extends ValidatedObject{
#Id
public String keyName;
#Parent Key<Department> department;
private Long fdID;
#Unindexed
private String userKeyName;
private String firstName;
private String lastName;
#Unindexed
private String address,city,county,state;
#Unindexed
private String cellPhone,homePhone,otherPhone;
public PersonnelInfo(){
}
public PersonnelInfo(String email){
keyName=email;
}
#Override
public Long getTimeUpdated() {
return timeUpdated;
}
#Override
public void setTimeUpdated(Long time) {
timeUpdated=time;
}
}
My query code is as follows:
Query<PersonnelInfo> q = ofy.query(PersonnelInfo.class).ancestor(tmp).filter("timeUpdated >", lastSync);
I am getting the "no matching index found" error everytime. The query works fine without the filter. Some of the entities are missing the "timeUpdated" field because I changed the schema. There are some entities that have been created after the schema change with timeUpdated values and they are not returned. Also I can do a GQL query on the datastore viewer like this:
select * where FROM PersonnelInfo timeUpdated > 0
and I am returned entities, which makes me believe the index is created. What am I doing wrong here? Any help would be appreciated!

You need a multi-property index defined in your datastore-indexes.xml. It should be added to datastore-indexes-auto.xml when you run the query in dev mode, but the result should look like this:
<datastore-index kind="PersonnelInfo" ancestor="true">
<property name="timeUpdated" direction="asc"/>
</datastore-index>

Related

Hibernate filters are not working for APIs returning single result

I have added Hibernate filters on my entities . These filters are applied on queries which fetch Collection of entity but not applied on queries which fetch single entity. Below is my code.
AOrganization.java
#MappedSuperclass
#FilterDef(name = "OrgFilter", parameters = { #ParamDef(name = "allowedOrgIdList", type = "long") })
#Filter(name = "OrgFilter", condition = "org_id in (:allowedOrgIdList)")
public class AOrganization implements Serializable {
#ManyToOne()
#JoinColumn(name = "org_id", nullable = true)
private Organization organization;
public Organization getOrganization() {
return organization;
}
public void setOrganization(Organization organization) {
this.organization = organization;
}
}
Site.java
#Data
#Entity
#Table(name = "site")
public class Site extends AOrganization{
private static final long serialVersionUID = 1L;
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
#Column(name = "id")
private long id;
#Column(name = "site_name")
private String siteName;
#Override
public String toString() {
return "Site [id=" + id + ", siteName=" + siteName + "]";
}
}
SiteService.java
public interface SiteService {
public List<Site> getAllSites();
public List<Site> getSiteBySiteName(String siteName);
public Site updateSiteName(Long id, String siteName);
}
SiteRepository.java
#Repository
public interface SiteRepository extends AOrganizationRepository<Site, Long> {
public List<Site> findBySiteName(String siteName);
public List<Site> findByOrganization_Id(Long orgId);
}
AOrganizationRepository.java
#NoRepositoryBean
public interface AOrganizationRepository<T, ID extends java.io.Serializable> extends CrudRepository<T, ID> {
}
SiteServiceImpl.java
#Service
public class SiteServiceImpl implements SiteService {
#Autowired
private EntityManager entityManager;
#Autowired
private SiteRepository siteRepository;
#Override
public List<Site> getAllSites() {
Iterable<Site> sites = siteRepository.findAll();
List<Site> allSites = new ArrayList<>();
sites.forEach(allSites::add);
return allSites;
}
#Override
public List<Site> getSiteBySiteName(String siteName) {
List<Site> allSites = siteRepository.findBySiteName(siteName);
return allSites;
}
#Override
public Site updateSiteName(Long id,String siteName) {
Site site = siteRepository.findById(id).get();
if(site == null)
return null;
site.setSiteName(siteName);
siteRepository.save(site);
return site;
}
}
AOrganizationAspect.java
#Aspect
#Component
#Slf4j
public class AOrganizationAspect {
#PersistenceContext
private EntityManager entityManager;
#Pointcut("execution(public * com.harshal.springboot.springfilter.repository.AOrganizationRepository+.*(..))")
protected void aOrganizationRepositoryRepositoryMethod() {
log.info("aOrganizationRepositoryRepositoryMethod");
}
#Around(value = "aOrganizationRepositoryRepositoryMethod()")
public Object enableOwnerFilter(ProceedingJoinPoint joinPoint) throws Throwable {
// Variable holding the session
Session session = null;
try {
// Get the Session from the entityManager in current persistence context
session = entityManager.unwrap(Session.class);
// Enable the filter
Filter filter = session.enableFilter("OrgFilter");
// Set the parameter from the session
List<Long> orgList = getAllowedOrgIdList();
filter.setParameterList("allowedOrgIdList", orgList);
} catch (Exception ex) {
// Log the error
log.error("Error enabling OrgFilter : Reason -" + ex.getMessage());
}
// Proceed with the joint point
Object obj = joinPoint.proceed();
// If session was available
if (session != null) {
// Disable the filter
session.disableFilter("OrgFilter");
}
// Return
return obj;
}
private List<Long> getAllowedOrgIdList() {
return Arrays.asList(2l);
}
}
So , hibernate filters are applied if method getSiteBySiteName is called and filters are not applied if findById method is called.
Below are queries :
For getSiteBySiteName :
select site0_.id as id1_2_, site0_.org_id as org_id3_2_,
site0_.site_name as site_nam2_2_ from site site0_ where site0_.org_id
in (?) and site0_.site_name=?
Please help . Thanks in advance.
For findById
select site0_.id as id1_2_0_, site0_.org_id as org_id3_2_0_,
site0_.site_name as site_nam2_2_0_, organizati1_.id as id1_1_1_,
organizati1_.address as address2_1_1_, organizati1_.org_name as
org_name3_1_1_ from site site0_ left outer join organization
organizati1_ on site0_.org_id=organizati1_.id where site0_.id=?
findById is using the EntityManager.find method and do not create a query.
Plus Hibernate Filters only work on queries.
You should write a query instead of using findById

camel jpa #Consumed is not being called

I'm trying to use #Consumed on jpa entity with camel.
this is my route :
<route id="incomingFileHandlerRoute">
<from
uri="jpa://com.menora.inbal.incomingFileHandler.Jpa.model.MessageToDB?consumer.nativeQuery=select
* from file_message where mstatus = 'TODO'&consumer.delay=5000&consumeDelete=false&consumeLockEntity=true&consumer.SkipLockedEntity=true" />
<to uri="bean:incomingFileHandler" />
</route>
and my entity:
#Entity
#Table(name = "file_message")
public class MessageToDB implements Serializable {
/**
*
*/
private static final long serialVersionUID = 1L;
#Id
private String uuid;
private String fileName;
private String body;
private String mstatus;
#Temporal(TemporalType.TIMESTAMP)
private Date mtimestamp;
#Consumed
public void updateMstatus() {
setMstatus(MessageStatus.DONE.name());
}
public String getUuid() {
return uuid;
}
public void setUuid(String uuid) {
this.uuid = uuid;
}
public String getFileName() {
return fileName;
}
public void setFileName(String fileName) {
this.fileName = fileName;
}
public String getBody() {
return body;
}
public void setBody(String body) {
this.body = body;
}
public String getMstatus() {
return mstatus;
}
public void setMstatus(String mstatus) {
this.mstatus = mstatus;
}
public Date getMtimestamp() {
return mtimestamp;
}
public void setMtimestamp(Date mtimestamp) {
this.mtimestamp = mtimestamp;
}
}
I do get to incomingFileHandler bean with results from db but I do not get to the Consumed method updateMstatus . The incomingFileHandler bean is getting called continuously as always there are results from db
I have a similar implementation with camel-jpa and annotations #Consumed and #PreConsumed in the entity but none of these methods is called.
I look the camel-jpa source code and found this in JpaConsumer.java:
protected DeleteHandler<Object> createPreDeleteHandler() {
// Look for #PreConsumed to allow custom callback before the Entity has been consumed
final Class<?> entityType = getEndpoint().getEntityType();
if (entityType != null) {
// Inspect the method(s) annotated with #PreConsumed
if entityType is null the entity class inst inspect the method annotated with #Consumed and #PreConsumed.
Solution: add entityType=com.xx.yy.MessageToDB to your URI to set Endpoint Entity type.

Too many entity_writes when creating multiple entities in cloud datastore

I am noticing a weird(?) behavior when I am creating a large number of entities (about 1000) in my datastore.
I have a REST method which creates 1000 entities of same kind. I am using
datastore.put(entityObject);
to create entities. When I check the trace timeline I see this:
If I check details of every subsequent put call it shows this:
And there are a ton of these put calls! Even the 'Insights' tab says:
Here is my entity class:
import com.googlecode.objectify.Key;
import com.googlecode.objectify.annotation.Entity;
import com.googlecode.objectify.annotation.Id;
import com.googlecode.objectify.annotation.Index;
import com.sm.task.entity.Task;
#Entity(name = "TMS.Task")
public class TaskEntity {
#Id
private String id;
private String title;
private String description;
#Index
private String status;
public Task toTask() {
return new Task(
id,
title,
description,
status);
}
public Key<TaskEntity> key() {
return Key.create(TaskEntity.class, id);
}
public static Key<TaskEntity> keyFor(String id) {
return Key.create(TaskEntity.class, id);
}
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public String getTitle() {
return title;
}
public void setTitle(String title) {
this.title = title;
}
public String getDescription() {
return description;
}
public void setDescription(String description) {
this.description = description;
}
public String getStatus() {
return status;
}
public void setStatus(String status) {
this.status = status;
}
}
I didn't find any setting which causes or controls this behavior. Can anyone please help me understand the reason of this issue and how to get rid of it?
Instead of making 1,000 putcalls to store 1,000 entities, you should make 2 put calls with 500 entities in each (just pass a list of entities to be saved). This is way faster and more efficient.

Unable to retrieve data from SQLite database because of table's name.(Hibernate)

I have a tabe in my database with name "Group".
This is my model code:
#Entity
#Table(name="Group")
public class Group {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
#Column(name="group_id")
private int id;
#Column(name="name")
private String name;
#Column(name="color")
private String color;
public Group(){
this(null,null);
}
public Group(String name, String color){
this.name=name;
this.color=color;
}
public int getId() {
return id;
}
public void setId(int id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getColor() {
return color;
}
public void setColor(String color) {
this.color = color;
}
}
I'm trying to retrieve data from database using:
Session session=mainApp.getSessionFactory().openSession();
groupList=FXCollections.observableArrayList(session.createCriteria(Group.class).list());
groupPicker.setItems(groupList);
session.close();
But it gives me an error(SQL error or missing database (near "Group": syntax error)). I know that it is happening because hibernate generates query similar to this: SELECT * FROM Group. To work properly it should be: SELECT * FROM "Group", but I do not know how to achieve this.
After I tried several different ways, I finally came up with a solution, which works. I just changed my model's table name to:#Table(name="\"Group\"").
If You are using XML somehow You can be interested in element delimited-identifiers to force quoting of all entities. More can be found Hibernate, MySQL and table named "Repeat" - strange behaviour

JAVA Google App Engine + Facebook API + GSON = Trouble with Javabean

I am trying to get the user's friends list from Facebook.
The problem seems to be the Javabean...
FBUser fbuser = new Gson().fromJson(jsonStr, FBUser.class);
public class FBUser implements Serializable {
private static final long serialVersionUID = -3154429420153433117L;
private String id;
private String name;
private String email;
private Friends friendsList = new Friends();
private FBUser() { }
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getEmail() {
return email;
}
public void setEmail(String email) {
this.email = email;
}
public List<Data> getFriendsList() {
return friendsList.getData();
}
public static class Friends implements Serializable {
private static final long serialVersionUID = 6991758772193514527L;
private List<Data> data;
private Friends() { }
public List<Data> getData() {
return data;
}
public void setData(List<Data> data) {
this.data = data;
}
public class Paging implements Serializable {
private static final long serialVersionUID = 1689816298710621080L;
private String next;
private Paging() { }
public String getNext() {
return next;
}
public void setNext(String next) {
this.next = next;
}
}
}
public class Data implements Serializable {
private static final long serialVersionUID = -5008541658519841090L;
private String id;
private String name;
private Data() { }
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
}
}
Json:
json: {"id":"10861234","name":"Whatever","email":"whatever\u0040gmail.com","friends":{"data":[{"name":"Someone","id":"10861234"},{"name" ...43"}],"paging":{"next":"https:\/\/graph.facebook.com\/10861234\/friends..."}}}
The fields ID, Name and Email I can retrieve succesfully... but the friendsList is null... =(
Maybe it is the way I am trying to get it from the nested class, any suggestions on that?
There is no friendsList in your JSON (or, there's no friends in your Java class - whichever way you'd like to look at it). Gson silently ignores anything in the JSON that is not present in your classes.
You have a field friends whose value is an object. That object has a field data which is an array of objects and a field paging which is another object.
You need to write Java classes that match that structure. You're ... close.
In your FBUser class change:
private Friends friendsList = new Friends();
to:
private Friends friends = new Friends();
or:
#SerializedName("friends")
private Friends friendsList = new Friends();
Then in your Friends class you need to add:
private Paging paging = new Paging();
Also note that you don't have to initialize these values unless you specifically don't want them to be non-null when using these classes elsewhere.

Resources