One of my entities has the following declaration for its ID:
#Id
private String oInstID;
public String getInstID(){return oInstID;}
public void initID(){
oInstID = OfyController.makeID(Partner.class, null);
}
Keep in mind that I have same declaration for my other entities as well.
I have the following testing statements after the ofy.save():
Sticky persisted = OfyController.ofy().load().type(Sticky.class).first().now();
String id = persisted.getInstID();
Sticky queried = OfyController.ofy().load().type(Sticky.class).id(id).now();
Sticky queried2 = OfyController.ofy().load().entity(persisted).now();
The persisted returned the entity
The id returned the ID of the entity
The queried returned null...which is what my problem is.
The queried2 returned the same entity as the persisted
Any idea why queried returned null?
Thanks!
My prior experience with Objectify is tiny and very stale, but what you're describing is consistent with eventual consistency. There's a bit of useful info in Storing Data with Objectify and Datastore.
Related
I am using GAE for my server where I have all my entities in Datastore. One of the entity has more than 2000 records, and it is taking almost 30 secs to read whole entity. So I wanted to use cache to improve performance.
I have tried Datastore objectify #cache annotation, but not finding
how to read from the stored cache. I have declared entity as below:
#Entity
#Cache
public class Devices{
}
Second thing I tried is memcache. I am storing whole List s
in key, but this is not storing, I couldn't see in console memcache,
but at the same time not showing any errors or exceptions while
storing objects.
putvalue("temp", List<Devices>)
public void putValue(String key, Object value) {
Cache cache = getCache();
logger.info(TAG + "getCache() :: storing memcache for key : " + key);
try {
if (cache != null) {
cache.put(key, value);
}
}catch (Exception e) {
logger.info(TAG + "getCache() :: exception : " + e);
}
}
When I tried to retrieve using getValue("temp"), it is returning
null or empty.
Object object = cache.get(key);
My main object is to limit the time to 5secs to get all the records of entity.
Can anyone suggest what I am doing wrong here? Or any better solution to retrieve the records fast from Datastore.
Datastore Objectify actually uses the App Engine Memcache service to cache your entity data globally when you use the #Cache annotation. However, as explained in the doc here, only get-by-key, save(), and delete() interact with the cache. Query operations are not cached.
Regarding the App Engine Memcache method, you may be hitting the limit for the maximum size of a cached data value which is 1 MiB, although I believe this raise an exception indeed.
Regarding the query itself, you may be better off using a keys_only query and then doing a key.get() on each returned key. That way, Memcache will be used for each record.
I want to test if an object exists in the datastore. I know its key. I am doing this right now by loading the entire object:
public boolean doesObjectExist(String knownFooId) {
Key<Foo> key = Key.create(Foo.class, knownFooId);
Foo foo = ofy().load().key(key).now();
if (foo != null) {
// yes it exists.
return true;
}
return false;
}
That must cost 1 read operation from the datastore. Is there a way to do it without having to load the entire object, that might be cheaper? In other words, a way that it would only cost 1 "small" operation?
Thanks
There's no way to do it cheaper.
Even if you just do a keys only query, the query is 1 Read operation + 1 Small operation per key fetched. (https://cloud.google.com/appengine/pricing#costs-for-datastore-calls)
Keep doing a get by key, which is just 1 Read.
public boolean doesObjectExist(String knownFooId) {
Key<Foo> fooKey = Key.create(Foo.class, knownFooId);
Key<Foo> datastoreKey = ofy().load().type(Foo.class).filterKey(fooKey).keys().first().now();
return datastoreKey.equals(fooKey);
}
From documentation:
QueryKeys keys()
Switches to a keys-only query. Keys-only responses are billed as "minor datastore operations" which are faster and free compared to fetching whole entities.
You could try to fetch the key, as far as I understand it'd be only a small operation.
// You can query for just keys, which will return Key objects much more efficiently than fetching whole objects
Iterable<Key<F>> allKeys = ofy().load().type(Foo.class).filter("id", knownFooId).keys();
It should work. Also take a look at the objectfy docs: https://github.com/objectify/objectify/wiki/Queries
i have a simple question
in the objectify documentation it says that "Only get(), put(), and delete() interact with the cache. query() is not cached"
http://code.google.com/p/objectify-appengine/wiki/IntroductionToObjectify#Global_Cache.
what i'm wondering - if you have one root entity (i did not use #Parent due to all the scalability issues that it seems to have) that all the other entities have a Key to, and you do a query such as
ofy.query(ChildEntity.class).filter("rootEntity", rootEntity).list()
is this completely bypassing the cache?
If this is the case, is there an efficient caching way to do a query on conditions - or for that matter can you cache a query with a parent where you would have to make an actual ancestor query like the following
Key<Parent> rootKey = ObjectifyService.factory().getKey(root)
ofy.query(ChildEntity.class).ancestor(rootKey)
Thank you
as to one of the comments below i've added an edit
sample dao (ignore the validate method - it just does some null & quantity checks):
this is a sample find all method inside a delegate called from the DAO that the request factory ServiceLocator is using
public List<EquipmentCheckin> findAll(Subject subject, Objectify ofy, Event event) {
final Business business = (Business) subject.getSession().getAttribute(BUSINESS_ATTRIBUTE);
final List<EquipmentCheckin> checkins = ofy.query(EquipmentCheckin.class).filter(BUSINESS_ATTRIBUTE, business)
.filter(EVENT_CONDITION, event).list();
return validate(ofy, checkins);
}
now, when this is executed i find that the following method is actually being called in my AbstractDAO.
/**
*
* #param id
* #return
*/
public T find(Long id) {
System.out.println("finding " + clazz.getSimpleName() + " id = " + id);
return ObjectifyService.begin().find(clazz, id);
}
Yes, all queries bypass Objectify's integrated memcache and fetch results directly from the datastore. The datastore provides the (increasingly sophisticated) query engine that understands how to return results; determining cache invalidation for query results is pretty much impossible from the client side.
On the other hand, Objectify4 does offer a hybrid query cache whereby queries are automagically converted to a keys-only query followed by a batch get. The keys-only query still requires the datastore, but any entity instances are pulled from (and populate on miss) memcache. It might save you money.
I have a Spring application which uses JPA (Hibernate) initially created with Spring Roo. I need to store Strings with arbitrary length, so for that reason I've annotated the field with #Lob:
public class MyEntity{
#NotNull
#Size(min = 2)
#Lob
private String message;
...
}
The application works ok in localhost but I've deployed it to an external server and it a problem with encoding has appeared. For that reason I'd like to check if the data stored in the PostgreSQL database is ok or not. The application creates/updates the tables automatically. And for that field (message) it has created a column of type:
text NOT NULL
The problem is that after storing data if I browse the table or just do a SELECT of that column I can't see the text but numbers. Those numbers seems to be identifiers to "somewhere" where that information is stored.
Can anyone tell me exactly what are these identifiers and if there is any way of being able to see the stored data in a #Lob columm from a pgAdmin or a select clause?
Is there any better way to store Strings of arbitrary length in JPA?
Thanks.
I would recommend skipping the '#Lob' annotation and use columnDefinition like this:
#Column(columnDefinition="TEXT")
see if that helps viewing the data while browsing the database itself.
Use the #LOB definition, it is correct. The table is storing an OID to the catalogs -> postegreSQL-> tables -> pg_largeobject table.
The binary data is stored here efficiently and JPA will correctly get the data out and store it for you with this as an implementation detail.
Old question, but here is what I found when I encountered this:
http://www.solewing.org/blog/2015/08/hibernate-postgresql-and-lob-string/
Relevant parts below.
#Entity
#Table(name = "note")
#Access(AccessType.FIELD)
class NoteEntity {
#Id
private Long id;
#Lob
#Column(name = "note_text")
private String noteText;
public NoteEntity() { }
public NoteEntity(String noteText) { this.noteText = noteText }
}
The Hibernate PostgreSQL9Dialect stores #Lob String attribute values by explicitly creating a large object instance, and then storing the UID of the object in the column associated with attribute.
Obviously, the text of our notes isn’t really in the column. So where is it? The answer is that Hibernate explicitly created a large object for each note, and stored the UID of the object in the column. If we use some PostgreSQL large object functions, we can retrieve the text itself.
Use this to query:
SELECT id,
convert_from(loread(
lo_open(note_text::int, x'40000'::int), x'40000'::int), 'UTF-8')
AS note_text
FROM note
I have an appengine app which has been running for about a year now, i have mainly been using JDO queries until now, but i am trying to collect stats and the queries are taking too long. I have the following entity (Device)
public class Device implements Serializable{
...
#Persistent
private Set<Key> feeds;// Key for the Feed entity
...
}
So I want to get a count of how many Devices have a certain Feed. I was doing it in JDOQL before as such (uses javax.jdo.Query):
Query query = pm.newQuery("select from Device where feeds.contains(:feedKey)");
Map<String, Object> paramsf = new HashMap<String, Object>();
paramsf.put("feedKey",feed.getId());
List<Device> results = (List<Device>) query.executeWithMap(paramsf);
Though this code times out now. I was trying to use the Datastore API so I could set chunk size,etc to see if i could speed the query up or use a cursor, but I am unsure how to search in a Set field. I was trying this (uses com.google.appengine.api.datastore.Query)
Query query = new Query("Device");
query.addFilter("feeds", FilterOperator.IN, feed.getId());
query.setKeysOnly();
final FetchOptions fetchOptions = FetchOptions.Builder.withPrefetchSize(100).chunkSize(100);
QueryResultList<Entity> results = dss.prepare(query).asQueryResultList(fetchOptions);
Essentially i am unsure how to search in the one-to-many filed (feeds) for a single key. Is it possible to index it somehow?
hope it makes sense....
Lists (and other things that are implemented as lists, like sets) are indexed individually. As a result, you can simply use an equality filter in your query, the same as if you were filtering on a single value rather than a list. A record will be returned if any of the items in the list match.