Using GAE, I am using javax's entity manager (javax.persistence.EntityManagerFactory) to generate an instance of the entity manager:
private static final EntityManagerFactory emfInstance = Persistence.createEntityManagerFactory("transactions-optional");
I retrieve from the datastore using the following code:
event = mgr.find(Event.class, id);
The problem I have is that if I retrieve the data for the first time, everything goes fine. However, if I go through the "Datastore Viewer" in the GAE dashboard to edit the values manually. The next time I call the data, using the "find" method, the values returned are the old value. I have to manually upload backend again in order to get the new values.
Any idea what is causing this? I would like mgr.find to always call the latest value. Thanks.
The entity is being cached. When you change it through the Datastore Viewer, the entity cached by your backend is not affected.
After you make a change in the Datastore viewer, click on the "Flush Memcache" button.
If this does not help, you may need to change configuration for your caching:
Level2 Caching is enabled by default. To get the previous default
behavior, set the persistence property datanucleus.cache.level2.type
to none. (Alternatively include the datanucleus-cache plugin in the
classpath, and set the persistence property
datanucleus.cache.level2.type to javax.cache to use Memcache for L2
caching.
Try flushing memcache and then try your query again.Most times the last persisted entity data is what's retrieved until you do this.
Related
My app is GAE standard.
I can edit an entity in the cloud console from a URL that starts with this:
https://console.cloud.google.com/datastore/entities/query
I'll save the entity and refresh the page and the new data is shown.
On my website, I then go to a page that retrieves the entity using the entity ID like this
e = Entity.get_by_id(1234)
But my website shows the old data! It seems like this shouldn't be possible.
My only solution is to then use remote api shell to get the entity (which shows the old data) and then update and then put the entity. My website then shows the new data.
How is it possible that updating an entity in the cloud console doesn't show up in production when getting an entity by its id?
Try something like this:
e = Entity.get_by_id(1234, use_cache=False, use_memcache=False)
or
the_key = ndb.Key(Entity, 12345)
the_key.get(use_cache=False, use_memcache=False)
There are other params you can set, like memcache_timeout and read_policy:
https://cloud.google.com/appengine/docs/standard/python/ndb/functions#context_options
This sounds like a caching problem - as the warning at the top of this page notes, updating entities via the datastore console doesn't update or flush the cache. You should try flushing the memcache via the console after you make an update, which will ensure that the query hits the datastore directly.
Trying to view items in memcache in GAE.
https://code.google.com/p/googleappengine/issues/detail?id=7245
http://googleappengine.blogspot.com/2012/08/app-engine-171-released.html
These links say "Namespaces in the Memcache Viewer - The Admin Console now supports retrieving memcache values that are stored in a non-default namespace."
But I still couldn't the view item trying different combinations.
tried, ns.key, ns_key, key (where ns is namespace, key is key set in the program).
I use Development SDK 1.7.6, with GO runtime. What should I enter it in the search space as seen in the picture, or what is the best way?
This is not currently implemented in the development server (but is available in the production admin console as pointer by Jesse in the comment).
You should open a new feature request in dev_appserver2 projects.
If you feel like contributing it yourself, you could easily add support by patching:
memcache_viewer.html to add a new <input> field to capture the namespace.
memcache_viewer.py to add namespace arg to memcache.get call
From the tutorial, which I confirmed by creating a simple project, the index.yaml file is auto-generated when a query is run. What I further observe is that until then the admin console (http://localhost:8080/_ah/admin/datastore) does not show the data-store.
My problem is this: I have a project for which data/entities are to be added manually through the datastore admin console. The website is only used to display/retrieve data, not to add data to the data-store.
How do I get my data-store to appear on the console so I can add data?
Yes, try retrieving from the empty data-store through the browser just so I can get the index.yaml to populate, etc. But that does not work.
The easiest way is probably just to create a small python script inside your project folder and create your entities in script. Assign it to a URL handler that you'll use once, then disable.
You can even do it from the python shell. It's very useful for debugging, but you'll need to set it up once.
http://alex.cloudware.it/2012/02/your-app-engine-app-in-python-shell.html
In order to do the same on production, use the remote_api:
https://developers.google.com/appengine/articles/remote_api
This is a very strange question.
The automatic creation of index.yaml only happens locally, and is simply to help you create that file and upload it to AppEngine. There is no automatic creation or update of that file once it's on the server: and as the documentation explains, no queries can be run unless the relevant index already exists in index.yaml.
Since you need indexes to run queries, you must create that file locally - either manually, or by running the relevant queries against your development datastore - then upload it along with your app.
However, this has nothing at all to do with whether the datastore viewer appears in the admin. Online, it will always show, but only entity kinds that actually have an instance in the store will be shown. The datastore viewer knows nothing about your models, it only knows about kinds that exist in the datastore.
On your development server you can use the interactive console to create/instantiate/save an entity, which should cause the entity class to appear in the datastore interface, like so:
from google.appengine.ext import ndb
class YourEntityModel(ndb.Model):
pass
YourEntityModel().put()
I have a grails application. I'd like to load data into the underlying database with something external to grails, perl, specifically. I know I have to update the hibernate sequence after external data loading, otherwise on the next create object in grails, hibernate throws an exception; but is there anything else I need to update? Do I have to clear the hibernate cache, for instance? This would seem to be a very common issue, but there's no discussion of it in the grails docs. Thanks.
Found this
http://grails.1312388.n4.nabble.com/Accessing-the-2nd-level-cache-to-allow-it-to-be-cleared-via-a-controller-or-service-td1390985.html
Hibernate has APIs for this. You can get the query cache via sessionFactory.getQueryCache() and clear it using
sessionFactory.queryCache.clear()
You can access a cache for a domain classes using its full class name, e.g.
def cache = sessionFactory.getSecondLevelCacheRegion('com.foo.bar.Book')
and clear it via
sessionFactory.getSecondLevelCacheRegion('com.foo.bar.Book').clear()
You can also call evict() on the sessionFactory for an entire class
sessionFactory.evict(com.foo.bar.Book)
or for an individual instance
sessionFactory.evict(com.foo.bar.Book, 42)
I am setting cache properly as my project does grab the correct result from it. I want to view it in the dev console memcache viewer (http://localhost:8080/_ah/amdin/memcache)
But when I paste in the key, it says "no such key". The memcache statement I use is:
stuff=memcache.get(userid, namespace='book')
So when I use the userid (aka key) in the memcache viewer, it fails. Am I to somehow add the namespace? I tried that and that failed. I also tried the entity key, that was a fail. any ideas?!?
To the best of my knowledge you can't retrieve a namespaced memcache key using the dev console memcache viewer.
You have more than an option though:
a. Use the interactive console with a code like this:
from google.appengine.api import memcache
print memcache.get(youruserid, namespace='book')
b. Create an admin section (Webapp+memcache api) to your web application adding this feature