My app is GAE standard.
I can edit an entity in the cloud console from a URL that starts with this:
https://console.cloud.google.com/datastore/entities/query
I'll save the entity and refresh the page and the new data is shown.
On my website, I then go to a page that retrieves the entity using the entity ID like this
e = Entity.get_by_id(1234)
But my website shows the old data! It seems like this shouldn't be possible.
My only solution is to then use remote api shell to get the entity (which shows the old data) and then update and then put the entity. My website then shows the new data.
How is it possible that updating an entity in the cloud console doesn't show up in production when getting an entity by its id?
Try something like this:
e = Entity.get_by_id(1234, use_cache=False, use_memcache=False)
or
the_key = ndb.Key(Entity, 12345)
the_key.get(use_cache=False, use_memcache=False)
There are other params you can set, like memcache_timeout and read_policy:
https://cloud.google.com/appengine/docs/standard/python/ndb/functions#context_options
This sounds like a caching problem - as the warning at the top of this page notes, updating entities via the datastore console doesn't update or flush the cache. You should try flushing the memcache via the console after you make an update, which will ensure that the query hits the datastore directly.
Related
Sorry for a general question. My situation looks so: i have mongodb database and 2 reactjs pages. In each page i want to fetch a different information from database. Depending by your practice, which is the best way to fetch data from mongodb in a reactjs component?
I would recommend reading up on the MERN stack - tons of guides available online via google and youtube. The gist would be that a typical web application will consist of a few key components. In this case:
1 - (React) The client page rendered to the user
2 - (Node + Express) The server which processes data, allows you to use endpoints to make changes to your application. These endpoints make the necessary database queries. You can use a database client to write these queries as JavaScript within your NodeJS endpoints.
3 - (MongoDB) Your database.
So for instance a typical CRUD app allows you to create, read, update, and delete. Let's say you are looking at making a standard TODO list app.
You would need to make requests to these endpoints to perform these operations.
You could have a POST to /todo which would then insert a new document into your database.
You would need a way to read the information from the page... say a GET request to /todos to read all items. Or also a GET request to /todo/:id to get a specific item.
You would need a way to update an existing item... say a PUT request to /todo/:id with the updates you want to take place.
Finally you would need a way to delete an item... a DELETE request to /todo/:id which would delete the item.
Each of these endpoints would make a request to insert / read / update / delete items from the database, and return content to the client React code --> which then displays it to the user.
Frontend side, in react.js call api data using fetch() method. Pass your Mongodb URI string. If you want data in slot based use limit() and Skip() function for pagination.
Follow MVC pattern where your frontend only calla controller api. And controller calls DAO methods for Mongodb. You can Use Mongodb Stitch for serverless app.sor data leak can be avoided forntend side. Mongodb has connecting pool max.100 so that each time client request Mongodb connection cashed object given from pool to further speed up your connection time.
I'm trying to delete a key from the memcache using the latest GAE console (https://console.developers.google.com/appengine/memcache). I used to be able to do this with the old console but I can't figure out the new one.
Sometimes I manually edit an entity with the GAE console and afterwards I need to delete the entity from the memcache so that my app uses the latest data. I know the entity is in the memcache since the old data is being served, but I can't find it with the memcache viewer.
Here is the new memcache console:
I leave Namespace blank because I don't use them.
I select Python String for the key type since I am using Python.
For the key, I've tried:
Key('Election', 6254893018906624)
aglzfm9wYXZvdGVyFQsSCEVsZWN0aW9uGICAgKSsmY4LDA
NDB9:aglzfm9wYXZvdGVyFQsSCEVsZWN0aW9uGICAgKSsmY4LDA
but none of them work.
Clicking the "Show all keys" link just produces a blank screen.
How do I get this to work?
Not sure for Python, but for Java, with Objectify, I have used this many times:
Namespace: ObjectifyCache
Key type: Java String
Key: aglzfm9wYXZvdGVyFQsSCEVsZWN0aW9uGICAgKSsmY4LDA (from your data)
Using GAE, I am using javax's entity manager (javax.persistence.EntityManagerFactory) to generate an instance of the entity manager:
private static final EntityManagerFactory emfInstance = Persistence.createEntityManagerFactory("transactions-optional");
I retrieve from the datastore using the following code:
event = mgr.find(Event.class, id);
The problem I have is that if I retrieve the data for the first time, everything goes fine. However, if I go through the "Datastore Viewer" in the GAE dashboard to edit the values manually. The next time I call the data, using the "find" method, the values returned are the old value. I have to manually upload backend again in order to get the new values.
Any idea what is causing this? I would like mgr.find to always call the latest value. Thanks.
The entity is being cached. When you change it through the Datastore Viewer, the entity cached by your backend is not affected.
After you make a change in the Datastore viewer, click on the "Flush Memcache" button.
If this does not help, you may need to change configuration for your caching:
Level2 Caching is enabled by default. To get the previous default
behavior, set the persistence property datanucleus.cache.level2.type
to none. (Alternatively include the datanucleus-cache plugin in the
classpath, and set the persistence property
datanucleus.cache.level2.type to javax.cache to use Memcache for L2
caching.
Try flushing memcache and then try your query again.Most times the last persisted entity data is what's retrieved until you do this.
From the tutorial, which I confirmed by creating a simple project, the index.yaml file is auto-generated when a query is run. What I further observe is that until then the admin console (http://localhost:8080/_ah/admin/datastore) does not show the data-store.
My problem is this: I have a project for which data/entities are to be added manually through the datastore admin console. The website is only used to display/retrieve data, not to add data to the data-store.
How do I get my data-store to appear on the console so I can add data?
Yes, try retrieving from the empty data-store through the browser just so I can get the index.yaml to populate, etc. But that does not work.
The easiest way is probably just to create a small python script inside your project folder and create your entities in script. Assign it to a URL handler that you'll use once, then disable.
You can even do it from the python shell. It's very useful for debugging, but you'll need to set it up once.
http://alex.cloudware.it/2012/02/your-app-engine-app-in-python-shell.html
In order to do the same on production, use the remote_api:
https://developers.google.com/appengine/articles/remote_api
This is a very strange question.
The automatic creation of index.yaml only happens locally, and is simply to help you create that file and upload it to AppEngine. There is no automatic creation or update of that file once it's on the server: and as the documentation explains, no queries can be run unless the relevant index already exists in index.yaml.
Since you need indexes to run queries, you must create that file locally - either manually, or by running the relevant queries against your development datastore - then upload it along with your app.
However, this has nothing at all to do with whether the datastore viewer appears in the admin. Online, it will always show, but only entity kinds that actually have an instance in the store will be shown. The datastore viewer knows nothing about your models, it only knows about kinds that exist in the datastore.
On your development server you can use the interactive console to create/instantiate/save an entity, which should cause the entity class to appear in the datastore interface, like so:
from google.appengine.ext import ndb
class YourEntityModel(ndb.Model):
pass
YourEntityModel().put()
I am setting cache properly as my project does grab the correct result from it. I want to view it in the dev console memcache viewer (http://localhost:8080/_ah/amdin/memcache)
But when I paste in the key, it says "no such key". The memcache statement I use is:
stuff=memcache.get(userid, namespace='book')
So when I use the userid (aka key) in the memcache viewer, it fails. Am I to somehow add the namespace? I tried that and that failed. I also tried the entity key, that was a fail. any ideas?!?
To the best of my knowledge you can't retrieve a namespaced memcache key using the dev console memcache viewer.
You have more than an option though:
a. Use the interactive console with a code like this:
from google.appengine.api import memcache
print memcache.get(youruserid, namespace='book')
b. Create an admin section (Webapp+memcache api) to your web application adding this feature