I added a new index to my GAE app, but I need to index old entities. Is there any way besides getting and reinserting to create index entries for existing entities?
No, you have to re-save all old entities. The Datastore only updates indexes when an entity is saved.
Google provides tools - gcd for Java and appcfg.py for Python/Go - that can add and remove indexes on your production datastore.
Related
We have a java application and we deployed this on Google App engine. We created around 150 indexes in datastore and which are running fine in production.
but somehow we missed indexes information in datastore-indexes-auto.xml and there is no any file with name datastore-indexes.xml.
Now we want to have datastore-indexes.xml / datastore-indexes-auto.xml with all existing indexes which serving in production now.
How can we do this? I checked appcfg/gcloud commands, there is no any command to import/download the indexes file from app engine application.
Thanks
You could download your deployed app code (How do I download a specific service's source code off of AppEngine?) or check it in StackDriver (similar to Google Cloud DataStore automatic indexing, but looking for the java-specific file(s) instead of index.yaml) and copy/paste the index configs from there.
Place those configs into your app's version-controlled datastore-indexes.xml file (create it if needed) - these will be the manually-maintained indexes. The development server will continue maintain the missing ones automatically in datastore-indexes-auto.xml. The datastore will combine the info from the 2 files at deployment time.
Note that the Datastore indexes are cumulative, they're not automatically deleted if fewer of them are present in newer versions of the xml files, they have to me manually vacuumed/deleted. So check that the index configs recovered with the above method(s) are indeed all of those displayed in the Datastore Indexes page, any missing ones would have to be reconstructed manually from that page info.
As of gcloud 211.0.0 you can list your composite indexes with gcloud beta datastore indexes list
I have a couple of Google App Engine versions in a project, and I use its Datastore. Each version creates a namespace on Datastore, where I have created some entities within a couple of kinds.
Now I've deleted some of the versions, and I want to delete their stuff on Datastore. I've deleted the entities, then the referred kinds had disappeared, but I can't find a way to delete the namespaces.
How can I delete a namespace in Google Cloud Datastore?
Each namespace stores a set of system kinds eg. statistics: https://cloud.google.com/datastore/docs/concepts/stats
Those are reculculated every 48h, and deleted if there are no more user data in that namespace. At that time namespace should disappear from your Datastore as well.
How can I use as NOT LIKE SQL operator in Python and Google App Engine data store? I want to filter string in database.
q = Post.all()
q.filter("text NOT LIKE", "%Something%")
This feature is not supported by AppEngine Datastore.
The datastore can query entities just by indexes (and get by ID and SELECT without any filters/order are basically queries over index on key).
This is an architectural limitation of the distributed data storage.
There are solution which can do what you want but Google Cloud Datastore is not one of them.
I accidentally deleted a bunch of entities earlier today in Google App Engine and found out later that some of them should not have been deleted. I have the keys for the entities I deleted as well as a lot of information about the entities stored in a secondary database. I was wondering if there was any way to create entities with specific keys in app engine or if there was alternative method of recovering these entities? I am very worried
I was wondering if there was any way to create entities with specific keys in app engine
You can create new entities with those same keys, just set the key property on the instance of the model before you put and it will use that key instead of generating a random one for you.
Also, check if your team has used datastore admin to take snapshots, if so, you may be able to recover them from there:
https://console.cloud.google.com/datastore/settings
So if I have a class of which I have a number saved in a datastore, and then seek to add a field later, how can I prevent all my previous objects breaking? Is there a way to retroactively set those fields so they're not null?
I'm using JDO.
It depends on how you are accessing the datastore. Which runtime (python/java) and which API are you using to access the datastore? The datastore itself is schemaless, so it dosen't care what is or isn't in a certain entity. On the Java side, if you use the low level datastore API, you wouldn't have any problems accessing the "old" entities and adding in the data you want to. However, if you are using JDO or JPA to access the datastore, you might get errors accessing the entities with missing data.