I have a local Datastore emulator instance running on my local machine (Mac OSX 10.11.6).
I have a bunch of data in there already, but I want to remove all data from the local database to start with a clean slate.
However I have already tried deleting the directory (where I thought) the data was being stored.
This message appears when starting the emulator.
WARNING: Reusing existing data in [/Users/myusername/.config/gcloud/emulators/datastore].
I deleted the entire datastore directory and it seems to be re-generated properly when the emulator is restarted. However, the old data is still there.
rm -rf /Users/myusername/.config/gcloud/emulators/datastore
I also confirmed that my Node App is connecting to the local emulator.
myusername$ $(gcloud beta emulators datastore env-init)
myusername$ gcloud beta emulators datastore env-init
export DATASTORE_DATASET=projectname
export DATASTORE_EMULATOR_HOST=localhost:8081
export DATASTORE_EMULATOR_HOST_PATH=localhost:8081/datastore
export DATASTORE_HOST=http://localhost:8081
export DATASTORE_PROJECT_ID=projectname
Datastore emulator supports resetting datastore when its running. This can be used to reset datastore when running tests.
Make POST request to http://localhost:8081/reset to clear the internal state of the emulator i.e
curl -X POST http://<emulator-host>:<port>/reset
see https://github.com/GoogleCloudPlatform/google-cloud-java/issues/1292 discussion
Stop the local dev server from running
Find its dev_appserver.datastore file and drag it to the trash
Restart dev server
A new, empty local datastore will be created.
Related
I'm running a local instance of a PHP app-engine project, I have got some buckets setup in GCP specifically for the local dev version, however instead of the data that I write to the bucket appearing online, they are being saved locally into the dev_appserver Datastore. I can see the files in the local admin interface (localhost:8000) under Datastore.
This is an issue as the application I'm developing has a companion app which needs to also access the bucket files.
The
--support_datastore_emulator=[true|false]
flag is documented under
dev_appserver.py -h
But it doesn't seem to have any effect when using =false.
So my question is: How do I stop the dev_appserver from using the local Datastore and make it use the 'real' buckets on the web?
Try setting the --default_gcs_bucket_name flag documented here to establish the default GCS bucket to use:
dev_appserver.py app.yaml --default_gcs_bucket_name gs://BUCKET-NAME
Apologies for the seemingly obvious question, but I figure the answer might help others. I can't for the life of me find documentation on the filepath within the Google App Engine VM (Cloud Shell) where I can find the static files being served from. I need to pull the latest upstream changes from a private github repo.
Note that I navigated elsewhere in the VM and even restarting the session didn't put me in a default project root path within the VM as I expected it to.
There are several issues to address here:
The Cloud Shell is a virtual shell
Google Cloud Shell is an interactive shell environment for Google
Cloud Platform.
The environment where you're working is a container running in a VM in a Google-owned project inside GCP.
You can verify this by checking the metadata server (only available for GCP VMs):
curl -H 'Metadata-Flavor:Google' "http://metadata.google.internal/computeMetadata/v1/?recursive=true&alt=text"
In the metadata provided you'll see how this container is created and configured.
The Cloud Shell is tied to the user, so you'll always access the same environment if you access it with the same credentials, no matter the project. However, if you access with a different user, you'll get a different environment.
You can't access GAE standard instances
GAE is a fully managed environment, and you won't be able to access it. In this way, you won't be able to find the root of the running app engine project.
However, by the way GAE deploys your code, it uses a staging bucket to gather the code before compiling. You can find your staging bucket through the App Engine Admin API. This is usually staging.<PROJECT_ID>.appspot.com, although you can change this configuration. You can get your files from there.
You can access GAE flex apps
However, the deployment in flex gets your files, build a Docker container with them, and then deploys this container inside a VM.
As per the docs, you can connect directly to your container by running:
gcloud app instances ssh [INSTANCE-NAME] --service [SERVICE] --version [VERSION]
docker exec -it gaeapp /bin/bash
Regarding your issue
According what you say in the comments of the question, your issue could come from a myriad of places. From changing the shell you're connecting to, to resetting your shell environment (deleting all the files), to a thousand different possible problems.
The best way to think about it is regard the Cloud Shell as a temporal environment to run commands, but not as a virtual machine.
Knowing that, you could mount a persistent filesystem (GCS through GCSFuse, Cloud Filestore, ...) to persist your work, or simply use Git to have your work always synced on a repo.
GAE Flex has some nice CI integrations, so that's a plus for going the Git route.
I am having the problem with Datastore Viewer when working with emulator. http://localhost:8990/_ah/admin/datastore? displays "Datastore has no entities in the Empty namespace. You need to add data programatically before you can use this tool to view and edit it" but when I look into ...\WEB-INF\appengine-generated\local_db.bin file I see it was updated and the entities are there. So for some reasons console doesn't see them. I am using SDK 97.0.0 but the console shows SDK v1.9.27
Check the location of your storage.bin when running the datastore emulator. Then use that storage path when running the dev_appserver.py:
dev_appserver.py test.yaml --storage_path=/path/.config/gcloud/emulators/datastore/WEB-INF/appengine-generated/local_db.bin
Also make sure to connect your appserver to the datastore emulator port.
How to clear datastore in GoogleAppEngine.
I want to clear my development data to do a test again, but I can found a way to do that
If you are running from the commandline, use the --clear_datastore flag, e.g.,
dev_appserver.py --clear_datastore=yes app
Otherwise, if you're running it off the included GAE launcher, go into the settings of your app (double click it), and there should be a little checkbox that says "clear datastore on launch" under Launch Settings.
reference:
https://cloud.google.com/appengine/docs/python/tools/devserver#Python_Using_the_Datastore
dev_appserver.py --clear_datastore myapp
assuming by "development data", you mean the data in the dev server.
Simply use Administering Your Datastore (Experimental)
Some Other ways
App Engine: How to "reset" the datastore?
Delete all data for a kind in Google App Engine
Interactive console is also a great way.
For Java, the following information can be seen in Using the Datastore - Clearing the Datastore at the end of the page (as of 2013/05/10):
The development web server uses a local version of the Datastore for testing your application, using local files. The data persists as long as the temporary files exist, and the web server does not reset these files unless you ask it to do so.
The file is named local_db.bin, and it is created in your application's WAR directory, in the WEB-INF/appengine-generated/ directory. To clear the Datastore, delete this file.
So, stop your server, delete the file, and restart it up.
On your local machine you can go to : http://localhost:8080/_ah/admin/datastore
I am developing an app locally in Google App Engine. I have built a small datastore for development purposes. Rebuilding it after every power cycle on my Mac got tedious so I made it permanent. Now I run my app locally with the following command:
/usr/local/bin/dev_appserver.py "--datastore_path=./permanent.datastore" appengine_prototype
Life is good. I have decided to deploy my app so I can test http post commands from a different machine. When I tried to register my current application id (example), I found that it was unavailable (shocker!). So I registered a different application id and planned to change my local application id to match. However, when I changed the
application: *app-id*
line in my app.yaml file, my app stopped recognizing my permanent datastore.
So, how can I change my application id to the one I registered, maintain the connection to the permanent datastore and then push the whole shebang online? I tried running the app twice locally, first with the permanent datastore referenced in the command and then without, hoping that the default temporary datastore would inherit from the previous permanent datastore. That didn't work. Do I need to start by copying the permanent datastore to the default temporary datastore? How would I do that? Any help would be much appreciated.
Thanks,
Dessie
If your intention is to eventually push your local data to your live environment anyway, then your best bet is to:
use bulkloader.py to backup your local data (while using oldid in your config)
then change your config to your newid
then use bulkloader.py to push your data to your new development server (ran with --datastore_path=./permanent.datastore2 or something)
then use bulkloader.py to push your data to the GAE production server
Details of bulkloader.py can be found in the docs and an example here