I am having the problem with Datastore Viewer when working with emulator. http://localhost:8990/_ah/admin/datastore? displays "Datastore has no entities in the Empty namespace. You need to add data programatically before you can use this tool to view and edit it" but when I look into ...\WEB-INF\appengine-generated\local_db.bin file I see it was updated and the entities are there. So for some reasons console doesn't see them. I am using SDK 97.0.0 but the console shows SDK v1.9.27
Check the location of your storage.bin when running the datastore emulator. Then use that storage path when running the dev_appserver.py:
dev_appserver.py test.yaml --storage_path=/path/.config/gcloud/emulators/datastore/WEB-INF/appengine-generated/local_db.bin
Also make sure to connect your appserver to the datastore emulator port.
Related
I am very, very new to Google App Engine, but I need to administer a WebApp2 (so, Python) -based website that uses it. Specifically I need to filter a queryset and delete certain model instances.
Building the site locally is fine. Locally I can get at the models by hitting
http://localhost:8000/console
in my browser. This pulls up the "interactive console", and in there I can put some code like
from application.models.user import User
and it will work. From there I can create / delete User objects in the Datastore, so this local interactive console seems to have ORM-like functionality. (I'm coming from a Django background, so that's what it reminds me of).
However I don't seem to have this option on the live website. I was hoping to find it at
https://console.cloud.google.com/home/dashboard?project=<my project>
And I thought I had found it when I found the control to "activate google cloud shell". This brings up a command line on the web server, but the website's codebase doesn't seem to exist here, so launching the Python cli and trying to import the applications models like I can do locally doesn't work. And even if I could, I doubt it would be a Django-like ORM the way it is on my local build.
Am I thinking about this in the wrong way? How do you create / filter / delete / etc. Datastore objects via the backend in a GAE / WebApp2 website? Is the "Google cloud shell" even part of the answer?
To get access to your app code in the cloud shell one option would be to create in the shell a local copy of your app repository. For an example (addressing a different question, true) you can peek at Google Cloud: How to deploy mirrored Repository. But I'm not 100% certain if that will automatically give you access to your app's datastore. Worth a try IMHO.
Another option is to hook the desired operations as handler actions inside your app itself and execute them from there. Eventually hidden/protected/restricted via authentication, for example for admin users only. This is what I use for one-time datastore migrations that I need from time to time when I make changes to my entity models.
Finally, but not really a programatic access - you could use the Datastore page in the developer console to manually find, read and modify your entities, see Managing Datastore from the Console.
Use the remote_api_shell. You run this locally where your application code lives. Then you can import your models, and perform the same queries etc, and add modify/delete entities. I use this frequently for a range of tasks. Updating more than 100K records can become slow using this method.
https://cloud.google.com/appengine/docs/standard/python/tools/remoteapi
This is the functionality I am referring to in the old style GAE console
At the bottom of this page, there is 'Backup Entities' button
I am not able to find the corresponding function in the new GAE console interface.
Is it possible to manually drive a backup process in the new GAE console as at the time of writing (4 Jun)?
Mark Cummins in the AppEngine-GoogleGroup has discovered that it's under Storage > Cloud Datastore > Settings.
It is missing (yet?) in the new UI just like a lot of other features.
But you still can access it directly at https://ah-builtin-python-bundle-dot-APPID.appspot.com/_ah/datastore_admin or even programmatically using this documentation.
I read about the DataStore admin, but can't find it in the console. Also, the doc says the feature is experimental?
More clarification:
We have an enterprise application, and we need to be able to take backups of our data at a certain frequency. Since we are using Google App Engine, we need to be able to take a backup off the App Engine DataStore. When I go into the Admin console, I can see the option to view the entities, I see "Dashboard","Query" and "Index" but nothing to take a backup of the DataStore data, say to something like CloudStorage. Also, when I go to localhost:8000/datastore, I see the same.
Datastore Admin is only available to Python applications - or rather, applications with a Python version. If your app is Java, you can deploy an empty/trivial app to a non-default version.
There's a note on this in the docs, here.
At the time of your request it was only available in the old GAE console.
Since few weeks it has been integrated in the new console as well.
Go to Storage/Cloud Datastore/Settings, then the first time you'll have to "Enable Datastore Admin", then you'll have a new button "Open Datastore Admin"
I am trying to follow the instructions for running a simple new Google App Engine web application locally (without Google Web Toolkit, just the Web App) named "tunes".
I am following these instructions. Step one is to make a run configuration. I made one using all the defaults; I checked that under the Server tab the "Run built-in Server" box is checked. However, when I Run the app, I get the Console; right below the console tab it says
<terminated> tunes [Web Application] C:\Program Files (x86)\Java\jre7\bin\javaw.exe (Feb 26, 2014, 5:35:44 PM)
then below the line is what looks like a classic Unix "Usage" error message in the console, in red type, whose first line is
Usage: <dev-appserver> [options] <app directory>
followed by a list of options, and then nothing else happens that I can see.
I tried pointing a browser at http://localhost:8888/tunes as suggested by the documentation, but Firefox could not find a server active at that port.
Under the Arguments tab in the Run Configuration is the following:
"-codeServerPort 9997 --port=8888 org.tunes.gaeproject.Tunes C:\Users\cdf\java\eclipse4.3.1workspace\tunes\war"
I can successfully deploy the web application to the Google App Engine site and run it there.
What do I have misconfigured?
Google AppEngine application is not meant to run on local server, and neither you could create its docker image etc to deploy it anywhere you wish.
Instead I suggest you to port your application to Google Compute Engine (GCE) first within your deployment setup, which might require minor code refactorings, and Kubernetes kinda auto-scaling deployment will functional equivalent to what you have now. But with approach you may port the application easily to local server setup, or a docker image to run from any virtualization environment
How to clear datastore in GoogleAppEngine.
I want to clear my development data to do a test again, but I can found a way to do that
If you are running from the commandline, use the --clear_datastore flag, e.g.,
dev_appserver.py --clear_datastore=yes app
Otherwise, if you're running it off the included GAE launcher, go into the settings of your app (double click it), and there should be a little checkbox that says "clear datastore on launch" under Launch Settings.
reference:
https://cloud.google.com/appengine/docs/python/tools/devserver#Python_Using_the_Datastore
dev_appserver.py --clear_datastore myapp
assuming by "development data", you mean the data in the dev server.
Simply use Administering Your Datastore (Experimental)
Some Other ways
App Engine: How to "reset" the datastore?
Delete all data for a kind in Google App Engine
Interactive console is also a great way.
For Java, the following information can be seen in Using the Datastore - Clearing the Datastore at the end of the page (as of 2013/05/10):
The development web server uses a local version of the Datastore for testing your application, using local files. The data persists as long as the temporary files exist, and the web server does not reset these files unless you ask it to do so.
The file is named local_db.bin, and it is created in your application's WAR directory, in the WEB-INF/appengine-generated/ directory. To clear the Datastore, delete this file.
So, stop your server, delete the file, and restart it up.
On your local machine you can go to : http://localhost:8080/_ah/admin/datastore