I am very, very new to Google App Engine, but I need to administer a WebApp2 (so, Python) -based website that uses it. Specifically I need to filter a queryset and delete certain model instances.
Building the site locally is fine. Locally I can get at the models by hitting
http://localhost:8000/console
in my browser. This pulls up the "interactive console", and in there I can put some code like
from application.models.user import User
and it will work. From there I can create / delete User objects in the Datastore, so this local interactive console seems to have ORM-like functionality. (I'm coming from a Django background, so that's what it reminds me of).
However I don't seem to have this option on the live website. I was hoping to find it at
https://console.cloud.google.com/home/dashboard?project=<my project>
And I thought I had found it when I found the control to "activate google cloud shell". This brings up a command line on the web server, but the website's codebase doesn't seem to exist here, so launching the Python cli and trying to import the applications models like I can do locally doesn't work. And even if I could, I doubt it would be a Django-like ORM the way it is on my local build.
Am I thinking about this in the wrong way? How do you create / filter / delete / etc. Datastore objects via the backend in a GAE / WebApp2 website? Is the "Google cloud shell" even part of the answer?
To get access to your app code in the cloud shell one option would be to create in the shell a local copy of your app repository. For an example (addressing a different question, true) you can peek at Google Cloud: How to deploy mirrored Repository. But I'm not 100% certain if that will automatically give you access to your app's datastore. Worth a try IMHO.
Another option is to hook the desired operations as handler actions inside your app itself and execute them from there. Eventually hidden/protected/restricted via authentication, for example for admin users only. This is what I use for one-time datastore migrations that I need from time to time when I make changes to my entity models.
Finally, but not really a programatic access - you could use the Datastore page in the developer console to manually find, read and modify your entities, see Managing Datastore from the Console.
Use the remote_api_shell. You run this locally where your application code lives. Then you can import your models, and perform the same queries etc, and add modify/delete entities. I use this frequently for a range of tasks. Updating more than 100K records can become slow using this method.
https://cloud.google.com/appengine/docs/standard/python/tools/remoteapi
Related
Is there a way to create a new Datastore entity kind via some interactive means for Go App Engine apps? The datastore viewer won't let me add new entity kinds, and the interactive console doesn't seem to support Go.
I'm trying to upload some configuration data to datastore, including sensitive data that I don't want to appear in code. So far the best method I can come up with is to write some code to write an empty configuration entity, deploy, run, then use the datastore viewer to set the values.
Thanks in advance.
You absolutely can add new Entity Kinds in the Console.
In the Cloud Console, Click on the Datastore menu item, then 'Create an Entity'. Handy link, just select your project name
Then in the Kind field, just type in your desired Kind name.
First of all your code is safe at AppEngine - nobody can download it. At least nobody outside of Google. You can deploy it to a dedicated version/module and restrict usage to yourself (check current user, ask pass-phrase, etc in your code).
Second you can use Remote API - this way you do not upload any code to AppEngine. You can create entities remotely as you wish using secure HTTPS connection.
So I started to play with Google Mobile Backend Starter.
Now I want to clean this instance and anything that this starter thingy created into the project (e.g. task queues, data store, etc...)
How do we achieve this?
Is this done through some command line somewhere along the lines described in this page?
appengine-java-sdk/bin
EDIT: I should have made it clearer, I don't intend to delete the project. I just want to "clean" it and replace with my own application. Anyway, I ended up using the appengine SDK tools mentioned above (Updating and Managing a Java App). It was a long process, and tedious. It could be improved.
What I did:
Using the appengine SDK tool, downloaded the application first. It prompted me for a password. I had to create a new "App Password" entry in my Google account, since it didn't accept my "usual" Google password (e.g. GMAIL)
To clean the Scheduled Tasks, edit cron.xml so that no entries are left. A sample empty cron.xml file is shown in the documentation page. Run the update procedure of the SDK tool for cron jobs
To clean the Queues, use the same approach (queue.xml)
Clean DataStore by going to DataStore Admin page (if applicable)
Upload your new application (either thru SDK, or thru Android/Eclipse AppEngine plugin)
There will now be 2 (or more) versions of your AppEngine. If necessary, make your newly uploaded application, the default version. This is done in the Developer Console.
Check the instances as well. Remove if necessary
I'm quite new to Google App Engine and it's cloud Datastore which is used for storing the backend's data by default. As far as I realized you can only view it's content within the developer console and you can create or edit entities there.
But is there any external tool from which you can connect to your datastore to create reports or administer the data? What is your experience?
In fact yes it's true you can only see data's from the admin console.
If you wish to see your data's in Google Drive Table and make a report you can, but for that you need to create a connector to your sheet (I already made one). It's exactly the same if you need update or import data's to your datastore.
I use this Technic to upload or refresh products on my e-shop GAE app.
In general if I need to see a report, I design a specific web page for that and I protect theme via a login / password. To see a well formatted report you can use jquery library or use Google Charts
I've been thinking lately about the pros and cons of using AppEngine.
My concern would be, when we create application for GAE, the front-end code (the UI stuff) is served from the same application instance in the GAE cloud as with the Datastore codes.
The question would be when my applications grows:
For GAE:
Do I need to create multiple instance of my application?
If so, what do I need to manually update all instances?
For Appscale:
Do I also need to create multiple instance of my application?
If so, what do I need to manually update all instances?
GAE starts new frontend instances automatically, you even can't create or update frontend instances. You just need to configure min/max latency, min/max idle instances in Application Settings. See docs for performance settings
Btw, there are also Backend Instances that can be Resident and started manually from Admin Console. But it's useful only when you need something very specific
You seem to have missed the whole point of AppEngine, which is that Google takes care of scaling your app for you automatically. You seem to be confusing 'instance' with 'version' - you have control over which version of your app is serving, but Google dynamically creates and kills instances of that app depending on load. That's the main benefit of using AppEngine in the first place.
I and my friend are working on a GWT-Google App Engine project, using Tortoise SVN and Google Code to synchronize the code.
We also synchronize the local_db.bin file in appengine-generated folder. But we cant get it work. After synchronize the db file, our local datastore is not updated as we expected.
That is a pain. Im worrying about our future, when our database get bigger and more complicated #A#.
Anyone please give me an advice. What should i do to synchronize our local datastore?
I have to suggestions:
1) Use remote api : https://developers.google.com/appengine/articles/remote_api to share a GAE hosted db locally.
2) Maybe you can use Gdrive to sync folders.
This is a really bad idea. Even if you weren't having trouble making both ends read from the same datastore file, the local datastore is in a binary format, and thus you won't both be able to work on the app at the same time, or you'll get merge conflicts you will be unable to resolve.
Instead, both for collaboration purposes and for testing and deployment, you should provide a set of test data you can easily load into the datastore. Store the test data in version control, and load it in using bulkloader or your own code.