I've playing around with Google App Engine and Google Datastore for a while now and I am facing the need to take regular backups of my stuff up on the cloud.
Is there any sort of general purpose tool that allows you to download all your data from a specific instance of google-datastore and restore it to another one?
If so, please enlighten me, if not - someone should do it!
You can use the BulkLoader that is described in the Google Appengine Docs
There is an opensource application available that I developed called "GAE datastore backup utility." It allows you to backup and restore your GAE entities through a GUI. It keeps track of your applications and your application's entities. It's currently only for Windows and Python based GAE apps but it sounds like something you are looking for. I see that you are looking for a solution for Java based apps. I will look into supporting this language in the next release.
http://code.google.com/p/gae-datastore-backup-utility/
Follow this tutorial for Exporting and Importing Entities: https://cloud.google.com/datastore/docs/export-import-entities
Windows:
set BUCKET=YOUR_BUCKET_NAME[/NAMESPACE_PATH]
gcloud datastore export --namespaces="(default)" gs://%BUCKET%
gcloud datastore import gs://%BUCKET%/[PATH]/[FILE].overall_export_metadata
Linux:
BUCKET="YOUR_BUCKET_NAME[/NAMESPACE_PATH]"
gcloud datastore export --namespaces="(default)" gs://${BUCKET}
gcloud datastore import gs://${BUCKET}/[PATH]/[FILE].overall_export_metadata
You may have a look at the following site:
http://aralbalkan.com/1837
It is about the backup and restore of the datastore.
http://code.google.com/appengine/articles/gae_backup_and_restore.html
I describe bulkloader.py no-config dump/restore in What's the best way to back up data on Google App Engine?
After years, this came out as a feature of the Datastore Administration in the app-engine dashboard, still experimental but still good news!
Related
What is better to code with php in google app engine or in amazon-ec2.
I think it is better in amazon-ec2 because they support datastore with php and google app engine doesn't, what do you think ?
While its not possible to access the appengine datastore, google has a new preview service for "cloud datastore", google "cloud datastore from php" and you will see how to use it.
You really cant compare AWS with appengine as one requires managing servers and scalability manually and their noSQL solution (dynamoDB) is a joke compared to google's datastore, for example in dynamoDB you must provision your writes beforehand and even if you are all day changing provisioning, it takes sometimes hours to propagate the new setting.
I had never use Google App Engine, but several times AWS systems, and sure, as AWS EC2 could be used as Linux Server Instance, I recommend you that provider. And coz' it seems that you use PHP, they have strong API for this langage. Have fun with AWS.
I'm looking for tool (data extract utility) to browse/extract data from the Google App Engine Datastore. I found one but it hasn't been released - GAEBrowser. Does anybody know of a similar tool? Thanks in advance!
I know one, of course, the datastore viewer in GAE admin console: https://appengine.google.com/datastore/explorer?&app_id={yourappid}
What's wrong with the official datastore viewer anyway?
You might also look into remote_api, which lets you write Python scripts that run on your own computer but access your app's Datastore: https://developers.google.com/appengine/articles/remote_api
I'm trying to deploy one of my java apps in GAE and I have used MongoDB. Unfortunately GAE doesn't allow me to create databases, however it suggests using Google Datastore. Are there any plugins or add-ons to use MongoDB.
You can't run MongoDB on GAE. You need to use the Google datastore.
Matt is right. If you want to deploy to Google's servers you can't use Mongo. If you are interesting in running your own server(s), check out AppScale.
If you really want to use MongoDB anyway, you'll have to find a third-party host that provides a REST interface through which you can make your transactions. I've personally tried and recommend MongoLab on App Engine. Then, in your code, use an HttpUrlConnection object to send/receive requests if you're doing it with Java.
I think you could deploy your app on app engine and create a separate vm instance in google compute engine that would have mongo installed, then you plug your GAE app on that vm for the db connection. This would allow you to keep all your app in the google cloud. See https://cloud.google.com/solutions/mongodb/ and http://docs.mongodb.org/ecosystem/platforms/google-compute-engine/
A lot of things have changed since this question has been asked, as of June 2021, you can use MongoDB Atlas with Google App Engine Standard running on Java 8 environment. I built my App with the Ktor framework that runs on App Engine and interacts with databases on MongoDB Atlas.
if you're still looking for an answer to this you might want to check this project that did exactly the same but in reverse, it will surely help you out with what you want to do.
http://code.google.com/appengine/articles/django-nonrel.html
you can use mongo, only if have something like on mongolab then use pymongo to connect and most important thing you need to enable socket api from gae dashboard.
it worked for me thanks
I want to export the datastore that I have running in App Engine into my local "standalone" version of the application. Anyone know how I can do that?
I've been looking around in the App Engine dashboard but can't find it.
Check out the docs on uploading and downloading data.
Gobo-Tools is designed for such tasks
It seems that there is no separate doc for java version of this except that it is said to do with com.google.apphosting.utils.remoteapi.RemoteApiServlet in the doc Nick provided.
Tip: If you have a Java app, you can use the Python appcfg.py tool by installing the Java
version of the remote_api handler, which is included with the Java runtime environment.
The handler servlet class is com.google.apphosting.utils.remoteapi.RemoteApiServlet.
Here is some relevant code example for java version of this API usage GAE/J datastore backup
I just want to know if it is possible to download and upload the whole database from Google App Engine to a SQLite db, in case I want to transfer my app from Google App Engine to another server?
What is the chance?
Thank you!
There is a project called approcket that synchronizes Google App Engine datastore and a local MySQL database. Maybe you should have a look.
In a related answer, I linked to the documentation for the bulk loader tool.
However you will be more interested in the rest of the document, especially the section on creating an exporter class for exportation of the data.