I'm using Google AppEngine with build-in datastore. But, I want move all datastore to my new VPS.
I'll use Apache Cassandra. How to move from GAE Datastore to Apache Cassandra?
My guess is you're looking at a tool such as the bulk loader/downloader:
http://code.google.com/appengine/docs/python/tools/uploadingdata.html
You'll want to export all your data into CSV, then write a script to import this into any new format you want.
You can not use the bulk downloader if you are using the "High Replication" datastore.
You can use a manual aproach such as listing all your entities as dictionaries. You will have a JSON formatted string. By using this you can generate your entities again suitable for your new system.
Related
Previously, we used Apache MySQL as data storage layer. And based on that we developed Spring-Hateoas APIs using Spring-Boot 1.2.7 - Spring-Data-JPA.
Now as per new requirement, we wan to develop a search engine.
Firstly, we were asked to move to Cassandra from MySQL as it is NoSQL.
Secondly, we need to build search engine, but we don't want to give up on Cassandra.
So how can we use Cassandra and Solr together and build search engine using Spring?
If you have been using Spring Data JPA before to access MySql it could be a good decision to go for the spring data projects for Cassandra and Solr:
http://projects.spring.io/spring-data-cassandra/
http://projects.spring.io/spring-data-solr/
Is that what your are asking for?
Is it possible to create AppEngine-independent applications with Python NDB API? I need to host some basic scripts with database on AE, but I don't want to vendor lockin into the service.
NDB was designed and built on App Engine. Turning it in to something portable would be a research project.
You could use no DB at all if you don't need to store data, or you can use Cloud SQL which is really a mysql like instance for your application. In any case, if you use AppScale you can move your App Engine application whereever you want, thus there is no vendor lock-in.
I am storing data in google cloud datastore via an appengine project and would like to download one of the entities as a CSV file.
I have set up gsutil so that it defaults to this appengine project.
I have also created a bucket under cloud storage.
Is there a way to use gsutil to move data into the bucket?
gsutil cp gs://bucket_name
seems a likely candidate.
But IS this the way to go? And if so, what is the structure of the URI?
Many thanks!
You can use the lookup method to retrieve and entity JSON presentation by key.
I just want to know if it is possible to download and upload the whole database from Google App Engine to a SQLite db, in case I want to transfer my app from Google App Engine to another server?
What is the chance?
Thank you!
There is a project called approcket that synchronizes Google App Engine datastore and a local MySQL database. Maybe you should have a look.
In a related answer, I linked to the documentation for the bulk loader tool.
However you will be more interested in the rest of the document, especially the section on creating an exporter class for exportation of the data.
I've playing around with Google App Engine and Google Datastore for a while now and I am facing the need to take regular backups of my stuff up on the cloud.
Is there any sort of general purpose tool that allows you to download all your data from a specific instance of google-datastore and restore it to another one?
If so, please enlighten me, if not - someone should do it!
You can use the BulkLoader that is described in the Google Appengine Docs
There is an opensource application available that I developed called "GAE datastore backup utility." It allows you to backup and restore your GAE entities through a GUI. It keeps track of your applications and your application's entities. It's currently only for Windows and Python based GAE apps but it sounds like something you are looking for. I see that you are looking for a solution for Java based apps. I will look into supporting this language in the next release.
http://code.google.com/p/gae-datastore-backup-utility/
Follow this tutorial for Exporting and Importing Entities: https://cloud.google.com/datastore/docs/export-import-entities
Windows:
set BUCKET=YOUR_BUCKET_NAME[/NAMESPACE_PATH]
gcloud datastore export --namespaces="(default)" gs://%BUCKET%
gcloud datastore import gs://%BUCKET%/[PATH]/[FILE].overall_export_metadata
Linux:
BUCKET="YOUR_BUCKET_NAME[/NAMESPACE_PATH]"
gcloud datastore export --namespaces="(default)" gs://${BUCKET}
gcloud datastore import gs://${BUCKET}/[PATH]/[FILE].overall_export_metadata
You may have a look at the following site:
http://aralbalkan.com/1837
It is about the backup and restore of the datastore.
http://code.google.com/appengine/articles/gae_backup_and_restore.html
I describe bulkloader.py no-config dump/restore in What's the best way to back up data on Google App Engine?
After years, this came out as a feature of the Datastore Administration in the app-engine dashboard, still experimental but still good news!