As Python 2 came to an end and App Engine 1 is no longer supported, we are migrating to App Engine 2. I have a sizable Datastore. Given that the two App Engines need to be on two different projects, is it possible to connect to the old App Engine's datastore as the new App Engines's datastore? Ideally using NDB.
The datastore of a GAE/GCP project can be accessed from an app in another project or even from outside Google Cloud, see How do I use Google datastore for my web app which is NOT hosted in google app engine?.
I didn't yet play with the python 3 ndb library (aka Cloud NDB), I can't exactly say if/how it can use another project's credentials. I'm unsure if you want to try to use it, though: from Migrating to Cloud NDB:
Cloud NDB is intended to replace the features in App Engine NDB, so it
will not support new features of Firestore in Datastore mode. New
Python 3 apps should use the Datastore mode client library
instead of Cloud NDB.
In the worst case the Datastore mode client library (actually the recommended one for python 3 GAE apps) should be able to access your python 2 app's datastore - it is generic, it can be used for any app, not just GAE. According to the docs its Client() method supports specifying a project and credentials (where you'd be using the service account for the python 2 app's project). Potentially of interest: GCP-The App Engine APIs are not available, with py 3
Related
Newbie to Datastore. I found two tutorials on the GCP site to use Datastore on App Engine. Which one should I use?
There are subtle differences in how the APIs work.
https://cloud.google.com/datastore/docs/datastore-api-tutorial
uses package (import com.google.appengine.api.datastore.Entity;)
vs.
https://cloud.google.com/appengine/docs/standard/java/building-app/cloud-datastore
uses package (import com.google.cloud.datastore.Entity)
Question: Is there a preferred package to use and call the datastore API while on App Engine - com.google.cloud.datastore.Entity vs. com.google.appengine.api.datastore.Entity?
The library with this package com.google.appengine.api.datastore (aka "Datastore API for Java") is intended for use by Java 7 and Java 8 applications which run on AppEngine.
The library with this package com.google.cloud.datastore (aka "Cloud Datastore client library") is intended for use by any Java 8 application regardless of where it is deployed (GKE, GCE, on-premises etc)
From the docs:
Datastore API for Java is a low-level Datastore API built into the App Engine SDK to provide direct access to all Datastore features and is described throughout the App Engine Datastore documentation for Java.
Cloud Datastore client library is a library that can be used by apps in the App Engine standard Java 8 runtime, by applications in the App Engine flexible environment, and by non App Engine applications as well.
So, according to Google, both are valid choices subject to these limitations ...
If you are running Java 7 then you cannot use Cloud Datastore client library
If your application is not deployed to AppEngine then you cannot use Datastore API for Java
These limitations describe the scenarios in which one or other of these libraries cannot be used. Google offers no advice on which one should be used. This is because the two libraries are functionally equivalent so, assuming that the limitations described above do not apply to your usage, the choice is probably moot.
I'm currently new to web developing and right now I started with a Spring Boot application which I converted to standard App Engine project. I'm wondering if it is possible to not convert my Spring Boot application to my App Engine project and still use Datastore as database?
Yes, it's possible to use the Cloud Datastore from an app in the GAE flexible environment, from outside GAE or even from outside the Google Cloud. From Cloud Datastore (emphasis mine):
You can access Cloud Datastore from anywhere using the Cloud
Datastore API. Use the Google Cloud client libraries to store and
retrieve data from Cloud Datastore.
The same Cloud Datastore data is available regardless of if you use
the App Engine libraries, the Google Cloud client libraries, or call
the API directly.
But you can't use the GAE Standard Environment Client Libraries, you have to use either the Cloud Datastore Client Libraries or the Cloud Datastore API v1.
Potentially of interest: the Deploying to the App Engine Flexible Environment guide happens to use a spring boot app as example.
Im creating a Node.js website that probably won't have loads of traffic, and was looking into cheap solutions to host the site. Came across Google cloud services offering free usage for their services with limits. A f1-mirco is more than enough for my needs, but I will happily pay for some usage if it goes over by any chance.
I wanted to setup a linux centOS 7 on GCE (which I already did), and run my application and REST API on it. Now here comes the problem.
I tried to use Google's datastore service, but it sprung an app engine instance and without it datastore won't work.
Is datastore entirely relying on app engine to function?? In the docs, it said if you use any of the client API, it requires app engine. What can I do to not use the client api and query data then? Don't want to use the app engine at the moment or datastore is just not for me then?
Thanks for any help!
Some of the underlying infrastructure of Cloud Datastore and App Engine are still tied together for creation, etc. So while creating an Cloud Datastore database also defines an App Engine instance for the project, it doesn't require you to use it. You don't get charged for App Engine either, unless you decide to deploy an App using it.
You should be totally fine use the Google Cloud Node client library on the f1 micro instance.
I have a situation where an existing GAE App (let's call it app A) is running, but for non-technical reasons can't be modified. As users migrates to a new client version, we need to migrate their data from app A to a new GAE app (which I'll call app B).
Is there a way that I can grant app B access to app A's live datastore without modifying app A? My not modifying I mean not having to deploy new code. Changing setting or permissions in the Cloud Console is fine.
In case it matters, both apps that I'm referring to are written in Go.
It might not be possible to share the datastore across multiple GAE apps using the Google App Engine Standard Environment Client Libraries. At least for python it's not possible, donno about go.
But the Cloud Datastore Client Libraries can be used to share a datastore across many apps, even from outside Google Cloud.
Regardless of the particular way the old app accesses the datastore (language/library/etc.) it can be configured from the Cloud Console to allow access to a remote app. The exact procedure steps are captured in How do I use Google datastore for my web app which is NOT hosted in google app engine?
The new app would be using the above-metioned client library with the old app's service account credentials (obtained in the above paragraph procedure) to access the old app's datastore.
ndb: (from google.appengine.ext import ndb)
datastore: (from gcloud import datastore)
What's the difference? I've seen both of them used, and hints they both save data to google datastore. Why are there two different implementations?
The Python NDB Client is specific to Python Applications running on Google App Engine. The datastore client removes that restriction and you can run your Python application anywhere, including Google App Engine, Google Compute Engine or anywhere else.
Exceprt form - https://cloud.google.com/appengine/docs/python/ndb/
The Google Datastore NDB Client Library allows App Engine Python
apps to connect to Cloud Datastore.
In addition, the NDB client has certain features (e.g. caching) while the other does not appeat to support.
The reason for the two implementations is that originally, the Datastore (called App Engine Datastore) was only available from inside App Engine (through a private RPC API). On Python, the only way to access this API was through an ORM-like library (NDB). As you can see on the import, it is part of the App Engine API.
Now Google has made the Datastore available outside of App Engine through a restful API called Cloud Datastore API. gcloud library is a client library that allows access to different rest APIs from Google Cloud, including the Cloud Datastore API.