I have an old AppEngine Java application using the AppEngine datastore. Is this what the marketing renamers at Google now (2019) call "Cloud Datastore"?
Can I create Google Cloud Functions that interact with the same datastore, and what are the steps needed to do so?
Yes, it's the same datastore. Also called/soon-to-be Cloud Firestore in Datastore mode (which all older apps will be converted to at some point).
Yes, you can access it from anywhere, even from outside Google Cloud. From Cloud Datastore (emphasis mine):
You can access Cloud Datastore from anywhere using the Cloud
Datastore API. Use the Google Cloud client libraries to store and
retrieve data from Cloud Datastore.
The same Cloud Datastore data is available regardless of if you use the App Engine libraries, the Google Cloud client libraries, or call
the API directly.
The major steps to access the datastore from a Cloud Function:
you can't use the GAE-specific client libraries like the one you likely used in your old app, you'll have to use one of the generic client libraries (or the REST or RPC APIs)
you'll have to give your CF's Identity/service account the proper access permissions, see Setting up authentication and Accessing your database from another platform.
Related
I'm currently new to web developing and right now I started with a Spring Boot application which I converted to standard App Engine project. I'm wondering if it is possible to not convert my Spring Boot application to my App Engine project and still use Datastore as database?
Yes, it's possible to use the Cloud Datastore from an app in the GAE flexible environment, from outside GAE or even from outside the Google Cloud. From Cloud Datastore (emphasis mine):
You can access Cloud Datastore from anywhere using the Cloud
Datastore API. Use the Google Cloud client libraries to store and
retrieve data from Cloud Datastore.
The same Cloud Datastore data is available regardless of if you use
the App Engine libraries, the Google Cloud client libraries, or call
the API directly.
But you can't use the GAE Standard Environment Client Libraries, you have to use either the Cloud Datastore Client Libraries or the Cloud Datastore API v1.
Potentially of interest: the Deploying to the App Engine Flexible Environment guide happens to use a spring boot app as example.
ndb: (from google.appengine.ext import ndb)
datastore: (from gcloud import datastore)
What's the difference? I've seen both of them used, and hints they both save data to google datastore. Why are there two different implementations?
The Python NDB Client is specific to Python Applications running on Google App Engine. The datastore client removes that restriction and you can run your Python application anywhere, including Google App Engine, Google Compute Engine or anywhere else.
Exceprt form - https://cloud.google.com/appengine/docs/python/ndb/
The Google Datastore NDB Client Library allows App Engine Python
apps to connect to Cloud Datastore.
In addition, the NDB client has certain features (e.g. caching) while the other does not appeat to support.
The reason for the two implementations is that originally, the Datastore (called App Engine Datastore) was only available from inside App Engine (through a private RPC API). On Python, the only way to access this API was through an ORM-like library (NDB). As you can see on the import, it is part of the App Engine API.
Now Google has made the Datastore available outside of App Engine through a restful API called Cloud Datastore API. gcloud library is a client library that allows access to different rest APIs from Google Cloud, including the Cloud Datastore API.
We got a couple millions data in the current GAE project using Google Cloud store. Mostly GPS point information. We want to be able to use all these GPS points in another demo instance, which is hosted in another GAE instance. Anyway we can do it?
Using Golang + Google App Engine
There is a Google Cloud Datastore API that you can use to access your Datastore data from any other deployment, including a different App Engine app. It's not available in Go, so you will have to mix in some Python or Java.
I am working on a project which needs to store profiles of people on the cloud.
Information includes multiple photos and multiple text fields. I don't need Messaging.
There is lot if confusion in the documentation provided Google Cloud Services.
I am confused about what storage services should I opt for out of the 3:
1-Google Cloud Services,
2-Google Datastore,
3-Google Cloud SQL
So the the things i need to confirm are:
0.Is there a storage limit on using Google cloud SQL?
1.Does Google Cloud Storage and Google cloud Datastore provide unlimited storage?
2.Can an Android user write data on the cloud. I heard from some where that the applications only have access to read the data and the developer needs to put the data as blob on the cloud him self when using Google Cloud Services. Is this fact true for all the 3 storage services.
3.Is the data fully 'Sharable'+'Searchable'?
In other words:
If an Android user stores data in cloud in Google Datastore(text)+Google Cloud Storage(image), can this data be accessed by another android user without any headache of permissions or authentication(after I authenticate my app/app-engine)?
4.Is it the best option to store the images in Google Cloud Storage and their URLs in Google Datastore?
5.Does all the three storage services need app-engine to work?
6.Are any limitations on each of these services?
(0) Cloud SQL has currently a limit of 250Gb.
(1) With regards to Cloud Storage, there isn't a limit you could reach.
(2) and (3) They're not created for easiness of searchability. They should be accessed through applications, that are authorized, i.e. is not a substitute to Google Drive or Dropbox.
If you're the owner of the project, you can "browse" the contents of your Cloud Storage, but it's not meant for that.
Furthermore, objects in Cloud Storage can't be modified once created. A change needs to create a new copy of the object.
(4) It's a good idea, and something is used by many developers who have their applications in Google App Engine.
(5) No, they can exist without you using Google App Engine, but as I said earlier, you'd probably need an "application" to allow your Web/Android users to interact with the data, and there's where GAE comes handy.
(6) Yes, your budget.
If you provide a more detailed use case, I could tell you what you'd need to do to get it done with the whole array of Google Cloud products.
I would like to import data from flat files stored in Google Drive into DataStore. Then use the full-text search and other query options to analyze the data using apps-script.
The script API doc shows how we can access Google Drive data from the apps-script.
Now, is there any API in apps-script to access DataStore from the scripts?
Google provides a (beta) REST API to access your data. Steps to enable are here.
However, BigQuery is usually better for the type of analysis you describe. See:
https://developers.google.com/apps-script/advanced/bigquery
At this point in time, you should consider writing your own Web Service to get you access to the DataStore. You can then access that Web Service hosted in your App Engine application from your App Scripts. A detailed example is provided over here.
Additionally, the Cloud DataStore is now provided under the Google Cloud Platform and while it is still in preview, there is an API available to interact with the Datastore. This API is exhaustive and allows for both read and write operations. But keep in mind that it is currently under Preview.