I would like to import data from flat files stored in Google Drive into DataStore. Then use the full-text search and other query options to analyze the data using apps-script.
The script API doc shows how we can access Google Drive data from the apps-script.
Now, is there any API in apps-script to access DataStore from the scripts?
Google provides a (beta) REST API to access your data. Steps to enable are here.
However, BigQuery is usually better for the type of analysis you describe. See:
https://developers.google.com/apps-script/advanced/bigquery
At this point in time, you should consider writing your own Web Service to get you access to the DataStore. You can then access that Web Service hosted in your App Engine application from your App Scripts. A detailed example is provided over here.
Additionally, the Cloud DataStore is now provided under the Google Cloud Platform and while it is still in preview, there is an API available to interact with the Datastore. This API is exhaustive and allows for both read and write operations. But keep in mind that it is currently under Preview.
Related
I have an old AppEngine Java application using the AppEngine datastore. Is this what the marketing renamers at Google now (2019) call "Cloud Datastore"?
Can I create Google Cloud Functions that interact with the same datastore, and what are the steps needed to do so?
Yes, it's the same datastore. Also called/soon-to-be Cloud Firestore in Datastore mode (which all older apps will be converted to at some point).
Yes, you can access it from anywhere, even from outside Google Cloud. From Cloud Datastore (emphasis mine):
You can access Cloud Datastore from anywhere using the Cloud
Datastore API. Use the Google Cloud client libraries to store and
retrieve data from Cloud Datastore.
The same Cloud Datastore data is available regardless of if you use the App Engine libraries, the Google Cloud client libraries, or call
the API directly.
The major steps to access the datastore from a Cloud Function:
you can't use the GAE-specific client libraries like the one you likely used in your old app, you'll have to use one of the generic client libraries (or the REST or RPC APIs)
you'll have to give your CF's Identity/service account the proper access permissions, see Setting up authentication and Accessing your database from another platform.
I am working on a project which needs to store profiles of people on the cloud.
Information includes multiple photos and multiple text fields. I don't need Messaging.
There is lot if confusion in the documentation provided Google Cloud Services.
I am confused about what storage services should I opt for out of the 3:
1-Google Cloud Services,
2-Google Datastore,
3-Google Cloud SQL
So the the things i need to confirm are:
0.Is there a storage limit on using Google cloud SQL?
1.Does Google Cloud Storage and Google cloud Datastore provide unlimited storage?
2.Can an Android user write data on the cloud. I heard from some where that the applications only have access to read the data and the developer needs to put the data as blob on the cloud him self when using Google Cloud Services. Is this fact true for all the 3 storage services.
3.Is the data fully 'Sharable'+'Searchable'?
In other words:
If an Android user stores data in cloud in Google Datastore(text)+Google Cloud Storage(image), can this data be accessed by another android user without any headache of permissions or authentication(after I authenticate my app/app-engine)?
4.Is it the best option to store the images in Google Cloud Storage and their URLs in Google Datastore?
5.Does all the three storage services need app-engine to work?
6.Are any limitations on each of these services?
(0) Cloud SQL has currently a limit of 250Gb.
(1) With regards to Cloud Storage, there isn't a limit you could reach.
(2) and (3) They're not created for easiness of searchability. They should be accessed through applications, that are authorized, i.e. is not a substitute to Google Drive or Dropbox.
If you're the owner of the project, you can "browse" the contents of your Cloud Storage, but it's not meant for that.
Furthermore, objects in Cloud Storage can't be modified once created. A change needs to create a new copy of the object.
(4) It's a good idea, and something is used by many developers who have their applications in Google App Engine.
(5) No, they can exist without you using Google App Engine, but as I said earlier, you'd probably need an "application" to allow your Web/Android users to interact with the data, and there's where GAE comes handy.
(6) Yes, your budget.
If you provide a more detailed use case, I could tell you what you'd need to do to get it done with the whole array of Google Cloud products.
I am writing an Android application that allows users to upload and share photos. The server is based on Google App Engine. App Engine's datastore does not allow to save file, so currently I just have URLs saved. Looking for a way to store files I read about Google Cloud Storage. My question is - if I'm looking for a host for user uploaded files, is Google Cloud Storage what I'm looking for?
Yes. Google Cloud Storage is the way forward. There is also the BlobStore API on App Engine that allows you to store large amounts of information but the road map seems to be clear i.e. use Google Cloud Storage moving forward.
The reason for going with GCS will also be influenced that eventually you might want various tools or utilities that people have written that work directly with GCS. With Blobstore API, you will need to write those utilities yourself or rely on Admin console's support for taking backup,etc - which is not really much.
In summary, go with GCS.
Yes thats what you want. Says the same if you read the docs about google cloud storage.
I have made a few stand alone apps that don't pull too much information from the internet. However, I want to make an app that allows users to post data which is then used by every user who has the app installed, like an online communal DB/datastore.
From what I've read it looks like Google App Engine is the solution and can be used in this sort of manner (looking at the "guestbook" demo app").
Can I use the Google app engine to provide data for an app?
If you're talking about an Android application, I think you can.
Depending on what kind of data you wish to store on your GAE datastore,
you can make a RESTful way of retreiving data from your GAE datastore.
For one example, you may be able to retrieve that data as a JSON object
Application sends GET request to the server, and the server responds with a JSON object.
I was having a look at the new Google Cloud Datastore and looks great. But there is something I could not understand... is it supposed to substitute Google App Engine Datastore? How can I use it inside GAE? What are the differences between both of them?
I have a GAE app in Java that uses 3 entities with thousands of rows each one, and I need to do joins quite often...
The cloud datastore is the App Engine datastore, for use outside App Engine. You won't get any benefit trying to use it with your App Engine app, unless you need other external apps to also have access to the data.
You certainly won't get more efficient joins. If you really need that, perhaps you should look into Cloud SQL, which is basically a version of MySQL you can use from App Engine.
They both are the same, in fact Google App Engine can use Google Cloud DataStore as one of its way to store data, the other options include Google Cloud SQL, Google Cloud Storage. You may select among these three according to the type of data you want to store and the way you want to access them.
From your question its clear that Google Cloud SQL would be right choice as no other options provide joins for retrieving results.
Cloud SQL is nothing but MySQL(the popular open source database) running on google platform. So you can perform your regular SQL-like queries to get your results.