I was having a look at the new Google Cloud Datastore and looks great. But there is something I could not understand... is it supposed to substitute Google App Engine Datastore? How can I use it inside GAE? What are the differences between both of them?
I have a GAE app in Java that uses 3 entities with thousands of rows each one, and I need to do joins quite often...
The cloud datastore is the App Engine datastore, for use outside App Engine. You won't get any benefit trying to use it with your App Engine app, unless you need other external apps to also have access to the data.
You certainly won't get more efficient joins. If you really need that, perhaps you should look into Cloud SQL, which is basically a version of MySQL you can use from App Engine.
They both are the same, in fact Google App Engine can use Google Cloud DataStore as one of its way to store data, the other options include Google Cloud SQL, Google Cloud Storage. You may select among these three according to the type of data you want to store and the way you want to access them.
From your question its clear that Google Cloud SQL would be right choice as no other options provide joins for retrieving results.
Cloud SQL is nothing but MySQL(the popular open source database) running on google platform. So you can perform your regular SQL-like queries to get your results.
Related
I've been wanting to use the "Documents and Indexes" feature of the Google Cloud platform, but due to the terribly constraints on AppEngine (e.g. I use gRPC, Cloud Endpoints etc etc). So instead I'm using GKE with Golang and it's all working good. My datastore is "Datastore" and I want to index some documents as I'm creating too many indexes that could result in exploding indexes.
How do we use the appengine search API from GKE? I get this error from Google Cloud Platform "not an App Engine context".. so does this mean that AppEngine standard gets the good stuff while users like me are left without these managed services and wondering if we will ever get these services?
So Google, will GKE/GCE users get:
Documents and Indexes (Search API)
Memcache service
Or do we have to roll up our own solution?
At this point you'll have to roll your own solution as App Engine search isn't available as a standalone product outside of App Engine Standard.
For caching you could follow the same advice given to App Engine Flexible users: https://cloud.google.com/appengine/docs/flexible/python/using-redislabs-memcache
Essentially use either Redis Labs Memcache/Redis Cloud offerings.
We got a couple millions data in the current GAE project using Google Cloud store. Mostly GPS point information. We want to be able to use all these GPS points in another demo instance, which is hosted in another GAE instance. Anyway we can do it?
Using Golang + Google App Engine
There is a Google Cloud Datastore API that you can use to access your Datastore data from any other deployment, including a different App Engine app. It's not available in Go, so you will have to mix in some Python or Java.
I am working on a project which needs to store profiles of people on the cloud.
Information includes multiple photos and multiple text fields. I don't need Messaging.
There is lot if confusion in the documentation provided Google Cloud Services.
I am confused about what storage services should I opt for out of the 3:
1-Google Cloud Services,
2-Google Datastore,
3-Google Cloud SQL
So the the things i need to confirm are:
0.Is there a storage limit on using Google cloud SQL?
1.Does Google Cloud Storage and Google cloud Datastore provide unlimited storage?
2.Can an Android user write data on the cloud. I heard from some where that the applications only have access to read the data and the developer needs to put the data as blob on the cloud him self when using Google Cloud Services. Is this fact true for all the 3 storage services.
3.Is the data fully 'Sharable'+'Searchable'?
In other words:
If an Android user stores data in cloud in Google Datastore(text)+Google Cloud Storage(image), can this data be accessed by another android user without any headache of permissions or authentication(after I authenticate my app/app-engine)?
4.Is it the best option to store the images in Google Cloud Storage and their URLs in Google Datastore?
5.Does all the three storage services need app-engine to work?
6.Are any limitations on each of these services?
(0) Cloud SQL has currently a limit of 250Gb.
(1) With regards to Cloud Storage, there isn't a limit you could reach.
(2) and (3) They're not created for easiness of searchability. They should be accessed through applications, that are authorized, i.e. is not a substitute to Google Drive or Dropbox.
If you're the owner of the project, you can "browse" the contents of your Cloud Storage, but it's not meant for that.
Furthermore, objects in Cloud Storage can't be modified once created. A change needs to create a new copy of the object.
(4) It's a good idea, and something is used by many developers who have their applications in Google App Engine.
(5) No, they can exist without you using Google App Engine, but as I said earlier, you'd probably need an "application" to allow your Web/Android users to interact with the data, and there's where GAE comes handy.
(6) Yes, your budget.
If you provide a more detailed use case, I could tell you what you'd need to do to get it done with the whole array of Google Cloud products.
What is better to code with php in google app engine or in amazon-ec2.
I think it is better in amazon-ec2 because they support datastore with php and google app engine doesn't, what do you think ?
While its not possible to access the appengine datastore, google has a new preview service for "cloud datastore", google "cloud datastore from php" and you will see how to use it.
You really cant compare AWS with appengine as one requires managing servers and scalability manually and their noSQL solution (dynamoDB) is a joke compared to google's datastore, for example in dynamoDB you must provision your writes beforehand and even if you are all day changing provisioning, it takes sometimes hours to propagate the new setting.
I had never use Google App Engine, but several times AWS systems, and sure, as AWS EC2 could be used as Linux Server Instance, I recommend you that provider. And coz' it seems that you use PHP, they have strong API for this langage. Have fun with AWS.
I would like to import data from flat files stored in Google Drive into DataStore. Then use the full-text search and other query options to analyze the data using apps-script.
The script API doc shows how we can access Google Drive data from the apps-script.
Now, is there any API in apps-script to access DataStore from the scripts?
Google provides a (beta) REST API to access your data. Steps to enable are here.
However, BigQuery is usually better for the type of analysis you describe. See:
https://developers.google.com/apps-script/advanced/bigquery
At this point in time, you should consider writing your own Web Service to get you access to the DataStore. You can then access that Web Service hosted in your App Engine application from your App Scripts. A detailed example is provided over here.
Additionally, the Cloud DataStore is now provided under the Google Cloud Platform and while it is still in preview, there is an API available to interact with the Datastore. This API is exhaustive and allows for both read and write operations. But keep in mind that it is currently under Preview.