I've been wanting to use the "Documents and Indexes" feature of the Google Cloud platform, but due to the terribly constraints on AppEngine (e.g. I use gRPC, Cloud Endpoints etc etc). So instead I'm using GKE with Golang and it's all working good. My datastore is "Datastore" and I want to index some documents as I'm creating too many indexes that could result in exploding indexes.
How do we use the appengine search API from GKE? I get this error from Google Cloud Platform "not an App Engine context".. so does this mean that AppEngine standard gets the good stuff while users like me are left without these managed services and wondering if we will ever get these services?
So Google, will GKE/GCE users get:
Documents and Indexes (Search API)
Memcache service
Or do we have to roll up our own solution?
At this point you'll have to roll your own solution as App Engine search isn't available as a standalone product outside of App Engine Standard.
For caching you could follow the same advice given to App Engine Flexible users: https://cloud.google.com/appengine/docs/flexible/python/using-redislabs-memcache
Essentially use either Redis Labs Memcache/Redis Cloud offerings.
Related
After reading available answers/comments on similar questions on this forum, it is now evident that GAE app is not straight-forward ready to be deployed on Compute engine. I fully understand that what all the managed services(mostly as APIs be it, datastore, document/index Search, memcache, cloud storage, task queues, cron jobs etc.), App Engine offered being a platform, won't be the same-fashioned accessible/integration-ready if available on Compute engine at all.
We have a 5 years old fully-grown App engine app now.
I am considering a scenario to support high-level of customization/control and adding third party softwares/middlewares to our server environment which is not possible with App engine. So if we have all solutions(Compute Engines, Container Engines etc.) other than App engine, to migrate our application to meet such requirements, what is the cost of such migration?
Need of server provisioning and configuration at Compute engines with different pricing model[Understood, should not be a problem :)]
Full or partial code rewrite to continue using the same APIs esp. Datastore, Cloud Storage, Task Queues, Cron jobs, Document Search, Memcache etc.[Need confirmation here and any reference/link to migration guide would be help!!]
Does this lead to risk of losing any managed service/API offered from App Engine? Document Search, Memcache, Task Queues, Cron jobs seem the possible candidates. Please confirm.
As per my reading, Big Query, Cloud storage, Pub-Sub APIs integration should not be much affected with such migration(Client-libraries or Rest APIs should still help!). Please confirm.
In nut-shell, We wanted it fully managed in the beginning so PaaS seemed the right choice 5 years ago. Now we want App minus platform-managed plus customized/flexible to our choice. How complicated this transition is going to be?
Full or partial code rewrite to continue using the same APIs esp. Datastore, Cloud Storage, Task Queues, Cron jobs, Document Search, Memcache etc.[Need confirmation here and any reference/link to migration guide would be help!!]
Unfortunately, some of those service only be provided on GAE, such as Document Search. But most of service can be use directly for GCP, such as Datastore, Cloud Storage. GAE Flexible Environment is much like GCP environment, so you can read this article first Migration to GAE Flexible Environment
In following articles also have some answer：
How to migrate Google App Engine Project to Compute Engine completely
Google App Engine Blobstore to Google Cloud Storage Migration Tool
Does this lead to risk of losing any managed service/API offered from App Engine? Document Search, Memcache, Task Queues, Cron jobs seem the possible candidates. Please confirm.
Yes, Document Search only available on GAE.
As per my reading, Big Query, Cloud storage, Pub-Sub APIs integration should not be much affected with such migration(Client-libraries or Rest APIs should still help!). Please confirm.
Yes, but you may need to change SDK or library. It dependency on your language and how to call those service by Rest API directly or SDK.
Im creating a Node.js website that probably won't have loads of traffic, and was looking into cheap solutions to host the site. Came across Google cloud services offering free usage for their services with limits. A f1-mirco is more than enough for my needs, but I will happily pay for some usage if it goes over by any chance.
I wanted to setup a linux centOS 7 on GCE (which I already did), and run my application and REST API on it. Now here comes the problem.
I tried to use Google's datastore service, but it sprung an app engine instance and without it datastore won't work.
Is datastore entirely relying on app engine to function?? In the docs, it said if you use any of the client API, it requires app engine. What can I do to not use the client api and query data then? Don't want to use the app engine at the moment or datastore is just not for me then?
Thanks for any help!
Some of the underlying infrastructure of Cloud Datastore and App Engine are still tied together for creation, etc. So while creating an Cloud Datastore database also defines an App Engine instance for the project, it doesn't require you to use it. You don't get charged for App Engine either, unless you decide to deploy an App using it.
You should be totally fine use the Google Cloud Node client library on the f1 micro instance.
What is the benefit of using luceneappengine over the search api provided by google app engine?
The luceneappengine project was born first of all to make easier the migration of an existing application to google app engine, and next to allow you to deploy a standard lucene application in google app engine environment.
The search api must be your first choice if you are developing a new google app engine application, unless if you think to move your application into another servlet container that will lack of search api.
Other considerations can be done on costs: luceneappengine uses the default datastore to save index (it means that will use datastore read and write), search api has a dedicated cost definition. You can find the cost details here. Unfortunately it's hard to predict the cost/operations of a luceneappengine application because it depends on indexed files (quantity, size, indexed fields etc..).
What is better to code with php in google app engine or in amazon-ec2.
I think it is better in amazon-ec2 because they support datastore with php and google app engine doesn't, what do you think ?
While its not possible to access the appengine datastore, google has a new preview service for "cloud datastore", google "cloud datastore from php" and you will see how to use it.
You really cant compare AWS with appengine as one requires managing servers and scalability manually and their noSQL solution (dynamoDB) is a joke compared to google's datastore, for example in dynamoDB you must provision your writes beforehand and even if you are all day changing provisioning, it takes sometimes hours to propagate the new setting.
I had never use Google App Engine, but several times AWS systems, and sure, as AWS EC2 could be used as Linux Server Instance, I recommend you that provider. And coz' it seems that you use PHP, they have strong API for this langage. Have fun with AWS.
I was having a look at the new Google Cloud Datastore and looks great. But there is something I could not understand... is it supposed to substitute Google App Engine Datastore? How can I use it inside GAE? What are the differences between both of them?
I have a GAE app in Java that uses 3 entities with thousands of rows each one, and I need to do joins quite often...
The cloud datastore is the App Engine datastore, for use outside App Engine. You won't get any benefit trying to use it with your App Engine app, unless you need other external apps to also have access to the data.
You certainly won't get more efficient joins. If you really need that, perhaps you should look into Cloud SQL, which is basically a version of MySQL you can use from App Engine.
They both are the same, in fact Google App Engine can use Google Cloud DataStore as one of its way to store data, the other options include Google Cloud SQL, Google Cloud Storage. You may select among these three according to the type of data you want to store and the way you want to access them.
From your question its clear that Google Cloud SQL would be right choice as no other options provide joins for retrieving results.
Cloud SQL is nothing but MySQL(the popular open source database) running on google platform. So you can perform your regular SQL-like queries to get your results.