Question:
Is there a way to Specify Zone for Google Cloud App Engine Flexible Environment to reside in? If not, what are the alternatives?
Context:
I'm having a setup where I use App Engine to write and reads to Bigtable. However I noticed a performance decrease, and during the debugging, I found a documentation from Google stating:
There are issues with the network connection. Network issues can reduce throughput and cause reads and writes to take longer than usual. In particular, you'll see issues if your clients are not running in the same zone as your Cloud Bigtable cluster.
In my case, my client is in a different region, by moving it to the same region had a huge increase in performance. However the performance issue still exist, and the recommendation from the documentation is to put client in the same zone as Bigtable.
I also considered using Container engine or Compute Engine where it is easier to specify the zone, but I want stay with App Engine for its autoscale functionality and managed services.
App Engine is a regional service:
App Engine is regional, which means the infrastructure that runs your
apps is located in a specific region and is managed by Google to be
redundantly available across all the zones within that region.
Taken from here.
You could indeed use GKE or GCE, while you're correct that these are not managed services like App Engine is, they do both support autoscaling.
Related
Our project was running in GCP compute engine. For scaling purpose, it is moved to app engine. We had rabbitmq implemented for push messages and chatbots in compute engine. In app engine it is not feasible to implement rabbitmq. So I was going through alternate options. There I found cloud task option. But I have doubts in certain areas even after reading their documentation
In my understanding, we need an app engine instance for cloud tasks. In that case, can I implement it in same project itself as a different service? Will this affect the performance of the existing project?
Is there any better solution than cloud tasks in this case?
You can implement additional services under your app in the App Engine as shown in this diagram.
By default, App Engine scales your app to match the load. Your apps will scale up the number of instances that are running to provide consistent performance, or scale down to minimize idle instances and reduces costs.
You can consider running a RabbitMQ Cluster on Google Kubernetes Engine. You can find more information in the following documentation: rabbitmq.
I'm trying to setup a web app in Google App Engine standard environment using Flask and MySQL.
I know Google App Engine is capable of scaling it's instance dynamically based on the performance needed but I'm unsure how I can do this for my MySQL servers.
For the MySQL connection I try to connect to Google Cloud SQL. I know how this works for a single instance but I would like to know whether it's possible to connect multiple instances and thereby allow the possiblity to dynamically scale my storage while using the same data.
So far I've found information how to do this on Google Computing Engine instances and Google Cloud SQL via this link but couldn't find out how this would work on Google App Engine standard environment.
Each GAE service instance will connect to your Google Cloud SQL as a separate client, using the same credentials. There's nothing to worry about in this respect, except maybe the cost of scaling/sizing your Google Cloud SQL instances if you have many GAE instances or limiting the number of GAE instances to keep the performance decent. From Best practices (the whole page is probably a useful read):
App Engine has the ability to automatically create more instances when load increases. If the number of App Engine instances increases
beyond what the Cloud SQL instance is sized to handle, your Cloud SQL
instance will not be able to meet the demand. You can avoid this issue
by limiting the maximum number of App Engine instances. For more
information, see Scaling elements.
As #dan-cornilescu correctly mentioned, Google App Engine automatically scales horizontally and handles the connections to your Cloud SQL instance without you having to care about.
But taking a look at your last comment on his answer :
Thank you, I understand that Google App Engine can scale dynamically
and let the created instances connect to one Google Cloud SQL
Instance, but I'm looking for a solution for the limited space and
performance on this Cloud SQL instance. Is there a way I can connect
Google App Engine to multiple instances and so increase the database
space on Google Cloud SQL?
I will go ahead and give you some options in order to address the limited space and performance of your Cloud SQL instance.
Regarding the limited space, when creating your Cloud SQL instance you can check the option “Enable automatic storage increases” under the “Configure machine type and storage” dropdown.
Regarding having multiple instances, remember that Cloud SQL instances scale vertically but not horizontally, so the first option I would recommend you is to switch to Google Cloud Spanner if what you are seeking is horizontal scalability. At the moment is the only database within Google Cloud products that allows to do so and is the choice to go.
You can also “fake out” the horizontal scalability by creating multiple Cloud SQL instances, which is indeed a best practice. While doing so, you only have to take into consideration that the amount of instances allowed per project is 40 and you will have to request an increase if you wish to have more.
In the documentation for App Engine it says:
Meeting your latency, availability, or durability requirements are
primary factors for selecting the region where your apps are run.
You cannot change an app's region after you set it.
In App Engine Standard Environment (with automatic scaling), what should I do if my application starts getting a lot of requests from a region far away from mine? For example if my region is us-east1 but I get a lot of requests from asia?
For some reason I thought that App Engine would spin up new instances in the region the request is coming from.
If wanting to build a globally accessible and fast app, this seems like a big limitation to me. Is GAE standard environment a bad choice then?
For the most part, Google App Engine is designed for small, simple and easy to deploy server instances.
If you need global load balancing and auto scaling, then you will need to select and configure the services yourself. Google's load balancer supports global geolocation based load balancing. You can define, in advance, which regions it will load balance to. Combined with Google Compute Engine and Instance Groups, you can define the global characteristics of your site.
The tradeoff for you is how much effort do you want to spend in planning, deploying and managing your setup. Google App Engine makes this easy, but you are limited in some options. Google Compute Engine takes more effort, but you have a larger set of options to chose from and manage.
Start with some research on Google's Load Balancer to better understand the options available to you.
GOOGLE CLOUD LOAD BALANCING
Im creating a Node.js website that probably won't have loads of traffic, and was looking into cheap solutions to host the site. Came across Google cloud services offering free usage for their services with limits. A f1-mirco is more than enough for my needs, but I will happily pay for some usage if it goes over by any chance.
I wanted to setup a linux centOS 7 on GCE (which I already did), and run my application and REST API on it. Now here comes the problem.
I tried to use Google's datastore service, but it sprung an app engine instance and without it datastore won't work.
Is datastore entirely relying on app engine to function?? In the docs, it said if you use any of the client API, it requires app engine. What can I do to not use the client api and query data then? Don't want to use the app engine at the moment or datastore is just not for me then?
Thanks for any help!
Some of the underlying infrastructure of Cloud Datastore and App Engine are still tied together for creation, etc. So while creating an Cloud Datastore database also defines an App Engine instance for the project, it doesn't require you to use it. You don't get charged for App Engine either, unless you decide to deploy an App using it.
You should be totally fine use the Google Cloud Node client library on the f1 micro instance.
i wanted to create a multiple instances of my application in either google cloud or EC2. I have two queries regarding this
1.How to achieve this?
Can we create a virtual instances by using zookeeper?
Google App Engine instances are started automatically, as your traffic raises. You may also have always on instances or backends instaces. Just read the docs: http://code.google.com/intl/pt-BR/appengine/docs/adminconsole/instances.html
Google App Engine is not adequate to use with Zookeper. Since Java code runs in a limited sandbox, you may not be able to communicate with Zookeper at all. Also, you will have to start and end you backends programmatically, leading you to lots of work.
As for EC2, see this:
http://www.mail-archive.com/zookeeper-user#hadoop.apache.org/msg01083.html