i wanted to create a multiple instances of my application in either google cloud or EC2. I have two queries regarding this
1.How to achieve this?
Can we create a virtual instances by using zookeeper?
Google App Engine instances are started automatically, as your traffic raises. You may also have always on instances or backends instaces. Just read the docs: http://code.google.com/intl/pt-BR/appengine/docs/adminconsole/instances.html
Google App Engine is not adequate to use with Zookeper. Since Java code runs in a limited sandbox, you may not be able to communicate with Zookeper at all. Also, you will have to start and end you backends programmatically, leading you to lots of work.
As for EC2, see this:
http://www.mail-archive.com/zookeeper-user#hadoop.apache.org/msg01083.html
Related
I was wondering how many threads can app engine support since I need it to calculate how many WebSocket connections I can support per instance.
Somewhere I read there are no threads, somewhere that you can use but no max number listed.
So can a single app engine instance support for example tomcats default 200 threads?
And if not what GCP service do you recommend for hosting such an app?
To create a connection with websocket you can have a look at this documentation.
And about the number of thread limits, you can specify by defining max_concurrent_requests as discussed in similar thread. It is also defined in the document, that the maximum limit of max_concurrent_requests is 1000.
If you are willing to check on other services provided by GCP, you can choose Cloud run or Google Kubernetes engine for containerized applications as mentioned here and also you can consider an option of installing tomcat on virtual machine.
Is it possible to run side containers or init containers in Google Cloud Run or App Engine? I couldn't find any documentation of this and trying to ask this on GCP support forums I was directed to ask at stack-overflow. If possible how would you accomplish this? I came across this repo but it wasn't helpful.
I know it is possible with GKS but trying to do the same with these services.
You can't, for now, running a Pod on Cloud Run, you can only run a container. However, it's possible to run a multi process container for helping you to achieve this.
You can find here and here a post from Ahmet for running several process in the same container.
Note: Ahmet is one of Cloud Run engineers at Google, you can rely on his articles!
As John mentioned, the answer is no. Cloud run and App Engine both provide you a simple way to deploy a containerized application int the cloud and have it run.
To keep this as simple and streamlined as possible, additional features that you see in pod specs are not available (such as init containers or running multiple containers).
For more complex deployments, it is recommended to use GKE
Question:
Is there a way to Specify Zone for Google Cloud App Engine Flexible Environment to reside in? If not, what are the alternatives?
Context:
I'm having a setup where I use App Engine to write and reads to Bigtable. However I noticed a performance decrease, and during the debugging, I found a documentation from Google stating:
There are issues with the network connection. Network issues can reduce throughput and cause reads and writes to take longer than usual. In particular, you'll see issues if your clients are not running in the same zone as your Cloud Bigtable cluster.
In my case, my client is in a different region, by moving it to the same region had a huge increase in performance. However the performance issue still exist, and the recommendation from the documentation is to put client in the same zone as Bigtable.
I also considered using Container engine or Compute Engine where it is easier to specify the zone, but I want stay with App Engine for its autoscale functionality and managed services.
App Engine is a regional service:
App Engine is regional, which means the infrastructure that runs your
apps is located in a specific region and is managed by Google to be
redundantly available across all the zones within that region.
Taken from here.
You could indeed use GKE or GCE, while you're correct that these are not managed services like App Engine is, they do both support autoscaling.
Is it possible to have 2 different applications (one written in Python and one in Java) in the same App Engine environment (Standard) and have them view the same Datastore?
From my understanding, Google Cloud Datastore is the storage option that comes along with any AppEngine application, but also exists as an external service any application can use.
Is there a way for one application to view the other application's "embedded" Datastore (without one application exposing an API to it)? If not, how can 2 different applications use the same storage? I haven't been able to find any documentation regarding Cloud Datastore urls or using them as a third party database.
You probably can use the REST API version https://cloud.google.com/datastore/docs/apis. The real question is, why do you need to 2 different apps here? Micro service is usually implemented in appengine using module on a single app, https://cloud.google.com/appengine/docs/standard/python/microservices-on-app-engine.
I am currently experimenting with the Google App Engine flexible environment, especially the feature allowing you to build custom runtimes by providing a Dockerfile.
Docker provides a really nice feature called docker-compose for defining and running multi-container Docker applications.
Now the question is, is there any way one can use the power of docker-compose within GAE? If the answer is no, what would be the best approach for deploying a multi-container application (for instance Nginx + PHP-FPM + RabbitMQ + Elasticsearch + Redis + MongoDB, ...) within GAE flexible environment using Docker?
It is not possible at this time to use docker-compose to have multiple application containers within a single App Engine instance. This does seem however to be by design.
Scaling application components independently
If you would like to have multiple application containers, you would need to deploy them as separate App Engine services. There would still only be a single application container per service instance but there could be multiple instances of each service. This would grant you the flexibility you seek of scaling each application component independently. In addition, if the application in a container were to hang, it could not affect other services as they would reside in different VMs.
An added benefit of deploying each component as a separate service is that one need not use the flexible environment for every service. For some very small tasks such as API backends or serving relatively slow-changing web content, the standard environment may suffice and may be less expensive at low resource levels.
Communication between components
Since one of your comments mentions getting instance IPs, I thought you might find inter-service communication useful. I'm not certain for what reason you wish to use VM instance IPs but a typical use case might be to communicate between instances or services. To do this without instance IPs, your best bet is to issue HTTP request from one service to another simply using the appropriate url. If you have a service called web and one called api, the web service can issue a request to api.mycustomdomain.com where your application is hosted and the api service will receive a request with the X-Appengine-Inbound-Appid header specified with your project ID. This can serve as a way a identifying the request as coming from your own application.
Multicontainer application using Docker
You mention many examples of applications including NGinx, PHP-FPM, RabbitMQ, etc.. With App Engine using custom runtimes, you can deploy any container to handle traffic as long as it responds to requests from port 8080. Keep in mind that the primary purpose of the application is to serve responses. The instances should be designed to start up and shut down quickly to be horizontally scalable. They should not be used to store any application data. That should remain outside of App Engine using tools like Cloud SQL, Cloud Datastore, BigQuery or your own Redis instance running on Compute Engine.
I hope this clarifies a few things and answers your questions.
You can follow following steps to create a container with docker-compose file in Google App Engine.
Follow link
You can build your custom image using docker-compose file
docker-compose build
Create a tag for local build
docker tag [SOURCE_IMAGE] [HOSTNAME]/[PROJECT-ID]/[IMAGE]
Push image to google registry
docker push [HOSTNAME]/[PROJECT-ID]/[IMAGE]
deploy Container
gcloud app deploy --image-url=[HOSTNAME]/[PROJECT-ID]/[IMAGE]
please add auth for docker commands to run.