I am currently experimenting with the Google App Engine flexible environment, especially the feature allowing you to build custom runtimes by providing a Dockerfile.
Docker provides a really nice feature called docker-compose for defining and running multi-container Docker applications.
Now the question is, is there any way one can use the power of docker-compose within GAE? If the answer is no, what would be the best approach for deploying a multi-container application (for instance Nginx + PHP-FPM + RabbitMQ + Elasticsearch + Redis + MongoDB, ...) within GAE flexible environment using Docker?
It is not possible at this time to use docker-compose to have multiple application containers within a single App Engine instance. This does seem however to be by design.
Scaling application components independently
If you would like to have multiple application containers, you would need to deploy them as separate App Engine services. There would still only be a single application container per service instance but there could be multiple instances of each service. This would grant you the flexibility you seek of scaling each application component independently. In addition, if the application in a container were to hang, it could not affect other services as they would reside in different VMs.
An added benefit of deploying each component as a separate service is that one need not use the flexible environment for every service. For some very small tasks such as API backends or serving relatively slow-changing web content, the standard environment may suffice and may be less expensive at low resource levels.
Communication between components
Since one of your comments mentions getting instance IPs, I thought you might find inter-service communication useful. I'm not certain for what reason you wish to use VM instance IPs but a typical use case might be to communicate between instances or services. To do this without instance IPs, your best bet is to issue HTTP request from one service to another simply using the appropriate url. If you have a service called web and one called api, the web service can issue a request to api.mycustomdomain.com where your application is hosted and the api service will receive a request with the X-Appengine-Inbound-Appid header specified with your project ID. This can serve as a way a identifying the request as coming from your own application.
Multicontainer application using Docker
You mention many examples of applications including NGinx, PHP-FPM, RabbitMQ, etc.. With App Engine using custom runtimes, you can deploy any container to handle traffic as long as it responds to requests from port 8080. Keep in mind that the primary purpose of the application is to serve responses. The instances should be designed to start up and shut down quickly to be horizontally scalable. They should not be used to store any application data. That should remain outside of App Engine using tools like Cloud SQL, Cloud Datastore, BigQuery or your own Redis instance running on Compute Engine.
I hope this clarifies a few things and answers your questions.
You can follow following steps to create a container with docker-compose file in Google App Engine.
Follow link
You can build your custom image using docker-compose file
docker-compose build
Create a tag for local build
docker tag [SOURCE_IMAGE] [HOSTNAME]/[PROJECT-ID]/[IMAGE]
Push image to google registry
docker push [HOSTNAME]/[PROJECT-ID]/[IMAGE]
deploy Container
gcloud app deploy --image-url=[HOSTNAME]/[PROJECT-ID]/[IMAGE]
please add auth for docker commands to run.
Related
I'm new to GCP and currently, I try to deploy all my applications on their services.
For an application in a single container I use CloudRun which I already really like.
Now I want to deploy an application that uses Angular in the frontend and spring in the backend and a SQL-DB (Postgres). Every part of this is in its own separated container.
Should I use for this purpose although CloudRun or does GCP have more suitable services I should consider to use if I want to host a scalable and serverless Application? So is there such a thing as the best practice for Frontend-backend architecture applications on GCP?
I recommend you to use these services:
Cloud SQL to host the database. It's managed for you and efficient
Cloud Run for the business tier (the spring application).
Be careful, with spring the cold start can take several seconds. I wrote an article on this (the article is quite old and the perf are now better on Cloud Run, but the latency on the first request exists (and take 5 - 7s for an hello world container)). Add several CPU (4 is a good number) to speed up the cold start or use the --min-instance parameter for this (or other solution that you can find in one of my articles)
For the front end, I recommend you to host the static files on Cloud Storage.
To serve this on internet, put a Load Balancer in front of this
Create a serverless network endpoint group (NEG) for Cloud Run service
Create a Cloud Storage backend to serve the static files.
Use the domain that you want and serve it in SSL
Optionally, use CDN to cache your static files.
CloudRun runs stateless containers. It doesn't make a distinction between frontend and backend, or worker jobs.
You can run your frontend, backend, admin code base as Cloud Run service.
Next to these you setup Cloud SQL for your operational database, and connect the Cloud Run services with the Cloud SQL connector so they are able to use for read/write queries.
I'm looking at moving my application to google cloud and I'm having a hard time understanding how best to organize my project. It seems like you can only have one App Engine application per project with services available to support a microservices architecture and instances representing the App Engine instances created via auto scaling.
What is the correct way to build an App Engine for my API server and an App Engine for my Web Server? Do I need to have a project for each? I'm essentially trying to accomplish the following:
It is straightforward to have a single GAE project implement both your website and an API. You can even do this within a single service. You could put each in a separate service, and the advantage of that is that you can update one without updating the other. For small projects, a disadvantage is that two services are more expensive than one (though GAE is quite inexpensive overall).
For prod vs dev, you'll need to explain your requirements a little more, but here are some thoughts.
Each GAE service has multiple versions. You can deploy your production version to www.mycompany.com and deploy a dev version to dev-dot-myapp.appspot.com (that's the way GAE does URLs for versions of your app). Both of these versions will access the same datastore so you need to be careful with the dev version so that it doesn't mess up your prod implementation.
If you have a dev situation that is bleeding edge and shouldn't be able to access the datastore of your production app, then you would create a different GAE project for that.
Here is a way to visualize it:
Google Cloud Project A
GAE Project A
production www service
production API service (this could be combined with production www service)
dev www service
dev API service
Google Cloud Project B
GAE Project B
bleeding edge www service
bleeding edge API service
Best Practices:
Your Dev and Prod should be in separate projects for both security and billing purposes.
App Engine:
You can only have one App Engine per project. This will create a problem for you to use App Engine for both API Server and Web Server. In this case, I would not use App Engine at all and instead look at Containers on Compute Engine or go for Kubernetes.
Even a single node Kubernetes Cluster will shock you with its flexibility and power. Containers on Compute Engine still have a lot flexibility and power too. If you like the concepts of App Engine Flexible, then you might really like containers. The exception here is that App Engine makes some concepts brain dead simple where you have more work in configuration for Containers or Kubernetes.
I am trying to run google app engine with memcache locally with go run *.go and I am getting the following response:
service bridge HTTP failed: Post http://appengine.googleapis.internal:10001/rpc_http: dial tcp: lookup appengine.googleapis.internal: no such host**
What is the proper way to run locally?
The library that you are trying to use is a App Engine standard environment library that is not available in the flexible environment. The flexible environment is built on top of Compute Engine and a very different system. You could try using MemoryStore for Redis. You also might be able to break your application into microservices. An App Engine project can have services on different runtime environments (mix of flex/std, mix of languages), but you'll have to be careful about how you communicate and work across the services. Its not necessarily a trivial thing to rearchitect your app though.
You have a Google Cloud project Project1 with App Engine enabled and serving your legacy application.
You want to rebuild your application using a different programming language and hence need a separate project (Project2) with its own separate App Engine.
You want to install some kind of reverse proxy appliance in front of your Project1 App Engine, so that you can gradually route more and more traffic to Project2's App Engine.
What's the best way to achieve this?
Google Cloud Load Balancer doesn't appear to work with App Engine (only Google Compute Engine), so I'm thinking establishing a reverse proxy server (e.g. NginX) container would be the best bet.
Depending on how you want to organize, you can duplicate your entire application within the same project by just deploying as a different service or as new versions of your existing service. For example, say you have two services frontend and backend for a simple web app. (Let's say they're both written in Python)
For deploying as a separate service, you can create a new version of your application in a different language, say Node.js. You can deploy the new services as frontend-node and backend-node.
If you don't want to do that, you can rewrite your application in a different language and then deploy as a new version of your existing services. In the "Versions" section, you can see your versions of the same service and they can be different language runtimes.
Either way, you can then use the "split-traffic" feature to customize and test implementations of your application.
Generally speaking, you should avoid using projects to isolate different variants and/or components of your application unless you really need too. App Engine services can each be a different runtime from each other so there's almost no point in provisioning a whole new project even though you're redeploying in a different language.
i wanted to create a multiple instances of my application in either google cloud or EC2. I have two queries regarding this
1.How to achieve this?
Can we create a virtual instances by using zookeeper?
Google App Engine instances are started automatically, as your traffic raises. You may also have always on instances or backends instaces. Just read the docs: http://code.google.com/intl/pt-BR/appengine/docs/adminconsole/instances.html
Google App Engine is not adequate to use with Zookeper. Since Java code runs in a limited sandbox, you may not be able to communicate with Zookeper at all. Also, you will have to start and end you backends programmatically, leading you to lots of work.
As for EC2, see this:
http://www.mail-archive.com/zookeeper-user#hadoop.apache.org/msg01083.html