I have a web app running packed in Docker and running in Google App Engine Flexible Environment.
Now I have a worker which also is packed in Docker, this one doesn't have any HTTP listeners.
Is it possible to run both containers in the same Google App Engine app?
In Google App Engine, currently there is no functionality to compose two or more docker images as a single application, like Docker compose or pods in Kubernetes.
I recommend having a look at This question and answer.
Basically the only option is multi-process container (using a startup script that starts both of your services).
Related
I have created a web app which uses react as frontend and django as backend. I have also added nginx proxy to my both backend and frontend. I use docker compose to build and start all of my containers and everything works perfectly.
Now i want to deploy it to google app engine and I have no idea how to do that. I found this well written article, but it uses aws. I want to use app engine because its free (for now).
It would be really helpful if someone could guide me through this.
App Engine Standard does not support containers but App Engine Flex does using Custom Runtimes.
Anyway, App Engine Flex does not support docker compose so you need to decouple your app into individual services (react, django, nginx (not necessary at all since GAE already has an NGINX proxy)) by using your own container for each element or use the already available runtimes when applies.
Then you can connect each other by using the App Engine URLs. This article can help you understand this approach (it is for NodeJS but the idea is the same).
App engine supports specific runtimes, not dockerized applications. You would need to deploy a python app engine app for the Django backend and a nodejs app for the react app.
How can I deploy multiple applications from same or different language/runtime originating from a single project in google cloud app engine?
Deploying Multiple Services to Google Cloud App engine.
Create the following files in the app root directory
Create app.yaml file with content:
runtime: nodejs14
service: default
Create myserviceone.yaml file with content:
runtime: nodejs14
service: myserviceone
Create myservicetwo.yaml file with content:
runtime: nodejs14
service: myservicetwo
Deploy using command
gcloud app deploy --project your_project_id app.yaml myserviceone.yaml myservicetwo.yaml
This will deploy the default service as well as my service one and my service two. Check out your cloud console.
You can access them using:
https://myserviceone-dot-yourProjectID.appspot.com/ or
http://myserviceone.yourProjectID.appspot.com/
https://myservicetwo-dot-yourProjectID.appspot.com/ or
http://myservicetwo.yourProjectID.appspot.com/
Today, App Engine has a one-to-one correspondence with a Google Cloud Console project. You cannot deploy multiple "apps" in the same project. However, you might still be able to do what you want depending on your application(s).
App Engine has the concept of "services," which are independent aspects of your application. Your App Engine app can have many services and each service can have its own language/runtime and even be on different App Engine environments.
You could have say a Python service on App Engine standard environment that is used to serve your simple Flask site, could have service that serves an API written in Java 8 on the Standard Environment, and could have yet another service in say Node.js running in the App Engine flexible environment.
Your "default" service is defined in your app.yaml file. Your other services can be defined in different folders and can have either an app.yaml with their service definitions or you could name them something else like backend.yaml. See this simple Python project for the layout of the configuration files.
For more conceptual information about services on App Engine, see Microservices Architecture on Google App Engine.
I am currently experimenting with the Google App Engine flexible environment, especially the feature allowing you to build custom runtimes by providing a Dockerfile.
Docker provides a really nice feature called docker-compose for defining and running multi-container Docker applications.
Now the question is, is there any way one can use the power of docker-compose within GAE? If the answer is no, what would be the best approach for deploying a multi-container application (for instance Nginx + PHP-FPM + RabbitMQ + Elasticsearch + Redis + MongoDB, ...) within GAE flexible environment using Docker?
It is not possible at this time to use docker-compose to have multiple application containers within a single App Engine instance. This does seem however to be by design.
Scaling application components independently
If you would like to have multiple application containers, you would need to deploy them as separate App Engine services. There would still only be a single application container per service instance but there could be multiple instances of each service. This would grant you the flexibility you seek of scaling each application component independently. In addition, if the application in a container were to hang, it could not affect other services as they would reside in different VMs.
An added benefit of deploying each component as a separate service is that one need not use the flexible environment for every service. For some very small tasks such as API backends or serving relatively slow-changing web content, the standard environment may suffice and may be less expensive at low resource levels.
Communication between components
Since one of your comments mentions getting instance IPs, I thought you might find inter-service communication useful. I'm not certain for what reason you wish to use VM instance IPs but a typical use case might be to communicate between instances or services. To do this without instance IPs, your best bet is to issue HTTP request from one service to another simply using the appropriate url. If you have a service called web and one called api, the web service can issue a request to api.mycustomdomain.com where your application is hosted and the api service will receive a request with the X-Appengine-Inbound-Appid header specified with your project ID. This can serve as a way a identifying the request as coming from your own application.
Multicontainer application using Docker
You mention many examples of applications including NGinx, PHP-FPM, RabbitMQ, etc.. With App Engine using custom runtimes, you can deploy any container to handle traffic as long as it responds to requests from port 8080. Keep in mind that the primary purpose of the application is to serve responses. The instances should be designed to start up and shut down quickly to be horizontally scalable. They should not be used to store any application data. That should remain outside of App Engine using tools like Cloud SQL, Cloud Datastore, BigQuery or your own Redis instance running on Compute Engine.
I hope this clarifies a few things and answers your questions.
You can follow following steps to create a container with docker-compose file in Google App Engine.
Follow link
You can build your custom image using docker-compose file
docker-compose build
Create a tag for local build
docker tag [SOURCE_IMAGE] [HOSTNAME]/[PROJECT-ID]/[IMAGE]
Push image to google registry
docker push [HOSTNAME]/[PROJECT-ID]/[IMAGE]
deploy Container
gcloud app deploy --image-url=[HOSTNAME]/[PROJECT-ID]/[IMAGE]
please add auth for docker commands to run.
The title says it all. Just to irritate DRY purists, I'll say it again.
Is it possible to get a list of instances my application is running on via gcloud?
In 2019, You can check the newest document : gcloud app services list - list your existing services
The newest command is:
gcloud app services list
The result will be something like this.
SERVICE NUM_VERSIONS
default 7
App Engine can mean traditional App Engine or Managed VMs, depending on the context. There is only one App Engine app per project, but you can have multiple modules (although only one of those module will serve web traffic, the rest are backend only) and multiple versions of each module.
As pgiecek mention,
gcloud preview app modules list
will show all the modules.
What you might be looking for is all your App Engine versions, for which you will still have to use appcfg
appcfg.py -A <your-project-id> [-M <module>] list_versions
If you are using Managed VMs, it will spin up your app on GCE instances, in which case
gcloud compute instances list
does in fact show your App Engine (MVM) instances.
It seems that currently it is not possible. At the moment gcloud tool supports App Engine (via Preview CLI command groups) but provides only limited number of operations and furthermore they are in BETA. You can check documentation here.
You can only list your existing deployed modules and versions as follows.
gcloud preview app modules list
As of Sep 2019 (Google Cloud SDK 237.0.0):
gcloud app instances list
Hi I like to know if it is possible to run phantomjs on Google App Engine.
My use case is to use Phantomjs to take snapshots of html pages served on GAE using a cron job.
Has this been done before ?
Google App Engine supports a number of language runtimes (Java, Python, PHP and Go), but you can use the Managed VMs feature of Google App Engine to run anything in a Docker container (such as PhantomJS), and you can also use Google Compute Engine to get a VM where you can also install anything you would like, in any language.
For reference, here's a tutorial for running Node.js on Managed VMs to help guide you in the right direction.
Alternatively, Google Cloud Run lets you run arbitrary stateless containers pn a serverless platform, so you can run your PhantomJS app there if you wish.
You cannot run phantomjs on Google App Engine.
Currently, Google App Engine supports Java, Python, PHP, and Go. (https://cloud.google.com/appengine/kb/?hl=en)
As #artjom-b mentioned you can use some services that aren't written in the supported languages (like memcache), but phantomjs is not one of them.
If you are asking about testing an application that you are building on Google App Engine, it is possible to run phantomjs tests against a google app engine application. You would just need to run phantomjs on a different service, such as an EC2 instance.