Can't deploy two-service docker image to google app engine - google-app-engine

I've built a docker image which consists of two parts:
simple nodejs app which is listening to port 8080
haskell service which is using snap framework (port 8000)
I know that it's better to run those two parts in different containers, but there is a reason to keep them in one. So I found a way how to run two services in one container with the use of supervisord.
In the dockerfile I expose 8080, and when I run the docker image locally it works just fine. I can make POST requests to nodejs app, which in its turn is making POST request to the haskellmodule using port 8000. I run it with the following command:
docker run -p 8080:8080 image_name
So I pushed the image to google container registry and deployed it with the use of --image-url flag. The deployment process goes without any error, though after that I cannot reach my app. If I look to the running version's logs, I see the following:
A /usr/lib/python2.7/dist-packages/supervisor/options.py:296: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security.
A 'Supervisord is running as root and it is searching '
A 2017-10-08 14:08:45,368 CRIT Supervisor running as root (no user in config file)
A 2017-10-08 14:08:45,368 WARN Included extra file "/etc/supervisor/conf.d/supervisord.conf" during parsing
A 2017-10-08 14:08:45,423 INFO RPC interface 'supervisor' initialized
A 2017-10-08 14:08:45,423 CRIT Server 'unix_http_server' running without any HTTP authentication checking
A 2017-10-08 14:08:45,424 INFO supervisord started with pid 1
A 2017-10-08 14:08:46,425 INFO spawned: 'haskellmodule' with pid 7
A 2017-10-08 14:08:46,427 INFO spawned: 'nodesrv' with pid 8
A 2017-10-08 14:08:47,429 INFO success: haskellmodule entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
A 2017-10-08 14:08:47,429 INFO success: nodesrv entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
A 2017-10-08 14:13:49,124 WARN received SIGTERM indicating exit request
A 2017-10-08 14:13:49,127 INFO waiting for haskellmodule, nodesrv to die
A 2017-10-08 14:13:49,128 INFO stopped: nodesrv (terminated by SIGTERM)
A 2017-10-08 14:13:49,138 INFO stopped: haskellmodule (terminated by SIGTERM)
Then it starts over and everything is repeated over and over again.
My Dockerfile:
FROM node:latest
RUN apt-get update
RUN curl -sSL https://get.haskellstack.org/ | sh
COPY ./nodesrv /nodesrv
COPY ./haskellmodule /haskellmodule
RUN mkdir /log
WORKDIR /haskellmodule
RUN stack build
WORKDIR /
RUN apt-get update && apt-get install -y supervisor
ADD ./configs/supervisord.conf /etc/supervisor/conf.d/supervisord.conf
EXPOSE 8080
ENTRYPOINT ["/usr/bin/supervisord"]
My supervisord config:
[supervisord]
nodaemon=true
[program:nodesrv]
command=node index.js
directory=/nodesrv/
user=root
[program:haskellmodule]
command=stack exec haskellmodule-exe
directory=/haskellmodule/
user=root
My app.yaml file I use for deployment:
runtime: custom
env: flex
So seems like google app engine is shutting supervisor down (taking into account that everything is working on localhost). What could be a reason of that?
Thanks in advance

You need to configure your app.yaml file to open ports 8080 and 8000. You need to do this in addition to opening the port in your Dockerfile with EXPOSE. The documentation for how to setup your app.yaml file is located here, and the example from the docs is copied below:
Add the following to your app.yaml:
network:
instance_tag: TAG_NAME
name: NETWORK_NAME
subnetwork_name: SUBNETWORK_NAME
forwarded_ports:
- PORT
- HOST_PORT:CONTAINER_PORT
- PORT/tcp
- HOST_PORT:CONTAINER_PORT/udp

Run supervisord with -n argument.
This will run supervisor in foreground.
Works fine for me in app engine flexible environment.
Thanks

Related

error on start nginx service on ubuntu vps

i am begginer user of vps, i have a reactJS app, and i wnat to deploy it on my ubuntu 18 vps with nginx.
I have followed the stpes of this tutorial Deploying create-react-app with Nginx and Ubuntu
i had already check all the steps, but when i put the command
sudo service nginx start
the system show's me, the next error message:
Job for nginx.service failed because the control process exited with error code.
See "systemctl status nginx.service" and "journalctl -xe" for details.
and when i put "journalctl -xe" shows me this:
nginx, error
ngnix, error
please help me friends
Look in your log file on messages before the error "Failed to startup nginx"
You will see the reason of problem.
bind() to 0.0.0.0:80 failed (98: Address already in use)
Looks like port 80 on your vps server is already in use by some application.
Port 80 used for HTTP services.
So most likely you already have run Apache HTTP server, or some other.
use this command to see what application use it
sudo netstat -tulpn | grep ":80"
If you see apache
tcp6 0 0 :::80 :::* LISTEN 349/apache2
then you can stop apache
# apache service name also can be httpd (use right command)
# sudo service httpd stop
sudo service apache stop
and run nginx
sudo service nginx start
But you should be sure that you don't use apache for another website.

DC/OS installation failure during preflight

I am using 5 cloud-based VMs to install DC/OS
1 mesos master
3 mesos agent
1 launching VM
I have installed Docker on my launching VM and start installing DC/OS. It is running successfully during install_prereqs stage without any errors. But it's failing during preflight with below errors for each of my VM system.
STDERR:
Connection to 129.114.18.235 closed.
STDOUT:
Running preflight checks /opt/dcos_install_tmp/dcos_install.sh: line 225: getenforce: command not found
Checking if docker is installed and in PATH: FAIL
Checking if unzip is installed and in PATH: FAIL
Checking if ipset is installed and in PATH: FAIL
Checking if systemd-notify is installed and in PATH: FAIL
/opt/dcos_install_tmp/dcos_install.sh: line 387: systemctl: command not found
Checking if systemctl is installed and in PATH: FAIL
Checking Docker is configured with a production storage driver: /opt/dcos_install_tmp/dcos_install.sh: line 285: docker: command not found
Do I need to install all the required software into my master and agents VMS? Please guide.
We have a similar setup but using straight vm's. We found docker needs to be running on all nodes, including masters, before running the install. Also, make sure you look at: /etc/sysconfig/docker-storageand have: DOCKER_STORAGE_OPTIONS= -s overlayset in the file on all nodes.
I don't believe this is the production setup but should get you running. You also may want to check the privilege of the user executing the install on the remote nodes, does it have permission to see/run systemctl?
I had the same error with the DC/OS web installer in version 1.9
I solved the error after double-checking the bootstraps machines's private key in the web form. To create the key, log into the bootstrap machine and run:
$ ssh-keygen -t rsa
$ for i in `cat dcos-ips.txt`; do ssh-copy-id root#$i; done
$ cat ~/.ssh/id_rsa

Google cloud sdks doesn't start preview for Docker image Mac OS

My app.yaml
runtime: custom
vm: true
api_version: 1
health_check:
enable_health_check: False
Dockerfile
# Use the official go docker image built on debian.
FROM golang:1.5.1
# Grab the source code and add it to the workspace.
ADD . /go/
# Install revel and the revel CLI.
RUN go get github.com/revel/revel
RUN go get github.com/revel/cmd/revel
# Use the revel CLI to start up our application.
ENTRYPOINT revel run 4quorum-appengine dev 8080
# Open up the port where the app is running.
EXPOSE 8080
I was working through this article
http://jbeckwith.com/2015/05/08/docker-revel-appengine/
Preview
I am trying to preview it:
gcloud preview app run app.yaml --custom-entrypoint "revel run 4quorum-appengine dev 8080"
WARNING: The `app run` command is deprecated and will soon be removed.
Please use dev_appserver.py (in the same directory as the `gcloud` command) instead.
Module [default] found in file [/Users/802619/Projects/src/4quorum_root/app.yaml]
INFO: Looking for the Dockerfile in /Users/802619/Projects/src/4quorum_root
INFO: Using Dockerfile found in /Users/802619/Projects/src/4quorum_root
INFO 2015-11-06 18:03:44,226 application_configuration.py:399] No version specified. Generated version id: 20151106t180344
INFO 2015-11-06 18:03:44,226 devappserver2.py:763] Skipping SDK update check.
INFO 2015-11-06 18:03:44,266 api_server.py:205] Starting API server at: http://localhost:62780
INFO 2015-11-06 18:03:44,272 dispatcher.py:197] Starting module "default" running at: http://localhost:8080
INFO 2015-11-06 18:03:44,277 admin_server.py:116] Starting admin server at: http://localhost:8000
ERROR 2015-11-06 18:03:44,282 instance.py:280] [Errno 2] No such file or directory
The same thing if trying dev_appserver.py
Deploy
Deploy also doesn't work. Fails because of timeout.
gcloud preview app deploy ./app.yaml
WARNING: Soon, deployments will set the deployed version to receive all traffic by
default.
To keep the current behavior (where new deployments do not receive any traffic),
use the `--no-promote` flag or run the following command:
$ gcloud config set app/promote_by_default false
To adopt the new behavior early, use the `--promote` flag or run the following
command:
$ gcloud config set app/promote_by_default true
Either passing one of the new flags or setting one of these properties will
silence this message.
You are about to deploy the following modules:
- vaulted-gift-112113/default (from [/Users/802619/Projects/src/4quorum_root/app.yaml])
Deployed URL: [https://20151106t204027-dot-vaulted-gift- 112113.appspot.com]
(add --promote if you also want to make this module available from
[https://vaulted-gift-112113.appspot.com])
Beginning deployment...
Verifying that Managed VMs are enabled and ready.
Provisioning remote build service.
Copying certificates for secure access. You may be prompted to create an SSH keypair.
Building and pushing image for module [default]
Saving [.dockerignore] to [/Users/802619/Projects/src/4quorum_root].
----------------------------- DOCKER BUILD OUTPUT ------------------------------
Step 0 : FROM golang:1.5.1
---> f6271e8f3723
Step 1 : ADD . /go/
---> 94fafc5e8a30
Removing intermediate container cfbe197f6e93
Step 2 : RUN go get github.com/revel/revel
---> Running in d7ad8c923144
---> b65877cf3049
Removing intermediate container d7ad8c923144
Step 3 : RUN go get github.com/revel/cmd/revel
---> Running in 2a9b3320ce47
---> 428defd008f3
Removing intermediate container 2a9b3320ce47
Step 4 : ENTRYPOINT revel run 4quorum-appengine dev 8080
---> Running in 8b9e38ec69ec
---> 3749ee8a6636
Removing intermediate container 8b9e38ec69ec
Step 5 : EXPOSE 8080
---> Running in a0e6c66b56c8
---> dafff62b9643
Removing intermediate container a0e6c66b56c8
Successfully built dafff62b9643
--------------------------------------------------------------------------------
Copying files to Google Cloud Storage...
Synchronizing files to [gs://staging.vaulted-gift-112113.appspot.com/].
Updating module [default]...|Deleted [https://www.googleapis.com/compute/v1/projects/vaulted-gift- 112113/zones/us-central1-f/instances/gae-builder-vm-20151106t204027].
Updating module [default]...failed.
ERROR: (gcloud.preview.app.deploy) Error Response: [4] Timed out creating VMs.
About to drop this.
Moved to heroku. Google App Engine is not ready yet.

Time out error when trying to create Google managed vm

I'm trying to create a managed vm for my node 4 application using google custom runtime.
I created the following Dockerfile:
FROM node:4.2.1
ENV PORT 8080
ADD package.json package.json
RUN npm install
ADD . .
CMD [ "npm", "start" ]
Along with this app.yaml:
# [START runtime]
runtime: custom
vm: true
api_version: 1
# [END runtime]
health_check:
enable_health_check: false
skip_files:
- ^(.*/)?#.*#$
- ^(.*/)?.*~$
- ^(.*/)?.*\.py[co]$
- ^(.*/)?.*/RCS/.*$
- ^(.*/)?\..*$
- ^(.*/)?.*/node_modules/.*$
- ^(.*/)?.*\.log$
I deploy the app using gcloud preview command:
gcloud preview app deploy app.yaml --promote
It seems like the docker is being built correctly but the at the end of the process I get this message:
Copying files to Google Cloud Storage...
Synchronizing files to [gs://staging.my-project-id.appspot.com/].
Updating module [default]...\Deleted [https://www.googleapis.com/compute/v1/projects/my-project-id/zones/us-central1-f/instances/gae-builder-vm-20151030t142257].
Updating module [default]...failed.
ERROR: (gcloud.preview.app.deploy) Error Response: [4] Timed out creating VMs.
I have my deployment working now. I have had to troubleshoot the same problem before, for another project, but I didn't have the code on hand, so I had to work through the problems again.
The deployment ran smoothly up until the last steps, where updating the module would timeout. This made me think it was something to do with the application starting up on VM and not responding appropriately, so the final hook would time out.
You'll find a lot of information here - https://cloud.google.com/appengine/docs/managed-vms/config . I checked the following things:
logging - ensure that you are writing to the correct log file. See https://cloud.google.com/appengine/docs/managed-vms/custom-runtimes#logging
ensure you have a .dockerignore file and are skipping files in app.yaml so you are not asking the process to copy across unneeded node_modules or log files
turn off health checking if you are not using it, or ensure you have the correct express.js routes configured for it
check that your environment variables are set and match what GAE can use. This was my final step - GAE will let you bind to a VM port on 8080. I had to pass through a NODE_ENV flag in my app.yaml which told the app to use 8080 and not 3000.
Lift the resources of the GAE instance in app.yaml. I specified two logical CPUs and made the ram 2 gig.
Good luck.

How can I debug a Dart AppEngine app

I'm now able to run a Dart app using
gcloud --verbosity debug preview app run app.yaml
and also to deploy and run on AppEngine
gcloud --verbosity debug preview app deploy app.yaml
but I haven't found a way to connect a debugger to the Dart app running on the development server.
I found http://dartbug.com/21067 but still couldn't find a way to make it work.
See also https://groups.google.com/a/dartlang.org/forum/#!topic/cloud/OK1nJtg7AjQ
Update 2015-02-27
The app can be run without Docker and then be debugged like any Dart command line application:
Source. https://groups.google.com/a/dartlang.org/d/msg/cloud/zrxgOHFz_lA/q5CdLLQPBAgJ
The API server is part of the App Engine SDK, and we are using it for
running tests in the appengine package. If you look at
https://github.com/dart-lang/appengine/blob/master/tool/run_tests.sh
you will see that it expects the environment variable
APPENGINE_API_SERVER.
The API server is in /platform/google_appengine/api_server.py
and takes a number of arguments. I just tested running it like this:
$ $CLOUD_SDK/platform/google_appengine/api_server.py \ -A
dev~test-application \ --api_port 4444 \ --high_replication \
--datastore_path /tmp/datastore
To run an app engine application outside the normal development server
requires that a number of environment variables are set. This worked
for my application:
$ GAE_LONG_APP_ID=test-application \ GAE_MODULE_NAME=default \
GAE_MODULE_VERSION=version \ GAE_PARTITION=dev \ API_PORT=4444 \
API_HOST=127.0.0.1 \ dart bin/server.dart
In the Dart Editor you cannot set environment variables for each
launch configuration, so they have to be set globally before starting
the Dart Editor. In WebStorm it is possible to have run configuration
specific environment variables.
This simple setup will of cause not support everything the normal
development server support. Some of the issues are:
Only one application at the time as it is always listening on port
8080 (can easily be made configurable) * The users API (mocking this
shouldn't be that difficult) * The modules API * No health-checks
(should not be a problem) * All HTTP headers are direct from the
client (no x-appengine- headers) * The admin web interface is not
available * Probably other stuff as well
This is all experimental, but it is one solution for a simpler
developer setup, which of cause does not match the deployment
environment as closely as the development server.
Running the API Server using Docker is also possible as the image
google/cloud-sdk with the Cloud SDK is on hub.docker.com.
Use the following Dockerfile
FROM google/cloud-sdk EXPOSE 4444 ENTRYPOINT
["/google-cloud-sdk/platform/google_appengine/api_server.py", \
"-A", "dev~test-application", \ "--api_port", "4444", \
"--high_replication", \ "--datastore_path", "/tmp/datastore"]
Build and run
$ docker build -t api_server . $ docker run -d -p 4444:4444 api_server
Change API_HOST above to 192.166.59.103 (of wherever your Docker
containers are) and run.
Regards, Søren Gjesse
Update 2014-11-27
Debugging from DartEditors debugger started working with the bleeding Dart build 1.8.0.edge_042017.
I assume that the next dev build (probably 1.9.0-dev1.0) will include the related fixes as well?
Detailed steps how this works can be found here: https://groups.google.com/a/dartlang.org/d/msg/cloud/OK1nJtg7AjQ/u-GzUDI-0VIJ
Build a custom Docker image with the latest Dart dev build 1.8.0-dev.4.6.
The Dart team is actually preparing an easy way to do this yourself (see https://github.com/dart-lang/dart_docker)
Installe the latest bleeding_edge on the host system (using this script https://gist.github.com/zoechi/d240f56a32ed5649797f or manual download from http://gsdview.appspot.com/dart-archive/channels/be/raw/latest/editor/darteditor-linux-x64.zip)
Add this to the app.yaml file
env_variables:
DBG_ENABLE: 'true'
# disable health-checking because this is so annoying during debugging
vm_health_check:
enable_health_check: False
See How to disable health checking for `gcloud preview app run` for more details about customizing health checking.
Launch the server code of your app with glcoud --verbosity debug app run app.yaml or glcoud --verbosity debug app run app.yaml index.yaml
Wait until the Docker container is ready (check with docker ps if the Command column shows a value starting with /dart_runtime/dart_
Open DartEditor
Open Menu Run > Remote Connection...
Connect to: Command-line VM
Host: localhost if you dont use boot2dockeror the IP address returned by the commandboot2docker ip`
Port: 5005
Select Folder... select the directory which contains the source code of your project.
Click OK
Set breakpoints and continue as usual.
Old
A first step is using the Observatory which includes a browser based debugger UI.
To make this work add the following lines to the app.yaml file
network:
forwarded_ports: ["8181"]
This might be useful as well to make the server.dart wait until we had the chance to set breakpoints using the observatory.
env_variables:
DART_VM_OPTIONS: '--pause-isolates-on-start'
boot2docker gives us the Docker ip (192.168.59.103) and after starting with gcloud preview app run app.yaml we can connect to http://192.168.59.103:8181 which should open the Observatory GUI.

Resources