I have an HTTP service written in Go. Inside main.go I have a global version string.
package main
var version string
Locally, I build using -ldflags "-X main.version=$VERSION where $VERSION is determined by the shell environment, like so:
VERSION=v0.16.0 go build ./cmd/app -ldflags "-X main.version=$VERSION
I've recently decided to trial Google App Engine and started with a basic YAML file:
runtime: go111
handlers:
- url: /.*
script: auto
What can I set in the YAML file in order to instruct GAE to build with the equivalent ldflags to bake in my version string?
I should also mention I use go modules with GO111MODULE=on locally when building.
You can't do it with you app.yaml file.
However, you can use Cloud build to build and deploy your app to App Engine.
In your cloudbuild.yaml you can add a line to the build step
args: ['build', '-a', '-installsuffix', 'cgo', '-ldflags', '''-w''', '-o', 'main', './main.go']
Related
Originally I had this in my app.yaml:
runtime: python39
- url: .*
script: main.app
I changed the script filename to gae.py and so updated app.yaml to thus:
runtime: python39
- url: .*
script: gae.app
The new version no longer starts up:
ModuleNotFoundError: No module named 'main'
I have tried changing the url but it's still trying to use a now-non-existent main.py to load the WSGI application. When I view the source of the version I see the correct app.yaml and file structure; I don't see main mentioned anywhere.
Any ideas?
The problem is not caching but rather that script is no longer used (and is ignored) in the Python 3 world of Google App Engine standard:
script: Optional. Specifies that requests to the specific handler should target your app. The only accepted value for the script element is auto because all traffic is served using the entrypoint command. In order to use static handlers, at least one of your handlers must contain the line script: auto or define an entrypoint element to deploy successfully.
(https://cloud.google.com/appengine/docs/standard/python3/config/appref#handlers_element)
The fix is to override entrypoint which defaults to:
/bin/sh -c exec gunicorn main:app --workers 2 -c /config/gunicorn.py
(https://cloud.google.com/appengine/docs/standard/python3/runtime#application_startup)
I have a django project I want to put into maintenance mode before I update (migrate) the database.
So, my github workflow
deploys my project with a variable MAINTENANCE_MODE set to true. This new deploy I understand will reboot any running instances, ensuring all instances only show my 'Site down for maintenance' 503.html page and won't be interacting with the database.
I launch a django VM in github actions, run my migrate, run collectstatic.
I set MAINTENANCE_MODE to false, I deploy a second time. This will re-enable production server with new code that now accesses a migrated database.
My question is, I am trying to use a single app.yaml file for both deploys. To pass the MAINTENANCE_MODE variable from github actions workflow to the app.yaml file, how can I do this?
I know you can import secrets like so:
runtime: python38
instance_class: F2
env_variables:
DB_URL: $ {{ secrets.DB_URL }}
But I don't know how to modify a secret in the workflow. Perhaps its not a secret but some other type of variable one can set in the workflow and access in the app.yaml?
So it appears that Google App Engine's yaml files do not support dynamic environment variable substitution. Static substitution (like using github's secrets) works, because github compiles the file with the github environment variable before the workflow runs, but there's no clear way to modify a file with a variable that is going to change during a workflow.
A method however that does work is to compile a new GAE yaml file during the workflow. Here's what I came up with in the end...
- name: Put in Maintenance mode
run: |
MAINTENANCE_MODE=1 envsubst < app_eng_staging.yml.template > app.yaml
cat app.yaml
gcloud app deploy --project staging-project --quiet
- name: Collectstatic and migrate
env:
RUNNING_ENVIRONMENT: 'Staging_Server'
DJANGO_DEBUG: 'False'
run: |
pipenv run python manage.py collectstatic --noinput
pipenv run python manage.py migrate
- name: Turn off Maintenance and Deploy
run: |
MAINTENANCE_MODE=0 envsubst < app_eng_staging.yml.template > app.yaml
gcloud app deploy --project staging-project --quiet
The trick is to use the linux envsubst command. We start with a app_eng_staging.yml.template file, which looks like so:
runtime: python38
instance_class: F2
env_variables:
RUNNING_ENVIRONMENT: 'Staging_Server'
StagingServerDB: $ {{ secrets.STAGINGSERVER_DB }}
MAINTENANCE_MODE: ${MAINTENANCE_MODE}
FRONTEND_URL: $ {{ secrets.STAGING_FRONTEND_URL }}
envsubst then populates ${MAINTENANCE_MODE} with the value 1 and the result is saved to a new file app.yaml.
After we finish working with out database migration, we can use envsubst to create a new app.yaml with MAINTENANCE_MODE set to zero (off), and re-deploy.
Neat huh?
There is a feature in GitHub Action for this called "GAE environment variable compiler"
Please read here
Build errors out with below output (Using a Rails app)
ERROR: (gcloud.app.deploy) There is a cloudbuild.yaml in the current directory, and the runtime field in /workspace/app.yaml is currently set to [runtime: ruby]. To use your cloudbuild.yaml to build a custom runtime, set the runtime field to [runtime: custom]. To continue using the [ruby] runtime, please remove the cloudbuild.yaml from this directory.
One way to deal with this is to change the name of the cloudbuild.yaml file to say cloud_build.yaml (you can also just move the file) and then go to your triggers in Cloud Build:
And change it from Autodetected to choosing your Cloud Build configuration file manually:
See this Github issue for some more information
Cloudbuild.yaml should work with App Engine Flexible without the need to use a custom runtime. As detailed in the error message, you cannot have the app.yaml and the cloudbuild.yaml in the same directory if you are deploying in a non-custom runtime, to remedy the situation, follow these steps:
Move the app.yaml and other ruby files into a subdirectory (use your original app.yaml, no need to use custom runtime)
Under your cloudbuild.yaml steps, modify the argument for app deploy by adding a third one specifying the app.yaml path.
Below is an example:
==================FROM:
steps:
- name: 'gcr.io/cloud-builders/gcloud'
args: ['app', 'deploy']
timeout: '1600s'
===================TO:
steps:
- name: 'gcr.io/cloud-builders/gcloud'
args: ['app', 'deploy', '[SUBDIRECTORY/app.yaml]']
timeout: '1600s'
I'm new on Google App Engine. And, I'm getting an issue that I can't solve.
I've a very simple app (developped in Go) like this :
main/
| model/
| | user.go
| main.go
| app.yaml
These are the imports of main.go :
import (
"github.com/julienschmidt/httprouter"
"log"
"net/http"
)
My code works well when I run it locally.
But, when I try to publish it on my Google App Engine instance, I receive this error :
$ gcloud app deploy
You are about to deploy the following services:
- <MY_APP_ENGINE_URL> (from [<MY_LOCAL_YAML_PATH>])
Deploying to URL: [<MY_APP_ENGINE_URL>]
Do you want to continue (Y/n)? Y
Beginning deployment of service [default]...
Some files were skipped. Pass `--verbosity=info` to see which ones.
You may also view the gcloud log file, found at
[<LOCAL_APP_ENGINE_LOG>].
File upload done.
Updating service [default]...failed.
ERROR: (gcloud.app.deploy) Error Response: [9] Deployment contains files that cannot be compiled: Compile failed:
2017/05/27 14:48:24 go-app-builder: build timing: 5×compile (301ms total), 0×link (0s total)
2017/05/27 14:48:24 go-app-builder: failed running compile: exit status 2
main.go:4: can't find import: "github.com/julienschmidt/httprouter"
What did I do wrong ?
EDIT :
This is the content of my app.yaml file :
runtime: go
api_version: go1
handlers:
- url: /.*
script: _go_app
App Engine environment doesn't contain your dependencies, you can add an script to do a go get ... for each one but it's too hacky and Go has a solution for that, we can save our dependencies in a vendor folder on the root of our project.
Quick solution:
# Instal godep:
go get -v -u github.com/tools/godep
cd your/project/path
godep save
Now try to deploy again, you'll see a vendor folder in your project, don't remove it and add it to your git source, that folder contains all your third party dependencies like your httprouter (it's my favorite :) )
Note You can use other tools to save your dependencies
I haven't used the gcloud tool, but back in the day when goapp was the tool you had to create github.com/julienschmidt/httprouter (with the lib's source in it, of course) directly under you'r main and then deploy.
AFAIK the App Engine's go version is currently 1.6 which means that while the vendoring is on by default, it can be switched off - perhaps thats the case and thats why #Yandry Pozo's suggestion doesn't work.
I am currently writing an application in Go and trying to deploy multiple services. I am running the following command : gcloud app deploy dispatch.yaml app/app.yaml mod1/mod1.yaml.
The app.yaml file corresponds to the default service and is successfully deployed however the service mod1 returns this error:
ERROR: (gcloud.app.deploy) Error Response: [9] Deployment contains files that cannot be compiled: Compile failed: 2016/07/22 18:17:14 go-app-builder: build timing: 1×compile (53ms total), 0×link (0 total) 2016/07/22 18:17:14 go-app-builder: failed running compile: exit status 1
mod1.go:4: can't find import: "myapp/mod1/web_console"
My-Macbook: myapp$ gcloud app deploy dispatch.yaml app/app.yaml mod1/mod1.yaml
My file structure is as follows:
/My_Project
/src
/myapp
/app
app.go
app.yaml
/mod1
mod1.go
mod1.yaml
/web_console
web_console.go
mod1.go :
package mod1
import (
"myapp/mod1/web_console"
)
func init() {
// Initializing Web Console establishes connection
// to the database and also creates routes
var wc *web_console.WebConsole
wc = web_console.NewWebConsole(true)
wc.Configure()
}
mod1.yaml :
module: mod1
runtime: go
api_version: go1
handlers:
- url: /.*
script: _go_app
app.yaml :
module: default
runtime: go
api_version: go1
handlers:
- url: /.*
script: _go_app
Thanks for taking the time to help!
Each GAE service/module is standalone and can not access anything outside the service/module dir which:
is the directory where the .yaml file exists
is what's uploaded during deployment
is considered the "root" directory for that service/module, everything is relative to it
In your particular case you need to make sure no service/module makes references to the parent myapp dir (which is nothing but an organisational placeholder for the app stuff, relevant on your side but with no actual presence on GAE). I believe your mod1.go import should looks like this:
package mod1
import (
"web_console"
)
But take it with a grain of salt, I'm not actually familiar with go.
It seems like I was using the wrong tool to deploy. I ran the command with goapp deploy app/app.yaml mod1/mod1.yaml and was able to successfully deploy the services without issue.