if I have my circle-ci deplying to google with e.g. 'gcloud app deploy app.yaml' is there a simple way I can pass a dev / prod variable and have it replaced in my app.yml file below ?
Sure, I could have multiple app.dev.yml, app.dev.yml files etc but theres a lot of duplication involved unless I can just override some by using some sort of base app.yml
runtime: nodejs
env: flex
service: email
env_variables:
PUBSUB_TOPIC: dev-email-integration-email
PUBSUB_VERIFICATION_TOKEN: <your-verification-token>
Sadly there really isn't a good way to do this. I'd actually suggest not using app.yaml for this purpose, and instead using a json file and nconf. That's how we do it in all of our nodejs samples.
Hope this helps!
Related
I'm trying to provide a .well-known folder under my Google App Engine Application I'm using the standard environment and the python27 runtime.
with a web-app-origin-association.json file to try the Progressive Web Apps as URL Handlers
origin trial from chrome.
I've added the following code to my app.yaml file under handlers:
# .well-known Ordner
- url: /.well-known/(.*)
static_files: well-known/\1
upload: well-known/.*
The folder in my project is named well-known without a dot cause I've read that there are problems when using a folder Name with a dot at the start of the foldername.
But the url https://example.com/.well-known/web-app-origin-associate.json isn't available instead it works without the dot:
What do I have to change in order to make it work under https://example.com/.well-known/web-app-origin-association.json?
You can use the workaround documented at "Make skip_files rule explicit and tweak to allow .well-known/* to upload":
^(.*/)?\.(?!well-known(?:/|$)).*$
You many want to migrate to Python 3 as described in the guide:
Starting on January 1, 2020, the Python community will no longer
update, fix bugs, or patch security issues for Python 2.7. We
recommend that you update apps that are still running in the Python 2
runtime of the App Engine standard environment to the Python 3 runtime
as soon as possible.
The best way i found out about is to just do it like that:
- url: /\.well-known
static_dir: .well-known
secure: always
and use the python39 runtime.
I know you can declare env_variables in your app.yaml as described in the app.yaml documentation. However, is it possible to include environment variables from your local environment into app.yaml when deploying.
As an example of what I'm trying to accomplish
# in app.yaml
runtime: python27
api_version:1
threadsafe: true
service: {{ $AN_ENVIRONMENT_VARIABLE }}
Yes, you can use includes: to specify an array of files to be included. And in the included file, you can specify env_variables: just like you do in app.yaml.
Example: app.yaml:
runtime: go
api_version: go1
env_variables:
FIST_VAR: myFirstVar
includes:
- credentials.yaml
credentials.yaml:
env_variables:
SECOND_VAR: mySecondVar
No, no such templating support exists for the app.yaml configuration files.
Side note: the app.yaml file is not only used to extract deployment instructions information, it's also used to configure the operation of the respective service on GAE. Making the service name configurable in such manner doesn't make a lot of sense unless the services being deployed are identical in every aspect (other than their name) - highly unlikely.
One possible approach for environment-specific deployment would be to have different version control branches for the app code, one for each environment, each having the desired app.yaml content.
Another one would be to wrap the deployment command in a script and perform the enviroment substitutions inside that script.
As for passing credentials info to the app a clean, straight-forward solution is not yet available. But approaches exist:
GAE: best practices for storing secret keys?
How to set environment variables/app secrets in Google App Engine
Google app engine: Best practice for hiding Rails secret keys?
How to handle sensitive configuration information when deploying app-engine applications?
I am developing an API in golang directly on the "App Engine flexible environment" (formerly known as "Managed VMs").
So far, i have been using this kind of import in my .go files :
import (
"appengine"
"appengine/datastore"
...)
Recently I decided to use Google Cloud Storage to store images. It requires the import of "cloud.google.com/go/storage". My problem is that i'm unable to deploy the app with this import (not found), or any other short version ("go/storage") like I use for the appengine import.
After much research, I found this : https://github.com/golang/appengine#user-content-3-update-code-using-deprecated-removed-or-modified-apis
It specifies how to migrate an application using short imports (deprecated, like mine) to full imports (with repository explicit like "google.golang.org/appengine")
I followed the procedure and used the script they provide to update my code (aefix). They also say to add this line to my app.yaml file :
vm : true
If I do, I got this error message running 'gcloud app deploy' :
ERROR: (gcloud.app.deploy) Your application does not satisfy all of the requirements for a runtime of type [go]. Please correct the errors and try again.
If I don't, none of my imports are working and I get the following error :
can't find import: "google.golang.org/appengine/datastore"
Here is my app.yaml file :
runtime: go
api_version: go2
#vm : true
handlers:
- url: /.*
script: _go_app
Of course, all the imports are on the server under $GOPATH/src/ so they're not really missing, more badly referenced I guess.
I'm stuck on this problem since several days, any help of any kind would be appreciated !
Thanks
So sorry - we have some docs to go update. You cannot use the golang/appengine package with the App Engine flexible environment. The aefix tool won't work here either. Instead of the App Engine Go SDK, you want to use the Go client library here:
https://github.com/GoogleCloudPlatform/google-cloud-go
If you were previously using vm:true, you will need to upgrade to env:flex - the instructions (and the note on the go app engine library) are here:
https://cloud.google.com/appengine/docs/flexible/go/upgrading
Let me know if you have any questions!
I've noticed that the developer console doesn't seem to expose anywhere where I can configure static environment variables.
Is the expectation on GAE that I will bundle those variables as part of the deployment from my build server? If so, is there any documentation on GAE/Google Cloud that covers why or details the philosophy?
Years have passed, and still it doesn't.
My workaround is to compile app.yaml during deployment process (locally or with CI).
For example, I have a template file app.tml.yaml file
runtime: python37
handlers:
- url: /static
static_dir: app/static/
- url: /.*
script: auto
env_variables:
DJANGO_GC_DATABASE_PASSWORD: ${DJANGO_GC_DATABASE_PASSWORD}
Then I call envsubst right before deployment
envsubst < ./app.tml.yaml > app.yaml and after that gcloud app deploy as usual. After the deployment is done app.yaml with sensitive data is deleted. Variables are read from local .env file or are set in CI system.
There also other approaches I found listed in this post: https://dev.to/mungell/google-cloud-app-engine-environment-variables-5990 but for me they are not convienient or generic enough.
Environment variables can be defined in your application's app.yaml
An example for a python/php/(maybe go?) app. Java uses a different format.
env_variables:
MY_ENV_VAR: 'some value here'
https://cloud.google.com/appengine/docs/python/config/appconfig#Python_app_yaml_Defining_environment_variables
You can set these values during your CI process as well if you need to by programmatically appending them to your app.yaml before deploying.
One solution is apparently https://cloud.google.com/secret-manager/docs, but I opted for the solution offered here:
Securely storing environment variables in GAE with app.yaml
First, put the environment variables in an env_variables.yaml, e.g.,
env_variables:
SECRET: 'my_secret'
Then, include this env_variables.yaml in the app.yaml
includes:
- env_variables.yaml
Finally, add the env_variables.yaml to .gitignore, so that the secret variables won't exist in the repository.
Further,
I commit env_variables.sample.yaml, with instructions and placeholder values which the next dev can fill in
in dev, I parse env_variables.yaml and add the vars to process.env so I have a single source of truth for those vars…
if (process.env.NODE_ENV === "development") {
try {
const fs = require("fs");
const yaml = require("js-yaml");
let fileContents = fs.readFileSync("./env_variables.yaml", "utf8");
let {env_variables} = yaml.load(fileContents);
console.log({ env_variables });
Object.keys(env_variables).forEach((v) => {
process.env[v] = env_variables[v];
});
} catch (error) {
res.status(500).end("500: Problem getting env vars");
return;
}
}
I'm adding my solution here as the quoted question specifies python, and this question is generic.
As with other PAAS solutions (eg Heroku, Netlify), if a user has access to the App Engine console, they can see the secrets (in this case by browsing the source files in the console).
On the tutorial for Apps Script/Appengine:
https://developers.google.com/apps-script/articles/appengine
When trying to run google_appengine/dev_appserver.py google-apps-script/ the response is:
WARNING 2012-09-06 14:56:33,570 rdbms_mysqldb.py:74] The rdbms API is not available because the MySQLdb library could not be loaded.
INFO 2012-09-06 14:56:33,840 appengine_rpc.py:163] Server: appengine.google.com
CRITICAL 2012-09-06 14:56:33,842 appcfg.py:561] The api_version specified in app.yaml (1) is not supported by this release of the SDK. The supported api_versions are ['3', 'go1'].
I have tried the following app.yaml, but it doesn't work.
application: google-apps-script-tutorial
version: 1
runtime: go
api_version: go1
handlers:
- url: /*
script: _go_app
Also with - url: /rpc and it doesn't work. Since the code is Python is it possible to get App script and Go linked up in app engine?
The code for that tutorial is in Python and Javascript. If you want to use the go runtime, you will have to rewrite the Python portion in Go.
That example demonstrates the use of a Google Apps Script frontend with a Google App Engine (GAE) backend written in Python. GAE currently runs apps written in Java, Python, Go and PHP.
That particular Python backend accepts and produces messages in JSON format. Therefore, to link Apps Script and Go similarly, using GAE or not, you would need to replicate the functionality of the Python backend using, probably, the net/http library and the encoding/json library.
For examples of using these libraries together, have a look at this, this and this.
For examples on using Go with GAE, have a look at this and this.
Hope that helps.