I have a Google App Engine app (Go lang, if that matters) that I would like to deploy more than once, with slightly different setup. Think production vs. QA.
env_variables in app.yaml seemed promising, but it seems I can only have one such file. For example, I don't see a way to call "goapp deploy" with app-qa.yaml.
How can I tweak deployment configuration? Is it possible to have more than one app.yaml, without custom script that copies files to a directory and manipulates app.yaml? Any other way to configure this?
My preference is to have the delta between the staging/QA and production reflected in (and controlled via) the VCS (git in my case):
my main development happens on the main branch, where the staging environment config is reflected in the .yaml files' contents
my production branch is pulled off the main branch and contains the production environment config - the .yaml files are modified accordingly
When I perform a deployment I do it from a workspace based on the appropriate branch depending on which config I need to deploy.
The delta between the production branch and the main branch is pretty much just deployment config changes. Whenever I'm happy with the staging results and I'm ready to deploy in production I just sync the production branch to the main refpoint corresponding to the OK'd staging deployment and deploy.
Another possible approach is to directly use the SDK's appcfg.py tool, which is what goapp deploy ultimately invokes anyways:
goapp deploy wraps the appcfg.py python tool provided in the SDK. You
can also invoke this tool directly if you need greater control over
the deployment:
The appcfg.py tool allows you to actually have and use alternate .yaml files residing in the same app/module directory (you'd probably have to use that anyways if you go with multiple or non-standard module configurations, since auto-detection from the app's dir won't work anymore):
appcfg.py update app-qa.yaml
App Engine Modules could be used for such purposes.
Related
I'm using GCP App Engine with auto scaling.
If I deploy a new version of the app code (Python 3 Flask app) with a simple change for control and test purposes, lets say I add a comment to one of the .js files I am not seeing that change to the file in the browser after it has been deployed.
I have 100% of traffic being served by the new version of the app. When I look at the source code for the version I can see the comment in there, but when I clear my browser cache and visit the page I only ever get the old version of the page (without the comment in the .js).
I have tried using the --promote and --no-cache values in the app deploy command, but no use. I have added:
default_expiration: "0d 0h 0m 0s"
To the app.yaml
I have also turned on object versioning of the storage account which app engine uses to try to ensure that only a single version of the file is available to be served - still no use.
The command I'm using to deploy is:
run: "gcloud app deploy app.yaml --quiet --promote --no-cache"
I can't understand why it should be so difficult to simply deploy a new version of the app and have the app engine serve the latest files; I must be doing something wrong but cannot see what.
Would appreciate any pointers.
The files are cached (even if for a short while and sometimes it takes time to clear it).
The trick is to make the urls (for the static files) unique for each deployment. This way, the browser is loading a 'distinct' url after each deployment. For example, you could append the environment variable, CURRENT_VERSION_ID to the url for all static elements. This means having something like (assuming Python/Jinja2)
src="/static/js/my_js_file.js?{{CURRENT_VERSION_ID}}"
os.environ['CURRENT_VERSION_ID'] changes for each deployment. There's a possibility this attribute is not available in newer runtimes. If so, just dump the environment variables and look for an attribute that is always present but the value changes (e.g. GAE_INSTANCE).
You could also just generate a random number each time your App is deployed and use that instead i.e.
src="/static/js/my_js_file.js?{{RANDOM_NUMBER}}"
Recently tried to update my Gaelyk project (yes, it's old, but it works well and I still use it), but Google App Engine will no longer accept the update. The error message returned is "Deployments using appcfg are no longer supported. See https://cloud.google.com/appengine/docs/deprecations". The thing is, I never used appcfg to deploy my application; I used Gaelyk and Gradle. But obviously Gaelyk must have used appcfg under the covers.
I did download the replacement Google Cloud SDK, but this new tool is not similar at all to how Gaelyk and Gradle worked. Is there anything I can do to get Gaelyk to work anymore? Or is Gaelyk just dead and I need to rewrite my application (like in Node.js or something instead of Groovy).
This will be hard, however I will try to help you as possible. I think you may try to migrate it somehow to app.yaml configuration of GAE.
I am not sure what plugins are used in the project. From Gaelyk temple project I can see that it's using appengine-geb which, according to the documentation, behind the scenes, is using gradle-appengine-plugin (there is wrong link on this doc, but proper is bellow).
On the github of gradle-appengine-plugin I have found following.
There is a note:
NOTE: All App Engine users are encouraged to transition to the new
gradle plugin for their projects.
And in FAQ part there is following information:
How do I deploy with gcloud?
If you're using gcloud to deploy your application, the newest version of app deploy > doesn't support war
directories, you will need to provide it with an app.yaml OR you can
use the appengineStage task to create a directory that is deployable
in /build/staged-app
$ ./gradlew appengineStage
$ gcloud app deploy build/staged-app/app.yaml --project [app id]
--version [some version]
NOTES:
You must explicitly define all config files your want to upload
(cron.yaml, etc)
This does not work with EAR formatted projects.
I think the best option will be to migrate to new appenine plugin or if not possible try to implement is with gcloud app deploy command crating the config files manually (at least app.yaml). And for this migration I can provide you this document.
I hope you will manage somehow...
I can confirm that Serge's answer on the Gaelyk Groups site works; the same procedure that he figured out also worked for me. To summarize:
Run gradlew appengineRun as run previously with Gaelyk.
Copy all jar files inside the build\exploded-app\WEB-INF\lib folder into a \src\main\webapp\web-inf\lib folder (for me the new lib folder did not exist previously).
To deploy, use the new required gcloud tool, and instead of running gradlew appengineUpdate (which fails now), instead run
gcloud app deploy appengine-web.xml where that XML file can be found in your webapp/WEB-INF directory. I navigated to that directory to run the gcloud command, but you can use a relative path there if your working directory is elsewhere. (There are a number of optional flags associated with the gcloud app deploy command, but I didn't need any of them.)
Serge needed to use these instructions to convert datastore-indexes.xml to index.yaml and run gcloud app deploy index.yaml, however, I didn't need to do this because I had no datastores.
I'm developing an app locally using Go and React with a project structure that looks somewhat like this:
/reactApp
/main.go
/api
/reactUI
While I am developing locally I am using the gcloud cli tool to run the Go code, and I'm using Node to run the UI. I need to handle CORS of course, but otherwise this works fairly well. The problem is there are too many files in the /reactUI directory for the gcloud cli tool to monitor them all. This means I need to manually start and stop the server every time I make a change to the Go source code.
The reactUI directory was created using the 'npx create-react-app' command. Since I'm running the react part of this app locally using node (and that handles file monitoring and auto-updating just fine), how can I get the gcloud tool to ignore the reactUI directory?
Basically I just want the gcloud tool to monitor the API part of the app, and not the UI part. If I make a change to the API part of the app the gcloud local server should automatically recompile and restart.
With https://issuetracker.google.com/issues/35895450 fixed the development server is supposed to not monitor changes in the files/directories matching the patterns specified in the skip_files section of the app/service's .yaml file.
So you can try to add a skip_files section to the go app's .yaml file specifying the patterns for the files/directories you want ignored. Be careful to also add the default patterns (otherwise you'd lose them).
I'm developing an app (front-end app that consumes an API) based on the create-react-app package. I'm using Heroku to deploy and currently have two deployments based on the same codebase, staging and production. These deployments should use different development/staging/production APIs that have different databases.
Is it possible to tell create-react-app to use different env variables based on how I run react-scripts start?
env.development
REACT_API: https://localhost/react_api
env.staging
REACT_API: https://myappstagingapi.heroku.com
env.production
REACT_API: https://myappproductionapi.heroku.com
How would I do this? And is this a good workflow?
Thank you very much!
I had the similar situation having more environments than production and development while deployment was done automatically from Github.
First of all, make sure you are using the deployment buildpack i.e.
https://github.com/mars/create-react-app-buildpack.git
You can add this URL in Settings in your project on Heroku, replacing NodeJS default buildpack for instance.
Full documentation is here:
https://elements.heroku.com/buildpacks/nhutphuongit/create-react-app-buildpack
If you follow the link above, you can see the chapter Environment variables. So, in order that your React App can process custom variables:
Add a one that suits you with REACT_APP_ prefix to your Heroku project environment variables (through Settings in Heroku Dashboard) Note that only the envs with this prefix will be visible in your process.env later so you should amend your code accordingly
Redeploy the application.
This should make the things work. I am sure it is also possible to keep .env file but I prefer to have more control via Heroku Dashboard.
My problem is that I want to create dev, stage, prod environments by using different GCP projects.
Basically they are running the same code, just running them in different isolated environments.
I'm using gcloud app deploy in command line to deploy app right now.
How can I efficiently deploy an app to different project?
Do I have to do gcloud init to change my configuration of default project every time?
There must be some better practices.
Or, is there a better way for me to set up dev... environments in the context of app engine?
Thanks.
Instead of using gcloud init to change your configuration each time, use this code to deploy to multiple projects faster:
gcloud app deploy -q --project [YOUR_PROJECT_ID]
If your projects have similar IDs, let's say: test1, test2, test3, test4, You can deploy to the four projects with one command. Use this code:
for i in {1..4}; do gcloud app deploy -q --project test${i}; done
The "standard" approach is to use versions, e.g.
qa.myApp.appspot.com
Once a version is ready for next step, you deploy it with a different version id.
One problem with using multiple projects is that you have to maintain a different data set for each project.
My preference is to have the different environments managed via the same version control as the code - one branch for each environment, keeping the deployments perfectly aligned with the natural flow of code changes, promoted via branch merges: dev -> stage -> production.
To minimize the risk of human error I try as much as possible to keep the deployment configs in the code itself (i.e. - have the app IDs, versions, etc. picked up from the .yaml files, not passed to the deploy cmd as args). The deployment cmds themselves are kept in a cheat-sheet file (too simple to warrant a full-blown script at this time), also git-controlled. Illustrated in this answer: https://stackoverflow.com/a/34111170/4495081
Deployments are done from separate, dedicated workspaces - one for each environment, based on the corresponding git branch (I never switch the branches in these workspaces). I just update the workspace corresponding to the desired environment to the version needed and copy-paste the deployment cmd from the workspace's cheat-sheet.
This model is IMHO CI/CD-ready and can easily be entirely automated.
For Python applications, you can set application in the app.yaml file. This allows you to use different data for each project. This is when you deploy using the appcfg.py command.
application: myproject
version: alpha-001
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /
script: home.app
If you don't want to change the application value in this file for each project, you can run the following:
appcfg.py -A <YOUR_PROJECT_ID> -V v1 update myapp/
https://cloud.google.com/appengine/docs/python/config/appref
If you do not specify the application in the file, use the --application option in the appcfg command when you deploy. This element is ignored when you deploy using the gcloud app deploy command.