As I understand it, gcloud app deploy triggers a Cloud Build job behind the scenes based on command line options and things like requirements.txt (for python runtime). What I am looking to do is extend these Cloud Build steps to include things like database migration and testing before the deploy.
I can see that I could run these extra steps within separate containers in a separate GCP project, and then as the final step do something like
- name: 'gcr.io/cloud-builders/gcloud'
args: [ 'app', 'deploy', 'app.yaml', '--project', 'my-gae-project', '--version', 'staging' ]
Does that seem like a good way to do it? Has anyone done this kind of thing before for a GAE project?
Check out the instructions for Building, Testing, and Deploying artifacts using Cloud Build. There is a section under "deploying artifacts" that is specific to App Engine and how you would set up your custom build to deploy to GAE.
Related
I'm using cloud build to automatically deploy new version of app when a specific github branch is updated.
Everything works great but I'd like to deploy the new version without starting it, keeping my previous version running.
I really prefer to schedule start of the new version in specific hours, but when cloud build finish the process, my new version is automatically running.
How can I change this behaviour?
To avoid that the new version is started you can deploy using the --no-promote flag.
To use this flag when deploying using gcloud just need to add it to the command and deploy like this:
gcloud app deploy --no-promote
To implement this as part of the CD with Cloud Build then you can edit the cloudbuild.yaml to add the flag like this:
steps:
- name: 'gcr.io/cloud-builders/gcloud'
args: ['app', 'deploy', '--no-promote']
timeout: '1600s'
Recently tried to update my Gaelyk project (yes, it's old, but it works well and I still use it), but Google App Engine will no longer accept the update. The error message returned is "Deployments using appcfg are no longer supported. See https://cloud.google.com/appengine/docs/deprecations". The thing is, I never used appcfg to deploy my application; I used Gaelyk and Gradle. But obviously Gaelyk must have used appcfg under the covers.
I did download the replacement Google Cloud SDK, but this new tool is not similar at all to how Gaelyk and Gradle worked. Is there anything I can do to get Gaelyk to work anymore? Or is Gaelyk just dead and I need to rewrite my application (like in Node.js or something instead of Groovy).
This will be hard, however I will try to help you as possible. I think you may try to migrate it somehow to app.yaml configuration of GAE.
I am not sure what plugins are used in the project. From Gaelyk temple project I can see that it's using appengine-geb which, according to the documentation, behind the scenes, is using gradle-appengine-plugin (there is wrong link on this doc, but proper is bellow).
On the github of gradle-appengine-plugin I have found following.
There is a note:
NOTE: All App Engine users are encouraged to transition to the new
gradle plugin for their projects.
And in FAQ part there is following information:
How do I deploy with gcloud?
If you're using gcloud to deploy your application, the newest version of app deploy > doesn't support war
directories, you will need to provide it with an app.yaml OR you can
use the appengineStage task to create a directory that is deployable
in /build/staged-app
$ ./gradlew appengineStage
$ gcloud app deploy build/staged-app/app.yaml --project [app id]
--version [some version]
NOTES:
You must explicitly define all config files your want to upload
(cron.yaml, etc)
This does not work with EAR formatted projects.
I think the best option will be to migrate to new appenine plugin or if not possible try to implement is with gcloud app deploy command crating the config files manually (at least app.yaml). And for this migration I can provide you this document.
I hope you will manage somehow...
I can confirm that Serge's answer on the Gaelyk Groups site works; the same procedure that he figured out also worked for me. To summarize:
Run gradlew appengineRun as run previously with Gaelyk.
Copy all jar files inside the build\exploded-app\WEB-INF\lib folder into a \src\main\webapp\web-inf\lib folder (for me the new lib folder did not exist previously).
To deploy, use the new required gcloud tool, and instead of running gradlew appengineUpdate (which fails now), instead run
gcloud app deploy appengine-web.xml where that XML file can be found in your webapp/WEB-INF directory. I navigated to that directory to run the gcloud command, but you can use a relative path there if your working directory is elsewhere. (There are a number of optional flags associated with the gcloud app deploy command, but I didn't need any of them.)
Serge needed to use these instructions to convert datastore-indexes.xml to index.yaml and run gcloud app deploy index.yaml, however, I didn't need to do this because I had no datastores.
I have an app that uses the following command to deploy a static app generated by create-react-app:
gcloud app deploy --project MY_PROJECT -v dev
After that, my app is available on myproject.appstop.com.
But I don't figure out how to deploy this app using a diff URL for each environment. Like.: dev.myproject.appspot.com, stg.myproject.com and so on.
If you know or have other ideas about how to solve this please share your thoughts.
It depends on what you understand by environment. I would definitely recommend you to create different projects for the different environments (my-project-dev, my-project-staging, my-project-test, my-project-prod...)
If you just want to have a different url for your deployment, you are doing ok by using the -v (version parameter).
Once you've deployed again:
gcloud app deploy --project MY_PROJECT -v test
you'll have both versions accessible at:
dev.myproject.appspot.com
test.myproject.appspot.com
Also, check Dan's answer below as it contains very relevant info.
To complement #MarCialR's answer, you can use SSL as well, but with *-dot-* URLs like https://test-dot-myproject.appspot.com, see Targeted routing.
But personally I'm not a big fan of a version-based environment, it brings trouble, see Continuous integration/deployment/delivery on Google App Engine, too risky?
I chose to implement environments at app level - each environment as a different app, each with its own URL. See Advantages of implementing CI/CD environments at GAE project/app level vs service/module level?
Since I'm using a wildcard custom-domain SSL certificate I'm simply mapping the different apps (environments) to different hosts/subdomains in my custom domain - thus ensuring everything works as expected with a custom domain at every point in the CI/CD pipeline, no surprises in production.
according to the official documentation for Google App Engine Standard environment (Go API) the "preferred tooling to deploy a project" is now the Cloud SDK and so we moved to gcloud from goapp.
We are unable to deploy Go projects to GAE because all the sub-packages of every given project can't be found at "deploy time".
The typical folder structure that we have been using for every GAE project was as follows:
-project-name
--app.yaml
--main.go
--assets
---package1
---package2
When global libraries were put in the system GOPATH everything worked smoothly.
Running gcloud app deploy we now get this:
You are about to deploy the following services:
- yourproject/default/123456789 (from [/Path/to/app.yaml])
Deploying to URL: [https://yourproject.appspot.com]
Do you want to continue (Y/n)? Y
Beginning deployment of service [default]...
ERROR: (gcloud.app.deploy) Staging command [/path/to/yourproject/app.yaml /var/folders/b6/5ydn0wdn64jd32sxzzz48b7c0000gn/T/tmpbd4oiG] failed with return code [1].
------------------------------------ STDOUT ------------------------------------
------------------------------------ STDERR ------------------------------------
2017/03/24 10:25:58 failed analyzing /path/to/yourproject: cannot find package "yourpackage" in any of:
($GOROOT not set)
/path/to/gopath/src/yourpackage (from $GOPATH)
GOPATH: /path/to/gopath
--------------------------------------------------------------------------------
while dev_appserver.py works perfectly keeping the same folder structure.
Did we miss something?
How can we deploy to Google App Engine Standard environment using gcloud?
If the project structure needs to be changed: how? Is there official documentation about it?
Thanks in advance,
Edit -- Further infos:
Luigi-Mac-Pro:path/to/yourproject distudio$ gcloud version
Google Cloud SDK 148.0.0
app-engine-go
app-engine-go-darwin-x86_64 1.9.50
app-engine-python 1.9.50
bq 2.0.24
bq-nix 2.0.24
core 2017.03.17
core-nix 2016.11.07
gcloud
gcloud-deps 2017.03.17
gcloud-deps-darwin-x86_64 2017.02.21
gsutil 4.23
gsutil-nix 4.22
Google recommends keeping your dependencies outside of the app directory, and using GOPATH to refer them. In your case, that would mean doing the following:
-project-name
--app.yaml
--main.go
where main.go contains
import (
"package1"
"package2"
)
And somewhere else:
-my_packages
--src
---package1
---package2
And then set GOPATH environment variable to path/to/my_packages prior to running dev_appserver and gcloud app deploy.
For the future
We are working out the long term solution for properly vendoring packages inside your app directory – likely using Go's future native package manager. I'm sorry to say that we don't have a good way of supporting sub-packages for gcloud app deploy. This was an unfortunate side effect of compatibility with the App Engine Flexible environment.
My problem is that I want to create dev, stage, prod environments by using different GCP projects.
Basically they are running the same code, just running them in different isolated environments.
I'm using gcloud app deploy in command line to deploy app right now.
How can I efficiently deploy an app to different project?
Do I have to do gcloud init to change my configuration of default project every time?
There must be some better practices.
Or, is there a better way for me to set up dev... environments in the context of app engine?
Thanks.
Instead of using gcloud init to change your configuration each time, use this code to deploy to multiple projects faster:
gcloud app deploy -q --project [YOUR_PROJECT_ID]
If your projects have similar IDs, let's say: test1, test2, test3, test4, You can deploy to the four projects with one command. Use this code:
for i in {1..4}; do gcloud app deploy -q --project test${i}; done
The "standard" approach is to use versions, e.g.
qa.myApp.appspot.com
Once a version is ready for next step, you deploy it with a different version id.
One problem with using multiple projects is that you have to maintain a different data set for each project.
My preference is to have the different environments managed via the same version control as the code - one branch for each environment, keeping the deployments perfectly aligned with the natural flow of code changes, promoted via branch merges: dev -> stage -> production.
To minimize the risk of human error I try as much as possible to keep the deployment configs in the code itself (i.e. - have the app IDs, versions, etc. picked up from the .yaml files, not passed to the deploy cmd as args). The deployment cmds themselves are kept in a cheat-sheet file (too simple to warrant a full-blown script at this time), also git-controlled. Illustrated in this answer: https://stackoverflow.com/a/34111170/4495081
Deployments are done from separate, dedicated workspaces - one for each environment, based on the corresponding git branch (I never switch the branches in these workspaces). I just update the workspace corresponding to the desired environment to the version needed and copy-paste the deployment cmd from the workspace's cheat-sheet.
This model is IMHO CI/CD-ready and can easily be entirely automated.
For Python applications, you can set application in the app.yaml file. This allows you to use different data for each project. This is when you deploy using the appcfg.py command.
application: myproject
version: alpha-001
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /
script: home.app
If you don't want to change the application value in this file for each project, you can run the following:
appcfg.py -A <YOUR_PROJECT_ID> -V v1 update myapp/
https://cloud.google.com/appengine/docs/python/config/appref
If you do not specify the application in the file, use the --application option in the appcfg command when you deploy. This element is ignored when you deploy using the gcloud app deploy command.