I would like to know how to deploy the application from bitbucket using pipelines to multiple Google Cloud projects.
Here is our current set up and it is working fine.
On Bitbucket, the application repo with development/UAT test/production branches, once the pull request approved and merged into development/production, it shall deploy to the GCP app engine through the pipelines.
The problem now, we want to isolate each client in GCP which mean each client will have its own GCP project, cloud SQL, App engines, storage bucket, etc...
I need some advice on how to change the deployment workflow in bitbucket and pipelines, so will work for the new set up.
For the branches setup on bitbucket, I'm thinking like below, but if I go for option2, then it seems too much if got more clients.
Option 1 (repo branches)
development/
UAT test/
validation/
production
Option 2 (repo branches)
development/
UAT test client1/
UAT test client2/
validation_client1/
validation_client2/
production_client1/
production_client2/
The first step, I know I have to create different app.yaml for each app engine service for each client, so it can deploy the app engine service to different CGP projects/bucket/SQL instance.
Also just found out the bitbucket-pipelines.yml only support 10 steps, if I create so many branches then it will over the limits for sure.
Does anyone have any suggestions about how should be set up?
Thanks,
You could create Cloud build triggers to specific bitbucket branches or repos (whatever your branching model is defined) and deploy the app engine implementation to the App engine service on the same project, and if you need to customize other steps, you could use custom steps as described here. Finally you can take a look at how to create a basic configuration file for Cloud Build if you are not very familiar with this product
Related
For my application, I have two different environments on GCP: staging and production. My static Angular application is currently deployed on GAE. Now I'm wondering if it's possible to deploy these two environments separately with two different URL's? Or is there another solution better suited for such a setup?
If not I'll probably have to switch back to Google Cloud Run.
Thanks in advance.
2 environments = 2 projects! It's easier and you have App Engine free tier duplicated (per projet)
If you deploy 2 time the same package on the same GAE (so, on the same project), you need to have a app.yaml per deployment. And thus your staging deployment pipeline isn't exactly the same as the production deployment pipeline. And the URL format also. The non default service has his URL prefixed with the service name. And you should have issues with handlers definitions, the scheduler (if you have one),....
No, the easiest is to have 1 project per environment
On GCP, I run Cloud Build from one project and deploy code to App Engine in another project. It looks like the project where build runs from needs to have App Engine Admin API enabled. Is it a real request or I missed the real configurations?
App Engine Admin API is required as it is used for any App Engine-related management operations
Also, based on the link above:
The Admin API provides you with:
An integration point for your development and build tools.
Tighter control around deploying new versions, including the ability to automate traffic
migration between two versions or traffic splitting across one or more versions.
The ability to programmatically manage applications across multiple Google Cloud projects.
So yes, if you plan on deploying code to App Engine using Cloud Build, you need GAE Admin API enabled.
I'm looking at moving my application to google cloud and I'm having a hard time understanding how best to organize my project. It seems like you can only have one App Engine application per project with services available to support a microservices architecture and instances representing the App Engine instances created via auto scaling.
What is the correct way to build an App Engine for my API server and an App Engine for my Web Server? Do I need to have a project for each? I'm essentially trying to accomplish the following:
It is straightforward to have a single GAE project implement both your website and an API. You can even do this within a single service. You could put each in a separate service, and the advantage of that is that you can update one without updating the other. For small projects, a disadvantage is that two services are more expensive than one (though GAE is quite inexpensive overall).
For prod vs dev, you'll need to explain your requirements a little more, but here are some thoughts.
Each GAE service has multiple versions. You can deploy your production version to www.mycompany.com and deploy a dev version to dev-dot-myapp.appspot.com (that's the way GAE does URLs for versions of your app). Both of these versions will access the same datastore so you need to be careful with the dev version so that it doesn't mess up your prod implementation.
If you have a dev situation that is bleeding edge and shouldn't be able to access the datastore of your production app, then you would create a different GAE project for that.
Here is a way to visualize it:
Google Cloud Project A
GAE Project A
production www service
production API service (this could be combined with production www service)
dev www service
dev API service
Google Cloud Project B
GAE Project B
bleeding edge www service
bleeding edge API service
Best Practices:
Your Dev and Prod should be in separate projects for both security and billing purposes.
App Engine:
You can only have one App Engine per project. This will create a problem for you to use App Engine for both API Server and Web Server. In this case, I would not use App Engine at all and instead look at Containers on Compute Engine or go for Kubernetes.
Even a single node Kubernetes Cluster will shock you with its flexibility and power. Containers on Compute Engine still have a lot flexibility and power too. If you like the concepts of App Engine Flexible, then you might really like containers. The exception here is that App Engine makes some concepts brain dead simple where you have more work in configuration for Containers or Kubernetes.
The title says it all. Just to irritate DRY purists, I'll say it again.
Is it possible to get a list of instances my application is running on via gcloud?
In 2019, You can check the newest document : gcloud app services list - list your existing services
The newest command is:
gcloud app services list
The result will be something like this.
SERVICE NUM_VERSIONS
default 7
App Engine can mean traditional App Engine or Managed VMs, depending on the context. There is only one App Engine app per project, but you can have multiple modules (although only one of those module will serve web traffic, the rest are backend only) and multiple versions of each module.
As pgiecek mention,
gcloud preview app modules list
will show all the modules.
What you might be looking for is all your App Engine versions, for which you will still have to use appcfg
appcfg.py -A <your-project-id> [-M <module>] list_versions
If you are using Managed VMs, it will spin up your app on GCE instances, in which case
gcloud compute instances list
does in fact show your App Engine (MVM) instances.
It seems that currently it is not possible. At the moment gcloud tool supports App Engine (via Preview CLI command groups) but provides only limited number of operations and furthermore they are in BETA. You can check documentation here.
You can only list your existing deployed modules and versions as follows.
gcloud preview app modules list
As of Sep 2019 (Google Cloud SDK 237.0.0):
gcloud app instances list
is it possible to deploy to the same appengine project from multiple users? Like if I add another user to have access to my appengine project how can the other user also deploy to my appengine? Is this possible?
Yes, multiple users can deploy to the same AppEngine app id. Just give them each permission from the cloud console.
At that point, every authorized developer is the same and can deploy in the normal fashion. Just keep in mind that users can/will overwrite each other's deploys if they push the same version identifier.
The most recent deploy of the same version wins and overwrites the earlier deploy. So, make sure your team workflow accommodates this.