I have Nuxt application that is running in a repository in Google Cloud Platform, that uses App Engine.
Everytime, I want to deploy it, I need to:
Get in Google Cloud Platform Console
cd to my repo folder
git pull changes in default branch
log in with an authenticated user
run the command npm run generate ( i am using generate because i am this article for full static Nuxt-full static
And finally run gcloud app deploy app.yaml --project project_id
app.yaml configuration:
runtime: nodejs12
instance_class: F2
handlers:
- url: /_nuxt
static_dir: .nuxt/dist/client
secure: always
- url: /(.*\.(gif|png|jpg|ico|txt))$
static_files: static/\1
upload: static/.*\.(gif|png|jpg|ico|txt)$
secure: always
- url: /.*
script: auto
secure: always
env_variables:
HOST: '0.0.0.0'
I have been reading Google Cloud Platform and Gitlab documentation about CI/CD, but they look like they are outdated.
The question is:
how can I configure an CI/CD (.gitlab-ci.yml) file in my master branch for: when I update this branch, update my App Engine application and make a new build for it?
I appreciate every help possible, and wish everyone that sees this message a Happy New Year S2.
If you use Gitlab as a self-hosted version, You can make CI/CD pipeline that automates your everyday manual deployments.
First, Install npm package and gcloud SDK in your Gitlab instance.
Second, Edit your CI/CD Pipeline as you described. It can be helpful if you use CI/CD Pipeline editor, GitLab provides.
ex: My .gitlab-ci.yml contents here. It's just one step, but you can have many of it.
With GitLab 15.5 (Oct. 2022), you can also checkout:
Deploy apps to Google Cloud with GitLab Cloud Seed
Cloud Seed allows GitLab and Google Cloud customers to migrate to the cloud using a single platform, consolidating their tech stack without slowing down their cloud adoption process.
Cloud Seed is built into the GitLab web UI and leverages CI/CD pipeline capabilities. It is specifically tailored to offer a frictionless developer experience for consuming Google Cloud services, supporting Service Accounts, Cloud Run, and Cloud SQL.
To develop this capability, GitLab worked with Google Cloud to build best-in-class experiences to simplify, automate and thus accelerate cloud resource provisioning, deployment automation and configuration. Google Cloud and GitLab worked together in an open-source model to make Cloud Seed available for paid and free users.
Cloud Seed’s easy-to-use and accessible format drives organic cloud adoption amongst users. Since it is a frictionless, open-source tool, developers and product teams can consume the Google Cloud services complementing its leadership’s cloud adoption efforts leading to an organic company-wide shift. As a result, GitLab is seeing increased bottom-up adoption of cloud services.
See:
Documentation
Accelerate cloud adoption with GitLab's open source partnership with Google Cloud
Issue Board
Related
On GCP, I run Cloud Build from one project and deploy code to App Engine in another project. It looks like the project where build runs from needs to have App Engine Admin API enabled. Is it a real request or I missed the real configurations?
App Engine Admin API is required as it is used for any App Engine-related management operations
Also, based on the link above:
The Admin API provides you with:
An integration point for your development and build tools.
Tighter control around deploying new versions, including the ability to automate traffic
migration between two versions or traffic splitting across one or more versions.
The ability to programmatically manage applications across multiple Google Cloud projects.
So yes, if you plan on deploying code to App Engine using Cloud Build, you need GAE Admin API enabled.
We're migrating a website from a VPS at DigialOcean to Google Cloud. Google Cloud probably won't be cheaper but as it's a small website that we do on the side we don't want to spend time on dev-ops too much. The techniques we're using are:
AngularJS as frontend
Flask, with Python3, as backend
PostgreSQL as database, with SQLAlchemy as ORM
GitLab as git repository, with GitLab CI
The frontend and the backend are two separate projects, communication with REST, and we would like to keep it this way. The reason is that we want to deploy one project without having to deploy the other. Don't think that we're unique in that way.
We have 2 environments: one production and one test and would like to keep it that way.
We don't have any previous experience with Google Cloud and it's products, but all examples we've found serves the frontend on a static-route from the backend from within a Google App Engine. From our point of view this doesn't seem like a great separation of concern, and we don't want to have a monolithic repository.
Is it possible to keep our setup with two separate projects and two different environments with Google Cloud?
Some more information
backend/app.yaml
runtime: python37
service: test-flask
backend/dispatch.yaml
dispatch:
- url: "*/api*"
service: test-flask
frontend/app.yaml
runtime: nodejs10
service: test-angularjs
handlers:
- url: /(.*\.(js|css|svg)?(.*))
static_files: dist/\1
upload: dist/(.*)
- url: /(.*\.(png|xml|)?(.*))
static_files: dist/\1
upload: dist/assets/(.*)
- url: /
static_files: dist/index.html
upload: dist/index.html
- url: /(.*(a|pa|friends|faq).*)
static_files: dist/index.html
upload: dist/index.html
It is possible to keep the Backend and Frontend in two different projects with two different App Engine applications and use HTTP requests to communicate between your separate applications. Here you can find the useful information about how requests are handled by a Google App Engine application deployed with Node.js runtime (the Frontend) and for the same with Python 3.7 runtime (the Backend).
I would recommend you to start by reading the relevant section of the Google Cloud Platform's documentation and specially this article to get a grasp for all the possibilities offered by Google Cloud Platform for serving websites. A comprehensive, yet not complete lists of architectural possibilities could be:
In one project deploy one App Engine application and use different services for the Frontend and Backend. Find here all the relevant information about the available methods to communicate between the services.
Use other products like virtual machines with Compute Engine.
Dockerize your applications and use Cloud Run services.
You can also use Cloud SQL for the PostgreSQL database and use Cloud Build and Cloud Source Repositories for a CI/CD pipeline. You can find more information here and use this quickstart to understand CD for App Engine with Cloud Build, as you intend for your application.
I would like to know how to deploy the application from bitbucket using pipelines to multiple Google Cloud projects.
Here is our current set up and it is working fine.
On Bitbucket, the application repo with development/UAT test/production branches, once the pull request approved and merged into development/production, it shall deploy to the GCP app engine through the pipelines.
The problem now, we want to isolate each client in GCP which mean each client will have its own GCP project, cloud SQL, App engines, storage bucket, etc...
I need some advice on how to change the deployment workflow in bitbucket and pipelines, so will work for the new set up.
For the branches setup on bitbucket, I'm thinking like below, but if I go for option2, then it seems too much if got more clients.
Option 1 (repo branches)
development/
UAT test/
validation/
production
Option 2 (repo branches)
development/
UAT test client1/
UAT test client2/
validation_client1/
validation_client2/
production_client1/
production_client2/
The first step, I know I have to create different app.yaml for each app engine service for each client, so it can deploy the app engine service to different CGP projects/bucket/SQL instance.
Also just found out the bitbucket-pipelines.yml only support 10 steps, if I create so many branches then it will over the limits for sure.
Does anyone have any suggestions about how should be set up?
Thanks,
You could create Cloud build triggers to specific bitbucket branches or repos (whatever your branching model is defined) and deploy the app engine implementation to the App engine service on the same project, and if you need to customize other steps, you could use custom steps as described here. Finally you can take a look at how to create a basic configuration file for Cloud Build if you are not very familiar with this product
I have been using Google App Engine Standard Environment(Python 2.7) since some days now. It provides a good local development server, providing Task queues, datastore and other services locally.
Recently i have been exploring App Engine Flex Environment (Python 3.7) for some of the projects. Within that i am trying use various services like Cloud Tasks, GCS, Pubsub etc. Some of the services like pubsub has an emulator to run it locally. I couldn't find such emulator for cloud tasks.
Is there any way to emulate/simulate Cloud Tasks locally?
I just built an emulator, see if it meets your needs.
Unfortunately, local server emulator is not available for App Engine Flex Environment (Python 3.7) [1] However you can use Cloud Pub/Sub in some cases as workaround [2].
[1] https://cloud.google.com/tasks/docs/migrating#features_in_task_queues_not_yet_available_via
[2] https://cloud.google.com/tasks/docs/comp-pub-sub
I built an in-process emulator for Python development.
See also some emulators that run in a separate process in localhost: Potato London’s gcloud-tasks-emulator and Aert van de Hulsbeek’s cloud-tasks-emulator, mentioned in the answer above.
I have a simple static website and I was wondering if I can host my website in google cloud?
I am not sure how much this can cost and how I can figure this out.
I already hosting my website somewhere but I am not happy with their email services. And I was thinking if I can host with Google Cloud and I can use google email as well.
Thank you!
If your requirement is to host static website (with not much traffic), I would suggest Google App Engine Standard than Google Cloud engine. With google app engine standard, if your site doesn't have any hits, it would be automatically taken down. Once there is a hit, it would be brought back automatically. You would save lot of money. Check the prices at https://cloud.google.com/pricing/
Regarding email, if you want google to host your business emails, its not free, you might want to start at https://gsuite.google.com/products/gmail/
Yes, Google App Engine can host your static website. Here's how I have my site setup.
main.go
package main
import (
"net/http"
)
func init() {
fs := http.FileServer(http.Dir("web"))
http.Handle("/", fs)
}
app.yaml
runtime: go
api_version: go1
handlers:
- url: /.*
script: _go_app
You'll need the Google Cloud SDK and app-engine-go component to test locally and deploy your website.
https://cloud.google.com/appengine/docs/standard/go/download
Once that's installed you can open a terminal and navigate to the folder with app.yaml in it then run gcloud app deploy
After the deploy is finished a new version will show up in your app engine account. You can test the deployed version by clicking on the version name in the table.
https://console.cloud.google.com/appengine/versions
Then you'll need to follow the instruction to point your domain to google's hosting
https://console.cloud.google.com/appengine/settings/domains
If you use more than the free quota then they'll charge you.
For example, 1GB of bandwidth per day is free then the $.12/GB after that
https://cloud.google.com/appengine/quotas#Requests
You can set your daily spending limit to $0 to prevent charges, but this will cause your website to go down once your free quote is reached.
https://console.cloud.google.com/appengine/settings
You can keep an eye on the cost from the app engine dashboard
https://console.cloud.google.com/appengine
You can but it's depend upon your requirement, like open source plateform can be host in share hosting whereas nodejs etc must be host ion dedicated server example vps.
Yes:
The most cost effective way of hosting a static (HTML) website on Google Cloud Platform is to store it in a cloud storage bucket. Since Cloud Storage comes with an always free tier, this can save a lot of money over time. See this article for details on hosting it on Cloud Storage https://cloud.google.com/storage/docs/hosting-static-website
Google's Firebase hosting would be another viable option.
Email:
IAAS/SAAS Email is not automatically built into hosting on GCP, but if you transfer your domain to Google Domains then you can seamlessly sign up for Googles G-Suite [Gmail based] email for your domain.