We're migrating a website from a VPS at DigialOcean to Google Cloud. Google Cloud probably won't be cheaper but as it's a small website that we do on the side we don't want to spend time on dev-ops too much. The techniques we're using are:
AngularJS as frontend
Flask, with Python3, as backend
PostgreSQL as database, with SQLAlchemy as ORM
GitLab as git repository, with GitLab CI
The frontend and the backend are two separate projects, communication with REST, and we would like to keep it this way. The reason is that we want to deploy one project without having to deploy the other. Don't think that we're unique in that way.
We have 2 environments: one production and one test and would like to keep it that way.
We don't have any previous experience with Google Cloud and it's products, but all examples we've found serves the frontend on a static-route from the backend from within a Google App Engine. From our point of view this doesn't seem like a great separation of concern, and we don't want to have a monolithic repository.
Is it possible to keep our setup with two separate projects and two different environments with Google Cloud?
Some more information
backend/app.yaml
runtime: python37
service: test-flask
backend/dispatch.yaml
dispatch:
- url: "*/api*"
service: test-flask
frontend/app.yaml
runtime: nodejs10
service: test-angularjs
handlers:
- url: /(.*\.(js|css|svg)?(.*))
static_files: dist/\1
upload: dist/(.*)
- url: /(.*\.(png|xml|)?(.*))
static_files: dist/\1
upload: dist/assets/(.*)
- url: /
static_files: dist/index.html
upload: dist/index.html
- url: /(.*(a|pa|friends|faq).*)
static_files: dist/index.html
upload: dist/index.html
It is possible to keep the Backend and Frontend in two different projects with two different App Engine applications and use HTTP requests to communicate between your separate applications. Here you can find the useful information about how requests are handled by a Google App Engine application deployed with Node.js runtime (the Frontend) and for the same with Python 3.7 runtime (the Backend).
I would recommend you to start by reading the relevant section of the Google Cloud Platform's documentation and specially this article to get a grasp for all the possibilities offered by Google Cloud Platform for serving websites. A comprehensive, yet not complete lists of architectural possibilities could be:
In one project deploy one App Engine application and use different services for the Frontend and Backend. Find here all the relevant information about the available methods to communicate between the services.
Use other products like virtual machines with Compute Engine.
Dockerize your applications and use Cloud Run services.
You can also use Cloud SQL for the PostgreSQL database and use Cloud Build and Cloud Source Repositories for a CI/CD pipeline. You can find more information here and use this quickstart to understand CD for App Engine with Cloud Build, as you intend for your application.
Related
I have Nuxt application that is running in a repository in Google Cloud Platform, that uses App Engine.
Everytime, I want to deploy it, I need to:
Get in Google Cloud Platform Console
cd to my repo folder
git pull changes in default branch
log in with an authenticated user
run the command npm run generate ( i am using generate because i am this article for full static Nuxt-full static
And finally run gcloud app deploy app.yaml --project project_id
app.yaml configuration:
runtime: nodejs12
instance_class: F2
handlers:
- url: /_nuxt
static_dir: .nuxt/dist/client
secure: always
- url: /(.*\.(gif|png|jpg|ico|txt))$
static_files: static/\1
upload: static/.*\.(gif|png|jpg|ico|txt)$
secure: always
- url: /.*
script: auto
secure: always
env_variables:
HOST: '0.0.0.0'
I have been reading Google Cloud Platform and Gitlab documentation about CI/CD, but they look like they are outdated.
The question is:
how can I configure an CI/CD (.gitlab-ci.yml) file in my master branch for: when I update this branch, update my App Engine application and make a new build for it?
I appreciate every help possible, and wish everyone that sees this message a Happy New Year S2.
If you use Gitlab as a self-hosted version, You can make CI/CD pipeline that automates your everyday manual deployments.
First, Install npm package and gcloud SDK in your Gitlab instance.
Second, Edit your CI/CD Pipeline as you described. It can be helpful if you use CI/CD Pipeline editor, GitLab provides.
ex: My .gitlab-ci.yml contents here. It's just one step, but you can have many of it.
With GitLab 15.5 (Oct. 2022), you can also checkout:
Deploy apps to Google Cloud with GitLab Cloud Seed
Cloud Seed allows GitLab and Google Cloud customers to migrate to the cloud using a single platform, consolidating their tech stack without slowing down their cloud adoption process.
Cloud Seed is built into the GitLab web UI and leverages CI/CD pipeline capabilities. It is specifically tailored to offer a frictionless developer experience for consuming Google Cloud services, supporting Service Accounts, Cloud Run, and Cloud SQL.
To develop this capability, GitLab worked with Google Cloud to build best-in-class experiences to simplify, automate and thus accelerate cloud resource provisioning, deployment automation and configuration. Google Cloud and GitLab worked together in an open-source model to make Cloud Seed available for paid and free users.
Cloud Seed’s easy-to-use and accessible format drives organic cloud adoption amongst users. Since it is a frictionless, open-source tool, developers and product teams can consume the Google Cloud services complementing its leadership’s cloud adoption efforts leading to an organic company-wide shift. As a result, GitLab is seeing increased bottom-up adoption of cloud services.
See:
Documentation
Accelerate cloud adoption with GitLab's open source partnership with Google Cloud
Issue Board
I have an API that I host through Google Cloud. The main entry point for the API is an App Engine instance (standard), which then needs to be able to call various cloud functions to execute its tasks.
To make the cloud functions secure, I want to set the ingress controls to "Allow internal only".
I've deployed the cloud functions and the App Engine in the same region (us-central1), but every time the App Engine instance tries to call a cloud function, it gets a 403 error. I've tried setting up a VPC connector for the App Engine, but that isn't helping.
Here's the app.yaml file:
service: my-test-app
runtime: nodejs10
env_variables:
STAGE: "dev"
instance_class: F1
vpc_access_connector:
name: "projects/my-test-project/locations/us-central1/connectors/test-vpc-connector"
The key cloud function configuration is the Ingress Setting. I have it set to Allow internal traffic only. If I set it to Allow all traffic, everything works (just highly insecure!)
Any suggestions on what I might be missing?
I found the solution! If I set the App Engine to a flex environment, everything works. It looks like the standard environment was the issue.
I have a static web client SPA serviced by a REST API. I'm trying to figure out the best way to host these apps on Google's Cloud Platform using App Engine to host the API, and Cloud Storage to host the static web client.
If I were doing this from scratch, a simple reverse proxy could manage routing traffic between the API and the client assets. To do the equivalent with GCP, I've looked at the following:
Google's Compute Engine supports content-based load balancing: though no equivalent for App Engine
the API on App Engine could proxy requests to Cloud Storage, though at the expense of unnecessary load on the API service
simply host the API and client on separate domains (App Engine and Cloud Storage respectively), and properly configure cross origin issues
Use Google Cloud Endpoints as a reverse proxy to route traffic appropriately between App Engine and Cloud Storage: haven't fully explored this option, though as of writing, Cloud Endpoints does not support routing to multiple hosts (which is defined only in v3 of the OpenAPI spec).
All of the above have limitations. What i'm trying to do seems fairly conventional, but I'm not sure what the path of least resistance is on GCP.
Google Cloud storage allow you to host a static website :
https://cloud.google.com/storage/docs/hosting-static-website
You don't need to use Endpoint or AppEngine as a reverse proxy
If you need to setup a load balancer based on route or if you need to setup ssl certificates you could use storage bucket as a service backend :
https://cloud.google.com/compute/docs/load-balancing/http/backend-bucket
Let's talk about serving SPA static files from Google App Engine.
The SPAs need to serve many routes to a single index.html, normally called rewrite rules.
App Engine can do that with a proper configured app.yaml handlers section.
For the real files part, you serve the real path:
- url: /assets/
static_dir: path/to/real/files
For these fake routes, serve the entrypoint index.html:
- url: /
static_file: path/to/index.html
upload: path/to/index.html
- url: /.*
static_file: path/to/index.html
upload: path/to/index.html
By this configuration, Google Frontend will serve the static files without hitting your backend.
Here's one Angular application and I deployed to App Engine, as an example:
Other stuff about securing APIs and CORS policies, you can consider using dispatch.yaml to avoid cross domain problem. Or serve from different domain with cloud endpoints (with IAP jwk configured).
As you have rightly observed, there are a number of complications that might come into play with your setup. The Google Cloud Storage is simply a Storage, which might not necessarily manage requests to GAE as well as you desire. Perhaps, using Endpoints would be a more viable solution in this case (considering your listed options), where you can use simple Javascripts to call Endpoints in your GAE applications from your Application Files in Google Cloud Storage. However, that being said, I think the better option is to move your static files into App Engine as described here. This will ease the complication of managing resources between two different technologies
How can I deploy multiple applications from same or different language/runtime originating from a single project in google cloud app engine?
Deploying Multiple Services to Google Cloud App engine.
Create the following files in the app root directory
Create app.yaml file with content:
runtime: nodejs14
service: default
Create myserviceone.yaml file with content:
runtime: nodejs14
service: myserviceone
Create myservicetwo.yaml file with content:
runtime: nodejs14
service: myservicetwo
Deploy using command
gcloud app deploy --project your_project_id app.yaml myserviceone.yaml myservicetwo.yaml
This will deploy the default service as well as my service one and my service two. Check out your cloud console.
You can access them using:
https://myserviceone-dot-yourProjectID.appspot.com/ or
http://myserviceone.yourProjectID.appspot.com/
https://myservicetwo-dot-yourProjectID.appspot.com/ or
http://myservicetwo.yourProjectID.appspot.com/
Today, App Engine has a one-to-one correspondence with a Google Cloud Console project. You cannot deploy multiple "apps" in the same project. However, you might still be able to do what you want depending on your application(s).
App Engine has the concept of "services," which are independent aspects of your application. Your App Engine app can have many services and each service can have its own language/runtime and even be on different App Engine environments.
You could have say a Python service on App Engine standard environment that is used to serve your simple Flask site, could have service that serves an API written in Java 8 on the Standard Environment, and could have yet another service in say Node.js running in the App Engine flexible environment.
Your "default" service is defined in your app.yaml file. Your other services can be defined in different folders and can have either an app.yaml with their service definitions or you could name them something else like backend.yaml. See this simple Python project for the layout of the configuration files.
For more conceptual information about services on App Engine, see Microservices Architecture on Google App Engine.
I have a simple static website and I was wondering if I can host my website in google cloud?
I am not sure how much this can cost and how I can figure this out.
I already hosting my website somewhere but I am not happy with their email services. And I was thinking if I can host with Google Cloud and I can use google email as well.
Thank you!
If your requirement is to host static website (with not much traffic), I would suggest Google App Engine Standard than Google Cloud engine. With google app engine standard, if your site doesn't have any hits, it would be automatically taken down. Once there is a hit, it would be brought back automatically. You would save lot of money. Check the prices at https://cloud.google.com/pricing/
Regarding email, if you want google to host your business emails, its not free, you might want to start at https://gsuite.google.com/products/gmail/
Yes, Google App Engine can host your static website. Here's how I have my site setup.
main.go
package main
import (
"net/http"
)
func init() {
fs := http.FileServer(http.Dir("web"))
http.Handle("/", fs)
}
app.yaml
runtime: go
api_version: go1
handlers:
- url: /.*
script: _go_app
You'll need the Google Cloud SDK and app-engine-go component to test locally and deploy your website.
https://cloud.google.com/appengine/docs/standard/go/download
Once that's installed you can open a terminal and navigate to the folder with app.yaml in it then run gcloud app deploy
After the deploy is finished a new version will show up in your app engine account. You can test the deployed version by clicking on the version name in the table.
https://console.cloud.google.com/appengine/versions
Then you'll need to follow the instruction to point your domain to google's hosting
https://console.cloud.google.com/appengine/settings/domains
If you use more than the free quota then they'll charge you.
For example, 1GB of bandwidth per day is free then the $.12/GB after that
https://cloud.google.com/appengine/quotas#Requests
You can set your daily spending limit to $0 to prevent charges, but this will cause your website to go down once your free quote is reached.
https://console.cloud.google.com/appengine/settings
You can keep an eye on the cost from the app engine dashboard
https://console.cloud.google.com/appengine
You can but it's depend upon your requirement, like open source plateform can be host in share hosting whereas nodejs etc must be host ion dedicated server example vps.
Yes:
The most cost effective way of hosting a static (HTML) website on Google Cloud Platform is to store it in a cloud storage bucket. Since Cloud Storage comes with an always free tier, this can save a lot of money over time. See this article for details on hosting it on Cloud Storage https://cloud.google.com/storage/docs/hosting-static-website
Google's Firebase hosting would be another viable option.
Email:
IAAS/SAAS Email is not automatically built into hosting on GCP, but if you transfer your domain to Google Domains then you can seamlessly sign up for Googles G-Suite [Gmail based] email for your domain.