I'm having trouble setting up my yaml files for google app engine. The configuration works correctly when my app.yaml file is in the root of the project but if it is within a subdirectory it does not build the correct source. I suspect I need to set the dir: option in the build config, but I have tried multiple variations and I can't get it to work.
Working file structure, deployed app is ~3mb in size.
src
deployment
└── staging
└── build.yaml
app.staging.yaml
# build.yaml
steps:
- name: node:12
entrypoint: yarn
- name: node:12
entrypoint: yarn
args: ['build']
- name: "gcr.io/cloud-builders/gcloud"
args: ["app", "deploy", "app.staging.yaml"]
timeout: "1800s"
Not working file structure, deployed app is ~1kb in size.
src
deployment
└── staging
└── build.yaml
└── app.yaml
# build.yaml
steps:
- name: node:12
entrypoint: yarn
- name: node:12
entrypoint: yarn
args: ['build']
- name: "gcr.io/cloud-builders/gcloud"
args: ["app", "deploy", "deployment/staging/app.yaml"]
timeout: "1800s"
In both scenarios I am kicking off the deployment with:
gcloud builds submit --config deployment/staging/build.yaml
What should my dir: be set to in the build.yaml steps so that the build step knows to build from root? Is there any way to debug this locally without having to upload the source every time?
Thanks!
A
you cannot have the app.yaml and the cloudbuild.yaml in the same directory if you are deploying in a non-custom runtime. Please see this comment
I believe by default cloud build expects the app.yaml file to be in the uppermost directory. I'm not sure if it's possible to change this in the cloud build settings.
Related
i am Kinda new to cloud build so i am kind of confused about what is happening.
first this is my file structure
cloudbuild.yaml
backend/
Dockerfile
app.yaml
I had an application which i dockerized and deployed to app engine felx in custom runtime
here's my Dockerfile
FROM mcr.microsoft.com/dotnet/sdk:5.0-buster-slim AS build
WORKDIR /app
# Copy csproj and restore as distinct layers
COPY *.csproj ./
RUN dotnet restore
COPY . ./
RUN dotnet publish -c Release -o out
FROM mcr.microsoft.com/dotnet/aspnet:5.0-buster-slim AS base
ENV ASPNETCORE_URLS=http://+:80;
WORKDIR /app
COPY --from=build /app/out .
EXPOSE 80
ENTRYPOINT ["dotnet", "myapp.dll"]
and this is my app engine flex file
runtime: custom
env: flex
manual_scaling:
instances: 1
resources:
cpu: 1
memory_gb: 0.5
disk_size_gb: 10
service: backend
network:
name: my-network
subnetwork_name: my-network-subnet
instance_tag: "backend"
forwarded_ports:
I have successfully deployed this app on app engine flex using this command
gcloud app deploy --appyaml=app.yaml
Then i added a cloudbuild.yaml file following this google doc
steps:
- name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
entrypoint: 'bash'
args: ['-c', 'gcloud config set app/cloud_build_timeout 2000 && gcloud app deploy --appyaml=backend/app.yaml']
as you can see in the cloudbuild.yaml i didnt add the timeout attribute because it gave me this error each time i tried to submit the build.
Error Response: [13] Error parsing cloudbuild.yaml for runtime custom: Argument is not an object: "2000s"
after removing the timeout attribute, cloud build started behaving in a weird way, it kept creating build jobs on its own until it reached over 20 builds.
i had to stop these builds manually because it exceeded the 120 minute free quota limit.
can some one tell me if my cloudbuild.yaml is the thing causing the issue or if its a problem with google cloud.
So the problem was writing the cloudbuild as a yaml file, instead i re-wrote it as a json file, i am not entirely sure why is the cloudbuild.yaml file was giving me errors but that was my solution.
{
"steps": [
{
"name": "gcr.io/google.com/cloudsdktool/cloud-sdk",
"entrypoint": "bash",
"args": [
"-c",
"gcloud config set app/cloud_build_timeout 1600 && gcloud app deploy --appyaml=app.yaml"
]
}
],
"timeout": "1600s"
}
Also the cloudbuild and app.yaml must be in the root of the branch with the cloudbuild file and Dockerfile.
I tried to deploy a Nest JS app to App Engine through Google Build Cloud & Manually
Both tries gives same error Modules not found
Error when deploying manually through Shell
Error when dploying using clouf build
My cloudbuild.yaml
- name: node:14.15.1
entrypoint: npm
args: ["install"]
- name: node:14.15.1
entrypoint: npm
args: ["run", "build"]
- name: node:14.15.1
entrypoint: npm
args: ["run", "create-env"]
env:
- "_APP_LUNE_DEV_TOKEN=${_APP_LUNE_DEV_TOKEN}"
- "_BUCKET_NAME=${_BUCKET_NAME}"
- "_FIREBASE_API_KEY=${_FIREBASE_API_KEY}"
- "_FIREBASE_APP_ID=${_FIREBASE_APP_ID}"
- "_FIREBASE_AUTH_DOMAIN=${_FIREBASE_AUTH_DOMAIN}"
- "_FIREBASE_MESSAGING_SENDER_ID=${_FIREBASE_MESSAGING_SENDER_ID}"
- "_FIREBASE_PROJECT_ID=${_FIREBASE_PROJECT_ID}"
- "_FIREBASE_STORAGE_BUCKET=${_FIREBASE_STORAGE_BUCKET}"
- "_GOOGLE_APPLICATION_CREDENTIALS=${_GOOGLE_APPLICATION_CREDENTIALS}"
- "_LUNE_API_BASE_URL=${_LUNE_API_BASE_URL}"
- "_STRIPE_SECRET_KEY=${_STRIPE_SECRET_KEY}"
- name: "gcr.io/cloud-builders/gcloud"
args: ["app", "deploy"]
timeout: "1600s"
options:
logging: CLOUD_LOGGING_ONLY
My app.yaml
runtime: nodejs14
service: backend
What I'm doing wrong ?
The problem was caused by Nest Js CLI Aliases, When I changed src/example/service to ../example/service the deployment worked
It's recommended to use typescript aliases in tsconfig file instead of CLI
I'm trying to publish a create-react-app on Gitlab, using its CI.
This is my .gitlab-ci.yml file:
stages:
- build
- pages
build:
image: node:16
stage: build
script:
- npm install
- npm build
artifacts:
paths:
- build/
pages:
image: alpine:latest
stage: deploy
variables:
GIT_STRATEGY: none # Do not clone git repo
script:
- mv build public
artifacts:
paths:
- public
rules:
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
But unfortunately I get this error:
This GitLab CI configuration is invalid: pages job: chosen stage does not exist; available stages are .pre, build, pages, .post.
What's wrong with my configuration?
You need to change your stages.
You have:
stages:
- build
- pages
but define:
pages:
image: alpine:latest
stage: deploy
The stage needs to match:
stages:
- build
- deploy <-------
I have a Django backend and an Angular frontend and i'm trying to deploy both of them to a Google App Engine Standard app as two services within the same app.
The directory looks like this:
-backend
--cloudbuild.yaml
--app.yaml
-- ...
-frontend
--cloudbuild.yaml
--app.yaml
-- ...
-cloudbuild.yaml
The main cloudbuild.yaml goes through each folder and looks for a cloudbuild.yaml file and submits the build:
steps:
- name: 'gcr.io/cloud-builders/gcloud'
entrypoint: 'bash'
args:
- '-c'
- |
for d in */; do
config="${d}cloudbuild.yaml"
if [[ ! -f "${config}" ]]; then
continue
fi
echo "Building $d ... "
(
gcloud builds submit $d --config=${config}
) &
done
wait
When i push the code to my Github repo, they get deployed successfully. However, both logs show something like below:
Step #2: Beginning deployment of service [default]...
Step #2: #============================================================#
Step #2: #= Uploading 0 files to Google Cloud Storage =#
Step #2: #============================================================#
Step #2: File upload done.
Step #2: Updating service [default]...
Step #2: ................................done.
Step #2: Setting traffic split for service [default]...
Step #2: .....done.
Step #2: Deployed service [default] to [https://thisapp.appspot.com]
So it seems like one is overriding the other as the default service. When I go to the services page, there is only one service which is the default one.
I know that we first have to deploy the default service and then the rest. However i thought since the main cloudbuild.yaml submits the build one by one, then there would be a default service and the second build would create a second service.
The other two cloudbuild.yaml files are as below:
steps:
# Install node packages
- name: 'gcr.io/cloud-builders/npm'
args: [ 'install' ]
# Build productive files
- name: 'gcr.io/cloud-builders/npm'
args: [ 'run', 'build', '--prod']
# Deploy to google cloud app egnine
- name: 'gcr.io/cloud-builders/gcloud'
args: ['app', 'deploy', '--version=prod']
steps:
- name: 'python:3.7'
entrypoint: python3
args: ['-m', 'pip', 'install', '-t', '.', '-r', 'requirements.txt']
- name: 'python:3.7'
entrypoint: python3
args: ['./manage.py', 'collectstatic', '--noinput']
- name: 'gcr.io/cloud-builders/gcloud'
args: ['app', 'deploy', '--version=prod']
I'd be grateful if you can help me on this.
Thanks
It is happening because in your app.yaml file, you have not provided the service tag. If service tag is not provided, the service will be deployed as default service. For e.g. when you deploy your frontend, it is getting deployed as default service. And then you deploy your backend, it is also getting deployed as default service overriding the existing default service. or the vice versa.
Add the service tag in your frontend's app.yaml:
service: frontend
runtime: nodejs10
You may keep the backend as default service i.e. no service tag in backend's app.yaml
I learned how to use Container Registry trigger for Google Cloud Functions deploy from the following tutorial.
Automatic serverless deployments with Cloud Source Repositories and Container Builder
I have Google App engine flexible app. The runtime is Node.js. I want to deploy the app triggered by git push. Are there any good references?
I'm using these example code. Manual deployment works normally.
* tree
.
├── app.js
├── app.yaml
└── package.json
* app.js
'use strict';
const express = require('express');
const app = express();
app.get('/', (req, res) => {
res.status(200).send('Hello, world!').end();
});
const PORT = process.env.PORT || 8080;
app.listen(PORT, () => {
console.log(`App listening on port ${PORT}`);
console.log('Press Ctrl+C to quit.');
});
* app.yaml
runtime: nodejs
env: flex
* package.json
{
"name": "appengine-hello-world",
"description": "Simple Hello World Node.js sample for Google App Engine Flexible Environment.",
"version": "0.0.1",
"private": true,
"license": "Apache-2.0",
"author": "Google Inc.",
"repository": {
"type": "git",
"url": "https://github.com/GoogleCloudPlatform/nodejs-docs-samples.git"
},
"engines": {
"node": ">=4.3.2"
},
"scripts": {
"deploy": "gcloud app deploy",
"start": "node app.js",
"lint": "samples lint",
"pretest": "npm run lint",
"system-test": "samples test app",
"test": "npm run system-test",
"e2e-test": "samples test deploy"
},
"dependencies": {
"express": "4.15.4"
},
"devDependencies": {
"#google-cloud/nodejs-repo-tools": "1.4.17"
},
"cloud-repo-tools": {
"test": {
"app": {
"msg": "Hello, world!"
}
},
"requiresKeyFile": true,
"requiresProjectId": true
}
}
* deploy command
$ gcloud app deploy
Update 1
I found a similar question.
How to auto deploy google app engine flexible using Container Registry with Build Trigger
I added cloudbuild.yaml.
steps:
# Build the Docker image.
- name: gcr.io/cloud-builders/docker
args: ['build', '-t', 'gcr.io/$PROJECT_ID/app', '.']
# Push it to GCR.
- name: gcr.io/cloud-builders/docker
args: ['push', 'gcr.io/$PROJECT_ID/app']
# Deploy your Flex app from the image in GCR.
- name: gcr.io/cloud-builders/gcloud
args: ['app', 'deploy', 'app.yaml', '--image-url=gcr.io/$PROJECT_ID/app']
# Note that this build pushes this image.
images: ['gcr.io/$PROJECT_ID/app']
However, I got an error. The error message is "error loading template: yaml: line 5: did not find expected key". I'm looking into it.
Update 2
The reason was invalid yaml format. I changed it like the following.
steps:
# Build the Docker image.
- name: gcr.io/cloud-builders/docker
args: ['build', '-t', 'gcr.io/$PROJECT_ID/app', '.']
# Push it to GCR.
- name: gcr.io/cloud-builders/docker
args: ['push', 'gcr.io/$PROJECT_ID/app']
# Deploy your Flex app from the image in GCR.
- name: gcr.io/cloud-builders/gcloud
args: ['app', 'deploy', 'app.yaml', '--image-url=gcr.io/$PROJECT_ID/app']
# Note that this build pushes this image.
images: ['gcr.io/$PROJECT_ID/app']
I got another error. The message is "error loading template: unknown field "images" in cloudbuild_go_proto.BuildStep"
Update 3
I noticed that "images" indent was wrong.
steps:
...
# Note that this build pushes this image.
images: ['gcr.io/$PROJECT_ID/app']
I encountered new error.
starting build "e3e00749-9c70-4ac7-a322-d096625b695a"
FETCHSOURCE
Initialized empty Git repository in /workspace/.git/
From https://source.developers.google.com/p/xxxx/r/bitbucket-zono-api-btc
* branch 0da6c8bf209c72b6406f3801f3eb66d346187f4e -> FETCH_HEAD
HEAD is now at 0da6c8b fix invalid yaml
BUILD
Starting Step #0
Step #0: Already have image (with digest): gcr.io/cloud-builders/docker
Step #0: unable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /workspace/Dockerfile: no such file or directory
Finished Step #0
ERROR
ERROR: build step 0 "gcr.io/cloud-builders/docker" failed: exit status 1
Yes. I don't have Dockerfile because I use Google App Engine flexible Environment Node.js runtime. It is not necessary Docker.
Update 4
I added Dockerfile
FROM gcr.io/google-appengine/nodejs
Then new error was occurred.
Step #2: ERROR: (gcloud.app.deploy) User [xxxxxxx#cloudbuild.gserviceaccount.com] does not have permission to access app [xxxx] (or it may not exist): App Engine Admin API has not been used in project xxx before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/appengine.googleapis.com/overview?project=xxx then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
Update 5
I enabled App Engine Admin API then next error has come.
Step #2: Do you want to continue (Y/n)?
Step #2: WARNING: Unable to verify that the Appengine Flexible API is enabled for project [xxx]. You may not have permission to list enabled services on this project. If it is not enabled, this may cause problems in running your deployment. Please ask the project owner to ensure that the Appengine Flexible API has been enabled and that this account has permission to list enabled APIs.
Step #2: Beginning deployment of service [default]...
Step #2: WARNING: Deployment of service [default] will ignore the skip_files field in the configuration file, because the image has already been built.
Step #2: Updating service [default] (this may take several minutes)...
Step #2: ...............................................................................................................................failed.
Step #2: ERROR: (gcloud.app.deploy) Error Response: [9]
Step #2: Application startup error:
Step #2: npm ERR! path /app/package.json
Step #2: npm ERR! code ENOENT
Step #2: npm ERR! errno -2
Step #2: npm ERR! syscall open
Step #2: npm ERR! enoent ENOENT: no such file or directory, open '/app/package.json'
Step #2: npm ERR! enoent This is related to npm not being able to find a file.
I changed my code tree but it did not work. I confirmed that Appengine Flexible API has been enabled. I have no idea what should I try next.
.
├── Dockerfile
├── app
│ ├── app.js
│ └── package.json
├── app.yaml
└── cloudbuild.yaml
Update 6
When I deploy manually, the artifact is like the following.
us.gcr.io/xxxxx/appengine/default.20180316t000144
Should I use this artifact...? I'm confused..
Update 7
Two builds are executed. I don't know whether this is correct.
Your Dockerfile doesn't copy source to the image.
You can move everything back to the same directory such that
.
├── app.js
├── app.yaml
├── cloudbuild.yaml
├── Dockerfile
└── package.json
but it doesn't matter.
Paste this into your Dockerfile and it should work:
FROM gcr.io/google-appengine/nodejs
# Working directory is where files are stored, npm is installed, and the application is launched
WORKDIR /app
# Copy application to the /app directory.
# Add only the package.json before running 'npm install' so 'npm install' is not run if there are only code changes, no package changes
COPY package.json /app/package.json
RUN npm install
COPY . /app
# Expose port so when container is launched you can curl/see it.
EXPOSE 8080
# The command to execute when Docker image launches.
CMD ["npm", "start"]
Edit: This is the cloudbuild.yaml I used:
steps:
- name: gcr.io/cloud-builders/docker
args: ['build', '-t', 'gcr.io/$PROJECT_ID/app', '.']
- name: gcr.io/cloud-builders/docker
args: ['push', 'gcr.io/$PROJECT_ID/app']
- name: gcr.io/cloud-builders/gcloud
args: ['app', 'deploy', 'app.yaml', '--image-url=gcr.io/$PROJECT_ID/app']
images: ['gcr.io/$PROJECT_ID/app']
A tech guy helped me. I changed directory structure and cloudbuild.yaml. Then it worked. Thanks.
* Code Tree
.
├── app
│ ├── app.js
│ ├── app.yaml
│ └── package.json
└── cloudbuild.yaml
* cloudbuild.yaml
steps:
- name: gcr.io/cloud-builders/npm
args: ['install', 'app']
- name: 'gcr.io/cloud-builders/gcloud'
args: ['app', 'deploy', 'app/app.yaml']