I want to deploy an update to our app but we have an issue with gcloud command which failed to "Conflicting SHA1 sum for file". Here is the output:
➜ gcloud app deploy xxx-xxx-xxxx.yaml --project xxx-xxx-xxxx --quiet
Services to deploy:
descriptor: [/home/vzool/Workspace/xxx-xxxx-xxxxxxxxxxx/xxx-xxx-xxxx.yaml]
source: [/home/vzool/Workspace/xxx-xxxx-xxxxxxxxxxx]
target project: [xxx-xxx-xxxx]
target service: [default]
target version: [20180917t110153]
target url: [https://xxx-xxx-xxxx.appspot.com]
Beginning deployment of service [default]...
╔════════════════════════════════════════════════════════════╗
╠═ Uploading 1 file to Google Cloud Storage ═╣
╚════════════════════════════════════════════════════════════╝
File upload done.
Updating service [default]...failed.
ERROR: (gcloud.app.deploy) Error Response: [3] The following errors occurred while copying files to App Engine:
File https://storage.googleapis.com/staging.xxx-xxx-xxxx.appspot.com/c819d7599ceb31df8d4370eed1981defcb5034c6 failed with: Conflicting SHA1 sum for file. Expected "ddb6af3b_cf4bff57_613fb78d_208480fa_1717b3c4" but received "c819d759_9ceb31df_8d4370ee_d1981def_cb5034c6".
Details: [
[
{
"#type": "type.googleapis.com/google.rpc.ResourceInfo",
"description": "Conflicting SHA1 sum for file. Expected \"ddb6af3b_cf4bff57_613fb78d_208480fa_1717b3c4\" but received \"c819d759_9ceb31df_8d4370ee_d1981def_cb5034c6\".",
"resourceName": "https://storage.googleapis.com/staging.xxx-xxx-xxxx.appspot.com/c819d7599ceb31df8d4370eed1981defcb5034c6",
"resourceType": "file"
}
]
]
I do delete this file c819d7599ceb31df8d4370eed1981defcb5034c6 but everytime new hash will be generated, delete entire block staging.xxx-xxx-xxxx.appspot.com and I did clear Memcache but with no luck trying all these options! :(
I don't know what to do anymore with this, any suggestions may help me out?
Thanks
I think after sometime in research the issue, the app itself has a live feature which lead to update one file while you open live page.
So, that change what gcloud uploaded.
gcloud upload then verify. And if file changed after upload and before verify gcloud will through this error on your face. Ha ha ha ^_^
OP's answer seemed to indicate that a file got touched during the gcloud deploy process.
The solution for me was to:
git checkout some random branch
git checkout the branch I was actually trying to deploy again
run gcloud app deploy . --version my_version_id --project my_project_id
Related
I started working on an app and suddenly it fails to deploy to appengine, with the following error message:
siim#pebble:~/projects/xyz$ gcloud app deploy
Services to deploy:
descriptor: [/home/siim/projects/xyz/app.yaml]
source: [/home/siim/projects/xyz]
target project: [xyz]
target service: [default]
target version: [20220313t182940]
target url: [https://xyz.uc.r.appspot.com]
target service account: [App Engine default service account]
Do you want to continue (Y/n)? y
Beginning deployment of service [default]...
╔════════════════════════════════════════════════════════════╗
╠═ Uploading 2 files to Google Cloud Storage ═╣
╚════════════════════════════════════════════════════════════╝
File upload done.
Updating service [default]...failed.
ERROR: (gcloud.app.deploy) Error Response: [9] Cloud build 016c4604-6ea6-47df-8144-e7f1471c6df6 status: FAILURE
removing /layers/google.go.gomod/gopath: unlinkat /layers/google.go.gomod/gopath/pkg/mod/github.com/golang/protobuf#v1.3.1/regenerate.sh: permission denied
Full build logs: https://console.cloud.google.com/cloud-build/builds;region=us-central1/016c4604-6ea6-47df-8144-e7f1471c6df6?project=576574664421
Clicking through to the build log, I see:
===> DETECTING
google.go.appengine_gomod 0.9.0
google.go.gomod 0.9.0
google.go.build 0.9.0
google.go.appengine 0.9.0
google.utils.label 0.0.2
===> ANALYZING
Previous image with name "us.gcr.io/xyz/app-engine-tmp/app/default/ttl-18h:69af35b6-6fc6-4167-88f3-da16f9cfc7e7" not found
Restoring metadata for "google.go.gomod:gopath" from cache
===> RESTORING
Restoring data for "google.go.gomod:gopath" from cache
===> BUILDING
=== App Engine Gomod (google.go.appengine_gomod#0.9.0) ===
--------------------------------------------------------------------------------
Running "cp --dereference -R . /layers/google.go.appengine_gomod/srv"
Done "cp --dereference -R . /layers/google.go.appengine_gomod/srv" (53.220262ms)
=== Go - Gomod (google.go.gomod#0.9.0) ===
DEBUG: go.mod SHA has changed: clearing GOPATH layer's cache
Failure: (ID: f51775d1) removing /layers/google.go.gomod/gopath: unlinkat /layers/google.go.gomod/gopath/pkg/mod/github.com/golang/protobuf#v1.3.1/regenerate.sh: permission denied
--------------------------------------------------------------------------------
Running "mv -f /builder/outputs/output-5577006791947779410 /builder/outputs/output"
Done "mv -f /builder/outputs/output-5577006791947779410 /builder/o..." (5.980458ms)
ERROR: failed to build: exit status 1
I'm completely mystified by this. AFAICT this is something AppEngine does internally so I don't think I have any way to debug this further? FWIW, my app is super simple ATM:
module xyz
go 1.17
require (
github.com/go-chi/chi/v5 v5.0.7
go.etcd.io/bbolt v1.3.6
google.golang.org/appengine/v2 v2.0.1
)
require (
github.com/golang/protobuf v1.3.1 // indirect
golang.org/x/sys v0.0.0-20200923182605-d9f96fdee20d // indirect
)
And:
runtime: go116
main: ./cmd/server
The solution was to go to container registry (maybe artifact registry soon) in the cloud console and delete all the containers there. The issue was quite likely just one of those images (maybe the build cache one) but my app is not released so I could just delete all of them.
I resolve it using --no-cache option, but its not perfect because cache is not enabled.
$ gcloud app deploy app.yaml --no-cache
I have a Java 11 GCP App Engine project and I'm trying to use different app.yaml files depending on the environment (e.g., app-dev.yaml, app-prod.yaml, etc). The yaml files in the /appengine directory like src/main/appengine/app-dev.yaml and so on.
There is an SO post about this already, but the answer doesn't work because it clobbers the descriptor which in Java should be the pom.xml (see my Approach 2 for more information).
Approach #1
UPDATE: Solved! In order to use this approach you must be on gcloud 298.0.0+
First, I tried using the --appyaml=APPYAML argument found in https://cloud.google.com/sdk/gcloud/reference/app/deploy#--appyaml:
gcloud app deploy [DEPLOYABLES …] [--appyaml=APPYAML] [--bucket=BUCKET] ...
I ran the following and received an error that the appyaml argument isn't recognized.
$ gcloud --project=my-project app deploy --appyaml=app-dev.yaml
ERROR: (gcloud.app.deploy) unrecognized arguments: --appyaml=app-dev.yaml
The fully qualified path to app-dev.yaml doesn't work either.
Approach #2
Next I found a slighly different syntax in https://cloud.google.com/appengine/docs/flexible/java/configuring-your-app-with-app-yaml that looks like this:
gcloud app deploy service-name-app.yaml
I tried the same locally but pointed to my custom app-dev.yaml like so, but it breaks:
$ gcloud --project=my-project app deploy src/main/appengine/app-dev.yaml
...
descriptor: [/Users/SomeDev/IdeaProjects/my-project/app-server/src/main/appengine/app-dev.yaml]
source: [/Users/SomeDev/IdeaProjects/my-project/app-server/src/main/appengine]
target project: [my-project]
target service: [default]
target version: [20200831abcdefg]
target url: [https://my-project.uc.r.appspot.com]
This breaks because it thinks the app-dev.yaml is the descriptor file instead of a pom.xml, so it errors out with the following:
Error message: did not find any jar files with a Main-Class manifest entry
To compare, I ran a normal deployment without a custom yaml file and you can see the pom.xml is the value of the descriptor.
$ gcloud --project=my-project app deploy
...
descriptor: [/Users/SomeDev/IdeaProjects/my-project/app-server/pom.xml]
source: [/Users/SomeDev/IdeaProjects/my-project/app-server]
target project: [my-project]
target service: [default]
target version: [20200831abcdefg]
target url: [https://my-project.uc.r.appspot.com]
Is there a recommended way to make this work, or is this the wrong approach entirely?
Looking at your "Approach #1" you have to upgrade your gcloud to version >= 298.0.0 here --appyaml parameter have been added - quite recently in Jun'20.
Looking at your "Approach #2". If you are running gcloud app deploy (without parameters) it search for descriptor app.yaml in current directory and if not found - than for pom.xml. If you want to use pom.xml from different localization you have to remove it from current directory. I didn't test it to the very end, just tested the descriptor value in summary.
Anyway I don't think using above is best way to do it. When you use pom.xml as descriptor it means that you are using feature called "deploy your Maven project as source code". Which is not main way to deploy app engine with maven.
According to my understanding if maven was used for build, its possible to use the jar in entrypoint of app.yaml file (reference) or maven goal appengine:deploy (reference + article that should be interesting).
I'm trying to deploy an application written in go to GAE. When I call gcloud app deploy it says this:
Updating service [default]...failed.
ERROR: (gcloud.app.deploy) Error Response: [9] Cloud build 64850a38-3b7b-4223-97bc-27643b8ab1d5 status: FAILURE.
Build error details: go build: cannot use -o with multiple packages
.
Check the build log for errors: https://console.cloud.google.com/gcr/builds/64850a38-3b7b-4223-97bc-27643b8ab1d5?project=609165667740
I have looked everywhere but I can't find information related to this error in this context.
I am trying to get a simple HelloWorld! deployment setup to AppEngine. I'm working with another friend/developer, who on his machine was able to get this simple gcloud app deploy to work successfully, but on my machine it is not working (classic works on my machine). I'm wondering if it's a permission issue with our buckets, but everything between me and my friend's AppEngine accounts are mirrored. I'm at my wit's and so any help is appreciated.
What I will describe to follow seems to be a rare find on the internet so I'm hopeful that someone knowledgeable can help.
I have the GoogleCloudSDK installed and authenticated.
I have a simple directory named "default" with an app.yaml and an index.html
When running the gcloud app deploy default command I receive the following error:
Beginning deployment of service [default]...
╔═════════════════════════════════════════════╗
╠═ Uploading 2 files to Google Cloud Storage ═╣
╚═════════════════════════════════════════════╝
File upload done.
Updating service [default]...failed.
ERROR: (gcloud.app.deploy) Error Response: [3] The following errors occurred while copying files to App Engine:
File https://storage.googleapis.com/staging.xxxxxxx.appspot.com/59add2715212c2a5fa68f05ead6ea6a307d279b5 failed with: Conflicting SHA1 sum for file. Expected "90d147b8_dd2137ab_8ec7f19f_c0891006_6585d3de" but received "59add271_5212c2a5_fa68f05e_ad6ea6a3_07d279b5".
Details: [
[
{
"#type": "type.googleapis.com/google.rpc.ResourceInfo",
"description": "Conflicting SHA1 sum for file. Expected \"90d147b8_dd2137ab_8ec7f19f_c0891006_6585d3de\" but received \"59add271_5212c2a5_fa68f05e_ad6ea6a3_07d279b5\".",
"resourceName": "https://storage.googleapis.com/staging.xxxxxxx.appspot.com/59add2715212c2a5fa68f05ead6ea6a307d279b5",
"resourceType": "file"
}
]
]
What I have tried
Deleting all of the files within the bucket.
Deleting the entire bucket at staging.xxxxxxx.appspot.com.
Uninstalling and reinstalling the SDK from https://cloud.google.com/sdk/. I'm on Windows 10 64bit.
Re-running gcloud init and going through those instructions.
Re-running gcloud auth application-default login and authenticating with my Google account.
Here are my example files
app.yaml
runtime: nodejs10
handlers:
- url: /
static_files: index.html
upload: index.html
index.html
<html>
<head>
<title>Test Title</title>
</head>
<body>
<p>
Hello from the static index.html file.
</p>
</body>
</html>
Summary of the issue:
Different size of the text files on the Windows local machine file system and in the Google Cloud Storage - failed SHA checksum compare.
Possible solutions:
Fix files locally before deploying app using Dos2Unix using Cigwin,
or deploy from Linux:
Upload deployment files to Cloud Shell and deploy from there.
Upload deployment files to Linux VM Instance.
For summary of Windows and Unix interoperability, check documentation
Use the old method for deploy - appcfg.py:
https://cloud.google.com/appengine/docs/standard/python/tools/appcfg-arguments
Works for me.
Or you can see this answer:
https://stackoverflow.com/a/52894113/8244338
I'm new on Google App Engine. And, I'm getting an issue that I can't solve.
I've a very simple app (developped in Go) like this :
main/
| model/
| | user.go
| main.go
| app.yaml
These are the imports of main.go :
import (
"github.com/julienschmidt/httprouter"
"log"
"net/http"
)
My code works well when I run it locally.
But, when I try to publish it on my Google App Engine instance, I receive this error :
$ gcloud app deploy
You are about to deploy the following services:
- <MY_APP_ENGINE_URL> (from [<MY_LOCAL_YAML_PATH>])
Deploying to URL: [<MY_APP_ENGINE_URL>]
Do you want to continue (Y/n)? Y
Beginning deployment of service [default]...
Some files were skipped. Pass `--verbosity=info` to see which ones.
You may also view the gcloud log file, found at
[<LOCAL_APP_ENGINE_LOG>].
File upload done.
Updating service [default]...failed.
ERROR: (gcloud.app.deploy) Error Response: [9] Deployment contains files that cannot be compiled: Compile failed:
2017/05/27 14:48:24 go-app-builder: build timing: 5×compile (301ms total), 0×link (0s total)
2017/05/27 14:48:24 go-app-builder: failed running compile: exit status 2
main.go:4: can't find import: "github.com/julienschmidt/httprouter"
What did I do wrong ?
EDIT :
This is the content of my app.yaml file :
runtime: go
api_version: go1
handlers:
- url: /.*
script: _go_app
App Engine environment doesn't contain your dependencies, you can add an script to do a go get ... for each one but it's too hacky and Go has a solution for that, we can save our dependencies in a vendor folder on the root of our project.
Quick solution:
# Instal godep:
go get -v -u github.com/tools/godep
cd your/project/path
godep save
Now try to deploy again, you'll see a vendor folder in your project, don't remove it and add it to your git source, that folder contains all your third party dependencies like your httprouter (it's my favorite :) )
Note You can use other tools to save your dependencies
I haven't used the gcloud tool, but back in the day when goapp was the tool you had to create github.com/julienschmidt/httprouter (with the lib's source in it, of course) directly under you'r main and then deploy.
AFAIK the App Engine's go version is currently 1.6 which means that while the vendoring is on by default, it can be switched off - perhaps thats the case and thats why #Yandry Pozo's suggestion doesn't work.