There are many answers to this question already, but they no longer work here in January 2021. All those answers come in 3 flavors:
Set local machine timeout with something like gcloud config set app/cloud_build_timeout 1600 then deploy with gcloud app deploy ...
Use a cloudbuild.yaml file instead with timeout: 1600s bits in the gcloud app deploy buildstep and the global configuration, then deploy with gcloud builds submit ...
Per google's own docs, don't set timeout: 1600s in the cloudbuild file, but rather do a mashup of the previous 2 flavors with a build step including args: ['-c', 'gcloud config set app/cloud_build_timeout 1600 && gcloud app deploy']
None of them have any impact on the app deploy build - it's stuck at 10mins. When using gcloud builds submit, it results in 2 Cloud Builds being kicked off: one for the cloudbuild.yaml, and one for the app engine deployment using buildpack. The above solutions can impact the first build, but once that first build kicks off the second build (gcloud app deploy, you can see at https://console.cloud.google.com/cloud-build in the Execution Details tab that the Timeout is still 10m.
IMO, solutions 2-3 are hacks since 1 doesn't work, but now that 2-3 don't work either, I'm looking for another hack. Does anyone have a solution which works in 2021? Since my app is using GAE Standard Environment, I can't prebuild an image - I'm stuck with Buildpack building my ruby app and pulling all the Gems every time, and this runs out the seemingly immutable 10m clock.
You cannot change the timeout property in App Engine standard and it's always 10min. The workaround is to use App Engine flex and this way you can use the gcloud config set app/cloud_build_timeout TIME_SECONDS.
There is a feature request to enable timeout edit for App Engine standard but seems still in progress.
Related
My gcloud app deploy fails due to timeout during build. After looking at logs, its taking more than 10 min and the App Engine Standard timeout can't be changed. But the previous all my builds went fine and the only change I noticed between successful and failed build logs are:
failed build is using and fetching the source and images from gae_runtime
Pulling image: asia.gcr.io/gae-runtimes/utilities/buildpack-shim:base_20211020a_18_04_RC00
successful build is fetching the image from
Pulling image: asia.gcr.io/fn-img/utilities/buildpack-shim:base_20211020a_18_04_RC00.
How to make the build to pull these image from fn-img and not gae-runtimes?
It turns out that you can only configure timeouts for App Engine builds if you are using the Flexible environment. You can configure the timeout by using Cloud Build to deploy your App Engine Standard service. As such you could avoid the 10 minutes timeout limit.
In this documentation page there is a step by step guide on how to do it but basically the key step is this one:
In the args field, invoke the gcloud app deploy command and set a timeout for App Engine to use when it invokes Cloud Build. This is required because Cloud Build build steps and builds have a default timeout of 10 minutes and App Engine deployments could take longer than that to complete. Specifying a longer timeout will make sure that the build doesn't timeout if gcloud app deploy takes longer than 10 minutes to complete.
steps:
- name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
entrypoint: 'bash'
args: ['-c', 'gcloud config set app/cloud_build_timeout 1600 && gcloud app deploy']
timeout: '1600s'
On the other hand, the App Engine standard environment does not allow the build timeout to be configured. If you're using Cloud Build for deploying on the App Engine standard environment, and your build is failing with a timeout error, consider using the App Engine flexible environment or Cloud Run instead of the App Engine standard environment.
We are deploying our Go app to Google app engine with the command: gcloud app deploy
This creates a cloud build, which normally works fine. However, sometimes there are some dependencies which are flaky (sometimes unavailable or down) or gone (revision no longer existing).
These issues break our complete deploy and build and we can only get rid of the dependency or wait until it is fixed on that side.
Solution for this is using a proxy for the go dependencies (like https://proxy.golang.org/), but I don't have any way of setting this setting for our app engine deployments.
Does anyone know how I can use a proxy for google app engine deploys to make our builds more stable?
Edit: Google Cloud at its finest. Made absolutely no changes but deployed fine this morning.
Error after ~10m of deployment (10+ deploy attempts).
I've changed the app to add a simply function with less than 10 lines of code. It has no affect on libraries and uses native python only and hence I'm not sure how it can affect the deployment.
I've tested the app and it runs fine locally.
Here is the error that I get:
ERROR: (gcloud.app.deploy) Cloud build failed. Check logs at
https://console.cloud.google.com/xxxxxxxx/ Failure status: UNKNOWN:
Error Response: [4] DEADLINE_EXCEEDED
Previously to that, logs don't show any errors, in fact everything is going as expected.
Newest entries first:
9302e2430a0e: Pushed
4f56eb74b6bf: Pushed
21df82f90a72: Layer already exists
f0e2b3558b28: Layer already exists
99c71ba2c817: Layer already exists
2483da9621d1: Layer already exists
af09d2110abe: Layer already exists
d968669f4b42: Pushed
4b1e707066a6: Layer already exists
55530b72c8c8: Layer already exists
62c169a7d462: Layer already exists
According to this you can change the build timeout setting with:
gcloud config set app/cloud_build_timeout 1000
Setting it to 1000 seconds in this example.
I have this problem too when deploying my custom image to AppEngine flex. My image requires a few ML models and takes around 2 hours to build and GAE does not seem to like it.
To build your container images outside of Google Cloud Platform, you must first upload your images to a container image repository before you can deploy your images to App Engine with the gcloud app deploy command.
In short, push your image to Google Container Registry or container image repository of your choice, then deploy with this command
gcloud app deploy --image-url gcr.io/YOUR_PROJECT_ID/YOUR_CONTAINER_IMAGE
More info here and here
For my case, I've found out the problem was that I have the same version for all gcloud app deploy command.
When there's traffic, the deployment will timeout.
I'm deploying an app in Google Cloud that has multiple microservices. For one of them, I wish to use a push queue. I configured my queue.yaml file as follows:
queue:
- name: equeue
rate: 10/s
retry_parameters:
task_retry_limit: 1
task_age_limit: 2d
I spin up the SDK and the queue is working fine, just as it should. But then I go to push it to my app on Google Cloud using:
gcloud app deploy --project=<projectname> --promote --stop-previous-version
The code all loads correctly, but the queue never gets created. I am getting errors from my code as follows:
API error 1 (taskqueue: UNKNOWN_QUEUE)
And when I look in the console, on https://console.cloud.google.com/appengine/taskqueues, the queue has not been created.
I've read the documentation and it seems to indicate that all I need to do is create the queue.yaml file and deploy the app (and that has worked for me in the past, albeit with an older version of the SDK.)
In case it helps, here are the components I'm running locally:
Google Cloud SDK 164.0.0
app-engine-go
app-engine-python 1.9.57
bq 2.0.24
core 2017.07.25
gcloud
gsutil 4.27
Thanks!
Dan
The queue.yaml is an app-level configuration file, not a service-specific one. See (marginally related) Why do I need to deploy a "default" app before I can deploy multiple services in GCP?
While in a single service app case deploying the default service might also update one or more of these app-level configs that is not guaranteed in a multi-service app case. Which is why each of these configs also comes with an individual dedicated method of deployment which can be performed independently of and/or without updating any of the app's services.
For queue.yaml in particular deployment can be done using gcloud app deploy:
gcloud app deploy [...] queues.yaml
or, if you're using the GAE SDK, using appcfg.py update_queues:
appcfg.py update_queues -A <app_id>
I used to deploy AppEngine apps with appcfg.py which, as far as I remember, was automatically deploying the index.yaml file as part of the app deployment.
Now that I'm using gcloud app deploy I've found out the hard way that I also have to run gcloud app deploy index.yaml to get the indices created.
Why was gcloud designed that way?
After indexes are uploaded, it can take a few minutes for them to actually build. While they build, any app that attempts to use those indexes will fail. The behavior in appcfg.py of uploading both at the same time can actually result in apps failing for a few minutes while the indexes build.
The behavior in gcloud app is more explicit to avoid these issues. If you are adding new indexes and new code that depends on it, you should use this procedure:
gcloud app deploy index.yaml
# wait for indexes to build
gcloud app deploy app.yaml