I'm very confused about deployment on app engine for my java application. After a couple of try by using gcloud app deploy --stop-previous-version and receive an error :
ERROR: (gcloud.app.deploy) INVALID_ARGUMENT: The following quotas were exceeded: IN_USE_ADDRESSES (quota: 8, used: 8 + needed: 2).
I decided to disable my project in the settings. But when I run :
gcloud app instances list
I'm still getting this output :
<code>
SERVICE VERSION ID VM_STATUS VM_LIVENESS DEBUG_MODE
default 20210916t134322 aef-default-20210916t134322-0v2b RUNNING TIMEOUT
default 20210916t134322 aef-default-20210916t134322-8k8f RUNNING TIMEOUT
default 20210916t134322 aef-default-20210916t134322-8p5v RUNNING TIMEOUT
default 20210916t134322 aef-default-20210916t134322-ggcs RUNNING TIMEOUT
default 20210916t134322 aef-default-20210916t134322-rjh0 RUNNING TIMEOUT
default 20210916t134322 aef-default-20210916t134322-t42r RUNNING TIMEOUT
default 20210916t134322 aef-default-20210916t134322-z9vw RUNNING TIMEOUT
</code>
If I try to delete an instance from the cli :
gcloud app versions delete 20210916t134322
I got the following exception
ERROR: (gcloud.app.versions.delete) The default service (module) may not be deleted, and must comprise at least one version.
And I'm still not able to deploy anymore my project !
Any hints ?
The reason it does not allow you to delete version: 20210916t134322 is because you need to have at least one version for the default service. You may follow this steps:
Create a new version
Migrate default service traffic to the new version
Delete the old version
As for the quota error, you may try to specify maximum instance and deploy this in app.yaml to avoid such error:
automatic_scaling:
min_num_instances: 1
max_num_instances: 7
Related
I am trying to deploy a image that I have stored on Google at eu.gcr.io. I want to set the enviroment variable NODE_ENV to production.
My command:
gcloud beta run deploy my-app-frontend
--image=eu.gcr.io/my-project/my-app-frontend/my-app-frontend:abcdefghijklmnopqrstuvwxyz123 --execution-environment=gen2 --region=europe-north1 --project=my-project && gcloud run services update-traffic my-app-frontend --to-latest --set-env-vars=[NODE_ENV=production]
Error:
OK Deploying... Done.
OK Creating Revision...
OK Routing traffic...
Done.
Service [my-app-frontend] revision [my-app-frontend-00024-nub] has been deployed and is serving 100 percent of traffic.
Service URL: https://my-app-frontend-abcdefghij-ab.a.run.app
ERROR: (gcloud.run.services.update-traffic) unrecognized arguments:
--set-env-vars (did you mean '--set-tags'?)
NODE_ENV=production
To search the help text of gcloud commands, run:
gcloud help -- SEARCH_TERMS
I have read the manual and set-env-vars is a part of it: https://cloud.google.com/sdk/gcloud/reference/run/deploy#--set-env-vars
What am I missing?
You've combined 2 commands - gcloud beta run deploy, gcloud run services update-traffic
--set-env-var is only applicable to the first command.
You can try this (I moved the flag to be part of the first command)
gcloud beta run deploy my-app-frontend --image=eu.gcr.io/my-project/my-app-frontend/my-app-frontend:abcdefghijklmnopqrstuvwxyz123 --execution-environment=gen2 --region=europe-north1 --set-env-vars=[NODE_ENV=production] --project=my-project && gcloud run services update-traffic my-app-frontend --to-latest
or you can run the 2 commands one after the other (I prefer to always include the -project flag when running commands to ensure I run it against my intended target)
a. First Command
gcloud beta run deploy my-app-frontend --image=eu.gcr.io/my-project/my-app-frontend/my-app-frontend:abcdefghijklmnopqrstuvwxyz123 --execution-environment=gen2 --region=europe-north1 --set-env-vars=[NODE_ENV=production] --project=my-project
b. Second command
gcloud run services update-traffic my-app-frontend --to-latest --project=my-project
I am deploying a node.js server to Google App Engine from Bitbucket pipeline environment and the last command in the script is: gcloud -q app deploy app.yaml --no-promote --verbosity=debug
The logs show that the service is deployed successfully but the script is not terminating, this is the last part of the log:
> DEBUG: Reading GCS logfile: 206 (read 10 bytes) PUSH DONE DEBUG:
> Operation [...] complete. Result: {...} DEBUG: Reading GCS logfile:
> 416 (no new content; keep polling)
> -------------------------------------------------------------------------------- DEBUG: Converted YAML to JSON: "{...}" DEBUG: Operation [...] not
> complete. Waiting to retry. Updating service [default] (this may take
> several minutes)... .DEBUG: Operation [...] not complete. Waiting to
> retry. ......DEBUG: Operation [...] not complete. Waiting to retry.
> .......DEBUG: Operation [...] not complete. Waiting to retry.
> ......DEBUG: Operation [...] not complete. Waiting to retry.
> .......DEBUG: Operation [...] not complete. Waiting to retry.
> .......DEBUG: Operation [...] not complete. Waiting to retry.
I tried to add readiness_check and liveness_check to app.yml but it didn't change the behaviour.
readiness_check:
path: "/api/public/logout"
check_interval_sec: 5
timeout_sec: 4
failure_threshold: 2
success_threshold: 2
app_start_timeout_sec: 300
liveness_check:
path: "/api/public/logout"
check_interval_sec: 30
timeout_sec: 4
failure_threshold: 2
success_threshold: 2
The main unknown here is what criteria does gcloud app deploy uses to determine termination condition?
Also, is there any bypass to this problem?
Update
The problem happens also when running the gcloud app deploy command from local environment (my laptop).
The problem does NOT happen when removing the --no-promote flag.
The gcloud app deploy command expects a well-formed and valid app.yml file, this is what determines its termination condition.
As you confirmed the deployment worked without the --no-promote flag, it could mean that something in the configuration expects the application to be already deployed and running, thus preventing the script to complete.
Another possible cause would be that the Google Cloud SDK version specified in bitbucket-pipelines.yml is an older one. Make sure you work with the latest. This consideration applies extensively to all dependencies in package.json, which might be conflicting with one another, especially when using older versions of Node.js.
This guide can help at building a sound configuration for Bitbucket-based deployments; although the example given is with Python, it might as well be used as a template for processing a Node.js pipeline.
Nb. in this solution, the Google Cloud SDK version is an older one (127.0.0), which will make this deployment fail, so it should be replaced with the latest (228.0.0 or higher). Also the guide omits another required API activation: Cloud Build API. I've notified the team to amend the solution.
I've tested several scenarios with a simple Node.js server, and could not reproduce the issue. Check my Github repository for the code.
For further help on this topic, please provide more hints, such as the content of the app.yml, bitbucket-pipelines.yml, and package.json files, as well as a description of the state of App Engine (services, versions).
In order to deploy the test repository to App Engine from Bitbucket, make sure the following is done on the project:
Enable API's:
App Engine Admin
Cloud Build
Create a Service Account with following permissions, and generate an API Key:
App Engine: Admin
Cloud Build: Editor
Storage: Object Admin
Writing this issue as I have no idea how to investigate it.
We're having problems in deploying an app engine flexible application.
The problem is, that the only error we get is the following:
GCLOUD: ERROR: (gcloud.app.deploy) Error Response: [4] Timed out waiting for the app infrastructure to become healthy.
I tried already the following:
Try a simple helloWorld app, to make sure it's not an application issue
Check quota settings -> All green
Check activity stream for warnings or errors
Check logs for warngings or errors
Grant owner role to service account which is deploying the app
App.yaml:
service: test-service # Id of the service
env: flex # Flex environment
runtime: java # Java runtime
runtime_config:
jdk: openjdk8 # use OpenJDK 8
resources:
cpu: 1
memory_gb: 2.8
gcloud version
Google Cloud SDK 214.0.0 alpha 2018.08.24
app-engine-java 1.9.64
app-engine-python 1.9.74 beta 2018.08.24 bq 2.0.34
cloud-datastore-emulator 2.0.2
core 2018.08.24
gsutil 4.33
kubectl 2018.08.24
pubsub-emulator 2018.08.24
After contacting the google technical support, we found out, that the default app engine service account didn't have the Editor role. After assigning the editor role the deployment worked again.
This error is often reported when your application has reached the quota limit for "In-use IP addresses". Similar error was reported on this Google Cloud Platform issue link. The default value for the in-use addresses is '8', and this quota value can be increased clicking the 'Edit' button in the Cloud Console — Ensure you are editing the value for In-use IP addresses.
The Google engineer confirmed that there is a planned improvement to the quota error details to be implemented in one of the next versions of gcloud SDK. You can track updates on the CloudSDK within this Google Group link
I am getting the following error while deploying the google app engine
ERROR: gcloud crashed (SSLHandshakeError): [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)
If you would like to report this issue, please run the following command:
gcloud feedback
To check gcloud for common problems, please run the following command:
gcloud info --run-diagnostics
I am using python 2.7 also tried turning off firewall settings but doesnot help. Any suggestions?
This is a common network issue seen when there is a networking proxy present on your network or antivirus and similar software that might prevent the connection.
As you mentioned the issue was solved when deactivating an antivirus software. If you still want to run the antivirus, you can configure it properly to allow the connection to GCP.
I tried this to avoid SSL certificate validation and successfully worked
gcloud config set auth/disable_ssl_validation True
we updated our google app engine health checks from the legacy version to the new version using and now our deployments are failing. Nothing else on the project has changed. We tested the default settings and then extended checks just in case.
This is the error:
ERROR: (gcloud.app.deploy) Error Response: [4] Your deployment has failed to become healthy in the allotted time and therefore was rolled back. If you believe this was an error, try adjusting the 'app_start_timeout_sec' setting in the 'readiness_check' section.
This is our app.yaml:
liveness_check:
check_interval_sec: 120
timeout_sec: 40
failure_threshold: 5
success_threshold: 5
initial_delay_sec: 500
readiness_check:
check_interval_sec: 120
timeout_sec: 40
failure_threshold: 5
success_threshold: 5
app_start_timeout_sec: 1500
Unfortunately, no matter the configuration, both the readiness and liveness checks are throwing 404s.
What could be causing the problem? and how can we debug this?
Is it possible to rollback to the legacy health checks?
This is usually caused when the application is still reading from the legacy health check flags and/or deploying the app using gcloud app deploy without enabling the updated health checks first. You can check this by:
1- Making sure the legacy health_check flag does not exist on your app.yaml.
2- Run gcloud beta app describe to see whether splitHealthChecks flag is set to true under featureSettings.
By default, HTTP requests from updated health checks are not forwarded to your application container. If you want to extend health checks to your application, then specify a path for liveness checks or readiness checks.
You can then enable updated health checks by using gcloud beta app update --split-health-checks --project [your-project-id]. See this public issue tracker or this article about Updated Health Checks about for more details.
In my case, I solved this issue by manually increasing memory allocation?
resources:
cpu: 1
memory_gb: 2
disk_size_gb: 10
Found this solution in a google forum:
https://groups.google.com/forum/#!topic/google-appengine/Po_-SkC5DOE
For those of you who want to migrate to the default settings for splitted health checks, follow these steps:
1) Remove health_check, liveness_check and readiness_check sections from your app.yaml file
2) Deploy to a newer version, This is important. So, for example, if your current version is production, change it to something else like prod in the command gcloud app deploy --version [new-version-name]