I am trying to push a Nodejs sample app Hello World but after pushing I am getting error.
ERROR: (gcloud.app.deploy) Error Response: [13] An internal error occurred while creating a Google Cloud Storage bucket.
There is nothing wrong with the code as it is just a sample app downloaded from google.
I have seen other SO posts related to this error but none of them helped.
Can anyone please tell me a Fix for this error.
I also tried to run command in debug using command and got following output-
gcloud app deploy app.yaml --verbosity debug
DEBUG: No bucket specified, retrieving default bucket.
DEBUG: Using bucket [gs://staging.united-backbone-186810.appspot.com].
DEBUG: Service [appengineflex.googleapis.com] is already enabled for project [united-backbone-186810]
Beginning deployment of service [default]...
INFO: Need Dockerfile to be generated for runtime nodejs
Building and pushing image for service [default]
INFO: Uploading [c:\users\sudha\appdata\local\temp\tmpaj1p9z\src.tgz] to [asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest]
DEBUG: Using runtime builder root [gs://runtime-builders/]
DEBUG: Loading runtimes manifest from [gs://runtime-builders/runtimes.yaml]
INFO: Reading [<googlecloudsdk.api_lib.storage.storage_util.ObjectReference object at 0x0000000006631400>]
DEBUG: Resolved runtime [nodejs] as build configuration [gs://runtime-builders/nodejs-default-builder-20171116155610.yaml]
INFO: Using runtime builder [gs://runtime-builders/nodejs-default-builder-20171116155610.yaml]
INFO: Reading [<googlecloudsdk.api_lib.storage.storage_util.ObjectReference object at 0x0000000006642390>]
Started cloud build [c3702b4b-7bc8-4861-944f-8490bf183078].
DEBUG: GCS logfile url is https://www.googleapis.com/storage/v1/b/staging.united-backbone-186810.appspot.com/o/log-c3702b4b-7bc8-4861-944f-8490bf183078.txt?alt=media
To see logs in the Cloud Console: https://console.cloud.google.com/gcr/builds/c3702b4b-7bc8-4861-944f-8490bf183078?project=united-backbone-186810
DEBUG: Operation [operations/build/united-backbone-186810/YzM3MDJiNGItN2JjOC00ODYxLTk0NGYtODQ5MGJmMTgzMDc4] not complete. Waiting 1s.
DEBUG: Reading GCS logfile: 416 (no new content; keep polling)
DEBUG: Operation [operations/build/united-backbone-186810/YzM3MDJiNGItN2JjOC00ODYxLTk0NGYtODQ5MGJmMTgzMDc4] not complete. Waiting 1s.
DEBUG: Reading GCS logfile: 206 (read 233 bytes)
------------------------------------------------- REMOTE BUILD OUTPUT --------------------------------------------------
starting build "c3702b4b-7bc8-4861-944f-8490bf183078"
FETCHSOURCE
Fetching storage object: gs://staging.united-backbone-186810.appspot.com/asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest#1511349522754483
DEBUG: Operation [operations/build/united-backbone-186810/YzM3MDJiNGItN2JjOC00ODYxLTk0NGYtODQ5MGJmMTgzMDc4] not complete. Waiting 1s.
DEBUG: Reading GCS logfile: 416 (no new content; keep polling)
DEBUG: Operation [operations/build/united-backbone-186810/YzM3MDJiNGItN2JjOC00ODYxLTk0NGYtODQ5MGJmMTgzMDc4] not complete. Waiting 1s.
DEBUG: Reading GCS logfile: 206 (read 398 bytes)
Copying gs://staging.united-backbone-186810.appspot.com/asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest#1511349522754483...
- [1 files][ 1.8 KiB/ 1.8 KiB]
DEBUG: Operation [operations/build/united-backbone-186810/YzM3MDJiNGItN2JjOC00ODYxLTk0NGYtODQ5MGJmMTgzMDc4] not complete. Waiting 1s.
DEBUG: Reading GCS logfile: 206 (read 2198 bytes)
Operation completed over 1 objects/1.8 KiB.
BUILD
Step #0: Pulling image: gcr.io/gcp-runtimes/nodejs/gen-dockerfile#sha256:196bc20ff8d91905dc071100399538814e2c619d0d27576c35a6405674da696c
Step #0: sha256:196bc20ff8d91905dc071100399538814e2c619d0d27576c35a6405674da696c: Pulling from gcp-runtimes/nodejs/gen-dockerfile
Step #0: Digest: sha256:196bc20ff8d91905dc071100399538814e2c619d0d27576c35a6405674da696c
Step #0: Status: Downloaded newer image for gcr.io/gcp-runtimes/nodejs/gen-dockerfile#sha256:196bc20ff8d91905dc071100399538814e2c619d0d27576c35a6405674da696c
Starting Step #0
Step #0: Checking for Node.js.
Finished Step #0
Step #1: Pulling image: gcr.io/cloud_builders/docker#sha256:8f8f572201e2b2ae876d8ca8b05c7d44df994e7ea8352c334ee5bae7ca3dc7f9
Step #1: sha256:8f8f572201e2b2ae876d8ca8b05c7d44df994e7ea8352c334ee5bae7ca3dc7f9: Pulling from cloud_builders/docker
Step #1: Digest: sha256:8f8f572201e2b2ae876d8ca8b05c7d44df994e7ea8352c334ee5bae7ca3dc7f9
Step #1: Status: Downloaded newer image for gcr.io/cloud_builders/docker#sha256:8f8f572201e2b2ae876d8ca8b05c7d44df994e7ea8352c334ee5bae7ca3dc7f9
Starting Step #1
Step #1: Sending build context to Docker daemon 11.26kB
Step #1: Step 1/5 : FROM gcr.io/google-appengine/nodejs#sha256:2c743f7509798cca81aaebaa339c899c4d1924153beb4a94df00ff6af238fcb2
Step #1: sha256:2c743f7509798cca81aaebaa339c899c4d1924153beb4a94df00ff6af238fcb2: Pulling from google-appengine/nodejs
Step #1: Digest: sha256:2c743f7509798cca81aaebaa339c899c4d1924153beb4a94df00ff6af238fcb2
Step #1: Status: Downloaded newer image for gcr.io/google-appengine/nodejs#sha256:2c743f7509798cca81aaebaa339c899c4d1924153beb4a94df00ff6af238fcb2
Step #1: ---> 669f53c480d3
Step #1: Step 2/5 : COPY . /app/
Step #1: ---> 388693438f26
Step #1: Removing intermediate container 45b4eecdfef0
Step #1: Step 3/5 : RUN /usr/local/bin/install_node '>=4.3.2'
Step #1: ---> Running in 67ea703659bc
Step #1: ---> 7b5b6d0283c5
Step #1: Removing intermediate container 67ea703659bc
Step #1: Step 4/5 : RUN npm install --unsafe-perm || ((if [ -f npm-debug.log ]; then cat npm-debug.log; fi) && false)
Step #1: ---> Running in 2d7b75885d6c
DEBUG: Operation [operations/build/united-backbone-186810/YzM3MDJiNGItN2JjOC00ODYxLTk0NGYtODQ5MGJmMTgzMDc4] not complete. Waiting 1s.
DEBUG: Reading GCS logfile: 206 (read 145 bytes)
Step #1: [91mnpm[0m[91m notice created a lockfile as package-lock.json. You should commit this file.
Step #1: [0madded 43 packages in 1.695s
DEBUG: Operation [operations/build/united-backbone-186810/YzM3MDJiNGItN2JjOC00ODYxLTk0NGYtODQ5MGJmMTgzMDc4] not complete. Waiting 1s.
DEBUG: Reading GCS logfile: 206 (read 1222 bytes)
Step #1: ---> e802843c73cd
Step #1: Removing intermediate container 2d7b75885d6c
Step #1: Step 5/5 : CMD npm start
Step #1: ---> Running in cef15318568d
Step #1: ---> b61594a5ec64
Step #1: Removing intermediate container cef15318568d
Step #1: Successfully built b61594a5ec64
Step #1: Successfully tagged asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest
Finished Step #1
PUSH
Pushing asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest
The push refers to a repository [asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838]
737512f7e42b: Preparing
ec3a6c686d39: Preparing
644f071ca81b: Preparing
226846715c53: Preparing
e0a3cc0c6e70: Preparing
c2003e396592: Preparing
8fc48a7a910e: Preparing
9aa804bf0e6a: Preparing
749e521e9c3d: Preparing
31cb62ec9f95: Preparing
c2003e396592: Waiting
8fc48a7a910e: Waiting
9aa804bf0e6a: Waiting
749e521e9c3d: Waiting
31cb62ec9f95: Waiting
226846715c53: Layer already exists
e0a3cc0c6e70: Layer already exists
644f071ca81b: Layer already exists
8fc48a7a910e: Layer already exists
c2003e396592: Layer already exists
31cb62ec9f95: Layer already exists
749e521e9c3d: Layer already exists
9aa804bf0e6a: Layer already exists
DEBUG: Operation [operations/build/united-backbone-186810/YzM3MDJiNGItN2JjOC00ODYxLTk0NGYtODQ5MGJmMTgzMDc4] not complete. Waiting 1s.
DEBUG: Reading GCS logfile: 206 (read 21 bytes)
ec3a6c686d39: Pushed
DEBUG: Operation [operations/build/united-backbone-186810/YzM3MDJiNGItN2JjOC00ODYxLTk0NGYtODQ5MGJmMTgzMDc4] not complete. Waiting 1s.
DEBUG: Reading GCS logfile: 206 (read 21 bytes)
737512f7e42b: Pushed
DEBUG: Operation [operations/build/united-backbone-186810/YzM3MDJiNGItN2JjOC00ODYxLTk0NGYtODQ5MGJmMTgzMDc4] not complete. Waiting 1s.
DEBUG: Reading GCS logfile: 206 (read 104 bytes)
latest: digest: sha256:6659b586325b131087fcbf872abf618fa1ee45503fff47cad7bb9e92d63bcd12 size: 2413
DONE
DEBUG: Operation [operations/build/united-backbone-186810/YzM3MDJiNGItN2JjOC00ODYxLTk0NGYtODQ5MGJmMTgzMDc4] complete. Result: {
"response": {
"finishTime": "2017-11-22T11:19:11.119096Z",
"status": "SUCCESS",
"timeout": "600s",
"startTime": "2017-11-22T11:18:45.353017219Z",
"logsBucket": "staging.united-backbone-186810.appspot.com",
"results": {
"images": [
{
"name": "asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838",
"digest": "sha256:6659b586325b131087fcbf872abf618fa1ee45503fff47cad7bb9e92d63bcd12"
},
{
"name": "asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest",
"digest": "sha256:6659b586325b131087fcbf872abf618fa1ee45503fff47cad7bb9e92d63bcd12"
}
],
"buildStepImages": [
"",
""
]
},
"createTime": "2017-11-22T11:18:44.769257383Z",
"#type": "type.googleapis.com/google.devtools.cloudbuild.v1.Build",
"source": {
"storageSource": {
"object": "asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest",
"bucket": "staging.united-backbone-186810.appspot.com"
}
},
"options": {
"substitutionOption": "ALLOW_LOOSE"
},
"steps": [
{
"args": [
"--runtime-image",
"gcr.io/google-appengine/nodejs#sha256:2c743f7509798cca81aaebaa339c899c4d1924153beb4a94df00ff6af238fcb2"
],
"name": "gcr.io/gcp-runtimes/nodejs/gen-dockerfile#sha256:196bc20ff8d91905dc071100399538814e2c619d0d27576c35a6405674da696c",
"env": [
"GAE_APPLICATION_YAML_PATH=app.yaml"
]
},
{
"args": [
"build",
"-t",
"asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest",
"."
],
"name": "gcr.io/cloud_builders/docker#sha256:8f8f572201e2b2ae876d8ca8b05c7d44df994e7ea8352c334ee5bae7ca3dc7f9",
"env": [
"GAE_APPLICATION_YAML_PATH=app.yaml"
]
}
],
"sourceProvenance": {
"resolvedStorageSource": {
"generation": "1511349522754483",
"object": "asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest",
"bucket": "staging.united-backbone-186810.appspot.com"
},
"fileHashes": {
"gs://staging.united-backbone-186810.appspot.com/asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest#1511349522754483": {
"fileHash": [
{
"type": "MD5",
"value": "RCwNC0JHUlmvZZz+C5RYsw=="
}
]
}
}
},
"projectId": "united-backbone-186810",
"images": [
"asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest"
],
"substitutions": {
"_GAE_APPLICATION_YAML_PATH": "app.yaml",
"_OUTPUT_IMAGE": "asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest"
},
"id": "c3702b4b-7bc8-4861-944f-8490bf183078",
"logUrl": "https://console.cloud.google.com/gcr/builds/c3702b4b-7bc8-4861-944f-8490bf183078?project=united-backbone-186810"
},
"done": true,
"name": "operations/build/united-backbone-186810/YzM3MDJiNGItN2JjOC00ODYxLTk0NGYtODQ5MGJmMTgzMDc4",
"metadata": {
"#type": "type.googleapis.com/google.devtools.cloudbuild.v1.BuildOperationMetadata",
"build": {
"finishTime": "2017-11-22T11:19:11.119096Z",
"status": "SUCCESS",
"timeout": "600s",
"startTime": "2017-11-22T11:18:45.353017219Z",
"logsBucket": "staging.united-backbone-186810.appspot.com",
"results": {
"images": [
{
"name": "asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838",
"digest": "sha256:6659b586325b131087fcbf872abf618fa1ee45503fff47cad7bb9e92d63bcd12"
},
{
"name": "asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest",
"digest": "sha256:6659b586325b131087fcbf872abf618fa1ee45503fff47cad7bb9e92d63bcd12"
}
],
"buildStepImages": [
"",
""
]
},
"id": "c3702b4b-7bc8-4861-944f-8490bf183078",
"source": {
"storageSource": {
"object": "asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest",
"bucket": "staging.united-backbone-186810.appspot.com"
}
},
"options": {
"substitutionOption": "ALLOW_LOOSE"
},
"steps": [
{
"args": [
"--runtime-image",
"gcr.io/google-appengine/nodejs#sha256:2c743f7509798cca81aaebaa339c899c4d1924153beb4a94df00ff6af238fcb2"
],
"name": "gcr.io/gcp-runtimes/nodejs/gen-dockerfile#sha256:196bc20ff8d91905dc071100399538814e2c619d0d27576c35a6405674da696c",
"env": [
"GAE_APPLICATION_YAML_PATH=app.yaml"
]
},
{
"args": [
"build",
"-t",
"asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest",
"."
],
"name": "gcr.io/cloud_builders/docker#sha256:8f8f572201e2b2ae876d8ca8b05c7d44df994e7ea8352c334ee5bae7ca3dc7f9",
"env": [
"GAE_APPLICATION_YAML_PATH=app.yaml"
]
}
],
"sourceProvenance": {
"resolvedStorageSource": {
"generation": "1511349522754483",
"object": "asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest",
"bucket": "staging.united-backbone-186810.appspot.com"
},
"fileHashes": {
"gs://staging.united-backbone-186810.appspot.com/asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest#1511349522754483": {
"fileHash": [
{
"type": "MD5",
"value": "RCwNC0JHUlmvZZz+C5RYsw=="
}
]
}
}
},
"projectId": "united-backbone-186810",
"images": [
"asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest"
],
"substitutions": {
"_GAE_APPLICATION_YAML_PATH": "app.yaml",
"_OUTPUT_IMAGE": "asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest"
},
"createTime": "2017-11-22T11:18:44.769257383Z",
"logUrl": "https://console.cloud.google.com/gcr/builds/c3702b4b-7bc8-4861-944f-8490bf183078?project=united-backbone-186810"
}
}
}
DEBUG: Reading GCS logfile: 416 (no new content; keep polling)
------------------------------------------------------------------------------------------------------------------------
DEBUG: Converted YAML to JSON: "{
"betaSettings": {
"module_yaml_path": "app.yaml",
"vm_runtime": "nodejs"
},
"env": "flex",
"handlers": [
{
"script": {
"scriptPath": "PLACEHOLDER"
},
"urlRegex": ".*"
}
],
"runtime": "vm"
}"
DEBUG: Received operation: [apps/united-backbone-186810/operations/51892505-4cb2-4b11-8431-ee7b62f0f236]
DEBUG: Operation [apps/united-backbone-186810/operations/51892505-4cb2-4b11-8431-ee7b62f0f236] not complete. Waiting to retry.
Updating service [default] (this may take several minutes).../DEBUG: Operation [apps/united-backbone-186810/operations/51892505-4cb2-4b11-8431-ee7b62f0f236] not complete. Waiting to retry.
Updating service [default] (this may take several minutes)...\DEBUG: Operation [apps/united-backbone-186810/operations/51892505-4cb2-4b11-8431-ee7b62f0f236] complete. Result: {
"metadata": {
"target": "apps/united-backbone-186810/services/default/versions/20171122t164838",
"method": "google.appengine.v1.Versions.CreateVersion",
"user": "prafactor9#gmail.com",
"insertTime": "2017-11-22T11:19:14.120Z",
"ephemeralMessage": "Deployment failed. Attempting to cleanup deployment artifacts.",
"#type": "type.googleapis.com/google.appengine.v1.OperationMetadataV1"
},
"done": true,
"name": "apps/united-backbone-186810/operations/51892505-4cb2-4b11-8431-ee7b62f0f236",
"error": {
"message": "An internal error occurred while creating a Google Cloud Storage bucket.",
"code": 13
}
}
Updating service [default] (this may take several minutes)...failed.
DEBUG: (gcloud.app.deploy) Error Response: [13] An internal error occurred while creating a Google Cloud Storage bucket.
Traceback (most recent call last):
File "C:\Users\sudha\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\calliope\cli.py", line 789, in Execute
resources = calliope_command.Run(cli=self, args=args)
File "C:\Users\sudha\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\calliope\backend.py", line 756, in Run
resources = command_instance.Run(args)
File "C:\Users\sudha\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\surface\app\deploy.py", line 65, in Run
parallel_build=False)
File "C:\Users\sudha\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\command_lib\app\deploy_util.py", line 587, in RunDeploy
flex_image_build_option=flex_image_build_option)
File "C:\Users\sudha\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\command_lib\app\deploy_util.py", line 395, in Deploy
extra_config_settings)
File "C:\Users\sudha\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\api_lib\app\appengine_api_client.py", line 188, in DeployService
message=message)
File "C:\Users\sudha\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\api_lib\app\operations_util.py", line 244, in WaitForOperation
sleep_ms=retry_interval)
File "C:\Users\sudha\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\api_lib\util\waiter.py", line 266, in WaitFor
sleep_ms=sleep_ms)
File "C:\Users\sudha\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\core\util\retry.py", line 222, in RetryOnResult
if not should_retry(result, state):
File "C:\Users\sudha\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\api_lib\util\waiter.py", line 260, in _IsNotDone
return not poller.IsDone(operation)
File "C:\Users\sudha\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\api_lib\app\operations_util.py", line 169, in IsDone
encoding.MessageToPyValue(operation.error)))
OperationError: Error Response: [13] An internal error occurred while creating a Google Cloud Storage bucket.
ERROR: (gcloud.app.deploy) Error Response: [13] An internal error occurred while creating a Google Cloud Storage bucket.
C:\Users\sudha\Desktop\nodejs-docs-samples\appengine\hello-world>
Related
I'm trying to deploy an application on Google App Engine but running into an error that doesn't include much information as to why the error is occurring. Here is the last lines of the log file on a run of: gcloud app deploy --verbosity=debug:
DEBUG: Converted YAML to JSON: "{
"automaticScaling": {
"maxTotalInstances": 2,
"minTotalInstances": 1
},
"betaSettings": {
"module_yaml_path": "app.yaml",
"vm_runtime": "ruby"
},
"entrypoint": {
"shell": "bundle exec rails server Puma -p $PORT"
},
"env": "flex",
"envVariables": {
"RAILS_ENV": "production"
},
"handlers": [
{
"script": {
"scriptPath": "PLACEHOLDER"
},
"urlRegex": ".*"
}
],
"runtime": "vm"
}"
Updating service [default] (this may take several minutes)...⠧DEBUG: Operation [apps/xxxxxxxx/operations/177ebe52-a136-403a-90e8-c71bc6cc94b9] not complete. Waiting to retry.
Updating service [default] (this may take several minutes)...⠧DEBUG: Operation [apps/xxxxxxxx/operations/177ebe52-a136-403a-90e8-c71bc6cc94b9] complete. Result: {
"done": true,
"error": {
"code": 13,
"message": "An internal error occurred."
},
"metadata": {
"#type": "type.googleapis.com/google.appengine.v1.OperationMetadataV1",
"insertTime": "2022-04-25T16:39:42.625Z",
"method": "google.appengine.v1.Versions.CreateVersion",
"target": "apps/xxxxxxxx/services/default/versions/20220425t085403",
"user": "xxxxxxxxxxxx"
},
"name": "apps/xxxxxxxx/operations/177ebe52-a136-403a-90e8-c71bc6cc94b9"
}
Updating service [default] (this may take several minutes)...failed.
DEBUG: (gcloud.app.deploy) Error Response: [13] An internal error occurred.
Traceback (most recent call last):
File "xxxxxxxx/google-cloud-sdk/lib/googlecloudsdk/calliope/cli.py", line 987, in Execute
resources = calliope_command.Run(cli=self, args=args)
File "xxxxxxxx/google-cloud-sdk/lib/googlecloudsdk/calliope/backend.py", line 809, in Run
resources = command_instance.Run(args)
File "xxxxxxxx/google-cloud-sdk/lib/surface/app/deploy.py", line 123, in Run
return deploy_util.RunDeploy(
File "xxxxxxxx/google-cloud-sdk/lib/googlecloudsdk/command_lib/app/deploy_util.py", line 690, in RunDeploy
deployer.Deploy(
File "xxxxxxxx/google-cloud-sdk/lib/googlecloudsdk/command_lib/app/deploy_util.py", line 471, in Deploy
self.api_client.DeployService(new_version.service, new_version.id,
File "xxxxxxxx/google-cloud-sdk/lib/googlecloudsdk/api_lib/app/appengine_api_client.py", line 230, in DeployService
return operations_util.WaitForOperation(
File "xxxxxxxx/google-cloud-sdk/lib/googlecloudsdk/api_lib/app/operations_util.py", line 307, in WaitForOperation
completed_operation = waiter.WaitFor(
File "xxxxxxxx/google-cloud-sdk/lib/googlecloudsdk/api_lib/util/waiter.py", line 261, in WaitFor
operation = PollUntilDone(
File "xxxxxxxx/google-cloud-sdk/lib/googlecloudsdk/api_lib/util/waiter.py", line 322, in PollUntilDone
operation = retryer.RetryOnResult(
File "xxxxxxxx/google-cloud-sdk/lib/googlecloudsdk/core/util/retry.py", line 249, in RetryOnResult
if not should_retry(result, state):
File "xxxxxxxx/google-cloud-sdk/lib/googlecloudsdk/api_lib/util/waiter.py", line 320, in _IsNotDone
return not poller.IsDone(operation)
File "xxxxxxxx/google-cloud-sdk/lib/googlecloudsdk/api_lib/app/operations_util.py", line 182, in IsDone
raise OperationError(requests.ExtractErrorMessage(
googlecloudsdk.api_lib.app.operations_util.OperationError: Error Response: [13] An internal error occurred.
ERROR: (gcloud.app.deploy) Error Response: [13] An internal error occurred.
The app.yaml is very basic and when I run my application locally, it runs fine. Here is the app.yaml:
entrypoint: bundle exec rails server Puma -p $PORT
env: flex
runtime: ruby
env_variables:
RAILS_ENV: "production"
automatic_scaling:
min_num_instances: 1
max_num_instances: 2
I tried looking at Logs Explorer to see if there was any more details as to why the failure is happening but no luck. Any insight on how to debug further would be greatly appreciated!
Update: I didn't notice but think it could be failing due to the application.tailwind.css file not existing in the output below. I'm not sure if that's actually the issue as it doesn't look fatal and doesn't indicate it's an error:
Step #1: When you HTTParty, you must party hard!
Step #1: Removing intermediate container 7bda307caaa7
Step #1: ---> bef7a98a7ec9
Step #1: Step 12/15 : RUN bundle exec rake assets:precompile || true
Step #1: ---> Running in e87dc15a12c0
Step #1: Specified input file /app/app/assets/stylesheets/application.tailwind.css does not exist.
Step #1: I, [2022-04-26T00:10:39.725889 #7] INFO -- : Writing /app/public/assets/application-37665e39018e5008ea3f942889791a8f602d3322c45b85d7e8470a13f7f59910.css
Step #1: I, [2022-04-26T00:10:39.726136 #7] INFO -- : Writing /app/public/assets/application-37665e39018e5008ea3f942889791a8f602d3322c45b85d7e8470a13f7f59910.css.gz
Step #1: Removing intermediate container a87dc15a12c0
Step #1: ---> 423dafb9ba66
Step #1: Step 13/15 : FROM augmented-base
Step #1: ---> 3958643d64af
Step #1: Step 14/15 : COPY --from=app-build /app/ /app/
Step #1: ---> 59e2cd88ca1a
Step #1: Step 15/15 : CMD exec bundle exec rails server Puma -p $PORT
Step #1: ---> Running in 73a5dd7ff3d2
Step #1: Removing intermediate container 73a5dd7ff3d2
Step #1: ---> c4785ef8d392
Step #1: Successfully built c4785ef8d392
Step #1: Successfully tagged us.gcr.io/xxxxxxxxxxxx/default.202220425t163957:latest
Finished Step #1
PUSH
Pushing us.gcr.io/xxxxxxxxxxxxxx/appengine/default.202220425t163957:latest
The push refers to repository [us.gcr.io/xxxxxxxxxxxxxxx/appengine/default.202220425t163957]
3642fcf3d105: Preparing
5ba2d620e9b8: Preparing
e6c3ee2a1dc7: Preparing
f254496b0294: Preparing
165e9843257f: Preparing
471f69049b89: Preparing
84ff92691f90: Preparing
b49bce339f97: Preparing
dcb7197db903: Preparing
84ff92691f90: Waiting
b49bce339f97: Waiting
471f69049b89: Waiting
dcb7197db903: Waiting
f254496b0294: Layer already exists
165e9843257f: Layer already exists
84ff92691f90: Layer already exists
471f69049b89: Layer already exists
b49bce339f97: Layer already exists
dcb7197db903: Layer already exists
e6c3ee2a1dc7: Pushed
5ba2d620e9b8: Pushed
3642fcf3d105: Pushed
latest: digest: sha256:f6ca40a7106d837827093fb5a9ac1e26b9576b9a877a28be90b2242a63510444 size: 2209
DONE
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Updating service [default] (this may take several minutes)...failed.
ERROR: (gcloud.app.deploy) Error Response: [13] An internal error occurred.
Update #2: After adding the missing asset file, it looks like another error is thrown:
Step #1: Browserslist: caniuse-lite is outdated. Please run:
Step #1: npx browserslist#latest --update-db
Step #1: Why you should do it regularly: https://github.com/browserslist/browserslist#browsers-data-updating
Step #1:
Step #1: Done in 979ms.
Step #1: I, [2022-04-30T16:05:52.871250 #8] INFO -- : Writing /app/public/assets/manifest-90aab26ade71e747f745b1227b433fd7d580a54f279ee2be6e1a67bb326bf03a.js
Step #1: I, [2022-04-30T16:05:52.872327 #8] INFO -- : Writing /app/public/assets/manifest-90aab26ade71e747f745b1227b433fd7d580a54f279ee2be6e1a67bb326bf03a.js.gz
Step #1: I, [2022-04-30T16:05:52.873483 #8] INFO -- : Writing /app/public/assets/application-79acb896e19757de4ac1a94d7a9ed4649c1c87e146846d806f93dcdc062c6a4d.css
Step #1: I, [2022-04-30T16:05:52.873736 #8] INFO -- : Writing /app/public/assets/application-79acb896e19757de4ac1a94d7a9ed4649c1c87e146846d806f93dcdc062c6a4d.css.gz
Step #1: I, [2022-04-30T16:05:52.874356 #8] INFO -- : Writing /app/public/assets/application.tailwind-727aaabcff3ab779544847b78f71b65d03963692205b3bba1fc81dbe96d77906.css
Step #1: I, [2022-04-30T16:05:52.874663 #8] INFO -- : Writing /app/public/assets/application.tailwind-727aaabcff3ab779544847b78f71b65d03963692205b3bba1fc81dbe96d77906.css.gz
Step #1: I, [2022-04-30T16:05:52.875667 #8] INFO -- : Writing /app/public/assets/tailwind-caafe944543690893f7a5b6d72c03227f5be6e8c9e571a9e8aecf63f7316a78e.css
Step #1: I, [2022-04-30T16:05:52.875936 #8] INFO -- : Writing /app/public/assets/tailwind-caafe944543690893f7a5b6d72c03227f5be6e8c9e571a9e8aecf63f7316a78e.css.gz
Step #1: Removing intermediate container 76f1174ba452
Step #1: ---> a6519edd8b36
Step #1: Step 14/16 : FROM augmented-base
Step #1: ---> f38133bf5629
Step #1: Step 15/16 : COPY --from=app-build /app/ /app/
Step #1: ---> d7b25b26c95f
Step #1: Step 16/16 : CMD exec bundle exec rails server -p $PORT
Step #1: ---> Running in 23b0e8a83de9
Step #1: Removing intermediate container 23b0e8a83de9
Step #1: ---> f251ce1fef21
Step #1: Successfully built f251ce1fef21
Step #1: Successfully tagged us.gcr.io/xxxxx/appengine/default.20220430t083823:latest
Finished Step #1
PUSH
Pushing us.gcr.io/xxxxxx/appengine/default.20220430t083823:latest
The push refers to repository [us.gcr.io/xxxxxxx/appengine/default.20220430t083823]
89e97bb1c87a: Preparing
a8b309f5a6e7: Preparing
653b234c4329: Preparing
2fe581f14dfe: Preparing
f254496b0294: Preparing
165e9843257f: Preparing
471f69049b89: Preparing
84ff92691f90: Preparing
b49bce339f97: Preparing
dcb7197db903: Preparing
165e9843257f: Waiting
471f69049b89: Waiting
84ff92691f90: Waiting
b49bce339f97: Waiting
dcb7197db903: Waiting
f254496b0294: Layer already exists
165e9843257f: Layer already exists
471f69049b89: Layer already exists
84ff92691f90: Layer already exists
b49bce339f97: Layer already exists
dcb7197db903: Layer already exists
2fe581f14dfe: Pushed
a8b309f5a6e7: Pushed
653b234c4329: Pushed
89e97bb1c87a: Pushed
latest: digest: sha256:180ac085461395ad3bd791f02ac93cd09db7ca8f2e30160560b867974e9e73fc size: 2420
DONE
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Updating service [default] (this may take several minutes)...failed.
ERROR: (gcloud.app.deploy) Error Response: [13] An internal error occurred.
I have two jobs and I want to execute the second one only when the first one has completed. Both are scheduled on cloud scheduler.
I am trying the get API to check the status of the first job but there is no data under the status field. Please note that I have tried to get this data when my first job was running.
{
"name": "projects/<project name>/locations/us-central1/jobs/<job name>",
"description": "Sample",
"appEngineHttpTarget": {
"httpMethod": "GET",
"appEngineRouting": {
"version": "test-v1",
"host": "test-v1.test.googleplex.com"
},
"relativeUri": "/api/v1/test",
"headers": {
"User-Agent": "AppEngine-Google; (+http://code.google.com/appengine)"
}
},
"userUpdateTime": "2020-07-17T11:44:16Z",
"state": "ENABLED",
"status": {},
"scheduleTime": "2020-07-18T11:00:00.834928Z",
"lastAttemptTime": "2020-07-17T11:44:30.439092Z",
"retryConfig": {
"maxRetryDuration": "0s",
"minBackoffDuration": "5s",
"maxBackoffDuration": "3600s",
"maxDoublings": 16
},
"schedule": "0 04 * * *",
"timeZone": "America/Los_Angeles",
"attemptDeadline": "18000s"
}
Where am I going wrong?
I was checking the documentation on this, and it looks like to get if a job is running you need to use state, as it will return the state of the job, but I could not find in the documentation a description on when the job is done.
It looks like once the job has finished, it will populate the field status, but status will give you a description of the http status from your job (so it will tell you if it failed or if it succeeded and specific information about the job failure or success)
So, in this case you would need to check if the state is ENABLED (this would tell you that the job is not paused or has other state) and if the status is populated (this would mean that the job finished and at the same time you will know if it succeeded or if it failed).
I am trying to use scikit_bring_your_own/container/decision_trees/train mode, running in AWS CLI, I had no issues. Trying to replicate in Creating Sagemaker Training Job , facing issue in loading data from S3 to docker image path.
In CLI command we used specify the docker run -v $(pwd)/test_dir:/opt/ml --rm ${image} train from where the input needs to referred.
In training job, mentioned the S3 bucket location and output path for model artifacts.
Error entered in the exception as in train - "container/decision_trees/train"
raise ValueError(('There are no files in {}.\n' +
'This usually indicates that the channel ({}) was incorrectly specified,\n' +
'the data specification in S3 was incorrectly specified or the role specified\n' +
'does not have permission to access the data.').format(training_path, channel_name))
Traceback (most recent call last):
File "/opt/program/train", line 55, in train
'does not have permission to access the data.').format(training_path, channel_name))
So not understanding is there any tweaking required or any access missing.
kindly help
If you set the InputDataConfig in the CreateTrainingJob API like this
"InputDataConfig": [
{
"ChannelName": "train",
"DataSource": {
"S3DataSource": {
"S3DataDistributionType": "FullyReplicated",
"S3DataType": "S3Prefix",
"S3Uri": "s3://<bucket>/a.csv"
}
},
"InputMode": "File",
},
{
"ChannelName": "eval",
"DataSource": {
"S3DataSource": {
"S3DataDistributionType": "FullyReplicated",
"S3DataType": "S3Prefix",
"S3Uri": "s3://<bucket>/b.csv"
}
},
"InputMode": "File",
}
]
SageMaker download the data specified above from S3 to the /opt/ml/input/data/channel_name directory in the Docker container. In this case, the algorithm container should be able to find the input data under
/opt/ml/input/data/train/a.csv
/opt/ml/input/data/eval/b.csv
You can find more details in https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-training-algo.html
I am trying to push a Nodejs sample app Hello World but after pushing I am getting error.
ERROR: (gcloud.app.deploy) Error Response: [13] An internal error
occurred while creating a Google Cloud Storage bucket.
There is nothing wrong with the code as it is just a sample app downloaded from google.
I have seen other SO posts related to this error but none of them helped.
Can anyone please tell me a Fix for this error.
I also tried to run command in debug using command and got following output.
gcloud app deploy app.yaml --verbosity debug
DEBUG: No bucket specified, retrieving default bucket.
DEBUG: Using bucket [gs://staging.united-backbone-186810.appspot.com].
DEBUG: Service [appengineflex.googleapis.com] is already enabled for project [united-backbone-186810]
Beginning deployment of service [default]...
INFO: Need Dockerfile to be generated for runtime nodejs
Building and pushing image for service [default]
INFO: Uploading [c:\users\sudha\appdata\local\temp\tmpaj1p9z\src.tgz] to [asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest]
DEBUG: Using runtime builder root [gs://runtime-builders/]
DEBUG: Loading runtimes manifest from [gs://runtime-builders/runtimes.yaml]
INFO: Reading [<googlecloudsdk.api_lib.storage.storage_util.ObjectReference object at 0x0000000006631400>]
DEBUG: Resolved runtime [nodejs] as build configuration [gs://runtime-builders/nodejs-default-builder-20171116155610.yaml]
INFO: Using runtime builder [gs://runtime-builders/nodejs-default-builder-20171116155610.yaml]
INFO: Reading [<googlecloudsdk.api_lib.storage.storage_util.ObjectReference object at 0x0000000006642390>]
Started cloud build [c3702b4b-7bc8-4861-944f-8490bf183078].
DEBUG: GCS logfile url is https://www.googleapis.com/storage/v1/b/staging.united-backbone-186810.appspot.com/o/log-c3702b4b-7bc8-4861-944f-8490bf183078.txt?alt=media
To see logs in the Cloud Console: https://console.cloud.google.com/gcr/builds/c3702b4b-7bc8-4861-944f-8490bf183078?project=united-backbone-186810
DEBUG: Operation [operations/build/united-backbone-186810/YzM3MDJiNGItN2JjOC00ODYxLTk0NGYtODQ5MGJmMTgzMDc4] not complete. Waiting 1s.
DEBUG: Reading GCS logfile: 416 (no new content; keep polling)
DEBUG: Operation [operations/build/united-backbone-186810/YzM3MDJiNGItN2JjOC00ODYxLTk0NGYtODQ5MGJmMTgzMDc4] not complete. Waiting 1s.
DEBUG: Reading GCS logfile: 206 (read 233 bytes)
------------------------------------------------- REMOTE BUILD OUTPUT --------------------------------------------------
starting build "c3702b4b-7bc8-4861-944f-8490bf183078"
FETCHSOURCE
Fetching storage object: gs://staging.united-backbone-186810.appspot.com/asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest#1511349522754483
DEBUG: Operation [operations/build/united-backbone-186810/YzM3MDJiNGItN2JjOC00ODYxLTk0NGYtODQ5MGJmMTgzMDc4] not complete. Waiting 1s.
DEBUG: Reading GCS logfile: 416 (no new content; keep polling)
DEBUG: Operation [operations/build/united-backbone-186810/YzM3MDJiNGItN2JjOC00ODYxLTk0NGYtODQ5MGJmMTgzMDc4] not complete. Waiting 1s.
DEBUG: Reading GCS logfile: 206 (read 398 bytes)
Copying gs://staging.united-backbone-186810.appspot.com/asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest#1511349522754483...
- [1 files][ 1.8 KiB/ 1.8 KiB]
DEBUG: Operation [operations/build/united-backbone-186810/YzM3MDJiNGItN2JjOC00ODYxLTk0NGYtODQ5MGJmMTgzMDc4] not complete. Waiting 1s.
DEBUG: Reading GCS logfile: 206 (read 2198 bytes)
Operation completed over 1 objects/1.8 KiB.
BUILD
Step #0: Pulling image: gcr.io/gcp-runtimes/nodejs/gen-dockerfile#sha256:196bc20ff8d91905dc071100399538814e2c619d0d27576c35a6405674da696c
Step #0: sha256:196bc20ff8d91905dc071100399538814e2c619d0d27576c35a6405674da696c: Pulling from gcp-runtimes/nodejs/gen-dockerfile
Step #0: Digest: sha256:196bc20ff8d91905dc071100399538814e2c619d0d27576c35a6405674da696c
Step #0: Status: Downloaded newer image for gcr.io/gcp-runtimes/nodejs/gen-dockerfile#sha256:196bc20ff8d91905dc071100399538814e2c619d0d27576c35a6405674da696c
Starting Step #0
Step #0: Checking for Node.js.
Finished Step #0
Step #1: Pulling image: gcr.io/cloud_builders/docker#sha256:8f8f572201e2b2ae876d8ca8b05c7d44df994e7ea8352c334ee5bae7ca3dc7f9
Step #1: sha256:8f8f572201e2b2ae876d8ca8b05c7d44df994e7ea8352c334ee5bae7ca3dc7f9: Pulling from cloud_builders/docker
Step #1: Digest: sha256:8f8f572201e2b2ae876d8ca8b05c7d44df994e7ea8352c334ee5bae7ca3dc7f9
Step #1: Status: Downloaded newer image for gcr.io/cloud_builders/docker#sha256:8f8f572201e2b2ae876d8ca8b05c7d44df994e7ea8352c334ee5bae7ca3dc7f9
Starting Step #1
Step #1: Sending build context to Docker daemon 11.26kB
Step #1: Step 1/5 : FROM gcr.io/google-appengine/nodejs#sha256:2c743f7509798cca81aaebaa339c899c4d1924153beb4a94df00ff6af238fcb2
Step #1: sha256:2c743f7509798cca81aaebaa339c899c4d1924153beb4a94df00ff6af238fcb2: Pulling from google-appengine/nodejs
Step #1: Digest: sha256:2c743f7509798cca81aaebaa339c899c4d1924153beb4a94df00ff6af238fcb2
Step #1: Status: Downloaded newer image for gcr.io/google-appengine/nodejs#sha256:2c743f7509798cca81aaebaa339c899c4d1924153beb4a94df00ff6af238fcb2
Step #1: ---> 669f53c480d3
Step #1: Step 2/5 : COPY . /app/
Step #1: ---> 388693438f26
Step #1: Removing intermediate container 45b4eecdfef0
Step #1: Step 3/5 : RUN /usr/local/bin/install_node '>=4.3.2'
Step #1: ---> Running in 67ea703659bc
Step #1: ---> 7b5b6d0283c5
Step #1: Removing intermediate container 67ea703659bc
Step #1: Step 4/5 : RUN npm install --unsafe-perm || ((if [ -f npm-debug.log ]; then cat npm-debug.log; fi) && false)
Step #1: ---> Running in 2d7b75885d6c
DEBUG: Operation [operations/build/united-backbone-186810/YzM3MDJiNGItN2JjOC00ODYxLTk0NGYtODQ5MGJmMTgzMDc4] not complete. Waiting 1s.
DEBUG: Reading GCS logfile: 206 (read 145 bytes)
Step #1: [91mnpm[0m[91m notice created a lockfile as package-lock.json. You should commit this file.
Step #1: [0madded 43 packages in 1.695s
DEBUG: Operation [operations/build/united-backbone-186810/YzM3MDJiNGItN2JjOC00ODYxLTk0NGYtODQ5MGJmMTgzMDc4] not complete. Waiting 1s.
DEBUG: Reading GCS logfile: 206 (read 1222 bytes)
Step #1: ---> e802843c73cd
Step #1: Removing intermediate container 2d7b75885d6c
Step #1: Step 5/5 : CMD npm start
Step #1: ---> Running in cef15318568d
Step #1: ---> b61594a5ec64
Step #1: Removing intermediate container cef15318568d
Step #1: Successfully built b61594a5ec64
Step #1: Successfully tagged asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest
Finished Step #1
PUSH
Pushing asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest
The push refers to a repository [asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838]
737512f7e42b: Preparing
ec3a6c686d39: Preparing
644f071ca81b: Preparing
226846715c53: Preparing
e0a3cc0c6e70: Preparing
c2003e396592: Preparing
8fc48a7a910e: Preparing
9aa804bf0e6a: Preparing
749e521e9c3d: Preparing
31cb62ec9f95: Preparing
c2003e396592: Waiting
8fc48a7a910e: Waiting
9aa804bf0e6a: Waiting
749e521e9c3d: Waiting
31cb62ec9f95: Waiting
226846715c53: Layer already exists
e0a3cc0c6e70: Layer already exists
644f071ca81b: Layer already exists
8fc48a7a910e: Layer already exists
c2003e396592: Layer already exists
31cb62ec9f95: Layer already exists
749e521e9c3d: Layer already exists
9aa804bf0e6a: Layer already exists
DEBUG: Operation [operations/build/united-backbone-186810/YzM3MDJiNGItN2JjOC00ODYxLTk0NGYtODQ5MGJmMTgzMDc4] not complete. Waiting 1s.
DEBUG: Reading GCS logfile: 206 (read 21 bytes)
ec3a6c686d39: Pushed
DEBUG: Operation [operations/build/united-backbone-186810/YzM3MDJiNGItN2JjOC00ODYxLTk0NGYtODQ5MGJmMTgzMDc4] not complete. Waiting 1s.
DEBUG: Reading GCS logfile: 206 (read 21 bytes)
737512f7e42b: Pushed
DEBUG: Operation [operations/build/united-backbone-186810/YzM3MDJiNGItN2JjOC00ODYxLTk0NGYtODQ5MGJmMTgzMDc4] not complete. Waiting 1s.
DEBUG: Reading GCS logfile: 206 (read 104 bytes)
latest: digest: sha256:6659b586325b131087fcbf872abf618fa1ee45503fff47cad7bb9e92d63bcd12 size: 2413
DONE
DEBUG: Operation [operations/build/united-backbone-186810/YzM3MDJiNGItN2JjOC00ODYxLTk0NGYtODQ5MGJmMTgzMDc4] complete. Result: {
"response": {
"finishTime": "2017-11-22T11:19:11.119096Z",
"status": "SUCCESS",
"timeout": "600s",
"startTime": "2017-11-22T11:18:45.353017219Z",
"logsBucket": "staging.united-backbone-186810.appspot.com",
"results": {
"images": [
{
"name": "asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838",
"digest": "sha256:6659b586325b131087fcbf872abf618fa1ee45503fff47cad7bb9e92d63bcd12"
},
{
"name": "asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest",
"digest": "sha256:6659b586325b131087fcbf872abf618fa1ee45503fff47cad7bb9e92d63bcd12"
}
],
"buildStepImages": [
"",
""
]
},
"createTime": "2017-11-22T11:18:44.769257383Z",
"#type": "type.googleapis.com/google.devtools.cloudbuild.v1.Build",
"source": {
"storageSource": {
"object": "asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest",
"bucket": "staging.united-backbone-186810.appspot.com"
}
},
"options": {
"substitutionOption": "ALLOW_LOOSE"
},
"steps": [
{
"args": [
"--runtime-image",
"gcr.io/google-appengine/nodejs#sha256:2c743f7509798cca81aaebaa339c899c4d1924153beb4a94df00ff6af238fcb2"
],
"name": "gcr.io/gcp-runtimes/nodejs/gen-dockerfile#sha256:196bc20ff8d91905dc071100399538814e2c619d0d27576c35a6405674da696c",
"env": [
"GAE_APPLICATION_YAML_PATH=app.yaml"
]
},
{
"args": [
"build",
"-t",
"asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest",
"."
],
"name": "gcr.io/cloud_builders/docker#sha256:8f8f572201e2b2ae876d8ca8b05c7d44df994e7ea8352c334ee5bae7ca3dc7f9",
"env": [
"GAE_APPLICATION_YAML_PATH=app.yaml"
]
}
],
"sourceProvenance": {
"resolvedStorageSource": {
"generation": "1511349522754483",
"object": "asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest",
"bucket": "staging.united-backbone-186810.appspot.com"
},
"fileHashes": {
"gs://staging.united-backbone-186810.appspot.com/asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest#1511349522754483": {
"fileHash": [
{
"type": "MD5",
"value": "RCwNC0JHUlmvZZz+C5RYsw=="
}
]
}
}
},
"projectId": "united-backbone-186810",
"images": [
"asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest"
],
"substitutions": {
"_GAE_APPLICATION_YAML_PATH": "app.yaml",
"_OUTPUT_IMAGE": "asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest"
},
"id": "c3702b4b-7bc8-4861-944f-8490bf183078",
"logUrl": "https://console.cloud.google.com/gcr/builds/c3702b4b-7bc8-4861-944f-8490bf183078?project=united-backbone-186810"
},
"done": true,
"name": "operations/build/united-backbone-186810/YzM3MDJiNGItN2JjOC00ODYxLTk0NGYtODQ5MGJmMTgzMDc4",
"metadata": {
"#type": "type.googleapis.com/google.devtools.cloudbuild.v1.BuildOperationMetadata",
"build": {
"finishTime": "2017-11-22T11:19:11.119096Z",
"status": "SUCCESS",
"timeout": "600s",
"startTime": "2017-11-22T11:18:45.353017219Z",
"logsBucket": "staging.united-backbone-186810.appspot.com",
"results": {
"images": [
{
"name": "asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838",
"digest": "sha256:6659b586325b131087fcbf872abf618fa1ee45503fff47cad7bb9e92d63bcd12"
},
{
"name": "asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest",
"digest": "sha256:6659b586325b131087fcbf872abf618fa1ee45503fff47cad7bb9e92d63bcd12"
}
],
"buildStepImages": [
"",
""
]
},
"id": "c3702b4b-7bc8-4861-944f-8490bf183078",
"source": {
"storageSource": {
"object": "asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest",
"bucket": "staging.united-backbone-186810.appspot.com"
}
},
"options": {
"substitutionOption": "ALLOW_LOOSE"
},
"steps": [
{
"args": [
"--runtime-image",
"gcr.io/google-appengine/nodejs#sha256:2c743f7509798cca81aaebaa339c899c4d1924153beb4a94df00ff6af238fcb2"
],
"name": "gcr.io/gcp-runtimes/nodejs/gen-dockerfile#sha256:196bc20ff8d91905dc071100399538814e2c619d0d27576c35a6405674da696c",
"env": [
"GAE_APPLICATION_YAML_PATH=app.yaml"
]
},
{
"args": [
"build",
"-t",
"asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest",
"."
],
"name": "gcr.io/cloud_builders/docker#sha256:8f8f572201e2b2ae876d8ca8b05c7d44df994e7ea8352c334ee5bae7ca3dc7f9",
"env": [
"GAE_APPLICATION_YAML_PATH=app.yaml"
]
}
],
"sourceProvenance": {
"resolvedStorageSource": {
"generation": "1511349522754483",
"object": "asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest",
"bucket": "staging.united-backbone-186810.appspot.com"
},
"fileHashes": {
"gs://staging.united-backbone-186810.appspot.com/asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest#1511349522754483": {
"fileHash": [
{
"type": "MD5",
"value": "RCwNC0JHUlmvZZz+C5RYsw=="
}
]
}
}
},
"projectId": "united-backbone-186810",
"images": [
"asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest"
],
"substitutions": {
"_GAE_APPLICATION_YAML_PATH": "app.yaml",
"_OUTPUT_IMAGE": "asia.gcr.io/united-backbone-186810/appengine/default.20171122t164838:latest"
},
"createTime": "2017-11-22T11:18:44.769257383Z",
"logUrl": "https://console.cloud.google.com/gcr/builds/c3702b4b-7bc8-4861-944f-8490bf183078?project=united-backbone-186810"
}
}
}
DEBUG: Reading GCS logfile: 416 (no new content; keep polling)
------------------------------------------------------------------------------------------------------------------------
DEBUG: Converted YAML to JSON: "{
"betaSettings": {
"module_yaml_path": "app.yaml",
"vm_runtime": "nodejs"
},
"env": "flex",
"handlers": [
{
"script": {
"scriptPath": "PLACEHOLDER"
},
"urlRegex": ".*"
}
],
"runtime": "vm"
}"
DEBUG: Received operation: [apps/united-backbone-186810/operations/51892505-4cb2-4b11-8431-ee7b62f0f236]
DEBUG: Operation [apps/united-backbone-186810/operations/51892505-4cb2-4b11-8431-ee7b62f0f236] not complete. Waiting to retry.
Updating service [default] (this may take several minutes).../DEBUG: Operation [apps/united-backbone-186810/operations/51892505-4cb2-4b11-8431-ee7b62f0f236] not complete. Waiting to retry.
Updating service [default] (this may take several minutes)...\DEBUG: Operation [apps/united-backbone-186810/operations/51892505-4cb2-4b11-8431-ee7b62f0f236] complete. Result: {
"metadata": {
"target": "apps/united-backbone-186810/services/default/versions/20171122t164838",
"method": "google.appengine.v1.Versions.CreateVersion",
"user": "prafactor9#gmail.com",
"insertTime": "2017-11-22T11:19:14.120Z",
"ephemeralMessage": "Deployment failed. Attempting to cleanup deployment artifacts.",
"#type": "type.googleapis.com/google.appengine.v1.OperationMetadataV1"
},
"done": true,
"name": "apps/united-backbone-186810/operations/51892505-4cb2-4b11-8431-ee7b62f0f236",
"error": {
"message": "An internal error occurred while creating a Google Cloud Storage bucket.",
"code": 13
}
}
Updating service [default] (this may take several minutes)...failed.
DEBUG: (gcloud.app.deploy) Error Response: [13] An internal error occurred while creating a Google Cloud Storage bucket.
Traceback (most recent call last):
File "C:\Users\sudha\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\calliope\cli.py", line 789, in Execute
resources = calliope_command.Run(cli=self, args=args)
File "C:\Users\sudha\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\calliope\backend.py", line 756, in Run
resources = command_instance.Run(args)
File "C:\Users\sudha\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\surface\app\deploy.py", line 65, in Run
parallel_build=False)
File "C:\Users\sudha\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\command_lib\app\deploy_util.py", line 587, in RunDeploy
flex_image_build_option=flex_image_build_option)
File "C:\Users\sudha\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\command_lib\app\deploy_util.py", line 395, in Deploy
extra_config_settings)
File "C:\Users\sudha\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\api_lib\app\appengine_api_client.py", line 188, in DeployService
message=message)
File "C:\Users\sudha\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\api_lib\app\operations_util.py", line 244, in WaitForOperation
sleep_ms=retry_interval)
File "C:\Users\sudha\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\api_lib\util\waiter.py", line 266, in WaitFor
sleep_ms=sleep_ms)
File "C:\Users\sudha\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\core\util\retry.py", line 222, in RetryOnResult
if not should_retry(result, state):
File "C:\Users\sudha\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\api_lib\util\waiter.py", line 260, in _IsNotDone
return not poller.IsDone(operation)
File "C:\Users\sudha\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\googlecloudsdk\api_lib\app\operations_util.py", line 169, in IsDone
encoding.MessageToPyValue(operation.error)))
OperationError: Error Response: [13] An internal error occurred while creating a Google Cloud Storage bucket.
ERROR: (gcloud.app.deploy) Error Response: [13] An internal error occurred while creating a Google Cloud Storage bucket.
C:\Users\sudha\Desktop\nodejs-docs-samples\appengine\hello-world>
Try by setting this configuration parameter first:
gcloud config set app/use_deprecated_preparation True
as proposed in Google's Public Issue Tracker
Try setting the configuration parameter with:
gcloud config set app/stop_previous_version true
I am trying to configure flink history server. I have followed the flink document here. I am able to bring up the history server but could only see the completed jobs, not the running jobs. Am I missing something or is it just the completed jobs are displayed in the history server. Kindly help.
Jobs currently in cluster :
localhost:8081/joboverview
{
"running": [
{
"jid": "e4e6edb76b887054d7aca460b7136937",
"name": "Filter",
"state": "RUNNING",
"start-time": 1505971363801,
"end-time": -1,
"duration": 11588615,
"last-modification": 1505971363832,
"tasks": {
"total": 1,
"pending": 0,
"running": 1,
"finished": 0,
"canceling": 0,
"canceled": 0,
"failed": 0
}
}
],
"finished": [
{
"jid": "ec16f4cf01192268150c750966cefd0d",
"name": "Flink Java Job at Thu Sep 21 10:52:09 IST 2017",
"state": "FINISHED",
"start-time": 1505971329989,
"end-time": 1505971330746,
"duration": 757,
"last-modification": 1505971330746,
"tasks": {
"total": 3,
"pending": 0,
"running": 0,
"finished": 3,
"canceling": 0,
"canceled": 0,
"failed": 0
}
}
]
}
Jobs available in History Server
localhost:8082/joboverview
{
"running": [],
"finished": [
{
"jid": "ec16f4cf01192268150c750966cefd0d",
"name": "Flink Java Job at Thu Sep 21 10:52:09 IST 2017",
"state": "FINISHED",
"start-time": 1505971329989,
"end-time": 1505971330746,
"duration": 757,
"last-modification": 1505971330746,
"tasks": {
"total": 3,
"pending": 0,
"running": 0,
"finished": 3,
"canceling": 0,
"canceled": 0,
"failed": 0
}
}
]
}
Flink History server config :
#==============================================================================
# HistoryServer
#==============================================================================
# The HistoryServer is started and stopped via bin/historyserver.sh (start|stop)
# Directory to upload completed jobs to. Add this directory to the list of
# monitored directories of the HistoryServer as well (see below).
jobmanager.archive.fs.dir: hdfs://<hdfs>:8020/sax/flink/jobmanager
# The address under which the web-based HistoryServer listens.
historyserver.web.address: localhost
# The port under which the web-based HistoryServer listens.
historyserver.web.port: 8082
# Comma separated list of directories to monitor for completed jobs.
historyserver.archive.fs.dir: hdfs://<hdfs>:8020/sax/flink/jobmanager
# Interval in milliseconds for refreshing the monitored directories.
historyserver.archive.fs.refresh-interval: 10000
The very first line of the Flink website you linked states:
Flink has a history server that can be used to query the statistics of
completed jobs after the corresponding Flink cluster has been shut
down.
which also states that
The HistoryServer allows you to query the status and statistics of
completed jobs that have been archived by a JobManager.
Now from what I see, you also have access to the monitoring api which you accessed with "localhost:8081/joboverview", If you want to access only the running jobs you can access them with "/joboverview/running" uri
You can find more information at the Monitoring API link here.