GAE deploying results in an error - google-app-engine

Question

I'm trying to deploy my projet on Google App Engine. It previously worked successfully but since yesterday, I can't deploy it.
I get the following error :
File upload done.
Updating service [default]...failed.
ERROR: (gcloud.app.deploy) Error Response: [13] The system encountered a fatal error
When looking at the log file, I get the following :
2017-02-17 10:15:51,710 DEBUG root Operation [apps/my-project/operations/4aced3ef-a018-4913-9a5b-b3f8f13ed4ae] not complete. Waiting 5s.
2017-02-17 10:15:56,876 DEBUG root Operation [apps/my-project/operations/4aced3ef-a018-4913-9a5b-b3f8f13ed4ae] not complete. Waiting 5s.
2017-02-17 10:16:02,173 DEBUG root Operation [apps/my-project/operations/4aced3ef-a018-4913-9a5b-b3f8f13ed4ae] not complete. Waiting 5s.
2017-02-17 10:16:07,352 DEBUG root Operation [apps/my-project/operations/4aced3ef-a018-4913-9a5b-b3f8f13ed4ae] not complete. Waiting 5s.
2017-02-17 10:16:12,654 DEBUG root Operation [apps/my-project/operations/4aced3ef-a018-4913-9a5b-b3f8f13ed4ae] not complete. Waiting 5s.
2017-02-17 10:16:17,817 DEBUG root Operation [apps/my-project/operations/4aced3ef-a018-4913-9a5b-b3f8f13ed4ae] not complete. Waiting 5s.
2017-02-17 10:16:23,119 DEBUG root Operation [apps/my-project/operations/4aced3ef-a018-4913-9a5b-b3f8f13ed4ae] not complete. Waiting 5s.
2017-02-17 10:16:28,296 DEBUG root Operation [apps/my-project/operations/4aced3ef-a018-4913-9a5b-b3f8f13ed4ae] not complete. Waiting 5s.
2017-02-17 10:16:33,475 DEBUG root Operation [apps/my-project/operations/4aced3ef-a018-4913-9a5b-b3f8f13ed4ae] not complete. Waiting 5s.
2017-02-17 10:16:38,650 DEBUG root Operation [apps/my-project/operations/4aced3ef-a018-4913-9a5b-b3f8f13ed4ae] not complete. Waiting 5s.
2017-02-17 10:16:43,838 DEBUG root Operation [apps/my-project/operations/4aced3ef-a018-4913-9a5b-b3f8f13ed4ae] complete. Result: {
"metadata": {
"target": "apps/my-project/services/default/versions/20170217t100612",
"method": "google.appengine.v1.Versions.CreateVersion",
"user": "user#gmail.com",
"insertTime": "2017-02-17T09:06:34.800Z",
"ephemeralMessage": "Creating new version configuration in service \"20170217t100612\".",
"endTime": "2017-02-17T09:16:43.099Z",
"#type": "type.googleapis.com/google.appengine.v1.OperationMetadataV1"
},
"done": true,
"name": "apps/my-project/operations/4aced3ef-a018-4913-9a5b-b3f8f13ed4ae",
"error": {
"message": "The system encountered a fatal error",
"code": 13
}
}
2017-02-17 10:16:43,838 DEBUG root (gcloud.app.deploy) Error Response: [13] The system encountered a fatal error
Traceback (most recent call last):
File "/opt/google-cloud-sdk/lib/googlecloudsdk/calliope/cli.py", line 742, in Execute
resources = args.calliope_command.Run(cli=self, args=args)
File "/opt/google-cloud-sdk/lib/googlecloudsdk/calliope/backend.py", line 739, in Run
resources = command_instance.Run(args)
File "/opt/google-cloud-sdk/lib/surface/app/deploy.py", line 59, in Run
return deploy_util.RunDeploy(args)
File "/opt/google-cloud-sdk/lib/googlecloudsdk/command_lib/app/deploy_util.py", line 413, in RunDeploy
all_services)
File "/opt/google-cloud-sdk/lib/googlecloudsdk/command_lib/app/deploy_util.py", line 255, in Deploy
endpoints_info)
File "/opt/google-cloud-sdk/lib/googlecloudsdk/api_lib/app/appengine_api_client.py", line 141, in DeployService
operation)
File "/opt/google-cloud-sdk/lib/googlecloudsdk/api_lib/app/operations_util.py", line 142, in WaitForOperation
encoding.MessageToPyValue(completed_operation.error)))
OperationError: Error Response: [13] The system encountered a fatal error
2017-02-17 10:16:43,841 ERROR root (gcloud.app.deploy) Error Response: [13] The system encountered a fatal error
2017-02-17 10:16:44,726 DEBUG root Metrics reporting process started...
There is a lot of similar line like "2017-02-17 10:15:51,710 DEBUG root Operation [apps/my-project/operations/4aced3ef-a018-4913-9a5b-b3f8f13ed4ae] not complete. Waiting 5s.".
When I take a look at the "Activity" tab on the GAE web console, here's what I see :
Sorry it's in French. It's written :
10:16 - Successful operation : create a version of AppEngine
10:06 - Create a version of AppEngine
So I suspect the deployment worked, but took a lot of time to be deployed (10 minutes). Also you can see that it always take 10 minutes to be successfully done, on all my attempts.
What is wrong?

Answer 1

From recent google-cloud-sdk issue 1443 (same symptoms as far as I can tell):
We suspect from the backend logs that the application is actually
deploying successfully but that part that checks if the application is
ready to serve is in fact returning a false negative and thus,
returning the error message you received. It is then stuck in this
inaccurate state. I've added the internal tracking number of that
issue to this report as well.
Can you try issuing requests to that application/service before,
during and after attempting to deploy? The hypothesis is that this
request should provide a positive response to a health check. If this
is received during the troubling deployment, it may trigger the
deployment to returning a positive when checking if the version is
ready to serve.
And the workaround suggestion worked:
As per your request: I issued requests before, during and after
deployment. The deployment was successful. 'gcloud app deploy'
returned normally without any errors reported I tested the GAE app and
indeed the code running in the GAE project is the newly deployed
version.