AppEngine App Deploy gives "Failed to copy file." - google-app-engine

I'm deploying a Laravel app to a fresh App Engine project. But I'm getting this error constantly. I tried creating a new project and deploying too.
╔════════════════════════════════════════════════════════════╗
╠═ Uploading 9625 files to Google Cloud Storage ═╣
╚════════════════════════════════════════════════════════════╝
File upload done.
Updating service [default]...failed.
ERROR: (gcloud.app.deploy) Error Response: [3] Errors were encountered while copying files to App Engine.
Details: [
[
{
"#type": "type.googleapis.com/google.rpc.ResourceInfo",
"description": "Failed to copy file.",
"resourceName": "https://storage.googleapis.com/staging.rsvp.appspot.com/df4bc71e8832337e997291648609c4e207b5aa55",
"resourceType": "file"
}
]
]
WHat is the problem here and how can I fix it?

The reason for this issue was, I had some large size files ( > 30Mb ) in my project folder. I removed them and redeployed, it worked without an issue.

As OP already mentioned, the reason is large files. Here is how you can also find out which file is causing the problem.
If you have an error such as
Details: [
[
{
"#type": "type.googleapis.com/google.rpc.ResourceInfo",
"description": "Failed to copy file.",
"resourceName": "https://storage.googleapis.com/staging.bemmu1-hrd.appspot.com/b463c152ee1498bd4d27c1ea67c7f8e82cb4b220",
"resourceType": "file"
}
]
]
Note the hash that appears in resourceName, in this case "b463c152ee1498bd4d27c1ea67c7f8e82cb4b220".
Search for it in the most recent log file (mine was /Users/bemmu/.config/gcloud/logs/2019.07.30/00.45.43.657001.log) and you'll find it as part of a really long string, in my case:
...'templates/envato/images/arrangement.png': {'sourceUrl': 'https://storage.googleapis.com/staging.bemmu1-hrd.appspot.com/b463c152ee1498bd4d27c1ea67c7f8e82cb4b220'...
From this I could see that in my case the offending file was arrangement.png

Related

Firebase hosting deployment failing

Running firebase deploy will run for a few minutes before giving me a timeout error Error: ESOCKETTIMEDOUT
I've successfully deployed multiple times earlier, without changing anything but the frontend of my project (written in React).
I have a separate folder with my cloud functions, which deploy perfectly with firebase deploy --only functions
I've tried regular deployment with firebase deploy as well as with --except functions flag but with no luck.
Also tried updating npm and firebase-tools
running firebase deploy --debug yields the following error after a few minutes:
[2019-09-12T16:25:59.115Z] FirebaseError: Task 6822eef99ebcce0df4baf9cc03e49b399d046c7cdfeb5b2a2ef2511c1df963d1 failed: retries exhausted after 6 attempts
[2019-09-12T16:25:59.115Z] [hosting][upload queue][FINAL] max=7278, min=7278, avg=7278, active=4, complete=2, success=1, errored=1, retried=25, total=6, elapsed=192701
[2019-09-12T16:25:59.127Z] FirebaseError: Task f98dce8e7830e8cd5881bc5e109453c208ff7fc643d61ad64ca34a7d098abc2c failed: retries exhausted after 6 attempts
[2019-09-12T16:25:59.127Z] FirebaseError: Task 5a7166c6cab1624c8a0c074fc535a23685c338be838012dced4006be3cc6dedd failed: retries exhausted after 6 attempts
[2019-09-12T16:25:59.128Z] FirebaseError: Task 76dd5b5551e5cd383875692455ff415bf8df56c97f67e4a3746e3c53c23478e1 failed: retries exhausted after 6 attempts
[2019-09-12T16:25:59.129Z] FirebaseError: Task 541bb1a8843dd0c62ea7e673f2260c85e0b456f35cfde358b6ab3be936e9c705 failed: retries exhausted after 6 attempts
[2019-09-12T16:25:59.211Z] Error: ESOCKETTIMEDOUT
at ClientRequest.<anonymous> (/usr/local/lib/node_modules/firebase-tools/node_modules/request/request.js:816:19)
at Object.onceWrapper (events.js:298:28)
at ClientRequest.emit (events.js:209:13)
at ClientRequest.EventEmitter.emit (domain.js:476:20)
at TLSSocket.emitRequestTimeout (_http_client.js:690:9)
at Object.onceWrapper (events.js:298:28)
at TLSSocket.emit (events.js:209:13)
at TLSSocket.EventEmitter.emit (domain.js:476:20)
at TLSSocket.Socket._onTimeout (net.js:468:8)
at listOnTimeout (internal/timers.js:531:17)
Error: Task 6822eef99ebcce0df4baf9cc03e49b399d046c7cdfeb5b2a2ef2511c1df963d1 failed: retries exhausted after 6 attempts
Delete the hidden folder in you project root directory.
.firebase/hosting.*.cache
Deleting the .firebase folder and tried firebase deploy command again. Things seemed to be working fine.
As #thmsdnnr mentioned, it seems that this error was caused by issues at firebase. Tried a new deployment first thing in the morning and everything went through first try.
There are two reasons to happen this.
by deleting .firebase folder in your root folder will solve problem.
if this dosen't work,
your internet connection may slow and size of files in your project may be bigger. so try with fast internet connection.
with slow internet, if you try deploying again and again, you will see that number of files uploading decreases on console. that means your files are getting uploaded, but it taking too much time, and firebase is getting exhausted.
as explained already by #thmsdnnr, it is indeed a firebase issue, but can be due to some issues at your end as well.
For example, for me it was the slow internet connection responsible for it.
Besides, you can always delete the ".firebase" folder, and re-build your project for starting afresh with hosting.
I also faced an issue like that
Error: Task 91525507ddf6d644ed40f9c46c3e30a2164f496acb1328512d4cf5eaf8d18768 failed: retries exhausted after 6 attempts
PS D:\React\amazon-clone> firebase deploy
But I fixed it. Put this change in your firebase.json file:
{
"hosting": {
"public": "build",
"ignore": [
"firebase.json",
"**/.*",
"**/node_modules/**"
],
"rewrites": [
{
"source": "**",
"destination": "/index.html"
}
]
}
}
to
{
"hosting": {
"public": "public",
"ignore": [
"firebase.json",
"**/.*",
"**/node_modules/**"
],
"rewrites": [
{
"source": "**",
"destination": "/index.html"
}
]
}
}
and it will work fine.
I was struggling with this problem too. After trying many methods, I solved the deployment issue as follows;
At first, I deleted node modules and reinstalled it again. I deleted firebase files (.firebase, firebase.json)
Then I applied them;
Update Firebase CLI via "npm i -g firebase-tools"
Log in to your account via "firebase login",
Initialize the project via "firebase init hosting"
And lastly deploy it via "firebase deploy --only hosting"
I was able to deploy my project on firebase with these steps. I hope someone else's problem is solved this way.
I recommend you to check here for detailed information.
As mentioned in the error massage: Timeout reached making request to https://url_path.
I found the same problem, the mine relates to the network.
So check your network connection.

AppEngine Deploy is Conflicting SHA1 sum for a file

I want to deploy an update to our app but we have an issue with gcloud command which failed to "Conflicting SHA1 sum for file". Here is the output:
➜ gcloud app deploy xxx-xxx-xxxx.yaml --project xxx-xxx-xxxx --quiet
Services to deploy:
descriptor: [/home/vzool/Workspace/xxx-xxxx-xxxxxxxxxxx/xxx-xxx-xxxx.yaml]
source: [/home/vzool/Workspace/xxx-xxxx-xxxxxxxxxxx]
target project: [xxx-xxx-xxxx]
target service: [default]
target version: [20180917t110153]
target url: [https://xxx-xxx-xxxx.appspot.com]
Beginning deployment of service [default]...
╔════════════════════════════════════════════════════════════╗
╠═ Uploading 1 file to Google Cloud Storage ═╣
╚════════════════════════════════════════════════════════════╝
File upload done.
Updating service [default]...failed.
ERROR: (gcloud.app.deploy) Error Response: [3] The following errors occurred while copying files to App Engine:
File https://storage.googleapis.com/staging.xxx-xxx-xxxx.appspot.com/c819d7599ceb31df8d4370eed1981defcb5034c6 failed with: Conflicting SHA1 sum for file. Expected "ddb6af3b_cf4bff57_613fb78d_208480fa_1717b3c4" but received "c819d759_9ceb31df_8d4370ee_d1981def_cb5034c6".
Details: [
[
{
"#type": "type.googleapis.com/google.rpc.ResourceInfo",
"description": "Conflicting SHA1 sum for file. Expected \"ddb6af3b_cf4bff57_613fb78d_208480fa_1717b3c4\" but received \"c819d759_9ceb31df_8d4370ee_d1981def_cb5034c6\".",
"resourceName": "https://storage.googleapis.com/staging.xxx-xxx-xxxx.appspot.com/c819d7599ceb31df8d4370eed1981defcb5034c6",
"resourceType": "file"
}
]
]
I do delete this file c819d7599ceb31df8d4370eed1981defcb5034c6 but everytime new hash will be generated, delete entire block staging.xxx-xxx-xxxx.appspot.com and I did clear Memcache but with no luck trying all these options! :(
I don't know what to do anymore with this, any suggestions may help me out?
Thanks
I think after sometime in research the issue, the app itself has a live feature which lead to update one file while you open live page.
So, that change what gcloud uploaded.
gcloud upload then verify. And if file changed after upload and before verify gcloud will through this error on your face. Ha ha ha ^_^
OP's answer seemed to indicate that a file got touched during the gcloud deploy process.
The solution for me was to:
git checkout some random branch
git checkout the branch I was actually trying to deploy again
run gcloud app deploy . --version my_version_id --project my_project_id

Google App Engine Deploy error 13 - Deployment Manager operation failed

I am trying to deploy an app first time and getting this at the very end of 'gcloud app deploy' operation.
ERROR: (gcloud.app.deploy) Error Response: [13] Deployment Manager operation failed, name: operation-1522364367335-5689513556e59-0732922b-1662dc1e, error:
[
{
"code":"RESOURCE_ERROR",
"location":"/deployments/aef-default-20180329t155754/resources/aef-default-20180329t155754-hcfw",
"message":{
"ResourceType":"compute.v1.firewall",
"ResourceErrorCode":"404",
"ResourceErrorMessage":{
"code":404,
"errors":[
{
"domain":"global",
"message":"The resource 'projects/kubernetes-staging/global/networks/default' was not found",
"reason":"notFound"
}
],
"message":"The resource 'projects/kubernetes-staging/global/networks/default' was not found",
"statusMessage":"Not Found",
"requestPath":"https://www.googleapis.com/compute/v1/projects/kubernetes-staging/global/firewalls",
"httpMethod":"POST"
}
}
},
{
"code":"RESOURCE_ERROR",
"location":"/deployments/aef-default-20180329t155754/resources/aef-default-20180329t155754-00it",
"message":{
"ResourceType":"compute.v1.instanceTemplate",
"ResourceErrorCode":"404",
"ResourceErrorMessage":{
"code":404,
"errors":[
{
"domain":"global",
"message":"The resource 'projects/kubernetes-staging/global/networks/default' was not found",
"reason":"notFound"
}
],
"message":"The resource 'projects/kubernetes-staging/global/networks/default' was not found",
"statusMessage":"Not Found",
"requestPath":"https://www.googleapis.com/compute/v1/projects/kubernetes-staging/global/instanceTemplates",
"httpMethod":"POST"
}
}
}
]
As the error states, the default global network (entitled 'default') hasn't been found, which appears to be the reason why your app isn't deploying.
You can create a global network by executing this command in Cloud Shell before deploying your app:
$ gcloud compute networks create default --subnet-mode auto
If you didn't modify the original default network, it would suggest that the issue is related to the network settings in your app.yaml configuration file. Information on configuring these settings can be found here.
If you follow the steps above but are still having trouble, I suggest that you create a new issue in the Public Issue Tracker and provide us with the contents of your app.yaml file as well as your Project ID, where I'd be happy to investigate further.
(Disclaimer: I work for Google Cloud Platform Support)

Connector Manifest not Validating in DataStudio

I am trying to test a connector I am building. I have created the manifest file (appsscript.json) and am trying to add this by Deployment ID to DataStudio. I keep getting an error that states:
The connector manifest could not be retrieved or is invalid. Check the connector and try again.
Here is a a copy/paste of my appsscripts.json file:
{
"timeZone": "America/New_York",
"dependencies": {
"libraries": [
{
"userSymbol": "OAuth2",
"libraryId": "1B7FSrk5Zi6L1rSxxTDgDEUsPzlukDsi4KGuTMorsTQHhGBzBkMun4iDF",
"version": "24"
}
]
},
"dataStudio": {
"name": "VALID NAME",
"company": "VALID NAME",
"logoUrl": "VALID LOGO",
"addonUrl": "VALID URL",
"supportUrl": "VALID URL",
"description": "VALID DESCRIPTION"
}
}
I would expect this to enable the workflow of enabling the connector and then allowing the OAuth flow to be tested.
Instead, I get this error:
The connector manifest could not be retrieved or is invalid. Check the connector and try again.
Can anyone advise why this is validating? I have followed these steps:
https://developers.google.com/datastudio/connector/use
Thanks!
Double check the permissions of your script. Especially if you are using corporate account (not from your #gmail.com address) the default sharing is corporate-only, therefore your script (and the manifest) is not available from outside.
Before Selecting the link, you need to select the Install add-on. It worked for me. I have enclosed the screenshot for reference
Check the installation URL if you are signed into multiple google accounts.
You may have to change to the u/0 section to u/1, or whichever number appears here for the relevant google account:
https://datastudio.google.com/**u/0**/datasources/create?connectorId=...

Exclude Thumbs.db from Sencha Cmd build

When executing "sencha app build production", I receive the following error message:
[ERR] BUILD FAILED
[ERR] com.sencha.exceptions.BasicException: Cannot find file : C:\apache24\htdoc
s\extjs\myapp\build\production\myapp\resources\images\Thumbs.db (Access denied)
I know that the answer could be "use a system tool to close the file handle". But I am tired of doing so, and since the Thumbs.db files are useless, I'd like to exclude copying them from the build process - once and for all times.
I tried via app.json, where I found an exclude for svn files:
"resources": [
"resources/images"
],
"ignore": [
"\.svn$",
"\.db$"
],
but that didn't yield the expected result. Any ideas?

Resources