I'm running gcloud app deploy on a simple python project and deploys are suddenly failing. I've tried rolling back the gcloud components version and rolling back my codebase.
We've tried multiple computers and networks and no one can deploy.
The error and logs don't provide much clarity:
Errors were encountered while copying files to App Engine.
Here's my app.yaml:
runtime: python27
api_version: 1
threadsafe: true
default_expiration: '1m'
handlers:
- url: /assets/
static_dir: assets/
secure: always
http_headers:
X-Frame-Options: "DENY"
Strict-Transport-Security: "max-age=2592000; includeSubdomains"
X-Content-Type-Options: "nosniff"
X-XSS-Protection: "1; mode=block"
- url: /build/
static_dir: build/
secure: always
http_headers:
X-Frame-Options: "DENY"
Strict-Transport-Security: "max-age=2592000; includeSubdomains"
X-Content-Type-Options: "nosniff"
X-XSS-Protection: "1; mode=block"
- url: /.*
script: main.app
secure: always
libraries:
- name: django
version: latest
- name: jinja2
version: latest
- name: webapp2
version: latest
skip_files:
- node_modules/
- public/
- src/
- ^.git/.*
- ^node_modules/(.*/)?
- .*node_modules
- ^js/(.*/)?
- ^less/(.*/)?
- bin/
- ^(.*/)?.*\.py[co]$
gcloud uses a cloud storage bucket to upload your files to Google (since Google can't read the files on your local computer). It's possible that you list access to the default bucket for this (which I believe is staging.$APPNAME.appspot.com).
You can try using the --bucket flag to stage code to a different bucket, or the gcloud beta app repair command to recreate the bucket if you find that it's missing.
Related
Here's the the app.yaml for the deploying build pages to gcloud:
runtime: nodejs16
handlers:
# Serve all static files with url ending with a file extension
- url: /(.*\..+)$
static_files: build/\1
upload: build/(.*\..+)$
# Serve all static images with url ending with a image extension
- url: /(.+\.(gif|png|jpg))$
static_files: build/images/\1
upload: build/images/.+\.(gif|png|jpg)$
# Catch all handler to index.html
- url: /.*
static_files: index.html
upload: index.html
Here's the build structure for the build:
Here are the build scripts for the project:
"scripts": {
"dev-server": "encore dev-server",
"dev": "encore dev",
"watch": "encore dev --watch",
"build": "encore production --progress"
}
When I deploy app on the gcloud here's the blank page I can find the logo png url but it's not being loaded with other components:
https://mintdapp.uc.r.appspot.com/build/images/logo.png
is the image address of the logo
Each service in your app has its own app.yaml file, which acts as a descriptor for its deployment. You must first create the app.yaml file for the default service before you can create and deploy app.yaml files for additional services within your app. For Node.js, the app.yaml is required to contain at least a runtime entry. For a brief overview, see Defining Runtime Settings . To handle deep links, you need a catch-all rule at the end to always serve index.html . However, before that, you need a rule which maps all your static content.
runtime: nodejs16 # or another supported version
instance_class: F2
env_variables:
BUCKET_NAME: "example-gcs-bucket"
handlers:
- url: /example1
static_dir: example1
- url: /.*
secure: always
redirect_http_response_code: 301
script: auto
You may also try rearranging your static assets in a folder rather than under the root. Create a directory named "staticfiles" and move all html files to it(except the homepage.html) example as below; and then try redeploying
url: /(.*\.html)
mime_type: text/html
static_files: staticfiles/\1
upload: staticfiles/(.*\.html)
I am deploying a web application to Google App Engine Standard edition using Java 8.
It is a documentation web site plus servlets that includes a huge amount of static web content (built with Jekyll).
I recently started getting this error trying to deploy the app:
ERROR: (gcloud.app.deploy) INVALID_ARGUMENT: This deployment has too many files.
New versions are limited to 10000 files for this app.
- '#type': type.googleapis.com/google.rpc.BadRequest
fieldViolations:
- description: This deployment has too many files.
New versions are limited to 10000 files for this app.
field: version.deployment.files[...]
My src/main/webapp folder has 5016 files.
But after running:
mvn clean install appengine:stage
my target/appengine-staging folder has 10113 files.
I see that the files are in 2 locations, both
target/appengine-staging
and
target/appengine-staging/__static__
Why do they appear twice?
Is this normal?
Or am I doing something wrong?
How can I avoid this duplication?
Or do I have to limit my files to under 5000?
Thanks for any advice.
Note: The generated app.yaml looks like this:
runtime: java8
instance_class: F1
inbound_services:
- warmup
derived_file_type:
- java_precompiled
threadsafe: True
auto_id_policy: default
beta_settings:
'source_reference': 'https://...'
api_version: 'user_defined'
handlers:
- url: (/.*/)
static_files: __static__\1index.html
upload: __NOT_USED__
require_matching_file: True
login: optional
secure: always
- url: (/)
static_files: __static__\1index.html
upload: __NOT_USED__
require_matching_file: True
login: optional
secure: always
- url: (/.*)
static_files: __static__\1
upload: __NOT_USED__
require_matching_file: True
login: optional
secure: always
- url: /.*
script: unused
login: optional
secure: always
skip_files: app.yaml
I have a flask + react application that is deployed on Google App Engine. Recently, I discovered that each time I deployed a new version to the GAE, my site would go down for a few hours, and several web pages cannot load correctly. I checked the console, the web application is trying to get the static files from the last version, which resulted in a 404 Error. Can anyone help me to find what the problem is?
Here is my app.yaml file:
runtime: python37
env: standard
default_expiration: "5m"
entrypoint: gunicorn -b :$PORT main:app --timeout 150
instance_class: F4
automatic_scaling:
max_instances: 5
min_instances: 1
min_pending_latency: "5s"
target_cpu_utilization: 0.75
inbound_services:
- warmup
handlers:
- url: /static/js/(.*)
static_files: build/static/js/\1
upload: build/static/js/(.*)
- url: /static/css/(.*)
static_files: build/static/css/\1
upload: build/static/css/(.*)
- url: /static/media/(.*)
static_files: build/static/media/\1
upload: build/static/media/(.*)
- url: /(.*\.(json|ico))$
static_files: build/\1
upload: build/.*\.(json|ico)$
- url: /
static_files: build/index.html
upload: build/index.html
- url: /.*
script: auto
I am here to answer my own question. I seem to find the problem and how to solve it.
The main problem seems to be a caching issue. For the app.yaml settings, although the default expiration time is set to 5m, the url with path don't have the expiration set. For example, page www.example.com/about will have a different caching time than the js package. This means when a new build folder is deployed, the js packages have been changed, but the www.example.com/about page generated by your backend application is still the old version, and it will try to request the js package from the previous build foler. Thus, causing the 404 error.
The way to solve this is to set the expiration time for your response generated by your backend application. I am using the Flask environment, so the code for that is (credited to this answer)
response["Cache-Control"] = "no-cache, no-store, must-revalidate" # HTTP 1.1.
response["Pragma"] = "no-cache" # HTTP 1.0.
response["Expires"] = "0" # Proxies.
the web application is trying to get the static files from the last version
So are these files that were removed in your new version that you just deployed?
In general, it sounds like your problem has to do with stale browser caches. I wouldn't remove static assets from your deployed app right away specifically for this reason.
I see you're using ReactJS. Are you using the features that combine all the js and css into a single file whose name contains a hash? This should help with cache-busting.
The part that's confusing is that you said it would go down for a few hours. You have default_expiration: "5m" in your app.yaml so a few hours sounds a bit extreme. Are you not doing a hard reload (or even a full reload) when you are trying to check out your changes in your browser?
We are trying to access .json file inside our python code in Google App Engine.
Here is our app.yaml
service: worker
instance_class: F2
runtime: python27
api_version: 1
threadsafe: true
automatic_scaling:
max_instances: 100
min_pending_latency: 200ms # default value
max_pending_latency: automatic
handlers:
- url: /stylesheets
static_dir: stylesheets
- url: /images
static_dir: images
- url: /javascripts
static_dir: javascripts
- url: /(.*\.(gif|png|jpg|json))$
static_files: static/\1
upload: static/.*\.(gif|png|jpg|json)$
application_readable: true
- url: /static
static_dir: static
application_readable: true
- url: /favicon\.ico
static_files: favicon.ico
upload: favicon\.ico
- url: /.*
script: worker.app
login: admin
libraries:
- name: webapp2
version: "2.5.2"
- name: jinja2
version: latest
We are using it in our application like this.
path = os.path.join(os.path.dirname(__file__)[0], '/static/google_web_app_script_credentials.json')
flow = InstalledAppFlow.from_client_secrets_file(path,self.GOOGLESHEETS_SCOPES )
But we are receiving the error.
IOError(2, 'No such file or directory')
We have tried various options of changing path including path = os.path.join(os.path.dirname(file)[0], '../static/google_web_app_script_credentials.json')
When we go to the GCP console -> Debug and review the current code we are able to see the .json file under static folder. This means that the file is there, we are not able to get the application to read it. This service is as "worker".
From your App Engine app, you don't have access to your static files. Move the file to another location within your project, and you will then be able to access it.
Even more importantly, you don't want to store credentials in your static folder! Anyone on the web will be able to access them via the URL!
I am servicing a system.
An old App Engine tool was used to deploy, but it is now obsolete and I need to use the gcloud command line.
However, gcloud is ignoring one system folder and deploying all others.
One big reason this must be happening is because they developed this system based on node_modules and didn't build the code before uploading. In short, I need the node_modules folder for the code to execute.
I already disabled .gcloudignore, but there have been no changes.
Source folder structure:
Folder Structure in App Engine:
I ran the --verbosity = info code on my gcloud deploy and it appeared that they were ignoring it:
My app.yaml:
runtime: python27
api_version: 1
threadsafe: true
libraries:
- name: jinja2
version: 2.6
- name: lxml
version: 2.3
- name: pycrypto
version: 2.6
handlers:
- url: /static
static_dir: app/static
secure: always
- url: /.*
script: manage.app
secure: always
builtins:
- remote_api: on
By default, flask use a "templates" folder to contain all your template files(any plain-text file, but usually .html or some kind of template language such as jinja2 ) & a "static" folder to contain all your static files(i.e. .js .css and your images).
So ideally you should not use /static folder
you will need to update app.yaml to include handler static_dir: static
runtime: python27
api_version: 1
threadsafe: true
libraries:
- name: ssl
version: latest
# [START handlers]
handlers:
- url: /static
static_dir: static
- url: /.*
script: main.app
# [END handlers]
The best way to use is
from flask import Flask
app = Flask(__name__, static_folder='static', static_url_path='')