I have the following handlers section in my app.yaml:
handlers:
- url: /(robots\.txt|sitemap\.xml)
static_files: \1
upload: (robots\.txt|sitemap\.xml)
secure: always
http_headers:
Strict-Transport-Security: 'max-age=63072000; includeSubDomains; preload'
- url: /.*
script: main.app
secure: always
http_headers:
Strict-Transport-Security: 'max-age=63072000; includeSubDomains; preload'
and another subdomain, served by the another submodule (static.yaml) has the following:
handlers:
- url: /
static_dir: files
secure: always
http_headers:
Access-Control-Allow-Origin: '*'
Strict-Transport-Security: 'max-age=63072000; preload'
I was able to deploy static.yaml without any issues to the appengine:
$ appcfg.py update static.yaml
12:48 PM Host: appengine.google.com
12:48 PM Application: XXXXXX; module: static; version: 1
12:48 PM
Starting update of app: XXXXXXXX, module: static, version: 1
12:48 PM Getting current resource limits.
12:48 PM Scanning files on local disk.
[...]
[...]
12:49 PM Checking if updated app version is serving.
12:49 PM Completed update of app: XXXXXX, module: static, version: 1
whereas, when I try to update the app.yaml configuration, I get:
$ appcfg.py update app.yaml
12:48 PM Host: appengine.google.com
Usage: appcfg.py [options] update <directory> | [file, ...]
appcfg.py: error: Error parsing .\app.yaml: Unexpected attribute "http_headers" for mapping type script.
in ".\app.yaml", line 31, column 1.
I understand that it means I'd have to handle HSTS configuration in my python script itself. But, I have ~10 handlers in the main.app interface. Instead of updating each of those to add the STS header, is there some alternative to do so at app.yaml level itself?
Checking the app.yaml reference on GAE, there is no mention of restriction of http_header directive in script type mapping.
You can use app.yaml to control HTTP headers for static file handlers and not dynamic handlers. You would need to set the header within your app code.
As the doc:
https://cloud.google.com/appengine/docs/flexible/nodejs/using-custom-domains-and-ssl
You cannot use Strict-Transport-Security headers unless your domain is whitelisted. To place your domain in the whitelist, contact ...
UPDATE
As of 2018, custom domains don't need to be whitelisted. In other words, HSTS headers are not stripped out anymore.
I was looking over http headers in app.yaml today and saw this. It appears to be related to your issue.
In addition, the header Strict-Transport-Security is removed from
responses served from any domains other than *.appspot.com.
https://cloud.google.com/appengine/docs/python/how-requests-are-handled#Python_Responses
Related
I have a flask + react application that is deployed on Google App Engine. Recently, I discovered that each time I deployed a new version to the GAE, my site would go down for a few hours, and several web pages cannot load correctly. I checked the console, the web application is trying to get the static files from the last version, which resulted in a 404 Error. Can anyone help me to find what the problem is?
Here is my app.yaml file:
runtime: python37
env: standard
default_expiration: "5m"
entrypoint: gunicorn -b :$PORT main:app --timeout 150
instance_class: F4
automatic_scaling:
max_instances: 5
min_instances: 1
min_pending_latency: "5s"
target_cpu_utilization: 0.75
inbound_services:
- warmup
handlers:
- url: /static/js/(.*)
static_files: build/static/js/\1
upload: build/static/js/(.*)
- url: /static/css/(.*)
static_files: build/static/css/\1
upload: build/static/css/(.*)
- url: /static/media/(.*)
static_files: build/static/media/\1
upload: build/static/media/(.*)
- url: /(.*\.(json|ico))$
static_files: build/\1
upload: build/.*\.(json|ico)$
- url: /
static_files: build/index.html
upload: build/index.html
- url: /.*
script: auto
I am here to answer my own question. I seem to find the problem and how to solve it.
The main problem seems to be a caching issue. For the app.yaml settings, although the default expiration time is set to 5m, the url with path don't have the expiration set. For example, page www.example.com/about will have a different caching time than the js package. This means when a new build folder is deployed, the js packages have been changed, but the www.example.com/about page generated by your backend application is still the old version, and it will try to request the js package from the previous build foler. Thus, causing the 404 error.
The way to solve this is to set the expiration time for your response generated by your backend application. I am using the Flask environment, so the code for that is (credited to this answer)
response["Cache-Control"] = "no-cache, no-store, must-revalidate" # HTTP 1.1.
response["Pragma"] = "no-cache" # HTTP 1.0.
response["Expires"] = "0" # Proxies.
the web application is trying to get the static files from the last version
So are these files that were removed in your new version that you just deployed?
In general, it sounds like your problem has to do with stale browser caches. I wouldn't remove static assets from your deployed app right away specifically for this reason.
I see you're using ReactJS. Are you using the features that combine all the js and css into a single file whose name contains a hash? This should help with cache-busting.
The part that's confusing is that you said it would go down for a few hours. You have default_expiration: "5m" in your app.yaml so a few hours sounds a bit extreme. Are you not doing a hard reload (or even a full reload) when you are trying to check out your changes in your browser?
I have a lot of trouble finding how to map multiple domains to multiple services in the GAE. Here is the configuration :
One application is a Go API, deployed in GAE in the standard environment
The second application is an Angular application, also deployed in GAE in the standard environment but as another service.
Here are the app.yaml files :
Go application app.yaml
runtime: go
api_version: go1.9
handlers:
- url: /.*
script: _go_app
Angular application app.yaml
service: stage
runtime: python27
api_version: 1
threadsafe: true
skip_files:
- ^(?!dist) # Skip any files not in the dist folder
handlers:
# Routing for bundles to serve directly
- url: /((?:inline|main|polyfills|styles|vendor)\.[a-z0-9]+\.bundle\.js)
secure: always
redirect_http_response_code: 301
static_files: dist/\1
upload: dist/.*
# Routing for a prod styles.bundle.css to serve directly
- url: /(styles\.[a-z0-9]+\.bundle\.css)
secure: always
redirect_http_response_code: 301
static_files: dist/\1
upload: dist/.*
# Routing for typedoc, assets and favicon.ico to serve directly
- url: /((?:assets|docs)/.*|favicon\.ico)
secure: always
redirect_http_response_code: 301
static_files: dist/\1
upload: dist/.*
# Any other requests are routed to index.html for angular to handle so we don't need hash URLs
- url: /.*
secure: always
redirect_http_response_code: 301
static_files: dist/index.html
upload: dist/index\.html
http_headers:
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Frame-Options: DENY
I have a domain and want to bind the Go API to api.domain.com and the Angular app to domain.com.
By going to App Engine > Settings > Custom Domains I managed to add the domain for my API and it is perfectly working.
But now, I cannot find a way to map domain.com to my Angular application. Going to the same settings does not gives me an option to map a different service to my domain.
Thanks for the help and have a nice day !
To map subdomains you can use a dispatch.yaml file. An example:
dispatch:
- url: "example.com/*"
service: default
- url: "api.example.com/*"
service: otherservice
And then run $ gcloud app deploy dispatch.yaml (it can be in any directory).
Once you have example.com added under App Engine > Settings > Custom Domains for the default service, you can add the subdomain api.example.com for the other service. Later you need to add the new subdomain DNS records to you domain registrar as pointed out in the console configuration.
You want to first map your naked domain.com to your app.
Then choose to add another domain and you'll have the option to add the api (or any other) domain.com subdomain to a specific service.
Note that you have a conflicting/overlapping handler pattern in the 2 services: - url: /.*, this won't work as GAE won't know to which service to direct such requests to, they'll all end up sent to the same service. You need to partition your requests URL namespaces in a non-overlapping manner and you'll likely need a dispatch.yaml file as well. See Mapping subdomain to a service in Google App Engine project for details.
I'm running gcloud app deploy on a simple python project and deploys are suddenly failing. I've tried rolling back the gcloud components version and rolling back my codebase.
We've tried multiple computers and networks and no one can deploy.
The error and logs don't provide much clarity:
Errors were encountered while copying files to App Engine.
Here's my app.yaml:
runtime: python27
api_version: 1
threadsafe: true
default_expiration: '1m'
handlers:
- url: /assets/
static_dir: assets/
secure: always
http_headers:
X-Frame-Options: "DENY"
Strict-Transport-Security: "max-age=2592000; includeSubdomains"
X-Content-Type-Options: "nosniff"
X-XSS-Protection: "1; mode=block"
- url: /build/
static_dir: build/
secure: always
http_headers:
X-Frame-Options: "DENY"
Strict-Transport-Security: "max-age=2592000; includeSubdomains"
X-Content-Type-Options: "nosniff"
X-XSS-Protection: "1; mode=block"
- url: /.*
script: main.app
secure: always
libraries:
- name: django
version: latest
- name: jinja2
version: latest
- name: webapp2
version: latest
skip_files:
- node_modules/
- public/
- src/
- ^.git/.*
- ^node_modules/(.*/)?
- .*node_modules
- ^js/(.*/)?
- ^less/(.*/)?
- bin/
- ^(.*/)?.*\.py[co]$
gcloud uses a cloud storage bucket to upload your files to Google (since Google can't read the files on your local computer). It's possible that you list access to the default bucket for this (which I believe is staging.$APPNAME.appspot.com).
You can try using the --bucket flag to stage code to a different bucket, or the gcloud beta app repair command to recreate the bucket if you find that it's missing.
My application contains Angular and Php Yii2 framework.
I hosted my application on Google App Engine.
Here is the contents of my app.yaml file:
threadsafe: true
runtime: php55
api_version: 2
handlers:
# The root URL (/) is handled by the Go application.
# No other URLs match this pattern.
- url: /(.+)
static_files: \1
upload: (.*)
- url: /web-service/*
script: web-service/yii
- url: /
static_files: index.html
upload: index.html
My Yii2 library is available in web-service directory, but when I call REST API from the postman, it then returns a '404 page not found' error.
Am I missing something in my app.yaml file?
Please help me solve this issue. My API call is something like this:
https://abcxyz.appspot.com/web-service/web/user-registration/login-user
Several problems:
api_version: 2 - there is no such version presently, set it to 1. From the api_version row in the Syntax table:
At this time, App Engine has one version of the php runtime
environment: 1
the order of the handlers in app.yaml matters, the first one with a matching pattern will be used. Your url: /(.+) pattern will match all of your /web-service/* requests as well, so static files uploads will be attempted instead of the script(s) you're expecting. Re-order your handlers with the most significant patterns preceeding the less significant ones.
your script: web-service/yii entry might not be OK if other php files need to be served from the web-service dir (the web-service/yii will always be the one served, regardless of the requested script). Instead I'd use the handler suggested in the Example (assuming the script names always end with .php):
# Serve php scripts.
- url: /(.+\.php)$
script: \1
Always check the request entries in the development server logs as a starting point to debug request failures.
First post here at stack overflow. Please forgive my posting errors.
I have spent a lot of time at this. I started with the 500 server error.
This long is stating python not found. My app is JS, CSS, and HTML only. (at this point) I have included the yaml, because I cant rule out for myself if I have errors there through my research.
Pointers are greatly appreciated.
Thanks.
My app.yaml:
application: application
version: secureable
runtime: python27
api_version: 1
threadsafe: false
handlers:
- url: /(.*\.(gif|png|jpg|ico|js|css))
static_files: \1
upload: (.*\.(gif|png|jpg|ico|js|css))
- url: /robots.txt
static_files: robots.txt
upload: robots.txt
- url: .*
script: main.py
inbound_services:
- mail
The error:
httpRequest: {
status: 500
0: {
logMessage: "File referenced by handler not found: main.py"
severity: "WARNING"
time: "2017-09-24T21:12:30.191830Z"
}
]
megaCycles: "2"
method: "GET"
requestId: resource: "/index.html"
startTime: "2017-09-24T21:12:30.138333Z"
status: 500
traceId: "618d060203d57aea2bfddc905e350698"
urlMapEntry: "main.py"
userAgent: "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:55.0) Gecko/20100101 Firefox/55.0"
versionId: "secureable"
}
receiveTimestamp: "2017-09-24T21:12:30.926277443Z"
resource: {
labels: {
module_id: "default"
project_id: "Application"
version_id: "secureable"
zone: "us9"
}
type: "gae_app"
}
severity: "WARNING"
timestamp: "2017-09-24T21:12:30.138333Z"
}
If your app is only HTML, CSS, and JS, you can remove the catch-all pointer to the Python script all together and instead use an app.yaml format like the one shown in the Hosting a Static Website on App Engine tutorial:
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /
static_files: www/index.html
upload: www/index.html
- url: /(.*)
static_files: www/\1
upload: www/(.*)
Later if you want to add server-side logic with a Python module, you can add in a handler with a script associated with it. When you take that step, you use an import style pointer in the form of [script_name].[var_pointing_to_wsgi_application_in_script]. So if you have main.py and within that a variable called application that is set to your WSGI application, then you would use script: main.application.
Commonly a WSGI application is either webapp2 (example) or Flask (example).
Your script: main.py statement in the handlers section of the app.yaml file is wrong, it should be script: main.app.
From the script row in the Handlers element table (sadly not properly formatted, including the quote from the page source to make it readable):
script
A script: directive must be a python import path, for example,
package.module.app that points to a WSGI application. The last
component of a script: directive using a Python module path is
the name of a global variable in the module: that variable must be a
WSGI app, and is usually called app by convention.