I'm getting warmup request every second in my GAE Python app, it results in 404, my app.yaml is below:
runtime: python37
entrypoint: gunicorn -w 4 -k uvicorn.workers.UvicornWorker api:app
instance_class: F4
inbound_services:
- warmup
handlers:
- url: /_ah/warmup
script: auto
- url: /.*
static_dir: /.*
http_headers:
Access-Control-Allow-Origin: "*"
# ...
I've tried removing the inbound_services: - warmup, but the requests didn't stop, they also appear to be originated from an earlier version, any idea on how can I stop them?
similar issue: Too many /_ah/warmup/ queries on Google App Engine
I'm not sure about stopping them, I would think that removing inbound_services: - warmup would stop it.
As for your original issue of the 404s though, you need to setup a handler in your python code for the url /_ah/warmup, since you specified script: auto
Related
I have a flask + react application that is deployed on Google App Engine. Recently, I discovered that each time I deployed a new version to the GAE, my site would go down for a few hours, and several web pages cannot load correctly. I checked the console, the web application is trying to get the static files from the last version, which resulted in a 404 Error. Can anyone help me to find what the problem is?
Here is my app.yaml file:
runtime: python37
env: standard
default_expiration: "5m"
entrypoint: gunicorn -b :$PORT main:app --timeout 150
instance_class: F4
automatic_scaling:
max_instances: 5
min_instances: 1
min_pending_latency: "5s"
target_cpu_utilization: 0.75
inbound_services:
- warmup
handlers:
- url: /static/js/(.*)
static_files: build/static/js/\1
upload: build/static/js/(.*)
- url: /static/css/(.*)
static_files: build/static/css/\1
upload: build/static/css/(.*)
- url: /static/media/(.*)
static_files: build/static/media/\1
upload: build/static/media/(.*)
- url: /(.*\.(json|ico))$
static_files: build/\1
upload: build/.*\.(json|ico)$
- url: /
static_files: build/index.html
upload: build/index.html
- url: /.*
script: auto
I am here to answer my own question. I seem to find the problem and how to solve it.
The main problem seems to be a caching issue. For the app.yaml settings, although the default expiration time is set to 5m, the url with path don't have the expiration set. For example, page www.example.com/about will have a different caching time than the js package. This means when a new build folder is deployed, the js packages have been changed, but the www.example.com/about page generated by your backend application is still the old version, and it will try to request the js package from the previous build foler. Thus, causing the 404 error.
The way to solve this is to set the expiration time for your response generated by your backend application. I am using the Flask environment, so the code for that is (credited to this answer)
response["Cache-Control"] = "no-cache, no-store, must-revalidate" # HTTP 1.1.
response["Pragma"] = "no-cache" # HTTP 1.0.
response["Expires"] = "0" # Proxies.
the web application is trying to get the static files from the last version
So are these files that were removed in your new version that you just deployed?
In general, it sounds like your problem has to do with stale browser caches. I wouldn't remove static assets from your deployed app right away specifically for this reason.
I see you're using ReactJS. Are you using the features that combine all the js and css into a single file whose name contains a hash? This should help with cache-busting.
The part that's confusing is that you said it would go down for a few hours. You have default_expiration: "5m" in your app.yaml so a few hours sounds a bit extreme. Are you not doing a hard reload (or even a full reload) when you are trying to check out your changes in your browser?
The current documentation of AppEngine for PHP 7.2 does not mention warmup requests. While I can configure some warmup settings, I cannot specify a specific handler.
app.yaml excerpt:
# url handlers
handlers:
- url: /_ah/warmup
script: warmup.php
login: admin
- url: /.*
script: auto
# run url request on startup
inbound_services:
- warmup
Error:
ERROR: (gcloud.app.deploy) INVALID_ARGUMENT: script field for handler '/_ah/warmup' must be set to 'auto' for runtime php72.
How can I securely receive warmup requests now?
I would like to host PrivateBin from a GCP App Engine, but I don't know how to configure the app.yaml file. If somebody has done this before, I would appreciate the help. :)
I have tried two options, but neither of them worked:
runtime: php55
api_version: 1
threadsafe: yes
handlers:
- url: /.*
script: index.php
&
runtime: php55
api_version: 1
handlers:
- url: /(.*.(ico$|jpg$|png$|gif$|htm$|html$|css$|js$|xml$))
static_files: \1
upload: (.*.(ico$|jpg$|png$|gif$|htm$|html$|css$|js$|xml$))
application_readable: true
- url: /(.+)
script: \1
- url: /
script: index.php
How do you intend to store your data?
https://github.com/PrivateBin/PrivateBin/blob/master/INSTALL.md#using-a-database-instead-of-flat-files
Looks like private bin uses the filesystem by default, which wont work on GAE Standard. You could do that in GAE flex, but the directory where you are reading and writing to would need to be a mounted Google Cloud Storage Bucket or something for the data to be persistent (as well as be available to multiple instances) https://cloud.google.com/storage/docs/gcs-fuse
The better route seems to be to configure privatebin to use a SQL db, which would be Google Cloud SQL:
https://cloud.google.com/sql/docs/
https://cloud.google.com/php/getting-started/using-cloud-sql-with-mysql
Does App Engine cache responses server side for either dynamic or static requests if I set Cache-Control headers? Documentation doesn't seem to clarify this either way https://cloud.google.com/appengine/docs/standard/php/how-requests-are-handled
I have an API that responds highly cachable responses, so it'd be nice to leverage any edge caching.
You can set the cache in your app.yaml file for static files
- url: /static
static_dir: static
expiration: 10m
You can set a a default cache in your app.yaml file
application: my-app
version: 1
runtime: python27
api_version: 1
threadsafe: yes
module: default
default_expiration: "1h"
instance_class: F2
For caching json/response data from request handlers you can use Memcache
I have created a simple API using Cloud Endpoints, which makes some translations using Google Translate API, caches them in Datastore and returns the list of results to Client.
The problem is that each time I make a call, which uses Google Translate API I receive 502 Bad Gateway error.
I've been debugging my endpoint from cloud console and the response is created successfully, so the 502 must be thrown when sending the result back to client. Another strange thing here is that I don't see 502 on any logging tool in Cloud Console.
I was trying to change region from europe-west to us-central, but no luck so far.
Does anyone have any ideas, what may be wrong here?
My cofnig of GAE:
runtime: java7
env: standard
threadsafe: true
instance_class: B2
inbound_services:
- warmup
handlers:
- url: '(/.*/)'
application_readable: false
static_files: "__static__\\1index.html"
require_matching_file: true
upload: __NOT_USED__
- url: (/)
application_readable: false
static_files: "__static__\\1index.html"
require_matching_file: true
upload: __NOT_USED__
- url: '(/.*)'
application_readable: false
static_files: "__static__\\1"
require_matching_file: true
upload: __NOT_USED__
- url: /
script: unused
- url: '/.*/'
script: unused
- url: '/_ah/.*'
script: unused
basic_scaling:
idle_timeout: 900s
max_instances: 2
When changing configuration of GAE and removing basic-scaling I get from Google translate following error:
com.google.cloud.translate.TranslateException: com.google.apphosting.api.ApiProxy$CancelledException: The API call urlfetch.Fetch() was cancelled because the overall HTTP request deadline was reached.
at io.stringx.repository.DatastoreTranslationRepository.translate (DatastoreTranslationRepository.java:53)
It turned out that Google Cloud Endpoints have 60s deadline limit, so I had to make my requests in chunks:
https://cloud.google.com/appengine/articles/deadlineexceedederrors#UrlFetch_Delays