I am deploying a Go application using AppEngine flexible. Below is my app.yaml. Sometimes after I deploy it stabilizes at 1 instance (it's a very low load application), but most of the time it constantly respawns upwards of 6 instances. My logs are filled with messages showing the new instances being created. There is nearly zero load on this application, why is AppEngine flexible constantly destroying and respawning instances?
Log showing constant respawning:
app.yaml
runtime: go
api_version: go1
env: flex
handlers:
- url: /.*
script: _go_app
health_check:
enable_health_check: True
check_interval_sec: 10
timeout_sec: 4
unhealthy_threshold: 2
healthy_threshold: 2
automatic_scaling:
min_num_instances: 1
max_num_instances: 10
cool_down_period_sec: 120 # default value
cpu_utilization:
target_utilization: 0.5
The problem was with my health check function. It originally looked like this:
func healthCheckHandler(w http.ResponseWriter, r *http.Request) {
return
}
I then discovered this sentence in the documentation on how instances are managed:
You can write your own custom health-checking code. It should reply to /_ah/health requests with a HTTP status code 200. The response must include a message body, however, the value of the body is ignored (it can be empty).
So I changed the health check function to write a simple "ok" in response:
func healthCheckHandler(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("ok"))
return
}
The instances now behave according to my autoscale settings! The respawning is gone.
I obviously should have read the documentation closer, but there was zero indication of a problem in the health check logs. All health checks looked like they were passing. Hopefully this info is helpful to others.
Related
I get the following from Lighthouse:
How do I change the Cache TTL on a Nuxt.js SSR website? I found some answers but nothing about Nuxt.js...
IMPORTANT: Deployed in Google App Engine
The specific answer for GAE Apps, is the parameter handlers.expiration in app.yaml file:
handlers:
- url: /_nuxt
static_dir: .nuxt/dist/client
expiration: 4d 5h
secure: always
Or if you want to configure it globally, set the default_expiration parameter at the root level:
default_expiration: 4d 5h
Allows d (days), h (hours), m (minutes) and s (seconds). Here's the docs
You can serve your static folder with custom cache policy following the render configuration.
As an example:
render: {
// Setting up cache for 'static' directory - a year in milliseconds
// https://web.dev/uses-long-cache-ttl
static: {
maxAge: 60 * 60 * 24 * 365 * 1000,
},
},
In addition to the answer, please note that - at the time of writing - the minimum time required by Lighthouse to pass is > 96.5d (Source : https://github.com/GoogleChrome/lighthouse/issues/11380)
I've followed the answer by #lmfresneda and managed to get the solution to work, making the cache time 30d : I still had the Lighthouse test fail until I changed it to "97d".
I am using app.yaml file to configure my app engine flexible. The file looks like the following,
runtime: java
env: flex
service: hello-service
health_check:
enable_health_check: True
check_interval_sec: 10
timeout_sec: 4
unhealthy_threshold: 2
healthy_threshold: 2
automatic_scaling:
min_num_instances: 3
max_num_instances: 10
cool_down_period_sec: 120 # default value
cpu_utilization:
target_utilization: 0.5
However, when I click the "view" link for the version list in the cloud console, I only can see the following in the popup,
runtime: java
env: flexible
threadsafe: true
automatic_scaling:
min_num_instances: 3
max_num_instances: 10
health_check:
enable_health_check: true
check_interval_sec: 10
timeout_sec: 4
unhealthy_threshold: 2
healthy_threshold: 2
As you can see, it is missing few "automatic_scaling" properties. I am not sure why. Do I need to stop and start the relevant version to see the changes?
It is most likely that the config values matching the default values are not displayed.
From the documentation the default values for the missing parameters:
cool_down_period_sec
The number of seconds that the autoscaler should wait before it
starts collecting information from a new instance. This prevents the
autoscaler from collecting information when the instance is
initializing, during which the collected usage would not be reliable.
The cool-down period must be greater than or equal to 60 seconds.
The default is 120 seconds.
target_utilization
Target CPU utilization (default 0.5). CPU use is averaged across
all running instances and is used to decide when to reduce or increase
the number of instances.
The cpu_utilization is likely not displayed because target_utilization (the only item under it) dissapeared.
It should be easy to check - just change the values for the missing configs slightly, re-deploy and see if the updated values are remebered or not.
I've specified my environment variables in app.yaml and it's being fetched when I'm running it on my local machine but once I have it deployed - it's not fetching it.
Here's how I've set it up:
application: some-application
version: 1
runtime: go
api_version: go1
threadsafe: true
handlers:
- url: /.*
script: main.go
secure: always
env_variables:
ENVIRONMENT_VAR1: 'some key'
ENVIRONMENT_VAR2: 'some key'
ENVIRONMENT_VAR3: 'some key'
And I'm using os.Getenv("ENVIRONMENT_VAR1") to retrieve the key and it works when I run it on my local but fails to work when deployed on google app engine.
It is undocumented in the official doc: Defining environment variables, but environment variables defined in app.yaml in production are not set before init() functions are called. They are only set before serving the first request.
This issue was reported here. Quoting an AppEngine engineer's answer:
Right. Due to the nature of the implementation, environment variables are not available in your init functions, unfortunately.
Although they are not tied to requests, they are not set until after all the init functions have already run, but are set before the first request is handled.
As a result, you could use a sync.DoOnce in your main handler to perform any actions required based on the value of an environment variable since it will be properly set by that time.
Example of achieving this with Once.Do():
var once = sync.Once{}
func MainHandler(w http.ResponseWriter, r *http.Request) {
once.Do(mysetup)
// do your regular stuff here
}
func mysetup() {
// This function is executed only once. Read / use env vars here.
var1 := os.Getenv("ENVIRONMENT_VAR1")
_ = var1 // use var1
}
The answer icza provided worked for me. In addition, I put the once.Do() call in a handler for the /_ah/start method so that it would be called immediately when GAE started up my application.
Is there a way to set all public links to have 'no-cache' in Google Cloud Storage?
I've seen solutions to use gsutil to set the "Cache-Control" upon file-upload, but I'm looking for a more permanent solution.
There was a conversation about providing a cache invalidation feature but I didn't quite follow the reasoning. Any explanations would be greatly appreciated!
it would be difficult to provide a cache invalidation feature because once served with a non-0 cache TTL any cache on the Internet (not just those under Google's control) is allowed (per HTTP spec) to cache the data
Thanks!
For a more permanent one-time-effort solution, with the current offerings on GCP, you can do this with Cloud Functions.
Create a new Funciton, set the Event type to "On (finalizing/creating) file in the selected bucket" - google.storage.object.finalize. Make sure to select the bucket you want this on. In the body of the function, set the cacheControl / Cache-Control attribute for the blob. The attribute name depends on the language. Here's my version in Python, using cache_control:
main.py:
match the function name below to the Entry point
from google.cloud import storage
def set_file_uncached(event, context):
file = event # auto-generated
print(f"Processing file: {file=}") # logging, if you want it
storage_client = storage.Client()
# we expect just one with that name
blob = storage_client.bucket(file["bucket"]).get_blob(file["name"])
if not blob:
# in case the blob is deleted before this executes
print(f"blob not found")
return None
blob.cache_control = "public, max-age=0" # or whatever you need
blob.patch()
requirements.txt
google-cloud-storage
From the logs: Function execution took 1712 ms, finished with status: 'ok'. This could have been faster but I've set the minimum to 0 instances so it needs to spin-up for each upload. Depending on your usage and cost constraints, you can set it to 1 or something higher.
Other settings:
Retry on failure: No/False
Region: [wherever your bucket is]
Memory allocated: 128 MB (smallest available currently)
Timeout: 5 seconds (smallest available currently, function shouldn't take longer)
Minimum instances: 0
Maximum instances: 1
I have a following problem. I have defined a cron job in Google App Engine, but my get method is not called (or to be precise it is called every other time - if I run it manually it doesn't do anything at the first time, but at the second it works flawlessly). This is output from logging for the call made by cron:
2011-07-04 11:39:08.500 /suggestions/ 200 489ms 70cpu_ms 0kb AppEngine-Google; (+http://code.google.com/appengine)
0.1.0.1 - - [04/Jul/2011:11:39:08 -0700] "GET /suggestions/ HTTP/1.1" 200 0 - "AppEngine-Google; (+http://code.google.com/appengine)" "bazinga-match.appspot.com" ms=489 cpu_ms=70 api_cpu_ms=0 cpm_usd=0.001975 queue_name=__cron task_name=a449e27ff383de24ff8fc5d5f05f2aae
As you can see it makes GET request on /suggestions/, but nothing happens, including my log messages (they are printed, when I run it second time manually). Do you have any idea, why this might be happening?
My handler:
class SuggestionsHandler(RequestHandler):
def get(self):
logging.debug('Creating suggestions')
for key in db.Query(User, keys_only=True).order('last_suggestion'):
make_suggestion(key)
logging.debug('Done creating suggestions')
print
print('Done creating suggestions')
This is my cron.yaml:
cron:
- description: daily suggestion creation
url: /suggestions/
schedule: every 6 hours
and proper section of my app.yaml:
- url: /suggestions/
script: cron.py
login: admin
You're missing this declaration from the bottom of your handler file, after main:
if __name__ == '__main__':
main()
The first time a handler script is run, App Engine simply imports it, and this snippet runs your main in that situation. On subsequent requests, App Engine runs the main function, if you defined one.