My instance has little to no traffic but I have a min-idle instance set to 1. What I notice is that whenever there is a random url (via some bot) that doesn't exist is accessed, it is considered a dynamic request since my catch all handler is auto. This is fine, except I see these 404 errors (404 because there are no http handlers associated with these url patterns even though the yaml defines a catch all pattern) resulting in instance restarts. Why should the instance restart if it runs into 404 errors?
I have all my dynamic handlers follow "/api" pattern and then a few that don't. So, I can explicitly list all valid patterns and map them to the auto handler. Would that then consider these random links as static but not present and throw 404 error (which I am fine with)? I want to make sure the instance doesn't keep running just because of some rouge requests.
I just did a local experiment (I don't presently have any quickly deployable play app) and it looks like your quite interesting idea could work.
I replaced the .* pattern previously catching all stragglers and routing them to my default service script (I'm using the python runtime) with specific patterns, then added this handler after all others:
- url: /(.*)$
static_files: images/\1
upload: images/.*
My images directory is real, holding static images (but for which I already have another handler with a more specific pattern).
With this in place I made a request to /crap and got, as expected (there is no images/crap file):
INFO 2019-11-08 03:06:02,463 module.py:861] default: "GET /crap
HTTP/1.1" 404 -
I added logging calls in my script handler's get() and dispatch() calls to confirm they're not actually getting invoked (the development server request logging casts a bit of doubt).
I also checked on an already deployed GAE app that requesting an image that matches a static handler pattern but which doesn't actually exist gets the 404 answer without causing a service's instance to be started (no instance was running at the time), i.e. it comes directly from the GAE's static content CDN.
So I think it's well worth a try with the go runtime, this could save some significant instance time for an app without a lot of activity faced with random bot traffic.
As for the instance restarts, I suspect what you see is just a symptom of your min-idle instance set to 1. Unlike a dynamic instance the idle (aka resident) instance is not normally meant to handle traffic, it's just ready to do it if/when needed. Only when there is no dynamic instance running (and able to handle incoming traffic efficiently) and a new request comes in that request is immediately routed to the idle instance. At that moment:
the idle instance becomes a dynamic one and will continue to serve traffic until it shuts due to inactivity or dies
a fresh idle instance is started to meet the min-idle configuration, it will remain idle until another similar event occurs
Note: your idea will help with the instance hours portion used by the dynamic instances, but not with the idle instance portion.
According to the documentation which quotes the following:
"When an instance responds to the request /_ah/startwith an HTTP status code of 200–299 or 404, it is considered to have started correctly and that it can handle additional requests. Otherwise, App Engine cancels the instance. Instances with manual scale adjustment restart immediately, while instances with basic scale adjustment restart only when necessary to deliver traffic."
You can find more detail about how instances are managed for Standard App Engine environment for Go 1.12 on the link: https://cloud.google.com/appengine/docs/standard/go112/how-instances-are-managed
As well, I recommend you to read the document "How instances are managed", on which quotes the following:
"Secondary routing
If a request matches the part [YOUR_PROJECT_ID].appspot.comof the host name, but includes the name of a service, version, or instance that does not exist, the service is routed default. Secondary routing does not apply to custom domains; requests sent to these domains will show an HTTP status code 404if the hostname is not valid."
https://cloud.google.com/appengine/docs/standard/go112/how-instances-are-managed
Related
I need to make sure some code is initialized on a GAE instance when it is spun up. Is there a way to detect in the code that the request is a "loading request"?
I tried adding the initialization code to my warmup handler but it does not seem that warmup is guaranteed to be called. I also tried giving each service in my application a single idle instance to increase the chance of warmup being called.
Note: this answer is written with the python 2.7 standard environment in mind, I'm not sure if something equivalent would be possible for all other languages/runtimes and how it'd work.
Indeed, the warm-up requests, if configured, are only effective if there is already at least one instance running for the service, which isn't always the case.
You could place the desired init code in the main service file which declares your service's app - that code is only executed when the service app is initialized, which happens exactly once per instance lifetime, regardless of the request being a loading or a warm-up one.
Something along these lines:
import ...
def my_desired_init_function():
pass # replace with what you need to do
my_desired_init_function()
app = webapp2.WSGIApplication(...)
I have a GWT based Java web application deployed to Google App Engine, in which the servelet reads and changes state held in memory. The client code might send requests to change this state and, subsequently, to change or read the same state. So it's important that all requests from the same instance of the client page go to the same instance of the application's version.
Since I don't expect a lot of traffic, I don't mind limiting the maximum number of instances to 1. But I'd like that one instance to exist more or less permanently. (If a user takes more than, say, an hour between requests, I don't mind if their data is lost.)
In detail, the way I'm managing state is that I have a static variable that points to a hash table, the hash table maps strings to states. On the first request from the client, a new unique string is created and a new state and a new entry is made in the hash table. The string is returned in the response. On subsequent requests, the client sends the string so that the servelet can find the state that it needs to mutate or read. I can't keep the state in a database because it is very complex and not at all serializable.
What are the ways to ensure that all requests from a given client instance go to the same server instance?
What are the ways to ensure that all requests from a given client instance go to the same server instance?
There are no ways, by design. If you want to persist state between requests reliably, use the datastore, with memcache as a cache.
Adding: You can also use cookies if your data storage is meager, obfuscating/encrypting them as needed.
App Engine assumes that applications hold no essential state between requests. That makes spinning up/shutting down instances a non-issue.
I second Dave's answer, GAE is not exactly the right fit for what you desire.
However there could be ways around it, but in only a few specific cases: if you're using the standard GAE environment with manual scaling and a subsequent request is always based on URLs embedded in the response to the previous request.
You could craft the URLs in a response to a request according to the targeted routing rules such that subsequent requests hit the same instance. From Targeted routing:
If you are still using backends or have manually-scaled services, you can target and send a request to a instance by
including the instance ID. The instance ID is an integer in the range
from 0 up to the total number of instances that are running, and can
be specified as follows:
Sends a request to a specific service and version within a specific instance:
https://INSTANCE_ID-dot-VERSION_ID-dot-SERVICE_ID-dot-MY_PROJECT_ID.appspot.com
http://INSTANCE_ID.VERSION_ID.SERVICE_ID.MY_CUSTOM_DOMAIN
Note: Targeting an instance is not supported in services that are configured for auto scaling or basic scaling. The instance ID must be
an integer in the range from 0, up to the total number of instances
running. Regardless of your scaling type or instance class, it is not
possible to send a request to a specific instance without targeting a
service or version within that instance.
To determine the instance ID you could use the modules API, for example:
// Get the instance handling the current request.
int currentInstance = modulesApi.getCurrentInstance();
Note that if the targeted instance goes down you will keep getting errors permanently (that instance will not come back), so you might want to think at a fall-back solution for going somehow to a non-instance-based based page from where you could hitch a flow on another instance.
But such solution is not available in the GAE flex environment. From Targeted routing:
Note: In the flexible environment, targeting an instance is not supported. It is not possible to send requests directly to a specific
instance.
I have appengine endpoint and trying to reduce latency on few first calls to newly created endpoint instance. Application is written in Java and endpoints are auto scaled.
To address this issue I configured idle instance, although even if instance is already created, first few calls routed to it consume some extra time. Following documentation I've implemented the custom servlet handling warm up requests and marked the EndpointsServlet as load on startup.
Inside the warm up servlet I've put code that initiates some commonly used services, load some data etc. Effect was almost impossible to notice.
After it I have implemented calls to methods exposed by the endpoint like that:
call("/_ah/api/teamly/v1/test/dummy")
It works for some cases (even most of them) and after calling few key methods instance is really ready to serve. The problem I'm facing now is that if I'm using auto scaling for some module I can't route the request to specific instance.
So the question is:
How should I properly warm up the endpoint instance to avoid load requests initiated from frontend.
You need to put a listener to /_ah/warmup and then make calls to any resources you want it to be warmed up. You can find detailed information at:
Configuring Warmup Requests to Improve Performance
I want to send a particular HTTP request (or otherwise communicate a message) to every (dynamic/autoscaled) instance which is currently running for a particular App Engine application.
My goal is to trigger each instance to discard some locally cached data (because I have just modified the underlying data and want them to reload it).
One possible solution is to store a value in Memcache, and have instances check this each time they handle a request to see if they should flush their cache. But this adds latency to every request.
Another possible solution would be to somehow stop all running instances. No fixed overhead, but some impact while instances are restarted.
An even less desirable solution would be to redeploy the application code in order to cause all instances to be stopped. This now adds additional delay on my end as a deployment takes some time.
You could use the management API to list instances for a given version, but I'd suggest that you'd probably want to use something like the PubSub API to create a subscription on each of your App Engine instances. Since each instance has its own subscription, any messages sent to the monitored queue will be received by all instances.
You can create the subscription at startup (the /_ah/start endpoint may be useful), and then delete it at shutdown (using the /_ah/stop endpoint).
I am calling a third party web service from app engine. This particular service is picky. I ran into an issue where calls would work fine for a while, then stop working, then start working again. I realized that if I manually stopped all instances in the admin console, that the calls would work again.
I setup a proxy to route the calls through that so I could see the headers and all detail. I think I have tracked the issue down to the following. After an instance has been up for a while (the app usually just needs 1 to 3 instances right now) app engine will start using the IP address of the destination as the value for the host header instead of the hostname. Well the service doesn't like that. Whether it should care is another matter.
So my question is, why does app engine use the ip address for the host header eventually instead of the hostname? And, of course, is there anything I can do about it? I know that I cannot set the host header, but maybe there is something else that can be done.
Thanks for any insight.
First, thank you for finding this behavior. We have had intermittent issues with urlfetch for a long time, and will try to detect if this is the issue.
One thing you could try is to target a specific instance/module:
http://instance.version.module.app-id.appspot.com
and cycle through the instances. If you just target the module, it will kill the instance after some inactivity. So, perhaps that would not trigger the GAE DNS shortcut.
Another trick would be to add a fake, random, query string after your url: ?foo=D7hfka67h. Perhaps that would prevent GAE from recognizing the repeat url, and trying to shortcut the DNS.