When splitting traffic by cookies, we're observing the following behavior:
GOOGAPPUID cookie does not receive a number between 0-999, but rather a string which resembles this form: xCgsIARD8AiDF1PjnBQ
Even though the client has the cookie he is not always sent to the same version, every few minutes the version the client is directed to is switched.
How do we remedy this?
So apparently it's a bug from 2015 (Google issue tracker) that has not been fixed yet.
We've implemented a partial solution though:
If you have versions A and B, split 30%/70% then if you set the GOOGAPPUID cookie manually to values between 0-299 you'll get to version A, or to values between 300-999 you'll get to version B, consistently, across requests.
Knowing that, on the relevant handlers of our app (you might need to do it for all handlers of your app), we check if the incoming request already has a GOOGAPPUID cookie AND that it's a number. If it is a number, we leave it as is.
If it's not present or not a number it means that we have not set it before. So we check on which version we are (because the App Engine traffic splitting works well, it's only the persistence cookie mechanism is broken), and if we're on version A, we set the cookie to a value between 0-299, if we're on version B we set the cookie to value of 300-999.
This patch will fill the gap of Google's issue with GOOGAPPUID cookies and will give your user a persistent experience across requests.
Related
I have an app running on Google app engine (Flask, python 3, flexible environment) using the Identity-Aware proxy to allow everyone in our organization (which uses GSuite) to control access. Recently we've been getting 413 errors.
When I looked at the cookies of the failing requests I expected to see one request cookie prefixed with GCP_IAAP_AUTH_TOKEN. Instead I see 11, each one slightly different. Their combined sizes put us over the 15kb header size limit indicated in the link below, causing a 413 error.
https://cloud.google.com/appengine/docs/flexible/go/how-requests-are-handled
I don't understand why there are so many cookies, or how to make them go away. Our users all use Chrome, and many but not all of them are intermittently running into this error. Those that aren't, when their cookies are inspected, show only a couple cookies with this prefix. See below for an example of what this collection of cookies looks like:
Eleven IAP cookies in a single header
Posting what ended up solving this particular instance of the problem in case something like it occurs to other people in the future.
The original IAP code for our project was written in 2018. At the time, IAP had a known issue requiring re-logging in every hour. The suggested workaround from this thread was to use a hidden iframe.
https://issuetracker.google.com/issues/69386592?pli=1
We followed that guidance, but Google fixed the underlying issue in June of 2019. Now, following that guidance causes a gradual accumulation of session cookies in the headers. Removing the no-longer-needed offending iframe code solved the problem.
I have a GWT based Java web application deployed to Google App Engine, in which the servelet reads and changes state held in memory. The client code might send requests to change this state and, subsequently, to change or read the same state. So it's important that all requests from the same instance of the client page go to the same instance of the application's version.
Since I don't expect a lot of traffic, I don't mind limiting the maximum number of instances to 1. But I'd like that one instance to exist more or less permanently. (If a user takes more than, say, an hour between requests, I don't mind if their data is lost.)
In detail, the way I'm managing state is that I have a static variable that points to a hash table, the hash table maps strings to states. On the first request from the client, a new unique string is created and a new state and a new entry is made in the hash table. The string is returned in the response. On subsequent requests, the client sends the string so that the servelet can find the state that it needs to mutate or read. I can't keep the state in a database because it is very complex and not at all serializable.
What are the ways to ensure that all requests from a given client instance go to the same server instance?
What are the ways to ensure that all requests from a given client instance go to the same server instance?
There are no ways, by design. If you want to persist state between requests reliably, use the datastore, with memcache as a cache.
Adding: You can also use cookies if your data storage is meager, obfuscating/encrypting them as needed.
App Engine assumes that applications hold no essential state between requests. That makes spinning up/shutting down instances a non-issue.
I second Dave's answer, GAE is not exactly the right fit for what you desire.
However there could be ways around it, but in only a few specific cases: if you're using the standard GAE environment with manual scaling and a subsequent request is always based on URLs embedded in the response to the previous request.
You could craft the URLs in a response to a request according to the targeted routing rules such that subsequent requests hit the same instance. From Targeted routing:
If you are still using backends or have manually-scaled services, you can target and send a request to a instance by
including the instance ID. The instance ID is an integer in the range
from 0 up to the total number of instances that are running, and can
be specified as follows:
Sends a request to a specific service and version within a specific instance:
https://INSTANCE_ID-dot-VERSION_ID-dot-SERVICE_ID-dot-MY_PROJECT_ID.appspot.com
http://INSTANCE_ID.VERSION_ID.SERVICE_ID.MY_CUSTOM_DOMAIN
Note: Targeting an instance is not supported in services that are configured for auto scaling or basic scaling. The instance ID must be
an integer in the range from 0, up to the total number of instances
running. Regardless of your scaling type or instance class, it is not
possible to send a request to a specific instance without targeting a
service or version within that instance.
To determine the instance ID you could use the modules API, for example:
// Get the instance handling the current request.
int currentInstance = modulesApi.getCurrentInstance();
Note that if the targeted instance goes down you will keep getting errors permanently (that instance will not come back), so you might want to think at a fall-back solution for going somehow to a non-instance-based based page from where you could hitch a flow on another instance.
But such solution is not available in the GAE flex environment. From Targeted routing:
Note: In the flexible environment, targeting an instance is not supported. It is not possible to send requests directly to a specific
instance.
How does automatic updates work in passbook available in backfield of the pass, how to use webserviceURL key to get update from the server.
in webserviceURL key i have provided remote path from where updated pass can be downloaded, but even after content changed in server, it does not reflects in pass.
This question has been asked so many times, in so many ways, in so many forums; mostly by people that are too lazy to read the manual, or are too inexperienced / incompetent to understand it.
The first paragraph of the Passbook Web Service Reference explains why simply adding a link to an updated pass will not work.
A REST-style web service protocol is used to communicate with your server about changes to passes, and to fetch the latest version of a pass when it has changed. The endpoints always begin with the web service URL, as specified in the pass, followed by the protocol version number. For example, a request for the latest version of the pass of type com.apple.pass.example and serial number ABC123 might look like the following:
The Passbook web service is an integral part of the Passbook eco system. Anyone wishing to issue passes that change their content (either in response to a push message or in response to a user requesting fresh content by pulling down on the back of the pass), needs to implement their own Passbook web service.
This entails building a server capable of responding to the following to authenticated requests from each device that has installed your pass.
There are 5 methods that your web service should respond to:
1. Registering a Device to Receive Push Notifications for a Pass
POST request to https://webServiceURL/v1/devices/deviceLibraryIdentifier/registrations/passTypeIdentifier/serialNumber
2. Getting the Serial Numbers for Passes Associated with a Device
GET request to https://webServiceURL/v1/devices/deviceLibraryIdentifier/registrations/passTypeIdentifier?passesUpdatedSince=tag
3. Getting the Latest Version of a Pass
GET request to https://webServiceURL/v1/passes/passTypeIdentifier/serialNumber
4. Unregistering a Device
DELETE request to https://webServiceURL/v1/devices/deviceLibraryIdentifier/registrations/passTypeIdentifier/serialNumber
5. Logging Errors
POST request to https://webServiceURL/v1/log
In order to have a pass respond to a manual refresh or push request you need to implement at least the first 3 methods. This is because the device will not issue any further requests until it has received a valid response to the registration request.
Furthermore, the web service must be available over https. An unsecured http service can be used for development, but production devices will only recognise a webServiceURL key that begins with https.
I'm using cookies so that mobile users can visit my site as desktop users. To do this, I give them a cookie - mob_yes.
Then, in a module, i use a drupal hook to see if the cookie is set.
I can see that the cookie IS getting set, but in my module (isset($_COOKIE["mob_yes"])) always returns false when using varnish.
In /etc/varnish/default.vlc I have the following:
if (req.http.Cookie) {
set req.http.Cookie = regsuball(req.http.Cookie, ";(mob_yes)=", "; \1=");
I'm really not sure what's going on here, but I only presume varnish is not unsetting that cookie temporarily? Does anyone have any idea what's going wrong here?
Thanks,
what do you mean by
I can see that the cookie IS getting set
you mean that you can see it in headers in firebug (client side) and then you see it on the server side with tcpdump / varnishlog / application (server side)?
code snippet from vcl is probably part of commonly used way of preserving important cookies by adding a space in front of them, deleting all that dont have ";[space]" combination and removing space at the end.
It is used later on to generate hash for specific url+cookies request.
i think you should check your vcl if its not removing any cookies if user is not logged in - it's a common practice to increase hitrate.
usually in vcl for drupal it's part which checks for DRUPAL_UID
In order to test some stuff, I've been using the trick of setting a cookie on the .appspot.com domain, which allows me to share data between two different .appspot.com GAE apps. This has stopped working in Firefox 6, and a google person told me:
It's because appspot.com was added to the public suffix list of
domains that modern browsers should not allow cookies to be set for:
http://mxr.mozilla.org/mozilla-central/source/netwerk/dns/effective_tld_names.dat?raw=1
which totally makes sense for regular users. But for my testing, I'd really like to have FF treat .appspot.com as a regular domain, not an "effective TLD". Is there any way to "whitelist" .appspot.com or otherwise override the effective_tld_names.dat used within FF?
(I need a hack that will work on OS X, in case that matters.)
Unfortunately for you, there is no supported way to do this. I checked the implementation of nsCookieService and method SetCookieStringInternal will reject any host names where it cannot get the base name from (which would be the case for "appspot.com" now - for the Firefox engine this is no longer a complete host name). This means that not even extensions can set a cookie for "appspot.com" other than by writing to the database "manually". But that doesn't help much here because method GetCookiesFromHost also uses the base domain for its checks so that any cookies for "appspot.com" will be ignored.
Given that effective_tld_names.dat is compiled into xul.dll the only way to change this data is using a hex editor on this file and for example replacing "appspot.com" by "bppspot.com" which should "disable" this rule. Note that a manipulation like this will break incremental updates and you will have to repeat it after the update.