GAE Flexible Environment Service error in memcache - google-app-engine

Question

I use jetty9-compat + java8 configuration and receive the following exception. Probably because of this I am not able to store objects in session, so in the end I cannot log in to my service.
[INFO] cze 28, 2016 11:17:56 AM com.google.apphosting.vmruntime.VmMetadataCache getMetadata
[INFO]
[INFO] INFO: Meta-data 'attributes/gae_affinity' path retrieval error: metadata
[INFO]
[INFO] cze 28, 2016 11:17:56 AM com.google.apphosting.vmruntime.VmApiProxyDelegate runSyncCall
[INFO]
[INFO] INFO: HTTP ApiProxy I/O error for memcache.Get: The target server failed to respond
[INFO]
[INFO] cze 28, 2016 11:17:56 AM com.google.appengine.api.memcache.LogAndContinueErrorHandler handleServiceError
[INFO]
[INFO] INFO: Service error in memcache
[INFO]
[INFO] com.google.appengine.api.memcache.MemcacheServiceException: RCP Failure for API call: memcache Get
[INFO]
[INFO] at com.google.apphosting.vmruntime.VmApiProxyDelegate.constructApiException(VmApiProxyDelegate.java:232)
[INFO]
[INFO] at com.google.apphosting.vmruntime.VmApiProxyDelegate.runSyncCall(VmApiProxyDelegate.java:195)
[INFO]
[INFO] at com.google.apphosting.vmruntime.VmApiProxyDelegate.makeApiCall(VmApiProxyDelegate.java:154)
[INFO]
[INFO] at com.google.apphosting.vmruntime.VmApiProxyDelegate.access$000(VmApiProxyDelegate.java:60)
[INFO]
[INFO] at com.google.apphosting.vmruntime.VmApiProxyDelegate$MakeSyncCall.call(VmApiProxyDelegate.java:436)
[INFO]
[INFO] at com.google.apphosting.vmruntime.VmApiProxyDelegate$MakeSyncCall.call(VmApiProxyDelegate.java:412)
[INFO]
[INFO] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[INFO]
[INFO] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[INFO]
[INFO] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[INFO]
[INFO] at java.lang.Thread.run(Thread.java:745)
[INFO]
[INFO]
[INFO]
[INFO] DEBUG 2016-06-28 11:17:56,493 api_server.py:277] Handled datastore_v3.Get in 0.0000
[INFO] cze 28, 2016 11:17:59 AM com.mysql.jdbc.log.Slf4JLogger logInfo

Answer 1

This issue is reproduced only locally, I face it too. I assume that it is caused by incorrect configuration of local memcached which is run by dev server.
I'll try to explain my point. In my case first request is always successful.
Because in case of first request Apache HttpClient opens a new connection and puts it into the pool.
There is source code https://github.com/GoogleCloudPlatform/appengine-java-vm-runtime/blob/master/appengine-managed-runtime/src/main/java/com/google/apphosting/vmruntime/VmApiProxyDelegate.java
All further requests are successful too, if they are executed with interval up to 10 seconds. But if there were no request within 10 seconds, next request fails.
It happens because memcached service closes connection from its side, but HttpClient doesn't know about it.
In VmApiProxyDelegate HttpClient is configured to close idle connections after 60 seconds. So, if I wait more than 1 minute instead of 10 seconds, my next request doesn't fail. Because in this case HttpClient opens new connection which is not closed yet.
To avoid this issue it would be correct to configure connection timeout from memcached stub side. But the documentation says that your requests to memcached can fail and you must handle these errors. So, I offer you to add error handler and some retry mechanism.
Of course all of the above fits your case, if everything is ok in your configuration. Taking into account that you did not provide examples of configuration and code, I assume that there everything is ok.