I'm using RemoteAPI to fetch entities from GAE Datastore, 300 at a time.
I'm doing something along the lines of:
while(!(emails = getEmails()).isEmpty()) {
Filter filter = new FilterPredicate("email", FilterOperator.IN, emails)
Query query = new Query("MyEntity").setFilter(filter);
QueryResultIterable<Entity> result = ds.prepare(query).asQueryResultIterable();
for (Entity entity : result) {
System.out.println(entity.getProperty("name"));
}
}
I'm processing something like 50k emails. The first time I ran this code it got to maybe 3/4 of the way, then it threw the following exception. Now it throws it after a single loop iteration is run.
com.google.appengine.tools.remoteapi.RemoteApiException: remote API call: I/O error
at com.google.appengine.tools.remoteapi.RemoteRpc.makeException(RemoteRpc.java:160)
at com.google.appengine.tools.remoteapi.RemoteRpc.callImpl(RemoteRpc.java:104)
at com.google.appengine.tools.remoteapi.RemoteRpc.call(RemoteRpc.java:50)
at com.google.appengine.tools.remoteapi.RemoteDatastore.runQuery(RemoteDatastore.java:156)
at com.google.appengine.tools.remoteapi.RemoteDatastore.handleRunQuery(RemoteDatastore.java:115)
at com.google.appengine.tools.remoteapi.RemoteDatastore.handleDatastoreCall(RemoteDatastore.java:93)
at com.google.appengine.tools.remoteapi.RemoteApiDelegate.makeDefaultSyncCall(RemoteApiDelegate.java:57)
at com.google.appengine.tools.remoteapi.StandaloneRemoteApiDelegate.makeSyncCall(StandaloneRemoteApiDelegate.java:47)
at com.google.appengine.tools.remoteapi.StandaloneRemoteApiDelegate$1.call(StandaloneRemoteApiDelegate.java:58)
at com.google.appengine.tools.remoteapi.StandaloneRemoteApiDelegate$1.call(StandaloneRemoteApiDelegate.java:54)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:152)
at java.net.SocketInputStream.read(SocketInputStream.java:122)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
at sun.security.ssl.InputRecord.read(InputRecord.java:480)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:934)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:891)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:275)
at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:690)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:633)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1324)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:468)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:338)
at com.google.appengine.repackaged.com.google.api.client.http.javanet.NetHttpResponse.<init>(NetHttpResponse.java:37)
at com.google.appengine.repackaged.com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:94)
at com.google.appengine.repackaged.com.google.api.client.http.HttpRequest.execute(HttpRequest.java:972)
at com.google.appengine.tools.remoteapi.OAuthClient.post(OAuthClient.java:54)
at com.google.appengine.tools.remoteapi.RemoteRpc.callImpl(RemoteRpc.java:102)
... 12 more
I can't figure out what the problem is, but the code seems to be evaluating the for() condition before throwing the exception.
Could this be a quota problem? The quota details screen doesn't show any problems and I can't find any relevant information in the documentation.
For future readers of this question, if you see occurrences of RemoteApiException: remote API call: I/O error which are happening consistently and not intermittently, this could be related to a disruption in network connectivity or possibly a remote issue on the App Engine side.
If the first possibility is ruled out, the best course of action is to report the issue on the Google App Engine issue tracker.
To fix this, first, check your Internet connection. Then clean all artifacts and build them again by (with IntelliJ):
Go to Build => Build Artifacts...
Focus on All Artifacts => Clean
Focus on All Artifacts => Build
Related
I recently got myself a new PC(Predator Helios 300) and I wanted to start using aws there but when I try to perform amplify init I get the error below even though I already did all the other steps such as configuration.
× Root stack creation failed
init failed
{ SignatureDoesNotMatch: Signature expired: 20190427T235724Z is now earlier than 20190428T094952Z (20190428T095452Z - 5 min.)
at Request.extractError (C:\Users\sahve\AppData\Roaming\npm\node_modules\#aws-amplify\cli\node_modules\aws-sdk\lib\protocol\query.js:50:29)
at Request.callListeners (C:\Users\sahve\AppData\Roaming\npm\node_modules\#aws-amplify\cli\node_modules\aws-sdk\lib\sequential_executor.js:106:20)
at Request.emit (C:\Users\sahve\AppData\Roaming\npm\node_modules\#aws-amplify\cli\node_modules\aws-sdk\lib\sequential_executor.js:78:10)
at Request.emit (C:\Users\sahve\AppData\Roaming\npm\node_modules\#aws-amplify\cli\node_modules\aws-sdk\lib\request.js:683:14)
at Request.transition (C:\Users\sahve\AppData\Roaming\npm\node_modules\#aws-amplify\cli\node_modules\aws-sdk\lib\request.js:22:10)
at AcceptorStateMachine.runTo (C:\Users\sahve\AppData\Roaming\npm\node_modules\#aws-amplify\cli\node_modules\aws-sdk\lib\state_machine.js:14:12)
at C:\Users\sahve\AppData\Roaming\npm\node_modules\#aws-amplify\cli\node_modules\aws-sdk\lib\state_machine.js:26:10
at Request.<anonymous> (C:\Users\sahve\AppData\Roaming\npm\node_modules\#aws-amplify\cli\node_modules\aws-sdk\lib\request.js:38:9)
at Request.<anonymous> (C:\Users\sahve\AppData\Roaming\npm\node_modules\#aws-amplify\cli\node_modules\aws-sdk\lib\request.js:685:12)
at Request.callListeners (C:\Users\sahve\AppData\Roaming\npm\node_modules\#aws-amplify\cli\node_modules\aws-sdk\lib\sequential_executor.js:116:18)
message:
'Signature expired: 20190427T235724Z is now earlier than 20190428T094952Z (20190428T095452Z - 5 min.)',
code: 'SignatureDoesNotMatch',
time: 2019-04-27T23:57:24.753Z,
requestId: 'ab179ef3-699b-11e9-bfe3-4ddc7ceb66ee',
statusCode: 403,
retryable: true }
After doing some research It seems to be a verification problem. Does anyone has experience with this or knows how to resolve this issue. Thanks a lot!
Any time you see an error like "is now earlier than" around some numbers that look like timestamps (20190427T235724Z -> 2019-04-27 23:57:24 UTC), that's an indicator that the error is time related. Time matters for cryptography in order to validate certificates (so that an attacker cannot break a certificate and use it after its expiration, among other reasons) [1]. In this case, either your clock or the remote server clock is wrongly set. Since the remote server in this case is AWS, it is highly unlikely that they have any significant clock drift, leaving you as the possible outlier.
Given that you mentioned a new computer, it is even more likely that this is due to an incorrectly set system clock.
Reset/synchronize your system clock and the error should disappear.
Reference [1]: https://security.stackexchange.com/q/72866/47422
How can I use ClusterClient to check if a job failed and why?
ClusterClient#getJobStatus may seem like a good first candidate but it only says if the job failed without any information regarding the exceptions.
The submission of the job is being done with a detached client therefore waiting for its ClusterClient#run to return a JobExecutionResult is not an option.
I've also tried
RestClusterClient#retrieveJob also does not work, failing with:
org.apache.flink.runtime.client.JobRetrievalException: Couldn't
retrieve leading JobManager. at
org.apache.flink.runtime.client.JobListeningContext.getJobManager(JobListeningContext.java:157)
at
org.apache.flink.runtime.client.JobListeningContext.getClassLoader(JobListeningContext.java:141)
at
org.apache.flink.runtime.client.JobClient.awaitJobResult(JobClient.java:262)
at
org.apache.flink.client.program.ClusterClient.retrieveJob(ClusterClient.java:586)
at java.lang.Thread.run(Thread.java:745)
Caused by:
org.apache.flink.runtime.leaderretrieval.LeaderRetrievalException:
Could not retrieve the leader gateway. at
org.apache.flink.runtime.util.LeaderRetrievalUtils.retrieveLeaderGateway(LeaderRetrievalUtils.java:82)
at
org.apache.flink.runtime.client.JobListeningContext.getJobManager(JobListeningContext.java:152)
... 10 more
Caused by: java.util.concurrent.TimeoutException: Futures
timed out after [10000 milliseconds] at
scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:223)
at
scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:227)
at scala.concurrent.Await$$anonfun$result$1.apply(package.scala:190)
at
scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
at scala.concurrent.Await$.result(package.scala:190) at
scala.concurrent.Await.result(package.scala) at
org.apache.flink.runtime.util.LeaderRetrievalUtils.retrieveLeaderGateway(LeaderRetrievalUtils.java:80)
... 11 more
Use NewClusterClient#requestJobResult which can be done using a RestClusterClient.
I have an app engine java project and am using objectify. I get a stack trace sporadically in the "stack driver error reporting" view of the app engine web console related to putting an item into memcache. This is the code:
try {
TestItem t = new TestItem(...);
ofy().save().entity(t).now();
} catch (Exception e) {
}
and this is the error I'll see sporadically:
com.googlecode.objectify.cache.MemcacheServiceRetryProxy invoke: Memcache operation failed, giving up
java.lang.reflect.InvocationTargetException
at com.google.appengine.runtime.Request.process-i4dx9s2kED3CVcPe(Request.java)
at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:44)
at com.googlecode.objectify.cache.MemcacheServiceRetryProxy.invoke(MemcacheServiceRetryProxy.java:68)
at com.sun.proxy.$Proxy9.putAll(Unknown Source)
at com.googlecode.objectify.cache.KeyMemcacheService.putAll(KeyMemcacheService.java:91)
at com.googlecode.objectify.cache.EntityMemcache.empty(EntityMemcache.java:319)
at com.googlecode.objectify.cache.CachingAsyncDatastoreService$5.trigger(CachingAsyncDatastoreService.java:445)
at com.googlecode.objectify.cache.TriggerFuture.isDone(TriggerFuture.java:87)
at com.googlecode.objectify.cache.TriggerFuture.get(TriggerFuture.java:102)
at com.googlecode.objectify.impl.ResultAdapter.now(ResultAdapter.java:34)
at com.googlecode.objectify.util.ResultWrapper.translate(ResultWrapper.java:22)
at com.googlecode.objectify.util.ResultWrapper.translate(ResultWrapper.java:10)
at com.googlecode.objectify.util.ResultTranslator.nowUncached(ResultTranslator.java:21)
at com.googlecode.objectify.util.ResultCache.now(ResultCache.java:30)
at com.googlecode.objectify.util.ResultWrapper.translate(ResultWrapper.java:22)
at com.googlecode.objectify.util.ResultWrapper.translate(ResultWrapper.java:10)
at com.googlecode.objectify.util.ResultTranslator.nowUncached(ResultTranslator.java:21)
at com.googlecode.objectify.util.ResultCache.now(ResultCache.java:30)
at com.me.test.Test.putSomethinInMemcache(Test.java:13)
...
Caused by: com.google.appengine.api.memcache.MemcacheServiceException: Memcache putAll: Unknown exception setting 1 keys
at com.google.appengine.api.memcache.MemcacheServiceApiHelper$RpcResponseHandler.handleApiProxyException(MemcacheServiceApiHelper.java:69)
at com.google.appengine.api.memcache.AsyncMemcacheServiceImpl$RpcResponseHandlerForPut.handleApiProxyException(AsyncMemcacheServiceImpl.java:349)
at com.google.appengine.api.memcache.MemcacheServiceApiHelper$1.absorbParentException(MemcacheServiceApiHelper.java:111)
at com.google.appengine.api.utils.FutureWrapper.handleParentException(FutureWrapper.java:52)
at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:91)
at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:89)
at com.google.appengine.api.memcache.MemcacheServiceImpl.quietGet(MemcacheServiceImpl.java:26)
at com.google.appengine.api.memcache.MemcacheServiceImpl.putAll(MemcacheServiceImpl.java:115)
... 52 more
It doesn't appear to be caught in the try-statement. I just see it in that admin console mentioned earlier.
Does anyone know what this means, or how I can catch it? My main worry is that there could be an old copy of the object stuck in memcache after this operation fails.
Using objectify 5.1.10.
Thanks
This is a get() operation. If memcache is unavailable during a get() operation, Objectify just reads from the datastore. The error is logged and performance suffers somewhat but the app marches on.
It is technically possible to have errors during write operations (any save() clears the cache entry; reads will repopulate the cache). This could in theory leave stale info in the cache. There's nothing that can really be done about this - if you can't clear the cache entry, it's going to be stuck there. My advice is that if you have sensitive data but want it cached, put a reasonable timeout on the cache entry (#Cache(expirationSeconds=60) or whatnot).
Hello everyone and thanks up front for your time,
I am working on a java-based GAE web application and now and then I get ApiProxy.ApplicationExceptions.
In the current case they appear randomly and come with the applicationError 108 when I open a write channel to a blob using the (yes I know, still experimental) FileStore API. Although the API is still in an experimental state, I'd like to handle the thrown exception correctly. Thus my question:
Where can I find a list of possible application errors including their descriptions?
As of right now it is not possible for me to figure out where the problem resides since the thrown exception does not contain something like a message, hint or reason phrase but only the error ID 108:
Caused by: com.google.apphosting.api.ApiProxy$ApplicationException: ApplicationError: 108:
at java.lang.Thread.getStackTrace(Thread.java:1495)
at com.google.apphosting.runtime.ApiProxyImpl.doSyncCall(ApiProxyImpl.java:240)
at com.google.apphosting.runtime.ApiProxyImpl.access$000(ApiProxyImpl.java:66)
at com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:183)
at com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:180)
at java.security.AccessController.doPrivileged(Native Method)
at com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java:180)
at com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java:66)
at com.googlecode.objectify.cache.TriggerFutureHook.makeSyncCall(TriggerFutureHook.java:154)
at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:107)
at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:56)
at com.google.appengine.api.files.FileServiceImpl.makeSyncCall(FileServiceImpl.java:584)
... 65 more
Also, the corresponding javadoc is quite conservative with giving information: https://developers.google.com/appengine/docs/java/javadoc/com/google/apphosting/api/ApiProxy.ApplicationException
Currently I bluntly cancel these requests with a 500, but since I am not sure what has happened I should probably do something else/more.
Thanksalot!
the best information I could get is from the Python source code :
http://code.google.com/p/googleappengine/source/browse/trunk/python/google/appengine/api/files/file_service_pb.py
Every now and then I come across this error. I usually end up cleaning all the database. what could be the cause?
This is on Win XP SP3, GAE 1.6.5, Objectify 3.0, GWT 2.4.
QueryResultIterator<Key<x>> it = query.fetchKeys().iterator();
ArrayList<Key<x>> keys = new ArrayList<Key<x>> (num);
while (it != null && it.hasNext()) { <== error
keys.add(it.next());
}
Here is the error -
Caused by: java.lang.NullPointerException
at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:90)
at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:90)
at com.google.appengine.api.datastore.FutureHelper.getInternal(FutureHelper.java:72)
at com.google.appengine.api.datastore.FutureHelper.quietGet(FutureHelper.java:33)
at com.google.appengine.api.datastore.QueryResultsSourceImpl.peekQueryResultAndIfFirstRecordIndexList(QueryResultsSourceImpl.java:175)
at com.google.appengine.api.datastore.QueryResultsSourceImpl.loadMoreEntities(QueryResultsSourceImpl.java:111)
at com.google.appengine.api.datastore.QueryResultsSourceImpl.loadMoreEntities(QueryResultsSourceImpl.java:98)
at com.google.appengine.api.datastore.QueryResultIteratorImpl.ensureLoaded(QueryResultIteratorImpl.java:164)
at com.google.appengine.api.datastore.QueryResultIteratorImpl.hasNext(QueryResultIteratorImpl.java:66)
at com.googlecode.objectify.util.TranslatingIterator.hasNext(TranslatingIterator.java:29)
It is a bug in GAEJ. I got this all the time, just wait for a day and it will work. Do not send any hit to the server. I know it is kind of bad. Even worse, I am a paying customer.
I hit the "Report production issue" every time it occurs and pray that someone will look into it.