Google App Engine time out? - google-app-engine

Question

In a Google App Engine app I used the following lines to read a page from a site :
String Url="http://...",line,Result="";
URL url=new URL(Url+"?r="+System.currentTimeMillis());
BufferedReader reader=new BufferedReader(new InputStreamReader(url.openStream()));
while ((line=reader.readLine())!=null) { Result+=line+"\n"; }
reader.close();
But I got the following error :
Uncaught exception from servlet
com.google.apphosting.api.DeadlineExceededException: This request (f5e2889605d27d42) started at 2011/09/07 03:20:41.458 UTC and was still executing at 2011/09/07 03:21:30.888 UTC.
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1326)
at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:276)
at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:82)
at com.google.appengine.tools.development.TimedFuture.get(TimedFuture.java:55)
at com.google.common.util.concurrent.ForwardingFuture.get(ForwardingFuture.java:69)
at com.google.apphosting.runtime.ApiProxyImpl.doSyncCall(ApiProxyImpl.java:177)
at com.google.apphosting.runtime.ApiProxyImpl.access$000(ApiProxyImpl.java:56)
at com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:150)
at com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:148)
at java.security.AccessController.doPrivileged(Native Method)
Seems it took longer than it would like to wait, what can I do if that site is slow ?

Answer 1

A DeadlineExceededException is thrown when your code, handling the request to your web application takes over 30 seconds to process. Presumably your code is taking a while to process because of the length of time it had to wait to receive data from some other site.
You can create a task on a task queue to fetch and process that data, and change your web request/response flow to reply with progress on your task.

Answer 2

If your code is running inside a request handler, then by default there is an App-Engine-enforced 60 second deadline. You can't change it. See the "Deadlines" row / "automatic scaling" column of the chart on this page under "Scaling types":
https://developers.google.com/appengine/docs/java/modules/
However, this code will be able to run for some hours if you change your module to use "manual scaling" and a "B1"-"B4" instance. Example:
default/src/main/webapp/WEB-INF/appengine-web.xml:
<?xml version="1.0" encoding="utf-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<application>myapp</application>
<module>default</module>
<version>1</version>
<threadsafe>true</threadsafe>
<instance-class>B1</instance-class>
<manual-scaling>
<instances>1</instances>
</manual-scaling>
</appengine-web-app>
On this type of instance, your requests will typically not time out for hours. (The documentation claims that the deadline is "indefinite".)