Should Exception catch DeadlineExceededError exceptions? - google-app-engine

Question

I have the code like below:
try:
response = urlfetch.Fetch(url, deadline=60, headers=headers, follow_redirects=False)
except Exception, error_message:
logging.exception('Failed, exception happened - %s' % error_message)
but sometimes it fails with DeadlineExceededError? Should not this DeadlineExceededError be caught by my code?
The exception details:
Traceback (most recent call last):
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 266, in Handle
result = handler(dict(self._environ), self._StartResponse)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1529, in __call__
rv = self.router.dispatch(request, response)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1102, in __call__
return handler.dispatch()
File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "/base/data/home/apps/s~myapp/1.375970700643773844/myapp.py", line 424, in get
url, response = call_api(origin=origin, destination=destination, date=date_.strftime('%Y-%m-%d'))
File "/base/data/home/apps/s~myapp/1.375970700643773844/myapp.py", line 288, in call_api
response = urlfetch.Fetch(url, deadline=60, headers=headers, follow_redirects=False)
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/urlfetch.py", line 270, in fetch
return rpc.get_result()
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 612, in get_result
return self.__get_result_hook(self)
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/urlfetch.py", line 375, in _get_fetch_result
rpc.check_success()
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 585, in check_success
self.__rpc, err)
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 196, in Call
for key, function, srv, num_args in self.__content:
DeadlineExceededError

Answer 1

To answer your specific question, "Should not this DeadlineExceededError be caught by my code?":
The DeadlineExceededError class extends BaseException instead of Exception, so your except clause will never be called.
from google.appengine.runtime import DeadlineExceededError
my_error = DeadlineExceededError()
isinstance(my_error, Exception) # False
isinstance(my_error, BaseException) # True

Answer 2

First approach
Try excepting the DeadlineExceededError class by importing
from google.appengine.runtime import DeadlineExceededError and doing it like this:
try:
response = urlfetch.Fetch(url, deadline=60, headers=headers, follow_redirects=False)
except DeadlineExceededError, error_message:
logging.exception('Failed, exception happened - %s' % error_message)
Read more about it on the documentation.
Second Approach
I've faced this error before, and an approach I've followed was not to set which exception class I was trying to catch. I just called except.
try:
response = urlfetch.Fetch(url, deadline=60, headers=headers, follow_redirects=False)
except:
logging.exception('First failure')
# try again your request as suggested by the documentation
try:
response = urlfetch.Fetch(url, deadline=60, headers=headers, follow_redirects=False)
except:
logging.exception('Second failure')
return None
But I wasn't able to catch the error message, and instead of just logging an error message, I tried the request once again when the exception was raised, as the link I posted above suggests.
Some good links you should read
You should read this about Dealing with DeadlineExceedError.
There is also Backends, which allow you to avoid this request timer; with backends, there is no time limit for generating and returning a request.
Hope this helps you.