I'd like to catch and handle DeadlineExceededError so users don't see the standard "Server Error" page that App Engine throws by default.
I know that DeadlineExceededErrors are not caught when overriding handle_exception in your request handler (we already do this).
I have tried, unsuccessfully so far, to use the custom error_handlers app.yaml configuration like so:
error_handlers:
- error_code: timeout
file: timeout.html
...but that also doesn't seem to catch DeadlineExceededErrors, unless I'm doing something wrong.
I am aware that I can use the following pattern to catch DeadlineExceededErrors inside particular request handlers:
class MainPage(webapp.RequestHandler):
def get(self):
try:
# Do stuff...
except DeadlineExceededError:
# Many Whelps! Handle it!
...but I would like to avoid adding this to every single request handler in my application.
How can I globally catch these elusive suckers?
One possible solution is to use webapp2, which is a pretty neat framework as it is and has a lot of useful stuff over the original webapp. With webapp2, you can handle the exception in the handle_500 method, as follows:
def BaseHandler(webapp2.RequestHandler):
def handle_500(request, response, exception):
if isinstance(exception, DeadlineExceededError):
response.write('Deadline exceeded!')
else:
response.write('A server error occurred!')
logging.exception(exception)
response.set_status(500)
Related
When I look at the logs in the Google Log Viewer for my GAE project, I see that often the logs that I write myself in the code are assigned to the wrong request. Most of the time the log is assigned to the request directly after the request that produced the log entry.
As the root of every application log in GAE must be a request, this means that the wrong request is sometimes marked as error, because another request before produced an error, but the log is somehow assigned to the request after that.
I don't really do anything special, I use Ktor as my servlet and have an interceptor that creates a log when an exception occurs before returning status 500.
I use Java logging via SLF4J with the google cloud logging handler, but before that I used logback via SLf4J and had the same problem.
The content of the logs itself is also correct, the returned status of the request, the level of the log entry, the message, everything is ok.
I thought that it may be because I use kotlin and switch coroutine contexts during a single request, but in some cases the point where I write the log and where I send the response are exactly next to each other, so I'm not sure if kotlin has anything to do with it.
My logging.properties:
# To use this configuration, add to system properties : -Djava.util.logging.config.file="/path/to/file"
#
.level = INFO
# it is recommended that io.grpc and sun.net logging level is kept at INFO level,
# as both these packages are used by Stackdriver internals and can result in verbose / initialization problems.
io.grpc.netty.level=INFO
sun.net.level=INFO
handlers=com.google.cloud.logging.LoggingHandler
# default : java.log
com.google.cloud.logging.LoggingHandler.log=custom_log
# default : INFO
com.google.cloud.logging.LoggingHandler.level=INFO
# default : ERROR
com.google.cloud.logging.LoggingHandler.flushLevel=WARNING
# default : auto-detected, fallback "global"
#com.google.cloud.logging.LoggingHandler.resourceType=container
# custom formatter
com.google.cloud.logging.LoggingHandler.formatter=java.util.logging.SimpleFormatter
java.util.logging.SimpleFormatter.format=%1$tY-%1$tm-%1$td %1$tH:%1$tM:%1$tS %4$-6s %2$s %5$s%6$s%n
#optional enhancers (to add additional fields, labels)
#com.google.cloud.logging.LoggingHandler.enhancers=com.example.logging.jul.enhancers.ExampleEnhancer
My logging relevant dependencies:
implementation "org.slf4j:slf4j-jdk14:1.7.30"
implementation "com.google.cloud:google-cloud-logging:1.100.0"
An example logging call:
exception<Throwable> { e ->
logger().error("Error", e)
call.respondText(e.message ?: "", ContentType.Text.Plain, HttpStatusCode.InternalServerError)
}
with logger() being:
import org.slf4j.Logger
import org.slf4j.LoggerFactory
inline fun <reified T : Any> T.logger(): Logger = LoggerFactory.getLogger(T::class.java)
Edit:
An example of the log in Google cloud. The first request has the query parameter GAID=cdda802e-fb9c-47ad-0794d394c913, but as you can see the error log for that request is in the one below, marked in red.
I've got some Objectify test code running in JUnit and I'm getting this error:
java.lang.IllegalStateException: You have not started an Objectify context. You are probably missing the ObjectifyFilter. If you are not running in the context of an http request, see the ObjectifyService.run() method.
at com.googlecode.objectify.ObjectifyService.ofy(ObjectifyService.java:44)
at com.googlecode.objectify.impl.ref.LiveRef.<init>(LiveRef.java:31)
at com.googlecode.objectify.Ref.create(Ref.java:26)
at com.googlecode.objectify.Ref.create(Ref.java:32)
at com.netbase.followerdownloader.repository.DownloadTaskRepositoryImpl.create(DownloadTaskRepositoryImpl.java:35)
at com.netbase.followerdownloader.repository.DownloadTaskRepositoryImplTest.setUp(DownloadTaskRepositoryImplTest.java:45)
How do I resolve this for test code?
Jeff Schnitzer answered this here: https://groups.google.com/forum/#!topic/objectify-appengine/8HinahG7irg. That link points to https://groups.google.com/forum/#!topic/objectify-appengine/O4FHC_i7EGk where Jeff suggests the following quick and dirty workaround:
My #BeforeMethod starts an objectify context (ObjectifyService.begin())
My #AfterMethod closes the objectify context
Jeff suggests we use ObjectifyService.run() instead but admits it's more work.
Here's how my implementation looks:
public class DownloadTaskRepositoryImplTest {
// maximum eventual consistency (see https://cloud.google.com/appengine/docs/java/tools/localunittesting)
private final LocalServiceTestHelper helper =
new LocalServiceTestHelper(new LocalDatastoreServiceTestConfig()
.setDefaultHighRepJobPolicyUnappliedJobPercentage(100));
private Closeable closeable;
#Before
public void setUp() {
helper.setUp();
ObjectifyRegistrar.registerDataModel();
closeable = ObjectifyService.begin();
}
#After
public void tearDown() {
closeable.close();
helper.tearDown();
}
I also had this issue and noticed that I had not added the ObjectifyFilter to my web.xml
<filter>
<filter-name>ObjectifyFilter</filter-name>
<filter-class>com.googlecode.objectify.ObjectifyFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>ObjectifyFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
I also had to include Objectify and guava jars in my WEB-INF>lib directory and include them in my build path.
I was facing the same error and this solusion worked for me
I have an app based on Endpoints that uses Objectify. When I leave it with the default/automatic scaling, everything works great. Once I enable basic scaling, though, I get the following exception when executing the endpoint method:
[INFO] java.lang.IllegalStateException: You have not started an Objectify context. You are probably missing the ObjectifyFilter. If you are not running in the context of an http request, see the ObjectifyService.run() method.
[INFO] at com.googlecode.objectify.ObjectifyService.ofy(ObjectifyService.java:44)
[INFO] at com.myco.myapp.dao.datastore.OfyService.ofy(OfyService.java:62)
The good news is that this goes away when you enable RequestDispatcher
support in the web.xml file like so. I think this is a documentation
issue, then, but I didn't know if everyone would agree if I edited the
Wiki page directly. Here is the proposed web.xml entry, which worked
for me:
<filter>
<filter-name>ObjectifyFilter</filter-name>
<filter-class>com.googlecode.objectify.ObjectifyFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>ObjectifyFilter</filter-name>
<url-pattern>/*</url-pattern>
<dispatcher>REQUEST</dispatcher>
<dispatcher>INCLUDE</dispatcher>
<dispatcher>FORWARD</dispatcher>
</filter-mapping>
Improving michael-osofsky answer, I add this to my ofy helper class
public static void registerDataModel() {
try {
factory().register(Profile.class);
} catch (Exception e){
e.printStackTrace();
}
}
and remplace
ObjectifyRegistrar.registerDataModel();
for this
OfyService.registerDataModel();
OfyService.java
public static void registerDataModel() {
try {
factory().register(Profile.class);
} catch (Exception e){
e.printStackTrace();
}
}
As Jeff Schnitzer says in the link provided by Michael Osofsky:
In your tests you should have some notion of a 'request' even if it is just conceptual. If "each test is a request by itself", then you can use #Before/#After in conjunction with ObjectifyService.begin() to demarcate the requests. However, this is probably not actually how your tests work - it isn't how my tests work.
He then goes on to say:
This would be prettier with JDK8 closures but the idea is straightforward - you're wrapping some unit of work in a context which represents a request. It would probably be smart to add even more context like authentication in that wrapper too.
I came up with the following implementation of his idea. With the solution below, you can ensure each call to a servlet handler gets a fresh Objectify session while still making your servlet handler calls in a single line of code. It also decouples your tests from explicitly worrying about Objectify, and allows you to add additional non-Objectify context around your servlet handlers.
My solution below works with Objectify 5.1.22. I tried using Objectify 6+, but I had problems that seem to be related to this.
First, define a custom Supplier that is able to capture the exceptions thrown by a servlet handler.
#FunctionalInterface
public interface ServletSupplier<T> {
T get()
throws ServletException, IOException;
}
Next, define a wrapper method that accepts your new custom Supplier as an input, and wrap the call to ServletSupplier.get() in a try-with-resources block that calls ObjectifyService.begin(). You must also register your entity classes before calling ServletSupplier.get().
public <T> T runInServletContext(ServletSupplier<T> servletMethod)
throws ServletException, IOException {
try (Closeable session = ObjectifyService.begin()) {
ObjectifyService.register(MyObj.class);
return servletMethod.get();
}
}
Finally, anywhere in your tests that you call the servlet handler you should do so using the wrapper method.
MyObj myObjPost = runInServletContext(() -> getServlet().doPost(request, response));
// Assert results of doPost call.
MyObj myObjGet = runInServletContext(() -> getServlet().doGet(request, response));
// Assert results of doGet call.
Just in case someone ends up here (as I originally did) looking up the same problem but for the ktor "main.kt" server instead of unit tests...
After looking at the ObjectifyFilter source code, I added
val closer = ObjectifyService.begin()
... real service here ...
closer.close()
around my actual servlet code and that fixed the problem.
Can error_handler be set for a blueprint?
#blueprint.errorhandler(404)
def page_not_found(error):
return 'This page does not exist', 404
edit:
https://github.com/mitsuhiko/flask/blob/18413ed1bf08261acf6d40f8ba65a98ae586bb29/flask/blueprints.py
you can specify an app wide and a blueprint local error_handler
You can use Blueprint.app_errorhandler method like this:
bp = Blueprint('errors', __name__)
#bp.app_errorhandler(404)
def handle_404(err):
return render_template('404.html'), 404
#bp.app_errorhandler(500)
def handle_500(err):
return render_template('500.html'), 500
errorhandler is a method inherited from Flask, not Blueprint.
If you are using Blueprint, the equivalent is app_errorhandler.
The documentation suggests the following approach:
def app_errorhandler(self, code):
"""Like :meth:`Flask.errorhandler` but for a blueprint. This
handler is used for all requests, even if outside of the blueprint.
"""
Therefore, this should work:
from flask import Blueprint, render_template
USER = Blueprint('user', __name__)
#USER.app_errorhandler(404)
def page_not_found(e):
""" Return error 404 """
return render_template('404.html'), 404
On the other hand, while the approach below did not raise any error for me, it didn't work:
from flask import Blueprint, render_template
USER = Blueprint('user', __name__)
#USER.errorhandler(404)
def page_not_found(e):
""" Return error 404 """
return render_template('404.html'), 404
add error handling at application level using the request proxy object:
from flask import request,jsonify
#app.errorhandler(404)
#app.errorhandler(405)
def _handle_api_error(ex):
if request.path.startswith('/api/'):
return jsonify(ex)
else:
return ex
flask Documentation
I too couldn't get the top rated answer to work, but here's a workaround.
You can use a catch-all at the end of your Blueprint, not sure how robust/recommended it is, but it does work. You could also add different error messages for different methods too.
#blueprint.route('/<path:path>')
def page_not_found(path):
return "Custom failure message"
Surprised others didn't mention miguelgrinberg's excellent tutorial.
https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-vii-error-handling
I found the sentry framework for error handling (links below). Seems overly complex. not sure of the threshold where it becomes useful.
https://flask.palletsprojects.com/en/1.1.x/errorhandling/
https://docs.sentry.io/platforms/python/guides/flask/
I combined previous excellent answers with the official docs from Flask, section 'Returning API Errors as JSON', in order to provide a more general approach.
Here is a working PoC that you can copy and paste on your registered blueprint API route handler (e.g. app/api/routes.py):
#blueprint.app_errorhandler(HTTPException)
def handle_exception(e):
"""Return JSON instead of HTML for HTTP errors."""
# start with the correct headers and status code from the error
response = e.get_response()
# replace the body with JSON
response.data = json.dumps({
"code": e.code,
"name": e.name,
"description": e.description,
})
response.content_type = "application/json"
return response
Flask doesnt support blueprint level error handlers for 404 and 500 errors. A BluePrint is a leaky abstraction. Its better to use a new WSGI App for this, if you need separate error handlers, this makes more sense.
Also i would recommend not to use flask, it uses globals all over the places, which makes your code difficult to manage if it grows bigger.
I'm using the webapp2 framework in Google App Engine (Python). In webapp2 exception handling: exceptions in the WSGI app it's described how to handle 404 errors in a function:
import logging
import webapp2
def handle_404(request, response, exception):
logging.exception(exception)
response.write('Oops! I could swear this page was here!')
response.set_status(404)
def handle_500(request, response, exception):
logging.exception(exception)
response.write('A server error occurred!')
response.set_status(500)
app = webapp2.WSGIApplication([
webapp2.Route('/', handler='handlers.HomeHandler', name='home')
])
app.error_handlers[404] = handle_404
app.error_handlers[500] = handle_500
How can I handle the 404 error in a webapp2.RequestHandler class, in the .get() method of that class?
Edit:
The reason I want to call a RequestHandler is to access the session (request.session). Otherwise I'm not able to pass the current user to the template of the 404 error page. i.e. on the StackOverflow 404 error page you can see your username. I would like to display the username of the current user on my website's 404 error page as well. Is this possible in a function or does it has to be a RequestHandler?
Correct code based on #proppy's answer:
class Webapp2HandlerAdapter(webapp2.BaseHandlerAdapter):
def __call__(self, request, response, exception):
request.route_args = {}
request.route_args['exception'] = exception
handler = self.handler(request, response)
return handler.get()
class Handle404(MyBaseHandler):
def get(self):
self.render(filename="404.html",
page_title="404",
exception=self.request.route_args['exception']
)
app = webapp2.WSGIApplication(urls, debug=True, config=config)
app.error_handlers[404] = Webapp2HandlerAdapter(Handle404)
The calling convention of error handler and request handler callables are different:
error_handlers takes (request, response, exception)
RequestHandler takes (request, response)
You may use something similar to Webapp2HandlerAdapter to adapt a webapp2.RequestHandler to a callable.
class Webapp2HandlerAdapter(BaseHandlerAdapter):
"""An adapter to dispatch a ``webapp2.RequestHandler``.
The handler is constructed then ``dispatch()`` is called.
"""
def __call__(self, request, response):
handler = self.handler(request, response)
return handler.dispatch()
But you would have to sneak the extra exception argument in request route_args.
Is it possible to create a catch-all global exception handler in Google App Engine using Python?
Basically, I want to catch all un-caught exceptions and gracefully handle it, while sending an email with the traceback to me.
Currently, for all uncaught errors, the users see a stacktrace with a snippet of code in it. This is undesirable.
Yes it is possible.
You can do it using the ereporter package that allows to receive exception reports from your application by email.
Ereporter will report two kind of exceptions:
exceptions logged with logging.exception('Your handled exception')
any uncaught exceptions
To catch all the exceptions, I would create a custom BaseHandler class overriding the handle_exception() method; all your request handlers should inherit from this Base class.
Have a look to Custom Error Responses too.
Here is a simple example of BaseHandler class:
class BaseHandler(webapp.RequestHandler):
def handle_exception(self, exception, debug_mode):
if debug_mode:
webapp.RequestHandler.handle_exception(self, exception, debug_mode)
else:
logging.exception(exception)
self.error(500)
self.response.out.write(template.render('templdir/error.html', {}))
You might want to call the original handle_exception by calling the following in your BaseHandler:
webapp.RequestHandler.handle_exception(self, exception, debug_mode)
Here it is in context.
from google.appengine.ext import webapp
import sys
import traceback
class BaseHandler(webapp.RequestHandler):
def handle_exception(self, exception, debug_mode):
from main import emaildevs
emaildevs('An error occurred on example.com', ''.join(traceback.format_exception(*sys.exc_info())))
webapp.RequestHandler.handle_exception(self, exception, debug_mode)
try:
call
except:
sendemail
http://docs.python.org/tutorial/errors.html