Is it possible to create a catch-all global exception handler in Google App Engine using Python?
Basically, I want to catch all un-caught exceptions and gracefully handle it, while sending an email with the traceback to me.
Currently, for all uncaught errors, the users see a stacktrace with a snippet of code in it. This is undesirable.
Yes it is possible.
You can do it using the ereporter package that allows to receive exception reports from your application by email.
Ereporter will report two kind of exceptions:
exceptions logged with logging.exception('Your handled exception')
any uncaught exceptions
To catch all the exceptions, I would create a custom BaseHandler class overriding the handle_exception() method; all your request handlers should inherit from this Base class.
Have a look to Custom Error Responses too.
Here is a simple example of BaseHandler class:
class BaseHandler(webapp.RequestHandler):
def handle_exception(self, exception, debug_mode):
if debug_mode:
webapp.RequestHandler.handle_exception(self, exception, debug_mode)
else:
logging.exception(exception)
self.error(500)
self.response.out.write(template.render('templdir/error.html', {}))
You might want to call the original handle_exception by calling the following in your BaseHandler:
webapp.RequestHandler.handle_exception(self, exception, debug_mode)
Here it is in context.
from google.appengine.ext import webapp
import sys
import traceback
class BaseHandler(webapp.RequestHandler):
def handle_exception(self, exception, debug_mode):
from main import emaildevs
emaildevs('An error occurred on example.com', ''.join(traceback.format_exception(*sys.exc_info())))
webapp.RequestHandler.handle_exception(self, exception, debug_mode)
try:
call
except:
sendemail
http://docs.python.org/tutorial/errors.html
Related
I'm trying to deploy appengine, but I'm seeing this error in the logs:
Uncaught exception from servlet
com.google.inject.ProvisionException: Unable to provision, see the following errors:
1) Error in custom provider, com.google.inject.OutOfScopeException: Cannot access scoped [sc.analysis.metrics.Metrics]. Either we are not currently inside an HTTP Servlet request, or you may have forgotten to apply com.google.inject.servlet.GuiceFilter as a servlet filter for this request.
at sc.analysis.metrics.MetricsModule.configure(MetricsModule.java:13)
while locating sc.analysis.metrics.Metrics
at sc.geo.management.geo.api.GeoAdminAPIv2.<init>(GeoAdminAPIv2.java:124)
while locating sc.geo.management.geo.api.GeoAdminAPIv2
1 error
at com.google.inject.internal.InternalProvisionException.toProvisionException(InternalProvisionException.java:226)
at com.google.inject.internal.InjectorImpl$1.get(InjectorImpl.java:1053)
at com.google.inject.spi.ProviderLookup$1.get(ProviderLookup.java:111)
at com.google.api.server.spi.guice.ServiceMap.get(ServiceMap.java:68)
at com.google.api.server.spi.guice.GuiceEndpointsServlet.createService(GuiceEndpointsServlet.java:36)
at com.google.api.server.spi.EndpointsServlet.createSystemService(EndpointsServlet.java:136)
at com.google.api.server.spi.EndpointsServlet.init(EndpointsServlet.java:57)
at com.google.inject.servlet.ServletDefinition.init(ServletDefinition.java:121)
at com.google.inject.servlet.ManagedServletPipeline.init(ManagedServletPipeline.java:82)
at com.google.inject.servlet.ManagedFilterPipeline.initPipeline(ManagedFilterPipeline.java:103)
at com.google.inject.servlet.GuiceFilter.init(GuiceFilter.java:220)
at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:140)
at org.eclipse.jetty.servlet.ServletHandler.lambda$initialize$0(ServletHandler.java:731)
at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:742)
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:755)
at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379)
at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449)
at com.google.apphosting.runtime.jetty94.AppEngineWebAppContext.startWebapp(AppEngineWebAppContext.java:274)
at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414)
at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:916)
at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288)
at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524)
at com.google.apphosting.runtime.jetty94.AppEngineWebAppContext.doStart(AppEngineWebAppContext.java:218)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at com.google.apphosting.runtime.jetty94.AppVersionHandlerFactory.doCreateHandler(AppVersionHandlerFactory.java:178)
at com.google.apphosting.runtime.jetty94.AppVersionHandlerFactory.createHandler(AppVersionHandlerFactory.java:112)
at com.google.apphosting.runtime.jetty94.AppVersionHandlerMap.getHandler(AppVersionHandlerMap.java:82)
at com.google.apphosting.runtime.jetty94.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:167)
at com.google.apphosting.runtime.RequestRunner.dispatchServletRequest(RequestRunner.java:264)
at com.google.apphosting.runtime.RequestRunner.dispatchRequest(RequestRunner.java:229)
at com.google.apphosting.runtime.RequestRunner.run(RequestRunner.java:194)
at com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:273)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.inject.OutOfScopeException: Cannot access scoped [sc.analysis.metrics.Metrics]. Either we are not currently inside an HTTP Servlet request, or you may have forgotten to apply com.google.inject.servlet.GuiceFilter as a servlet filter for this request.
at com.google.inject.servlet.GuiceFilter.getContext(GuiceFilter.java:165)
at com.google.inject.servlet.GuiceFilter.getOriginalRequest(GuiceFilter.java:147)
at com.google.inject.servlet.ServletScopes$1$1.get(ServletScopes.java:107)
at com.google.inject.internal.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:39)
at com.google.inject.internal.InjectorImpl$1.get(InjectorImpl.java:1050)
at sc.analysis.metrics.StaticMetricsHolder.get(StaticMetricsHolder.java:27)
at sc.analysis.metrics.StaticMetricsHolder.get(StaticMetricsHolder.java:19)
at sc.util.ScDatastore.findEntity(ScDatastore.java:765)
at sc.util.ScDatastore.findEntity(ScDatastore.java:747)
at sc.util.Datastore.findEntity(Datastore.java:289)
at picaboo.entity.util.RegistryEntities.findRegistryEntity(RegistryEntities.java:98)
at picaboo.entity.util.RegistryEntities.findRegistryEntity(RegistryEntities.java:94)
at picaboo.entity.util.RegistryEntities.findOrCreateRegistryEntity(RegistryEntities.java:116)
at sc.registry.RegistrySetting.getEntity(RegistrySetting.java:125)
at sc.registry.RegistrySetting.getUncachedValue(RegistrySetting.java:207)
at sc.registry.RegistrySetting.fetchLatestValue(RegistrySetting.java:186)
at sc.registry.RegistrySetting.updateIfNecessary(RegistrySetting.java:151)
at sc.registry.RegistrySetting.getValue(RegistrySetting.java:196)
at sc.registry.RegistrySetting.getValue(RegistrySetting.java:35)
at sc.registry.ConvertedSetting.reloadIfNecessary(ConvertedSetting.java:62)
at sc.registry.ConvertedSetting.getValue(ConvertedSetting.java:34)
at sc.geo.management.geo.api.AdminApiIngestion.<init>(AdminApiIngestion.java:62)
at sc.geo.management.geo.api.AdminApiIngestion.<init>(AdminApiIngestion.java:47)
at sc.geo.management.geo.api.GeoAdminAPI.<init>(GeoAdminAPI.java:327)
at sc.geo.management.geo.api.GeoAdminAPIv2.<init>(GeoAdminAPIv2.java:124)
at sc.geo.management.geo.api.GeoAdminAPIv2$$FastClassByGuice$$855da3d.newInstance(<generated>)
at com.google.inject.internal.DefaultConstructionProxyFactory$FastClassProxy.newInstance(DefaultConstructionProxyFactory.java:89)
at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:114)
at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:91)
at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:306)
at com.google.inject.internal.InjectorImpl$1.get(InjectorImpl.java:1050)
... 33 more
This error is thrown when the servlet starts.
It's always the Metrics Module, but the Metrics Module looks correct in terms of injection:
package sc.analysis.metrics;
import com.google.inject.AbstractModule;
import com.google.inject.Singleton;
import com.google.inject.servlet.RequestScoped;
public class MetricsModule extends AbstractModule {
#Override
protected void configure() {
bind(Metrics.class).to(MetricsImpl.class).in(RequestScoped.class);
bind(GlobalMetrics.class).to(GlobalMetricsImpl.class).in(Singleton.class);
requestStaticInjection(StaticMetricsHolder.class);
requestStaticInjection(StaticGlobalMetricsHolder.class);
requestStaticInjection(ScopeSafeMetricsHolder.class);
}
}
None of the other injections seem to have issues; is there sommething I'm missing? I don't know much about guice to be honest, but the code that calls the Metrics (GeoAdminAPI) uses a provider:
public class GeoAdminAPIv2 extends GeoAdminAPI {
#Inject
GeoAdminAPIv2(...,
final Provider<Metrics> metrics,
...)
When I look at the logs in the Google Log Viewer for my GAE project, I see that often the logs that I write myself in the code are assigned to the wrong request. Most of the time the log is assigned to the request directly after the request that produced the log entry.
As the root of every application log in GAE must be a request, this means that the wrong request is sometimes marked as error, because another request before produced an error, but the log is somehow assigned to the request after that.
I don't really do anything special, I use Ktor as my servlet and have an interceptor that creates a log when an exception occurs before returning status 500.
I use Java logging via SLF4J with the google cloud logging handler, but before that I used logback via SLf4J and had the same problem.
The content of the logs itself is also correct, the returned status of the request, the level of the log entry, the message, everything is ok.
I thought that it may be because I use kotlin and switch coroutine contexts during a single request, but in some cases the point where I write the log and where I send the response are exactly next to each other, so I'm not sure if kotlin has anything to do with it.
My logging.properties:
# To use this configuration, add to system properties : -Djava.util.logging.config.file="/path/to/file"
#
.level = INFO
# it is recommended that io.grpc and sun.net logging level is kept at INFO level,
# as both these packages are used by Stackdriver internals and can result in verbose / initialization problems.
io.grpc.netty.level=INFO
sun.net.level=INFO
handlers=com.google.cloud.logging.LoggingHandler
# default : java.log
com.google.cloud.logging.LoggingHandler.log=custom_log
# default : INFO
com.google.cloud.logging.LoggingHandler.level=INFO
# default : ERROR
com.google.cloud.logging.LoggingHandler.flushLevel=WARNING
# default : auto-detected, fallback "global"
#com.google.cloud.logging.LoggingHandler.resourceType=container
# custom formatter
com.google.cloud.logging.LoggingHandler.formatter=java.util.logging.SimpleFormatter
java.util.logging.SimpleFormatter.format=%1$tY-%1$tm-%1$td %1$tH:%1$tM:%1$tS %4$-6s %2$s %5$s%6$s%n
#optional enhancers (to add additional fields, labels)
#com.google.cloud.logging.LoggingHandler.enhancers=com.example.logging.jul.enhancers.ExampleEnhancer
My logging relevant dependencies:
implementation "org.slf4j:slf4j-jdk14:1.7.30"
implementation "com.google.cloud:google-cloud-logging:1.100.0"
An example logging call:
exception<Throwable> { e ->
logger().error("Error", e)
call.respondText(e.message ?: "", ContentType.Text.Plain, HttpStatusCode.InternalServerError)
}
with logger() being:
import org.slf4j.Logger
import org.slf4j.LoggerFactory
inline fun <reified T : Any> T.logger(): Logger = LoggerFactory.getLogger(T::class.java)
Edit:
An example of the log in Google cloud. The first request has the query parameter GAID=cdda802e-fb9c-47ad-0794d394c913, but as you can see the error log for that request is in the one below, marked in red.
I'm building a new spottily app using cocoaLibSpotify.
Immediately after instantiating the session with initializeSharedSessionWithApplicationKey, I call attemptLoginWithUserName with a valid username/password pair.
The app then immediately crashes with
+[NSURL urlWithSpotifyLink:]: unrecognized selector sent to class 0x23e826c
2014-01-23 14:05:09.476 MercuryDockAssistant[44744:3f03] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '+[NSURL urlWithSpotifyLink:]: unrecognized selector sent to class 0x23e826c'
*** First throw call stack:
(0x22c2012 0x20e7e7e 0x234d2ad 0x22b1bbc 0x22b194e 0x44196 0x43d46 0x35aac 0x4398c 0x3305c 0x2e50d 0x2281920 0x2244d31 0x2268724 0x2267f44 0x2273f91 0x2e7d5 0xcd20d5 0xcd2034 0x2d0c5fb 0x2d0c485 0x2d11cf2)
libc++abi.dylib: terminate called throwing an exception
So it looks like the NSURL extensions are not being recognised. Any ideas?
Ensure you are importing the header wherever you are using spotify extensions like urlWithSpotifyLink:.
#import "CocoaLibSpotify.h"
Also, as mentioned in the CocoaLibSpotify readme, you need to add the -ObjC and -all_load flags to the "Other Linker Flags" build setting in Xcode.
Can error_handler be set for a blueprint?
#blueprint.errorhandler(404)
def page_not_found(error):
return 'This page does not exist', 404
edit:
https://github.com/mitsuhiko/flask/blob/18413ed1bf08261acf6d40f8ba65a98ae586bb29/flask/blueprints.py
you can specify an app wide and a blueprint local error_handler
You can use Blueprint.app_errorhandler method like this:
bp = Blueprint('errors', __name__)
#bp.app_errorhandler(404)
def handle_404(err):
return render_template('404.html'), 404
#bp.app_errorhandler(500)
def handle_500(err):
return render_template('500.html'), 500
errorhandler is a method inherited from Flask, not Blueprint.
If you are using Blueprint, the equivalent is app_errorhandler.
The documentation suggests the following approach:
def app_errorhandler(self, code):
"""Like :meth:`Flask.errorhandler` but for a blueprint. This
handler is used for all requests, even if outside of the blueprint.
"""
Therefore, this should work:
from flask import Blueprint, render_template
USER = Blueprint('user', __name__)
#USER.app_errorhandler(404)
def page_not_found(e):
""" Return error 404 """
return render_template('404.html'), 404
On the other hand, while the approach below did not raise any error for me, it didn't work:
from flask import Blueprint, render_template
USER = Blueprint('user', __name__)
#USER.errorhandler(404)
def page_not_found(e):
""" Return error 404 """
return render_template('404.html'), 404
add error handling at application level using the request proxy object:
from flask import request,jsonify
#app.errorhandler(404)
#app.errorhandler(405)
def _handle_api_error(ex):
if request.path.startswith('/api/'):
return jsonify(ex)
else:
return ex
flask Documentation
I too couldn't get the top rated answer to work, but here's a workaround.
You can use a catch-all at the end of your Blueprint, not sure how robust/recommended it is, but it does work. You could also add different error messages for different methods too.
#blueprint.route('/<path:path>')
def page_not_found(path):
return "Custom failure message"
Surprised others didn't mention miguelgrinberg's excellent tutorial.
https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-vii-error-handling
I found the sentry framework for error handling (links below). Seems overly complex. not sure of the threshold where it becomes useful.
https://flask.palletsprojects.com/en/1.1.x/errorhandling/
https://docs.sentry.io/platforms/python/guides/flask/
I combined previous excellent answers with the official docs from Flask, section 'Returning API Errors as JSON', in order to provide a more general approach.
Here is a working PoC that you can copy and paste on your registered blueprint API route handler (e.g. app/api/routes.py):
#blueprint.app_errorhandler(HTTPException)
def handle_exception(e):
"""Return JSON instead of HTML for HTTP errors."""
# start with the correct headers and status code from the error
response = e.get_response()
# replace the body with JSON
response.data = json.dumps({
"code": e.code,
"name": e.name,
"description": e.description,
})
response.content_type = "application/json"
return response
Flask doesnt support blueprint level error handlers for 404 and 500 errors. A BluePrint is a leaky abstraction. Its better to use a new WSGI App for this, if you need separate error handlers, this makes more sense.
Also i would recommend not to use flask, it uses globals all over the places, which makes your code difficult to manage if it grows bigger.
I'd like to catch and handle DeadlineExceededError so users don't see the standard "Server Error" page that App Engine throws by default.
I know that DeadlineExceededErrors are not caught when overriding handle_exception in your request handler (we already do this).
I have tried, unsuccessfully so far, to use the custom error_handlers app.yaml configuration like so:
error_handlers:
- error_code: timeout
file: timeout.html
...but that also doesn't seem to catch DeadlineExceededErrors, unless I'm doing something wrong.
I am aware that I can use the following pattern to catch DeadlineExceededErrors inside particular request handlers:
class MainPage(webapp.RequestHandler):
def get(self):
try:
# Do stuff...
except DeadlineExceededError:
# Many Whelps! Handle it!
...but I would like to avoid adding this to every single request handler in my application.
How can I globally catch these elusive suckers?
One possible solution is to use webapp2, which is a pretty neat framework as it is and has a lot of useful stuff over the original webapp. With webapp2, you can handle the exception in the handle_500 method, as follows:
def BaseHandler(webapp2.RequestHandler):
def handle_500(request, response, exception):
if isinstance(exception, DeadlineExceededError):
response.write('Deadline exceeded!')
else:
response.write('A server error occurred!')
logging.exception(exception)
response.set_status(500)