Is it possible to recover from panic on google app engine? - google-app-engine

I'm wondering if it's possible to recover from a panic. It seems GAE has it's own panic recovery mechanism but I can't find any hook to handle it on my app.

Handlers in an AppEngine webapp are registered in the same way as would be in a normal Go application. You just don't have to call http.ListenAndServe() explicitly (because it will be by the platform), and handler registration happens in an init() function (not in main()).
Having said that, the same panic-recover wrapping works on AppEngine too, and unfortunately there is no other, better way.
Take a look at this example: it uses a function registered with HandleFunc() and a Handler registered with Handle() to handle 2 URL patterns, but both intentionally panics (they refuse to serve):
func myHandleFunc(w http.ResponseWriter, r *http.Request) {
panic("I'm myHandlerFunc and I refuse to serve!")
}
type MyHandler int
func (m *MyHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
panic("I'm MyHandler and I refuse to serve!")
}
func main() {
http.HandleFunc("/myfunc", myHandleFunc)
http.Handle("/myhandler", new(MyHandler))
panic(http.ListenAndServe(":8080", nil))
}
Directing your browser to http://localhost:8080/myfunc and http://localhost:8080/myhandler results in HTTP 500 status: internal server error (or an empty response depending on where you check it).
The general idea is to use recover to "catch" the panics from handlers (spec: Handling panics). We can "wrap" handle functions or Handlers in a way that we first register a defer statement which is called even if the rest of the function panics, and in which we recover from the panicing state.
See these 2 functions:
func protectFunc(hf func(http.ResponseWriter,
*http.Request)) func(http.ResponseWriter, *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
defer func() {
r := recover()
if r != nil {
// hf() paniced, we just recovered from it.
// Handle error somehow, serve custom error page.
w.Write([]byte("Something went bad but I recovered and sent this!"))
}
}()
hf(w, r)
}
}
func protectHandler(h http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
defer func() {
r := recover()
if r != nil {
// h.ServeHTTP() paniced, we just recovered from it.
// Handle error somehow, serve custom error page.
w.Write([]byte("Something went bad but I recovered and sent this!"))
}
}()
h.ServeHTTP(w, r)
})
}
The first one takes a function and returns one which calles the one we passed but recovers from panicing state if one was initiated.
The second one takes a Handler and returns another Handler which similarly calls the passed one but also handles panics and restores normal execution.
Now if we register handler functions and Handlers protected by these methods, the registered handlers will never panic (assuming the code after restoring normal execution does not panic):
http.HandleFunc("/myfunc-protected", protectFunc(myHandleFunc))
http.Handle("/myhandler-protected", protectHandler(new(MyHandler)))
Visiting http://localhost:8080/myfunc-protected and http://localhost:8080/myhandler-protected URLs resuls in HTTP 200 status (OK) with the message:
Something went bad but I recovered and sent this!

Related

How to use cloud.google.com/go/datastore with AppEngine?

I'm migrating my Golang AppEngine app to 1.12+, and I need to switch to cloud.google.com/go/datastore. It's not clear to me how to use it with AppEngine, could someone please verify my assumptions?
My assumption is that somewhere inside main() I can run (note the context.Background()):
db, err := datastore.NewClient(context.Background(), datastore.DetectProjectID)
if err != nil {
panic(err)
}
defer db.Close()
And then from my handlers I can use that db:
func blah(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
key := datastore.NameKey("blah", blah, nil)
db.Get(ctx, key, blah2)
}
Am I correct? Or do I need to run datastore.NewClient() separately from each web handler?
I run datastore.NewClient() for each web handler. I only do this once for each web handler and only if that code needs to use the datastore. Also, make sure that you Close() it at the end of your handler otherwise you'll leak memory.

How to mock a ping command

I'm using https://github.com/DATA-DOG/go-sqlmock
and trying to mock a connection to a db.
Now, I need to mock a ping command (for load balancing purposes). However, I can't find any information about how to do that.
For example, I would like to write a test like this
db, mock, _ := sqlmock.New()
// ExpectPing does not exist, but it is there anything similar?
mock.ExpectPing().WillReturnError("mock error")
err := db.Ping()
if err==nil{
t.Fatal("there should have been an error")
}
in addition, I need to inject this mocked object into a function. let's say New(*sql.DB) that outputs a new structure. so it must be compatible with *sql.DB struct. for this reason, sqlmock seems a good choice. However, I've not found the way to mock the ping command.
it is there any way to do this using this library?
if not, is there any other database/sql mock library that could do the trick?
Update: As of Jan 6, 2020, this functionality has been added to go-sqlmock, and is included in the v1.4.0 release.
Original answer:
No, there is "nothing similar." The Ping and PingContext methods depend on the driver implementing the Pinger interface, so you can't mimic it by, for example, expecting a 'SELECT 1' or something.
There is already an issue requesting to add this. It seems to not have gotten much attention. I suspect creating a PR (which is probably about 3 lines of code) would greatly increase the chance of having that feature added.
If your goal is to make a Ping fail, you should be able to mimic that by connecting to an invalid DB endpoint (either with or without sqlmock).
You certainly can mock db itself:
type mockedDB struct {
*sql.DB
}
func (db *mockedDB) Ping() error {
return errors.New("not implemented")
}
func Example_mockedDB_Ping() {
db, _, _ := sqlmock.New()
defer db.Close()
mdb := mockedDB{db}
fmt.Println("mdb.Ping(): ", mdb.Ping())
// Output: mdb.Ping(): not implemented
}
but I don't understand what is the purpose of such experiment.
In the same way you can put mock into new type and define ExpectPing on it.
I needed to mock the Ping() functionality as well. This is how I solved it:
type mockDB struct {
ReturnError error
}
func (db *mockDB) Ping() error {
return db.ReturnError
}
func (db *mockDB) PingContext(ctx context.Context) error {
return db.ReturnError
}
// Pinger to be able to mock & ask just for a ping
type Pinger interface {
PingContext(ctx context.Context) error
Ping() error
}
// DatabasePingCheck returns a Check that validates connectivity to a database/sql.DB using Ping().
func DatabasePingCheck(pinger Pinger, timeout time.Duration) Check {
return func() error {
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
if pinger == nil {
return fmt.Errorf("pinger is nil")
}
return pinger.PingContext(ctx)
}
}

How do I add custom tracing to a second-generation App Engine application in Go?

Google App Engine now supports Go 1.11 via the new second-generation standard environment. While converting an older standard environment application to the second generation, it wasn't obvious how to combine tracing information from the app engine infrastructure with custom tracing I added to the application using OpenCensus.
Even though I had created a stackdriver exporter and registered traces, I wasn't seeing custom trace information in the stackdriver console attached to inbound requests.
The key is I was missing is understanding how span context is communicated to the serving app. Google leverages the X-Cloud-Trace-Context header to propagate span context within requests sent to your serving instances, and the go.opencensus.io/exporter/stackdriver/propagation library provides an implementation to extract and persist this information within http requests.
Don't forget to create a stackdriver exporter, and register traces to it. The docs for the exporter library show an example of this.
// CreateSpanFromRequest returns a context and span based on the http.Request.
// If no existing spancontext is found, this will start a new span.
// Modifies existing request to contain the generated span's context.
func CreateSpanFromRequest(name string, r *http.Request) (context.Context, *trace.Span) {
var span *trace.Span
ctx := r.Context()
httpFormat := &propagation.HTTPFormat{}
sc, ok := httpFormat.SpanContextFromRequest(r)
if ok {
ctx, span = trace.StartSpanWithRemoteParent(ctx, name, sc)
} else {
ctx, span = trace.StartSpan(ctx, name)
}
// Write the span context into the http.Request. We do this to
// to enable chaining handlers together more easily.
httpFormat.SpanContextToRequest(span.SpanContext(), r)
return ctx, span
}
Using this, I was able to add custom spans to my handlers that would be properly associated with the incoming request information in stackdriver:
func indexHandler(w http.ResponseWriter, r *http.Request) {
_, span := CreateSpanFromRequest("indexHandler", r)
defer span.End()
if r.URL.Path != "/" {
http.NotFound(w, r)
return
}
fmt.Fprint(w, "Hello, World!")
}

How do I get the current namespace in and App Engine delay function using Go?

If I set the namespace of my context.Context and call a delay function:
ctx := appengine.NewContext(r)
ctx, err := appengine.Namespace(ctx, "mynamespace")
delayFunc.Call(ctx)
How can I find its name:
var delayFunc = delay.Func("my-func", func(ctx context.Context) {
// How do I extract "mynamespace" from ctx?
})
Is the following acceptable practice?
var delayFunc = delay.Func("my-func", func(ctx context.Context) {
n := datastore.NewKey(ctx, "E", "e", 0, nil).Namespace()
// n == "mynamespace"
})
It works but feels like a hack.
Unfortunately you're out of luck. Appengine does not provide an (exported) API call to access the namespace associated with the context.
The namespace association with the context is handled by the appengine/internal package, but "programs should not use this package directly". The context with namespace is obtained by the internal.NamespacedContext() call, and the namespace "extraction" from context is implemented in internal.NamespaceFromContext(). These are not part of the public API, so you can't (shouldn't) use them.
You basically have 2 options. One is the "hacky" way you presented, which works and you may continue to use it.
Another one is to manually handle it, e.g. by manually putting the namespace into the context with your own key, e.g.:
var namespaceKey = "myNsKey"
ctx = context.WithValue(ctx, namespaceKey, "mynamespace")
And when you need it, you can get it like:
ns := ctx.Value(namespaceKey)
Yes, this has the burden of having to manually update it, and if you'd forgot, you would get an "invalid" or empty namespace. So personally I would go with your "hacky" way for now (until this functionality is added to the public API, if ever).
If you go with the manual way, to get rid of the "risk" factor, you could create a helper function which could take care of this along with the appengine.Namespace() call, so you would not forget about it and it would be safe. It could look like this:
func SetNS(ctx context.Context, ns string) context.Context {
ctx = ctx, err := appengine.Namespace(ctx, ns)
if err != nil {
// handle error
}
ctx = context.WithValue(ctx, namespaceKey, ns)
return ctx
}
And using it:
ctx = SetNS(ctx, "mynamespace")
But it may be a rare case when you need to access namespace from the context, as when you would need it, it might just be enough to pass the context to the proper (Appengine) API call which can extract it from the context.

Can I use Goroutines in Google App Engine (Standard Environment)?

The following example seems to work, but is it safe to use? My goal is to do some very light background processing (whereas an actual task queue job feels too heavy).
func MyHandler(w http.ResponseWriter, r *http.Request) {
go func() {
// do something ...
}()
return // 200
}
Goroutines that outlive the request are not supported, but you can use runtime.RunInBackground to execute code in a background goroutine:
func MyHandler(w http.ResponseWriter, r *http.Request) {
err := runtime.RunInBackground(c, func(c appengine.Context) {
// do something...
})
return // 200
}
The provided function will be invoked with a background context that is distinct from (and may outlast) the provided context. Note that there is a limit of 10 simultaneous background requests per instance. Here is another example.
Please note that Goroutines that live within the context of a request, are supported though:
The Go runtime environment for App Engine provides full support for
goroutines, but not for parallel execution: goroutines are scheduled
onto a single operating system thread. This single-thread restriction
may be lifted in future versions. Multiple requests may be handled
concurrently by a given instance; that means that if one request is,
say, waiting for a datastore API call, another request may be
processed by the same instance. (Source)

Resources