Stackdriver Trace with Google App Engine Go 1.11 runtime - google-app-engine

I would like to get Stackdriver Trace to work on the new Go 1.11 standard runtime on Google App Engine. This is all still labeled as beta so maybe it just doesn't quite work yet. However, I followed the step-by-step directions and it isn't working. I deployed the code (almost) exactly as it is listed in the link and my traces are flat (i.e. doesn't include the waterfall view I would expect with the incoming request at the top and the outgoing request nested underneath).
Sample code
package main
import (
"fmt"
"log"
"net/http"
"os"
"contrib.go.opencensus.io/exporter/stackdriver"
"contrib.go.opencensus.io/exporter/stackdriver/propagation"
"go.opencensus.io/plugin/ochttp"
"go.opencensus.io/trace"
)
func main() {
// Create and register a OpenCensus Stackdriver Trace exporter.
exporter, err := stackdriver.NewExporter(stackdriver.Options{
ProjectID: os.Getenv("GOOGLE_CLOUD_PROJECT"),
})
if err != nil {
log.Fatal(err)
}
trace.RegisterExporter(exporter)
client := &http.Client{
Transport: &ochttp.Transport{
// Use Google Cloud propagation format.
Propagation: &propagation.HTTPFormat{},
},
}
handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
req, _ := http.NewRequest("GET", "https://www.google.com/robots.txt", nil)
// The trace ID from the incoming request will be
// propagated to the outgoing request.
req = req.WithContext(r.Context())
// The outgoing request will be traced with r's trace ID.
resp, err := client.Do(req)
if err != nil {
log.Fatal(err)
}
defer resp.Body.Close()
fmt.Fprint(w, "OK")
})
http.Handle("/foo", handler)
log.Fatal(http.ListenAndServe(":"+os.Getenv("PORT"), &ochttp.Handler{}))
}
Sample trace:

As written in the reply comment in the original question, can you try the config for sampling?: trace.AlwaysSample()
You can find some comments about sampling rate in OpenCensus Trace documentation and godoc of OpenCensus Trace library:
By default, traces will be sampled relatively rarely. To change the sampling frequency for your entire program, call ApplyConfig. Use a ProbabilitySampler to sample a subset of traces, or use AlwaysSample to collect a trace on every run:

Related

Parse multipart form on Google App Engine

A project I'm working on depends on having a service hosted on Google App Engine parse from SendGrid. The following code is an example of what we're doing:
package sendgrid_failure
import (
"net/http"
"fmt"
"google.golang.org/appengine"
"google.golang.org/appengine/log"
)
func init() {
http.HandleFunc("/sendgrid/parse", sendGridHandler)
}
func sendGridHandler(w http.ResponseWriter, r *http.Request) {
ctx := appengine.NewContext(r)
err := r.ParseMultipartForm(-1)
if err != nil {
log.Errorf(ctx, "Unable to parse form: %v", err)
}
fmt.Fprint(w, "Test.")
}
When SendGrid POSTs its multipart form, the console shows similar to the following:
2018/01/04 23:44:08 ERROR: Unable to parse form: open /tmp/multipart-445139883: no file writes permitted on App Engine
App Engine doesn't allow you to read/write files, but Golang appears to need it to parse. Is there an App Engine specific library to parse multipart forms, or should we be using a different method from the standard net/http library entirely? We're using the standard go runtime.
The documentation for ParseMultipartForm says:
The whole request body is parsed and up to a total of maxMemory bytes of its file parts are stored in memory, with the remainder stored on disk in temporary files.
The server attempts to write all files to disk because the application passed -1 as maxMemory. Use a value larger than the size of the files you expect to upload.

How to make a get request mid application

How would I go about creating a GET HTTP request mid application in GAE? I don't want to have it as a handler function, I simply have a URL that I need to get the response body from.
Use the urlfetch package.
ctx := appengine.NewContext(r) // r is the *http.Request arg to the handler
client := urlfetch.Client(ctx)
resp, err := client.Get("http://example.com")
if err != nil {
// handle the error
}
body := resp.Body // body is an io.Reader containing the response body
Here's a complete example.
I haven't used it so apologies if this doesn't work. According to GAE's docs you probably want to use urlfetch to get a *http.Client something like (N.B. the context package is standard in the just released Go 1.7):
import (
"context" // Go 1.7
// "golang.org/x/net/context" // Go < 1.7
"google.golang.org/appengine/urlfetch"
)
client := urlfetch.Client(context.Background())
resp, err := client.Get("http://example.com/")

Cross platform go code for appengine

What is the GO appropriate way to create a FetchUrl/GetURL function that works from the command line and works from google app engine with its custom way to fetch a url.
I have basic code that fetches and processes some data on a URL. I want to be able to call it from code I use on my desktop, and code deployed to app engine.
Hopefully thats clear, if not please let me know and Ill clarify.
If you have some code which works both on local machine and on AppEngine environment, you have nothing to do.
If you need to do something which should or must be done differently on AppEngine, then you need to detect the environment and write different code for the different environments.
This detection and code selection is easiest done using build constraints. You can put a special comment line in the beginning of your .go file, and it may or may not be compiled and run depending on the environment.
Quoting from The Go Blog: The App Engine SDK and workspaces (GOPATH):
The App Engine SDK introduces a new build constraint term: "appengine". Files that specify
// +build appengine
will be built by the App Engine SDK and ignored by the go tool. Conversely, files that specify
// +build !appengine
are ignored by the App Engine SDK, while the go tool will happily build them.
So for example you can have 2 separate .go files, one for AppEngine and one for local (non-AppEngine) environment. Define the same function in both (with same parameter list), so no matter in which environment the code is built, the function will have one declaration. We will use this signature:
func GetURL(url string, r *http.Request) ([]byte, error)
Note that the 2nd parameter (*http.Request) is only required for the AppEngine (in order to be able to create a Context), so in the implementation for local env it is not used (can even be nil).
An elegant solution can take advantage of the http.Client type which is available in both the standard environment and in AppEngine, and which can be used to do an HTTP GET request. An http.Client value can be acquired differently on AppEngine, but once we have an http.Client value, we can proceed the same way. So we will have a common code that receives an http.Client and can do the rest.
Example implementation can look like this:
url_local.go:
// +build !appengine
package mypackage
import (
"net/http"
)
func GetURL(url string, r *http.Request) ([]byte, error) {
// Local GetURL implementation
return GetClient(url, &http.Client{})
}
url_gae.go:
// +build appengine
package mypackage
import (
"google.golang.org/appengine"
"google.golang.org/appengine/urlfetch"
"net/http"
)
func GetURL(url string, r *http.Request) ([]byte, error) {
// Appengine GetURL implementation
ctx := appengine.NewContext(r)
c := urlfetch.Client(ctx)
return GetClient(url, c)
}
url_common.go:
// No build constraint: this is common code
package mypackage
import (
"net/http"
)
func GetClient(url string, c *http.Client) ([]byte, error) {
// Implementation for both local and AppEngine
resp, err := c.Get(url)
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, err
}
return body, nil
}
You could get some clues in the golang/appengine project itself.
For instance, its remote_api/client.go provides the client for connecting remotely to a user's production application.

Why doesn't fmt.Println work in Google app engine

I built a simple web app using google app engine and golang. in code below, I use fmt.Println twice to print out somehting for debugging purpose. I have no problem running the app. everything works except nothing print out on the terminal.
func HomeHandler(w http.ResponseWriter, r *http.Request) {
c := appengine.NewContext(r)
q := datastore.NewQuery("Post").Ancestor(goblogKey(c)).Order("-CreatedOn").Limit(10)
//posts := make([]entity.Post, 0, 10)
var posts []entity.Post
if _, err := q.GetAll(c, &posts); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
fmt.Println(string(len(posts)) + "...........")
postList := []dto.Post{}
for _, val := range posts {
newpost := dto.Post{
Post: val,
BodyHTML: template.HTML(val.Body),
}
fmt.Println(val.Title)
postList = append(postList, newpost)
}
page := dto.PageData{Title: "Home", Posts: postList}
templates.ExecuteTemplate(w, "index", page)
}
In the real appengine enviroment you can't se anything output to stdout.
Appengine context give you away to log (that you can check in you appengine admin's page and in console playground).
func HomeHandler(w http.ResponseWriter, r *http.Request) {
c := appengine.NewContext(r)
c.Debugf("The message: %s", "foo")
...
Read more: https://developers.google.com/appengine/docs/go/reference#Context
standard i/o Or Error is used communicate with app server used by the devleoper. In production system there's no meaning of using standard i/o. In production systems log is used to track the results. In app engine there's some limitations. like fmt, socket etc.
Its always better to use log when testing or running program in remote server.

How to send 204 No Content with Go http package?

I built a tiny sample app with Go on Google App Engine that sends string responses when different URLs are invoked. But how can I use Go's http package to send a 204 No Content response to clients?
package hello
import (
"fmt"
"net/http"
"appengine"
"appengine/memcache"
)
func init() {
http.HandleFunc("/", hello)
http.HandleFunc("/hits", showHits)
}
func hello(w http.ResponseWriter, r *http.Request) {
name := r.Header.Get("name")
fmt.Fprintf(w, "Hello %s!", name)
}
func showHits(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "%d", hits(r))
}
func hits(r *http.Request) uint64 {
c := appengine.NewContext(r)
newValue, _ := memcache.Increment(c, "hits", 1, 0)
return newValue
}
According to the package docs:
func NoContent(w http.ResponseWriter, r *http.Request) {
// Set up any headers you want here.
w.WriteHeader(http.StatusNoContent) // send the headers with a 204 response code.
}
will send a 204 status to the client.
Sending 204 response from your script means your instance still need to run and cost you money. If you are looking for a caching solution. Google got it and it's called Edge Cache.
You only need to response with the following headers and Google will automatically cache your response in multiple servers nearest to the users (that is, replying with 204). This greatly enhance your site's speed and reduce instance cost.
w.Header().Set("Cache-Control", "public, max-age=86400")
w.Header().Set("Pragma", "Public")
You can adjust the max-age, but do it wisely.
By the way, it seems billing must be enabled in order to use Edge Cache

Resources