How would I go about creating a GET HTTP request mid application in GAE? I don't want to have it as a handler function, I simply have a URL that I need to get the response body from.
Use the urlfetch package.
ctx := appengine.NewContext(r) // r is the *http.Request arg to the handler
client := urlfetch.Client(ctx)
resp, err := client.Get("http://example.com")
if err != nil {
// handle the error
}
body := resp.Body // body is an io.Reader containing the response body
Here's a complete example.
I haven't used it so apologies if this doesn't work. According to GAE's docs you probably want to use urlfetch to get a *http.Client something like (N.B. the context package is standard in the just released Go 1.7):
import (
"context" // Go 1.7
// "golang.org/x/net/context" // Go < 1.7
"google.golang.org/appengine/urlfetch"
)
client := urlfetch.Client(context.Background())
resp, err := client.Get("http://example.com/")
Related
I would like to get Stackdriver Trace to work on the new Go 1.11 standard runtime on Google App Engine. This is all still labeled as beta so maybe it just doesn't quite work yet. However, I followed the step-by-step directions and it isn't working. I deployed the code (almost) exactly as it is listed in the link and my traces are flat (i.e. doesn't include the waterfall view I would expect with the incoming request at the top and the outgoing request nested underneath).
Sample code
package main
import (
"fmt"
"log"
"net/http"
"os"
"contrib.go.opencensus.io/exporter/stackdriver"
"contrib.go.opencensus.io/exporter/stackdriver/propagation"
"go.opencensus.io/plugin/ochttp"
"go.opencensus.io/trace"
)
func main() {
// Create and register a OpenCensus Stackdriver Trace exporter.
exporter, err := stackdriver.NewExporter(stackdriver.Options{
ProjectID: os.Getenv("GOOGLE_CLOUD_PROJECT"),
})
if err != nil {
log.Fatal(err)
}
trace.RegisterExporter(exporter)
client := &http.Client{
Transport: &ochttp.Transport{
// Use Google Cloud propagation format.
Propagation: &propagation.HTTPFormat{},
},
}
handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
req, _ := http.NewRequest("GET", "https://www.google.com/robots.txt", nil)
// The trace ID from the incoming request will be
// propagated to the outgoing request.
req = req.WithContext(r.Context())
// The outgoing request will be traced with r's trace ID.
resp, err := client.Do(req)
if err != nil {
log.Fatal(err)
}
defer resp.Body.Close()
fmt.Fprint(w, "OK")
})
http.Handle("/foo", handler)
log.Fatal(http.ListenAndServe(":"+os.Getenv("PORT"), &ochttp.Handler{}))
}
Sample trace:
As written in the reply comment in the original question, can you try the config for sampling?: trace.AlwaysSample()
You can find some comments about sampling rate in OpenCensus Trace documentation and godoc of OpenCensus Trace library:
By default, traces will be sampled relatively rarely. To change the sampling frequency for your entire program, call ApplyConfig. Use a ProbabilitySampler to sample a subset of traces, or use AlwaysSample to collect a trace on every run:
I'm trying to create a signed URL to download a file in Google Cloud Storage, from an App Engine app written in Go. There's a nifty signing method in App Engine which I'm using, which [in theory] allows me to avoid putting a private key in my app. However, the URLs don't appear to work: I always get a 403 "SignatureDoesNotMatch" error. Any ideas?
Here's the code:
func createDownloadURL(c context.Context, resource string, validUntil time.Time, bucket string) (string, error) {
serviceAccount, err := appengine.ServiceAccount(c)
if err != nil {
return "", err
}
// Build up the string to sign.
validUntilString := strconv.FormatInt(validUntil.Unix(), 10)
toSign := []string{
"GET", // http verb (required)
"", // content MD5 hash (optional)
"", // content type (optional)
validUntilString, // expiration (required)
resource, // resource (required)
}
// Sign it.
_, signedBytes, err := appengine.SignBytes(c, []byte(strings.Join(toSign, "\n")))
if err != nil {
return "", err
}
signedString := base64.StdEncoding.EncodeToString(signedBytes)
// Build and return the URL.
arguments := url.Values{
"GoogleAccessId": {serviceAccount},
"Expires": {validUntilString},
"Signature": {signedString},
}
return fmt.Sprintf("https://storage.googleapis.com/%s/%s?%s", bucket, resource, arguments.Encode()), nil
}
Solved. There were 2 problems with my code.
I forgot to include the bucket name when building up toSign. Fix:
fmt.Sprintf("/%s/%s", bucket, resource), // resource (required)
This returned an AccessDenied error -- progress!
The second mistake was using the XML API storage.googleapis.com instead of the authenticated browser endpoint storage.cloud.google.com. Fix:
return fmt.Sprintf("https://storage.cloud.google.com/%s/%s?%s", bucket, resource, arguments.Encode()), nil
This works.
StringToSign may require uninterpreted newlines. Could you give this a try:
_, signedBytes, err := appengine.SignBytes(c, []byte(strings.Join(toSign, "\\n"))) // escaped newline
Writing a function that signs URLs is tricky, since due to the nature of encryption it's very difficult to tell what's wrong when things don't work. You may find it easier to use a library like gcloud-golang, which has a SignedURL method.
I built a simple web app using google app engine and golang. in code below, I use fmt.Println twice to print out somehting for debugging purpose. I have no problem running the app. everything works except nothing print out on the terminal.
func HomeHandler(w http.ResponseWriter, r *http.Request) {
c := appengine.NewContext(r)
q := datastore.NewQuery("Post").Ancestor(goblogKey(c)).Order("-CreatedOn").Limit(10)
//posts := make([]entity.Post, 0, 10)
var posts []entity.Post
if _, err := q.GetAll(c, &posts); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
fmt.Println(string(len(posts)) + "...........")
postList := []dto.Post{}
for _, val := range posts {
newpost := dto.Post{
Post: val,
BodyHTML: template.HTML(val.Body),
}
fmt.Println(val.Title)
postList = append(postList, newpost)
}
page := dto.PageData{Title: "Home", Posts: postList}
templates.ExecuteTemplate(w, "index", page)
}
In the real appengine enviroment you can't se anything output to stdout.
Appengine context give you away to log (that you can check in you appengine admin's page and in console playground).
func HomeHandler(w http.ResponseWriter, r *http.Request) {
c := appengine.NewContext(r)
c.Debugf("The message: %s", "foo")
...
Read more: https://developers.google.com/appengine/docs/go/reference#Context
standard i/o Or Error is used communicate with app server used by the devleoper. In production system there's no meaning of using standard i/o. In production systems log is used to track the results. In app engine there's some limitations. like fmt, socket etc.
Its always better to use log when testing or running program in remote server.
I built a tiny sample app with Go on Google App Engine that sends string responses when different URLs are invoked. But how can I use Go's http package to send a 204 No Content response to clients?
package hello
import (
"fmt"
"net/http"
"appengine"
"appengine/memcache"
)
func init() {
http.HandleFunc("/", hello)
http.HandleFunc("/hits", showHits)
}
func hello(w http.ResponseWriter, r *http.Request) {
name := r.Header.Get("name")
fmt.Fprintf(w, "Hello %s!", name)
}
func showHits(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "%d", hits(r))
}
func hits(r *http.Request) uint64 {
c := appengine.NewContext(r)
newValue, _ := memcache.Increment(c, "hits", 1, 0)
return newValue
}
According to the package docs:
func NoContent(w http.ResponseWriter, r *http.Request) {
// Set up any headers you want here.
w.WriteHeader(http.StatusNoContent) // send the headers with a 204 response code.
}
will send a 204 status to the client.
Sending 204 response from your script means your instance still need to run and cost you money. If you are looking for a caching solution. Google got it and it's called Edge Cache.
You only need to response with the following headers and Google will automatically cache your response in multiple servers nearest to the users (that is, replying with 204). This greatly enhance your site's speed and reduce instance cost.
w.Header().Set("Cache-Control", "public, max-age=86400")
w.Header().Set("Pragma", "Public")
You can adjust the max-age, but do it wisely.
By the way, it seems billing must be enabled in order to use Edge Cache
Where can one read logs created by calling function:
log.Print("Message")
The tab "Logs" under Main seems to only display information about what URLs were called, but without any debug information that would be displayed by the application.
As described in the documentation, you should use the Context interface to log instead of log, if you want your logs to show up in the console.
c := appengine.NewContext(r)
c.Infof("Requested URL: %v", r.URL)
If you are using the new App Engine package google.golang.org/appengine, in the README:
Logging methods that were on appengine.Context are now functions in google.golang.org/appengine/log
So you should use
c := appengine.NewContext(r)
log.Infof(c, "Requested URL: %v", r.URL)
The same context object must be passed around in other method calls.
Here is an example:
func handleSign(w http.ResponseWriter, r *http.Request) {
c := appengine.NewContext(r)
if err := r.ParseForm(); err != nil {
writeError(c, err)
return
}
}
func writeError(c appengine.Context, err os.Error) {
c.Errorf("%v", err)
}