GAE Golang - urlfetch timeout? - google-app-engine

Question

I am having issues with urlfetch's timeouts on Google App Engine in Go. The app does not appear to want to take a longer timeout than about 5 seconds (it ignores a longer timeout and times out after its own time).
My code is:
var TimeoutDuration time.Duration = time.Second*30
func Call(c appengine.Context, address string, allowInvalidServerCertificate bool, method string, id interface{}, params []interface{})(map[string]interface{}, error){
data, err := json.Marshal(map[string]interface{}{
"method": method,
"id": id,
"params": params,
})
if err != nil {
return nil, err
}
req, err:=http.NewRequest("POST", address, strings.NewReader(string(data)))
if err!=nil{
return nil, err
}
tr := &urlfetch.Transport{Context: c, Deadline: TimeoutDuration, AllowInvalidServerCertificate: allowInvalidServerCertificate}
resp, err:=tr.RoundTrip(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, err
}
result := make(map[string]interface{})
err = json.Unmarshal(body, &result)
if err != nil {
return nil, err
}
return result, nil
}
No matter what I try to set TimeoutDuration to, the app times out after about 5 seconds. How prevent it from doing that? Did I make some error in my code?

Answer 1

You need to pass the time duration like this (otherwise it will default to the 5 sec timeout):
tr := &urlfetch.Transport{Context: c, Deadline: time.Duration(30) * time.Second}
Update Jan 2 2016:
With the new GAE golang packages (google.golang.org/appengine/*), this has changed. urlfetch no longer receives a deadline time duration in the transport.
You should now set the timeout via the new context package. For example, this is how you would set a 1 minute deadline:
func someFunc(ctx context.Context) {
ctx_with_deadline, _ := context.WithTimeout(ctx, 1*time.Minute)
client := &http.Client{
Transport: &oauth2.Transport{
Base: &urlfetch.Transport{Context: ctx_with_deadline},
},
}

Answer 2

Try the code below:
// createClient is urlfetch.Client with Deadline
func createClient(context appengine.Context, t time.Duration) *http.Client {
return &http.Client{
Transport: &urlfetch.Transport{
Context: context,
Deadline: t,
},
}
}
Here is how to use it.
// urlfetch
client := createClient(c, time.Second*60)
Courtesy #gosharplite

Answer 3

Looking at the source code of Go's appengine:
http://code.google.com/p/appengine-go/source/browse/appengine/urlfetch/urlfetch.go
and the protobuffer generated code:
http://code.google.com/p/appengine-go/source/browse/appengine_internal/urlfetch/urlfetch_service.pb.go
Looks like there should not be a problem with Duration itself.
My guess is that the whole application inside appengine timeouts after 5 seconds.

Answer 4

for me, this worked:
ctx_with_deadline, _ := context.WithTimeout(ctx, 15*time.Second)
client := urlfetch.Client(ctx_with_deadline)

Answer 5

This is had now changed with the recent updates to the library . Now the Duration of timeout/delay have to carried by the context , urlfetch.transport no more has the Deadline field in it . context.WithTimeout or context.WithDeadline is the method to use , here is the link https://godoc.org/golang.org/x/net/context#WithTimeout