For some reason nothing gets saved when the test code below is run. I have other api methods that when run (not through tests, this is just the first test) do save.
When I check the database stats via localhost:8000, it can be seen that nothing is being inserted.
Update: After copying and pasting the code below and wrapping it is GET request handler with some hardcoded data it does save to the database. So this seems like an issue with the testing aetest.Context that is used. I have added the code for the NewTestHandler helper code.
Method to create the context within the tests
func NewTestHandler(handlerFunc func(appengine.Context, http.ResponseWriter, *http.Request)) http.HandlerFunc {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
c, _ := aetest.NewContext(nil)
handlerFunc(c, w, r)
})
}
Error (update: the key that is generated returns 0 when calling .IntId())
// happens in the .Get() error handling
--- err datastore: internal error: server returned the wrong number of entities
Model
package app
import "time"
type League struct {
Name string `json:"name"`
Location string `json:"location"`
CreatedAt time.Time
}
Code
func (api *LeagueApi) Create(c appengine.Context, w http.ResponseWriter, r *http.Request) {
// data
var league League
json.NewDecoder(r.Body).Decode(&league)
defer r.Body.Close()
// save to db
key := datastore.NewIncompleteKey(c, "leagues", nil)
if _, err := datastore.Put(c, key, &league); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
var leagueCheck League
if err := datastore.Get(c, key, &leagueCheck); err != nil {
log.Println("--- err", err)
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
// json response
if err := json.NewEncoder(w).Encode(league); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
}
Test
func Test_LeagueReturnedOnCreate(t *testing.T) {
league := League{Name: "foobar"}
data, _ := json.Marshal(league)
reader := bytes.NewReader(data)
// setup request and writer
r, _ := http.NewRequest("POST", "/leagues", reader)
w := httptest.NewRecorder()
// make request
api := LeagueApi{}
handler := tux.NewTestHandler(api.Create)
handler.ServeHTTP(w, r)
// extract api response
var leagueCheck League
json.NewDecoder(w.Body).Decode(&leagueCheck)
if leagueCheck.Name != "foobar" {
t.Error("should return the league")
}
// ensure the league is in the db
}
Related
I want to run 2 goroutines parallel in App Engine, so that when the first goroutine finish its job, the handler doesn't need to wait the second goroutine - it stops the secend goroutine and returns the result to the client. Is this possible? I tried it with context.WithCancel(), but it didn't work (I use go1.6).
Here is my code:
package mytest
import (
"net/http"
"sync"
"time"
"golang.org/x/net/context"
"google.golang.org/appengine"
"google.golang.org/appengine/log"
"google.golang.org/appengine/urlfetch"
)
func init() {
http.HandleFunc("/test", handlerTest)
http.HandleFunc("/testwait10s", handlerTest10s)
http.HandleFunc("/testwait5s", handlerTest5s)
}
func handlerTest(w http.ResponseWriter, r *http.Request) {
c := appengine.NewContext(r)
ctx, _ := context.WithTimeout(c, 30*time.Second)
ctx1, ctx1Cancel := context.WithCancel(ctx)
ctx2, ctx2Cancel := context.WithCancel(ctx)
var wg sync.WaitGroup
wg.Add(2)
go func() {
defer wg.Done()
log.Infof(ctx1, "Go1 begin ...")
client1 := urlfetch.Client(ctx1)
_, err := client1.Get("http://APP_NAME.appspot.com/testwait5s")
if err != nil {
log.Errorf(ctx1, "Go1 failed: %v", err)
}
ctx2Cancel()
log.Infof(ctx1, "Go1 over ...")
}()
go func() {
defer wg.Done()
log.Infof(ctx2, "Go2 begin ...")
client2 := urlfetch.Client(ctx2)
_, err := client2.Get("http://APP_NAME.appspot.com/testwait10s")
if err != nil {
log.Errorf(ctx2, "Go2 failed %v", err)
}
ctx1Cancel()
log.Infof(ctx2, "Go2 over ...")
}()
wg.Wait()
log.Infof(ctx1, "Go1 and GO2 over")
}
func handlerTest10s(w http.ResponseWriter, r *http.Request) {
time.Sleep(10 * time.Second)
return
}
func handlerTest5s(w http.ResponseWriter, r *http.Request) {
time.Sleep(5 * time.Second)
return
}
Any ideas? Thanks!
Just create a notification channel and send there a signal that one of computations is over and you can proceed without waiting for the other.
func handlerTest(w http.ResponseWriter, r *http.Request) {
c := appengine.NewContext(r)
ctx, cancel := context.WithTimeout(c, 30*time.Second)
done := make(chan error, 2)
work := func(url, name string) {
log.Infof(ctx, "%s begin ...", name)
client := urlfetch.Client(ctx)
req, err := http.NewRequest(http.MethodGet, url, nil)
if err != nil {
log.Errorf(ctx, "%s failed: %v", name, err)
done <- err
return
}
req = req.WithContext(ctx)
_, err = client.Do(req)
done <- err
if err != nil {
log.Errorf(ctx, "%s failed: %v", name, err)
return
}
cancel()
log.Infof(ctx, "%s over ...", name)
}
go work("go1", "http://APP_NAME.appspot.com/testwait5s")
go work("go2", "http://APP_NAME.appspot.com/testwait10s")
for i := 0; i < cap(done); i++ {
if err := <-done; err == nil {
log.Infof(ctx, "executed without errors")
return
}
}
log.Error(ctx, "both computations have failed")
}
You can try reducing the value of wg.Add() to wg.Add(1) instead of wg.Add(2).
When one go-routine completes, wg.Done() will reduce the counter value by 1. So, In this case, the WaitGroup (wg) counter value will become ZERO. As a result, wg.Wait() on last line, will not wait for other go-routines to complete.
Note that, if the value of wg counter falls below zero, it will panic the remaining go-routines. So, the go-routines will be exited forcefully.
On app engine I have a large number of entities of a particular kind.
I want to run a function on each entity (e.g. edit the entity or copy it)
I would do this in a taskqueue but a taskqueue is limited to 10 minutes runtime and each function call is prone to many kinds of errors. What is the best way to do this?
Here's my solution although I'm hoping someone out there has a better solution. I also wonder if this is prone to fork bombs e.g. if the task runs twice, it will set off two chains of iteration.. ! I'm only using it to iterate a few hundred thousand entities, although the operation on each entity is expensive.
First I create a taskqueue for running each individual function call on an entity one at a time:
queue:
- name: entity-iter
rate: 100/s
max_concurrent_requests: 1
retry_parameters:
task_retry_limit: 3
task_age_limit: 30m
min_backoff_seconds: 200
and then I have an iterate entity method which, given the kind, will call your delay func on each entity with the key.
package sysadmin
import (
"google.golang.org/appengine/datastore"
"golang.org/x/net/context"
"google.golang.org/appengine/log"
"google.golang.org/appengine/delay"
"google.golang.org/appengine/taskqueue"
)
func ForEachEntity(kind string, f *delay.Function) *delay.Function {
var callWithNextKey *delay.Function // func(c context.Context, depth int, cursorString string) error
callWithNextKey = delay.Func("something", func(c context.Context, depth int, cursorString string) error {
q := datastore.NewQuery(kind).KeysOnly()
if cursorString != "" {
if curs, err := datastore.DecodeCursor(cursorString); err != nil {
log.Errorf(c, "error decoding cursor %v", err)
return err
} else {
q = q.Start(curs)
}
}
it := q.Run(c)
if key, err := it.Next(nil); err != nil {
if err == datastore.Done {
log.Infof(c, "Done %v", err)
return nil
}
log.Errorf(c, "datastore error %v", err)
return err
} else {
curs, _ := it.Cursor()
if t, err := f.Task(key); err != nil {
return err
} else if _, err = taskqueue.Add(c, t, "entity-iter"); err != nil {
log.Errorf(c, "error %v", err)
return err
}
if depth - 1 > 0 {
if err := callWithNextKey.Call(c, depth - 1, curs.String()); err != nil {
log.Errorf(c, "error2 %v", err)
return err
}
}
}
return nil
})
return callWithNextKey
}
example usage:
var DoCopyCourse = delay.Func("something2", CopyCourse)
var DoCopyCourses = ForEachEntity("Course", DoCopyCourse)
func CopyCourses(c context.Context) {
//sharedmodels.MakeMockCourses(c)
DoCopyCourses.Call(c, 9999999, "")
}
How do I connect a backend server in go lang to angularjs? I know angularjs communicates with go via $http or $resource services but what part of the go code links communicates with angular once all the data structs have been made? Would this be the encoded/marshalled json or do we create some kind of route...
I am a newbie looking to start this project after studying angular and go but this is the part i don't understand - what is the end point from go that angulars $resource or $http service links with?
Create a http endpoint that reads json string, unmarshalls it, does some logic with it and writes a json string to the response. E.g:
func main() {
http.HandleFunc("/api/", apiHandler)
logInfo(fmt.Sprintf("Starting server on port %d", serverPort))
err := http.ListenAndServe(fmt.Sprintf(":%d", serverPort), nil)
if err != nil {
logError(err)
}
}
func apiHandler(w http.ResponseWriter, r *http.Request) {
//ensure its a post
method := r.Method
if method != "POST" {
fail(w, "Invalid http method")
return
}
requestData, err := ioutil.ReadAll(r.Body)
if err != nil {
fail(w, err.Error())
return
}
respData, err := doBusinessLogicWithData(requestData)
if err != nil {
fail(w, err.Error())
return
}
respJSONBytes, err := json.Marshal(respData)
if err != nil {
fail(w, err.Error())
return
}
fmt.Fprintln(w, string(respJSONBytes))
}
func fail(w http.ResponseWriter, message string) {
result := &struct {
Success bool
Message string
}{
Success: false,
Message: message,
}
resultJSONBytes, err := json.Marshal(result)
if err != nil {
logError(err)
fmt.Fprintln(w, "Unable to generate result")
return
}
fmt.Fprintln(w, string(resultJSONBytes))
}
When I try to inject an appengine.Context from a middleware doing this:
//Share Context
m.Use(func(r *http.Request) {
c := appengine.NewContext(r)
c, err := appengine.Namespace(c, namespace)
if err != nil {
c.Debugf("[Namespace] %s", err)
}
m.Map(c)
})
I get this Panic saying that apparently there is no appengine.Context to be injected:
PANIC
Value not found for type appengine.Context
<pre>github.com/go-martini/martini/router.go:320 (0xc3731d)
(*routeContext).run: panic(err)
github.com/go-martini/martini/router.go:221 (0xc36729)
(*route).Handle: context.run()
github.com/go-martini/martini/router.go:112 (0xc35628)
(*router).Handle: route.Handle(context, res)
app/nc_backend.go:37 (0xc30fe0)
Router.Handle.fm: m.Action(r.Handle)
go/src/pkg/runtime/asm_amd64.s:340 (0xc23b82)
go/src/pkg/reflect/value.go:474 (0xd41bd3)
go/src/pkg/reflect/value.go:345 (0xd40c65)
github.com/codegangsta/inject/inject.go:102 (0xd8449a)
(*injector).Invoke: return reflect.ValueOf(f).Call(in), nil
github.com/go-martini/martini/martini.go:165 (0xc33607)
(*context).run: _, err := c.Invoke(c.handler())
github.com/go-martini/martini/martini.go:156 (0xc33500)
(*context).Next: c.run()
github.com/go-martini/martini/recovery.go:140 (0xc37a4b)
func.004: c.Next()
go/src/pkg/runtime/asm_amd64.s:339 (0xc23b22)
go/src/pkg/reflect/value.go:474 (0xd41bd3)
go/src/pkg/reflect/value.go:345 (0xd40c65)
github.com/codegangsta/inject/inject.go:102 (0xd8449a)
(*injector).Invoke: return reflect.ValueOf(f).Call(in), nil
github.com/go-martini/martini/martini.go:165 (0xc33607)
(*context).run: _, err := c.Invoke(c.handler())
github.com/go-martini/martini/martini.go:69 (0xc32b08)
(*Martini).ServeHTTP: m.createContext(res, req).run()
go/src/pkg/net/http/server.go:1496 (0xc98dda)
go/src/pkg/appengine_internal/api_prod.go:246 (0xc26e3f)
go/src/pkg/appengine_internal/api_prod.go:212 (0xc268c5)
go/src/pkg/runtime/asm_amd64.s:340 (0xc23b82)
go/src/pkg/reflect/value.go:474 (0xd41bd3)
go/src/pkg/reflect/value.go:345 (0xd40c65)
_:410 (0xcf6255)
go/src/pkg/runtime/proc.c:279 (0xc170a0)
What am I doing wrong?
appengine.Context is an interface, so you have to use the alternative MapTo. Also, according to the docs, mapping should be performed on the martini Context, not on the Martini object itself.
So your code should be:
m.Use(func(c martini.Context, req *http.Request) {
ctx := appengine.NewContext(req)
ctx, err := appengine.Namespace(ctx, namespace)
if err != nil {
ctx.Debugf("[Namespace] %s", err)
}
c.MapTo(ctx, (*appengine.Context)(nil))
})
I am trying to learn Go with GAE.
I have created 2 handlers. One for saving an object to datastore and the other retrieve it and output to screen. The problem is that when i retrieve the UserAccount object from datastore, every values inside the object are gone.
Any help would be appreciate.
Output:
a/c count: 2
val: core.UserAccount{idString:"", deviceId:""}
val: core.UserAccount{idString:"", deviceId:""}
type UserAccount struct {
idString string
deviceId string
}
func create_account(w http.ResponseWriter, r *http.Request) {
c := appengine.NewContext(r)
idstr := "ABCDEFGH"
devId := r.FormValue("deviceId")
newAccount := UserAccount{ idString: idstr, deviceId: devId,}
key := datastore.NewIncompleteKey(c, "UserAccount", nil)
_, err := datastore.Put(c, key, &newAccount)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
fmt.Fprintf(w, "val: %#v \n", newAccount)
}
func get_info(w http.ResponseWriter, r *http.Request) {
c := appengine.NewContext(r)
q := datastore.NewQuery("UserAccount")
accounts := make([]UserAccount, 0, 10)
if _, err := q.GetAll(c, &accounts); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
fmt.Fprintf(w, "a/c count: %v \n", len(accounts))
for i := 0; i < len(accounts); i++ {
fmt.Fprintf(w, "val: %#v \n", accounts[i])
}
}
If the datastore API uses reflection, which I presume it does, it cannot access struct fields that aren't exported, i.e. field names that do not begin with a capital letter.
Export them and it should work.