I'm new to Google AppEngine & to Go & to Web stacks - so it is very likely something I'm doing wrong here, but I can't figure it out :)
I'm trying to write a simple clock app in Go where the the server pushes to the client & a JS handler updates an HTML element. I'm using the TaskQueue api to reschedule the server side update, and I've changed the default queue frequency to be once per second. The server sends a payload string that contains the number of times the task URL has been hit & the current time. The hit count updates at a rate of 1 per second, as expected - but the time stamp barely changes. The log output I'm seeing in the dev_appserver console looks correct
INFO 2012-07-05 03:04:31,955 dev_appserver.py:2952] "POST /update HTTP/1.1" 200 -
INFO 2012-07-05 03:04:31,985 dev_appserver.py:2952] "POST /update HTTP/1.1" 200 -
INFO 2012-07-05 03:04:32,015 dev_appserver.py:2952] "POST /update HTTP/1.1" 200 -
INFO 2012-07-05 03:04:32,043 dev_appserver.py:2952] "POST /update HTTP/1.1" 200 -
I've pasted my code below (is a pastebin better?).
Cheers!
Brad
package clock
import (
"fmt"
"appengine"
"appengine/channel"
"appengine/datastore"
"appengine/taskqueue"
"html/template"
"net/http"
"time"
// "appengine/user"
)
type Greeting struct {
Author string
Content string
Date time.Time
}
func init() {
http.HandleFunc("/", root)
http.HandleFunc("/update", update)
}
type TemplateFiller struct {
Time string
Token string
}
var clientId string = "clockclient"
func root(w http.ResponseWriter, r *http.Request) {
c := appengine.NewContext(r)
q := datastore.NewQuery("Greeting").Order("-Date").Limit(10)
greetings := make([]Greeting, 0, 10)
if _, err := q.GetAll(c, &greetings); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
token, _ := channel.Create(c, clientId);
tf := TemplateFiller{time.Now().String(), token}
if err := guestbookTemplate.Execute(w, tf); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
reschedule(c)
}
func reschedule(c appengine.Context) {
t := taskqueue.NewPOSTTask("/update", map[string][]string{"token": {clientId}})
if _, err := taskqueue.Add(c, t, ""); err != nil {
return
}
}
var hits int = 0
func update(w http.ResponseWriter, r *http.Request) {
c := appengine.NewContext(r)
channel.Send(c, clientId, fmt.Sprintln(hits) + time.Now().Format(time.RFC3339))
hits = hits + 1
//channel.Send(c, clientId, time.Now().Format(time.RFC3339))
reschedule(c)
}
var guestbookTemplate = template.Must(template.New("").Parse(guestbookTemplateHTML))
const guestbookTemplateHTML = `
<html>
<script type="text/javascript" src="/_ah/channel/jsapi"></script>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.7.2/jquery.min.js"></script>
<body>
<div id="timediv">No time</div>
<script type="text/javascript">
$(document).ready(function(){
onMessage = function(msg) {
$("#timediv").html(msg.data);
}
channel = new goog.appengine.Channel('{{.Token}}');
socket = channel.open();
//socket.onopen = onOpened;
socket.onmessage = onMessage;
//socket.onerror = onError;
//socket.onclose = onClose;
});
</script>
</body>
</html>
`
The dev_server queue and timing behaves nothing like how the production server behaves (timing wise).
It is a very bad idea to depend on the queue to give you a constat and stable timer interup.
Related
I've got a push task queue in a Go App Engine application. When we try to enqueue tasks in testing for whatever reason the tasks always return 404.
Our app.yaml:
runtime: go
api_version: go1.9
handlers:
- url: /worker/.*
script: _go_app
login: admin
- url: /.*
script: _go_app
The actual task invocation:
func Handler(w http.ResponseWriter, r *http.Request) {
ctx := appengine.NewContext(r)
t := taskqueue.NewPOSTTask("/worker", map[string][]string{"key": {"val"}})
_, err := taskqueue.Add(ctx, t, "")
if err != nil {
log.Errorf(ctx, "Failed to add task");
}
fmt.Fprintf(w, "Success");
}
A still-incomplete handler, but it exists!
func Worker(w http.ResponseWriter, r *http.Request) {
ctx := appengine.NewContext(r)
log.Infof(ctx, "Worker succeeded")
}
and finally, proof that we actually added the path to our router:
func init() {
http.HandleFunc("/", Handler)
http.HandleFunc("/worker", Worker)
}
When we actually run tests, we always get the following logging output:
INFO 2018-05-03 09:51:11,794 module.py:846] default: "POST /worker HTTP/1.1" 404 19
WARNING 2018-05-03 09:51:11,794 taskqueue_stub.py:2149] Task � failed to execute. This task will retry in 0.100 seconds
INFO 2018-05-03 09:51:11,897 module.py:846] default: "POST /worker HTTP/1.1" 404 19
WARNING 2018-05-03 09:51:11,897 taskqueue_stub.py:2149] Task � failed to execute. This task will retry in 0.200 seconds
INFO 2018-05-03 09:51:12,101 module.py:846] default: "POST /worker HTTP/1.1" 404 19
WARNING 2018-05-03 09:51:12,101 taskqueue_stub.py:2149] Task � failed to execute. This task will retry in 0.400 seconds
Note that the /worker endpoint returns 302 when I try to ping it via an API client like Paw, so the route seems to have been configured correctly. The 404 only arises when I try to run things in a test.
Why is this returning 404? I've tried running tests around the example push queue in their documentation have run into the same issue there - is there some sort of missing configuration flag I'm failing to pass to goapp?
I've pushed up a GitHub repo with a minimal replicable example here
The order for running goapp is top-bottom but you need to be specific in your app.yaml. In your case this will works:
package main
import (
"fmt"
"net/http"
"google.golang.org/appengine"
"google.golang.org/appengine/log"
"google.golang.org/appengine/taskqueue"
)
func Handler(w http.ResponseWriter, r *http.Request) {
ctx := appengine.NewContext(r)
t := taskqueue.NewPOSTTask("/worker", map[string][]string{"key": {"val"}})
_, err := taskqueue.Add(ctx, t, "")
if err != nil {
log.Errorf(ctx, "Failed to add task")
}
fmt.Fprintf(w, "Success")
}
func Worker(w http.ResponseWriter, r *http.Request) {
ctx := appengine.NewContext(r)
log.Infof(ctx, "Worker succeeded")
}
func init() {
http.HandleFunc("/", Handler)
http.HandleFunc("/worker", Worker)
}
For this you need to map url like:
runtime: go
api_version: go1.9
handlers:
- url: /worker
script: _go_app
login: admin
- url: /.*
script: _go_app
The result is:
See that worker is running two times. This is ocurring because GET /favicon.ico is entering in GET /.* mapping. So this is only details for you!
UPDATE (05/14/2018):
In your test you use aetest.NewInstance(), that runs dev_appserver.py in ioutil.TempDir("", "appengine-aetest"), that writes your own main.go and app.yaml. See above in instance_vm.go:
i.appDir, err = ioutil.TempDir("", "appengine-aetest")
if err != nil {
return err
}
defer func() {
if err != nil {
os.RemoveAll(i.appDir)
}
}()
err = os.Mkdir(filepath.Join(i.appDir, "app"), 0755)
if err != nil {
return err
}
err = ioutil.WriteFile(filepath.Join(i.appDir, "app", "app.yaml"), []byte(i.appYAML()), 0644)
if err != nil {
return err
}
err = ioutil.WriteFile(filepath.Join(i.appDir, "app", "stubapp.go"), []byte(appSource), 0644)
if err != nil {
return err
}
//... others codes
const appYAMLTemplate = `
application: %s
version: 1
runtime: go
api_version: go1
vm: true
handlers:
- url: /.*
script: _go_app
`
const appSource = `
package main
import "google.golang.org/appengine"
func main() { appengine.Main() }
`
So you need to create your own server-instance. This is a way:
//out buffer
var out bytes.Buffer
//start server
c := exec.Command("goapp", "serve") //default port=8080, adminPort=8000
c.Stdout = &out
c.Stderr = &out
c.Start()
defer c.Process.Kill()
//delay to wait server is completed
time.Sleep(10 * time.Second)
//... others codes
//quit server
quitReq, err := http.NewRequest("GET", "http://localhost:8000/quit", nil)
_, err := client.Do(quitReq)
if err != nil {
fmt.Errorf("GET /quit handler error: %v", err)
}
To test your Handler function do:
//create request (testing Handler func)
req, err := http.NewRequest("GET", "http://localhost:8080/", nil)
if err != nil {
t.Fatal(err.Error())
}
//do GET
client := http.DefaultClient
resp, err := client.Do(req)
if err != nil {
t.Error(err)
}
defer resp.Body.Close()
//delay to wait for the worker to execute
time.Sleep(10 * time.Second)
To retrieve the result and test it:
//read response
b, _ := ioutil.ReadAll(resp.Body)
resp_content := string(b)
//checking
if !strings.Contains(resp_content, "Handler Success") {
t.Errorf("Handler not working")
}
//log server content
logserver := out.String()
if !strings.Contains(logserver, "Worker succeeded") {
t.Errorf("Worker not working")
}
//log response
t.Logf(logserver)
The result is:
UPDATE: Link for Github: https://github.com/ag-studies/go-appengine-sample
Hope this help!!
Your handlers pattern is /worker/.* but you are issuing tasks to /worker.
You should do 1 of the following:
Either change pattern to /worker.* or just /worker
Or issue request(s) to /worker/ or /worker/some-task-name.
I have created a go application that sends an email using net/smtp and PlainAuth. And everything is working fine. But when I deploy this same code on the google app engine it is not sending. If I serve using goapp serve it work locally. Below is the code:
func sendMail(body string, subject string) {
var err error
var doc bytes.Buffer
auth := smtp.PlainAuth("",
"from#gmail.com",
"********",
"smtp.gmail.com")
parameters := &SmtpTemplateData{
"from#gmail.com",
"to#gmail.com",
"Subject",
body,
}
buffer := new(bytes.Buffer)
template := template.Must(template.New("emailTemplate").Parse(emailScript()))
template.Execute(buffer, parameters)
t := template.New("emailTemplate1")
t, err = t.Parse(emailTemplate)
if err != nil {
fmt.Println("error trying to parse mail template")
}
err = t.Execute(&doc, parameters)
smtp.SendMail(
"smtp.gmail.com:587",
auth,
"from#gmail.com",
[]string{"to#gmail.com"},
doc.Bytes(),
)
}
Hey im trying to implement a resumable upload to cloud storage .
But im getting a Status:"401 Unauthorized", StatusCode:401
And im assuming that it`s something with the bearer but i can't figure out another way to send the bearer Token.
I've been able to delete files with the GetClinet method.
func GetClinet(c endpoints.Context) *http.Client {
cli := &http.Client{
Transport: &oauth2.Transport{
Source: google.AppEngineTokenSource(c, storage.ScopeReadWrite),
Base: &urlfetch.Transport{Context: c},
},
}
return cli
}
client := GetClinet(con)
url := "https://storage.googleapis.com/bucketName/file.txt"
b := r.Header.Get("Authorization") //parse the bearer from user request
r, err = http.NewRequest("POST", url, nil)
r.Header.Add("Content-Type", "text/plain")
r.Header.Add("Content-Length", "0")
r.Header.Add("x-goog-resumable", "start")
r.Header.Add("Authorization", b)
resp, err := client.Do(r)
I'm having similar difficulties, not sure if your situation is the same as mine. My goal is to
User client uploads meta-data to my server
Server authenticates and stores in DB
Server asks Google Cloud Storage for resumabale upload url
Server returns url to client
Client uses url to upload media to GCS
I'm getting 401 unauthorized returns too. Maybe I'm not setting the headers correctly? or the storage.SignedURL doesn't handle resumable uploads properly...?
I've been able to upload files using a Service Account PEM file
func setGcsFile(c context.Context, fileId string, content []byte) error {
expiration := time.Now().Add(time.Second * 30) //expire in 30 seconds
data, err = ioutil.ReadFile(serviceAccountPEMFilename)
if err != nil {
fmt.Printf("error: %v", err)
panic(err)
}
log.Debugf(c, "Getting upload url")
opts := &storage.SignedURLOptions{
GoogleAccessID: googleAccessID,
PrivateKey: data,
Method: "PUT",
Expires: expiration,
}
putURL, err := storage.SignedURL(bucket, fileId, opts)
if err != nil {
log.Debugf(c, "%v", err)
return err
}
log.Debugf(c, "PUT URL : %v\n", putURL)
client := urlfetch.Client(c)
req, err := http.NewRequest("PUT", putURL, bytes.NewBuffer(content))
res, err := client.Do(req)
if err != nil {
log.Debugf(c, "%v", err)
return err
}
res.Body.Close()
log.Debugf(c, "Response Code: %s\n", res.Status)
return nil
}
I'm thinking about pulling and modifying the storage.SignedURL function next.
I am having trouble getting the AppEngine Blobstore upload sample to work. Looking further, I see that Blobstore.Send() always returns HTTP 206 "Partial response".
package blobstore_example
import (
"html/template"
"io"
"net/http"
"appengine"
"appengine/blobstore"
)
func serveError(c appengine.Context, w http.ResponseWriter, err error) {
w.WriteHeader(http.StatusInternalServerError)
w.Header().Set("Content-Type", "text/plain")
io.WriteString(w, "Internal Server Error")
c.Errorf("%v", err)
}
var rootTemplate = template.Must(template.New("root").Parse(rootTemplateHTML))
const rootTemplateHTML = `
<html><body>
<form action="{{.}}" method="POST" enctype="multipart/form-data">
Upload File: <input type="file" name="file"><br>
<input type="submit" name="submit" value="Submit">
</form></body></html>
`
func handleRoot(w http.ResponseWriter, r *http.Request) {
c := appengine.NewContext(r)
uploadURL, err := blobstore.UploadURL(c, "/upload", nil)
if err != nil {
serveError(c, w, err)
return
}
w.Header().Set("Content-Type", "text/html")
err = rootTemplate.Execute(w, uploadURL)
if err != nil {
c.Errorf("%v", err)
}
}
func handleServe(w http.ResponseWriter, r *http.Request) {
blobstore.Send(w, appengine.BlobKey(r.FormValue("blobKey")))
}
func handleUpload(w http.ResponseWriter, r *http.Request) {
c := appengine.NewContext(r)
blobs, _, err := blobstore.ParseUpload(r)
if err != nil {
serveError(c, w, err)
return
}
file := blobs["file"]
if len(file) == 0 {
c.Errorf("no file uploaded")
http.Redirect(w, r, "/", http.StatusFound)
return
}
http.Redirect(w, r, "/serve/?blobKey="+string(file[0].BlobKey), http.StatusFound)
}
func init() {
http.HandleFunc("/", handleRoot)
http.HandleFunc("/serve/", handleServe)
http.HandleFunc("/upload", handleUpload)
}
Firefox 32 does not play this and says:
Video can't be played because the file is corrupt.
Yet, if I use wget to save the file I get the exact mp4 file with all bytes included - and local player plays the mp4 fine.
Capturing the HTTP packets I see that Firefox is not asking for a byte-range in the HTTP Request - yet the Blobstore response includes:
Content-Range: bytes 0-245778/245779\r\n
(which by the way is the full size of the mp4 file).
So why does Blobstore think it has to return HTTP 206? ...and how to get it not to?
My following code
package inqzincrm
import (
"github.com/gorilla/pat"
"github.com/gorilla/sessions"
"net/http"
)
var store = sessions.NewCookieStore([]byte("X12h8v6BZC4QJl53KfNLshtr85gkC5OZ"), []byte("X12h8vasdf6BZC4QJl53KfNLshtr85gk"))
func init() {
r := pat.New()
r.Get("/", Home)
http.Handle("/", r)
}
and in handler,
package inqzincrm
import (
"appengine"
"html/template"
"net/http"
)
var aTmplt = template.Must(template.ParseFiles(
"inqzincrm/templates/base.html",
"inqzincrm/templates/index.html",
))
func Home(w http.ResponseWriter, r *http.Request) {
c := appengine.NewContext(r)
session, err := store.Get(r, "x")
c.Infof("HOST: %s", r.Host)
if session.IsNew {
session.Options.Domain = r.Host
session.Options.Path = "/"
session.Options.MaxAge = 0
session.Options.HttpOnly = false
session.Options.Secure = false
}
if err != nil {
c.Infof("Error getting session: %v", err)
}
c.Infof("Requested URL: %v", session.Values["foo"])
session.Values["foo"] = "asdf"
if err := aTmplt.ExecuteTemplate(w, "index.html", nil); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
session.Save(r, w)
}
do not set any cookie on browser side. how ever result and err are nil indicating no problem with the functions.
How I should debug further ?
I am using Ubuntu 64 bit, Google App Engine, Go with Gorilla tool kit.
Have a look at my answer to a previous question here.
In short form:
import (
"github.com/gorilla/sessions"
"net/http"
)
// Authorization Key
var authKey = []byte("somesecret")
// Encryption Key
var encKey = []byte("someothersecret")
var store = sessions.NewCookieStore(authKey, encKey)
func initSession(r *http.Request) *sessions.Session {
session, _ := store.Get(r, "my_cookie") // Don't ignore the error in real code
if session.IsNew { //Set some cookie options
session.Options.Domain = "example.org"
session.Options.MaxAge = 0
session.Options.HttpOnly = false
session.Options.Secure = true
}
return session
}
Then, in your handlers:
func ViewPageHandler(w http.ResponseWriter, r *http.Request) {
session := initSession(r)
session.Values["page"] = "view"
session.Save(r, w)
....
Your code seems to do the same thing, so without more of an example I can't see any problem with it. I can say that the example I've posted is (very slightly modified) from a working server.
the code session.save() should come before template execution code. it has been mentioned no where. but that was the whole mistake.