Upstream prematurely closed connection while reading response header from upstream, client: - google-app-engine

I'd appreciate some help. I have deployed Go on Google App Engine. The server shuts down for some time which means we cannot get the messages coming in. The trigger is from the upstream prematurely closed connection while reading response header from upstream, client: The error:
2019/10/27 08:25:26 [error] 23#23: *166 upstream prematurely closed connection while reading response header from upstream, client: 169.x.x.x, server: _, request: "POST /api/v2/money/topup HTTP/1.1", upstream: "http://127.0.0.1:8081/api/v2/money/topup", host: “x.x.x”
{
insertId: "5db55476000502bed07e4583" 
labels: {
clone_id: "00c61b117c10abbe48ef11bb1f4ad5987e3ba1611745455daf2a1d6634c62289065e6f871e6c" 
}
logName: "projects/showcase/logs/%2Fvar%2Flog%2Fnginx%2Ferror.log" 
receiveTimestamp: "2019-10-27T08:25:26.334404825Z" 
resource: {
labels: {…} 
type: "gae_app" 
}
textPayload: "2019/10/27 08:25:26 [error] 23#23: *166 upstream prematurely closed connection while reading response header from upstream, client: 169.254.1.1, server: _, request: "POST /api/v2/money/topup HTTP/1.1", upstream: "http://127.0.0.1:8081/api/v2/money/topup", host: “x.x.x”” 
timestamp: "2019-10-27T08:25:26.328382Z" 
}
The image breakdown is here: error log
My Nginx.conf configuration setting:
events {
worker_connections 768;
}
http {
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 650;
keepalive_requests 10000;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
# Logs will appear on the Google Developer's Console when logged to this
# directory.
access_log /var/log/app_engine/app.log;
error_log /var/log/app_engine/app.log;
gzip on;
gzip_disable "msie6";
server {
# Google App Engine expects the runtime to serve HTTP traffic from
# port 8080.
listen 8080;
root /usr/share/nginx/www;
index index.html index.htm;
}
}
And for the app.yaml is:
runtime: go111
instance_class: F2
automatic_scaling:
min_idle_instances: 5
max_idle_instances: automatic # default value
min_pending_latency: 30ms # default value
max_pending_latency: automatic
max_concurrent_requests: 50
The function that is being called:
func (h Handler) UpdateTopUps(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
var payload model.C2BCallBack
defer r.Body.Close()
err := json.NewDecoder(r.Body).Decode(&payload)
if err != nil {
http.Error(w, "Bad Request", http.StatusBadRequest)
return
}
str := spew.Sdump(payload)
log.Println(str)
id := uuid.NewV4()
if err != nil {
fmt.Printf("Something went wrong: %s", err)
return
}
if len(payload.MSISDN) > 0 {
amount := strings.Split(payload.TransAmount, ".")
balance := strings.Split(payload.OrgAccountBalance, ".")
var phoneNumber string
if strings.HasPrefix(payload.MSISDN, "07") {
phoneNumber = "255" + payload.MSISDN[1:]
} else {
phoneNumber = payload.MSISDN
}
res := h.database.Create(&model.MpesaTransaction{MpesaTransactionId: id, TransID: payload.TransID, TransTime: payload.TransTime,
TransAmount: amount[0], BusinessShortCode: payload.BusinessShortCode, OrgAccountBalance: balance[0],
MSISDN: phoneNumber, FirstName: payload.FirstName, MiddleName: payload.MiddleName, LastName: payload.LastName})
if err != nil {
http.Error(w, "Internal Server Error", http.StatusInternalServerError)
return
}
h.updateWalletBalance(res.Value)
}
}
It works sometimes and then fails with the above error.

Related

Aws ec2 with lb does not return 101 on websocket

i have a problem with my app.
Basically while everything runs smooth on localhost, when i deploy to aws websocket connections does not work.
The problem is that when I make an wss request on ec2 it seems it does let the response to get out of the machine ,so instead of becoming 101 and stable a connection my app hangs on pending until 1 min pass and get a timeout, then it returns 200 but no connection establish.
Api is written in go lang with gin framework and app is written in js with React framework.
While asking some people for their advise they told me that I can "wrap" an wss request with https and pass it through lb and upgrade it with nginx, but I really cannot find anything similar.
Here is my backend code
main.go
{
ws.GET("/:id", func(c *gin.Context) {
websocket.ServeWs(c, pool)
})
}
websocket.go
ar upgrader = websocket.Upgrader{
ReadBufferSize: 1024,
WriteBufferSize: 1024,
CheckOrigin: func(r *http.Request) bool { return true },
}
func Upgrade(w http.ResponseWriter, r *http.Request) (*websocket.Conn, error) {
ws, err := upgrader.Upgrade(w, r, nil)
if err != nil {
log.Println(err)
return ws, err
}
return ws, nil
}
func ServeWs(c *gin.Context, pool *Pool) {
w := c.Writer
r := c.Request
conn, err := Upgrade(w, r)
if err != nil {
fmt.Fprintf(w, "%+V\n", err)
}
client := &Client{
Conn: conn,
Pool: pool,
ID: c.Param("id"),
}
pool.Register <- client
client.Read()
}
Here is react code
const socket = new WebSocket(
`${
process.env.REACT_APP_ENV === "dev"
? "ws://localhost:8080"
: "wss://api.blabla.prod"
}/ws/${sessionStorage.getItem("userID")}`
);
let WebSocketConnect = () => {
console.log("Attempting Connection...");
socket.onopen = () => {
console.log("Successfully Connected");
};
socket.onclose = (event) => {
console.log("Socket Closed Connection: ", event);
// WebSocketConnect();
};
socket.onerror = (error) => {
console.log("Socket Error: ", error);
};
};
let sendMsg = (msg) => {
// console.log("sending msg: ", msg);
socket.send(msg);
};
export { WebSocketConnect, sendMsg, socket };
Here is nginx config
server {
listen 80;
server_name api.blabla.prod;
if ($http_x_forwarded_proto = 'http') {
return 301 https://$server_name$request_uri;
}
location /ws {
proxy_pass "http://127.0.0.1:8080";
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
location / {
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $http_host;
proxy_pass "http://127.0.0.1:8080";
}
}
Thanks in advance!
The issue seems to be in your nginx configuration, I would replicate this for your /ws endpoint. Seems to be missing the $proxy_add_x_forwarded_for
server {
listen 80;
server_name goddit.pro; // change this
location / { // You can leave it at / or set it for another location, i.e. /ws
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-NginX-Proxy true;
proxy_pass http://localhost:8080/;
proxy_redirect off;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
The issue could also be in your javaScript, here's how you connect and upgrade.
window.onload = function () {
// Somewhere in your HTML document
var conn;
// Then you need to connect
function connectWs(id) {
if (window["WebSocket"]) {
conn = new WebSocket("wss://" + document.location.host + "/id/" + id);
conn.onopen = function (evt) {
var item = document.createElement("li");
item.innerHTML = "<em>Connected to " + id + "</em>";
log.appendChild(item);
};
conn.onclose = function (evt) {
var item = document.createElement("li");
item.innerHTML = "<em>Connection to " + id + " closed</em>";
log.appendChild(item);
};
conn.onmessage = function (evt) {
var message = JSON.parse(evt.data);
// console.log(message);
printMessage(message);
};
} else {
var item = document.createElement("li");
item.innerHTML = "<b>Your browser does not support WebSockets.</b>";
log.appendChild(item);
}
}
connectWs(id);
}
After you shared the error logs from the React console:
Close Code 1006 is a special code that means the connection was closed abnormally (locally) by the browser implementation.
If your browser client reports close code 1006, then you should be looking at the websocket.onerror(evt) event for details.

Docker REST API Golang

Hey guys i'm a new to docker, and i don't understand how to install my Golang application that have a routes, controller, and database config to Docker,after I searched on the internet, all I found was how to run 1 main.go file only, do i need to make route image, controller image, and database image to make my application run on docker?? or just make main.go image in docker to access all ?? sorry for my bad grammar.. i hope you guys understand what i'm talking about appriciate your help
Here my Main.go code
package main
import (
"dikota/config"
"dikota/models"
"dikota/routes"
"fmt"
"github.com/jinzhu/gorm"
)
var err error
func main() {
config.DB, err = gorm.Open("mysql", config.DbURL(config.BuildDBConfig()))
if err != nil {
fmt.Println("Status:", err)
}
//config.DB
defer config.DB.Close()
config.DB.AutoMigrate(&models.Konten{})
config.DB.AutoMigrate(&models.User{})
r := routes.SetupRouter()
//running
r.Run()
}
}
Here my routes code
package routes
import (
"dikota/controllers"
"time"
"github.com/gin-gonic/gin"
cors "github.com/itsjamie/gin-cors"
)
//SetupRouter Function
func SetupRouter() *gin.Engine {
router := gin.Default()
// Apply the middleware to the router (works with groups too)
router.Use(cors.Middleware(cors.Config{
Origins: "*",
Methods: "GET, PUT, POST, DELETE,OPTIONS",
RequestHeaders: "Origin, Authorization, Content-Type",
ExposedHeaders: "",
MaxAge: 50 * time.Second,
Credentials: true,
ValidateHeaders: false,
}))
// grp1 := r.Group("/user-api")
// {
// grp1.GET("user", controllers.GetUsers)
// grp1.POST("user", controllers.CreateUser)
// grp1.GET("user/:id", controllers.GetUserByID)
// grp1.PUT("user/:id", controllers.UpdateUser)
// grp1.DELETE("user/:id", controllers.DeleteUser)
// }
grp1 := router.Group("/konten-api")
{
grp1.GET("konten", controllers.GetKontens)
grp1.POST("createkonten", controllers.CreateKonten)
// grp1.GET("konten/:id", controllers.GetKontenByID)
// grp1.PUT("konten/:id", controllers.UpdateKonten)
// grp1.DELETE("konten/:id", controllers.DeleteKonten)
}
return router
}
Here my database config code
package config
import (
"fmt"
"github.com/jinzhu/gorm"
)
//DB variable
var DB *gorm.DB
//DBConfig struct
type DBConfig struct {
Host string
Port int
User string
DBName string
Password string
}
//BuildDBConfig function
func BuildDBConfig() *DBConfig {
dbConfig := DBConfig{
Host: "localhost",
Port: 3306,
User: "root",
Password: "dikota123",
DBName: "DIKOTA",
}
return &dbConfig
}
//DbURL function
func DbURL(dbConfig *DBConfig) string {
return fmt.Sprintf(
"%s:%s#tcp(%s:%d)/%s?charset=utf8&parseTime=True&loc=Local",
dbConfig.User,
dbConfig.Password,
dbConfig.Host,
dbConfig.Port,
dbConfig.DBName,
)
}

deploy go api does not send any response

I've created a Go API and deploy but when i see the logs there is no Println and no response is send to the client
the main:
func main() {
http.HandleFunc("/v1/", handler)
appengine.Main()
}
func handler(w http.ResponseWriter, r *http.Request) {
if r.Method != "GET" {
http.Error(w, "not allowed", http.StatusMethodNotAllowed)
return
}
...
}
the yaml file:
runtime: go
api_version: go1.8
handlers:
- url: /.*
script: _go_app
the url that i'm accessing:
myapp.appspot.com/v1/param
(what is this _go_app?)
the problem is that if i pass a "POST" request the error works just fine, but if i make a valid request the response take forever.
full code: https://goplay.space/#yiIp42FKzzj
In AppEngine you must use the AppEngine logger, passing in the context from the request.
Example from the godoc:
c := appengine.NewContext(r)
query := &log.Query{
AppLogs: true,
Versions: []string{"1"},
}
for results := query.Run(c); ; {
record, err := results.Next()
if err == log.Done {
log.Infof(c, "Done processing results")
break
}
if err != nil {
log.Errorf(c, "Failed to retrieve next log: %v", err)
break
}
log.Infof(c, "Saw record %v", record)
}

App engine push task always returns 404 in test

I've got a push task queue in a Go App Engine application. When we try to enqueue tasks in testing for whatever reason the tasks always return 404.
Our app.yaml:
runtime: go
api_version: go1.9
handlers:
- url: /worker/.*
script: _go_app
login: admin
- url: /.*
script: _go_app
The actual task invocation:
func Handler(w http.ResponseWriter, r *http.Request) {
ctx := appengine.NewContext(r)
t := taskqueue.NewPOSTTask("/worker", map[string][]string{"key": {"val"}})
_, err := taskqueue.Add(ctx, t, "")
if err != nil {
log.Errorf(ctx, "Failed to add task");
}
fmt.Fprintf(w, "Success");
}
A still-incomplete handler, but it exists!
func Worker(w http.ResponseWriter, r *http.Request) {
ctx := appengine.NewContext(r)
log.Infof(ctx, "Worker succeeded")
}
and finally, proof that we actually added the path to our router:
func init() {
http.HandleFunc("/", Handler)
http.HandleFunc("/worker", Worker)
}
When we actually run tests, we always get the following logging output:
INFO 2018-05-03 09:51:11,794 module.py:846] default: "POST /worker HTTP/1.1" 404 19
WARNING 2018-05-03 09:51:11,794 taskqueue_stub.py:2149] Task � failed to execute. This task will retry in 0.100 seconds
INFO 2018-05-03 09:51:11,897 module.py:846] default: "POST /worker HTTP/1.1" 404 19
WARNING 2018-05-03 09:51:11,897 taskqueue_stub.py:2149] Task � failed to execute. This task will retry in 0.200 seconds
INFO 2018-05-03 09:51:12,101 module.py:846] default: "POST /worker HTTP/1.1" 404 19
WARNING 2018-05-03 09:51:12,101 taskqueue_stub.py:2149] Task � failed to execute. This task will retry in 0.400 seconds
Note that the /worker endpoint returns 302 when I try to ping it via an API client like Paw, so the route seems to have been configured correctly. The 404 only arises when I try to run things in a test.
Why is this returning 404? I've tried running tests around the example push queue in their documentation have run into the same issue there - is there some sort of missing configuration flag I'm failing to pass to goapp?
I've pushed up a GitHub repo with a minimal replicable example here
The order for running goapp is top-bottom but you need to be specific in your app.yaml. In your case this will works:
package main
import (
"fmt"
"net/http"
"google.golang.org/appengine"
"google.golang.org/appengine/log"
"google.golang.org/appengine/taskqueue"
)
func Handler(w http.ResponseWriter, r *http.Request) {
ctx := appengine.NewContext(r)
t := taskqueue.NewPOSTTask("/worker", map[string][]string{"key": {"val"}})
_, err := taskqueue.Add(ctx, t, "")
if err != nil {
log.Errorf(ctx, "Failed to add task")
}
fmt.Fprintf(w, "Success")
}
func Worker(w http.ResponseWriter, r *http.Request) {
ctx := appengine.NewContext(r)
log.Infof(ctx, "Worker succeeded")
}
func init() {
http.HandleFunc("/", Handler)
http.HandleFunc("/worker", Worker)
}
For this you need to map url like:
runtime: go
api_version: go1.9
handlers:
- url: /worker
script: _go_app
login: admin
- url: /.*
script: _go_app
The result is:
See that worker is running two times. This is ocurring because GET /favicon.ico is entering in GET /.* mapping. So this is only details for you!
UPDATE (05/14/2018):
In your test you use aetest.NewInstance(), that runs dev_appserver.py in ioutil.TempDir("", "appengine-aetest"), that writes your own main.go and app.yaml. See above in instance_vm.go:
i.appDir, err = ioutil.TempDir("", "appengine-aetest")
if err != nil {
return err
}
defer func() {
if err != nil {
os.RemoveAll(i.appDir)
}
}()
err = os.Mkdir(filepath.Join(i.appDir, "app"), 0755)
if err != nil {
return err
}
err = ioutil.WriteFile(filepath.Join(i.appDir, "app", "app.yaml"), []byte(i.appYAML()), 0644)
if err != nil {
return err
}
err = ioutil.WriteFile(filepath.Join(i.appDir, "app", "stubapp.go"), []byte(appSource), 0644)
if err != nil {
return err
}
//... others codes
const appYAMLTemplate = `
application: %s
version: 1
runtime: go
api_version: go1
vm: true
handlers:
- url: /.*
script: _go_app
`
const appSource = `
package main
import "google.golang.org/appengine"
func main() { appengine.Main() }
`
So you need to create your own server-instance. This is a way:
//out buffer
var out bytes.Buffer
//start server
c := exec.Command("goapp", "serve") //default port=8080, adminPort=8000
c.Stdout = &out
c.Stderr = &out
c.Start()
defer c.Process.Kill()
//delay to wait server is completed
time.Sleep(10 * time.Second)
//... others codes
//quit server
quitReq, err := http.NewRequest("GET", "http://localhost:8000/quit", nil)
_, err := client.Do(quitReq)
if err != nil {
fmt.Errorf("GET /quit handler error: %v", err)
}
To test your Handler function do:
//create request (testing Handler func)
req, err := http.NewRequest("GET", "http://localhost:8080/", nil)
if err != nil {
t.Fatal(err.Error())
}
//do GET
client := http.DefaultClient
resp, err := client.Do(req)
if err != nil {
t.Error(err)
}
defer resp.Body.Close()
//delay to wait for the worker to execute
time.Sleep(10 * time.Second)
To retrieve the result and test it:
//read response
b, _ := ioutil.ReadAll(resp.Body)
resp_content := string(b)
//checking
if !strings.Contains(resp_content, "Handler Success") {
t.Errorf("Handler not working")
}
//log server content
logserver := out.String()
if !strings.Contains(logserver, "Worker succeeded") {
t.Errorf("Worker not working")
}
//log response
t.Logf(logserver)
The result is:
UPDATE: Link for Github: https://github.com/ag-studies/go-appengine-sample
Hope this help!!
Your handlers pattern is /worker/.* but you are issuing tasks to /worker.
You should do 1 of the following:
Either change pattern to /worker.* or just /worker
Or issue request(s) to /worker/ or /worker/some-task-name.

Golang google storage resumable upload HTTP 401

Hey im trying to implement a resumable upload to cloud storage .
But im getting a Status:"401 Unauthorized", StatusCode:401
And im assuming that it`s something with the bearer but i can't figure out another way to send the bearer Token.
I've been able to delete files with the GetClinet method.
func GetClinet(c endpoints.Context) *http.Client {
cli := &http.Client{
Transport: &oauth2.Transport{
Source: google.AppEngineTokenSource(c, storage.ScopeReadWrite),
Base: &urlfetch.Transport{Context: c},
},
}
return cli
}
client := GetClinet(con)
url := "https://storage.googleapis.com/bucketName/file.txt"
b := r.Header.Get("Authorization") //parse the bearer from user request
r, err = http.NewRequest("POST", url, nil)
r.Header.Add("Content-Type", "text/plain")
r.Header.Add("Content-Length", "0")
r.Header.Add("x-goog-resumable", "start")
r.Header.Add("Authorization", b)
resp, err := client.Do(r)
I'm having similar difficulties, not sure if your situation is the same as mine. My goal is to
User client uploads meta-data to my server
Server authenticates and stores in DB
Server asks Google Cloud Storage for resumabale upload url
Server returns url to client
Client uses url to upload media to GCS
I'm getting 401 unauthorized returns too. Maybe I'm not setting the headers correctly? or the storage.SignedURL doesn't handle resumable uploads properly...?
I've been able to upload files using a Service Account PEM file
func setGcsFile(c context.Context, fileId string, content []byte) error {
expiration := time.Now().Add(time.Second * 30) //expire in 30 seconds
data, err = ioutil.ReadFile(serviceAccountPEMFilename)
if err != nil {
fmt.Printf("error: %v", err)
panic(err)
}
log.Debugf(c, "Getting upload url")
opts := &storage.SignedURLOptions{
GoogleAccessID: googleAccessID,
PrivateKey: data,
Method: "PUT",
Expires: expiration,
}
putURL, err := storage.SignedURL(bucket, fileId, opts)
if err != nil {
log.Debugf(c, "%v", err)
return err
}
log.Debugf(c, "PUT URL : %v\n", putURL)
client := urlfetch.Client(c)
req, err := http.NewRequest("PUT", putURL, bytes.NewBuffer(content))
res, err := client.Do(req)
if err != nil {
log.Debugf(c, "%v", err)
return err
}
res.Body.Close()
log.Debugf(c, "Response Code: %s\n", res.Status)
return nil
}
I'm thinking about pulling and modifying the storage.SignedURL function next.

Resources