I'm trying to send requests to Apple's APNS service using their HTTP/2 API, and my service is working fine locally, but once it's on a Managed VM it seems the underlying sockets are dying after a few minutes and the Go HTTP library is unable to handle it gracefully.
What I see is the requests working fine (getting responses) for a while, but if it's idle for a few minutes the connections will take minutes to time out with read tcp 172.17.0.4:35395->17.172.234.19:443: read: connection timed out (apparently ignoring the 10 second timeout I specified).
I've previously had keep-alive issues with Managed VMs specifically, but Google has indicated it should be fixed. Does anyone know how to avoid this issue?
I'm creating an HTTP/2 client in this way:
func NewClient() *http.Client {
cert, err := tls.LoadX509KeyPair("secrets/prod_voip.pem", "secrets/prod_voip.key")
if err != nil {
log.Fatalln(err)
}
config := &tls.Config{
Certificates: []tls.Certificate{cert},
}
config.BuildNameToCertificate()
dialer := &net.Dialer{
Timeout: 10 * time.Second,
}
transport := &http.Transport{
Dial: dialer.Dial,
TLSClientConfig: config,
}
// Explicitly enable HTTP/2 as TLS-configured clients don't auto-upgrade.
// See: https://github.com/golang/go/issues/14275
if err := http2.ConfigureTransport(transport); err != nil {
log.Fatalln(err)
}
return &http.Client{Transport: transport}
}
Related
I am using Django channels with React websockets. I am unable to open multiple simulataneous websockets if the consumer of the first websocket channel is busy with some activity. I want to open multiple simultaneous websockets where consumers is doing something for each individual websocket.
The module versions are:
asgiref 3.6.0
daphne 3.0.2
django-cors-headers 3.13.0
djangorestframework 3.14.0
djangorestframework-simplejwt 5.2.2
From the snippet below, once one websocket completes the task the second websocket can be connected (in a separate tab) but while 1st websocket is busy (sleeping), the other websockets in the new browser tabs fails after handshake with the following error:
**
WebSocket HANDSHAKING /ws/socket-server/monitor-rack-kpis/ushoeuhrti/ [127.0.0.1:49228]
django.channels.server INFO WebSocket HANDSHAKING /ws/socket-server/monitor-rack-kpis/ushoeuhrti/ [127.0.0.1:49228]
daphne.http_protocol DEBUG Upgraded connection ['127.0.0.1', 49228] to WebSocket
daphne.ws_protocol DEBUG WebSocket closed for ['127.0.0.1', 49228]
WebSocket DISCONNECT /ws/socket-server/monitor-rack-kpis/ushoeuhrti/ [127.0.0.1:49228]
django.channels.server INFO WebSocket DISCONNECT /ws/socket-server/monitor-rack-kpis/ushoeuhrti/ [127.0.0.1:49228]
daphne.server WARNING Application instance <Task pending name='Task-56' coro=<StaticFilesWrapper.__call__() running at /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/channels/staticfiles.py:44> wait_for=<Future pending cb=[_chain_future.<locals>._call_check_cancel() at /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/futures.py:387, Task.task_wakeup()]>> for connection <WebSocketProtocol client=['127.0.0.1', 49216] path=b'/ws/socket-server/monitor-rack-kpis/vhofoefwmr/'> took too long to shut down and was killed.
**
I tried using url in routing instead of re_path but that didn't help either
settings.py
`
ASGI_APPLICATION = 'backend.asgi.application'
CHANNEL_LAYERS={'default':{'BACKEND':'channels.layers.InMemoryChannelLayer'}}
`
asgi.py
application = ProtocolTypeRouter({
'http':get_asgi_application(),
'websocket':AuthMiddlewareStack(
URLRouter(logstatus.routing.websocket_urlpatterns)
)
})
routing.py
websocket_urlpatterns=[
re_path(r'^ws/socket-server/monitor-rack-kpis/(?P<username>[A-Za-z]+)/', consumers.InfluxWritePromesthusConsumer.as_asgi())
]
consumer.py
class InfluxWritePromesthusConsumer(WebsocketConsumer):
def ws_connect(message):
message.reply_channel.send({"accept": True})
def receive(self, text_data):
print(f"\nReceiving from: {self.channel_name}\n")
t = 0
while t<=100:
self.send(text_data=json.dumps({
'type':"LearnSocket",
'message': "Received Messages"
}))
t += 10
time.sleep(10)
Frontend- React.JS
const randN = generate()
console.log(randN)
const socket = new WebSocket('ws://127.0.0.1:8000/ws/socket-server/monitor-rack-kpis/'+ randN+ '/');
console.log(socket)
socket.onopen=function(e){
console.log(socket.readyState)
if (user && user.access) {
socket.send("Hi");
//Receiving response from WebSocket
socket.onmessage=function(e){
//let data1=JSON.parse(e.data);
console.log(e.data, new Date());
//setOutputResponse(data1['message']);
}
}
}
I've been developing with Google App Engine Go for awhile, but I haven't touched it in a year. I tried updating my application from "go1" (1.9?) to "go111", and I'm currently getting some weird errors without any explanation as to what's going on.
The errors include:
The request failed because the instance could not start successfully
Container called exit(1).
500 Internal server error
etc.
None of these point me to any specific line in my code where something would go wrong, nor explain anything more meaningful...
I'm guessing the error is stemming from me upgrading between the golang versions. I had to change the app package into main, add a main function to the application, update the appengine package to a newer version, update the gsuite app, add a cloud compilation widget thingy, change app.yaml script from go to auto, etc.
All in all, I'm lost. A Similar SE question yielded no good answers. Someone else suggested app.yaml might be at fault, so here is mine:
runtime: go111
handlers:
- url: /static
static_dir: static
- url: /res
static_dir: res
- url: /update
script: auto
secure: always
login: admin
- url: /.*
script: auto
secure: always
login: optional
Debug console log is very unhelpful:
And the main file looks essentially like:
package main
import (
"fmt"
"google.golang.org/appengine"
"html/template"
"log"
"net/http"
)
var MainTemplate *template.Template
func main() {
http.HandleFunc("/", hello)
var err error
MainTemplate, err = template.ParseFiles("html/main.html")
if err != nil {
log.Printf("ERROR! %v\n", err.Error())
panic(err)
}
log.Printf("Hello world!\n")
}
func hello(w http.ResponseWriter, r *http.Request) {
c := appengine.NewContext(r)
//................................//
MainTemplate.Execute(w, nil)
}
Anything else it could be?
Okay, after some help from a few comments, here are a few issues that I had to fix to get my code to work:
I converted my init() function into main() without adding a way to listen to requests. The code just ran through the main() function and exited without an error, hence the problems with debugging.
appengine.NewContext(r) is deprecated apparently, so I had to switch those statements to r.Context(). MEANWHILE, Appengine Datastore is still using golang.org/x/net/context and not just context, so if you want to use things like RunInTransaction(), DON'T update your imports, context casts into golang.org/x/net/context just fine
If you follow the official examples provided by Google, you will most likely run into errors like textPayload: "2019/10/20 22:32:46 http: panic serving 127.0.0.1:16051: not an App Engine context. Instead, your code needs to look like the below example.
Since the former app package is now the main package, make sure that any files that are referenced in app.yaml (favicon.ico for example) are in a proper position in relation to the main package (I had to move mine to a different folder to avoid errors popping up every request...).
package main
import (
"google.golang.org/appengine"
"html/template"
"net/http"
)
var MainTemplate *template.Template
func init() {
http.HandleFunc("/", hello)
MainTemplate, _= template.ParseFiles("html/main.html")
}
func main() {
appengine.Main()
}
func hello(w http.ResponseWriter, r *http.Request) {
c := r.Context()
//................................//
MainTemplate.Execute(w, nil)
}
This ought to work. appengine.Main() apparently connects you to all the appengine functionality needed to use the context for datastore / memcache / whatever operations.
Thank you to the commenters that helped me get over the first hump!
Your code is missing the part to start a listener. Add this to your code:
port := os.Getenv("PORT")
if port == "" {
port = "8080"
log.Printf("Defaulting to port %s", port)
}
log.Fatal(http.ListenAndServe(fmt.Sprintf(":%s", port), nil))
I am trying to create an endpoint test when using appengine. Unfortunately, the tests keep failing because of the lack of a schema (and host) within the url used when creating the test *Request struct. When running appengine tests a server is spawned for that specific test that runs on a semi-random port number, which makes it seemingly impossible to define the full url to perform the test on.
The official docs on running tests like this are very sparse and only give half of an example, so I am left scratching my head on how to get this to work.
This is the error that I get from the marked line within the code snippet
Error: Received unexpected error "Post /auth: unsupported protocol scheme \"\""
Test Code
func TestEndpoints_Auth(t *testing.T) {
// input data
account := Account{
AuthProvider: "facebook",
AuthProviderId: "123345456",
}
b, _ := json.Marshal(&account)
reader := bytes.NewReader(b)
// test server
inst, err := aetest.NewInstance(nil)
if !assert.NoError(t, err) { return }
defer inst.Close()
// request
client := http.Client{}
req, err := inst.NewRequest("POST", "/auth", reader)
if !assert.NoError(t, err) { return }
req.Header.Add(AppAuthToken, "foobar")
resp, err := client.Do(req)
if !assert.NoError(t, err) { return } // <=== Where the error occurs
// tests
if !assert.Nil(t, err) { return }
assert.Equal(t, http.StatusCreated, resp.StatusCode)
}
Logs
[GIN-debug] POST /auth --> bitbucket.org/chrisolsen/chriscamp.(*endpoints).Auth-fm (5 handlers)
[GIN-debug] GET /accounts/me --> bitbucket.org/chrisolsen/chriscamp.(*endpoints).GetMyAccount-fm (7 handlers)
INFO 2016-04-22 13:23:39,278 devappserver2.py:769] Skipping SDK update check.
WARNING 2016-04-22 13:23:39,278 devappserver2.py:785] DEFAULT_VERSION_HOSTNAME will not be set correctly with --port=0
WARNING 2016-04-22 13:23:39,345 simple_search_stub.py:1126] Could not read search indexes from c:\users\chris\appdata\local\temp\appengine.testapp\search_indexes
INFO 2016-04-22 13:23:39,354 api_server.py:205] Starting API server at: http://localhost:54461
INFO 2016-04-22 13:23:41,043 dispatcher.py:197] Starting module "default" running at: http://localhost:54462
INFO 2016-04-22 13:23:41,046 admin_server.py:116] Starting admin server at: http://localhost:54466
I was really hoping to perform api blackbox tests, but that seems to be undoable with appengine. Instead I am now performing the tests on the endpoint directly.
req, _ := inst.NewRequest("POST", "/auth", reader)
req.Header.Add(AppAuthToken, "foobar")
resp := httptest.NewRecorder()
handlePostAuth(resp, req)
assert.Equal(t, http.StatusCreated, resp.Code)
package helloworld
import (
"fmt"
"net/http"
"appengine"
"appengine/user"
)
func init() {
fmt.Print("hello")
http.HandleFunc("/", handler)
}
func handler(w http.ResponseWriter, r *http.Request) {
c := appengine.NewContext(r)
u := user.Current(c)
if u == nil {
url, err := user.LoginURL(c, r.URL.String())
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
w.Header().Set("Location", url)
w.WriteHeader(http.StatusFound)
return
}
fmt.Fprintf(w, "Hello, %v!", u)
}
Throw the following error in goapp serve output
(saucy)adam#localhost:~/projects/ringed-land-605/default$ goapp serve -host 0.0.0.0 .
INFO 2014-06-08 23:57:47,862 devappserver2.py:716] Skipping SDK update check.
INFO 2014-06-08 23:57:47,877 api_server.py:171] Starting API server at: http://localhost:48026
INFO 2014-06-08 23:57:47,923 dispatcher.py:182] Starting module "default" running at: http://0.0.0.0:8080
INFO 2014-06-08 23:57:47,925 admin_server.py:117] Starting admin server at: http://localhost:8000
ERROR 2014-06-08 23:57:48,759 http_runtime.py:262] bad runtime process port ['hello46591\n']
Removing the fmt.Print() fixes the issue. My question is why does that happen?
When starting the runtime process, the Go Development Server (in the App Engine Go SDK) is reading the single line response found in your helloworld's init.
This modifies the _start_process_flavor flag in http_runtime.py; consequently, the HTTP runtime attempts to read the line for direction on which port to listen to.
Read the single line response expected in the start process file. [...] The START_PROCESS_FILE flavor uses a file for the runtime instance to report back the port it is listening on.
In this case, hello isn't a valid port to listen on.
Try using Go's log package instead:
log.Print("hello")
I'm getting a permission denied error when I make a call to another web service from within my go code.
part of the handler code on my server side go program:
client := http.Client{}
if resp, err := client.Get("http://api.wipmania.com/" + r.RemoteAddr); err != nil {
c.Errorf("Error getting location from ip: %s", err.Error())
}
From the logs:
Error getting location from ip: Get http://api.wipmania.com/30.30.30.30: permission denied
I saw a similar qn here. Still unable to figure it out. Can you please tell me what is the right way to do this and if any permissions have to be set on the appengine console?
App Engine applications can communicate with other applications and can access other resources on the web by fetching URLs. An app can use the URL Fetch service to issue HTTP and HTTPS requests and receive responses.
Try:
import (
"fmt"
"net/http"
"appengine"
"appengine/urlfetch"
)
func handler(w http.ResponseWriter, r *http.Request) {
c := appengine.NewContext(r)
client := urlfetch.Client(c)
if resp, err := client.Get("http://api.wipmania.com/" + r.RemoteAddr); err != nil {
c.Errorf("Error getting location from ip: %s", err.Error())
}
// ...
}