I'm getting a permission denied error when I make a call to another web service from within my go code.
part of the handler code on my server side go program:
client := http.Client{}
if resp, err := client.Get("http://api.wipmania.com/" + r.RemoteAddr); err != nil {
c.Errorf("Error getting location from ip: %s", err.Error())
}
From the logs:
Error getting location from ip: Get http://api.wipmania.com/30.30.30.30: permission denied
I saw a similar qn here. Still unable to figure it out. Can you please tell me what is the right way to do this and if any permissions have to be set on the appengine console?
App Engine applications can communicate with other applications and can access other resources on the web by fetching URLs. An app can use the URL Fetch service to issue HTTP and HTTPS requests and receive responses.
Try:
import (
"fmt"
"net/http"
"appengine"
"appengine/urlfetch"
)
func handler(w http.ResponseWriter, r *http.Request) {
c := appengine.NewContext(r)
client := urlfetch.Client(c)
if resp, err := client.Get("http://api.wipmania.com/" + r.RemoteAddr); err != nil {
c.Errorf("Error getting location from ip: %s", err.Error())
}
// ...
}
Related
There's a problem where I don't know why a context.Context was changed once I pass it to a different package on Google App Engine.
The following code works fine when running on App Engine:
package main
import (
"net/http"
"log"
"google.golang.org/appengine"
)
func main() {
http.HandleFunc("/", myHandler)
appengine.Main()
}
func myHandler(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
account, err := appengine.ServiceAccount(ctx)
if err != nil {
log.Println("[myHandler] error:", err)
} else {
log.Println("[myHandler] ServiceAccount:", account)
}
w.Write([]byte("ok"))
}
I could retrieve the ServiceAccount successfully when accessing /, and everything was good.
However, when I passed the context from main.go to another package, the function call didn't work. The following was added to main.go:
import (
// other stuff
"github.com/adlerhsieh/q_context/handlers"
)
func main() {
http.HandleFunc("/", myHandler)
appengine.Main()
}
func myHandler(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
account, err := appengine.ServiceAccount(ctx)
if err != nil {
log.Println("[myHandler] error:", err)
} else {
log.Println("[myHandler] ServiceAccount:", account)
}
handlers.AnotherFunc(ctx) // <--- added this
w.Write([]byte("ok"))
}
Another package:
package handlers
import (
"log"
"context"
"google.golang.org/appengine"
)
func AnotherFunc(ctx context.Context) {
account, err := appengine.ServiceAccount(ctx)
if err != nil {
log.Println("[AnotherFunc] error:", err)
} else {
log.Println("[AnotherFunc] ServiceAccount:", account)
}
}
When I ran it on App Engine, the log said:
2019/09/04 09:36:30 [myHandler] ServiceAccount: myaccount#gmail.com
2019/09/04 09:36:30 [AnotherFunc] error: not an App Engine context
The function calls are the same, but just in different packages. I dug in the package itself and found that it uses the key here (which leads to here) to setup the context. And here to check whether that value was setup properly. However, that value seem to be modified/changed so that the second function call couldn't get it. Even if I omitted the first function call and went straight to the second one, it still has the same error.
Any idea why context object was modified when passing to another package?
The following is my app.yaml:
runtime: go111
service: default
instance_class: F1
automatic_scaling:
min_idle_instances: 0
max_idle_instances: automatic
min_pending_latency: automatic
max_pending_latency: automatic
max_concurrent_requests: 30
handlers:
- url: /.*
script: auto
login: admin
nobuild_files:
- vendor
env_variables:
ENV: 'dev'
GO111MODULE: 'off'
Here is the GitHub repo link.
Thank you!
It turns out that my code actually worked. It's because of some other operation error.
However, I'll just post the issue that actually caused it so it can help those who have the same issue.
With the new go111 runtime, it treats packages from non-root directory or its subdirectories as a different type of package. This caused the problem with "not an App Engine context". I'll just call it an "outcast" package for now (cause I'm not entirely sure why's that).
For example:
- appengine
- main.go
- handlers
- handlers.go <-- this is a regular package
- appengine
- main.go
- handlers
- handlers.go < -- this is an outcast package
An outcast package would have issues handling context.Context generated from App Engine, as pointed out in my question.
The mechanism of App Engine knowing that the context is created from App Engine, is using a built-in value that can only be retrieved from its internal package (with an un-exported pointer-string key). When passing the context to an outcast package, we can no longer retrieve the value from the context. It's still a mystery for me that why the value disappeared, but it's probably because of some Go compiling mechanism.
The solution would be moving the main.go to the top-level directory in the project, so that there would be no outcast package anywhere.
I have an API, currently am trying to consume one of its endpoints. The endpoint is for POST requests, the endpoint is working as expected. The API is running in the cloud, I tested it with curl and it was perfect, then from my react app I was trying to consume it but I get 403 status code.
Watching in the console of the browser I see that I get that error on a OPTIONS request, and the POST never get done. Here is a screenshot of the result displayed in the console:
Then, I made a simple HTML file with a form, there I placed the required inputs, and the action pointing to this endpoint and it worked pretty well. Then, I don't know where would be the error? I have enabled CORS in the API
In the API I am using Gorilla/mux and I have something like this:
// Set up a router and some routes
r := mux.NewRouter()
r.HandleFunc("/", handleHome)
//some other routes
headersOk := handlers.AllowedHeaders([]string{"*"})
originsOk := handlers.AllowedOrigins([]string{"*"})
methodsOk := handlers.AllowedMethods([]string{"GET", "HEAD", "POST", "PUT", "OPTIONS"})
// Start http server
port := fmt.Sprintf(":%d", SomePort)
http.ListenAndServe(port, handlers.CORS(originsOk, headersOk, methodsOk)(r))
Using:
"github.com/gorilla/mux"
"github.com/gorilla/handlers"
The message that I am getting in the browser is (in Spanish):
Solicitud desde otro origen bloqueada: la política de mismo origen
impide leer el recurso remoto en https://miURL (razón: falta la
cabecera CORS 'Access-Control-Allow-Origin').
In English: basically the server is rejecting the request because the CORS header is not present.
So, what have I done wrong in my router configuration?
With rs/cors you should solve CORS issues pretty easily.
On your server.go
package main
import (
. . .
"fmt"
"log"
"net/http"
"github.com/gorilla/mux"
"github.com/rs/cors"
"../myhandler"
)
func main() {
fmt.Println("Settin up server, enabling CORS . . .")
c := cors.New(cors.Options{
AllowedOrigins: []string{"*"}, // All origins
AllowedMethods: []string{"GET"}, // Allowing only get, just an example
})
router := mux.NewRouter()
// Example handler
router.HandleFunc("/test", myhandler.TestHandler())
http.Handle("/", router)
// Bind to port 8000 and pass our router in and pass the cors Handler
log.Fatal(http.ListenAndServe(":8000"), c.Handler(router)))
fmt.Println("Server is ready and is listening at port :8000 . . .")
}
And on your testhandler.go, let's suppose you want to accept Content-Type: application/json
. . .
func TestHandler func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusOK)
return
}
I am trying to create an endpoint test when using appengine. Unfortunately, the tests keep failing because of the lack of a schema (and host) within the url used when creating the test *Request struct. When running appengine tests a server is spawned for that specific test that runs on a semi-random port number, which makes it seemingly impossible to define the full url to perform the test on.
The official docs on running tests like this are very sparse and only give half of an example, so I am left scratching my head on how to get this to work.
This is the error that I get from the marked line within the code snippet
Error: Received unexpected error "Post /auth: unsupported protocol scheme \"\""
Test Code
func TestEndpoints_Auth(t *testing.T) {
// input data
account := Account{
AuthProvider: "facebook",
AuthProviderId: "123345456",
}
b, _ := json.Marshal(&account)
reader := bytes.NewReader(b)
// test server
inst, err := aetest.NewInstance(nil)
if !assert.NoError(t, err) { return }
defer inst.Close()
// request
client := http.Client{}
req, err := inst.NewRequest("POST", "/auth", reader)
if !assert.NoError(t, err) { return }
req.Header.Add(AppAuthToken, "foobar")
resp, err := client.Do(req)
if !assert.NoError(t, err) { return } // <=== Where the error occurs
// tests
if !assert.Nil(t, err) { return }
assert.Equal(t, http.StatusCreated, resp.StatusCode)
}
Logs
[GIN-debug] POST /auth --> bitbucket.org/chrisolsen/chriscamp.(*endpoints).Auth-fm (5 handlers)
[GIN-debug] GET /accounts/me --> bitbucket.org/chrisolsen/chriscamp.(*endpoints).GetMyAccount-fm (7 handlers)
INFO 2016-04-22 13:23:39,278 devappserver2.py:769] Skipping SDK update check.
WARNING 2016-04-22 13:23:39,278 devappserver2.py:785] DEFAULT_VERSION_HOSTNAME will not be set correctly with --port=0
WARNING 2016-04-22 13:23:39,345 simple_search_stub.py:1126] Could not read search indexes from c:\users\chris\appdata\local\temp\appengine.testapp\search_indexes
INFO 2016-04-22 13:23:39,354 api_server.py:205] Starting API server at: http://localhost:54461
INFO 2016-04-22 13:23:41,043 dispatcher.py:197] Starting module "default" running at: http://localhost:54462
INFO 2016-04-22 13:23:41,046 admin_server.py:116] Starting admin server at: http://localhost:54466
I was really hoping to perform api blackbox tests, but that seems to be undoable with appengine. Instead I am now performing the tests on the endpoint directly.
req, _ := inst.NewRequest("POST", "/auth", reader)
req.Header.Add(AppAuthToken, "foobar")
resp := httptest.NewRecorder()
handlePostAuth(resp, req)
assert.Equal(t, http.StatusCreated, resp.Code)
I'm trying to send requests to Apple's APNS service using their HTTP/2 API, and my service is working fine locally, but once it's on a Managed VM it seems the underlying sockets are dying after a few minutes and the Go HTTP library is unable to handle it gracefully.
What I see is the requests working fine (getting responses) for a while, but if it's idle for a few minutes the connections will take minutes to time out with read tcp 172.17.0.4:35395->17.172.234.19:443: read: connection timed out (apparently ignoring the 10 second timeout I specified).
I've previously had keep-alive issues with Managed VMs specifically, but Google has indicated it should be fixed. Does anyone know how to avoid this issue?
I'm creating an HTTP/2 client in this way:
func NewClient() *http.Client {
cert, err := tls.LoadX509KeyPair("secrets/prod_voip.pem", "secrets/prod_voip.key")
if err != nil {
log.Fatalln(err)
}
config := &tls.Config{
Certificates: []tls.Certificate{cert},
}
config.BuildNameToCertificate()
dialer := &net.Dialer{
Timeout: 10 * time.Second,
}
transport := &http.Transport{
Dial: dialer.Dial,
TLSClientConfig: config,
}
// Explicitly enable HTTP/2 as TLS-configured clients don't auto-upgrade.
// See: https://github.com/golang/go/issues/14275
if err := http2.ConfigureTransport(transport); err != nil {
log.Fatalln(err)
}
return &http.Client{Transport: transport}
}
package helloworld
import (
"fmt"
"net/http"
"appengine"
"appengine/user"
)
func init() {
fmt.Print("hello")
http.HandleFunc("/", handler)
}
func handler(w http.ResponseWriter, r *http.Request) {
c := appengine.NewContext(r)
u := user.Current(c)
if u == nil {
url, err := user.LoginURL(c, r.URL.String())
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
w.Header().Set("Location", url)
w.WriteHeader(http.StatusFound)
return
}
fmt.Fprintf(w, "Hello, %v!", u)
}
Throw the following error in goapp serve output
(saucy)adam#localhost:~/projects/ringed-land-605/default$ goapp serve -host 0.0.0.0 .
INFO 2014-06-08 23:57:47,862 devappserver2.py:716] Skipping SDK update check.
INFO 2014-06-08 23:57:47,877 api_server.py:171] Starting API server at: http://localhost:48026
INFO 2014-06-08 23:57:47,923 dispatcher.py:182] Starting module "default" running at: http://0.0.0.0:8080
INFO 2014-06-08 23:57:47,925 admin_server.py:117] Starting admin server at: http://localhost:8000
ERROR 2014-06-08 23:57:48,759 http_runtime.py:262] bad runtime process port ['hello46591\n']
Removing the fmt.Print() fixes the issue. My question is why does that happen?
When starting the runtime process, the Go Development Server (in the App Engine Go SDK) is reading the single line response found in your helloworld's init.
This modifies the _start_process_flavor flag in http_runtime.py; consequently, the HTTP runtime attempts to read the line for direction on which port to listen to.
Read the single line response expected in the start process file. [...] The START_PROCESS_FILE flavor uses a file for the runtime instance to report back the port it is listening on.
In this case, hello isn't a valid port to listen on.
Try using Go's log package instead:
log.Print("hello")