We have a Golang application running in the Google App Engine, which reads a signing key using cloud.google.com/go/storage. When trying to deploy a newly updated version, it started receiving errors saying that it could not find default credentials, and thus not access the storage.
bucketName, err := file.DefaultBucketName(ctx)
if err != nil {
log.Errorf(ctx, "failed to get default GCS bucket name: %v", err)
return nil, nil, err
}
client, err = storage.NewClient(ctx)
if err != nil {
log.Errorf(ctx, "failed to create client: %v", err)
return nil, nil, err
}
(ctx is the context.Context we create using appengine.NewContext() with the incoming http.Request object as parameter; file is google.golang.org/appengine/file)
This yielded this error log:
"2020-06-15 09:16:51.809 CEST
failed to create client: dialing: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information"
Tracing the error through the golang libraries, we found that it seems that the application fails to resolve the metadata hostname that is supposed to be set when this runs on the Google Cloud platform.
We have worked around the issue by adding this to the app.yaml file, but it feels wrong to have to do that:
env_variables:
GCE_METADATA_HOST: "169.254.169.254"
This forces testOnGCE() in metadata.go (https://github.com/googleapis/google-cloud-go/blob/master/compute/metadata/metadata.go#L117) to return true, and lets us read the signing key from storage.
Is this a bug in the Google Cloud platform that we need to work around, or is there anything wrong with the code above? It has worked fine since we added it in 2018 and only started failing when deploying code in June 2020, and the deploy that broke only contained changes to the client JavaScript code, the golang backend was not touched. We tried redeploying it several times, but the error message was 100 % reproducible until we added the workaround.
Related
I am working on connecting to the Facebook API using MongoDB.
I am receiving an error that states: "server selection error: server selection timeout, current topology: { Type: Unknown, Servers: [{ Addr: localhost:27017, Type: Unknown, Last error: connection() error occured during connection handshake: dial tcp [::1]:27017: connectex: No connection could be made because the target machine actively refused it. }, ] }"
In attempts to fix this error, I have tried switching to servers "mongodb://localhost:27018" and "mongodb://localhost:27019"
I have also scanned my code for errors but can't find anything. I am still very new to programming, so any help or additional advice/ knowledge is appreciated.
Code Below:
package main
import (
"context"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
"log"
)
// Client is exported Mongo Database client
var Client *mongo.Client
// ConnectDatabase is used to connect the MongoDB database
func ConnectDatabase() {
log.Println("Database connecting...")
// Set client options
clientOptions := options.Client().ApplyURI("mongodb://localhost:27017")
// Connect to MongoDB
client,err := mongo.Connect(context.TODO(), clientOptions)
Client = client
if err != nil{
log.Fatal(err)
}
// Check the connection
err = Client.Ping(context.TODO(), nil)
if err != nil {
log.Fatal(err)
}
log.Println("Database Connected.")
}
Found the incredibly simple answer. I hadn't downloaded the MongoDB community server and installed it as a service!
After installing I ran my code with http://localhost:8000/ and it worked immediately.
Link here: https://www.mongodb.com/try/download/community
I am in the process of upgrading my golang AppEngine app from Go v1.9 to Go v1.12. As part of that, I'm replacing the appengine specific packages for accessing the datastore with the standard GCP datastore client library.
Now when I run my code, any call to get or set data in the datastore hangs with no error. In a debugging attempt I added a timeout to the context; when the request times out I get the error "latest connection error: connection error: desc = "transport: Error while dialing dial tcp: address true: missing port in address"
I thought it might be an issue with needing to create a service account, so I followed the steps here: Datastore mode Client Libraries, but that didn't solve the problem. I also tried calling gcloud auth activate-service-account --key-file google-creds.json from the command line, but it also seemed to have no effect.
Any ideas what I'm doing wrong? Here's a snippet of accessing the datastore from the application
client, err := datastore.NewClient(tctx, "my-project-id")
if err != nil {
log.Panic("Error creating datastore client: ", err)
}
q = datastore.NewQuery("Project").Order("-LastUpdated")
_, err := client.GetAll(tctx, q, &projects)
if err != nil {
log.Panicf("Error getting all projects: %v", err)
return
}
I am trying to deploy a simple go language code on Google's app engine. This is the code I am trying to deploy.
https://github.com/GoogleCloudPlatform/golang-samples/tree/master/appengine/go11x/static
main.go
package main
import (
"fmt"
"html/template"
"log"
"net/http"
"os"
"path/filepath"
"time"
)
var (
indexTmpl = template.Must(
template.ParseFiles(filepath.Join("templates", "index.html")),
)
)
func main() {
http.HandleFunc("/", indexHandler)
// Serve static files out of the public directory.
// By configuring a static handler in app.yaml, App Engine serves all the
// static content itself. As a result, the following two lines are in
// effect for development only.
public := http.StripPrefix("/public", http.FileServer(http.Dir("public")))
http.Handle("/public/", public)
port := os.Getenv("PORT")
if port == "" {
port = "8080"
log.Printf("Defaulting to port %s", port)
}
log.Printf("Listening on port %s", port)
log.Fatal(http.ListenAndServe(fmt.Sprintf(":%s", port), nil))
}
// indexHandler uses a template to create an index.html.
func indexHandler(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/" {
http.NotFound(w, r)
return
}
type indexData struct {
Logo string
Style string
RequestTime string
}
data := indexData{
Logo: "/public/gcp-gopher.svg",
Style: "/public/style.css",
RequestTime: time.Now().Format(time.RFC822),
}
if err := indexTmpl.Execute(w, data); err != nil {
log.Printf("Error executing template: %v", err)
http.Error(w, "Internal server error", http.StatusInternalServerError)
}
}
When I deploy this code using gcloud app deploy and use the browser to load the webpage, I see error
2019-08-24 06:32:19 default[] "GET / HTTP/1.1" 500
2019-08-24 06:32:20 default[] panic: open templates/index.html: no such file or directory goroutine 1 [running]: html/template.Must(0x0, 0x800800, 0xc000078f90, 0
x0) /usr/local/go/src/html/template/template.go:372 +0x54
My app.yaml file looks like this. It has static mentioned but nothing about templates.
runtime: go111
handlers:
# Configure App Engine to serve any static assets.
- url: /public
static_dir: public
# Use HTTPS for all requests.
- url: /.*
secure: always
redirect_http_response_code: 301
script: auto
Question:
How do I handle templates and other small files that I want the application to read? Mine is a toy application so I do not need cloud storage or any such solution. I just want to read from a (local) directory.
So... I tested this deployment in 3 different ways and what I found was that:
Directly cloning the repo with git clone, cd to the static folder, and deploying from there, reproduced the issue, but only if I did it from my Google Cloud Shell.
a. I later found out that the Go version I had in the Cloud Shell was the Go 1.12.
b. I created a new VM instance to test it from a fresh Go 1.11 environment and the same process worked like a treat.
The same process as the above, but instead of deploying from the static, I moved its content to a different directory and then I deployed it from there.
a. This worked in the VM instance and in the Cloud Shell.
As suggested in the Quickstart for Go 1.11 in the App Engine Standard Environment, I downloaded the sample code using the go get command, cd to the static folder and deployed from there.
a. This also worked in both environments.
My suggestion is to always try to download Google's golang samples using the go get command, as it is suggested in the guides and it did not mess with the App Engine deployment in the tests I made.
It is also important to mention that both environments had the same Cloud SDK version, which is the 259.
I'm trying to use the app engine VM for my project (env: Go). But am having problems I still haven't been able to resolve after days of searching:
I can't use the package "google.golang.org/appengine" and "google.golang.org/cloud/*"
When I deploy my code which contains the above packages I can deploy successfully, but when I access the URL, return always "502 Bad Gateway".
If I deploy the code without the above package, all works very well
The problem above happens after an update some of my package. Because I get this problem after I did "go get -u ...", before that all works well
When I see the last commit here (package "google.golang.org/appengine"): https://github.com/golang/appengine/commit/25b8450bec636c6b6e3b9b33d3a3f55230b10812
I don't understand when replace "package helloworld" -> "package main". If I depoy with "package main", I don't know how to access the handler. For example:
func main() {
http.HandleFunc("/hello", helloHandler)
appengine.Main()
}
After deploy, how to test?? Because "my_app.appspot.com/hello" return 404, page not found
I don't know if there is someone else having the same problem. Please help, thanks!!!
I think you get 502 because of panic on attempt to get Context. You can see it with command sudo docker logs -f gaeapp after connecting to VM instance via ssh.
It seems that all examples for appengine vm are broken and you should use init(), not main() for initialization:
package notmainpackage
import (
"net/http"
"google.golang.org/appengine"
"google.golang.org/appengine/log"
)
func init() {
http.HandleFunc("/", handle)
appengine.Main()
}
func handle(w http.ResponseWriter, r *http.Request) {
ctx := appengine.NewContext(r)
log.Infof(ctx, "got appengine context")
}
Google App Engine SDK for PHP (at local environment) returns error on any try to use Google Cloud Storage. Error message is:
Fatal error: Uncaught exception 'google\appengine\runtime\RPCFailedError' with message 'Remote implementation for app_identity_service.GetAccessToken failed' in /media/data/home/vladimir/setup/gae/google_appengine/php/sdk/google/appengine/runtime/RemoteApiProxy.php on line 92
It exactly repeats the problem described here:
App engine update breaks CloudStorage in dev php env
Test code from the question above shows the same result.
I tried App Engine SDK for PHP versions 1.9.19, 1.9.20, 1.9.21 without success.
On Win10 this issue can be solved by generating an application-default credentials file:
D:\Workspace\Sourcecode>gcloud auth application-default login
Credentials saved to file: [C:\Users\Otje\AppData\Roaming\gcloud\application_default_credentials.json]
And then setting the environment in commandline:
D:\Workspace\Sourcecode>SET GOOGLE_APPLICATION_DEFAULT=C:\Users\Otje\AppData\Roaming\gcloud\application_default_credentials.json
Seems to me GAE on local just outputs limited error information when it can't find the right credentials to succesfully connect to remote endpoint.
It seems that it was GAE's server side issue. They fixed it. I discovered that it started to work as expected today without any changes applied by me.