So I am trying to use https://github.com/mongodb/mongo-go-driver to connect to a mongo database in golang.
Here is my connection handler:
var DB *mongo.Database
func CreateConnectionHandler()(*mongo.Database, error){
fmt.Println("inside createConnection in database package")
godotenv.Load()
fmt.Println("in CreateConnectionHandler and SERVER_CONFIG: ")
fmt.Println(os.Getenv("SERVER_CONFIG"))
uri:=""
if os.Getenv("SERVER_CONFIG")=="kubernetes"{
fmt.Println("inside kubernetes db config")
uri = "mongodb://patientplatypus:SUPERSECRETPASSDOOT#
mongo-release-mongodb.default.svc.cluster.local:27017/
platypusNEST?authMechanism=SCRAM-SHA-1"
}else if os.Getenv("SERVER_CONFIG")=="compose"{
fmt.Println("inside compose db config")
uri = "mongodb://datastore:27017"
}
ctx, _ := context.WithTimeout(context.Background(), 10*time.Second)
client, err := mongo.Connect(ctx, uri)
if err != nil {
return nil, fmt.Errorf("mongo client couldn't connect: %v", err)
}
DB := client.Database("platypusNEST")
return DB, nil
}
And the error I am getting:
api | database/connection.go:29:30: cannot use uri (type
string) as type *options.ClientOptions in argument to mongo.Connect
So I tried replacing uri with the connection string like this:
client, err := mongo.Connect(ctx, "mongodb://datastore:27017")
But I still got the error.
Compare this with what is in the documentation:
ctx, _ := context.WithTimeout(context.Background(), 10*time.Second)
client, err := mongo.Connect(ctx, "mongodb://localhost:27017")
And it is exactly the same! I'm really not sure why there is this error. Any ideas?
To those who come searching - the docs are out of date as of this posting, but their latest push here: https://github.com/mongodb/mongo-go-driver/commit/32946b1f8b9412a6a94e68ff789575327bb257cf has them doing this with the connect:
client, err := mongo.NewClient(options.Client().ApplyURI(uri))
You will also now need to import the options package. Happy hacking.
EDIT: thanks vcanales for finding this - you're a gentleman and a scholar.
In addition to the accepted answer - this snippet below may be improved by using an environment variable to pass in the Mongodb URL.
package main
import (
"context" //use import withs " char
"fmt"
"time"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
"go.mongodb.org/mongo-driver/mongo/readpref"
)
func ConnectMongo() {
var (
client *mongo.Client
mongoURL = "mongodb://localhost:27017"
)
// Initialize a new mongo client with options
client, err := mongo.NewClient(options.Client().ApplyURI(mongoURL))
// Connect the mongo client to the MongoDB server
ctx, _ := context.WithTimeout(context.Background(), 10*time.Second)
err = client.Connect(ctx)
// Ping MongoDB
ctx, _ = context.WithTimeout(context.Background(), 10*time.Second)
if err = client.Ping(ctx, readpref.Primary()); err != nil {
fmt.Println("could not ping to mongo db service: %v\n", err)
return
}
fmt.Println("connected to nosql database:", mongoURL)
}
func main() {
ConnectMongo()
}
More information on options and readpref respectively:
https://docs.mongodb.com/manual/reference/method/cursor.readPref/index.html
https://docs.mongodb.com/manual/core/read-preference/
Related
Im new to the Golang language and im coming from Nodejs, where it was quite simple to access database and manipulate db within a http request handler.Now i want to do the same in Golang and i cant access the db variable from the handlers.
Lets say i want to get users on a get request from a postgres db.
func getHandler(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "get req made")
rows, error := db.Query("SELECT id, name, age FROM new_table LIMIT $1", 4)
if error != nil {
panic(error)
}
defer rows.Close()
for rows.Next() {
var id int
var name string
var age int
error = rows.Scan(&id, &name, &age)
if error != nil {
panic(error)
}
fmt.Println(id, name, age)
}
error = rows.Err()
if error != nil {
panic(error)
}
}```
And i get error: undeclared name: db, if i use this code inside the main function where the db connection is located, its working fine.
How can i use the db variable outside the scope where its declared?
Probably your db variable is created in your main func when you try to connect to the database. The problem is, that db variable will have a scope within your main func only. So to work, you need to declare it globally at package level.
So in your main.go declare a variable outside of your main func, then use it everywhere.
package main
var db *DB
func main() {
var err error
db, err = sql.Connect(...)
if err != nil {
log.Fatal(err)
}
defer db.Close()
//Start HTTP server
}
But if you use global variables, you must check whenever it supports multi threaded access. Your db connection will work fine, but you have to read some tutorial about variable scopes and mutexes in go.
If db is from another package make sure it has public access. You will need to start with a capital letter e.g. Db
This would be a basic database function in a separate file using gorm.io
package boot
import (
"fmt"
"log"
"os"
"gorm.io/driver/postgres"
"gorm.io/gorm"
)
var DB *gorm.DB
func ConnectDB() {
var err error
dsn := fmt.Sprintf("host=%s user=%s password=%s dbname=%s port=%s sslmode=disable",
os.Getenv("PG_HOST"),
os.Getenv("PG_USER"),
os.Getenv("PG_PASSWORD"),
os.Getenv("PG_DBNAME"),
os.Getenv("PG_PORT"),
)
DB, err = gorm.Open(postgres.Open(dsn), &gorm.Config{})
if err != nil {
log.Println(err)
panic("Failled to connect to Database. ")
}
}
I am able to insert the entities with default namespace but I need to achieve multitenancy. Below is the code i am using to insert the entities and get the entities,but I need to assign namespaces for each entity.So i have followed below link but i am unable to set namespace please help me to fix
https://cloud.google.com/appengine/docs/standard/go/multitenancy/multitenancy
I found that there are two packages
1."cloud.google.com/go/datastore" - By using this package i am able to insert the entities with out namespace
2."google.golang.org/appengine/datastore" - I found this package from multi tenancy link by google,
I got the clarity that I need to use this package to assign namespaces but I am getting errors while using this package, Please help me to fix this
package main
import (
"fmt"
"net/http"
"google.golang.org/appengine"
"google.golang.org/appengine/datastore"
)
type user struct {
UserName string
Password string
First string
Last string
}
func main() {
fmt.Println("hi")
http.HandleFunc("/", handle)
http.Handle("/favicon.ico", http.NotFoundHandler())
http.ListenAndServe(":8080", nil)
}
func handle(w http.ResponseWriter, r *http.Request) {
ctx := appengine.NewContext(r)
k := datastore.NewKey(ctx, "user", "stringID", 0, nil)
e := new(user)
if err := datastore.Get(ctx, k, e); err != nil {
http.Error(w, err.Error(), 500)
return
}
old := e.First
e.First = r.URL.Path
if _, err := datastore.Put(ctx, k, e); err != nil {
http.Error(w, err.Error(), 500)
return
}
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
fmt.Fprintf(w, "old=%q\nnew=%q\n", old, e.First)
}
Error :
While running the code from local host I am getting below error
E:/GoWork/src/google.golang.org/appengine/internal/api.go:270 +0x186
google.golang.org/appengine.WithContext(0xd60000, 0xc04200c320, 0xc042174000, 0xd60000, 0xc04200c320)
E:/GoWork/src/google.golang.org/appengine/appengine.go:73 +0x46
google.golang.org/appengine.NewContext(0xc042174000, 0x987460, 0xc04216c034)
E:/GoWork/src/google.golang.org/appengine/appengine.go:66 +0x6e
main.handle(0x959da0, 0xc042180000, 0xc042174000)
E:/GoWork/src/simplystdatastore/main.go:27 +0x4a
net/http.HandlerFunc.ServeHTTP(0x7e49c8, 0x959da0, 0xc042180000, 0xc042174000)
c:/go/src/net/http/server.go:1942 +0x4b
net/http.(*ServeMux).ServeHTTP(0x987460, 0x959da0, 0xc042180000, 0xc042174000)
c:/go/src/net/http/server.go:2238 +0x137
net/http.serverHandler.ServeHTTP(0xc042075550, 0x959da0, 0xc042180000, 0xc042174000)
c:/go/src/net/http/server.go:2568 +0x99
net/http.(*conn).serve(0xc0421280a0, 0x95a3a0, 0xc04216e040)
c:/go/src/net/http/server.go:1825 +0x619
created by net/http.(*Server).Serve
c:/go/src/net/http/server.go:2668 +0x2d5
I have a simple question, I been playing with dev_appserver testing my Go app locally.
I saved some keys:
client, err = datastore.NewClient(ctx, utils.ProjectID)
key := datastore.NewKey(ctx, KindSpace, "", id, nil)
if _, err := client.Put(ctx, key, value); err != nil {
return err
}
However, When connecting to: http://localhost:8000/datastore my datastore is empty.
I then tried to retrieve the keys from inside my code using:
client, err = datastore.NewClient(ctx, utils.ProjectID)
entity := new(Space)
key := datastore.NewKey(ctx, KindSpace, "", id, nil)
if err := client.Get(ctx, key, entity); err != nil {
return nil, err
}
and indeed it returned the keys I saved. Do you have any idea what do I miss? Does it have something to do with the database namespace maybe?
Is there anyway to read the local data base from the terminal?
Google's AppEngine service provides an eventually consistent database for storage of application data in production. For testing, Google provides a similar database that emulates the consistency characteristics of the production database. Testing requirements may vary so Google supplies command line parameters to their test server, dev_appserver.py, that modify the consistency characteristics as needed for testing.
I am using the goapp tools to run our automated test environment for my AppEngine development. goapp test is responsible for running our automated server API tests. goapp test does not appear to have a way of setting the datastore's consistency level via the command line parameters, unlike dev_appserver.py, even though goapp test launches dev_appserver.py at some point during the testing process.
Is there a way to pass command line parameters to dev_appserver.py from goapp test? If not, is there an alternative method for setting the consistency from the command line?
I presume you are using the aetest package.
If this is the case you should set the StronglyConsistentDatastore member of the aetest.Options struct accordingly.
Here is an example:
hello.go
package hello
import (
"fmt"
"net/http"
"time"
"appengine"
datastore "appengine/datastore"
)
type Employee struct {
FirstName string
LastName string
HireDate time.Time
}
func init() {
http.HandleFunc("/", handler)
}
func handler(w http.ResponseWriter, r *http.Request) {
ctx := appengine.NewContext(r)
q := datastore.NewQuery("Employee").Filter("FirstName =", "Antonio")
var people []Employee
if _, err := q.GetAll(ctx, &people); err != nil {
fmt.Fprintf(w, "Error %v", err)
}
fmt.Fprintf(w, "Matches %v", len(people))
}
hello_test.go
package hello
import (
"time"
"testing"
"net/http/httptest"
"appengine"
"appengine/aetest"
datastore "appengine/datastore"
)
const consistency_strong = true; // set to false and the test will fail
func TestMyHandler(t *testing.T) {
options := &aetest.Options{StronglyConsistentDatastore: consistency_strong}
inst, err := aetest.NewInstance(options)
if err != nil {
t.Fatalf("Failed to create instance: %v", err)
}
defer inst.Close()
req, err := inst.NewRequest("GET", "/", nil)
if err != nil {
t.Fatalf("Failed to create req1: %v", err)
}
ctx := appengine.NewContext(req)
employee := &Employee{
FirstName: "Antonio",
LastName: "Salieri",
HireDate: time.Now(),
}
key := datastore.NewIncompleteKey(ctx, "Employee", nil)
_, err = datastore.Put(ctx, key, employee)
if err != nil {
t.Fatalf("Error setting test data: %v", err)
}
w := httptest.NewRecorder()
handler(w, req)
if w.Body.String() != "Matches 1" {
t.Fatalf("Expected 1 record got %v", w.Body)
}
}
As you have mentioned the go tools merely wrap dev_appserver.py. This means that if you are not using aetest you can always run your app with a chosen consistency policy like so:
/usr/local/go_appengine/dev_appserver.py --datastore_consistency_policy consistent .
I am having issues with urlfetch's timeouts on Google App Engine in Go. The app does not appear to want to take a longer timeout than about 5 seconds (it ignores a longer timeout and times out after its own time).
My code is:
var TimeoutDuration time.Duration = time.Second*30
func Call(c appengine.Context, address string, allowInvalidServerCertificate bool, method string, id interface{}, params []interface{})(map[string]interface{}, error){
data, err := json.Marshal(map[string]interface{}{
"method": method,
"id": id,
"params": params,
})
if err != nil {
return nil, err
}
req, err:=http.NewRequest("POST", address, strings.NewReader(string(data)))
if err!=nil{
return nil, err
}
tr := &urlfetch.Transport{Context: c, Deadline: TimeoutDuration, AllowInvalidServerCertificate: allowInvalidServerCertificate}
resp, err:=tr.RoundTrip(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, err
}
result := make(map[string]interface{})
err = json.Unmarshal(body, &result)
if err != nil {
return nil, err
}
return result, nil
}
No matter what I try to set TimeoutDuration to, the app times out after about 5 seconds. How prevent it from doing that? Did I make some error in my code?
You need to pass the time duration like this (otherwise it will default to the 5 sec timeout):
tr := &urlfetch.Transport{Context: c, Deadline: time.Duration(30) * time.Second}
Update Jan 2 2016:
With the new GAE golang packages (google.golang.org/appengine/*), this has changed. urlfetch no longer receives a deadline time duration in the transport.
You should now set the timeout via the new context package. For example, this is how you would set a 1 minute deadline:
func someFunc(ctx context.Context) {
ctx_with_deadline, _ := context.WithTimeout(ctx, 1*time.Minute)
client := &http.Client{
Transport: &oauth2.Transport{
Base: &urlfetch.Transport{Context: ctx_with_deadline},
},
}
Try the code below:
// createClient is urlfetch.Client with Deadline
func createClient(context appengine.Context, t time.Duration) *http.Client {
return &http.Client{
Transport: &urlfetch.Transport{
Context: context,
Deadline: t,
},
}
}
Here is how to use it.
// urlfetch
client := createClient(c, time.Second*60)
Courtesy #gosharplite
Looking at the source code of Go's appengine:
http://code.google.com/p/appengine-go/source/browse/appengine/urlfetch/urlfetch.go
and the protobuffer generated code:
http://code.google.com/p/appengine-go/source/browse/appengine_internal/urlfetch/urlfetch_service.pb.go
Looks like there should not be a problem with Duration itself.
My guess is that the whole application inside appengine timeouts after 5 seconds.
for me, this worked:
ctx_with_deadline, _ := context.WithTimeout(ctx, 15*time.Second)
client := urlfetch.Client(ctx_with_deadline)
This is had now changed with the recent updates to the library . Now the Duration of timeout/delay have to carried by the context , urlfetch.transport no more has the Deadline field in it . context.WithTimeout or context.WithDeadline is the method to use , here is the link https://godoc.org/golang.org/x/net/context#WithTimeout