How to set namespace in appengine using go lang? - google-app-engine

I am able to insert the entities with default namespace but I need to achieve multitenancy. Below is the code i am using to insert the entities and get the entities,but I need to assign namespaces for each entity.So i have followed below link but i am unable to set namespace please help me to fix
https://cloud.google.com/appengine/docs/standard/go/multitenancy/multitenancy
I found that there are two packages
1."cloud.google.com/go/datastore" - By using this package i am able to insert the entities with out namespace
2."google.golang.org/appengine/datastore" - I found this package from multi tenancy link by google,
I got the clarity that I need to use this package to assign namespaces but I am getting errors while using this package, Please help me to fix this
package main
import (
"fmt"
"net/http"
"google.golang.org/appengine"
"google.golang.org/appengine/datastore"
)
type user struct {
UserName string
Password string
First string
Last string
}
func main() {
fmt.Println("hi")
http.HandleFunc("/", handle)
http.Handle("/favicon.ico", http.NotFoundHandler())
http.ListenAndServe(":8080", nil)
}
func handle(w http.ResponseWriter, r *http.Request) {
ctx := appengine.NewContext(r)
k := datastore.NewKey(ctx, "user", "stringID", 0, nil)
e := new(user)
if err := datastore.Get(ctx, k, e); err != nil {
http.Error(w, err.Error(), 500)
return
}
old := e.First
e.First = r.URL.Path
if _, err := datastore.Put(ctx, k, e); err != nil {
http.Error(w, err.Error(), 500)
return
}
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
fmt.Fprintf(w, "old=%q\nnew=%q\n", old, e.First)
}
Error :
While running the code from local host I am getting below error
E:/GoWork/src/google.golang.org/appengine/internal/api.go:270 +0x186
google.golang.org/appengine.WithContext(0xd60000, 0xc04200c320, 0xc042174000, 0xd60000, 0xc04200c320)
E:/GoWork/src/google.golang.org/appengine/appengine.go:73 +0x46
google.golang.org/appengine.NewContext(0xc042174000, 0x987460, 0xc04216c034)
E:/GoWork/src/google.golang.org/appengine/appengine.go:66 +0x6e
main.handle(0x959da0, 0xc042180000, 0xc042174000)
E:/GoWork/src/simplystdatastore/main.go:27 +0x4a
net/http.HandlerFunc.ServeHTTP(0x7e49c8, 0x959da0, 0xc042180000, 0xc042174000)
c:/go/src/net/http/server.go:1942 +0x4b
net/http.(*ServeMux).ServeHTTP(0x987460, 0x959da0, 0xc042180000, 0xc042174000)
c:/go/src/net/http/server.go:2238 +0x137
net/http.serverHandler.ServeHTTP(0xc042075550, 0x959da0, 0xc042180000, 0xc042174000)
c:/go/src/net/http/server.go:2568 +0x99
net/http.(*conn).serve(0xc0421280a0, 0x95a3a0, 0xc04216e040)
c:/go/src/net/http/server.go:1825 +0x619
created by net/http.(*Server).Serve
c:/go/src/net/http/server.go:2668 +0x2d5

Related

Mongo-Go-Driver Failing to Connect

So I am trying to use https://github.com/mongodb/mongo-go-driver to connect to a mongo database in golang.
Here is my connection handler:
var DB *mongo.Database
func CreateConnectionHandler()(*mongo.Database, error){
fmt.Println("inside createConnection in database package")
godotenv.Load()
fmt.Println("in CreateConnectionHandler and SERVER_CONFIG: ")
fmt.Println(os.Getenv("SERVER_CONFIG"))
uri:=""
if os.Getenv("SERVER_CONFIG")=="kubernetes"{
fmt.Println("inside kubernetes db config")
uri = "mongodb://patientplatypus:SUPERSECRETPASSDOOT#
mongo-release-mongodb.default.svc.cluster.local:27017/
platypusNEST?authMechanism=SCRAM-SHA-1"
}else if os.Getenv("SERVER_CONFIG")=="compose"{
fmt.Println("inside compose db config")
uri = "mongodb://datastore:27017"
}
ctx, _ := context.WithTimeout(context.Background(), 10*time.Second)
client, err := mongo.Connect(ctx, uri)
if err != nil {
return nil, fmt.Errorf("mongo client couldn't connect: %v", err)
}
DB := client.Database("platypusNEST")
return DB, nil
}
And the error I am getting:
api | database/connection.go:29:30: cannot use uri (type
string) as type *options.ClientOptions in argument to mongo.Connect
So I tried replacing uri with the connection string like this:
client, err := mongo.Connect(ctx, "mongodb://datastore:27017")
But I still got the error.
Compare this with what is in the documentation:
ctx, _ := context.WithTimeout(context.Background(), 10*time.Second)
client, err := mongo.Connect(ctx, "mongodb://localhost:27017")
And it is exactly the same! I'm really not sure why there is this error. Any ideas?
To those who come searching - the docs are out of date as of this posting, but their latest push here: https://github.com/mongodb/mongo-go-driver/commit/32946b1f8b9412a6a94e68ff789575327bb257cf has them doing this with the connect:
client, err := mongo.NewClient(options.Client().ApplyURI(uri))
You will also now need to import the options package. Happy hacking.
EDIT: thanks vcanales for finding this - you're a gentleman and a scholar.
In addition to the accepted answer - this snippet below may be improved by using an environment variable to pass in the Mongodb URL.
package main
import (
"context" //use import withs " char
"fmt"
"time"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
"go.mongodb.org/mongo-driver/mongo/readpref"
)
func ConnectMongo() {
var (
client *mongo.Client
mongoURL = "mongodb://localhost:27017"
)
// Initialize a new mongo client with options
client, err := mongo.NewClient(options.Client().ApplyURI(mongoURL))
// Connect the mongo client to the MongoDB server
ctx, _ := context.WithTimeout(context.Background(), 10*time.Second)
err = client.Connect(ctx)
// Ping MongoDB
ctx, _ = context.WithTimeout(context.Background(), 10*time.Second)
if err = client.Ping(ctx, readpref.Primary()); err != nil {
fmt.Println("could not ping to mongo db service: %v\n", err)
return
}
fmt.Println("connected to nosql database:", mongoURL)
}
func main() {
ConnectMongo()
}
More information on options and readpref respectively:
https://docs.mongodb.com/manual/reference/method/cursor.readPref/index.html
https://docs.mongodb.com/manual/core/read-preference/

GAE/Go: Namespace not working

I am trying to store datastore record in namespace MyNameSpace with GAE/Go, but the code below not working.
import (
"cloud.google.com/go/datastore"
"github.com/gin-gonic/gin"
"google.golang.org/appengine"
)
func Save(c *gin.Context, list []MyStruct) ([]MyStruct, error) {
r := c.Request
ctx := appengine.NewContext(r)
ctx_with_namespace, err := appengine.Namespace(ctx, "MyNameSpace")
if err != nil {
return nil, err
}
client, err := datastore.NewClient(ctx_with_namespace, "MyProject")
if err != nil {
return nil, err
}
var keyList []*datastore.Key
for _, v := range list {
key := datastore.NameKey("MyStruct", v.ColA, nil)
keyList = append(keyList, key)
}
_, err = client.PutMulti(ctx_with_namespace, keyList, list)
return list,nil
}
This code creates records in the default namespace, not MyNameSpace.
Does cloud.google.com/go/datastore ignores namespace setting?
I found this document
November 8, 2016
Breaking changes to datastore: contexts no longer hold namespaces;
instead you must set a key's namespace explicitly. Also, key functions
have been changed and renamed.
The WithNamespace function has been removed. To specify a namespace in
a Query, use the Query.Namespace method:
q := datastore.NewQuery("Kind").Namespace("ns")
All the fields of Key are exported. That means you can construct any Key with a struct literal:
k := &Key{Kind: "Kind", ID: 37, Namespace: "ns"}
I realized I should explicitly set namespace, but it is very inconvenient. I migrated to use google.golang.org/appengine/datastore

problems with dev_appserver datastore file

I have a simple question, I been playing with dev_appserver testing my Go app locally.
I saved some keys:
client, err = datastore.NewClient(ctx, utils.ProjectID)
key := datastore.NewKey(ctx, KindSpace, "", id, nil)
if _, err := client.Put(ctx, key, value); err != nil {
return err
}
However, When connecting to: http://localhost:8000/datastore my datastore is empty.
I then tried to retrieve the keys from inside my code using:
client, err = datastore.NewClient(ctx, utils.ProjectID)
entity := new(Space)
key := datastore.NewKey(ctx, KindSpace, "", id, nil)
if err := client.Get(ctx, key, entity); err != nil {
return nil, err
}
and indeed it returned the keys I saved. Do you have any idea what do I miss? Does it have something to do with the database namespace maybe?
Is there anyway to read the local data base from the terminal?

Cannot change consistency using "goapp test"

Google's AppEngine service provides an eventually consistent database for storage of application data in production. For testing, Google provides a similar database that emulates the consistency characteristics of the production database. Testing requirements may vary so Google supplies command line parameters to their test server, dev_appserver.py, that modify the consistency characteristics as needed for testing.
I am using the goapp tools to run our automated test environment for my AppEngine development. goapp test is responsible for running our automated server API tests. goapp test does not appear to have a way of setting the datastore's consistency level via the command line parameters, unlike dev_appserver.py, even though goapp test launches dev_appserver.py at some point during the testing process.
Is there a way to pass command line parameters to dev_appserver.py from goapp test? If not, is there an alternative method for setting the consistency from the command line?
I presume you are using the aetest package.
If this is the case you should set the StronglyConsistentDatastore member of the aetest.Options struct accordingly.
Here is an example:
hello.go
package hello
import (
"fmt"
"net/http"
"time"
"appengine"
datastore "appengine/datastore"
)
type Employee struct {
FirstName string
LastName string
HireDate time.Time
}
func init() {
http.HandleFunc("/", handler)
}
func handler(w http.ResponseWriter, r *http.Request) {
ctx := appengine.NewContext(r)
q := datastore.NewQuery("Employee").Filter("FirstName =", "Antonio")
var people []Employee
if _, err := q.GetAll(ctx, &people); err != nil {
fmt.Fprintf(w, "Error %v", err)
}
fmt.Fprintf(w, "Matches %v", len(people))
}
hello_test.go
package hello
import (
"time"
"testing"
"net/http/httptest"
"appengine"
"appengine/aetest"
datastore "appengine/datastore"
)
const consistency_strong = true; // set to false and the test will fail
func TestMyHandler(t *testing.T) {
options := &aetest.Options{StronglyConsistentDatastore: consistency_strong}
inst, err := aetest.NewInstance(options)
if err != nil {
t.Fatalf("Failed to create instance: %v", err)
}
defer inst.Close()
req, err := inst.NewRequest("GET", "/", nil)
if err != nil {
t.Fatalf("Failed to create req1: %v", err)
}
ctx := appengine.NewContext(req)
employee := &Employee{
FirstName: "Antonio",
LastName: "Salieri",
HireDate: time.Now(),
}
key := datastore.NewIncompleteKey(ctx, "Employee", nil)
_, err = datastore.Put(ctx, key, employee)
if err != nil {
t.Fatalf("Error setting test data: %v", err)
}
w := httptest.NewRecorder()
handler(w, req)
if w.Body.String() != "Matches 1" {
t.Fatalf("Expected 1 record got %v", w.Body)
}
}
As you have mentioned the go tools merely wrap dev_appserver.py. This means that if you are not using aetest you can always run your app with a chosen consistency policy like so:
/usr/local/go_appengine/dev_appserver.py --datastore_consistency_policy consistent .

KeysOnly Function dont return Keys

I experiment a little bit with GAE, but now I have a problem. First of all I store some stuff into the datastore, with an NewIncompleteKey.
So there is the issue. My Website sends timestamps (I handle them as "ID"s) to the back-end. I parse then and want to delete them from the datastore.
I thought I can do this.
type Food struct{
Id int64
Course string
Name string
Date string
Price float64
}
...Some Code...
func deleteEntries(mealsID []string, r *http.Request) int{
// Get context from
c := appengine.NewContext(r);
for _,id := range mealsID{
var key *datastore.Key = nil
q := datastore.NewQuery("Meal").Ancestor(mealStoreKey(c)).Filter("Course =", "dessert").KeysOnly()
_, err := q.GetAll(c, key)
if err != nil{
return 0
}
log.Printf("Here the keys: %T %v ", key, key)
log.Printf("%v ", id)
e := datastore.Delete(c, key)
if e != nil{
return 33
}
}
return len(mealsID)
}
But it doesn't work, because I get an error at the datastore.Delete() function. Anyone an idea?
Edit:
Part I:
keys, err := q.GetAll(c, nil)
…
err = datastore.DeleteMulti(c, keys)
Thanks to Dave.
Part II:
I passed an String as Filter vaule to the query, but it have to be an Int64 same as in the datastore. Note to my self: You have to pass also the same type of var to the query.
func deleteEntries(mealsID []string, r *http.Request) int{
// Get context from
c := appengine.NewContext(r);
for _,id := range mealsID{
ID,_ := strconv.Atoi(id)
q:= datastore.NewQuery("Meal").Ancestor(mealStoreKey(c)).Filter("Id =", ID ).KeysOnly()
keys, err := q.GetAll(c, nil)
if err != nil{
return 0
}
log.Printf("ID: %v ", id)
log.Printf("Keys: %v ", keys)
e := datastore.DeleteMulti(c, keys)
if e != nil{
log.Printf("%v ", e)
return 0
}
}
return len(mealsID)
}
The keys are returned from GetAll. So you should instead write:
keys, err := q.GetAll(c, nil)
…
err = datastore.DeleteMulti(c, keys)
GetAll ignores the dst parameter for keys-only requests – datastore reference. So, in the example above, key will still be set to nil which explains the error.

Resources