I have a simple question, I been playing with dev_appserver testing my Go app locally.
I saved some keys:
client, err = datastore.NewClient(ctx, utils.ProjectID)
key := datastore.NewKey(ctx, KindSpace, "", id, nil)
if _, err := client.Put(ctx, key, value); err != nil {
return err
}
However, When connecting to: http://localhost:8000/datastore my datastore is empty.
I then tried to retrieve the keys from inside my code using:
client, err = datastore.NewClient(ctx, utils.ProjectID)
entity := new(Space)
key := datastore.NewKey(ctx, KindSpace, "", id, nil)
if err := client.Get(ctx, key, entity); err != nil {
return nil, err
}
and indeed it returned the keys I saved. Do you have any idea what do I miss? Does it have something to do with the database namespace maybe?
Is there anyway to read the local data base from the terminal?
Related
So I am trying to use https://github.com/mongodb/mongo-go-driver to connect to a mongo database in golang.
Here is my connection handler:
var DB *mongo.Database
func CreateConnectionHandler()(*mongo.Database, error){
fmt.Println("inside createConnection in database package")
godotenv.Load()
fmt.Println("in CreateConnectionHandler and SERVER_CONFIG: ")
fmt.Println(os.Getenv("SERVER_CONFIG"))
uri:=""
if os.Getenv("SERVER_CONFIG")=="kubernetes"{
fmt.Println("inside kubernetes db config")
uri = "mongodb://patientplatypus:SUPERSECRETPASSDOOT#
mongo-release-mongodb.default.svc.cluster.local:27017/
platypusNEST?authMechanism=SCRAM-SHA-1"
}else if os.Getenv("SERVER_CONFIG")=="compose"{
fmt.Println("inside compose db config")
uri = "mongodb://datastore:27017"
}
ctx, _ := context.WithTimeout(context.Background(), 10*time.Second)
client, err := mongo.Connect(ctx, uri)
if err != nil {
return nil, fmt.Errorf("mongo client couldn't connect: %v", err)
}
DB := client.Database("platypusNEST")
return DB, nil
}
And the error I am getting:
api | database/connection.go:29:30: cannot use uri (type
string) as type *options.ClientOptions in argument to mongo.Connect
So I tried replacing uri with the connection string like this:
client, err := mongo.Connect(ctx, "mongodb://datastore:27017")
But I still got the error.
Compare this with what is in the documentation:
ctx, _ := context.WithTimeout(context.Background(), 10*time.Second)
client, err := mongo.Connect(ctx, "mongodb://localhost:27017")
And it is exactly the same! I'm really not sure why there is this error. Any ideas?
To those who come searching - the docs are out of date as of this posting, but their latest push here: https://github.com/mongodb/mongo-go-driver/commit/32946b1f8b9412a6a94e68ff789575327bb257cf has them doing this with the connect:
client, err := mongo.NewClient(options.Client().ApplyURI(uri))
You will also now need to import the options package. Happy hacking.
EDIT: thanks vcanales for finding this - you're a gentleman and a scholar.
In addition to the accepted answer - this snippet below may be improved by using an environment variable to pass in the Mongodb URL.
package main
import (
"context" //use import withs " char
"fmt"
"time"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
"go.mongodb.org/mongo-driver/mongo/readpref"
)
func ConnectMongo() {
var (
client *mongo.Client
mongoURL = "mongodb://localhost:27017"
)
// Initialize a new mongo client with options
client, err := mongo.NewClient(options.Client().ApplyURI(mongoURL))
// Connect the mongo client to the MongoDB server
ctx, _ := context.WithTimeout(context.Background(), 10*time.Second)
err = client.Connect(ctx)
// Ping MongoDB
ctx, _ = context.WithTimeout(context.Background(), 10*time.Second)
if err = client.Ping(ctx, readpref.Primary()); err != nil {
fmt.Println("could not ping to mongo db service: %v\n", err)
return
}
fmt.Println("connected to nosql database:", mongoURL)
}
func main() {
ConnectMongo()
}
More information on options and readpref respectively:
https://docs.mongodb.com/manual/reference/method/cursor.readPref/index.html
https://docs.mongodb.com/manual/core/read-preference/
I am trying to store datastore record in namespace MyNameSpace with GAE/Go, but the code below not working.
import (
"cloud.google.com/go/datastore"
"github.com/gin-gonic/gin"
"google.golang.org/appengine"
)
func Save(c *gin.Context, list []MyStruct) ([]MyStruct, error) {
r := c.Request
ctx := appengine.NewContext(r)
ctx_with_namespace, err := appengine.Namespace(ctx, "MyNameSpace")
if err != nil {
return nil, err
}
client, err := datastore.NewClient(ctx_with_namespace, "MyProject")
if err != nil {
return nil, err
}
var keyList []*datastore.Key
for _, v := range list {
key := datastore.NameKey("MyStruct", v.ColA, nil)
keyList = append(keyList, key)
}
_, err = client.PutMulti(ctx_with_namespace, keyList, list)
return list,nil
}
This code creates records in the default namespace, not MyNameSpace.
Does cloud.google.com/go/datastore ignores namespace setting?
I found this document
November 8, 2016
Breaking changes to datastore: contexts no longer hold namespaces;
instead you must set a key's namespace explicitly. Also, key functions
have been changed and renamed.
The WithNamespace function has been removed. To specify a namespace in
a Query, use the Query.Namespace method:
q := datastore.NewQuery("Kind").Namespace("ns")
All the fields of Key are exported. That means you can construct any Key with a struct literal:
k := &Key{Kind: "Kind", ID: 37, Namespace: "ns"}
I realized I should explicitly set namespace, but it is very inconvenient. I migrated to use google.golang.org/appengine/datastore
I am having problems saving programs records when using a parent key for the account.
This code fails with error "invalid key" (see bottom for complete):
key := datastore.NewIncompleteKey(ctx, "programs", actKey)
_, err = datastore.Put(ctx, key, &Program{Name: names[i]})
This passes:
key := datastore.NewIncompleteKey(ctx, "programs", nil)
_, err = datastore.Put(ctx, key, &Program{Name: names[i]})
Complete code:
// insert test account
actKey := datastore.NewIncompleteKey(ctx, "accounts", nil)
_, err = datastore.Put(ctx, actKey, &Account{Name: "Chris Olsenio"})
if err != nil {
log.Errorf(ctx, "Insert test account %v", err.Error())
c.AbortWithError(http.StatusInternalServerError, err)
return
}
var names = []string {"Low Impact", "Running"}
for i := 0; i < len(names); i++ {
key := datastore.NewIncompleteKey(ctx, "programs", actKey)
_, err = datastore.Put(ctx, key, &Program{Name: names[i]})
if err != nil {
log.Errorf(ctx, "Insert test programs %v", err.Error())
c.AbortWithError(http.StatusInternalServerError, err)
return
}
}
The problem is that when you create an incomplete key:
actKey := datastore.NewIncompleteKey(ctx, "accounts", nil)
Which you use to save an entity:
_, err = datastore.Put(ctx, actKey, &Account{Name: "Chris Olsenio"})
It works, but note that if the key passed is an incomplete key (it is in your case), datastore.Put() returns a new, unique key generated by the datastore. You don't store the returned new key, but you should!
When you try to create and save "programs" entities:
key := datastore.NewIncompleteKey(ctx, "programs", actKey)
datastore.NewIncompleteKey() expects either a nil parent key, of if it is provided, it must be a complete key (cannot be incomplete). You pass actKey which is an incomplete key, hence the "invalid key" error message.
Solution is simple: store the generated new key, and pass the new, complete key as the parent key:
actKey := datastore.NewIncompleteKey(ctx, "accounts", nil)
actKey, err = datastore.Put(ctx, actKey, &Account{Name: "Chris Olsenio"})
If err is nil, actKey will now be a complete key and therefore can be used as parent key when creating other keys with datastore.NewIncompleteKey() or datastore.NewKey().
Google's AppEngine service provides an eventually consistent database for storage of application data in production. For testing, Google provides a similar database that emulates the consistency characteristics of the production database. Testing requirements may vary so Google supplies command line parameters to their test server, dev_appserver.py, that modify the consistency characteristics as needed for testing.
I am using the goapp tools to run our automated test environment for my AppEngine development. goapp test is responsible for running our automated server API tests. goapp test does not appear to have a way of setting the datastore's consistency level via the command line parameters, unlike dev_appserver.py, even though goapp test launches dev_appserver.py at some point during the testing process.
Is there a way to pass command line parameters to dev_appserver.py from goapp test? If not, is there an alternative method for setting the consistency from the command line?
I presume you are using the aetest package.
If this is the case you should set the StronglyConsistentDatastore member of the aetest.Options struct accordingly.
Here is an example:
hello.go
package hello
import (
"fmt"
"net/http"
"time"
"appengine"
datastore "appengine/datastore"
)
type Employee struct {
FirstName string
LastName string
HireDate time.Time
}
func init() {
http.HandleFunc("/", handler)
}
func handler(w http.ResponseWriter, r *http.Request) {
ctx := appengine.NewContext(r)
q := datastore.NewQuery("Employee").Filter("FirstName =", "Antonio")
var people []Employee
if _, err := q.GetAll(ctx, &people); err != nil {
fmt.Fprintf(w, "Error %v", err)
}
fmt.Fprintf(w, "Matches %v", len(people))
}
hello_test.go
package hello
import (
"time"
"testing"
"net/http/httptest"
"appengine"
"appengine/aetest"
datastore "appengine/datastore"
)
const consistency_strong = true; // set to false and the test will fail
func TestMyHandler(t *testing.T) {
options := &aetest.Options{StronglyConsistentDatastore: consistency_strong}
inst, err := aetest.NewInstance(options)
if err != nil {
t.Fatalf("Failed to create instance: %v", err)
}
defer inst.Close()
req, err := inst.NewRequest("GET", "/", nil)
if err != nil {
t.Fatalf("Failed to create req1: %v", err)
}
ctx := appengine.NewContext(req)
employee := &Employee{
FirstName: "Antonio",
LastName: "Salieri",
HireDate: time.Now(),
}
key := datastore.NewIncompleteKey(ctx, "Employee", nil)
_, err = datastore.Put(ctx, key, employee)
if err != nil {
t.Fatalf("Error setting test data: %v", err)
}
w := httptest.NewRecorder()
handler(w, req)
if w.Body.String() != "Matches 1" {
t.Fatalf("Expected 1 record got %v", w.Body)
}
}
As you have mentioned the go tools merely wrap dev_appserver.py. This means that if you are not using aetest you can always run your app with a chosen consistency policy like so:
/usr/local/go_appengine/dev_appserver.py --datastore_consistency_policy consistent .
I experiment a little bit with GAE, but now I have a problem. First of all I store some stuff into the datastore, with an NewIncompleteKey.
So there is the issue. My Website sends timestamps (I handle them as "ID"s) to the back-end. I parse then and want to delete them from the datastore.
I thought I can do this.
type Food struct{
Id int64
Course string
Name string
Date string
Price float64
}
...Some Code...
func deleteEntries(mealsID []string, r *http.Request) int{
// Get context from
c := appengine.NewContext(r);
for _,id := range mealsID{
var key *datastore.Key = nil
q := datastore.NewQuery("Meal").Ancestor(mealStoreKey(c)).Filter("Course =", "dessert").KeysOnly()
_, err := q.GetAll(c, key)
if err != nil{
return 0
}
log.Printf("Here the keys: %T %v ", key, key)
log.Printf("%v ", id)
e := datastore.Delete(c, key)
if e != nil{
return 33
}
}
return len(mealsID)
}
But it doesn't work, because I get an error at the datastore.Delete() function. Anyone an idea?
Edit:
Part I:
keys, err := q.GetAll(c, nil)
…
err = datastore.DeleteMulti(c, keys)
Thanks to Dave.
Part II:
I passed an String as Filter vaule to the query, but it have to be an Int64 same as in the datastore. Note to my self: You have to pass also the same type of var to the query.
func deleteEntries(mealsID []string, r *http.Request) int{
// Get context from
c := appengine.NewContext(r);
for _,id := range mealsID{
ID,_ := strconv.Atoi(id)
q:= datastore.NewQuery("Meal").Ancestor(mealStoreKey(c)).Filter("Id =", ID ).KeysOnly()
keys, err := q.GetAll(c, nil)
if err != nil{
return 0
}
log.Printf("ID: %v ", id)
log.Printf("Keys: %v ", keys)
e := datastore.DeleteMulti(c, keys)
if e != nil{
log.Printf("%v ", e)
return 0
}
}
return len(mealsID)
}
The keys are returned from GetAll. So you should instead write:
keys, err := q.GetAll(c, nil)
…
err = datastore.DeleteMulti(c, keys)
GetAll ignores the dst parameter for keys-only requests – datastore reference. So, in the example above, key will still be set to nil which explains the error.