I'm actually learning go, following some tutorial as this one to build an Resftul API app.
First time Using Go and mongoDB, I don't understand well, how to get specific key in certain document of my collection.
Actually I have this object model :
type Numobject struct {
ID bson.ObjectId `bson:"_id" json:"id"`
Text string `bson:"text" json:"text"`
Number int `bson:"number" json:"number"`
Found bool `bson:"found" json:"found"`
Type string `bson:"type" json:"type"`
}
And I can have a specific object by ID with this function :
// Find Object by ID
func (m *NumObjectDAO) FindByNumber(id string) (Numobject, error) {
var numObject Numobject
err := db.C(COLLECTION).FindId(bson.ObjectIdHex(id)).One(&numObject)
return numObject, err
}
I call my method in main.go as follow
package main
import (
"encoding/json"
"log"
"net/http"
"github.com/gorilla/mux"
"gopkg.in/mgo.v2/bson"
. "github.com/sandaleRaclette/03-coddingChallenge/config"
. "github.com/sandaleRaclette/03-coddingChallenge/dao"
. "github.com/sandaleRaclette/03-coddingChallenge/models"
)
// Save the configuration of mongodatabase (localhost and which db use) in Config array
var config = Config{}
var dao = NumObjectDAO{}
// GET list of all objects
func AllObjectsEndPoint(w http.ResponseWriter, r *http.Request) {
movies, err := dao.FindAll()
if err != nil {
respondWithError(w, http.StatusInternalServerError, err.Error())
return
}
respondWithJson(w, http.StatusOK, movies)
}
// GET an Object by its ID
func FindObjectEndpoint(w http.ResponseWriter, r *http.Request) {
params := mux.Vars(r)
numObject, err := dao.FindByNumber(params["number"])
if err != nil {
respondWithError(w, http.StatusBadRequest, "Invalid Object ID")
return
}
respondWithJson(w, http.StatusOK, numObject)
}
// Respond Methods
func respondWithError(w http.ResponseWriter, code int, msg string) {
respondWithJson(w, code, map[string]string{"error": msg})
}
func respondWithJson(w http.ResponseWriter, code int, payload interface{}) {
response, _ := json.Marshal(payload)
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(code)
w.Write(response)
}
// Parse the configuration file 'config.toml', and establish a connection to DB
func init() {
config.Read()
dao.Server = config.Server
dao.Database = config.Database
dao.Connect()
}
// Define HTTP request routes and define the differents endpoints
func main() {
r := mux.NewRouter()
r.HandleFunc("/api/v1/trivia", AllObjectsEndPoint).Methods("GET")
r.HandleFunc("/api/v1/trivia/{number}", FindObjectEndpoint).Methods("GET")
if err := http.ListenAndServe(":3000", r); err != nil {
log.Fatal(err)
}
}
There is other methods than getting an object by ID? How can I get an object with a specific key as Number or Type following my model ?
I want to obtain something like this when I GET "/api/v1/trivia/45000000" :
{
"id": "5aa554c89d63b0d3580449a5",
"text": "45000000 is the number of turkeys Americans eat at Thanksgiving annually.",
"number": 45000000,
"found": true,
"type": "trivia"
}
I'm actually looking here for some answers but I have some difficulties with query... If someone can give me beginner explanation...
Use Find for general queries:
err := db.c(COLLECTION).Find(bson.D{{"Number", 10}}).One(&numobjecct)
Related
I would like to connect my server that was written in Go with a MongoDB but I'm not sure how to do it in an efficient way. A couple of examples I found implemented it like shown below.
libs/mongodb/client.go
package mongodb
import (
"context"
"log"
"project/keys"
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
)
func GetClient() *mongo.Database {
client, err := mongo.Connect(
context.Background(),
options.Client().ApplyURI(keys.GetKeys().MONGO_URI),
)
if err != nil {
log.Fatal(err)
}
return client.Database(keys.GetKeys().MONGO_DB_NAME)
}
services/user/findOne.go
package userservices
import (
"context"
"log"
"project/libs/mongodb"
"project/models"
"go.mongodb.org/mongo-driver/bson"
)
func FindOne(filter bson.M) (models.User, error) {
var user models.User
collection := mongodb.GetClient().Collection("users")
result := collection.FindOne(context.TODO(), filter)
if result.Err() != nil {
return user, result.Err()
}
if err := result.Decode(&user); err != nil {
log.Println("Failed to decode user with error:", err)
return user, err
}
return user, nil
}
The GetClient function returns a database instance that is then used throughout the app. This seems to work, but I'm wondering if this really is best practice as it seems to create a new connection every time a new client is requested as shown in the second code snippet or is that assumption incorrect? I also thought about converting GetClient to a singleton, that always returns the same database instance but how would a lost connection be handled in that case? Thank you
I do it this way. Do it once at the service start and then pass the MongoDatastore object around to orchestrator, service layers and repository layers. I am using the "github.com/mongodb/mongo-go-driver/mongo" driver for mongo. I think it internally monitors and recycles idle connections. Hence, we don't have to bother about broken connections as long as reference to the mongo.Client object is not lost.
const CONNECTED = "Successfully connected to database: %v"
type MongoDatastore struct {
db *mongo.Database
Session *mongo.Client
logger *logrus.Logger
}
func NewDatastore(config config.GeneralConfig, logger *logrus.Logger) *MongoDatastore {
var mongoDataStore *MongoDatastore
db, session := connect(config, logger)
if db != nil && session != nil {
// log statements here as well
mongoDataStore = new(MongoDatastore)
mongoDataStore.db = db
mongoDataStore.logger = logger
mongoDataStore.Session = session
return mongoDataStore
}
logger.Fatalf("Failed to connect to database: %v", config.DatabaseName)
return nil
}
func connect(generalConfig config.GeneralConfig, logger *logrus.Logger) (a *mongo.Database, b *mongo.Client) {
var connectOnce sync.Once
var db *mongo.Database
var session *mongo.Client
connectOnce.Do(func() {
db, session = connectToMongo(generalConfig, logger)
})
return db, session
}
func connectToMongo(generalConfig config.GeneralConfig, logger *logrus.Logger) (a *mongo.Database, b *mongo.Client) {
var err error
session, err := mongo.NewClient(generalConfig.DatabaseHost)
if err != nil {
logger.Fatal(err)
}
session.Connect(context.TODO())
if err != nil {
logger.Fatal(err)
}
var DB = session.Database(generalConfig.DatabaseName)
logger.Info(CONNECTED, generalConfig.DatabaseName)
return DB, session
}
You may now create your repository as below:-
type TestRepository interface{
Find(ctx context.Context, filters interface{}) []Document, error
}
type testRepository struct {
store *datastore.MongoDatastore
}
func (r *testRepository) Find(ctx context.Context , filters interface{}) []Document, error{
cur, err := r.store.GetCollection("some_collection_name").Find(ctx, filters)
if err != nil {
return nil, err
}
defer cur.Close(ctx)
var result = make([]models.Document, 0)
for cur.Next(ctx) {
var currDoc models.Document
err := cur.Decode(&currDoc)
if err != nil {
//log here
continue
}
result = append(result, currDoc)
}
return result, err
}
I solved it doing this
var CNX = Connection()
func Connection() *mongo.Client {
// Set client options
clientOptions := options.Client().ApplyURI("mongodb://localhost:27017")
// Connect to MongoDB
client, err := mongo.Connect(context.TODO(), clientOptions)
if err != nil {
log.Fatal(err)
}
// Check the connection
err = client.Ping(context.TODO(), nil)
if err != nil {
log.Fatal(err)
}
fmt.Println("Connected to MongoDB!")
return client
}
//calll connection
func main() {
collection := db.CNX.Database("tasks").Collection("task")
}
output "Connected to MongoDB!"
{
"machines": [{
"name": "relay_1",
"totalmem": "3G",
"machinemem": "6G"
}, {
"name": "relay_2",
"totalmem": "30G",
"machinemem": "4G"
}]
}
tried doing the parsing using the below code
Run: func(cmd *cobra.Command, args []string) {
fmt.Println("relay called")
conf, _ = rootCmd.Flags().GetString("conf")
if conf != "" {
fmt.Println("From Create Command : ", conf)
}
data, err := ioutil.ReadFile("file.txt") // data has type []byte
if err != nil {
log.Fatal(err)
}
var result []map[string]interface{}
json.Unmarshal(data, &result)
relays := result["relays"].(map[string]interface{})
for key, relay := range relays {
fmt.Println("name :", relay["name"],
"totalmem:", relay["totalmem"],
"relaymem:", relay["relaymem"])
}
},
But I am getting the error as below which indicates that the type is invalid
cmd/create_relay.go:54:29: invalid type assertion: result["relays"].(map[string]) (non-interface type map[string]interface {} on left)
Can someone let me know how to parse the json below by using the interfaces as below
It worked with the below code.
type Relays struct {
Relays []Relay `json:"relays"`
}
type Relay struct {
Name string `json:"name"`
Totalmem string `json:"totalmem"`
Relaymem string `json:"relaymem"`
}
// relayCmd represents the relay command
var createrelayCmd = &cobra.Command{
Use: "relay",
Short: "A brief description of your command",
Long: `A longer description that spans multiple lines and likely contains examples
and usage of using your command. For example:
Cobra is a CLI library for Go that empowers applications.
This application is a tool to generate the needed files
to quickly create a Cobra application.`,
Run: func(cmd *cobra.Command, args []string) {
fmt.Println("relay called")
conf, _ = rootCmd.Flags().GetString("conf")
if conf != "" {
fmt.Println("From Create Command : ", conf)
jsonFile, err := os.Open(conf)
// if we os.Open returns an error then handle it
if err != nil {
fmt.Println(err)
}
fmt.Println("Successfully Opened users.json")
// defer the closing of our jsonFile so that we can parse it later on
defer jsonFile.Close()
data, err := ioutil.ReadAll(jsonFile) // data has type []byte
if err != nil {
log.Fatal(err)
}
var relays Relays
json.Unmarshal(data,&relays)
for i := 0; i < len(relays.Relays); i++ {
fmt.Println("User Name: " + relays.Relays[i].Name)
fmt.Println("User Totalmem: " + relays.Relays[i].Totalmem)
fmt.Println("User Relaymem: " + relays.Relays[i].Relaymem)
}
}
},
}
result in your code is an array of map and you are using result["relays"] which is invalid.
Your code should be something like this:
Run: func(cmd *cobra.Command, args []string) {
fmt.Println("relay called")
conf, _ = rootCmd.Flags().GetString("conf")
if conf != "" {
fmt.Println("From Create Command : ", conf)
}
data, err := ioutil.ReadFile("file.txt") // data has type []byte
if err != nil {
log.Fatal(err)
}
var result map[string]interface{}
json.Unmarshal(data, &result)
relays := result["relays"].([]interface{})
for _, relay := range relays {
relayM := relay.(map[string]interface{})
fmt.Println("name :", relayM["name"].(string),
"totalmem:", relayM["totalmem"].(string),
"relaymem:", relayM["relaymem"].(string))
}
},
How can I insert an array of documents into MongoDB with mgo library using only a single DB call as in db.collection.insert()?
I have the following Transaction structure:
type Transaction struct {
Brand string `json:"brand"`
Name string `json:"name"`
Plu string `json:"plu"`
Price string `json:"price"`
}
From a POST request I will recieve an array of these structures. I want to insert them into MongoDB as individual documents but using a single DB call as explained in db.collection.insert()
I tried using c.Insert of mgo
The following is the code snippet:
func insertTransaction(c *gin.Context) {
var transactions []Transaction
err := c.BindJSON(&transactions)
if err != nil {
c.AbortWithStatusJSON(http.StatusBadRequest, map[string]string{"error":"invalid JSON"})
return
}
err = InsertTransactons(transactions)
if err != nil {
c.AbortWithStatusJSON(http.StatusInternalServerError, &map[string](interface{}){
"status": "error",
"code": "500",
"message": "Internal server error",
"error": err,
})
return
}
c.JSON(http.StatusCreated, &map[string](interface{}){
"status": "success",
"code": "0",
"message": "created",
})
}
func InsertTransactons(u []Transaction) error {
s := GetSession()
defer s.Close()
c := s.DB(DB).C(TransactionColl)
err := c.Insert(u...)
if err != nil {
return err
}
return nil
}
But as I compile and run the code, I get the following error:
go/database.go:34:17: cannot use u (type *[]Transaction) as type
[]interface {} in argument to c.Insert
You cannot pass []*Transaction as []interface{}. You need to convert each Transaction to inferface{} to change its memory layout.
var ui []interface{}
for _, t := range u{
ui = append(ui, t)
}
Pass ui to c.Insert instead
Create slice of interface for document structs by appending and then inserting data using Bulk insert which takes variable arguments.
type Bulk struct {
// contains filtered or unexported fields
}
func (b *Bulk) Insert(docs ...interface{})
For inserting documents in Bulk
const INSERT_COUNT int = 10000
type User struct {
Id bson.ObjectId `bson:"_id,omitempty" json:"_id"`
Email string `bson:"email" json:"email"`
}
func (self *User) Init() {
self.Id = bson.NewObjectId()
}
Call Bulk() function on collection returned from db connection. Bulk() function returns pointer to *Bulk.
bulk := dbs.Clone().DB("").C("users").Bulk()
bulk.Insert(users...)
Assign it to variable which will be used to call Insert() method using Bulk pointer receiver.
func main(){
// Database
dbs, err := mgo.Dial("mongodb://localhost/")
if err != nil {
panic(err)
}
// Collections
uc := dbs.Clone().DB("").C("users")
defer dbs.Clone().DB("").Session.Close()
for n := 0; n < b.N; n++ {
count := INSERT_COUNT
users := make([]interface{}, count)
for i := 0; i < count; i++ {
loop_user := User{}
loop_user.Init()
loop_user.Email = fmt.Sprintf("report-%d#example.com", i)
users[i] = loop_user
}
bulk := uc.Bulk()
bulk.Unordered()
bulk.Insert(users...)
_, bulkErr := bulk.Run()
if bulkErr != nil {
panic(err)
}
}
}
How do I connect a backend server in go lang to angularjs? I know angularjs communicates with go via $http or $resource services but what part of the go code links communicates with angular once all the data structs have been made? Would this be the encoded/marshalled json or do we create some kind of route...
I am a newbie looking to start this project after studying angular and go but this is the part i don't understand - what is the end point from go that angulars $resource or $http service links with?
Create a http endpoint that reads json string, unmarshalls it, does some logic with it and writes a json string to the response. E.g:
func main() {
http.HandleFunc("/api/", apiHandler)
logInfo(fmt.Sprintf("Starting server on port %d", serverPort))
err := http.ListenAndServe(fmt.Sprintf(":%d", serverPort), nil)
if err != nil {
logError(err)
}
}
func apiHandler(w http.ResponseWriter, r *http.Request) {
//ensure its a post
method := r.Method
if method != "POST" {
fail(w, "Invalid http method")
return
}
requestData, err := ioutil.ReadAll(r.Body)
if err != nil {
fail(w, err.Error())
return
}
respData, err := doBusinessLogicWithData(requestData)
if err != nil {
fail(w, err.Error())
return
}
respJSONBytes, err := json.Marshal(respData)
if err != nil {
fail(w, err.Error())
return
}
fmt.Fprintln(w, string(respJSONBytes))
}
func fail(w http.ResponseWriter, message string) {
result := &struct {
Success bool
Message string
}{
Success: false,
Message: message,
}
resultJSONBytes, err := json.Marshal(result)
if err != nil {
logError(err)
fmt.Fprintln(w, "Unable to generate result")
return
}
fmt.Fprintln(w, string(resultJSONBytes))
}
For some reason nothing gets saved when the test code below is run. I have other api methods that when run (not through tests, this is just the first test) do save.
When I check the database stats via localhost:8000, it can be seen that nothing is being inserted.
Update: After copying and pasting the code below and wrapping it is GET request handler with some hardcoded data it does save to the database. So this seems like an issue with the testing aetest.Context that is used. I have added the code for the NewTestHandler helper code.
Method to create the context within the tests
func NewTestHandler(handlerFunc func(appengine.Context, http.ResponseWriter, *http.Request)) http.HandlerFunc {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
c, _ := aetest.NewContext(nil)
handlerFunc(c, w, r)
})
}
Error (update: the key that is generated returns 0 when calling .IntId())
// happens in the .Get() error handling
--- err datastore: internal error: server returned the wrong number of entities
Model
package app
import "time"
type League struct {
Name string `json:"name"`
Location string `json:"location"`
CreatedAt time.Time
}
Code
func (api *LeagueApi) Create(c appengine.Context, w http.ResponseWriter, r *http.Request) {
// data
var league League
json.NewDecoder(r.Body).Decode(&league)
defer r.Body.Close()
// save to db
key := datastore.NewIncompleteKey(c, "leagues", nil)
if _, err := datastore.Put(c, key, &league); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
var leagueCheck League
if err := datastore.Get(c, key, &leagueCheck); err != nil {
log.Println("--- err", err)
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
// json response
if err := json.NewEncoder(w).Encode(league); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
}
Test
func Test_LeagueReturnedOnCreate(t *testing.T) {
league := League{Name: "foobar"}
data, _ := json.Marshal(league)
reader := bytes.NewReader(data)
// setup request and writer
r, _ := http.NewRequest("POST", "/leagues", reader)
w := httptest.NewRecorder()
// make request
api := LeagueApi{}
handler := tux.NewTestHandler(api.Create)
handler.ServeHTTP(w, r)
// extract api response
var leagueCheck League
json.NewDecoder(w.Body).Decode(&leagueCheck)
if leagueCheck.Name != "foobar" {
t.Error("should return the league")
}
// ensure the league is in the db
}