Parsing an array of json objects in GO - arrays

I am trying to parse given API response into a struct.
It seems to be an array.
[
{
"host_name" : "hostname",
"perf_data" : "",
"plugin_output" : "All good",
"state" : 0
}
]
I cannot figure out how to create struct for it, I came up with:
type ServiceData struct {
HostName string `json:"host_name"`
PerfData string `json:"perf_data"`
PluginOutput string `json:"plugin_output"`
State int `json:"state"`
}
defer resp.Body.Close()
buf := new(bytes.Buffer)
buf.ReadFrom(resp.Body)
jsonStr := buf.String()
var servicedata ServiceData
json.Unmarshal([]byte(jsonStr), &servicedata)
But no luck.
Should I perhaps remove square brackets from the initial response?
Could somebody point me in the right direction?

You may unmarshal JSON arrays into Go slices. So unmarshal into a value of type []ServiceData or []*ServiceData:
var servicedata []*ServiceData
A working demo:
func main() {
var result []*ServiceData
if err := json.Unmarshal([]byte(src), &result); err != nil {
panic(err)
}
fmt.Printf("%+v", result[0])
}
const src = `[
{
"host_name" : "hostname",
"perf_data" : "",
"plugin_output" : "All good",
"state" : 0
}
]`
Which outputs (try it on the Go Playground):
&{HostName:hostname PerfData: PluginOutput:All good State:0}
Also note that you may unmarshal "directly from the body", no need to read the body first.
Use json.Decoder to do that:
var servicedata []*ServiceData
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
// handle error
}
It's much shorter, easier to read and it's more efficient.

Related

How to parse the json array of the format below

{
"machines": [{
"name": "relay_1",
"totalmem": "3G",
"machinemem": "6G"
}, {
"name": "relay_2",
"totalmem": "30G",
"machinemem": "4G"
}]
}
tried doing the parsing using the below code
Run: func(cmd *cobra.Command, args []string) {
fmt.Println("relay called")
conf, _ = rootCmd.Flags().GetString("conf")
if conf != "" {
fmt.Println("From Create Command : ", conf)
}
data, err := ioutil.ReadFile("file.txt") // data has type []byte
if err != nil {
log.Fatal(err)
}
var result []map[string]interface{}
json.Unmarshal(data, &result)
relays := result["relays"].(map[string]interface{})
for key, relay := range relays {
fmt.Println("name :", relay["name"],
"totalmem:", relay["totalmem"],
"relaymem:", relay["relaymem"])
}
},
But I am getting the error as below which indicates that the type is invalid
cmd/create_relay.go:54:29: invalid type assertion: result["relays"].(map[string]) (non-interface type map[string]interface {} on left)
Can someone let me know how to parse the json below by using the interfaces as below
It worked with the below code.
type Relays struct {
Relays []Relay `json:"relays"`
}
type Relay struct {
Name string `json:"name"`
Totalmem string `json:"totalmem"`
Relaymem string `json:"relaymem"`
}
// relayCmd represents the relay command
var createrelayCmd = &cobra.Command{
Use: "relay",
Short: "A brief description of your command",
Long: `A longer description that spans multiple lines and likely contains examples
and usage of using your command. For example:
Cobra is a CLI library for Go that empowers applications.
This application is a tool to generate the needed files
to quickly create a Cobra application.`,
Run: func(cmd *cobra.Command, args []string) {
fmt.Println("relay called")
conf, _ = rootCmd.Flags().GetString("conf")
if conf != "" {
fmt.Println("From Create Command : ", conf)
jsonFile, err := os.Open(conf)
// if we os.Open returns an error then handle it
if err != nil {
fmt.Println(err)
}
fmt.Println("Successfully Opened users.json")
// defer the closing of our jsonFile so that we can parse it later on
defer jsonFile.Close()
data, err := ioutil.ReadAll(jsonFile) // data has type []byte
if err != nil {
log.Fatal(err)
}
var relays Relays
json.Unmarshal(data,&relays)
for i := 0; i < len(relays.Relays); i++ {
fmt.Println("User Name: " + relays.Relays[i].Name)
fmt.Println("User Totalmem: " + relays.Relays[i].Totalmem)
fmt.Println("User Relaymem: " + relays.Relays[i].Relaymem)
}
}
},
}
result in your code is an array of map and you are using result["relays"] which is invalid.
Your code should be something like this:
Run: func(cmd *cobra.Command, args []string) {
fmt.Println("relay called")
conf, _ = rootCmd.Flags().GetString("conf")
if conf != "" {
fmt.Println("From Create Command : ", conf)
}
data, err := ioutil.ReadFile("file.txt") // data has type []byte
if err != nil {
log.Fatal(err)
}
var result map[string]interface{}
json.Unmarshal(data, &result)
relays := result["relays"].([]interface{})
for _, relay := range relays {
relayM := relay.(map[string]interface{})
fmt.Println("name :", relayM["name"].(string),
"totalmem:", relayM["totalmem"].(string),
"relaymem:", relayM["relaymem"].(string))
}
},

How to encode/marshal response from my server in multiple object JSON array

I am trying to format my response from a server in an array of multiple object JSON. I tried slice and []Struct but I believe I am not able to understand their usage correctly.
I tried the method mentioned here - https://play.golang.org/p/9OEPzbf0Me0 but didn't work in my case
I am using golang to get response for a query parameter
router.HandleFunc("/v1.0/singles", GetInfo).Methods("GET").Queries("Group", "{group}")
//controller.go
func SendToServer(jsonValue []byte, command string) (models.JSON, error) {
var pdatar models.JSON
s := []string{"http://172.xx.xxx.xx:xxxxx/", command}
response, err := http.Post(strings.Join(s, ""), "application/json", bytes.NewBuffer(jsonValue))
data, err := ioutil.ReadAll(response.Body)
err = json.Unmarshal(data, &pdatar)
mdatar := make(models.JSON)
mdatar["ID"] = pdatar["id"]
mdatar["Name"] = pdatar["name"]
fmt.Println(mdatar) //This is how it is coming map[ID:[877,235], Name:[FCU, GHP]]
/*Another attempt
var temp models.Arr
json.Unmarshal(data, &temp)
fmt.Println(temp) ---- returning null
*/
return mdatar, nil
}
func (c Controller) GetSingle(pdata models.Single) (models.JSON, error) {
mdata := make(models.JSON)
mdata["group"] = pdata.Group
jsonValue, err := json.Marshal(mdata)
pdatar, err := SendToServer(jsonValue, "func_info")
return pdatar, nil
// json.NewEncoder(w).Encode(pdatar) //sending this to my main function
}
//model.go
type JSON map[string]interface{}
type Single struct {
ID int `json:"id"`
Name string `json:"name"`
}
type Data []Single
type Arr [][]string
I have a response from my API in a format
{
"id":[877,235,312,429],
"name" ["FCU","GHP","c","d"],
"group":["x","x","y","z"]
}
With Current Code I am receiving a response
{"id":[877 235], "name": ["FCU" "GHP"]
Though I am expecting a response
{
"data":
[
{"id": 877
"name" : "FCU"
}
{"id": 235
"name": "GHP"
}
]
}
Your mdatarr doesn't change the format any, so this result makes sense.
If the data received from the server is in the format:
{
"id": [877,235,312,429],
"name" ["FCU","GHP","c","d"],
"group": ["x","x","y","z"]
}
then you will need translate it. Ideally you'd fix the server so that it sends data in the format:
{
{
"id": 877,
"name": "FCU",
"group": "x",
},
...
}
If the server could send something like the above, then you could simply unmarshal it into a models.Data object.
If that is not an option, and you really need to do the translation client-side, then you'll need a for loop that does something like this:
ids := pdatarr["id"]
names := pdatarr["name"]
...
if len(ids) != len(names) {
return nil, errors.New("Invalid input format")
}
var mdatar models.Data
for i := range(ids) {
mdatar = append(mdatar, models.Single{ids[i], names[i], ...})
}
return models.JSON{"data": mdatar}, nil

Dynamically insert multiple documents using mgo golang mongodb

How can I insert an array of documents into MongoDB with mgo library using only a single DB call as in db.collection.insert()?
I have the following Transaction structure:
type Transaction struct {
Brand string `json:"brand"`
Name string `json:"name"`
Plu string `json:"plu"`
Price string `json:"price"`
}
From a POST request I will recieve an array of these structures. I want to insert them into MongoDB as individual documents but using a single DB call as explained in db.collection.insert()
I tried using c.Insert of mgo
The following is the code snippet:
func insertTransaction(c *gin.Context) {
var transactions []Transaction
err := c.BindJSON(&transactions)
if err != nil {
c.AbortWithStatusJSON(http.StatusBadRequest, map[string]string{"error":"invalid JSON"})
return
}
err = InsertTransactons(transactions)
if err != nil {
c.AbortWithStatusJSON(http.StatusInternalServerError, &map[string](interface{}){
"status": "error",
"code": "500",
"message": "Internal server error",
"error": err,
})
return
}
c.JSON(http.StatusCreated, &map[string](interface{}){
"status": "success",
"code": "0",
"message": "created",
})
}
func InsertTransactons(u []Transaction) error {
s := GetSession()
defer s.Close()
c := s.DB(DB).C(TransactionColl)
err := c.Insert(u...)
if err != nil {
return err
}
return nil
}
But as I compile and run the code, I get the following error:
go/database.go:34:17: cannot use u (type *[]Transaction) as type
[]interface {} in argument to c.Insert
You cannot pass []*Transaction as []interface{}. You need to convert each Transaction to inferface{} to change its memory layout.
var ui []interface{}
for _, t := range u{
ui = append(ui, t)
}
Pass ui to c.Insert instead
Create slice of interface for document structs by appending and then inserting data using Bulk insert which takes variable arguments.
type Bulk struct {
// contains filtered or unexported fields
}
func (b *Bulk) Insert(docs ...interface{})
For inserting documents in Bulk
const INSERT_COUNT int = 10000
type User struct {
Id bson.ObjectId `bson:"_id,omitempty" json:"_id"`
Email string `bson:"email" json:"email"`
}
func (self *User) Init() {
self.Id = bson.NewObjectId()
}
Call Bulk() function on collection returned from db connection. Bulk() function returns pointer to *Bulk.
bulk := dbs.Clone().DB("").C("users").Bulk()
bulk.Insert(users...)
Assign it to variable which will be used to call Insert() method using Bulk pointer receiver.
func main(){
// Database
dbs, err := mgo.Dial("mongodb://localhost/")
if err != nil {
panic(err)
}
// Collections
uc := dbs.Clone().DB("").C("users")
defer dbs.Clone().DB("").Session.Close()
for n := 0; n < b.N; n++ {
count := INSERT_COUNT
users := make([]interface{}, count)
for i := 0; i < count; i++ {
loop_user := User{}
loop_user.Init()
loop_user.Email = fmt.Sprintf("report-%d#example.com", i)
users[i] = loop_user
}
bulk := uc.Bulk()
bulk.Unordered()
bulk.Insert(users...)
_, bulkErr := bulk.Run()
if bulkErr != nil {
panic(err)
}
}
}

Convert json single element arrays to strings

In Go I have to parse this json:
{
"response": [
{
"message": [
"hello world"
],
"misc": [
{
"timestamp": [
"2017-06-28T05:52:39.347Z"
],
"server": [
"server-0101"
]
}
]
}
]
}
I'd like to get an object in Go that doesn't include all the unnecessary arrays of with a single string. The source json will never have more than one string in each array.
So the end result that I'd like to get would be this json:
{
"response": {
"message": "hello world",
"misc": {
"timestamp": "2017-06-28T05:52:39.347Z",
"server": "server-0101"
}
}
}
Or an equivalent object in Go.
Right now I have to use Response[0].Misc[0].Timestamp[0] to access the data which seems weird.
You can override the default behaviour of json.Marshal / json.Unmarshal methods for a struct, by defining its own MarshalJSON or UnmarshalJSON properly.
Here there is an excerpt for the code of a simplified version of the struct you need to decode.
type Response struct {
Message string `json:"message"`
}
// UnmarshalJSON overrides the default behaviour for JSON unmarshal method.
func (r *Response) UnmarshalJSON(data []byte) error {
auxResponse := &struct {
Message []string `json:"message"`
}{}
if err := json.Unmarshal(data, &auxResponse); err != nil {
return err
}
// Consider to add some checks on array length :)
r.Message = auxResponse.Message[0]
return nil
}
You can access the full working example here.
I suggest you to read this interesting article about custom JSON encode/decode with golang.
I'd like to get an object in Go that doesn't include all the unnecessary arrays of with a single string.
The hard way: Parse the JSON by hand (write our own parser).
The sensible way: Unmarshal via package encoding/json into some Go type matching the JSON or into some generic interface{} and copy the pieces into a different, simpler Go type afterwards.
Creating your own unmarshaller is probably best, but this is a quick way to simulate what you want to achieve.
package main
import (
"encoding/json"
"fmt"
)
// JSON ...
var JSON = `{
"response": [
{
"message": [
"hello world"
],
"misc": [
{
"timestamp": [
"2017-06-28T05:52:39.347Z"
],
"server": [
"server-0101"
]
}
]
}
]
}
`
type rawObject struct {
Response []struct {
Message []string `json:"message"`
Misc []interface{} `json:"misc"`
} `json:"response"`
}
type clean struct {
Message string `json:"message"`
Misc map[string]interface{} `json:"misc"`
}
func main() {
var o rawObject
var c clean
// init map
c.Misc = make(map[string]interface{})
// unmarshall the raw data
json.Unmarshal([]byte(JSON), &o)
for _, res := range o.Response { // I assume there should only be one response, don't know why this is wrapped as an array
// assume message is not really an array
c.Message = res.Message[0]
// convert []interface to map[string]interface
for _, m := range res.Misc {
for k, v := range m.(map[string]interface{}) {
c.Misc[k] = v
}
}
}
fmt.Printf("%+v\n", c)
}
What i don't like about this answer is that it isn't very reusable..so a function should probably be made and more error checking (part of creating a custom unmarshaller). If this were used in heavy production it might run into some memory issues, as I have to create a raw object to create a clean object.. but as a one off script it does the job. I my clean struct doesn't add response as a type because i find it to be redundant.

Parsing a JSON array of JSON objects?

I am trying to get each JSON object out of a JSON array. I get this data via a HTTP post.
I know what my data will look like:
{
"array":[
{
"entity_title":"University of Phoenix",
"entity_org_name":"CS Club",
"possible_user_name":"Johnny Ive",
"posibble_user_email":"Johhny.Ive#uop.edu",
"user_position_title":"President",
"msg_body_id":4
},
{
"entity_title":"University of San Francisco",
"entity_org_name":"Marketing club",
"possible_user_name":"steve jobs",
"posibble_user_email":"steven.job#uop.edu",
"user_position_title":"Student",
"msg_body_id":5
}
]
}
My example code and my structs look like this:
type MsgCreateUserArray struct {
CreateUser []MsgCreateUserJson `json:"createUserArray"`
}
type MsgCreateUserJson struct {
EntityTitleName string `json:"entity_title_name"`
EntityOrgName string `json:"entity_org_name"`
PossibleUserName string `json:"possible_user_name"`
PossibleUserEmail string `json:"possible_user_email"`
UserPositionTitle string `json:"user_position_title"`
MsgBodyId string `json:"msg_body_id, omitempty"`
}
func parseJson(rw http.ResponseWriter, request *http.Request) {
decodeJson := json.NewDecoder(request.Body)
var msg MsgCreateUserArray
err := decodeJson.Decode(&msg)
if err != nil {
panic(err)
}
log.Println(msg.CreateUser)
}
func main() {
http.HandleFunc("/", parseJson)
http.ListenAndServe(":1337", nil)
}
I am not sure where how to iterate over the JSON array and get the JSON objects and then just work with the JSON objects.
Try this as your structs,
type MsgCreateUserArray struct {
CreateUser []MsgCreateUserJson `json:"array"`
}
type MsgCreateUserJson struct {
EntityOrgName string `json:"entity_org_name"`
EntityTitle string `json:"entity_title"`
MsgBodyID int64 `json:"msg_body_id,omitempty"`
PosibbleUserEmail string `json:"posibble_user_email"`
PossibleUserName string `json:"possible_user_name"`
UserPositionTitle string `json:"user_position_title"`
}
Your entity_title_name is not named correctly, nor is the top level array. After you decode into a MsgCreateUserArray you can iterate over the CreateUser slice to get each MsgCreateUserJson

Resources