Get Array of Nested JSON Struct in GO - arrays

I am new to GoLang, and have a question about filling an array from nested JSON data. I have looked through Stack overflow yesterday and cannot find this exact topic, only threads that are similar, but do not provide a direct solution.
Lets say I have some nested JSON data like what is given below:
How can I create a nested struct to fill an array of the close prices. My code is given below.
My goal is to have an array where, arr = {157.92, 142.19, 148.26}
Thanks in advance! I greatly appreciate any help!
{
"history": {
"day": [
{
"date": "2019-01-02",
"open": 154.89,
"high": 158.85,
"low": 154.23,
"close": 157.92,
"volume": 37039737
},
{
"date": "2019-01-03",
"open": 143.98,
"high": 145.72,
"low": 142.0,
"close": 142.19,
"volume": 91312195
},
{
"date": "2019-01-04",
"open": 144.53,
"high": 148.5499,
"low": 143.8,
"close": 148.26,
"volume": 58607070
}
...
]
}
}
// DATA STRUCTURE
type Hist struct {
History string `json:"history"`
}
type Date struct {
Day string `json:"day"`
}
type Price struct {
Close []string `json:"close"`
}
// HISTORICAL QUOTES
func get_quotes(arg1 string, arg2 string, arg3 string, arg4 string) []string {
// arg1 = ticker symbol, arg2 = start, arg3 = end, arg4 = access token
// TRADIER API
apiUrl := "https://sandbox.tradier.com/v1/markets/history?symbol=" + arg1 + "&interval=daily&start=" + arg2 + "&end=" + arg3
u, _ := url.ParseRequestURI(apiUrl)
urlStr := u.String()
client := &http.Client{}
r, _ := http.NewRequest("GET", urlStr, nil)
r.Header.Add("Authorization", "Bearer "+arg4)
r.Header.Add("Accept", "application/json")
resp, _ := client.Do(r)
responseData, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatal(err)
}
fmt.Println(resp.Status)
fmt.Println(string(responseData))
var response Price
json.NewDecoder(resp.Body).Decode(&response)
fmt.Println(response.Close)
return response.Close
}

Something like this should give you what you need. Your data structure does not correctly reflect the response from the API. I used this tool to quickly convert the JSON value into a Go struct type. Once you're decoding the response correctly, then it's just a matter of iterating over each Day struct and appending the close value to an output array.
I added the core stuff for decoding and mapping the values. You can handle customizing the client, request, and headers however you'd like by combining what you've already got.
package main
import (
"encoding/json"
"fmt"
"log"
"net/http"
)
type Response struct {
History History `json:"history"`
}
type Day struct {
Date string `json:"date"`
Open float64 `json:"open"`
High float64 `json:"high"`
Low float64 `json:"low"`
Close float64 `json:"close"`
Volume int `json:"volume"`
}
type History struct {
Day []Day `json:"day"`
}
func main() {
prices, err := closePrices()
if err != nil {
log.Fatal(err)
}
fmt.Println(prices)
}
func closePrices() (out []float64, err error) {
resp, err := http.Get("...")
if err != nil {
return
}
r := Response{}
err = json.NewDecoder(resp.Body).Decode(&r)
if err != nil {
return
}
for _, d := range r.History.Day {
out = append(out, d.Close)
}
return
}

Related

How to unmarshal objects inside array

I am trying to get access to object's values inside of array
[
{
"name": "London",
"lat": 51.5073219,
"lon": -0.1276474,
"country": "GB",
"state": "England"
}
]
I use this code to unmarshal it
content, err := ioutil.ReadAll(res.Body)
if err != nil {
log.Fatal(err)
}
var data []ResponseData
err = json.Unmarshal(content, &data)
if err != nil {
log.Fatal(err)
}
This is my struct
type ResponseData struct {
Name string `json:"name"`
Lat float32 `json:"lat"`
Lon float32 `json:"lon"`
Country string `json:"country"`
State string `json:"state"`
}
I need to simply fmt.Println(data.Lat, data.Lon) later.
The code you presented should unmarshal your JSON successfully; the issue is with the way you are trying to use the result. You say you want to use fmt.Println(data.Lat, data.Lon) but this will not work because data is a slice ([]ResponseData) not a ResponseData. You could use fmt.Println(data[0].Lat, data[0].Lon) (after checking the number of elements!) or iterate through the elements.
The below might help you experiment (playground - this contains a little more content than below):
package main
import (
"encoding/json"
"fmt"
"log"
)
const rawJSON = `[
{
"name": "London",
"lat": 51.5073219,
"lon": -0.1276474,
"country": "GB",
"state": "England"
}
]`
type ResponseData struct {
Name string `json:"name"`
Lat float32 `json:"lat"`
Lon float32 `json:"lon"`
Country string `json:"country"`
State string `json:"state"`
}
func main() {
var data []ResponseData
err := json.Unmarshal([]byte(rawJSON), &data)
if err != nil {
log.Fatal(err)
}
if len(data) == 1 { // Would also work for 2+ but then you are throwing data away...
fmt.Println("test1", data[0].Lat, data[0].Lon)
}
for _, e := range data {
fmt.Println("test2", e.Lat, e.Lon)
}
}

Getting file content into a multidimensional string var

I'm using the fsnotify packet to wait for changes in a json file.
I have two problems with this code. The first one is regarding the info returned by ReadFile function. Looks like when I print something returned by the function is empty.
Second issue is regarding the fsnotify that is not reading the file the first time unless i do some modification on the content. I must read the file from the beggining as well.
type Information struct {
Info []Info `json:"info"`
}
type Info struct {
Type string `json:"type"`
News []New `json:"news"`
}
type New struct {
Name string `json:"name"`
Read bool `json:"read"`
}
func ReadFile(file_name string) *Information {
jsonFile, err := os.Open(file_name)
if err != nil {
fmt.Println(err)
}
fmt.Println("Successfully Opened file_name.json")
defer jsonFile.Close()
byteValue, _ := ioutil.ReadAll(jsonFile)
var infor Information
json.Unmarshal(byteValue, &infor)
return &infor
}
// main function
func main() {
// read json file using fsnotify to wait for changes
watcher, err := fsnotify.NewWatcher()
if err != nil {
panic(err)
}
err = watcher.Add(file_json)
if err != nil {
panic(err)
}
for {
select {
case ev, ok := <-watcher.Events:
log.Println("event:", ev)
if !ok {
return
}
if ev.Op&fsnotify.Write == fsnotify.Write {
data := ReadFile(file_name)
fmt.Print("INFORMATION ABOUT FILE:\n")
for _, info := range data.Info {
fmt.Printf("Info type: %s\n", info.Type) // Here is not printing the result of info.Type
for _, news := range info.News {
fmt.Printf("News Name: %s\n", news.Name) // Here is not printing even "News Name:" or News Read:"
fmt.Printf("News Read: %s\n", strconv.FormatBool(news.Read))
}
}
}
case err := <-watcher.Errors:
log.Println("error:", err)
}
}
}
This is the json file:
{
    "info": [
      {
        "type": "general",
        "news": [
          { "name": "abc",  "read": true },
          { "name": "def",  "read": true }
        ]
      },
{
"type": "confidential",
"news": [
{ "name": "xxx", "read": false },
{ "name": "yyy", "read": false }
]
},
]
}
type Information struct {
Info []Info `json:"info"`
}
type Info struct {
Type string `json:"type"`
News []New `json:"news"` // Here should be a slice define.
}
type New struct {
Name string `json:"name"`
Read bool `json:"read"`
}
func ReadFile(file_name string) *Information {
jsonFile, err := os.Open(file_name)
if err != nil {
fmt.Println(err)
}
fmt.Println("Successfully Opened file_name.json")
defer jsonFile.Close()
byteValue, _ := ioutil.ReadAll(jsonFile)
var infor Information
json.Unmarshal(byteValue, &infor)
return &infor
}
func main() {
data := ReadFile("./data.json")
for _, news := range data.Info {
for _, v := range news.News {
name := v.Name
// Add you want to do
fmt.Println(name)
}
}
}
You can not get like this:
getInfo = [general][abc, true, def, true]
[confidential][xxx, false, yyy, false]

How to encode/marshal response from my server in multiple object JSON array

I am trying to format my response from a server in an array of multiple object JSON. I tried slice and []Struct but I believe I am not able to understand their usage correctly.
I tried the method mentioned here - https://play.golang.org/p/9OEPzbf0Me0 but didn't work in my case
I am using golang to get response for a query parameter
router.HandleFunc("/v1.0/singles", GetInfo).Methods("GET").Queries("Group", "{group}")
//controller.go
func SendToServer(jsonValue []byte, command string) (models.JSON, error) {
var pdatar models.JSON
s := []string{"http://172.xx.xxx.xx:xxxxx/", command}
response, err := http.Post(strings.Join(s, ""), "application/json", bytes.NewBuffer(jsonValue))
data, err := ioutil.ReadAll(response.Body)
err = json.Unmarshal(data, &pdatar)
mdatar := make(models.JSON)
mdatar["ID"] = pdatar["id"]
mdatar["Name"] = pdatar["name"]
fmt.Println(mdatar) //This is how it is coming map[ID:[877,235], Name:[FCU, GHP]]
/*Another attempt
var temp models.Arr
json.Unmarshal(data, &temp)
fmt.Println(temp) ---- returning null
*/
return mdatar, nil
}
func (c Controller) GetSingle(pdata models.Single) (models.JSON, error) {
mdata := make(models.JSON)
mdata["group"] = pdata.Group
jsonValue, err := json.Marshal(mdata)
pdatar, err := SendToServer(jsonValue, "func_info")
return pdatar, nil
// json.NewEncoder(w).Encode(pdatar) //sending this to my main function
}
//model.go
type JSON map[string]interface{}
type Single struct {
ID int `json:"id"`
Name string `json:"name"`
}
type Data []Single
type Arr [][]string
I have a response from my API in a format
{
"id":[877,235,312,429],
"name" ["FCU","GHP","c","d"],
"group":["x","x","y","z"]
}
With Current Code I am receiving a response
{"id":[877 235], "name": ["FCU" "GHP"]
Though I am expecting a response
{
"data":
[
{"id": 877
"name" : "FCU"
}
{"id": 235
"name": "GHP"
}
]
}
Your mdatarr doesn't change the format any, so this result makes sense.
If the data received from the server is in the format:
{
"id": [877,235,312,429],
"name" ["FCU","GHP","c","d"],
"group": ["x","x","y","z"]
}
then you will need translate it. Ideally you'd fix the server so that it sends data in the format:
{
{
"id": 877,
"name": "FCU",
"group": "x",
},
...
}
If the server could send something like the above, then you could simply unmarshal it into a models.Data object.
If that is not an option, and you really need to do the translation client-side, then you'll need a for loop that does something like this:
ids := pdatarr["id"]
names := pdatarr["name"]
...
if len(ids) != len(names) {
return nil, errors.New("Invalid input format")
}
var mdatar models.Data
for i := range(ids) {
mdatar = append(mdatar, models.Single{ids[i], names[i], ...})
}
return models.JSON{"data": mdatar}, nil

Dynamically insert multiple documents using mgo golang mongodb

How can I insert an array of documents into MongoDB with mgo library using only a single DB call as in db.collection.insert()?
I have the following Transaction structure:
type Transaction struct {
Brand string `json:"brand"`
Name string `json:"name"`
Plu string `json:"plu"`
Price string `json:"price"`
}
From a POST request I will recieve an array of these structures. I want to insert them into MongoDB as individual documents but using a single DB call as explained in db.collection.insert()
I tried using c.Insert of mgo
The following is the code snippet:
func insertTransaction(c *gin.Context) {
var transactions []Transaction
err := c.BindJSON(&transactions)
if err != nil {
c.AbortWithStatusJSON(http.StatusBadRequest, map[string]string{"error":"invalid JSON"})
return
}
err = InsertTransactons(transactions)
if err != nil {
c.AbortWithStatusJSON(http.StatusInternalServerError, &map[string](interface{}){
"status": "error",
"code": "500",
"message": "Internal server error",
"error": err,
})
return
}
c.JSON(http.StatusCreated, &map[string](interface{}){
"status": "success",
"code": "0",
"message": "created",
})
}
func InsertTransactons(u []Transaction) error {
s := GetSession()
defer s.Close()
c := s.DB(DB).C(TransactionColl)
err := c.Insert(u...)
if err != nil {
return err
}
return nil
}
But as I compile and run the code, I get the following error:
go/database.go:34:17: cannot use u (type *[]Transaction) as type
[]interface {} in argument to c.Insert
You cannot pass []*Transaction as []interface{}. You need to convert each Transaction to inferface{} to change its memory layout.
var ui []interface{}
for _, t := range u{
ui = append(ui, t)
}
Pass ui to c.Insert instead
Create slice of interface for document structs by appending and then inserting data using Bulk insert which takes variable arguments.
type Bulk struct {
// contains filtered or unexported fields
}
func (b *Bulk) Insert(docs ...interface{})
For inserting documents in Bulk
const INSERT_COUNT int = 10000
type User struct {
Id bson.ObjectId `bson:"_id,omitempty" json:"_id"`
Email string `bson:"email" json:"email"`
}
func (self *User) Init() {
self.Id = bson.NewObjectId()
}
Call Bulk() function on collection returned from db connection. Bulk() function returns pointer to *Bulk.
bulk := dbs.Clone().DB("").C("users").Bulk()
bulk.Insert(users...)
Assign it to variable which will be used to call Insert() method using Bulk pointer receiver.
func main(){
// Database
dbs, err := mgo.Dial("mongodb://localhost/")
if err != nil {
panic(err)
}
// Collections
uc := dbs.Clone().DB("").C("users")
defer dbs.Clone().DB("").Session.Close()
for n := 0; n < b.N; n++ {
count := INSERT_COUNT
users := make([]interface{}, count)
for i := 0; i < count; i++ {
loop_user := User{}
loop_user.Init()
loop_user.Email = fmt.Sprintf("report-%d#example.com", i)
users[i] = loop_user
}
bulk := uc.Bulk()
bulk.Unordered()
bulk.Insert(users...)
_, bulkErr := bulk.Run()
if bulkErr != nil {
panic(err)
}
}
}

decoding nested json objects in go

I found some posts on how to decoding json nested objects in go, I tried to apply the answers to my problem, but I only managed to find a partial solution.
My json file look like this:
{
"user":{
"gender":"male",
"age":"21-30",
"id":"80b1ea88-19d7-24e8-52cc-65cf6fb9b380"
},
"trials":{
"0":{"index":0,"word":"WORD 1","Time":3000,"keyboard":true,"train":true,"type":"A"},
"1":{"index":1,"word":"WORD 2","Time":3000,"keyboard":true,"train":true,"type":"A"},
},
"answers":{
"training":[
{"ans":0,"RT":null,"gtAns":"WORD 1","correct":0},
{"ans":0,"RT":null,"gtAns":"WORD 2","correct":0}
],
"test":[
{"ans":0,"RT":null,"gtAns":true,"correct":0},
{"ans":0,"RT":null,"gtAns":true,"correct":0}
]
}
}
Basically I need to parse the information inside it and save them into go structure. With the code below I managed to extract the user information, but it looks too complicated to me and it won't be easy to apply the same thing to the "answers" fields which contains 2 arrays with more than 100 entries each. Here the code I'm using now:
type userDetails struct {
Id string `json:"id"`
Age string `json:"age"`
Gender string `json:"gender"`
}
type jsonRawData map[string]interface {
}
func getJsonContent(r *http.Request) ( userDetails) {
defer r.Body.Close()
jsonBody, err := ioutil.ReadAll(r.Body)
var userDataCurr userDetails
if err != nil {
log.Printf("Couldn't read request body: %s", err)
} else {
var f jsonRawData
err := json.Unmarshal(jsonBody, &f)
if err != nil {
log.Printf("Error unmashalling: %s", err)
} else {
user := f["user"].(map[string]interface{})
userDataCurr.Id = user["id"].(string)
userDataCurr.Gender = user["gender"].(string)
userDataCurr.Age = user["age"].(string)
}
}
return userDataCurr
}
Any suggestions? Thanks a lot!
You're doing it the hard way by using interface{} and not taking advantage of what encoding/json gives you.
I'd do it something like this (note I assumed there was an error with the type of the "gtAns" field and I made it a boolean, you don't give enough information to know what to do with the "RT" field):
package main
import (
"encoding/json"
"fmt"
"io"
"log"
"strconv"
"strings"
)
const input = `{
"user":{
"gender":"male",
"age":"21-30",
"id":"80b1ea88-19d7-24e8-52cc-65cf6fb9b380"
},
"trials":{
"0":{"index":0,"word":"WORD 1","Time":3000,"keyboard":true,"train":true,"type":"A"},
"1":{"index":1,"word":"WORD 2","Time":3000,"keyboard":true,"train":true,"type":"A"}
},
"answers":{
"training":[
{"ans":0,"RT":null,"gtAns":true,"correct":0},
{"ans":0,"RT":null,"gtAns":true,"correct":0}
],
"test":[
{"ans":0,"RT":null,"gtAns":true,"correct":0},
{"ans":0,"RT":null,"gtAns":true,"correct":0}
]
}
}`
type Whatever struct {
User struct {
Gender Gender `json:"gender"`
Age Range `json:"age"`
ID IDString `json:"id"`
} `json:"user"`
Trials map[string]struct {
Index int `json:"index"`
Word string `json:"word"`
Time int // should this be a time.Duration?
Train bool `json:"train"`
Type string `json:"type"`
} `json:"trials"`
Answers map[string][]struct {
Answer int `json:"ans"`
RT json.RawMessage // ??? what type is this
GotAnswer bool `json:"gtAns"`
Correct int `json:"correct"`
} `json:"answers"`
}
// Using some custom types to show custom marshalling:
type IDString string // TODO custom unmarshal and format/error checking
type Gender int
const (
Male Gender = iota
Female
)
func (g *Gender) UnmarshalJSON(b []byte) error {
var s string
err := json.Unmarshal(b, &s)
if err != nil {
return err
}
switch strings.ToLower(s) {
case "male":
*g = Male
case "female":
*g = Female
default:
return fmt.Errorf("invalid gender %q", s)
}
return nil
}
func (g Gender) MarshalJSON() ([]byte, error) {
switch g {
case Male:
return []byte(`"male"`), nil
case Female:
return []byte(`"female"`), nil
default:
return nil, fmt.Errorf("invalid gender %v", g)
}
}
type Range struct{ Min, Max int }
func (r *Range) UnmarshalJSON(b []byte) error {
// XXX could be improved
_, err := fmt.Sscanf(string(b), `"%d-%d"`, &r.Min, &r.Max)
return err
}
func (r Range) MarshalJSON() ([]byte, error) {
return []byte(fmt.Sprintf(`"%d-%d"`, r.Min, r.Max)), nil
// Or:
b := make([]byte, 0, 8)
b = append(b, '"')
b = strconv.AppendInt(b, int64(r.Min), 10)
b = append(b, '-')
b = strconv.AppendInt(b, int64(r.Max), 10)
b = append(b, '"')
return b, nil
}
func fromJSON(r io.Reader) (Whatever, error) {
var x Whatever
dec := json.NewDecoder(r)
err := dec.Decode(&x)
return x, err
}
func main() {
// Use http.Get or whatever to get an io.Reader,
// (e.g. response.Body).
// For playground, substitute a fixed string
r := strings.NewReader(input)
// If you actually had a string or []byte:
// var x Whatever
// err := json.Unmarshal([]byte(input), &x)
x, err := fromJSON(r)
if err != nil {
log.Fatal(err)
}
fmt.Println(x)
fmt.Printf("%+v\n", x)
b, err := json.MarshalIndent(x, "", " ")
if err != nil {
log.Fatal(err)
}
fmt.Printf("Re-marshalled: %s\n", b)
}
Playground
Of course if you want to reuse those sub-types you could pull them out of the "Whatever" type into their own named types.
Also, note the use of a json.Decoder rather than reading in all the data ahead of time. Usually try and avoid any use of ioutil.ReadAll unless you really need all the data at once.

Resources