Unmarshal JSON Array in Go - arrays

I've read a few answers here and none seem to help. I keep getting the same error:
json: cannot unmarshal array into Go value of type A
type A struct {
Arr []string
}
type MA []A
func UnmarshalJSON() (ma MA, err error) {
jsonFile, err := os.Open(aJsonFile)
// error handling
defer jsonFile.Close()
byteValue, _ := ioutil.ReadAll(jsonFile)
err = json.Unmarshal(byteValue, &ma)
if err != nil {
return ma, err
}
return ma, nil
}
The json looks like this:
[
["Name", "Another Name", "Another"],
["A", "B"],
["W", "X", "Y", "Z"],
["Foo", "Bar"]
]
I've tried various different things from top answers when searching and as previously stated, nothing has worked.
I'm still decently new to Go and having to unmarshall stuff (I'm currently unmarshalling to [][]string but I want to use structs). What am I doing wrong?

A struct has fields; fields have names; the name of the field in your struct is Arr. So the json input would need to be [{ "arr": ["list", "of", "names"]}, {"arr": ["more", "names"]}] for instance, given the example you have above.
You can, however, define an UnmarshalJSON on your type named A, like this:
func (p *A) UnmarshalJSON(data []byte) error {
var s []string
err := json.Unmarshal(data, &s)
if err != nil {
return err
}
p.Arr = s
return nil
}
This receiver function named UnmarshalJSON takes a pointer to an A object, plus some sort of valid json input. Its job is to unmarshal that json. In this case we attempt to unmarshal into an ordinary slice-of-string—the variable s—which works as long as the json itself is a valid initializer for slice-of-strings.
If the unmarshal succeeds we then set p.Arr, knowing that the array is meant just for the thing named Arr (which is in fact the only member of the structure type) and return nil (no error).
Note that the last few lines could be written as:
if err == nil {
p.Arr = s
}
return err
which is shorter, but Go conventions generally handle the error case first, rather than letting it flow through. (I actually prefer the shorter code myself, slightly, but use whatever your group likes.)
(Given an UnmarshalJSON receiver on the type, there probably should be a json marshaler on the type, too, but my minimal test-and-example on the Go playground does not have one.)

your json content should like this
[
{
"Arr": [
"a0",
"a1"
]
},
{
"Arr": [
"b0",
"b1"
]
},
{
"Arr": [
"c0",
"c1"
]
}
]

Related

How to validate JSON when using unmarshal into generic interface?

I want to validate byte array data if it contains valid JSON using unmarsall method into interface.
package main
import (
"encoding/json"
"fmt"
)
func isJSON(s string) bool {
var js map[string]interface{}
return json.Unmarshal([]byte(s), &js) == nil
}
func main() {
var tests = []string{
`{"a":"b"}`,
`[{"a":"b"},{"a":"b"}]`,
}
for _, t := range tests {
fmt.Printf("isJSON(%s) = %v\n\n", t, isJSON(t))
}
}
Both input test parameters are valid JSON strings, but it validate based on the interface 'map[string]interface{}'
{
"a": "b"
}
[{
"a": "b"
}, {
"a": "b"
}]
I want to validate the JSON text. JSON text is a serialized object or array. Hence looking for a solution which support all valid cases for JSON text as I added test cases in playground.
How can I make this interface i.e var map[string]interface{} generic so it support both test cases of valid JSON string?
Solved: Playground Link
Don't use map[string]interface{} but simply interface{}. The second example is of type []interface{} and there are more types of valid json.
Here your working code. I added a few more cases of valid json.
playground
Here the code if you want to allow only maps and slices:
playground

In Go, how to Unmarshal bson byte[] data into an array of structs?

What's the best way to Unmarshal bson byte[] data into an array of structs, when the array of structs is passed into an interface{} parameter?
For demonstration purposes, in the following code, I use bson.Marshal() on the inStructArr to get the byte[] type of data. This is so I can use bson.Unmarshal(...) to pipe into the outStructArr.
import "gopkg.in/mgo.v2/bson"
type User struct {
Name string
}
func DecodeArrData(inStructArr, outStructArr interface{}) {
inStructArrData, _ := bson.Marshal(inStructArr)
bson.Unmarshal(inStructArrData, outStructArr) // <-- Error happens here
// What's the right way of accomplishing this?
}
func Main() {
outUsers := &[]User{}
inUsers := []User{User{"A"}, User{"B"}}
DecodeArrData(inUsers, outUsers)
}
When I do this, the error-message I get is: Unsupported document type for unmarshalling: []User. What's the right way of doing this?
Thanks in advance!
The Marshal and Unmarshal functions work with BSON documents, not BSON arrays.
Wrap the slices in a struct to provide the document expected by the package:
func DecodeArrData(inStructArr, outStructArr interface{}) error {
in := struct{ Data interface{} }{Data: inStructArr}
inStructArrData, err := bson.Marshal(in)
if err != nil {
return err
}
var out struct{ Data bson.Raw }
if err := bson.Unmarshal(inStructArrData, &out); err != nil {
return err
}
return out.Data.Unmarshal(outStructArr)
}
If you are willing to take advantage of an undocumented feature of the Marshal function and add some BSON format knowledge to your application, then you can omit the wrapper.
The undocumented feature of Marshal is that it encodes slices as BSON arrays. The BSON array can be decoded using a bson.Raw value with Kind set the the BSON code for arrays (the value 4) and Data set to the array data:
func DecodeArrData(inStructArr, outStructArr interface{}) error {
inStructArrData, err := bson.Marshal(inStructArr)
if err != nil {
return err
}
raw := bson.Raw{Kind: 4, Data: inStructArrData}
return raw.Unmarshal(outStructArr)
}

Golang generic JSON marshalling

I'm working on s small server-client project based on JSON communication. But i run into issues. I'm trying to create a response struct with a generic message body. This means I have an map with a key as string and a json raw message as value. In the end the message body should work for any type (strings, integers, arrays)
package main
import (
"encoding/json"
"fmt"
)
type ServerResponse struct {
Code int `json:"code" bson:"code"`
Type string `json:"type" bson:"type"`
Body map[string]json.RawMessage `json:"body" bson:"body"`
}
func NewServerResponse() *ServerResponse {
return &ServerResponse{Body: make(map[string]json.RawMessage)}
}
func main(){
serverResponse := NewServerResponse()
serverResponse.Code = 100
serverResponse.Type = "molly"
serverResponse.Body["string"] = json.RawMessage("getIt")
serverResponse.Body["integer"] = json.RawMessage{200}
serverResponse.Body["array"] = json.RawMessage(`["a", "b", "c"]`)
if d, err := json.Marshal(&serverResponse); err != nil{
fmt.Println("Error " + err.Error())
}else{
fmt.Println(string(d))
}
}
But the output is as follow.
{
"code":100,
"type":"molly",
"body": {
"array":"WyJhIiwgImIiLCAiYyJd",
"integer":"yA==",
"string":"Z2V0SXQ="
}
}
It seems like the values are Base64 encoded and inside double quotes. Tihs should be the expected output
{
"code":100,
"type":"molly",
"body": {
"array":["a", "b", "c"],
"integer":200,
"string":"getIt"
}
}
Is this even possible? Or do I have to write a specific struct type for every response?
The raw message must be valid JSON.
Add quotes to the string to make it a valid JSON string.
serverResponse.Body["string"] = json.RawMessage("\"getIt\"")
JSON numbers are a sequence of decimal bytes. A number is not the value of a single byte as attempted in the question.
serverResponse.Body["integer"] = json.RawMessage("200")
This one works as you expected.
serverResponse.Body["array"] = json.RawMessage(`["a", "b", "c"]`)
The program in the question compiles and runs with errors. Examining those errors and fixing them leads to my suggestions above.
An alternative approach is to replace the use of json.RawMessage with interface{}:
type ServerResponse struct {
Code int `json:"code" bson:"code"`
Type string `json:"type" bson:"type"`
Body map[string]interface{} `json:"body" bson:"body"`
}
Set the response body like this:
serverResponse.Body["string"] = "getIt"
serverResponse.Body["integer"] = 200
serverResponse.Body["array"] = []string{"a", "b", "c"}
You can use json.RawMessage values:
serverResponse.Body["array"] = json.RawMessage(`["a", "b", "c"]`)
Playground example

Serialize a mixed type JSON array in Go

I want to return a structure that looks like this:
{
results: [
["ooid1", 2.0, "Söme text"],
["ooid2", 1.3, "Åther text"],
]
}
That's an array of arrags that is string, floating point number, unicode character.
If it was Python I'd be able to:
import json
json.dumps({'results': [["ooid1", 2.0, u"Söme text"], ...])
But in Go you can't have an array (or slice) of mixed types.
I thought of using a struct like this:
type Row struct {
Ooid string
Score float64
Text rune
}
But I don't want each to become a dictionary, I want it to become an array of 3 elements each.
We can customize how an object is serialized by implementing the json.Marshaler interface. For our particular case, we seem to have a slice of Row elements that we want to encode as an array of heterogenous values. We can do so by defining a MarshalJSON function on our Row type, using an intermediate slice of interface{} to encode the mixed values.
This example demonstrates:
package main
import (
"encoding/json"
"fmt"
)
type Row struct {
Ooid string
Score float64
Text string
}
func (r *Row) MarshalJSON() ([]byte, error) {
arr := []interface{}{r.Ooid, r.Score, r.Text}
return json.Marshal(arr)
}
func main() {
rows := []Row{
{"ooid1", 2.0, "Söme text"},
{"ooid2", 1.3, "Åther text"},
}
marshalled, _ := json.Marshal(rows)
fmt.Println(string(marshalled))
}
Of course, we also might want to go the other way around, from JSON bytes back to structs. So there's a similar json.Unmarshaler interface that we can use.
func (r *Row) UnmarshalJSON(bs []byte) error {
arr := []interface{}{}
json.Unmarshal(bs, &arr)
// TODO: add error handling here.
r.Ooid = arr[0].(string)
r.Score = arr[1].(float64)
r.Text = arr[2].(string)
return nil
}
This uses a similar trick of first using an intermediate slice of interface{}, using the unmarshaler to place values into this generic container, and then plop the values back into our structure.
package main
import (
"encoding/json"
"fmt"
)
type Row struct {
Ooid string
Score float64
Text string
}
func (r *Row) UnmarshalJSON(bs []byte) error {
arr := []interface{}{}
json.Unmarshal(bs, &arr)
// TODO: add error handling here.
r.Ooid = arr[0].(string)
r.Score = arr[1].(float64)
r.Text = arr[2].(string)
return nil
}
func main() {
rows := []Row{}
text := `
[
["ooid4", 3.1415, "pi"],
["ooid5", 2.7182, "euler"]
]
`
json.Unmarshal([]byte(text), &rows)
fmt.Println(rows)
}
You can read a full example here.
Use []interface{}
type Results struct {
Rows []interface{} `json:"results"`
}
You will then have to use type assertion if you want to access the values stored in []interface{}
for _, row := range results.Rows {
switch r := row.(type) {
case string:
fmt.Println("string", r)
case float64:
fmt.Println("float64", r)
case int64:
fmt.Println("int64", r)
default:
fmt.Println("not found")
}
}
Some clumsy, but you can
type result [][]interface{}
type results struct {
Results result
}
Working example https://play.golang.org/p/IXAzZZ3Dg7

Unmarshalling top-level JSON array in Go

I'm learning Go by writing a simple http server and I need to handle some JSON responses.
With an object response, I can unmarshal it idiomatically with 2 lines of code:
structResult := Foo{}
json.Unmarshal(structBody, &structResult)
I don't know how to do the same for an array response (see the example below). Is there a way to specify (possibly via json tag) that top-level array should go into a given struct field?
package main
import "fmt"
import "encoding/json"
type Foo struct {
Id uint64 `json:"id"`
Name string `json:"name"`
}
type BaseResult struct {
Error string `json:"error"`
}
type FooResult struct {
BaseResult
Foos []Foo
}
func main() {
// Simple and works.
structBody := []byte(`{"id": 1,"name": "foo"}`)
structResult := Foo{}
json.Unmarshal(structBody, &structResult)
fmt.Printf("%#v\n", structResult)
// Doesn't work.
arrayBody := []byte(`[{"id": 1,"name": "foo"},{"id": 2,"name": "bar"},{"id": 3,"name": "foobar"}]`)
arrayResult := FooResult{}
json.Unmarshal(arrayBody, &arrayResult)
fmt.Printf("%#v\n", arrayResult)
}
I know I could make FooResult an array:
type FooResult []Foo
but then I lose the ability to specify base object which I would like to use to store error message and such. I also know that I can unmarshal into &fooResult.Foos directly, but I want the code to work with both objects and arrays.
UPDATE
Implementing UnmarshalJSON as suggested by #dyoo partially solves my problem, but I was hoping that I could use BaseResult to store parse error in case JSON has a different structure:
arrayBody := []byte(`{"error": "foo"}`)
arrayResult := FooResult{}
json.Unmarshal(arrayBody, &arrayResult)
fmt.Printf("%#v\n", arrayResult)
Of course I could implement more complex logic inside UnmarshalJSON - but isn't there a simpler way to do it?
You can implement the json.Unmarshaler interface in your FooResult, to customize exactly how it responds to unmarshaling. (Similarly, there's a json.Marshaler interface.)
Add:
func (f *FooResult) UnmarshalJSON(bs []byte) error {
return json.Unmarshal(bs, &f.Foos)
}
after which your code should otherwise work. http://play.golang.org/p/oMdoB2e-rB
You might try something like:
func (f *FooResult) UnmarshalJSON(bs []byte) error {
err1 := json.Unmarshal(bs, &f.BaseResult)
err2 := json.Unmarshal(bs, &f.Foos)
if err1 != nil && err2 != nil {
// Arbitrarily choose an error.
return err1
}
return nil
}
although even this is beginning to look dubious. Handling union type results is not quite what the json library is designed to handle automatically for you. You'll need to explicitly code the coercion logic if your JSON has dynamic type.
See: How to unmarshall an array of different types correctly? and http://blog.golang.org/json-and-go for related issues.
Just specify Foos when you Unmarshal
package main
import "fmt"
import "encoding/json"
type Foo struct {
Id uint64 `json:"id"`
Name string `json:"name"`
}
type BaseResult struct {
Error string `json:"error"`
}
type FooResult struct {
BaseResult
Foos []Foo
}
func main() {
// Simple and works.
structBody := []byte(`{"id": 1,"name": "foo"}`)
structResult := Foo{}
json.Unmarshal(structBody, &structResult)
fmt.Printf("%#v\n", structResult)
// Doesn't work.
arrayBody := []byte(`[{"id": 1,"name": "foo"},{"id": 2,"name": "bar"},{"id": 3,"name": "foobar"}]`)
arrayResult := FooResult{}
if err := json.Unmarshal(arrayBody, &arrayResult.Foos); err != nil {
arrayResult.BaseResult.Error = string(arrayBody)
}
fmt.Printf("%#v\n", arrayResult)
}

Resources