Handling Null JSON Array in Go using struct - arrays

we have struct and getting null after append struct in golang.
Find below struct with my some part of code in golang.
type XmloutRoomRate struct {
CancellationPolicy Policies `bson:"cancellationPolicy" json:"cancellationPolicy"`
}
type Policies struct {
Policies []RoomPolicies `bson:"policies" json:"policies"`
}
type RoomPolicies struct {
Amount float64 `bson:"amount" json:"amount"`
DaysBeforeCheckIn int `bson:"daysBeforeCheckIn" json:"daysBeforeCheckIn"`
}
cancelPolicyMain := Policies{}
cancelPolicy := RoomPolicies{}
if cancelAmount < 0 {
cancelPolicy.Amount = cancelAmount
cancelPolicy.DaysBeforeCheckIn = cancelDay
cancelPolicyMain.Policies = append(cancelPolicyMain.Policies, cancelPolicy)
}else{
cancelPolicyMain = agodaPolicies{}
cancelPolicyMain.Policies = append(cancelPolicyMain.Policies)
}
when data present getting proper data structure.
"cancellationPolicy": {
"policies": [
{
"amount": 5141.58,
"daysBeforeCheckIn": 5
}
]
}
But when data not available getting struct with null value.
"cancellationPolicy": {
"policies": null
}
We need my actual output with blank array [].
"cancellationPolicy": {
"policies": []
}

nil slice values are marshaled into JSON null values. This is documented at json.Marshal():
Array and slice values encode as JSON arrays, except that []byte encodes as a base64-encoded string, and a nil slice encodes as the null JSON value.
Non-nil empty slices are marshaled into empty JSON arrays. So simply initialize Policies.Policies to a non-nil empty slice, and it will be [] in the output:
cancelPolicyMain = Policies{Policies: []RoomPolicies{}}
Test code:
const cancelDay = 1
for cancelAmount := -500.0; cancelAmount <= 501; cancelAmount += 1000 {
cancelPolicyMain := Policies{}
cancelPolicy := RoomPolicies{}
if cancelAmount < 0 {
cancelPolicy.Amount = cancelAmount
cancelPolicy.DaysBeforeCheckIn = cancelDay
cancelPolicyMain.Policies = append(cancelPolicyMain.Policies, cancelPolicy)
} else {
cancelPolicyMain = Policies{Policies: []RoomPolicies{}}
cancelPolicyMain.Policies = append(cancelPolicyMain.Policies)
}
x := XmloutRoomRate{cancelPolicyMain}
if err := json.NewEncoder(os.Stdout).Encode(x); err != nil {
panic(err)
}
}
Output (try it on the Go Playground):
{"cancellationPolicy":{"policies":[{"amount":-500,"daysBeforeCheckIn":1}]}}
{"cancellationPolicy":{"policies":[]}}

In an array, the meaning of a "null" entry is clear: It means this array entry is missing.
In an object aka dictionary, there are different ways to indicate "no entry": The key might not be there. Or the key might not be there, but with an empty array as value. Or the key might be there, but with a "null" value.
You really need agreement between the provider of the data and the client processing it, what each of these mean. And since it's often hard to change what the data provider does, translate what you get into what you need.
So you have to decide what it means if "policies" does not exist as a key, or if it exists as a null value. I've seen software that wouldn't produce arrays with one element, but would provide the single element instead. So "policies": { "amount": ..., "daysBeforeCheckin": ... } would also be possible. You decide what you accept, what you treat as an array, and how you change from the form you got to the form you want.

Related

Searching for a specific key-value in GoLang buffer variable

I have a variable buffer that stores a set of key-value pairs in the array of the form:
[{"Key":"area1", "Record": {"name":"belfast","type":"surburban","validity":"true"}},{Key,Record},{Key,Record}....] i.e a set of Key-Record pairs in a buffer array.
Now I want to retrieve only the key-record pairs that have a specific record entry, for example i want only records that have the value "true" in validity, I want to return all key-record pairs that have the validity field value as true. Any suggestions ? Thanks
Here is a code segment of how the key-record pairs are created after which I want to filter key-records that have validity as true
var buffer bytes.Buffer
buffer.WriteString("[")
bArrayMemberAlreadyWritten := false
for resultsIterator.HasNext() {
queryResponse, err := resultsIterator.Next()
if err != nil {
return shim.Error(err.Error())
}
// Add a comma before array members, suppress it for the first array member
if bArrayMemberAlreadyWritten == true {
buffer.WriteString(",")
}
buffer.WriteString("{\"Key\":")
buffer.WriteString("\"")
buffer.WriteString(queryResponse.Key)
buffer.WriteString("\"")
buffer.WriteString(", \"Record\":")
// Record is a JSON object, so we write as-is
buffer.WriteString(string(queryResponse.Value))
buffer.WriteString("}")
bArrayMemberAlreadyWritten = true
}
buffer.WriteString("]")
So the buffer array has the key-record pairs and I want to filter it.
If you can unmarshal the data into a struct, you can use the following code:
package main
import (
"encoding/json"
"fmt"
)
type DataStructure struct {
Key string `json:"Key"`
Record struct {
Name string `json:"name"`
Type string `json:"type"`
Validity bool `json:"validity"`
} `json:"Record"`
}
var data string = `[{"Key":"area1", "Record": {"name":"belfast","type":"surburban","validity":true}},{"Key":"area1", "Record": {"name":"belfast","type":"surburban","validity":false}}]`
func main() {
var datastruct []DataStructure
var result []DataStructure
if err := json.Unmarshal([]byte(data), &datastruct); err != nil {
panic(err)
}
for _, item := range datastruct {
if item.Record.Validity {
result = append(result, item)
}
}
fmt.Println(result)
}
Input:
[{"Key":"area1", "Record": {"name":"belfast","type":"surburban","validity":true}},{"Key":"area1", "Record": {"name":"belfast","type":"surburban","validity":false}}]
Expected result:
[{Key:area1 Record:{Name:belfast Type:surburban Validity:true}}]

Range over elements of a protobuf array in Go

I have a Protobuf structure defined as so in my .proto file:
message Msg{
message SubMsg {
string SubVariable1 = 1;
int32 SubVariable2 = 2;
...
}
string Variable1 = 1;
repeated SubMsg Variable2 = 2;
...
}
I pull data into this structure using the https://godoc.org/google.golang.org/protobuf/encoding/protojson package when consuming data from a JSON API, as so:
Response, err := Client.Do(Request)
if err != nil {
log.Error(err)
}
DataByte, err := ioutil.ReadAll(Response.Body)
if err != nil {
log.Error(err)
}
DataProto := Msg{}
err = protojson.Unmarshal(DataByte, &DataProto)
if err != nil {
log.Error(err)
}
What I want to be able to do is to range over the elements of Variable2 to be able to access the SubVariables using the protoreflect API, for which I have tried both:
Array := DataProto.GetVariable2()
for i := range Array {
Element := Array[i]
}
and also:
DataProto.GetVariable2().ProtoReflect().Range(func(fd protoreflect.FieldDescriptor, v protoreflect.Value) {
…
return true})
The first of which fails with error message:
cannot range over DataProto.GetVariable2() (type *SubMsg)
despite the fact DataProto.GetVariable2() returns a variable of type []*Msg_SubMsg.
The second of which fails with:
DataProto.GetVariable2.ProtoReflect undefined (type []*SubMsg has no field or method ProtoReflect)
which suggests that DataProto.GetVariable2() does indeed return an array unlike what is suggested in the error returned in my first approach. This makes sense to me as the protoreflect API only allows this method to be called on a defined message, not an array of those messages. There therefore must be another way of accessing the elements of these arrays to be able to make use of the protoreflect API (for which I have been unsuccessful in finding and answer to on the web thus far).
Could someone help me make sense of these seemingly conflicting error messages? Has anyone had any success iterating over a Protobuf array themselves?
Thanks in advance.
You'll want to treat your Array variable as a List, which means you can't use Range() as in your second attempt. It's close though. Here is a functional example of iterating through and inspecting nested messages:
import (
"testing"
"google.golang.org/protobuf/reflect/protoreflect"
)
func TestVariable2(t *testing.T) {
pb := &Msg{
Variable2: []*Msg_SubMsg{
{
SubVariable1: "string",
SubVariable2: 1,
},
},
}
pbreflect := pb.ProtoReflect()
fd := pbreflect.Descriptor().Fields().ByJSONName("Variable2")
if !fd.IsList() {
t.Fatal("expected a list")
}
l := pbreflect.Get(fd).List()
for i := 0; i < l.Len(); i++ {
// should test that we are now inspecting a message type
li := l.Get(i).Message()
li.Range(func(lifd protoreflect.FieldDescriptor, liv protoreflect.Value) bool {
t.Logf("%v: %v", lifd.Name(), liv)
return true
})
}
}
Run with go test -v ./... if you want to see output

Golang array in struct issue

I have problems using a array of structs inside another struct. The issue is that when I fill the structure with data from JSON data in the function below, it is filled correctly. When I directly try to access the data outside of the loop within a new loop the data is not there. So I guess that I'm filling the copied data structure instead of the reference to it so it's only valid inside the first loop. Though I have tried to allocate memory for it and still the same issue.
I guess that I failed somewhere and need directions. See some comments below in the code snippet.
type Spaces struct {
Items []*Space `json:"items"`
}
type Space struct {
Id string `json:"id"`
Messages []Message `json:"items"`
}
type Messages struct {
Items []Message `json:"items"`
}
// spaces are marshalled first so that there is a array of spaces
// with Id set. Then the function below is called.
func FillSpaces(space_id string) {
for _,s := range spaces.Items {
if s.Id == space_id {
// I tried to allocate with: s.Messages = &Messages{} without any change.
json.Unmarshal(f, &s) // f is JSON data
fmt.Printf(" %s := %v\n", s.Id, len(s.Messages))) // SomeId := X messages (everything seems fine!)
break
}
}
// Why is the messages array empty here when it was not empty above?
for _,s := range spaces.Items {
if s.Id == space_id {
fmt.Printf("%v", len(s.Messages))) // Length is 0!?
}
}
}
The application is unmarshaling to the variable s defined in the loop. Unmarshal to the slice element instead:
for i, s := range spaces.Items {
if s.Id == space_id {
err := json.Unmarshal(f, &spaces.Items[i]) // <-- pass pointer to element
if err != nil {
// handle error
}
break
}
}

Serialize a mixed type JSON array in Go

I want to return a structure that looks like this:
{
results: [
["ooid1", 2.0, "Söme text"],
["ooid2", 1.3, "Åther text"],
]
}
That's an array of arrags that is string, floating point number, unicode character.
If it was Python I'd be able to:
import json
json.dumps({'results': [["ooid1", 2.0, u"Söme text"], ...])
But in Go you can't have an array (or slice) of mixed types.
I thought of using a struct like this:
type Row struct {
Ooid string
Score float64
Text rune
}
But I don't want each to become a dictionary, I want it to become an array of 3 elements each.
We can customize how an object is serialized by implementing the json.Marshaler interface. For our particular case, we seem to have a slice of Row elements that we want to encode as an array of heterogenous values. We can do so by defining a MarshalJSON function on our Row type, using an intermediate slice of interface{} to encode the mixed values.
This example demonstrates:
package main
import (
"encoding/json"
"fmt"
)
type Row struct {
Ooid string
Score float64
Text string
}
func (r *Row) MarshalJSON() ([]byte, error) {
arr := []interface{}{r.Ooid, r.Score, r.Text}
return json.Marshal(arr)
}
func main() {
rows := []Row{
{"ooid1", 2.0, "Söme text"},
{"ooid2", 1.3, "Åther text"},
}
marshalled, _ := json.Marshal(rows)
fmt.Println(string(marshalled))
}
Of course, we also might want to go the other way around, from JSON bytes back to structs. So there's a similar json.Unmarshaler interface that we can use.
func (r *Row) UnmarshalJSON(bs []byte) error {
arr := []interface{}{}
json.Unmarshal(bs, &arr)
// TODO: add error handling here.
r.Ooid = arr[0].(string)
r.Score = arr[1].(float64)
r.Text = arr[2].(string)
return nil
}
This uses a similar trick of first using an intermediate slice of interface{}, using the unmarshaler to place values into this generic container, and then plop the values back into our structure.
package main
import (
"encoding/json"
"fmt"
)
type Row struct {
Ooid string
Score float64
Text string
}
func (r *Row) UnmarshalJSON(bs []byte) error {
arr := []interface{}{}
json.Unmarshal(bs, &arr)
// TODO: add error handling here.
r.Ooid = arr[0].(string)
r.Score = arr[1].(float64)
r.Text = arr[2].(string)
return nil
}
func main() {
rows := []Row{}
text := `
[
["ooid4", 3.1415, "pi"],
["ooid5", 2.7182, "euler"]
]
`
json.Unmarshal([]byte(text), &rows)
fmt.Println(rows)
}
You can read a full example here.
Use []interface{}
type Results struct {
Rows []interface{} `json:"results"`
}
You will then have to use type assertion if you want to access the values stored in []interface{}
for _, row := range results.Rows {
switch r := row.(type) {
case string:
fmt.Println("string", r)
case float64:
fmt.Println("float64", r)
case int64:
fmt.Println("int64", r)
default:
fmt.Println("not found")
}
}
Some clumsy, but you can
type result [][]interface{}
type results struct {
Results result
}
Working example https://play.golang.org/p/IXAzZZ3Dg7

How to set new value to struct member of explicit type from interface{} value (reflection)? Golang

I want to understand some subtle moments of using reflect package. Please, see example below, it describes better what I want to know:
type Robot struct {
id int
model string
}
func change(i interface{}, fields ...string) {
v := reflect.ValueOf(i).Elem()
// here I emulate function by slice that could return any value,
// so here I need to check if I can store incoming values to existing struct
returns := []interface{}{100, "Something"}
for i, name := range fields {
x := reflect.ValueOf(&returns[i]).Elem()
//check if value representing x is the same of struct member
v.FieldByName(name).Set(x)
// ^ here I want to store 100 to Robot.id when i = 0,
// and "Something" to Robot.model when i = 1
}
}
func main() {
robot := &Robot{id: 1, model: "T310"}
change(robot, "model", "id")
// now robot become as follows: &Robot{100, "Something"}
}
Why does it need for?
// It is need for retrieving values from sql DB into struct members
// (only for training purposes :))
// Example:
f := func(q string, structs interface{}, fields ...string) {
rows, _ := db.Query(q)
for i := 0; rows.Next(); i++ {
rows.Scan(&structs[i])
// very dirty here! it's hard to understand how to implement it
}
}
var robots = []*Robot
f("select id, model from robots", robots, "id", "model")
// now each member of robots var should contain values from DB
I tried to be descriptive and explain as short as possible. I hope you understand me..
You can only set exported fields via reflection, so capitalize those first. Otherwise, if you're counting on positional values, make sure they are properly aligned.
Something like this for example: http://play.golang.org/p/ItnjwwJnxe
type Robot struct {
ID int
Model string
}
func change(i interface{}, fields ...string) {
returns := []interface{}{100, "Something"}
v := reflect.ValueOf(i).Elem()
for i, name := range fields {
val := reflect.ValueOf(returns[i])
v.FieldByName(name).Set(val)
}
}
func main() {
robot := &Robot{ID: 1, Model: "T310"}
fmt.Println(robot)
change(robot, "ID", "Model")
fmt.Println(robot)
}

Resources