I have a variable buffer that stores a set of key-value pairs in the array of the form:
[{"Key":"area1", "Record": {"name":"belfast","type":"surburban","validity":"true"}},{Key,Record},{Key,Record}....] i.e a set of Key-Record pairs in a buffer array.
Now I want to retrieve only the key-record pairs that have a specific record entry, for example i want only records that have the value "true" in validity, I want to return all key-record pairs that have the validity field value as true. Any suggestions ? Thanks
Here is a code segment of how the key-record pairs are created after which I want to filter key-records that have validity as true
var buffer bytes.Buffer
buffer.WriteString("[")
bArrayMemberAlreadyWritten := false
for resultsIterator.HasNext() {
queryResponse, err := resultsIterator.Next()
if err != nil {
return shim.Error(err.Error())
}
// Add a comma before array members, suppress it for the first array member
if bArrayMemberAlreadyWritten == true {
buffer.WriteString(",")
}
buffer.WriteString("{\"Key\":")
buffer.WriteString("\"")
buffer.WriteString(queryResponse.Key)
buffer.WriteString("\"")
buffer.WriteString(", \"Record\":")
// Record is a JSON object, so we write as-is
buffer.WriteString(string(queryResponse.Value))
buffer.WriteString("}")
bArrayMemberAlreadyWritten = true
}
buffer.WriteString("]")
So the buffer array has the key-record pairs and I want to filter it.
If you can unmarshal the data into a struct, you can use the following code:
package main
import (
"encoding/json"
"fmt"
)
type DataStructure struct {
Key string `json:"Key"`
Record struct {
Name string `json:"name"`
Type string `json:"type"`
Validity bool `json:"validity"`
} `json:"Record"`
}
var data string = `[{"Key":"area1", "Record": {"name":"belfast","type":"surburban","validity":true}},{"Key":"area1", "Record": {"name":"belfast","type":"surburban","validity":false}}]`
func main() {
var datastruct []DataStructure
var result []DataStructure
if err := json.Unmarshal([]byte(data), &datastruct); err != nil {
panic(err)
}
for _, item := range datastruct {
if item.Record.Validity {
result = append(result, item)
}
}
fmt.Println(result)
}
Input:
[{"Key":"area1", "Record": {"name":"belfast","type":"surburban","validity":true}},{"Key":"area1", "Record": {"name":"belfast","type":"surburban","validity":false}}]
Expected result:
[{Key:area1 Record:{Name:belfast Type:surburban Validity:true}}]
Related
we have struct and getting null after append struct in golang.
Find below struct with my some part of code in golang.
type XmloutRoomRate struct {
CancellationPolicy Policies `bson:"cancellationPolicy" json:"cancellationPolicy"`
}
type Policies struct {
Policies []RoomPolicies `bson:"policies" json:"policies"`
}
type RoomPolicies struct {
Amount float64 `bson:"amount" json:"amount"`
DaysBeforeCheckIn int `bson:"daysBeforeCheckIn" json:"daysBeforeCheckIn"`
}
cancelPolicyMain := Policies{}
cancelPolicy := RoomPolicies{}
if cancelAmount < 0 {
cancelPolicy.Amount = cancelAmount
cancelPolicy.DaysBeforeCheckIn = cancelDay
cancelPolicyMain.Policies = append(cancelPolicyMain.Policies, cancelPolicy)
}else{
cancelPolicyMain = agodaPolicies{}
cancelPolicyMain.Policies = append(cancelPolicyMain.Policies)
}
when data present getting proper data structure.
"cancellationPolicy": {
"policies": [
{
"amount": 5141.58,
"daysBeforeCheckIn": 5
}
]
}
But when data not available getting struct with null value.
"cancellationPolicy": {
"policies": null
}
We need my actual output with blank array [].
"cancellationPolicy": {
"policies": []
}
nil slice values are marshaled into JSON null values. This is documented at json.Marshal():
Array and slice values encode as JSON arrays, except that []byte encodes as a base64-encoded string, and a nil slice encodes as the null JSON value.
Non-nil empty slices are marshaled into empty JSON arrays. So simply initialize Policies.Policies to a non-nil empty slice, and it will be [] in the output:
cancelPolicyMain = Policies{Policies: []RoomPolicies{}}
Test code:
const cancelDay = 1
for cancelAmount := -500.0; cancelAmount <= 501; cancelAmount += 1000 {
cancelPolicyMain := Policies{}
cancelPolicy := RoomPolicies{}
if cancelAmount < 0 {
cancelPolicy.Amount = cancelAmount
cancelPolicy.DaysBeforeCheckIn = cancelDay
cancelPolicyMain.Policies = append(cancelPolicyMain.Policies, cancelPolicy)
} else {
cancelPolicyMain = Policies{Policies: []RoomPolicies{}}
cancelPolicyMain.Policies = append(cancelPolicyMain.Policies)
}
x := XmloutRoomRate{cancelPolicyMain}
if err := json.NewEncoder(os.Stdout).Encode(x); err != nil {
panic(err)
}
}
Output (try it on the Go Playground):
{"cancellationPolicy":{"policies":[{"amount":-500,"daysBeforeCheckIn":1}]}}
{"cancellationPolicy":{"policies":[]}}
In an array, the meaning of a "null" entry is clear: It means this array entry is missing.
In an object aka dictionary, there are different ways to indicate "no entry": The key might not be there. Or the key might not be there, but with an empty array as value. Or the key might be there, but with a "null" value.
You really need agreement between the provider of the data and the client processing it, what each of these mean. And since it's often hard to change what the data provider does, translate what you get into what you need.
So you have to decide what it means if "policies" does not exist as a key, or if it exists as a null value. I've seen software that wouldn't produce arrays with one element, but would provide the single element instead. So "policies": { "amount": ..., "daysBeforeCheckin": ... } would also be possible. You decide what you accept, what you treat as an array, and how you change from the form you got to the form you want.
Recently I was trying to an implementation of a Mutation Request using GoLang as a Graphql Server, Basically this is the query that i send: As you can see its an array of object that contains name and an array of strings
mutation{
CellTest(cells:[{name:"lero",child:["1","2"]},{name:"lero2",child:["12","22"]}]){
querybody
}
}
In my Go code I have a type object that is gonna set the values sent
type Cell struct {
name string `json:"name"`
child []string `json:"child"`
}
and a custom array that is gonna be []Cell
type Cells []*Cell
However when the request is received by GO I get this:
Note that this is the print of cellsInterface
[map[child:[1 2] name:lero] map[child:[12 22] name:lero2]]
How can i get each value and assign those in my Array Cells
something like this:
Cells[0] = {name="first",child={"1","2"}}
Cells[1] = {name="second",child={"hello","good"}}
this is my current attempt:
var resolvedCells Cells
cellsInterface := params.Args["cells"].([]interface{})
cellsByte, err := json.Marshal(cellsInterface)
if err != nil {
fmt.Println("marshal the input json", err)
return resolvedCells, err
}
if err := json.Unmarshal(cellsByte, &resolvedCells); err != nil {
fmt.Println("unmarshal the input json to Data.Cells", err)
return resolvedCells, err
}
for cell := range resolvedCells {
fmt.Println(cellsInterface[cell].([]interface{}))
}
However this only split the cells array into 0 and 1.
Range through the map values in the result and append those values to Cell slice. If you are getting an object from json. Then you can unmarshall the bytes into Cell.
The result when unmarshalling should be a slice of Cell struct as
var resolvedCells []Cell
if err := json.Unmarshal(cellsByte, &resolvedCells); err != nil {
fmt.Println("unmarshal the input json to Data.Cells", err)
}
fmt.Println(resolvedCells)
Working Code on Go playground
Or if you want to use pointers loop over the resolvedCell as
type Cells []*Cell
func main() {
var resolvedCells Cells
if err := json.Unmarshal(cellsByte, &resolvedCells); err != nil {
fmt.Println("unmarshal the input json to Data.Cells", err)
}
fmt.Println(*resolvedCells[1])
for _, value := range resolvedCells{
fmt.Println(value)
fmt.Printf("%+v",value.Child) // access child struct value of array
}
}
Playground example
In my database, each row corresponds to a struct
type datum struct{
Id *string `json:"task_id"`
Status *string `json:"status"`
AccountId *string `json:"account_id"`
.... // many more fields, all of pointer types
}
On the webpage, the user can query on several fields of datum (say account_id and status). The server will return all data that satisfy the query with a projection of the fields (say Id, account_id and status).
Right now, I wrote a HTTP handler to
Extract the query as a datum object from the request:
body, err := ioutil.ReadAll(r.Body)
condition := datum{}
err = json.Unmarshal(body, &condition)
Use the partially filled datum object to query the database, only the non-nil fields translate to SELECT ... WHERE ..=... The query result is saved in query_result []datum
Write the query_result into json object for reply:
reply := map[string]interface{}{
"reply": query_result,
}
data, err := json.Marshal(reply)
The problem is that in the reply many of the fields are nil, but I still send them, which is wasteful. On the other hand, I don't want to change the datum struct to include omitempty tag because in the database a value entry has all fields non-nil.
In this case, shall I define a new struct just for the reply? Is there a way to define this new struct using datum struct, instead of hard code one?
Is there a better design for this query feature?
You have several options, with choice depending what is more wasteful/expensive in your particular case:
Just use pointers+omitempty in the original struct.
Prepare a custom response object. But you'll need to copy/convert the values from the original struct into its export version.
Write a custom marshaller, that will be exploring your struct and creating an export-ready variant, this way being more dynamic/automatic that #1.
While #1 needs no comments, and #2 to some extend covered by Gepser above, here's how you can address this with a custom marshaller (the idea is to re-assemble your output skipping nil fields):
package main
import (
"fmt"
"encoding/json"
"reflect"
)
type datum struct {
Id *string `json:"task_id"`
Status *string `json:"status"`
AccountId *string `json:"account_id"`
}
type Response struct {
Reply []datum `json:"reply"`
}
func main() {
var query_result []datum
// mocking a query result with records with nil fields
val_id_a := "id-a"
val_status := "status-b"
d1 := datum{
Id: &val_id_a,
Status: &val_status,
}
query_result = append(query_result, d1)
val_id_b := "id-b"
val_account_id := "account-id-b"
d2 := datum{
Id: &val_id_b,
AccountId: &val_account_id,
}
query_result = append(query_result, d2)
reply := &Response{
Reply: query_result,
}
data, err := json.Marshal(reply)
if err != nil {
panic(err)
}
fmt.Printf("%+v\n", string(data))
}
// MarshalJSON is a custom JSON marshaller implementation for Response object.
func (r *Response) MarshalJSON() ([]byte, error) {
a := struct {
Reply []map[string]interface{} `json:"reply"`
}{}
for _, v := range r.Reply {
a.Reply = append(a.Reply, converter(v))
}
return json.Marshal(a)
}
// converter converts a struct into a map, skipping fields with nil values.
func converter(in interface{}) map[string]interface{} {
out := make(map[string]interface{})
v := reflect.ValueOf(in)
for i := 0; i < v.NumField(); i++ {
f := v.Type().Field(i)
tag := f.Tag.Get("json")
if tag != "" && !v.Field(i).IsNil() {
out[tag] = v.Field(i).Interface()
}
}
return out
}
The approach I suggest (is the one I use) is the new struct with omitempty tag, for example:
type datumResponse struct{
Id *string `json:"task_id,omitempty"`
Status *string `json:"status,omitempty"`
AccountId *string `json:"account_id,omitempty"`
.... // many more fields
}
and there is no option to write your new struct using the fields of the old one if there is not substructs or you don't write an array of structs.
I'm trying to learn Go (or Golang) and can't seem to get it right. I have 2 texts files, each containing a list of words. I'm trying to count the amount of words that are present in both files.
Here is my code so far :
package main
import (
"fmt"
"log"
"net/http"
"bufio"
)
func stringInSlice(str string, list []string) bool {
for _, v := range list {
if v == str {
return true
}
}
return false
}
func main() {
// Texts URL
var list = "https://gist.githubusercontent.com/alexcesaro/c9c47c638252e21bd82c/raw/bd031237a56ae6691145b4df5617c385dffe930d/list.txt"
var url1 = "https://gist.githubusercontent.com/alexcesaro/4ebfa5a9548d053dddb2/raw/abb8525774b63f342e5173d1af89e47a7a39cd2d/file1.txt"
//Create storing arrays
var buffer [2000]string
var bufferUrl1 [40000]string
// Set a sibling counter
var sibling = 0
// Read and store text files
wordList, err := http.Get(list)
if err != nil {
log.Fatalf("Error while getting the url : %v", err)
}
defer wordList.Body.Close()
wordUrl1, err := http.Get(url1)
if err != nil {
log.Fatalf("Error while getting the url : %v", err)
}
defer wordUrl1.Body.Close()
streamList := bufio.NewScanner(wordList.Body)
streamUrl1 := bufio.NewScanner(wordUrl1.Body)
streamList.Split(bufio.ScanLines)
streamUrl1.Split(bufio.ScanLines)
var i = 0;
var j = 0;
//Fill arrays with each lines
for streamList.Scan() {
buffer[i] = streamList.Text()
i++
}
for streamUrl1.Scan() {
bufferUrl1[j] = streamUrl1.Text()
j++
}
//ERROR OCCURRING HERE :
// This code if i'm not wrong is supposed to compare through all the range of bufferUrl1 -> bufferUrl1 values with buffer values, then increment sibling and output FIND
for v := range bufferUrl1{
if stringInSlice(bufferUrl1, buffer) {
sibling++
fmt.Println("FIND")
}
}
// As a testing purpose thoses lines properly paste both array
// fmt.Println(buffer)
// fmt.Println(bufferUrl1)
}
But right now, my build doesn't even succeed. I'm only greeted with this message:
.\hello.go:69: cannot use bufferUrl1 (type [40000]string) as type string in argument to stringInSlice
.\hello.go:69: cannot use buffer (type [2000]string) as type []string in argument to stringInSlice
bufferUrl1 is an array: [4000]string. You meant to use v (each
string in bufferUrl1). But in fact, you meant to use the second
variable—the first variable is the index which is ignored in the code
below using _.
type [2000]string is different from []string. In Go, arrays and slices are not the same. Read Go Slices: usage and internals. I've changed both variable declarations to use slices with the same initial length using make.
These are changes you need to make to compile.
Declarations:
// Create storing slices
buffer := make([]string, 2000)
bufferUrl1 := make([]string, 40000)
and the loop on Line 69:
for _, s := range bufferUrl1 {
if stringInSlice(s, buffer) {
sibling++
fmt.Println("FIND")
}
}
As a side-note, consider using a map instead of a slice for buffer for more efficient lookup instead of looping through the list in stringInSlice.
https://play.golang.org/p/UcaSVwYcIw has the fix for the comments below (you won't be able to make HTTP requests from the Playground).
I want to understand some subtle moments of using reflect package. Please, see example below, it describes better what I want to know:
type Robot struct {
id int
model string
}
func change(i interface{}, fields ...string) {
v := reflect.ValueOf(i).Elem()
// here I emulate function by slice that could return any value,
// so here I need to check if I can store incoming values to existing struct
returns := []interface{}{100, "Something"}
for i, name := range fields {
x := reflect.ValueOf(&returns[i]).Elem()
//check if value representing x is the same of struct member
v.FieldByName(name).Set(x)
// ^ here I want to store 100 to Robot.id when i = 0,
// and "Something" to Robot.model when i = 1
}
}
func main() {
robot := &Robot{id: 1, model: "T310"}
change(robot, "model", "id")
// now robot become as follows: &Robot{100, "Something"}
}
Why does it need for?
// It is need for retrieving values from sql DB into struct members
// (only for training purposes :))
// Example:
f := func(q string, structs interface{}, fields ...string) {
rows, _ := db.Query(q)
for i := 0; rows.Next(); i++ {
rows.Scan(&structs[i])
// very dirty here! it's hard to understand how to implement it
}
}
var robots = []*Robot
f("select id, model from robots", robots, "id", "model")
// now each member of robots var should contain values from DB
I tried to be descriptive and explain as short as possible. I hope you understand me..
You can only set exported fields via reflection, so capitalize those first. Otherwise, if you're counting on positional values, make sure they are properly aligned.
Something like this for example: http://play.golang.org/p/ItnjwwJnxe
type Robot struct {
ID int
Model string
}
func change(i interface{}, fields ...string) {
returns := []interface{}{100, "Something"}
v := reflect.ValueOf(i).Elem()
for i, name := range fields {
val := reflect.ValueOf(returns[i])
v.FieldByName(name).Set(val)
}
}
func main() {
robot := &Robot{ID: 1, Model: "T310"}
fmt.Println(robot)
change(robot, "ID", "Model")
fmt.Println(robot)
}