strategy for REST API in go - database

Question

In my database, each row corresponds to a struct
type datum struct{
Id *string `json:"task_id"`
Status *string `json:"status"`
AccountId *string `json:"account_id"`
.... // many more fields, all of pointer types
}
On the webpage, the user can query on several fields of datum (say account_id and status). The server will return all data that satisfy the query with a projection of the fields (say Id, account_id and status).
Right now, I wrote a HTTP handler to
Extract the query as a datum object from the request:
body, err := ioutil.ReadAll(r.Body)
condition := datum{}
err = json.Unmarshal(body, &condition)
Use the partially filled datum object to query the database, only the non-nil fields translate to SELECT ... WHERE ..=... The query result is saved in query_result []datum
Write the query_result into json object for reply:
reply := map[string]interface{}{
"reply": query_result,
}
data, err := json.Marshal(reply)
The problem is that in the reply many of the fields are nil, but I still send them, which is wasteful. On the other hand, I don't want to change the datum struct to include omitempty tag because in the database a value entry has all fields non-nil.
In this case, shall I define a new struct just for the reply? Is there a way to define this new struct using datum struct, instead of hard code one?
Is there a better design for this query feature?

Answer 1

You have several options, with choice depending what is more wasteful/expensive in your particular case:
Just use pointers+omitempty in the original struct.
Prepare a custom response object. But you'll need to copy/convert the values from the original struct into its export version.
Write a custom marshaller, that will be exploring your struct and creating an export-ready variant, this way being more dynamic/automatic that #1.
While #1 needs no comments, and #2 to some extend covered by Gepser above, here's how you can address this with a custom marshaller (the idea is to re-assemble your output skipping nil fields):
package main
import (
"fmt"
"encoding/json"
"reflect"
)
type datum struct {
Id *string `json:"task_id"`
Status *string `json:"status"`
AccountId *string `json:"account_id"`
}
type Response struct {
Reply []datum `json:"reply"`
}
func main() {
var query_result []datum
// mocking a query result with records with nil fields
val_id_a := "id-a"
val_status := "status-b"
d1 := datum{
Id: &val_id_a,
Status: &val_status,
}
query_result = append(query_result, d1)
val_id_b := "id-b"
val_account_id := "account-id-b"
d2 := datum{
Id: &val_id_b,
AccountId: &val_account_id,
}
query_result = append(query_result, d2)
reply := &Response{
Reply: query_result,
}
data, err := json.Marshal(reply)
if err != nil {
panic(err)
}
fmt.Printf("%+v\n", string(data))
}
// MarshalJSON is a custom JSON marshaller implementation for Response object.
func (r *Response) MarshalJSON() ([]byte, error) {
a := struct {
Reply []map[string]interface{} `json:"reply"`
}{}
for _, v := range r.Reply {
a.Reply = append(a.Reply, converter(v))
}
return json.Marshal(a)
}
// converter converts a struct into a map, skipping fields with nil values.
func converter(in interface{}) map[string]interface{} {
out := make(map[string]interface{})
v := reflect.ValueOf(in)
for i := 0; i < v.NumField(); i++ {
f := v.Type().Field(i)
tag := f.Tag.Get("json")
if tag != "" && !v.Field(i).IsNil() {
out[tag] = v.Field(i).Interface()
}
}
return out
}

Answer 2

The approach I suggest (is the one I use) is the new struct with omitempty tag, for example:
type datumResponse struct{
Id *string `json:"task_id,omitempty"`
Status *string `json:"status,omitempty"`
AccountId *string `json:"account_id,omitempty"`
.... // many more fields
}
and there is no option to write your new struct using the fields of the old one if there is not substructs or you don't write an array of structs.