I'm working on a hashing function for a map[string]interface{}
Most of the hashing libs required []byte as input to compute the hash.
I tried to Marshal using the json.Marshal for simple maps it works correct but when i add some complexity and shuffled the items then json.Marshal fails to give me a consistent byte array output
package main
import (
"encoding/json"
"fmt"
)
func main() {
data := map[string]interface{}{
"id": "124",
"name": "name",
"count": 123456,
"sites": []map[string]interface{}{
{
"name": "123445",
"count": 234324,
"id": "wersfs",
},
{
"id": "sadcacasca",
"name": "sdvcscds",
"count": 22,
},
},
"list": []int{5, 324, 123, 123, 123, 14, 34, 52, 3},
}
data1 := map[string]interface{}{
"name": "name",
"id": "124",
"sites": []map[string]interface{}{
{
"id": "sadcacasca",
"count": 22,
"name": "sdvcscds",
},
{
"count": 234324,
"name": "123445",
"id": "wersfs",
},
},
"count": 123456,
"list": []int{123, 14, 34, 52, 3, 5, 324, 123, 123},
}
jsonStr, _ := json.Marshal(data)
jsonStr1, _ := json.Marshal(data1)
fmt.Println(jsonStr)
fmt.Println(jsonStr1)
for i := 0; i < len(jsonStr); i++ {
if jsonStr[i] != jsonStr1[i] {
fmt.Println("Byte arrays not equal")
}
}
}
This is what I have tried and it fails to give me a consistent output.
Moreover i was thinking to write a function which will do the sorting of the map and values as well, but then got stuck on how do I sort the
"sites": []map[string]interface{}
I tried json.Marshal and also sorting the map but got stuck
Your data sructures are not equivalent. According to JSON rules arrays are ordered, therefore [123, 14, 34, 52, 3, 5, 324, 123, 123] is not the same as [5, 324, 123, 123, 123, 14, 34, 52, 3]. No wonders the hashes are different. If you need different arrays with the same elements to produce the same hash, you need to canonicalize the arrays before hashing. E.g. sort them.
Here is how it could be done: https://go.dev/play/p/OHq7jsX_cNw
Before serilizing it recursively gos down the maps and arrays and prepares all arrays:
// Prepares data by sorting arrays in place
func prepare(data map[string]any) map[string]any {
for _, value := range data {
switch v := value.(type) {
case []int:
prepareIntArray(v)
case []string:
prepareStringArray(v)
case []map[string]any:
prepareMapArrayById(v)
for _, obj := range v {
prepare(obj)
}
case map[string]any:
prepare(v)
}
}
return data
}
// Sorts int array in place
func prepareIntArray(a []int) {
sort.Ints(a)
}
// Sorts string array in place
func prepareStringArray(a []string) {
sort.Strings(a)
}
// Sorts an array of objects by "id" fields
func prepareMapArrayById(mapSlice []map[string]any) {
sort.Slice(mapSlice, func(i, j int) bool {
return getId(mapSlice[i]) < getId(mapSlice[j])
})
}
// Extracts "id" field from JSON object. Returns empty string if there is no "id" or it is not a string.
func getId(v map[string]any) string {
idAny, ok := v["id"]
if !ok {
return ""
}
idStr, ok := idAny.(string)
if ok {
return idStr
} else {
return ""
}
}
As both the marshaled outputs are basically string representations of the same map in different sequences, if you sort their characters, they become equal.
following this logic, if you sort both jsonStr and jsonStr1, the sorted []byte(s) will be exactly equal. which then you can use to formulate your hash value.
check my solution here
Related
I have two lists of maps. We'll call them A, which exists within a database, and B, which is live results from a sensor.
A shares Key/Values From B
Example looks like:
A = [
{
"created_at": "2020-09-19T17:25:29.547354",
"id": 1,
"ip_address": "192.168.1.1",
"mac_address": "xx:xx:xx:xx:xx:xx",
},
{
"created_at": "2020-09-19T17:25:29.564472",
"id": 2,
"ip_address": "192.168.1.2",
"mac_address": "xx:xx:xx:xx:xx:xx",
},
{
"created_at": "2020-09-19T17:25:29.564472",
"id": 3,
"ip_address": "192.168.1.3",
"mac_address": "xx:xx:xx:xx:xx:xx",
}
]
B = [
{
'mac_address': 'xx:xx:xx:xx:xx:xx',
'ip_address': '192.168.1.1',
'status': True
},
{
'mac_address': 'xx:xx:xx:xx:xx:xx',
'ip_address': '192.168.1.2',
'status': True
}
]
What's the best way way to find out any missing maps from B compared to A by the Value ip_address.
For example, we can tell by looking at the above that the map which contains the ip_address "192.168.1.3" doesn't exist within B. The aim is to try and find a list of values which don't exist between the two, if any.
The expected output is a list like: ["192.168.1.3"]
I come up with semi efficient solution:
package main
import (
"fmt"
)
func main() {
a := []map[string]interface{}{
{
"created_at": "2020-09-19T17:25:29.547354",
"id": 1,
"ip_address": "192.168.1.1",
"mac_address": "xx:xx:xx:xx:xx:xx",
},
{
"created_at": "2020-09-19T17:25:29.564472",
"id": 2,
"ip_address": "192.168.1.2",
"mac_address": "xx:xx:xx:xx:xx:xx",
},
{
"created_at": "2020-09-19T17:25:29.564472",
"id": 3,
"ip_address": "192.168.1.3",
"mac_address": "xx:xx:xx:xx:xx:xx",
},
}
b := []map[string]interface{}{
{
"mac_address": "xx:xx:xx:xx:xx:xx",
"ip_address": "192.168.1.1",
"status": true,
},
{
"mac_address": "xx:xx:xx:xx:xx:xx",
"ip_address": "192.168.1.2",
"status": true,
},
}
c, d := collectIpAddresses(a), collectIpAddresses(b)
var missing []string
for k := range c {
if !d[k] {
missing = append(missing, k)
}
}
for k := range d {
if !c[k] {
missing = append(missing, k)
}
}
fmt.Println(missing)
}
// stores data in more efficient data-structure for searching
func collectIpAddresses(a []map[string]interface{}) map[string]bool {
b := make(map[string]bool, len(a))
for _, v := range a {
b[v["ip_address"].(string)] = true
}
return b
}
This is a good solution because it provides O(m+n) complexity (where m is len(a) and n is len(b)).
In contrary, that solution which uses a loop within a loop will have a complexity of O(m*n). That complexity will dramatically reduce the performance of the algorithm on larger datasets.
Although, because allocating is extremely slow, on a dataset as provided by OP, the last solution will provide better results. This might be a catch depending the size of the dataset to iterate.
there
I have a JSON response as below:
"log": [{
"a": 0.40,
"b": "ED",
}, {
"c": 82,
"d": "ABC",
}, {
"e": 36,
"f": 23,
}, {
"g": 12,
"h": 40,
}
]
I need to count a number of lists in a 'log' object to be able to address the last one and find a specific element in it. The response is dynamic and has different amount of lists in it (in this case 4)
I tried log[-1], and examples form js-arrays.feature as in the link below:
https://github.com/intuit/karate/blob/master/karate-junit4/src/test/java/com/intuit/karate/junit4/demos/js-arrays.feature#L83
It is easy to find a number of elements in a list, but I frequently have variable amount of lists and I cant make it work
Many Thanks,
Take into account that log[-1] would return you undefined due to there is no index -1 in the array. For get the number of elements in your array you must do
log.length
Also about the link you've posted
log[log.length-1]; //This will return the last element of the array in this case { "g": 12, "h": 40, }
I have array data like this:
work_list = {
"data": {
"1": {
"team": "Design",
"members": nil,
"workload": {
"process": {
"total": 50,
"finish": 36,
"un_finish": 14,
}
}
},
"2": {
"team": "Account",
"members": 15,
"workload": {
"process": {
"total": 30,
"finish": 20,
"un_finish": 10,
}
}
}
}
}
I want to calculte total amount of work both team by:
list = work_list["data"]
count = 0
list.each do |num|
num.each do |details|
work = num["workload"]["process"]["total"]
count += work
end
end
puts "The total amount of works: #{count}"
The error here is: "undefined method `each' for nil:NilClass (NoMethodError)"
The problem with your code is that you're using : to build a hash:
"workload": {
"process": {
"total": 50,
"finish": 36,
"un_finish": 14,
}
}
Which is valid syntax in Ruby, but the keys of this hash are going to be symbols, not strings as expected, so you need to access the data of this hash by symbols:
data_hash = work_list[:"data"]
count = 0
data_hash.each do |index, data|
work = data[:"workload"][:"process"][:"total"]
count += work
end
puts "The total amount of works: #{count}"
If you want to use strings, you need to construct hash as following:
"workload" => {
"process" => {
"total" => 50,
"finish" => 36,
"un_finish" => 14,
}
}
You are iteration over a hash not array
list.each do |num, info|
work = info["workload"]["process"]["total"]
count += work
end
I am trying to create an array from already created array. Array that I have is
{
"id": 1,
"category": "fruits",
"name": "Apple",
"description": "Apple is my favorite fruit."
}
{
"id": 2,
"category": "colors",
"name": "Red",
"description": "Red color is always charming."
}
{
"id": 3,
"category": "flowers",
"name": "Lotus",
"description": "It is one of the most beautiful flowers in this world."
}
{
"id": 4,
"category": "colors",
"name": "Pink",
"description": "A romantic color, mostly liked by women."
}
{
"id": 5,
"category": "flowers",
"name": "Rose",
"description": "I love roses."
}
{
"id": 6,
"category": "fruits",
"name": "Mango",
"description": "Mango is one of my favorite fruits."
}
Now I need to create an array and populate data like:
"elements":{
"fruits":{
0:{
"id": 1,
"category": "fruits",
"name": "Apple",
"description": "Apple is my favorite fruit."
}
1:{
"id": 6,
"category": "fruits",
"name": "Mango",
"description": "Mango is one of my favorite fruits."
}
}
"flowers":{
0:{
"id": 3,
"category": "flowers",
"name": "Lotus",
"description": "It is one of the most beautiful flowers in this world."
}
1:{
"id": 5,
"category": "flowers",
"name": "Rose",
"description": "I love roses."
}
}
"colors":{
0:{
"id": 2,
"category": "colors",
"name": "Red",
"description": "Red color is always charming."
}
1:{
"id": 4,
"category": "colors",
"name": "Pink",
"description": "A romantic color, mostly liked by women."
}
}
}
What I have tried is:
arr := make(map[string]interface{})
arrCate := make(map[string]interface{})
arrCateFlower := make(map[int]interface{})
arrCateColor := make(map[int]interface{})
arrCateFruit := make(map[int]interface{})
for index, data := range dataVals{
if(data.Category == "flower"){
arrCateFlower[index] = data
}
if(data.Category == "colors"){
arrCateColor[index] = data
}
if(data.Category == "fruits"){
arrCateFruit[index] = data
}
}
arrCate["flowers"] = arrCateFlower
arrCate["colors"] = arrCateColor
arrCate["fruits"] = arrCateFruit
arr["elements"] = arrCate
Where dataVals contain the unformatted data given at the top. By applying the above code I am able to get the proper output. But I don't think it is efficient way. If I try something like
arr := make(map[string]interface{})
arrCate := make(map[string]interface{})
for _, data := range dataVals{
arrCate[data.Category] = data
}
arr["elements"] = arrCate
Then I get something like:
"elements":{
"fruits":{
"id": 6,
"category": "fruits",
"name": "Mango",
"description": "Mango is one of my favorite fruits."
}
"flowers":{
"id": 5,
"category": "flowers",
"name": "Rose",
"description": "I love roses."
}
"colors":{
"id": 4,
"category": "colors",
"name": "Pink",
"description": "A romantic color, mostly liked by women."
}
}
the last elements of that particular category in the loop. I don't understand how can I get all the elements in the array without using any static values in code.
I have already spent hours in this. Can anyone please tell what am i missing in it?
https://play.golang.org/p/y-I6Fb_61R
I hope you can live with the additional outer {} pair.
And without the outer {} pair: https://play.golang.org/p/SSTgln0qJc
To not just have a bunch of links and to enable easy criticism of my solution by others, I include the code here, slightly redacted:
package main
import (
"fmt"
"encoding/json"
"log"
"strings"
)
var dataAsString = `` //put data between the ``
type Item struct {
Id int `json:"id"`
Category string `json:"category"`
Name string `json:"name"`
Description string `json:"description"`
}
type CategoryToItemSliceMap map[string][]Item
type CategoryToIndexItemMap map[string]map[int]Item
func main() {
// first read the data, we use a decoder as the input was given
// as a stream of seperate json objects and not a big single one.
decoder := json.NewDecoder(strings.NewReader(dataAsString))
var ourData []Item
for decoder.More() {
var it Item
err := decoder.Decode(&it)
if err != nil {
log.Fatalln(err)
}
ourData = append(ourData, it)
}
// collect items according to categories
catToItemSlice := CategoryToItemSliceMap{}
for _,v := range ourData {
catToItemSlice[v.Category] = append(catToItemSlice[v.Category],v)
}
// turn those slices into int -> Item maps so we get the index numbers
// in the encoded json
catToIndexItemMap := CategoryToIndexItemMap{}
for k,v := range catToItemSlice {
if catToIndexItemMap[k] == nil {
catToIndexItemMap[k] = map[int]Item{}
}
for index, item := range v {
catToIndexItemMap[k][index] = item
}
}
// easiest way to get the "elements: " without an additional outer {}
// brace pair
fmt.Printf("elements: ")
// We only have one json object in the output and that is a map, so we
// can use Unmarshal and don't need a streaming encoder. And get nice
// indentation with MarshalIndent.
out, err := json.MarshalIndent(catToIndexItemMap, "", " ")
if err != nil {
log.Fatalln(err)
}
fmt.Println(string(out))
}
// If atributes names equals , set value to fields
// target receive values of object
func Assign(target interface{}, object interface{}) {
t := reflect.ValueOf(target).Elem()
o := reflect.ValueOf(object).Elem()
for i := 0; i < o.NumField(); i++ {
for j := 0; j < t.NumField(); j++ {
if o.Type().Field(i).Name == t.Type().Field(j).Name {
t.Field(j).Set(o.Field(i))
}
}
}
}
// Use this exemple objects interfaces diffrents but fields are equals
// Assign(&target, &object)
I wanto print a complicate dataset like bellowing
package main
import (
"fmt"
// "reflect"
)
func main() {
var initData []interface{}
initData[0] = map[string]interface{}{
"name": "k1",
"type": "line",
"data": []int{1, 2, 3, 4},
}
initData[1] = map[string]interface{}{
"name": "k2",
"type": "circle",
"data": []int{11, 12, 13, 14},
}
for _, data := range initData {
for k, v := range data {
fmt.Println(k)
for _, sv := range v {
fmt.Println(" " + sv)
}
}
}
}
but get an error
./maps-in-maps.go:56: cannot range over data (type interface {})
any help? to try to concert the interface typed dataset to interable like arry or slice
You must type assert each interface{} level first, in this case, use a type assertion at map[string]interface{} level since we know a definite type, and a type switch to check anticipated types for each nested value in the next level.
initData := []interface{}{
map[string]interface{}{
"name": "k1",
"type": "line",
"data": []int{1, 2, 3, 4},
},
map[string]interface{}{
"name": "k2",
"type": "circle",
"data": []int{11, 12, 13, 14},
},
}
for _, data := range initData {
for _, v := range data.(map[string]interface{}) {
switch t := v.(type) {
case string, []int:
fmt.Println(t)
default:
fmt.Println("wrong type")
}
}
}
}
Run playground here.