I have JSON with one object for each site's traffic for 3 dates.
I want to merge these three objects together on "Date". Example below.
I'm using Ruby. What is the easiest way to do this?
Start JSON:
[
{
"Google":[
{
"Date":"2015-01-01",
"Value":100
},
{
"Date":"2015-02-01",
"Value":200
},
{
"Date":"2015-03-01",
"Value":300
}
]
},
{
"Yahoo":[
{
"Date":"2015-01-01",
"Value":1200
},
{
"Date":"2015-02-01",
"Value":1300
},
{
"Date":"2015-03-01",
"Value":1400
}
]
},
{
"Bing":[
{
"Date":"2015-01-01",
"Value":500
},
{
"Date":"2015-02-01",
"Value":600
},
{
"Date":"2015-03-01",
"Value":700
}
]
}
]
End JSON:
[
{
"Date":"2015-01-01",
"Google":100,
"Yahoo":1200,
"Bing":500
},
{
"Date":"2015-01-02",
"Google":200,
"Yahoo":1200,
"Bing":600
},
{
"Date":"2015-01-03",
"Google":300,
"Yahoo":1400,
"Bing":700
}
]
result = array.inject({}) do | a, e |
site, data = e.first
data.each do | x |
a[x[:Date]] ||= {}
a[x[:Date]][site] = x[:Value]
end
a
end
gives you a hash with the dates as keys. This can be transformed to the array by:
result.map { | k, v | v.update(:Date => k) }
Assuming exactly this structure, what might work is
results = []
your_array[0]['Google'].each_with_index do |item, index|
date = item['Date']
provider_values = your_array.inject({}) do |memo, current|
provider = current.keys[0]
value = current[provider][index]['Value']
memo[provider] = value
memo
end
results.push({'Date' => date}.merge(provider_values))
end
I'm currently on Windows so I can't be 100% sure of correctness, however fixing any syntax errors should be easy.
Related
Again I'm stuck with a nested Array of objects. I want to flatten it out, but I do have to keep some nested objects. The Problem I'm running into: How to rename the keys of the nested objects since I have an undefined number of nested objects. There might be 3 of them oder 8. So property1_3 has to be renamed to eg property1_3_1, property1_3_2 - depending on how many objects are in the original json data. And how to aply them to the correct parent object.
The json data I recieve looks like:
data = [{
"property1_1": "value1_1",
"property1_2": "value1_2",
"property1_3": [
[{
"subproperty1_1_1": "subvalue1_1_1",
"subproperty1_1_2": "subvalue1_1_2"
}],
[{
"subproperty1_2_1": "subvalue1_2_1",
"subproperty1_2_2": "subvalue1_2_2"
}]
]
},
{
"property2_1": "value2_1",
"property2_2": "value2_2",
"property2_3": [
[{
"subproperty2_1_1": "subvalue2_2_1",
"subproperty2_1_2": "subvalue2_2_2"
}],
[{
"subproperty2_2_1": "subvalue2_2_1",
"subproperty2_2_2": "subvalue2_2_2"
}],
[{
"subproperty2_3_1": "subvalue2_2_1",
"subproperty2_3_2": "subvalue2_2_2"
}]
]
}
]
What I want to achieve now is:
data = [
{
"property1_1": "value1_1",
"property1_2": "value1_2",
"property1_3_index1": {"subproperty1_1_1":"subvalue1_1_1", "subproperty1_1_2":"subvalue1_1_2"},
"property1_3_index2": {"subproperty1_2_1":"subvalue1_2_1", "subproperty1_2_2":"subvalue1_2_2"}
},
{
"property2_1": "value2_1",
"property2_2": "value2_2",
"property2_3_index1": {"subproperty2_1_1":"subvalue2_2_1", "subproperty2_1_2":"subvalue2_2_2"},
"property2_3_index2": {"subproperty2_2_1":"subvalue2_2_1", "subproperty2_2_2":"subvalue2_2_2"},
"property2_3_index3": {"subproperty2_3_1":"subvalue2_2_1", "subproperty2_3_2":"subvalue2_2_2"}
}
]
My last try was:
transformData(input) {
const testArray = [];
input.map(obj => {
for (const prop in obj) {
if (obj.hasOwnProperty(prop) && Array.isArray(obj[prop])) {
for (const [index, element] of obj[prop].entries()) {
testArray.push(element[0]);
}
}
}
});
}
but this only leeds to an array with all the single subobjects in one array. I'm also not quite sure if it's best trying to convert the original data or to build a new array as I tried before.
I finally found a way to achieve this.
transformData(input) {
return input.map(obj => {
for (const prop in obj) {
if (obj.hasOwnProperty(prop) && Array.isArray(obj[prop])) {
for (let i = 0; i < obj[prop].length; i++) {
const name = prop + (i + 1).toString();
obj[name] = obj[prop].flat(1)[i];
}
delete obj[prop];
}
}
return obj;
});
}
Sorry guys if my way of asking the question in the title is not correct.
I am working on a project on react js and I am getting the data like this
[
{
"count": [
{
"1": 16
},
{
"1": 149
}
],
"day": "2019-08-27"
}
]
now this is my first time I am dealing with this kind of data and I really have no idea how can I show it like this I am really sorry guys I literally can't even show what I have tried because it does not seem relevant
[
{
count: 165
day:"2019-08-27"
}
}
Assuming the data you're getting is under a variable called data you could use reduce:
The below makes the assumption the count is always an array of objects with just 1 key called '1'.
const newData = data.map(datum => {
datum.count = datum.count.reduce((count, item) => {
return count + item['1']
}, 0)
return datum
})
You can try something like this:
let arr = [
// item
{
count: [
{
"1": 16
},
{
"1": 149
}
],
day: "2019-08-27"
}
];
arr.map(item => {
Object.keys(item).map(key => {
console.log(item[key])
// if item[key] is iterable
if(Array.isArray(item[key])) {
item[key].map(val => {
console.log(item)
})
} else {
console.log(item[key])
}
});
});
The concept is that for Objects you do a Object.keys().something and for an array you do a arr.map(item => ...)
I have an object with an array that looks like this:
some_object = {
some_array: [
{ id: "foo0" },
{ id: "foo1" },
{ id: "foo2" },
{ id: "foo3" },
]
}
And I have an input of another array that I want to rearrange the array in
target_order = [
{ id: "foo0", new_position: 3 },
{ id: "foo3", new_position: 0 },
{ id: "foo1", new_position: 2 },
{ id: "foo2", new_position: 1 }
]
How do I go about using the second target_order array to modify the order of the first some_object[:some_array]?
I recommend you use sort_by with a custom block that finds the position of the item in the new array.
new_array = some_object[:some_array].sort_by do |item|
order = target_order.detect { |order| order[:id] == item[:id] }
next unless order
order[:new_position]
end
This returns the following value.
=> [{:id=>"foo2"}, {:id=>"foo1"}, {:id=>"foo0"}, {:id=>"foo3"}]
Further considerations
Perhaps you wanted to give each item a position in a list instead of just sorting them. For instance
target_order = [
{ id: "foo0", new_position: 0 },
{ id: "foo1", new_position: 2 }
]
would give
=> [{ id: "foo0" }, nil, { id: "foo1" }]
To do this, you should use each_with_object instead of sort_by.
new_array = target_order.each_with_object([]) do |order, memo|
item = some_object[:some_array].detect { |item| item[:id] == order[:id] }
next unless item
memo[order[:new_position]] = item
end
Just to be simplistic, this is what I would do ...
temp_arr = []
target_order.each do |o|
x = some_json_object[:some_array].find { |i| o[:id] == i[:id] }
temp_arr[o[:new_position] - 1] = x
end
some_json_object = {
"some_array": temp_arr
}
If there is one to one correspondence between some_array and target_order elements, maybe you can do a direct assignment, something like:
some_object[:some_array] = target_order.sort_by{ |h| h[:new_position] }.map { |h| h.delete_if { |k, _| k == :new_position } }
So, you'll end up with
some_object #=> {:some_array=>[{:id=>"foo3"}, {:id=>"foo2"}, {:id=>"foo1"}, {:id=>"foo0"}]}
There is no need to sort, which has time-complexity of O(n*log(n)). Here is a O(n) solution.
{ some_array: target_order.each_with_object([]) { |h,a|
a[h[:new_position]] = h.slice(:id) } }
#=> {:some_array=>[{:id=>"foo3"}, {:id=>"foo2"}, {:id=>"foo1"}, {:id=>"foo0"}]}
Note that there is no reference to some_object.
If some_object is to be modified in place:
some_object[:some_array] = target_order.each_with_object([]) { |h,a|
a[h[:new_position]] = h.slice(:id) }
some_object
#=> {:some_array=>[{:id=>"foo3"}, {:id=>"foo2"}, {:id=>"foo1"}, {:id=>"foo0"}]}
Using Enumerable#sort_by, though less efficient, one could write:
{ some_array: target_order.sort_by { |h| h[:new_position] }.map { |h| h.slice(:id) } }
#=> {:some_array=>[{:id=>"foo3"}, {:id=>"foo2"}, {:id=>"foo1"}, {:id=>"foo0"}]}
I have an array of hashes:
[
{
"June" => { "A" => { 3 => 48.4 } }
},
{
"January" => { "C" => { 2 => 88.0} }
},
{
"January"=> { "B" => { 2 => 44.0} }
},
{
"January"=> { "C" => { 4 => 48.8} }
}
]
I need to group each similar hash key into an array of the subsequent values like the following:
{
"June" => [{ "A" => [{ 3 => 48.4 }]] },
"January" => [
{ "B" => [{ 2 => 44.0}],
{ "C" => [{ 2 => 88.0}, { 4 => 48.8}],
] }
}
I am looking for an efficient method of grouping these elements. Can anyone help me master this hash of hashes?
I am trying to avoid looping through the base array and grouping manually. I was hoping that map (or some other enumerable method) might give what I want. When I used reduce(Hash.new, :merge), it came close but it used the last hash for each month key instead of adding it to an array.
Note: I added the following after gaining a clearer understanding of the question. My original answer is below.
Here is the OP's array of hashes, modified slightly.
arr = [{ "June" =>{ "A"=>{ 3=>48.4 } } },
{ "January"=>{ "C"=>{ 2=>88.0 } } },
{ "January"=>{ "B"=>{ "D"=>{ 2=>44.0 } } } },
{ "January"=>{ "C"=>{ 2=>10.0 } } },
{ "January"=>{ "C"=>{ 4=>48.8 } } }]
The hash to be constructed appears to be the following.
{ "June" =>[{ "A"=>[{ 3=>48.4 }] }],
"January"=>[{ "B"=>[{ "D"=>[{ 2=>44.0 }] }] }],
"C"=>[{ 2=>98.0, 4=>48.8 }] }] }
Note that 88.0 + 10.0 #=> 98.0 in 2=>98.0.
Observe that all the arrays within arr contain a single element, a hash. That being the case, those arrays serve no useful purpose. I therefore suggest the following hash be constructed instead:
{ "June" =>{ "A"=>{ 3=>48.4 } },
"January"=>{ "B"=>{ "D"=>{ 2=>44.0 } } },
"C"=>{ 2=>98.0, 4=>48.8 } } }
This can be produced with the following recursive method.
def recurse(arr)
arr.map(&:flatten).
group_by(&:first).
each_with_object({}) do |(k,v),h|
o = v.map(&:last)
h.update(k=>o.first.is_a?(Hash) ? recurse(o) : o.sum )
end
end
recurse(arr)
#=> {"June"=>{"A"=>{3=>48.4}},
# "January"=>{"C"=>{2=>98.0, 4=>48.8}, "B"=>{"D"=>{2=>44.0}}}}
(Original answer follows)
Here are two ways to obtain the desired hash. I assume that arr is your array of hashes.
#1 Use the form of Hash::new that takes a block
arr.each_with_object(Hash.new { |h,k| h[k] = [] }) do |g,h|
k, v = g.to_a.first
h[k] << v
end
# => {"June"=>[{"A"=>{3=>48.4}}],
# "January"=>[{"C"=>{2=>88.0}}, {"B"=>{2=>44.0}}, {"C"=>{4=>48.8}}]}
#2 Use Enumerable#group_by
arr.map(&:first).
group_by(&:first).
tap { |h| h.keys.each { |k| h[k] = h[k].map(&:last) } }
# => {"June"=>[{"A"=>{3=>48.4}}],
# "January"=>[{"C"=>{2=>88.0}}, {"B"=>{2=>44.0}}, {"C"=>{4=>48.8}}]}
The steps are as follows.
a = arr.map(&:first)
#=> [["June", {"A"=>{3=>48.4}}], ["January", {"C"=>{2=>88.0}}],
# ["January", {"B"=>{2=>44.0}}], ["January", {"C"=>{4=>48.8}}]]
b = a.group_by(&:first)
#=> {"June"=>[["June", {"A"=>{3=>48.4}}]],
# "January"=>[["January", {"C"=>{2=>88.0}}], ["January", {"B"=>{2=>44.0}}],
# ["January", {"C"=>{4=>48.8}}]]}
c = b.tap { |h| h.keys.each { |k| h[k] = h[k].map(&:last) } }
#=> {"June"=>[{"A"=>{3=>48.4}}],
# "January"=>[{"C"=>{2=>88.0}}, {"B"=>{2=>44.0}}, {"C"=>{=>48.8}}]}
Let me elaborate the last step. Inside tap's block, we compute the following.
h = b
d = h.keys
#=> ["June", "January"]
The first element of d is passed to each's block and the block variable is assigned to that element.
k = d.first
#=> "June"
The block calculation is as follows.
e = h[k]
#=> [["June", {"A"=>{3=>48.4}}]]
f = e.map(&:last)
#=> [{"A"=>{3=>48.4}}]
h[k] = f
#=> [{"A"=>{3=>48.4}}]
b #=> {"June"=>[{"A"=>{3=>48.4}}],
# "January"=>[["January", {"C"=>{2=>88.0}}],
# ["January", {"B"=>{2=>44.0}}],
# ["January", {"C"=>{4=>48.8}}]]}
Next, d[1] ("January") is passed to each's block and similar calculations are performed.
Rather than using Object#tap I could have written
h = arr.map(&:first).
group_by(&:first)
h.keys.each { |k| h[k] = h[k].map(&:last) }
h
tap merely avoids the creation of local variable h and the need to have a final line equal to h.
I have a nested field with arrays in array in JSON like the following:
{
"foo": {
"bar": [
[
"a",
"b"
],
[
"c",
"d"
]
]
}
}
The following is my config file:
input {
file {
codec => "json"
path => "pathtofile"
type => "footype"
start_position => "beginning"
}
}
filter {
json {
source => "message"
remove_field => [ "host", "message", "path" ]
}
}
output {
elasticsearch {
action => "index"
index => "bar"
hosts => [ "http://localhost:9200" ]
}
}
I got the following error:
09:40:47.725 [[main]>worker0] WARN logstash.outputs.elasticsearch -
Failed action. {:status=>400, :action=>["index", {:_id=>nil,
:_index=>"bar", :_type=>"footype", :_routing=>nil},
2017-02-13T01:40:30.387Z myconnection %{message}],
:response=>{"index"=>{"_index"=>"bar", "_type"=>"footype",
"_id"=>"AVo1IN0vK2jgwdCXqZ-q", "status"=>400,
"error"=>{"type"=>"illegal_argument_exception", "reason"=>"mapper
[foo.bar] of different type, current_type [long], merged_type
[text]"}}}}
I have a feeling that it's the array problem. I have done some research and know that array is not well supported. But I need to ingest the array in elasticsearch. Is there a way to actually do that?
Any helps will be appreciated.
I solved this by using a ruby filter:
ruby {
code => '
j = 0
for i in event.get("[foo][bar]") do
#i is an array element in the big array
l = 0
for k in i do
event.set("item#" + j.to_s + "#" + l.to_s, k)
l = l + 1
end
j = j + 1
end
'
}
This will eventually produce fields
item#0#0 = "a"
item#0#1 = "b"
item#1#0 = "c"
item#1#1 = "d"