This question already has an answer here:
Select multiple fields at different levels
(1 answer)
Closed 2 years ago.
I have a file, let's call it heroes.json, where part of the data is a nested array of object, superpowers:
[
{
"hero": "Superman",
"id": "123",
"realName": "Clark Kent",
"age": "?",
"superpowers": [
{
"name": "speed",
"num": "1",
"des": "Faster than a speeding bullet.",
"value": "50"
},
{
"name": "strength",
"num": "2",
"des": "More powerful than a locomotive.",
"value": "100"
}
],
"weakness": "kryptonite"
},
{
"hero": "Batman",
"id": "456",
...
I want to select hero and superpowers, and keep only name and des keys within superpowers, like:
[
{
"hero": "Superman",
"superpowers": [
{
"name": "speed",
"des": "Faster than a speeding bullet."
},
{
"name": "strength",
"des": "More powerful than a locomotive."
}
]
},
{
"hero": "Batman",
"superpowers": [
...
It wouldn't be hard to write an iterator to do this, but I want to try jq as I'm new to this tool and it seems useful to learn.
So I experimented on jqplay until it delivered the needed format. I don't know if it's optimal, but this worked:
jq '[.[] | {hero, superpowers: [ .superpowers[] | {name, des} ] } ]'
(A graphQL-like filter syntax would make this easier.)
Note: The output required for my json differs from what is mentioned in this question and answer, and I have avoided using map (iterator) in the solution. In other words, I'm not asking the same question, or presenting the same answer.
It would be helpful to know if my solution is optimal.
Related
This question already has an answer here:
Output the results of select operation in an array - jq
(1 answer)
Closed 2 years ago.
I'm using the select function with jq to parse for items in a list that contain a certain value. I want the resulting output to be a json list, but jq gives me the objects individually:
Example:
$ a='{
"FOO": {
"name": "Donald",
"location": "Stockholm"
},
"BAR": {
"name": "Walt",
"location": "Stockholm"
},
"BAZ": {
"name": "Jack",
"location": "Whereever"
}
}'
$ echo $a | jq '.[] | select(.location=="Stockholm")'
{
"name": "Donald",
"location": "Stockholm"
}
{
"name": "Walt",
"location": "Stockholm"
}
Instead I want the output to be a json list like this:
[
{
"name": "Donald",
"location": "Stockholm"
},
{
"name": "Walt",
"location": "Stockholm"
}
]
How can I do this with jq?
In general, you can stick square brackets around any expression to gather all its outputs into an array.
[.[] | select(.location=="Stockholm")]
Sometimes it makes sense not to break up the input array in the first place, but use map to transform it:
map(select(.location=="Stockholm"))
Can anyone help me get the correct jq command to flatten the below example? I've seen a few other posts and I'm hacking away at it but can't seem to get it. I'd greatly appreciate any help.
Input:
[
{
"name": "level1",
"children": [
{
"name": "level2",
"children": [
{
"name": "level3-1",
"children": []
},
{
"name": "level3-2",
"children": []
}
]
}
]
}
]
Output:
[
{
"displayName": "level1",
"parent": ""
},
{
"displayName": "level2",
"parent": "level1"
},
{
"displayName": "level3-1",
"parent": "level2"
},
{
"displayName": "level3-2",
"parent": "level2"
}
]
Here's a straightforward solution that does not involve a helper function and actually solves a more general problem. It is based on the idea of beginning by adding a "parent" key to each child, and then using .. to collect all the name/parent pairs.
So first consider:
[ walk(if type=="object" and has("children")
then .name as $n | .children |= map(.parent = $n)
else . end)
| ..
| select(type=="object" and has("name"))
| {displayName: .name, parent}
]
This meets the requirements except that for the top-level (parentless) object, it produces a .parent value of null. That would generally be more JSON-esque than "", but if the empty string is really required, one has simply to replace the last non-trivial line above by:
| {displayName: .name, parent: (.parent // "")}
With a simple recursive function:
def f: .name as $parent | .children[] | {$parent, displayName: .name}, f;
[ {name: "", children: .} | f ]
Online demo
I have a file1.json with structure like this :
[
{
"uri": "features/hdp.feature",
"id": "as-a-user-i-want-to-use-house-detailed-page",
"keyword": "Feature",
"name": "As a user I want to use house detailed page",
"description": "",
"line": 2,
"tags": [
{
"name": "#hdp",
"line": 1
}
],
"elements": [
{
As you can see - it is an array with nested key:value pairs and other arrays. I need to convert it to ruby hash, but when I'm performing JSON.parse(file1) - it creates an array (http://prntscr.com/lqio6r) with ruby hashes, arrays and so on. If I'm performing JSON.parse(file1).reduce Hash.new, :merge or JSON.parse(file1).reduce Hash.new, :update) - as one of the answers on StackOverflow supposed - the result hash losses about 60% of .json content. Can you please advice on how can I convert json file to ruby hash (without any data losses)?
UPD - not truncated array - https://gist.githubusercontent.com/M1khah/3337507e3ca1544e6098bc726bca90cb/raw/c8262ad753bd0eebf1180e111acd016ffc07d1a5/gistfile1.txt
Hash with hashes - something like this instead of an array with nested hashes
{
{
"uri": "features/hdp.feature",
"id": "as-a-user-i-want-to-use-house-detailed-page",
"keyword": "Feature",
"name": "As a user I want to use house detailed page",
"description": "",
"line": 2,
"tags": [
{
"name": "#hdp",
"line": 1
}
],
"elements": [
{
}
I've got a more complex JQ expression that's handling an array of objects.
The input looks like this:
[
{ "key": "1", "value": "value 1"},
{ "key": "2", "value": "value 2"},
{ "key": "1", "value": "value 3"},
]
What I want to get is this:
{
"1": { "values": ["value 1", "value 3"] },
"2": { "values": ["value 2"] }
}
or, for my use case:
{
"1": [ "value 1", "value 3" ],
"2": [ "value 2" ]
}
would also be OK.
I've already tried to use … | { (.key): [.value] } but the result is (to no real surprise to me) that later occurrences of keys simply overwrite already existing ones. What I want to accomplish is something like "create a new key/value pair or add .value to an already existing one's 'values' array".
The drawback of a solution relying on group_by is that group_by requires a sort, which is unnecessary here. In this response, I'll show how to avoid any sorting by using a generic (and generally useful) jq function that "melds" an array of JSON objects, essentially by popping the value at each key into an array, and then concatenating corresponding arrays.
# input should be an array of objects
def meld:
reduce .[] as $o
({}; reduce ($o|keys)[] as $key (.; .[$key] += [$o[$key]] ));
Let's also define some data:
def data:
[
{ "key": "1", "value": "value 1"},
{ "key": "2", "value": "value 2"},
{ "key": "1", "value": "value 3"}
]
;
Then the filter:
data | map([.] | from_entries) | meld
produces:
{"1":["value 1","value 3"],"2":["value 2"]}
OK, after finally finding out what I wanted I also understand that my previous filters didn't keep the input array but resulted in objects being output after each other. So that was basically the reason why all examples I found wouldn't work.
I wanted to group by keys (hence the key/value requirement), which group_by already does, but wouldn't work.
From grouping working its only a small step to my solution (unique keys, values in arrays).
'… group_by(.key) | map({ "key": .[0].key, "values": map(.value) | unique })'
The output now looks like this, which is perfectly fine for my requirements:
[
{
"key": "1",
"values": [
"value 1",
"value 3"
],
},
{
"key": "2",
"values": [
"value 2"
]
}
]
I've been looking at some StackOverflow cases such as this case, but I cannot find an example with a document structure close to this one.
Below is an example of one document within my collection artistTags. All documents follow the same structure.
{
"_id": ObjectId("5500aaeaa7ef65c7460fa3d9"),
"toptags": {
"tag": [
{
"count": "100",
"name": "Hip-Hop"
},
{
"count": "97",
"name": "french rap"
},
...{
"count": "0",
"name": "seen live"
}
],
"#attr": {
"artist": "113"
}
}
}
1) How can I find() this document using the "artist" value (here "113")?
2) How can I retrieve all "artist" values having a specific "name" value (say "french rap") ?
Referring to chridam answer here above:
db.collection.find({"toptags.#attr.artist": "113"})