this is my data structure:
[
{
"name": "name1",
"organizations": [
{
"name": "name2",
"spaces": [
{
"name": "name3",
"otherkey":"otherval"
},
{
"name": "name4",
"otherkey":"otherval"
}
]
}
]
},
{
"name": "name21",
"organizations": [
{
"name": "name22",
"spaces": [
{
"name": "name23",
"otherkey":"otherval"
},
{
"name": "name24",
"otherkey":"otherval"
}
]
}
]
}
]
i just want to keep name=name1, remove the nested array object with name=name4 and want to keep the rest of the object intact. I tried with map(select) but this will just give me the full object. Is it possible to work with del on specific subarrays and keep the rest as it is?
result should be the following. in addition i want to avoid enumeration all attributes to keep on outer objects:
[
{
"name": "name1",
"organizations": [
{
"name": "name2",
"spaces": [
{
"name": "name3",
"otherkey":"otherval"
}
]
}
]
}
]
any idea? thanks!
A very targeted solution would be:
path(.[0].organizations[0].spaces) as $target
| (getpath($target) | map(select(.name != "name4"))) as $new
| setpath($target; $new)
If permissible, though, you might consider:
walk(if type == "object" and .spaces|type == "array"
then .spaces |= map(select(.name != "name4"))
else . end)
or:
del(.. | .spaces? // empty | .[] | select(.name == "name4") )
(If your jq does not have walk/1 then its jq definition can easily be found by googling.)
You can use the below and it will remove the "name": "name4" array only.
jq 'del(.[] | .organizations? | .[] | .spaces?|.[] | select(.name? == "name4"))' yourJsonFile.json
Here is a solution using select, reduce, tostream and delpaths
map(
select(.name == "name1")
| reduce (tostream|select(length==2)) as [$p,$v] (
.
; if [$p[-1],$v] == ["name","name4"] then delpaths([$p[:-1]]) else . end
)
)
I took a similar approach as #peak but inverted it, so instead of selecting what you want and setting that in the output we're selecting what we don't want and deleting it.
[path(.organizations[0].spaces[]| select(.name == "name4")] as $trash | delpaths($trash)
Related
Let's say I have more namespaces with the similar k8s resource (some might have different images used). I am trying to get .metadata.namespace using jq from the following json object (let's call it test.json):
{
"items": [
{
"metadata": {
"name": "app",
"namespace": "test1"
},
"spec": {
"components": [
{
"database": {
"from": "service",
"value": "redis"
},
"image": "test.com/lockmanager:1.1.1",
"name": "lockmanager01",
"replicas": 2,
"type": "lockmanager"
},
{
"database": {
"from": "service",
"value": "postgresql"
},
"image": "test.com/jobmanager:1.1.1",
"name": "jobmanager01",
"replicas": 2,
"type": "jobmanager"
}
]
}
}
]
}
if following condition is met:
.spec.components[].type == "jobmanager" and .spec.components[].image != "test.com/jobmanager:1.1.1"
but can't find the correct statement.
I tried:
cat test.json | jq '.items[] | select((.spec.components[].name? | contains("jobmanager01")) and (.spec.components[].image != "test.com/jobmanager:1.1.1")) | .metadata.namespace''
but it returns all namespaces and, moreover, those I am interested in (because I know they contain different image), are returned twice.
Please advise what am I doing wrong?
You state that the selection criterion is:
.spec.components[].type == "jobmanager" and
.spec.components[].image != "test.com/jobmanager:1.1.1"
but that does not make much sense, given the semantics of .[].
I suspect you meant that you want to select items from .spec.components such that
.type == "jobmanager" and .image != "test.com/jobmanager:1.1.1"
If that's the case, you could use any, so that your query would look like this:
.items[]
| select( any(.spec.components[];
(.name? | contains("jobmanager01")) and
.image != "test.com/jobmanager:1.1.1") )
| .metadata.namespace
all distinct
If you want all the distinct .namespace values satisfying the condition, you could go with:
[.items[]
| .metadata.namespace as $it
| .spec.components[]
| select( (.name? | contains("jobmanager01")) and
.image != "test.com/jobmanager:1.1.1" )
| $it]
| unique[]
Efficient version of "all-distinct" solution
To avoid unnecessary checks, if .namespace is always a string, we could write:
reduce .items[] as $item ({};
$item.metadata.namespace as $it
| if .[$it] then . # already seen
elif any( $item.spec.components[];
((.name? | contains("jobmanager01")) and
.image != "test.com/jobmanager:1.1.1") )
then .[$it] = true
else . end )
| keys_unsorted[]
Can anyone help me get the correct jq command to flatten the below example? I've seen a few other posts and I'm hacking away at it but can't seem to get it. I'd greatly appreciate any help.
Input:
[
{
"name": "level1",
"children": [
{
"name": "level2",
"children": [
{
"name": "level3-1",
"children": []
},
{
"name": "level3-2",
"children": []
}
]
}
]
}
]
Output:
[
{
"displayName": "level1",
"parent": ""
},
{
"displayName": "level2",
"parent": "level1"
},
{
"displayName": "level3-1",
"parent": "level2"
},
{
"displayName": "level3-2",
"parent": "level2"
}
]
Here's a straightforward solution that does not involve a helper function and actually solves a more general problem. It is based on the idea of beginning by adding a "parent" key to each child, and then using .. to collect all the name/parent pairs.
So first consider:
[ walk(if type=="object" and has("children")
then .name as $n | .children |= map(.parent = $n)
else . end)
| ..
| select(type=="object" and has("name"))
| {displayName: .name, parent}
]
This meets the requirements except that for the top-level (parentless) object, it produces a .parent value of null. That would generally be more JSON-esque than "", but if the empty string is really required, one has simply to replace the last non-trivial line above by:
| {displayName: .name, parent: (.parent // "")}
With a simple recursive function:
def f: .name as $parent | .children[] | {$parent, displayName: .name}, f;
[ {name: "", children: .} | f ]
Online demo
I have a JSON structure that has repeated keys per message. I would like to combine these into one array per message.
[
{
"id": 1,
"PolicyItems": [
{
"accesses": [
{
"isAllowed": true,
"type": "drop"
},
{
"isAllowed": true,
"type": "select"
}
],
"groups": [],
"users": ["admin"]
}
]
},
{
"id": 2,
"PolicyItems": [
{
"accesses": [
{
"isAllowed": true,
"type": "drop"
}
{
"isAllowed": true,
"type": "update"
}
],
"groups": [],
"users": [
"admin",
"admin2"
]
}
]
}]
I have this:
cat ranger_v2.json | jq -r '[.[] | {"id", "access_type":(.policyItems[].accesses[] | .type)}]'
But this outputs:
[
{
"id": 1,
"access_type": "drop"
},
{
"id": 1,
"access_type": "select"
},
{
"id": 2,
"access_type": "drop"
},
{
"id": 2,
"access_type": "update"
}
]
However, what I want is to output:
[{
"id": 1,
"access_type": ["drop|select"]
},
{
"id": 2,
"access_type": ["drop|update"]
}]
Any ideas how I could do this? I'm a bit stumped!
The values could be 'drop' and 'select', but equally could be anything, so I don't want to hard code these.
Let's start by observing that with your input, the filter:
.[]
| {id, access_type: [.PolicyItems[].accesses[].type]}
produces the two objects:
{
"id": 1,
"access_type": [
"drop",
"select"
]
}
{
"id": 2,
"access_type": [
"drop",
"update"
]
}
Now it's a simple matter to tweak the above filter so as to produce the desired format:
[.[]
| {id, access_type: [.PolicyItems[].accesses[].type]}
| .access_type |= [join("|")] ]
Or equivalently, the one-liner:
map({id, access_type: [[.PolicyItems[].accesses[].type] | join("|")]})
I found something that I can work with.
If I wrap the query with []...
cat ranger_v2.json | jq -r '[.[] | {"id", "access_type":([.policyItems[].accesses[] | .type])}]'
... it produces this type of output:
[
{
"id": 1,
"access_type": ["drop","select"]
},
{
"id": 2,
"access_type": ["drop","update"]
}
]
I can then use the following:
(if (."access_type" | length > 0 ) then . else ."access_type" = [""] end )]
and
(."access_type" | #tsv)
Before I can convert to #csv and use sed to replace the tab with a pipe.
#csv' | sed -e "s/[\t]\+/|/g"
It may not be the most economical way of getting what I need, but it works for me. (Please let me know if there's a better way of doing it.)
cat ranger_v2.json | jq -r '[.[] | {"id", "access_type":([.policyItems[].accesses[] | .type])}] | .[] | [(if (."access_type" | length > 0 ) then . else ."access_type" = [""] end )] | .[] | [.id, (."access_type" | #tsv)] | #csv' | sed -e "s/[\t]\+/|/g"
input
{
"apps": [
{
"name": "whatever1",
"id": "ID1"
},
{
"name": "whatever2",
"id": "ID2",
"dep": [
"a.jar"
]
},
{
"name": "whatever3",
"id": "ID3",
"dep": [
"a.jar",
"b.jar"
]
}
]
}
output
{
"apps": [
{
"name": "whatever1",
"id": "ID1",
"dep": [
"b.jar"
]
},
{
"name": "whatever2",
"id": "ID2",
"dep": [
"a.jar",
"b.jar"
]
},
{
"name": "whatever3",
"id": "ID3",
"dep": [
"a.jar",
"b.jar"
]
}
]
}
in the above example
whatever1 does not have dep, so create one.
whatever2 has dep and does not have b.jar, so add b.jar
whatever3 aready has dep and b.jar is there so untouched.
what i have tried.
# add blindly, whatever3 is not right
cat dep.json | jq '.apps[].dep += ["b.jar"]'
# missed one level and whatever3 is gone.
cat dep.json | jq '.apps | map(select(.dep == null or (.dep | contains(["b.jar"]) | not)))[] | .dep += ["b.jar"]'
For the sake of clarity, let's define a helper function for performing the core task:
# It is assumed that the input is an object
# that either does not have the specified key or
# that it is array-valued
def ensure_has($key; $value):
if has($key) and (.[$key] | index($value)) then .
else .[$key] += [$value]
end ;
The task can now be accomplished in a straightforward way:
.apps |= map(ensure_has("dep"; "b.jar"))
Alternatively ...
.apps[] |= ensure_has("dep"; "b.jar")
after some trial and error, it looks like this is one way to do it.
cat dep.json | jq '.apps[].dep |= (. + ["b.jar"] | unique)'
I want to use jq to remove all objects with a given name from all arrays in the input data. For example deleting "Name1" from this:
{
"Category1": [
{
"name": "Name1",
"desc": "Desc1"
},
{
"name": "Name2",
"desc": "Desc2"
}
],
"Category2": [
{
"name": "Name1",
"desc": "Desc1"
},
{
"name": "Name3",
"desc": "Desc3"
}
],
"Category3": [
{
"name": "Name4",
"desc": "Desc4"
}
]
}
Should yield this:
{
"Category1": [
{
"name": "Name2",
"desc": "Desc2"
}
],
"Category2": [
{
"name": "Name3",
"desc": "Desc3"
}
],
"Category3": [
{
"name": "Name4",
"desc": "Desc4"
}
]
}
I haven't worked with jq, or indeed JSON, much and after several hours of googling and experimenting I haven't been able to figure it out. How would I do this?
The closest I managed was this:
cat input | jq 'keys[] as $k | .[$k] |= map( select( .name != "Name1"))'
This does filter each of the arrays but returns the result as three separate objects and this is not what I want.
If the structure of your input JSON is always as seen on your example, try this:
map_values(map(select(.name != "Name1")))
Here is a solution that will remove all objects with the specified name, wherever they occur. It uses the generic function walk/1,
which is a built-in in versions of jq > 1.5, and can therefore be omitted if your jq includes it, but there is no harm in including it redundantly, e.g. in a jq script.
# Apply f to composite entities recursively, and to atoms
def walk(f):
. as $in
| if type == "object" then
reduce keys[] as $key
( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f
elif type == "array" then map( walk(f) ) | f
else f
end;
walk(if type == "object" and .name == "Name1" then empty else . end)
If you really only want to remove objects from arrays, then you could use:
walk(if type == "array" then map(select( type != "object" or .name != "Name1")) else . end)
Here is a solution which uses reduce and del
reduce keys[] as $k (
.
; del(.[$k][] | select(.name == "Name1"))
)