Modify keys in multiple nested objects/arrays - arrays

I want to modify the value of all x keys in a json that looks like:
{
"a": {
"b": {
"c": [
{
"0": {
"x": 23,
"name": "AS"
}
},
{
"1": {
"x": 23,
"name": "AS"
}
},
{
"2": {
"x": 23,
"name": "Fe"
}
},
{
"3": {
"x": 23,
"name": "Pl"
}
}
]
}
}
}
I have tried multiple approaches, but I can't modify the value of x and obtain the full json as a result. All I managed to do is modify the value of x and obtain the last array as a result.
Here is the closest I have been to achieve the result: https://jqplay.org/s/Wx741btZOg

Using |= one can simply perform the update by writing:
.a.b.c |= [.[]|.[].x=97]
or perhaps more clearly:
.a.b.c |= map(.[].x=97)
If you really do want to "modify the value of all x keys", then you could use walk:
walk(if type == "object" and has("x") then .x=97 else . end)
(If your jq does not have walk, then you can snarf its def from the web, e.g. from builtin.jq )

To change all x values to 97, you can try this jq command:
<file jq '.a.b.c as $in | .a.b.c=[ $in[] | .[].x=97 ]'
The command stores the parent of the object in the variable $in such that you can modify one of its sub element.

Related

Updating a json file using jq while keep the exact same input (except changed value)

To keep things simple, I have that file.js:
{
"devices": [
{
"label": "label1",
"percent": 10
},
{
"label": "label2",
"percent": 20
}
]
}
Since I browsed Google and StackOverflow a lot, I actually know how to update, let's say, the label2 from 20% to 50% :
jq '.devices[] | select(.label == "label2").percent = 50' file.json
But here's the ouput:
{
"label": "label1",
"percent": 10
}
{
"label": "label2",
"percent": 50
}
The devices object (?) is not there anymore.
I'd like to find a way to keep the exact same input format.
I guess I should rid of the pipe, but I don't know how.
I've browsed Google and I found the
to_entries / with_entries / from_entries, that seem to keep the whole thing, but I don't know how to merge eveything together.
Thank you.
Make the traversal and filtering all part of the assignment's LHS using parentheses:
jq '(.devices[] | select(.label == "label2").percent) = 50' file.json
Demo
Alternatively, update |= to a map of the whole .devices array:
jq '.devices |= map(select(.label == "label2").percent = 50)' file.json
Demo
Output:
{
"devices": [
{
"label": "label1",
"percent": 10
},
{
"label": "label2",
"percent": 50
}
]
}

How get index of array with jq

I want search this string tb1qpvtnfqqs3cp4ly4375km7n5sga8hkdkujkm854 in that structure
{
"txid": "67bc5194442dc350312a7c0a5fc7ef912c31bf00b23349b4c3afdf177c91fb2f",
"hash": "8392ded0647e4166eda342cee409c7d0e1e3ffab24de41866d2e6a7bd0a245b3",
"version": 2,
"size": 245,
"vsize": 164,
"weight": 653,
"locktime": 1764124,
"vin": [
{
"txid": "69eed058cbd18b3bf133c8341582adcd76a4d837590d3ae8fa0ffee1d597a8c3",
"vout": 0,
"scriptSig": {
"asm": "0014759fc698313da549948940508df6db93a319096e",
"hex": "160014759fc698313da549948940508df6db93a319096e"
},
"txinwitness": [
"3044022014a8eb758063c52bc970d42013e653f5d3fb3c190b55f7cfa72680280cc5138602202a873b5cad4299b2f52d8cccb4dcfa66fa6ec256d533788f54440d4cdad7dd6501",
"02ec8ba22da03ed1870fe4b9f9071067a6a1fda6f582c5c858644e44bd401bfc0a"
],
"sequence": 4294967294
}
],
"vout": [
{
"value": 0.37841708,
"n": 0,
"scriptPubKey": {
"asm": "0 686bc8ce41505642c96f3eb99919fff63f4c0f11",
"hex": "0014686bc8ce41505642c96f3eb99919fff63f4c0f11",
"reqSigs": 1,
"type": "witness_v0_keyhash",
"addresses": [
"tb1qdp4u3njp2pty9jt086uejx0l7cl5crc3x3phwd"
]
}
},
{
"value": 0.00022000,
"n": 1,
"scriptPubKey": {
"asm": "0 0b173480108e035f92b1f52dbf4e90474f7b36dc",
"hex": "00140b173480108e035f92b1f52dbf4e90474f7b36dc",
"reqSigs": 1,
"type": "witness_v0_keyhash",
"addresses": [
"tb1qpvtnfqqs3cp4ly4375km7n5sga8hkdkujkm854"
]
}
}
],
"hex": "02000000000101c3a897d5e1fe0ffae83a0d5937d8a476cdad821534c833f13b8bd1cb58d0ee690000000017160014759fc698313da549948940508df6db93a319096efeffffff022c6b410200000000160014686bc8ce41505642c96f3eb99919fff63f4c0f11f0550000000000001600140b173480108e035f92b1f52dbf4e90474f7b36dc02473044022014a8eb758063c52bc970d42013e653f5d3fb3c190b55f7cfa72680280cc5138602202a873b5cad4299b2f52d8cccb4dcfa66fa6ec256d533788f54440d4cdad7dd65012102ec8ba22da03ed1870fe4b9f9071067a6a1fda6f582c5c858644e44bd401bfc0a1ceb1a00",
"blockhash": "000000009acb8b4f06a97beb23b3d9aeb3df71052dabec94465933b564c27f50",
"confirmations": 2,
"time": 1591687001,
"blocktime": 1591687001
}
I'd like to get the index of vout, in this case 1. is it possible with jq?
It's not clear what exactly you want.
I guess you want the n of the element of vout that contains the given address in its addresses list. That can be achieved with
jq '.vout[]
| select(.scriptPubKey.addresses[] == "tb1qpvtnfqqs3cp4ly4375km7n5sga8hkdkujkm854")
| .n
' file.json
You can also use
select((.scriptPubKey.addresses[]
| contains("tb1qpvtnfqqs3cp4ly4375km7n5sga8hkdkujkm854")))
to search for the address.
The following assumes that you want the index in .vout of the first object which has the given string as a leaf value, and that you have in mind using 0 as the index origin.
A simple and reasonably efficient jq program that finds all such indices is as follows:
.vout
| range(0;length) as $i
| if any(.[$i]|..;
. == "tb1qpvtnfqqs3cp4ly4375km7n5sga8hkdkujkm854")
then $i
else empty
end
With the given input, this in fact yields 1, which is in accordance with the problem description, so we seem to be on right track.
To get the first index, you could wrap the above in first(...), but in that case the result would be the empty stream if there is no occurrence. So perhaps you would prefer to wrap the above in first(...) // null
You could try something like this:
$vout={{ your json }}
$value="tb1qpvtnfqqs3cp4ly4375km7n5sga8hkdkujkm854"
result=$(echo "$vout" | jq -r '.[0] | select($value)')

variant of jq from_entries that collate values for each key occurrence

Can I use jq to run a filter that behaves similarly to from_entries, with the one difference being, if multiple entries for the same key are encountered, it will collate the values into an array, rather than just use the last value?
If so, what filter would achieve this? For example, if my input is:
[
{
"key": "a",
"value": 1
},
{
"key": "b",
"value": 2
},
{
"key": "a",
"value": 3
},
{
"key": "b",
"value": 4
}
]
then the desired output would be:
{ "a": [1,3], "b": [2,4] }
Note that, using 'from_entries' alone as the filter, the resulting values are just the last value (that is, { "a": 3, "b": 4 })
With your example and the following lines in merge.jq:
def merge_entries:
reduce .[] as $pair ({}; .[$pair["key"]] += [$pair["value"]] );
merge_entries
the invocation: jq -c -f merge.jq
yields:
{"a":[1,3],"b":[2,4]}
You could also use the invocation:
jq 'reduce .[] as $p ({}; .[$p.key] += [$p.value])'

JSON: use jq to edit specific values in nested arrays

I'm trying to update values within an array inside an array using the utility jq. I've pasted the sample json below.
More specifically: Within the sheets array, and then within the formulas array, I'd like to change each columnName with a value of "MONTH" to "YEAR". I'd like to do the same for within the sheets array, within the columnStyles array, change each incidence of "MONTH" also to "YEAR"
This jq filter gets me the list of columnNames.
.sheets[1] | .formulas[] | .columnName
How can I edit the entire file in place by just updating the values I want? Do I use map with if?
And what if I wanted to edit a portion of a value? For example, in a forumlaString property, just changing the part of the string that contains MONTH but leaving the rest intact?
{
"version": "6.1.1",
"className": "xyz",
"sheets": [
{
"name": "Pass1",
"sheetId": "95e6c2cd-abbe-46c1-8012-bdf37438b9b7",
"keep": true,
"formulas": [
{
"columnName": "SAMPLE_PROVIDER",
"columnId": "0",
"columnIndex": 0,
"formulaString": "\u003dGROUPBY(#Raw!SAMPLE_PROVIDER)"
},
{
"columnName": "MONTH",
"columnId": "1",
"columnIndex": 1,
"formulaString": "\u003dGROUPBY(#Raw!MONTH)"
}
],
"columnStyles": [
{
"columnId": "0",
"name": "SAMPLE_PROVIDER",
"width": 206,
"thousandSeparator": true
},
{
"columnId": "1",
"name": "MONTH",
"width": 100,
"thousandSeparator": true
}
],
"nextColumnId": 2
},
{
"name": "Transform1",
"sheetId": "49071c1c-fa84-4ae3-92c1-b63175a6b26c",
"keep": true,
"formulas": [
{
"columnName": "SAMPLE_PROVIDER",
"columnId": "0",
"columnIndex": 0,
"formulaString": "\u003d#Pass1!SAMPLE_PROVIDER"
},
{
"columnName": "MONTH",
"columnId": "1",
"columnIndex": 1,
"formulaString": "\u003d#Pass1!MONTH"
}
],
"columnStyles": [
{
"columnId": "0",
"name": "SAMPLE_PROVIDER",
"width": 179,
"thousandSeparator": true
},
{
"columnId": "1",
"name": "MONTH",
"width": 100,
"thousandSeparator": true
}
],
"nextColumnId": 3
}
],
"advancedSchedulingInUse": true,
"errorHandlingMode": "IGNORE"
}
To change the columnName field in the desired containers, you can use
jq '(.sheets[] | .formulas[]? | .columnName | select(.=="MONTH")) |= "YEAR"' tmp.json
(The ? avoids an error if there is no key formula.)
To replace MONTH with YEAR in formula strings, replace each formulaString value with a possible modified string returned by sub.
jq '(.sheets[] | .formulas[]? | .formulaString) |= sub("MONTH"; "YEAR")' tmp.json
(sub requires jq 1.5, compiled with the Oniguruma library.)
To combine these into a single jq filter? I'm not sure yet; I have only a tenuous understanding of why either one alone works.
It looks like you're updating more than just fields in the formulas arrays, but a little bit of everything.
If you want to indiscriminately change all occurrences of the string "MONTH" to "YEAR", you could do this:
(.. | strings) |= sub("MONTH"; "YEAR")
This may be a task for walk/1.
(If your jq does not have walk/1, then you can copy its definition from https://github.com/stedolan/jq/blob/master/src/builtin.jq)
For example, if you want to change "MONTH" to "YEAR" whenever "MONTH" appears as the value of a key in an object, then the following would do the job:
jq 'walk(if type == "object"
then with_entries(.value |= (if . == "MONTH" then "YEAR" else . end))
else . end)' input.json
Equivalently:
jq 'walk(if type == "object"
then with_entries(if .value == "MONTH" then .value = "YEAR" else . end)
else . end)' input.json
These can easily be modified in accordance with similar requirements.

JQ: Remove object from multiple arrays

I want to use jq to remove all objects with a given name from all arrays in the input data. For example deleting "Name1" from this:
{
"Category1": [
{
"name": "Name1",
"desc": "Desc1"
},
{
"name": "Name2",
"desc": "Desc2"
}
],
"Category2": [
{
"name": "Name1",
"desc": "Desc1"
},
{
"name": "Name3",
"desc": "Desc3"
}
],
"Category3": [
{
"name": "Name4",
"desc": "Desc4"
}
]
}
Should yield this:
{
"Category1": [
{
"name": "Name2",
"desc": "Desc2"
}
],
"Category2": [
{
"name": "Name3",
"desc": "Desc3"
}
],
"Category3": [
{
"name": "Name4",
"desc": "Desc4"
}
]
}
I haven't worked with jq, or indeed JSON, much and after several hours of googling and experimenting I haven't been able to figure it out. How would I do this?
The closest I managed was this:
cat input | jq 'keys[] as $k | .[$k] |= map( select( .name != "Name1"))'
This does filter each of the arrays but returns the result as three separate objects and this is not what I want.
If the structure of your input JSON is always as seen on your example, try this:
map_values(map(select(.name != "Name1")))
Here is a solution that will remove all objects with the specified name, wherever they occur. It uses the generic function walk/1,
which is a built-in in versions of jq > 1.5, and can therefore be omitted if your jq includes it, but there is no harm in including it redundantly, e.g. in a jq script.
# Apply f to composite entities recursively, and to atoms
def walk(f):
. as $in
| if type == "object" then
reduce keys[] as $key
( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f
elif type == "array" then map( walk(f) ) | f
else f
end;
walk(if type == "object" and .name == "Name1" then empty else . end)
If you really only want to remove objects from arrays, then you could use:
walk(if type == "array" then map(select( type != "object" or .name != "Name1")) else . end)
Here is a solution which uses reduce and del
reduce keys[] as $k (
.
; del(.[$k][] | select(.name == "Name1"))
)

Resources