In the example JSON at the bottom of this question, how can I count the number of key/value pairs in the array "Tags" using JMESPath?
According to the JMESPath documentation, I can do this using the count() function -
For example, the following expression creates an array containing the total number of elements in the foo object followed by the value of foo["bar"].
However, it seems that the documentation is incorrect. Using the JMESPath website, the query Reservations[].Instances[].[count(#), Tags] yeilds the result [ [ null ] ]. I then tested via the AWS command line and an error was returned -
Unknown function: count()
Is there actually a way of doing this using JMESPath?
Example JSON -
{
"Reservations": [
{
"Instances": [
{
"InstanceId": "i-asdf1234",
"InstanceName": "My Instance",
"Tags": [
{
"Value": "Value1",
"Key": "Key1"
},
{
"Value": "Value2",
"Key": "Key2"
},
{
"Value": "Value3",
"Key": "Key3"
},
{
"Value": "Value4",
"Key": "Key4"
}
]
}
]
}
]
}
The answer here is that the JMESPath documentation is shocking, and for some reason I was seeing out of date documentation (check the bottom right corner of the screen to see what version you are viewing.
I can do what I need to do using the length() function -
Reservations[].Instances[].Tags[] | length(#)
I managed to incorporate this usage of length length(Tags[*]) within a larger statement I think is useful and wanted to share:
aws ec2 describe-instances --region us-west-2 --query 'Reservations[*].Instances[*].{id: InstanceId, ami_id: ImageId, type: InstanceType, tag_count: length(Tags[*])}' --profile prod --output table;
--------------------------------------------------------------------
| DescribeInstances |
+--------------+-----------------------+------------+--------------+
| ami_id | id | tag_count | type |
+--------------+-----------------------+------------+--------------+
| ami-abc123 | i-redacted1 | 1 | m3.medium |
| ami-abc456 | i-redacted2 | 7 | m3.xlarge |
| ami-abc789 | i-redacted3 | 12 | t2.2xlarge |
+--------------+-----------------------+------------+--------------+
Related
I say this stack overflow anser with a simple jq command to convert a JSON to a csv file, but I need to improve it further.
Say I have the following JSON:
[
{
"name": "foo",
"env": "dev",
"version": "1.24"
},
{
"name": "bar",
"env": "staging",
"version": "1.21"
},
{
"name": "boo",
"env": "prod",
"version": "1.23"
},
{
"name": "far",
"env": "prod",
"version": "1.24"
}
]
How does one create the CSV with only the "name" and "version" fields?
My current command is:
jq -r '(map(keys) | add | unique) as $cols | map(.[] | {name, version} as $row | $cols | map($row[.])) as $rows | $cols, $rows[] | #csv'
This is not working. Can anyone provide some help?
Thanks!
If you know the two column names anyway, you could simply extract them directly using .name and .version:
<file jq -r '["name", "version"], (.[] | [.name, .version]) | #csv'
You can also use your $cols variable, so the names only appear once:
<file jq -r '["name", "version"] as $cols | $cols, (.[] | [.[$cols[]]]) | #csv'
Or import them dynamically, e.g. using --args:
<file jq -r '$ARGS.positional, (.[] | [.[$ARGS.positional[]]]) | #csv' \
--args name version
Output:
"name","version"
"foo","1.24"
"bar","1.21"
"boo","1.23"
"far","1.24"
Given the example JSON below:
{
"account_number": [
"123456"
],
"account_name": [
"name"
],
"account_id": [
654321
],
"username": [
"demo"
]
}
I'd like to get:
{
"account_number": "123456",
"account_name": "name",
"account_id": 654321,
"username": "demo"
}
Currently, I'm brute forcing it with | sed 's/\[//g' | sed 's/\]//g' | jq '.' ... but of course, that's ugly and causes issues if any of the values contain [ or ].
I've been unsuccessful with jq's flatten and other loops and mapping techniques like | jq -s '{Item:.[]} | .Item |add' to try and flatten the single-item arrays. Ideally, it would work where it would flatten arrays [...] to flat elements/objects {...}. Either way something better than replacing all occurrences of square brackets.
Short and sweet:
map_values(first)
Use with_entries, changing each value to the first element of itself:
jq 'with_entries(.value |= .[0])' file.json
I am trying to do what I think should be a fairly simple filter but I keep running into errors. I have this JSON:
{
"versions": [
{
"archived": true,
"description": "Cod version 3.3/Sprint 8",
"id": "11500",
"name": "v 3.3",
"projectId": 11500,
"releaseDate": "2016-03-15",
"released": true,
"self": "https://xxxxxxx.atlassian.net/rest/api/2/version/11500",
"startDate": "2016-02-17",
"userReleaseDate": "14/Mar/16",
"userStartDate": "16/Feb/16"
},
{
"archived": true,
"description": "Hot fix",
"id": "12000",
"name": "v3.3.1",
"projectId": 11500,
"releaseDate": "2016-03-15",
"released": true,
"self": "https://xxxxxxx.atlassian.net/rest/api/2/version/12000",
"startDate": "2016-03-15",
"userReleaseDate": "14/Mar/16",
"userStartDate": "14/Mar/16"
},
{
"archived": false,
"id": "29704",
"name": "Sync-diff v1.0.0",
"projectId": 11500,
"releaseDate": "2022-02-16",
"released": true,
"self": "https://xxxxxxx.atlassian.net/rest/api/2/version/29704",
"startDate": "2022-02-06",
"userReleaseDate": "15/Feb/22",
"userStartDate": "05/Feb/22"
}
]
}
I just want to return any userReleaseDate that ends with '22'
I can get the boolean result by:
jq '.versions[].userReleaseDate | endswith("22")'
prints out false, false, true
But I am not sure how to retrieve the objects. I tried variations of this:
[.versions[] as $keys | $keys select(endswith("22"))]
and each threw an error. Any help would be appreciated.
This was so close:
jq '.versions[].userReleaseDate | endswith("22")'
Rather than outputting whether they end with 22 or not, you want to select the values which end with 22. Fixed:
jq '.versions[].userReleaseDate | select( endswith("22") )'
Now, your question asks for the dates that end with 22, but the title suggests you want the objects. For that, you'd want something a little different. We want to select from the versions, not from the dates.
jq '.versions[] | select( .userReleaseDate | endswith("22") )' # As a stream
jq '[ .versions[] | select( .userReleaseDate | endswith("22") ) ]' # As an array
jq '.versions | map( select( .userReleaseDate | endswith("22") ) )' # As an array
There are a number of issues with [ .versions[] as $keys | $keys select(endswith("22")) ].
The keys of array element aren't usually called keys but indexes. $indexes would be a better name.
Except .versions[] gets the values of the array elements, not the keys/indexes. $values would be a better name.
Except the variable only takes on a single value at a time. $value would be a better name.
$version would be an even better name.
There's a | missing between $keys and select(endswith("22")).
There's no mention of userReleaseDate anywhere.
The result is placed in an array (because of the [ ]). There's no need or desire for this.
You could use
.versions[] as $version | $version.userReleaseDate | select(endswith("22"))
or
.versions[].userReleaseDate as $date | $date | select(endswith("22"))
But these are just overly-complicated versions of
jq '.versions[].userReleaseDate | select( endswith("22") )'
Use select directly on the list of objects, extract and check the release date inside its argument:
jq '.versions[] | select(.userReleaseDate | endswith("22"))'
I intend to extract a csv with a row for each sub array item.
Given a json array with a sub array. e.g. like this one:
[
{
"foo": 108,
"bar": ["a","b"]
},
{
"foo": 201,
"bar": ["c","d"]
}
]
It is possible to fetch the data by utilizing an intermediate object.
.[] | { "y": .foo, "x": .bar[] }| [.y,.x] | #csv
https://jqplay.org/s/922RlkbFNA
But I'd like to express it in a less elaborate form.
However the following does not work :( :
.[] | [ (.foo, .bar[]) ] | #csv
PS: I struggle to find a fitting headline
In three lines:
.[]
| [.foo] + (.bar[]|[.])
| #csv
or maybe less obscurely:
.[]
| .bar[] as $bar
| [.foo, $bar]
| #csv
I've got JSON that, among other top-level content, includes the following:
{
"organizationStructure": [
{
"id": 212119,
"key": "level2"
},
{
"id": 212112,
"key": "level1"
}
]
}
How can I filter by the key to find only a given id (such as that for "level2")?
Or, to keep it simpler (or more complicated, depending on your point of view):
jq '.organizationalStructure[] |
select(.key == "level1") | {id: .id}'
Often, it's nice to clean up the output:
jq -r '.organizationalStructure[] |
select(.key == "level1") | {id: .id}.id'
As per PesaThe's suggestion in the comments, this can be simplified to:
jq -r '.organizationalStructure[] |
select(.key == "level1").id'
and {.id: id} can be written simply {id}
This can be done using select. Note if there is more than one item with the key of level2 this will only return the first:
.organizationalStructure | map(select(.key == "level2") | .id)[0]