Flattening a Sub-Array in a Powershell Object, including parent object property - arrays

Given the following JSON
[
{
"key": "James",
"things": [
{
"id": 123,
"name": "PRD"
},
{
"id": 124,
"name": "PRE"
}
]
},
{
"key": "Susan",
"things": [
{
"id": 125,
"name": "PRF"
},
{
"id": 126,
"name": "PRG"
}
]
}
]
Which, I've easily converted into a Powershell object:
$json = '[{"key":"James", "things":[{"id":123,"name":"PRD"},{"id":124,"name":"PRE"}]},{"key":"Susan", "things":[{"id":125,"name":"PRF"},{"id":126,"name":"PRG"}]}]'
$obj = $json | ConvertFrom-Json
How do I flatten the sub-array things and include the key from the parent object, so that my result set is
key id name
--- -- ----
James 123 PRD
James 124 PRE
Susan 125 PRF
Susan 126 PRG
I've used the following to flatten the sub-array:
$obj | % { $_.things}
Which returns me
id name
-- ----
123 PRD
124 PRE
125 PRF
126 PRG
But, I can't quite figure out what to do next.
Any help would be much appreciated.

You loop into each key, then loop into each things, since you want 1 result per thing, and build a PSObject using the current key, id and name.
Here you go.
# initial code sample
$json = '[{"key":"James", "things":[{"id":123,"name":"PRD"},{"id":124,"name":"PRE"}]},{"key":"Susan", "things":[{"id":125,"name":"PRF"},{"id":126,"name":"PRG"}]}]'
$obj = $json | ConvertFrom-Json
# Loop needed to flatten the object.
foreach ($i in $obj) {
foreach ($t in $i.things) {
[PSCustomObject]#{
key = $i.key
id = $t.id
name = $t.name
}
}
}
Output
key id name
--- -- ----
James 123 PRD
James 124 PRE
Susan 125 PRF
Susan 126 PRG

you can use built-in flattening with select as follows:
($json | ConvertFrom-Json) | Select-Object key -ExpandProperty things

Related

Powershell - json File - select only first value

I've got this json file:
[
{
"group": [313, 312, 313, 311],
"group_name": ["example", "example", "example1"],
"status": ["not_available_time", "no_time"]
}
]
And in Powershell I want the output to be:
313, example, not_available_time
So I always need just the first value.
I've tried this:
$responseJson = ConvertFrom-Json $response.Content
$group = $responseJson.group[0]
$name = $responseJson.group_name[0]
$status = $responseJson.status[0]
That works. But only if there is always more than one value.
For example: if "group_name" in the JSON file would only hold the value "example" (and not ["example", "example", "example1"]) => the output would only be "e" instead of "example". It takes not the first value, but only the first letter if there is only one value.
How can I select the first value?
BR and thanks!
There are 2 safe methods to ensure you're always selecting the first Value, be it an array, a scalar or null:
You can use the array subexpression operator #(), which will ensure that even if the value is $null or [System.Management.Automation.Internal.AutomationNull]::Value the result will always be an array (object[]):
#([System.Management.Automation.Internal.AutomationNull]::Value).GetType().Name # => Object[]
#($null).GetType().Name # => Object[]
As an alternative, Select-Object with the -First 1 argument, the difference is the returned object's Type remains the same:
(1 | Select-Object -First 1).GetType().Name # => Int32
('a', 'b' | Select-Object -First 1).GetType().Name # => String
$null -eq ($null, $null | Select-Object -First 1) # => True
Both alternatives are valid and depends on your use case which one fits your need better:
$json = #'
[
{
"group": [313, 312, 313, 311],
"group_name": "single_element",
"status": null
}
]
'# | ConvertFrom-Json
$json.PSObject.Properties | ForEach-Object {
$arr = #($_.Value)[0]
$slo = $_.Value | Select-Object -First 1
[pscustomobject]#{
'ArraySubExpression' = "[ {0}: $arr ]" -f $_.Name
'Select-Object' = "[ {0}: $slo ]" -f $_.Name
}
}
ArraySubExpression Select-Object
------------------ -------------
[ group: 313 ] [ group: 313 ]
[ group_name: single_element ] [ group_name: single_element ]
[ status: ] [ status: ]

JQ - access nested square brackets with fields with no names

trying to access a field in the list array via jq. The fields doesnt have a name for me to gain access to and extract. Please assist?
Trying to extract John and Smith.
$ cat test.txt
{
"content": {
"list": [
[
[
"name",
"John",
123
],
[
"surname",
"Smith",
345
],
1
]
]
}
}
$ jq -r '.content | {name: ."list"}' test.txt
{
"name": [
[
[
"name",
"John",
123
],
[
"surname",
"Smith",
345
],
1
]
]
}
You could do something as naive as:
$ jq -r '.content.list[][][1]?' test.json
John
Smith
Which will extract the second field from the array third nested arrays, and ignore the numeric literal.
Alternative you could manipulate the data before-hand to make it easier to manipulate afterwards:
$ jq '.content.list | map(map({ (.[0]): .[1] }?) | add)'
[
{
"name": "John",
"surname": "Smith"
}
]
Extracting the name(s) would be as simple as just using | [].name:
$ jq '.content.list | map(map({ (.[0]): .[1] }?) | add) | .[].name'
"John"

jq: map field array with different length

I'm working with those JSONs:
{
"extension": [
{
"url": "url1",
"system": "system1"
},
{
"url": "url2",
"system": "system2"
}
]
}
{
"extension": [
{
"url": "url3",
"system": "system3"
}
]
}
As you can see, both JSON objects have different .extension lenght.
I'm using this command in order to map input JSONs:
jq --raw-output '[.extension[] | .url, .system] | #csv'
You can find jqplay here.
I'm getting that:
"url1","system1","url2","system2"
"url3","system3"
What I would like to get is:
"url1","system1","url2","system2"
"url3","system3",,
Any ideas about how I could map those "fields" "correctly"?
Flip the table twice using transpose | transpose to fill up the slots missing from the unrigged square shape with null:
jq -rs 'map(.extension) | transpose | transpose[] | map(.url, .system) | #csv'
"url1","system1","url2","system2"
"url3","system3",,
Demo
A fairly efficient solution:
def pad:
(map(length)|max) as $mx
| map( . + [range(length;$mx)|null] );
[inputs | [.extension[] | (.url, .system)]]
| pad[]
| #csv
This of course should be used with the -n command-line option.

jq how to filter by field value in list

having an input like this:
[
{
"foo": "aaa",
"bar": 111
},
{
"foo": "bbb",
"bar": 111
},
{
"foo": "ccc",
"bar": 222
},
{
"foo": "aaa",
"bar": 333
},
{
"foo": "ddd",
"bar": 444
}
]
i would like to select all objects with "foo" key equal to "aaa" or "bbb". so the solution is obvious:
.[] | select ( .foo=="aaa" or .foo=="bbb" )
(https://jqplay.org/s/x7FGo1uQNW)
but i would like to enhance it and replace that x=y or x=z to sql'ish styled x in (y,z). and i got stuck, as the natural try:
.[] | select ( .foo in (["aaa", "bbb"]) )
results in an error:
jq: error: syntax error, unexpected IDENT, expecting ';' or ')' (Unix shell quoting issues?) at , line 1:
i also tried this:
.[] | select ( .foo | in (["aaa", "bbb"]) )
but also not great...
jq: error (at :21): Cannot check whether array has a string key
is this even posible?
hmm, i managed to do it with this:
.[] | select(.foo as $tmpvar | ["aaa", "bbb"] | index ($tmpvar ) )
https://jqplay.org/s/g7AyRgARdU
According to this answer:
https://stackoverflow.com/a/46470951/2244766
in versions above 1.5 there is a new IN operator that makes life a bit easier:
.[] | select(.foo|IN("aaa","bbb"))
The SQL-style operators don't work well for me as a straight-forward selection mechanism; I believe they have a very specific use-case, for which they are uniquely suitable for, and for anything else they're (at best) clunky. At least that's been my experience. And I haven't really figured out what that specific use-case is, either.
With all of that as a backdrop, my recommendation is to use a simple regex test:
map(select(.foo | test("aaa|bbb")))
Given the example JSON:
<~> $ jq . /tmp/so4229.json
[
{
"foo": "aaa",
"bar": 111
},
{
"foo": "bbb",
"bar": 111
},
{
"foo": "ccc",
"bar": 222
},
{
"foo": "aaa",
"bar": 333
},
{
"foo": "ddd",
"bar": 444
}
]
the above filter would result in:
<~> $ jq 'map(select(.foo | test("aaa|bbb")))' /tmp/so4229.json
[
{
"foo": "aaa",
"bar": 111
},
{
"foo": "bbb",
"bar": 111
},
{
"foo": "aaa",
"bar": 333
}
]
If you need to generate the regex based on other data within the JSON, you can do that, too:
. as $data | map(select(.bar==111) | .foo) | join("|") as $regex | . = $data | map(select(.foo | test($regex)))
which would result in:
<~> $ jq '. as $data | map(select(.bar==111) | .foo) | join("|") as $regex | . = $data | map(select(.foo | test($regex)))' /tmp/so4229.json
[
{
"foo": "aaa",
"bar": 111
},
{
"foo": "bbb",
"bar": 111
},
{
"foo": "aaa",
"bar": 333
}
]
There may be a better way to run through the JSON twice (once to get the regex values, once to use it).

I am trying to convert json file containing multiple nested arrays to csv file using powershell

I am trying to convert json file to csv file using Powershell. The json file that I am trying to convert has multiple nested arrays. I want to expand them all. Following is the file structure and the code I am working with:
This is a snapshot of my feed.json file:
{
"result": {
"problems": [
{
"id": "AHR157689",
"displayName": "YOCETJE",
"impact": "STRUCTURE",
"status": "OPEN",
"tagsOfEntities": [],
"ranked": [
{
"entityId": "843675746378564876",
"entityName": "HGFUTGYJDH",
"severityLevel": "8957685N8Y",
}
],
"affectedCounts": {
"INFRA": 1,
"STRUCTURE": 0,
"APPLICATION": 0,
},
"recoveredCounts": {
"INFRA": 0,
"STRUCTURE": 0,
"APPLICATION": 0,
},
"RootCause": true
}
]
}
}
And below is the code I am working with:
Get-Content C:\Documents\feed.json -Raw | ConvertFrom-Json | Select -Expand result | Select -Expand problems | Select * -Expand tagsOfEntities | Select * -Expand ranked | ConvertTo-Csv -NoTypeInformation | Out-File C:\Documents\output.csv
The above code is giving the correct output csv file but it runs for 2 hours. Am I doing something wrong?
This is the output in output.csv fil. This all data comes in one cell.
CONTEXTLESS,"1","Application","843675746378564876","843675746378564876","8957685N8Y","AHR157689","YOCETJE","STRUCTURE","OPEN","#{"INFRA": 1; "STRUCTURE": 0,"APPLICATION": 0}","#{"INFRA": 0;"STRUCTURE": 0;"APPLICATION": 0}","true"
There a two problems with your input JSON.
There are commas with nothing after, causing it to fail parsing with 'invalid JSON primitive' errors.
There was a curly bracket near the end that should have been a square bracket.
Also, the Powershell is selecting results but the JSON has result. Similarly, it's selecting problem but the JSON has problems.
Here's an example that corrects the JSON and a few other issues:
#"
{
"result": {
"problems": [
{
"id": "AHR157689",
"displayName": "YOCETJE",
"impact": "STRUCTURE",
"status": "OPEN",
"tagsOfEntities": [],
"ranked": [
{
"entityId": "843675746378564876",
"entityName": "HGFUTGYJDH",
"severityLevel": "8957685N8Y"
}
],
"affectedCounts": {
"INFRA": 1,
"STRUCTURE": 0,
"APPLICATION": 0
},
"recoveredCounts": {
"INFRA": 0,
"STRUCTURE": 0,
"APPLICATION": 0
},
"RootCause": true
}
]
}
}
"# | ConvertFrom-Json | Select -Expand result | Select -Expand problems |
Select * -Expand ranked | ConvertTo-Csv -NoTypeInformation | Out-File .\output.csv

Resources