Powershell: Reformat JSON nested Arrays to Array of Objects - arrays

I'm trying to modify a badly formated JSON Textfile, that currently looks like this:
[
["val1", "val2", "val3", "val4"],
["val5", "val6", "val7", "val8"],
["val9", "val10", "val11", "val12"]
]
and I have an other array containing field names
["title1", "title2", "title3", "title4"]
I want to output a final.json textfile looking like this:
[
{"title1": "val1", "title2": "val2", "title3": "val3", "title4": "val4"},
{"title1": "val5", "title2": "val6", "title3": "val7", "title4": "val8"},
{"title1": "val9", "title2": "val10", "title3": "val11", "title4": "val12"}
]
I guess the best way would be to take each row, split by , and then adding them back together foreach-ing over the title names, but I'm not quite sure on how to do this in PowerShell.

Since you're dealing with structured data here, I think the best way is to parse the JSON, and work with the resulting objects. Create the objects you want, then convert back to JSON:
$j1 = #'
[
["val1", "val2", "val3", "val4"],
["val5", "val6", "val7", "val8"],
["val9", "val10", "val11", "val12"]
]
'#
$j2 = #'
["title1", "title2", "title3", "title4"]
'#
$a1 = $j1 | ConvertFrom-Json
$a2 = $j2 | ConvertFrom-Json
0..($a1.Count-1) | ForEach-Object {
$i = $_
$props = #{}
0..($a2.Count-1) | ForEach-Object {
$props[$a2[$_]] = $a1[$i][$_]
}
New-Object PSOBject -Property $props
} | ConvertTo-Json
ConvertTo-Json and ConvertFrom-Json are the cmdlets you need to serialize/deserialize the JSON. Then you just work with the objects.
In this case I'm going through each top level array in $a1 and creating a hashtable that contains the properties you want. Then I'm creating a PSObject with those properties. That gets returned from the ForEach-Object cmdlet (the result is an array of those objects) which then gets piped directly into ConvertTo-Json to give you the output needed.

I think a better approach is to read in the first JSON format via ConvertFrom-Json, then take that array of arrays and for each row, create a PSCustomObject from a hashtable e.g. [PSCustomObject]#{title1=$arr[$row][0]; title2=$arr[$row][1];...}. Once you then have that array of PSCustomObject, convert that back to JSON with ConvertTo-Json.

Related

Paired enumeration of two arrays [duplicate]

This question already has answers here:
Powershell foreach regarding multiple collections
(1 answer)
Display multiple array values in list in Powershell
(2 answers)
Closed 3 months ago.
I have 2 arrays: I need to sort them in foreach to modify 1 file.
1 array with dbs name like:
$dbs = ['Honda', 'Toyota', 'BMW', 'Opel']
2nd array with their size like:
$dbsSize = ['123', '300', '222', '143']
and I can sort names in foreach like
foreach ($db in $dbs){
# some logic..
}
How can I sort 2nd array in the same foreach so I can use it in the same loop
Like this:
foreach ($db in $dbs) {
# some logic with $db
# and here I need some logic with $dbsSize
}
but to sort out $dbsSize like $dbs because I need the result like :
Honda
123
Toyota
300
etc.
Do I need to create another loop?
It seems to me you have two arrays where the indices determine which element of the one array belongs to the element in the other array.
It is unclear how you created them and why you did not create an array of objects with both properties together in the first place, but you can still do that like this:
$dbs = 'Honda', 'Toyota', 'BMW', 'Opel'
$dbsSize = '123', '300', '222', '143'
$result = for($i = 0; $i -lt [math]::Max($dbs.Count, $dbsSize.Count); $i++) {
[PsCustomObject]#{
Car = $dbs[$i]
Size = $dbsSize[$i]
}
}
# sort the items on 'Car' for instance
$result = $result | Sort-Object Car
# show on screen
$result
# save to structured CSV file you can open in Excel
$result | Export-Csv -Path 'X:\Somewhere\cars.csv' -UseCulture -NoTypeInformation
Result on screen using the above exampe looks like
Car Size
--- ----
BMW 222
Honda 123
Opel 143
Toyota 300
I don't see any sorting in your code, I only see enumeration. Seems like you want to enumerate both collections sequentially, in that case you would need to use a for loop instead of foreach:
$dbs = "['Honda', 'Toyota', 'BMW', 'Opel']" | ConvertFrom-Json
$dbsSize = "['123', '300', '222', '143']" | ConvertFrom-Json
for($i = 0; $i -lt [math]::Max($dbs.Count, $dbsSize.Count); $i++) {
$dbs[$i]
$dbsSize[$i]
}
If you want to use a foreach loop and both collections have the same Length, then you can get the items of one of them (your choice which) via indexing [ ]:
$i = 0
foreach($item in $dbs) {
$item
$dbsSize[$i++]
}
If you're looking to merge both arrays into objects, an easy way to do it is with the function from this answer, the usage in this case would be:
$dbs, $dbsSize | Join-Array -Columns Car, Size

Powershell, JSON and Hashtables

My JSON
[
{
"solution": "abc",
"solutionName": "abc_test",
"solutionShortcode": "",
"isManaged": false
},
{
"solution": "def",
"solutionName": "def_test",
"solutionShortcode": "def1",
"isManaged": true
}
]
What I need to do is take the solutionName and the solutionShortcode from each and insert them into a new hashtable - i thought this would work...
$buildDictionary = #{}
$b = Get-Content -Raw -Path ./temp.json | ConvertFrom-Json
foreach ($a in $b.solutionName.GetEnumerator()) {
$name = $b.solutionName.toString()
$shortcode = $b.solutionShortcode.toString()
if ([string]::IsNullOrEmpty($shortcode)) {
$shortcode = "0xxx"
}
$buildDictionary.Add("$name","$shortcode")
}
Write-Output $buildDictionary
Basically what I need to do is not that complex but there must be something I am missing about the ConvertFrom-Json cmdlet because this is not working as expected.
Essentially I need to add a generic value if the "shortcode" is empty and take insert the solutionName and solutionShortcode into the "buildDictionary" hashtable as a key/value pair.
What I have currently errors with "Key already added" error messages.. my resulting hashtable ends up with a single row consisting of a System.Object and not a key=value pair with string content.
Would appreciate some insight into what is wrong and why it is wrong.
Thanks!
...should do what you want:
$buildDictionary = #{}
$json = ConvertFrom-Json -InputObject (gc '.\temp.json' -raw)
$json | %{
If (!$_.solutionShortCode){
$_.solutionShortCode = 'someValue'
$buildDictionary.add($_.solutionName,$_.solutionShortcode)
}
}
If solutionShortCode is empty it sets 'someValue' as value and adds solutionName as Key to the HashTable $buildDictionary with the value solutionShortCode.

Powershell nested JSON to csv conversion

I have a rather peculiar nested JSON where in some instances a key - value pair occurs as normal, but in others the type of the key appears in a further nesting.
{"metadata":{"systemId":"da1895","legalEntity":"A0"},"recordContent":{"positionDate":"2019-04-08 00:00:00.0","account":{"string":"G32"},"seg":{"string":"S"},"strike":{"double":4.4}}}
{"metadata":{"systemId":"45364d","legalEntity":"5G"},"recordContent":{"positionDate":"2019-04-08 00:00:00.0","account":{"string":"G81"},"seg":{"string":"S"},"strike":{"double":5.0}}}
In the example you can see metadata's fields are straightforward key-value pairs, but underneath recordContent, we have positionDate which is a straightforward key-value but "account":{"string":"G32"} and "strike":{"double":4.4} are not.
I'd like to ditch the type information and arrive at a CSV structure as follows:
systemId, legalEntity, positionDate, account,seg,strike
da1895, A0, 2019-04-08 00:00:00.0,G32, S, 4.4
4536d, 5G, 2019-04-08 00:00:00.0,G81, S, 5.0
Any ideas on how to convert such a structure to CSV using Powershell?
Here's what I tried:
$TemplateParametersFile = "c:\data\output.json"
$JsonParameters = Get-Content $TemplateParametersFile | ConvertFrom-Json
$metadatafields = $JsonParameters.metadata[0].PSObject.Properties.Name
$recordcontentfields = $JsonParameters.recordContent[0].PsObject.Properties.Name
$oData = New-Object PSObject
$metadatafields |
ForEach {
Add-Member -InputObject $oData -NotePropertyName ($_) -NotePropertyValue $JsonParameters.metadata.($_)
}
$recordcontentfields |
ForEach {
Add-Member -InputObject $oData -NotePropertyName ($_) -NotePropertyValue $JsonParameters.recordContent.($_)
}
This gave me:
$oData
systemId : {da1895, 45364d}
legalEntity : {A0, 5G}
positionDate : {2019-04-08 00:00:00.0, 2019-04-08 00:00:00.0}
account : {#{string=G32}, #{string=G81}}
seg : {#{string=S}, #{string=S}}
strike : {#{double=4.4}, #{double=5.0}}
I'm a bit stuck now and the above doesn't convert to csv.
Note that other than metadata and recordContent, I've not hardcoded any fieldnames and I'd like to maintain that flexibility in case the JSON structure changes.
Thanks
I suggest collecting the property-name-value pairs iteratively in an ordered hashtable ([ordered] #{}), which can then be cast to [pscustomobject] to convert it to a custom object.
No property names are hard-coded in the following solution, but the object-graph structure is assumed to follow the pattern in your sample JSON, which is limited to one level of nesting - if you need to process arbitrarily nested objects, this answer may be a starting point.
Reflection (discovery of the property names and values) is performed via the intrinsic .psobject property that PowerShell makes available on all objects.
# Parse sample JSON into an array of [pscustomobject] graphs.
$fromJson = ConvertFrom-Json #'
[
{"metadata":{"systemId":"da1895","legalEntity":"A0"},"recordContent":{"positionDate":"2019-04-08 00:00:00.0","account":{"string":"G32"},"seg":{"string":"S"},"strike":{"double":4.4}}}
,
{"metadata":{"systemId":"45364d","legalEntity":"5G"},"recordContent":{"positionDate":"2019-04-08 00:00:00.0","account":{"string":"G81"},"seg":{"string":"S"},"strike":{"double":5.0}}}
]
'#
# Initialize an aux. ordered hashtable to collect the property-name-value
# pairs in.
$oht = [ordered] #{}
$fromJson | ForEach-Object {
$oht.Clear()
# Loop over top-level properties.
foreach ($topLevelProp in $_.psobject.Properties) {
# Loop over second-level properties.
foreach ($prop in $topLevelProp.Value.psobject.Properties) {
if ($prop.Value -is [System.Management.Automation.PSCustomObject]) {
# A nested value: Use the value of the (presumed to be one-and-only)
# property of the object stored in the value.
$oht[$prop.Name] = $prop.Value.psobject.Properties.Value
}
else {
# A non-nested value: use as-is.
$oht[$prop.Name] = $prop.Value
}
}
}
# Construct and output a [pscustomobject] from the aux. ordered hashtble.
[pscustomobject] $oht
} |
ConvertTo-Csv # Replace this with Export-Csv to export to a file.
The above yields:
"systemId","legalEntity","positionDate","account","seg","strike"
"da1895","A0","2019-04-08 00:00:00.0","G32","S","4.4"
"45364d","5G","2019-04-08 00:00:00.0","G81","S","5"
A few years ago, I wrote a reusable Flatten-Object function for this.
The only difference is that it combines the (sub)property names with the parent property names as they might not be unique:
$JsonParameters |Flatten-Object |Format-Table
metadata.systemId metadata.legalEntity recordContent.positionDate recordContent.account.string recordContent.seg.string recordContent.strike.double
----------------- -------------------- -------------------------- ---------------------------- ------------------------ ---------------------------
da1895 A0 2019-04-08 00:00:00.0 G32 S 4.4
45364d 5G 2019-04-08 00:00:00.0 G81 S 5
Try this:
$data = ConvertFrom-Json #"
[
{"metadata":{"systemId":"da1895","legalEntity":"A0"},"recordContent":{"positionDate":"2019-04-08 00:00:00.0","account":{"string":"G32"},"seg":{"string":"S"},"strike":{"double":4.4}}},
{"metadata":{"systemId":"45364d","legalEntity":"5G"},"recordContent":{"positionDate":"2019-04-08 00:00:00.0","account":{"string":"G81"},"seg":{"string":"S"},"strike":{"double":5.0}}}
]
"#
$data | Select-Object -Property #{l="systemId"; e={$_.metadata.systemId}}, #{l="legalEntity"; e={$_.metadata.legalEntity}},
#{l="positionDate"; e={$_.recordContent.positionDate}}, #{l="account"; e={$_.recordContent.account.string}},
#{l="seg"; e={$_.recordContent.seg.string}}, #{l="strike"; e={$_.recordContent.strike.double}} | Export-Csv
This should work with any nested psobject.
$json = #'
{"metadata":{"systemId":"da1895","legalEntity":"A0"},"recordContent":{"positionDate":"2019-04-08 00:00:00.0","account":{"string":"G32"},"seg":{"string":"S"},"strike":{"double":4.4}}}
'#
$obj = ConvertFrom-Json $json
$obj.recordContent | gm -MemberType NoteProperty | % {
$prop = $_.name
if ($obj.recordContent.$prop.GetType().name -eq 'pscustomobject') {
$obj.recordContent.$prop = $obj.recordContent.$prop.psobject.Members | where membertype -eq noteproperty | select -ExpandProperty value
}
$obj.metadata | add-member -MemberType NoteProperty -Name $prop -Value $obj.recordContent.$prop
}
$newobj = $obj.metadata
$newobj

Powershell Compare 2 Arrays of Hashtables based on a property value

I have one array of hashtables like the one below:
$hashtable1 = #{}
$hashtable1.name = "aaa"
$hashtable1.surname =#()
$hashtable1.surname += "bbb"
$hashtable2 = #{}
$hashtable2.name = "aaa"
$hashtable2.surname =#()
$hashtable2.surname += "ccc"
$hashtable3 = #{}
$hashtable3.name = "bbb"
$hashtable3.surname = #()
$hashtable3.surname += "xxx"
$A = #($hashtable1; $hashtable2; $hashtable3)
I need to iterate though the array and I need to find out duplicates based on hashtable[].name
Then I need to group those hashtable.surname to hashtable[].surname so that the result will be an array of hashtables that will group all for name all the surnames:
$hashtable1.name = "aaa"
$hashtable1.surname = ("bbb","ccc")
$hashtable3.name = "bbb"
$hashtable3.surname = ("xxx")
I was looking into iterating to empty array
+
I have found this link:
powershell compare 2 arrays output if match
but I am not sure on how to reach into the elements of the hashtable.
My options:
I was wondering if -contain can do it.
I have read about compare-object but I am not sure it can be done like that.
(It looks a bit scary in the moment)
I am on PS5.
Thanks for your help,
Aster
You can group your array items by the names using a scriptblock like so.
Once grouped, you can easily build your output to do what you seek.
#In PS 7.0+ you can use Name directly but earlier version requires the use of the scriptblock when dealing with arrays of hashtables.
$Output = $A | Group-Object -Property {$_.Name} | % {
[PSCustomObject]#{
Name = $_.Name
Surname = $_.Group.Surname | Sort-Object -Unique
}
}
Here is the output variable content.
Name Surname
---- -------
aaa {bbb, ccc}
bbb xxx
Note
Improvements have been made in PS 7.0 that allows you to use simply the property name (eg: Name) in Group-Object for arrays of hashtables, just like you would do for any other arrays type. For earlier version though, these particular arrays must be accessed by passing the property in a scriptblock, like so: {$_.Name}
References
MSDN - Group_Object
SS64 - Group Object
Dr Scripto - Use a Script block to create custom groupings in PowerShell

Remove Empty Object in JSON Array using powershell

I have this JSON and I want to remove the empty object inside value array using powershell. Anyone can help?
{ "value" : [{},{},{"name": "a"}] }
# Parse the JSON into an object graph.
$obj = '{ "value" : [{},{},{"name": "a"}] }' | ConvertFrom-Json
# Filter out the empty objects, by counting the number of properties
# via the .psobject.Properties collection, available on any object in PowerShell.
# Note the need for #(...) to ensure that the result of the filtering remains
# an array.
$obj.Value = #($obj.Value | Where-Object { $_.psobject.Properties.Count -gt 0 })
# Reconvert to JSON.
$obj | ConvertTo-Json -Compress
The above yields:
{"value":[{"name":"a"}]}
See also:
ConvertFrom-Json
ConvertTo-Json
Where-Object

Resources