Format an array of hashtables as table with Powershell - arrays

I wish to display an array of hastables as a Table. I found several threads dealing with this issue, but so far no solution has worked for me (see here and here).
My array consists of 18 hashtables in this form:
Array = #(
#{Company=1}
#{Company=2}
#{Company=3}
#{Contact=X}
#{Contact=Y}
#{Contact=Y}
#{Country=A}
#{Country=B}
#{Country=C}
)
I would like to get the following output:
Company Contact Country
------------------------------
1 X A
2 Y B
3 Z C
I've tried the following:
$Array | ForEach-Object { New-Object -Type PSObject -Property $_ } | Format-Table
That displays the following:
Company
----------
1
2
3
A Format-List works better; I then get this:
Company: 1 2 3
Contact: X Y Z
Country: A B C
Is there any way to accomplish my desired output?

You can do the following if you want to work with CSV data and custom objects:
$Array = #(
#{Company=1}
#{Company=2}
#{Company=3}
#{Contact='X'}
#{Contact='Y'}
#{Contact='Z'}
#{Country='A'}
#{Country='B'}
#{Country='C'}
)
$keys = $Array.Keys | Get-Unique
$data = for ($i = 0; $i -lt $Array.($Keys[0]).Count; $i++) {
($Keys | Foreach-Object { $Array.$_[$i] }) -join ','
}
($Keys -join ','),$data | ConvertFrom-Csv | Format-Table
Explanation:
Since $Array is an array of hash tables, then the .Keys property will return all the keys of those hash tables. Since we only care about unique keys when building our object, Get-Unique is used to remove duplicates.
$Array.($Keys[0]).Count counts the number of items in a group. $Keys[0] here will be Company. So it returns the number of hash tables (3) that contain the key Company.
$Array.Company for example returns all values from the hash tables that contain the key Company. The Foreach-Object loops through each unique key's value at a particular index ($i). Once each key-value is read at a particular index, the values are joined by a comma.
When the loop completes, ($Keys -join ','),$data outputs the data in CSV format, which is piped into ConvertFrom-Csv to create a custom object.
Note: that if your data contains commas, you may want to consider the alternative method below.
Alernatively, you can work with hash tables and custom objects with the following:
$Array = #(
#{Company=1}
#{Company=2}
#{Company=3}
#{Contact='X'}
#{Contact='Y'}
#{Contact='Z'}
#{Country='A'}
#{Country='B'}
#{Country='C'}
)
$keys = $Array.Keys | Get-Unique
$data = for ($i = 0; $i -lt $Array.($Keys[0]).Count; $i++) {
$hash = [ordered]#{}
$Keys | Foreach-Object { $hash.Add($_,$Array.$_[$i]) }
[pscustomobject]$hash
}
$data | Format-Table

Related

Query PSCustomObject Array for row with largest value

I'm trying to find the row with an attribute that is larger than the other row's attributes. Example:
$Array
Name Value
---- ----
test1 105
test2 101
test3 512 <--- Selects this row as it is the largest value
Here is my attempt to '1 line' this but It doesn't work.
$Array | % { If($_.value -gt $Array[0..($Array.Count)].value){write-host "$_.name is the largest row"}}
Currently it outputs nothing.
Desired Output:
"test1 is the largest row"
I'm having trouble visualizing how to do this efficiently with out some serious spaghetti code.
You could take advantage of Sort-Object to rank them by the property "Value" like this
$array = #(
[PSCustomObject]#{Name='test1';Value=105}
[PSCustomObject]#{Name='test2';Value=101}
[PSCustomObject]#{Name='test3';Value=512}
)
$array | Sort-Object -Property value -Descending | Select-Object -First 1
Output
Name Value
---- -----
test3 512
To incorporate your write host you can just run the one you select through a foreach.
$array | Sort-Object -Property value -Descending |
Select-Object -First 1 | Foreach-Object {Write-host $_.name,"has the highest value"}
test3 has the highest value
Or capture to a variable
$Largest = $array | Sort-Object -Property value -Descending | Select-Object -First 1
Write-host $Largest.name,"has the highest value"
test3 has the highest value
PowerShell has many built in features to make tasks like this easier.
If this is really an array of PSCustomObjects you can do something like:
$Array =
#(
[PSCustomObject]#{ Name = 'test1'; Value = 105 }
[PSCustomObject]#{ Name = 'test2'; Value = 101 }
[PSCustomObject]#{ Name = 'test3'; Value = 512 }
)
$Largest = ($Array | Sort-Object Value)[-1].Name
Write-host $Largest,"has the highest value"
This will sort your array according to the Value property. Then reference the last element using the [-1] syntax, then return the name property of that object.
Or if you're a purist you can assign the variable like:
$Largest = $Array | Sort-Object Value | Select-Object -Last 1 -ExpandProperty Name
If you want the whole object just remove .Name & -ExpandProperty Name respectively.
Update:
As noted PowerShell has some great tools to help with common tasks like sorting & selecting data. However, that doesn't mean there's never a need for looping constructs. So, I wanted to make a couple of points about the OP's own answer.
First, if you do need to reference array elements by index use a traditional For loop, which might look something like:
For( $i = 0; $i -lt $Array.Count; ++$i )
{
If( $array[$i].Value -gt $LargestValue )
{
$LargestName = $array[$i].Name
$LargestValue = $array[$i].Value
}
}
$i is commonly used as an iteration variable, and within the script block is used as the array index.
Second, even the traditional loop is unnecessary in this case. You can stick with the ForEach loop and track the largest value as and when it's encountered. That might look something like:
ForEach( $Row in $array )
{
If( $Row.Value -gt $LargestValue )
{
$LargestName = $Row.Name
$LargestValue = $Row.Value
}
}
Strictly speaking you don't need to assign the variables beforehand, though it may be a good practice to precede either of these with:
$LargestName = ""
$LargestValue = 0
In these examples you'd have to follow with a slightly modified Write-Host command
Write-host $LargestName,"has the highest value"
Note: Borrowed some of the test code from Doug Maurer's Fine Answer. Considering our answers were similar, this was just to make my examples more clear to the question and easier to test.
Figured it out, hopefully this isn't awful:
$Count = 1
$CurrentLargest = 0
Foreach($Row in $Array) {
# Compare This iteration vs the next to find the largest
If($Row.value -gt $Array.Value[$Count]){$CurrentLargest = $Row}
Else {$CurrentLargest = $Array[$Count]}
# Replace the existing largest value with the new one if it is larger than it.
If($CurrentLargest.Value -gt $Largest.Value){ $Largest = $CurrentLargest }
$Count += 1
}
Write-host $Largest.name,"has the highest value"
Edit: its awful, look at the other answers for a better way.

Appending objects to arrays in Powershell

I have the following code:
$DataType = "X,Y,Z"
$Data = "1,2,3"
$Table = #()
for ($i = 0; $i -le ($DataType.Count-1); $i++)
{
$Properties = #{$DataType[$i]=$Data[$i]}
$Object = New-Object -TypeName PSCustomObject -Property $Properties
$Table += $Object
}
$Table | Format-Table -AutoSize
I get this output:
X
-
1
What I would like to get is:
X Y Z
- - -
1 2 3
Thanks for your help!
Cutting a long story short:
$DataType, $Data | ConvertFrom-Csv
X Y Z
- - -
1 2 3
Ok, it needs a little explanation:
PowerShell will automatically unroll the array of strings ($DataType, $Data) and supply it as individual line items to the pipeline. The ConvertFrom-Csv cmdlet supports supplying the input table through the pipeline as separate lines (strings).
You can do the following instead:
$DataType = "X","Y","Z"
$Data = 1,2,3
$hash = [ordered]#{}
for ($i = 0; $i -lt $DataType.Count; $i++) {
$hash.Add($DataType[$i],$Data[$i])
}
$table = [pscustomobject]$hash
Explanation:
The code creates two collections, $DataType and $Data, of three items. $hash is an ordered hash table. [ordered] is used to preserve the order at which key-value pairs are added to the hash table. Since $hash is the object type hashtable, it contains the .Add(key,value) method for adding key-value pairs.
Since the [pscustomobject] type accelerator can be cast on a hash table, we can simply use the syntax [pscustomobject]$hash to create a new object.
If we consider your attempt, your variables are actually single strings rather than collections. Surrounding a value with quotes causes PowerShell to expand the inner contents as a string. When you index a string rather than a collection, you index the characters in the string rather than the entire item. You need to quote the individual elements between the commas so that the , acts as a separator rather than part of the string. You can see this behavior in the code below:
# DataType as a string
$DataType = "X,Y,Z"
$DataType[1]
,
# DataType as an array or collection
$DataType = "X","Y","Z"
$DataType[1]
Y
If you receive your data from another output in the current format, you can manipulate using $DataType = $DataType.Split(',') in order to create a collection. Alternatively you can treat the data as comma-separated and use the Import-Csv or ConvertFrom-Csv commands as in iRon's answer provided you order your strings properly.
Inside of your loop, you are adding three new objects to your collection $table rather than creating one object with three properties. $table += $Object creates an array called $table that appends a new item to the previous list from $table. If this was your original intention, you can view your collection by running $table | Format-List once you fix your $DataType and $Data variables.
When a collection is enumerated, the default table view displays the properties of the first object in a collection. Any succeeding objects will only display values for the first object's matching properties. So if object1 has properties X and Y and object2 has properties Y and Z, the console will only display values for properties X and Y for both objects. Format-List overrides this view and displays all properties of all objects. See below for an example of this behavior:
$obj1
X Y
- -
1 2
$obj2
Y Z
- -
3 4
$array = $obj1,$obj2
# Table View
$array
X Y
- -
1 2
3
# List View
$array | Format-List
X : 1
Y : 2
Y : 3
Z : 4
It seems that you want to create a single object with a property for each value in the arrays $DataType/$Data, but the problems are...
Neither $DataType nor $Data are arrays.
By creating your object inside the for loop you will create one object per iteration.
Since $DataType is a scalar variable $DataType.Count returns 1. Ordinarily, testing for $DataType.Count-1 would mean the loop never gets entered, but by the grace of using -le (so 0 -le 0 returns $true) instead of -lt, it does for exactly one iteration. Thus, you do get your single result object, but with only the first property created.
To fix this, let's create $DataType and $Data as arrays, as well as creating one set of properties before the loop to be used to create one result object after the loop...
...
$DataType = "X,Y,Z" -split ','
$Data = "1,2,3" -split ','
$Properties = #{}
for ($i = 0; $i -lt $DataType.Count; $i++)
{
$Properties[$DataType[$i]] = $Data[$i]
}
New-Object -TypeName PSCustomObject -Property $Properties | Format-Table -AutoSize
You'll also notice that $i -le ($DataType.Count-1) has been simplified to $i -lt $DataType.Count. On my system the above code outputs...
Y Z X
- - -
2 3 1
The properties are correct, but the order is not what you wanted. This is because Hashtable instances, such as $Properties, have no ordering among their keys. To ensure that the properties are in the order you specified in the question, on PowerShell 3.0 and above you can use this to preserve insertion order...
$Properties = [Ordered] #{}
What if you initialized $Table as an appendable like so:
$Table = New-Object System.Collections.ArrayList
for ($i = 0; $i -le ($DataType.Count-1); $i++)
{
$Properties = #{$DataType[$i]=$Data[$i]}
$Object = New-Object -TypeName PSCustomObject -Property $Properties
$Table.Add ( $Object )
}
Reformat your logic as needed.
One solution to this problem (if the inputs were two separate arrays):
$DataType = #( 'X','Y','Z' )
$Data = #( '1','2','3' )
$Table = New-Object psobject
for ($i = 0; $i -le ( $DataType.Count-1 ); $i++)
{
$Table | Add-Member -Name "$( $DataType[$i] )" -Value ( $Data[$i] ) -MemberType NoteProperty
}
$Table

Efficient way to remove duplicates from large 2D arrays in PowerShell

I have a large set of data roughly 10 million items that I need to process efficiently and quickly removing duplicate items based on two of the six column headers.
I have tried grouping and sorting items but it's horrendously slow.
$p1 = $test | Group-Object -Property ComputerSeriaID,ComputerID
$p2 = foreach ($object in $p1.group) {
$object | Sort-Object -Property FirstObserved | Select-Object -First 1
}
The goal would be to remove duplicates by assessing two columns while maintaining the oldest record based on first observed.
The data looks something like this:
LastObserved : 2019-06-05T15:40:37
FirstObserved : 2019-06-03T20:29:01
ComputerName : 1
ComputerID : 2
Virtual : 3
ComputerSerialID : 4
LastObserved : 2019-06-05T15:40:37
FirstObserved : 2019-06-03T20:29:01
ComputerName : 5
ComputerID : 6
Virtual : 7
ComputerSerialID : 8
LastObserved : 2019-06-05T15:40:37
FirstObserved : 2019-06-03T20:29:01
ComputerName : 9
ComputerID : 10
Virtual : 11
ComputerSerialID : 12
You might want to clean up your question a little bit, because it's a little bit hard to read, but I'll try to answer the best I can with what I can understand about what you're trying to do.
Unfortunately, with so much data there's no way to do this quickly. String Comparison and sorting are done by brute force; there is no way to reduce the complexity of comparing each character in one string against another any further than measuring them one at a time to see if they're the same.
(Honestly, if this were me, I'd just use export-csv $object and perform this operation in excel. The time tradeoff to scripting something like this only once just wouldn't be worth it.)
By "Items" I'm going to assume that you mean rows in your table, and that you're not trying to retrieve only the strings in the rows you're looking for. You've already got the basic idea of select-object down, you can do that for the whole table:
$outputFirstObserved = $inputData | Sort-Object -Property FirstObserved -Unique
$outputLastObserved = $inputData | Sort-Object -Property LastObserved -Unique
Now you have ~20 million rows in memory, but I guess that beats doing it by hand. All that's left is to join the two tables. You can download that Join-Object command from the powershell gallery with Install-Script -Name Join and use it in the way described. If you want to do this step yourself, the easiest way would be to squish the two tables together and sort them again:
$output = $outputFirstObserved + $outputLastObserved
$return = $output | Sort-Object | Get-Unique
Does this do it? It keeps the one it finds first.
$test | sort -u ComputerSeriaID, ComputerID
I created this function to de-duplicate my multi-dimensional arrays.
Basically, I concatenate the contents of the record, add this to a hash.
If the concatenate text already exists in the hash, don't add it to the array to be returned.
Function DeDupe_Array
{
param
(
$Data
)
$Return_Array = #()
$Check_Hash = #{}
Foreach($Line in $Data)
{
$Concatenated = ''
$Elements = ($Line | Get-Member -MemberType NoteProperty | % {"$($_.Name)"})
foreach($Element in $Elements)
{
$Concatenated += $line.$Element
}
If($Check_Hash.$Concatenated -ne 1)
{
$Check_Hash.add($Concatenated,1)
$Return_Array += $Line
}
}
return $Return_Array
}
Try the following script.
Should be as fast as possible due to avoiding any pipe'ing in PS.
$hashT = #{}
foreach ($item in $csvData) {
# Building hash table key
$key = '{0}###{1}' -f $item.ComputerSeriaID, $item.ComputerID
# if $key doesn't exist yet OR when $key exists and "FirstObserverd" is less than existing one in $hashT (only valid when date provided in sortable format / international format)
if ((! $hashT.ContainsKey($key)) -or ( $item.FirstObserved -lt $hashT[$key].FirstObserved )) {
$hashT[$key] = $item
}
}
$result = $hashT.Values

Export array with "sub-array"

I am currently trying to automate license counting in Office 365 across multiple partner tenants using PowerShell.
My current code (aquired from the internet) with some modifications gives me this output:
Column A Column B Column C
-------- -------- --------
CustA LicA,LicB 1,3
CustB LicA,LicB 7,3
CustC LicA 4
But the output I want from this code is:
Column A Column B Column C
-------- -------- --------
CustA LicA 1
LicB 3
CustB LicA 7
LicB 3
Here is my current code which is exported using Export-Csv -NoType:
$tenantID = (Get-MsolPartnerContract).tenantid
foreach($i in $tenantID){
$tenantName = Get-MsolPartnerInformation -TenantId $i
$tenantLicense = Get-MsolSubscription -TenantId $i
$properties = [ordered]#{
'Company' = ($tenantName.PartnerCompanyName -join ',')
'License' = ($tenantLicense.SkuPartNumber -join ',')
'LicenseCount' = ($tenantLicense.TotalLicenses -join ',')
}
$obj = New-Object -TypeName psobject -Property $properties
Write-Output $obj
}
I have tried this along with several other iterations of code which all fail catastophically:
$properties = [ordered]#{
'Company' = ($tenantName.PartnerCompanyName -join ','),
#{'License' = ($tenantLicense.SkuPartNumber -join ',')},
#{'LicenseCount' = ($tenantLicense.TotalLicenses -join',')}
}
I was thinking about making a "sub-array" $tenantLicense.SkuPartnumber and $tenantLicense.TotalLicenses, but I'm not quite sure how to approach this with appending it to the object or "main-array".
A second loop for each $tenantLIcense should do the trick for you. I don't have access to an environment like yours so I cannot test this.
$tenantID | ForEach-Object{
$tenantName = Get-MsolPartnerInformation -TenantId $_
$tenantLicense = Get-MsolSubscription -TenantId $_
# Make an object for each $tenantLicense
$tenantLicense | ForEach-Object{
$properties = [ordered]#{
'Company' = $tenantName.PartnerCompanyName
'License' = $_.SkuPartNumber
'LicenseCount' = $_.TotalLicenses
}
# Send the new object down the pipe.
New-Object -TypeName psobject -Property $properties
}
}
Since you have multiple $tenantLicenses that have the same company name lets just loop over those and use the same company name in the output. Assuming this worked it would not have the same output as you desired since there no logic to omit company in subsequent rows. I would argue that is it better this way since you can sort the data now with out loss of data / understanding.
Notice I change foreach() to ForEach-Object. This makes it simpler to send object down the pipe.
Without providing the code solution I can say that you need to build up the array.
In terms of programming, you will need to iterate the array ARRAY1 and populate another one ARRAY2with the extra rows. For example if columns A,B are simple value and C is an array of 3 items, then you would add 3 rows in the new table with A,B,C1, A,B,C2 and A,B,C3. On each iteration of the loop you need to calculate all the permutations, for example in your case the ones generated by columnB and columnC.
This should be also possible with pipelining using the ForEach-Object cmdlet but that is more difficult and as you mentioned your relatively new relationship with powershell I would not pursuit this path, unless of coarse you want to learn.

How to create and populate an array in Powershell based on a dynamic variable?

I've been struggling with this for a couple of days, and I'm not sure how to conquer it. I need to do the following:
Import a csv of users with the following values:
ID, Name, Region
Create an array based on the Region values that I can then use to populate with ID's and Names with that region, ie.
Array_SEA
AA_SCOM, Adam Andrews, SEA
Array_OAK
BB_SCOM, Bob Barker, OAK
Here's the code I've got right now:
$list2 = ipcsv .\TSE_Contact_List.csv | sort-object BU
$arraylist =#()
foreach ($vitem in $list2)
{
$arraylist += New-Object PsObject -Property #{'Array' = "Array_" + $vitem.bu}
}
foreach ($varray in $arraylist)
{
$arr = new-variable -Name $varray
$arr.value += $varray.array
$arr
}
This produces the following error for records with a duplicate regions:
New-Variable: A variable with name '#{Array=Array_SCA}' already exists.
I'm also getting the following when it tries to add values:
Property 'value' cannot be found on this object; make sure it exists and is settable.
I get that I'm not actually creating arrays in the second section, but I'm not sure how to pass the output of the variable to an array name without turning the variable declaration into the array name, if that makes sense.
I've tried the following with hash tables, and it gets closer:
$list2 = ipcsv .\TSE_Contact_List.csv | sort-object BU
$arraylist =#{}
foreach ($vitem in $list2){$arraylist[$vitem.bu] = #()}
foreach ($record in $list2)
{
$arraylist[$vitem.bu] += ($record.SCOMID,$record.Name,$record.BU)
Write-host "Array: "
$arraylist[$vitem.bu]
write-host ""
}
The output on this shows no errors, but it just keeps showing the added fields for all of the records for each iteration of the list, so I don't think that it's actually assigning each unique BU to the array name.
I like the hashtable-approach, but I would finetune it a little. Try:
$list2 = ipcsv .\TSE_Contact_List.csv | sort-object BU
$arraylist = #{}
foreach ($vitem in $list2){
if($arraylist.ContainsKey($vitem.BU)) {
#Array exists, add item
$arraylist[($vitem.BU)] += $vitem
} else {
#Array not found, creating it
$arraylist[($vitem.BU)] = #($vitem)
}
}
#TEST: List arrays and number of entries
$arraylist.GetEnumerator() | % {
"Array '$($_.Key)' has $($_.Value.Count) items"
}
You could also use Group-Object like:
$list2 = ipcsv .\TSE_Contact_List.csv | Group-Object BU
#TEST: List groups(regions) and number of entries
$list2 | % {
"Region '$($_.Name)' has $(#($_.Group).Count) items"
}

Resources