Sort Hashtable with Arrays as values - arrays

Description: I'm building a PowerShell-script that searches for files, then gives them unique names, copies them and then verifies them via hash-calculation - I chose to split the script in functions for each step, so it's easier to maintain the whole thing.
To get all values from one function to the other, I chose to use [hashtable]$FooBar - inside $FooBar, there are multiple arrays, such as FullName or OutputPath (which may change per file as they will be copied to subfolders named yyyy-mm-dd). All arrays are correlating with each other (meaning that index 1 contains all values of the first file, index 2 the values for the second file,...) and this works fine as of now.
A short simplified visualisation:
$FooBar = #{}
$FooBar.FullName = #()
$FooBar.Size = #()
$FooBar.Ext = #()
Get-ChildItem | ForEach-Object {
$FooBar.FullName += $_.FullName
$FooBar.Size += $_.Length
$FooBar.Ext += $_.Extension
}
However, I now need to sort them all by one value-set of one of the arrays, e.g. the size. Or, visualised again:
# From:
$FooBar
Name Value
---- -----
fullname {D:\AAA.XYZ, D:\BBB.ZYX, D:\CCC.YZX}
size {222, 111, 555}
extension {.XYZ, .ZYX, .YZX}
# To:
$FooBar = $FooBar | Sort-Object -Property Size -Descending
$FooBar
Name Value
---- -----
fullname {D:\CCC.YZX, D:\AAA.XYZ, D:\BBB.ZYX}
size {555, 222, 111}
extension {.YZX, .XYZ, .ZYX}
I tried $FooBar.GetEnumerator() | Sort-Object -Property Size, but this does not change anything. Google turned up suggestions on how to sort an array of hashtables, but in my case, it's the other way round, and I can't get my head around this because I don't even understand why this is a problem in the first place.
So my question is: is there any way to sort all arrays inside the hashtable by the value-set of one of the arrays? I can't get my head around this.
Disclaimer: I'm a PowerShell-autodidact with no reasonable background in scripting/programming, so it might well be that my "include everything in one hashtable"-solution isn't going to work at all or might be extremely inefficient - if so, please tell me.

The easiest way to go about what I believe you are trying to do is Select-Object
$fooBar = Get-ChildItem | Select-Object FullName, Size, Extension
This will create an array of new objects that only have the desired properties. The reason this works and your method doesn't is because Sort-Object works on properties and the property you are specifying is behind a few layers.
If you need more flexibility than just exact properties, you can create your own like this
$fooBar = Get-ChildItem | Select-Object #{Name = 'SizeMB'; Expression = {$_.Size / 1MB}}
Or manually create new properties with the [PSCustomObject] type accelerator:
$fooBar = Get-ChildItem | ForEach-Object {
[PSCustomObject]#{
FullName = $_.FullName
Extension = $_.Extension
Size = $_.Size
}
}
Update
If you need to add additional properties to the object after it's initially created you have a few options.
Add-Member
The most common method by far is by using the Add-Member cmdlet.
$object | Add-Member -MemberType NoteProperty -Name NewProperty -Value 'MyValue'
$object
Something important to keep in mind is that by default this cmdlet does not return anything. So if you place the above statement at the end of a function and do not separately return the object, your function won't return anything. Make sure you either use the -PassThru parameter (this is also useful for chaining Add-Member commands) or call the variable afterwards (like the example above)
Select-Object
You can select all previous properties when using calculated properties to add members. Keep in mind, because of how Select-Object works, all methods from the source object will not be carried over.
$fooBar | Select-Object *, #{Name = 'NewProperty'; Expression = {'MyValue'}}
psobject.Properties
This one is my personal favorite, but it's restricted to later versions of PowerShell and I haven't actually seen it used by anyone else yet.
$fooBar.psobject.Properties.Add([psnoteproperty]::new('NewProperty', 'MyValue'))
$fooBar
Each member type has it's own constructor. You can also add methods to $fooBar.psobject.Methods or either type to $fooBar.psobject.Members. I like this method because it feels more explicit, and something about adding members with members feels right.
Summary
The method you choose is mostly preference. I would recommend Add-Member if possible because it's the most used, therefore has better readability and more people who can answer questions about it.
I would also like to mention that it's usually best to avoid adding additional members if at all possible. A function's return value should ideally have a reliable form. If someone is using your function and they have to guess when a property or method will exist on your object it becomes very difficult to debug. Obviously this isn't a hard and fast rule, but if you need to add a member you should at least consider if it would be better to refactor instead.

For all practical purposes I'd strongly suggest you just store the objects you need in a single array, sort that once and then reference the individual properties of each object when needed:
$FooBar = Get-ChildItem |Sort-Object -Property Length
# Need the Extension property of the object at index 4?
$FooBar[4].Extension
To answer your actual question:
Array.Sort() has an overload that takes keys and values arrays separately. You could make a copy of the array you want to sort on for each other property you want to sort:
# Create hashtable of correlated arrays
$FooBar = #{}
$FooBar.FullName = #()
$FooBar.Size = #()
$FooBar.Ext = #()
# Types cast explicitly to avoid Array.Sort() calling .CompareTo() on the boxing object
Get-ChildItem | ForEach-Object {
$FooBar.FullName += [string]$_.FullName
$FooBar.Size += [int]$_.Length
$FooBar.Ext += [string]$_.Extension
}
# Define name of reference array property
$SortKey = 'Size'
# Sort all arrays except for the reference array
$FooBar.Keys |Where-Object {$_ -ne $SortKey} |ForEach-Object {
# Copy reference values to new array
$Keys = $FooBar[$SortKey].Clone()
# Sort values in target array based on reference values
[array]::Sort($Keys,$FooBar[$_])
}
# Finally sort the reference array
[array]::Sort($FooBar[$SortOn])
The above only works as long as the reference array is made up of value types

PowerShell makes working with objects ridiculously easy.
Try:
$FooBar = Get-Childitem
$FooBar | Get-Member
This will tell you that $Foobar actually contains objects of FileInfo and DirectoryInfo type, and show you the Properties available.
$FooBarSortedBySizeDesc = $FooBar | Sort-Object Length -Descending
$FooBarFullNamesOnly = $FooBar.FullName

Related

Why is my ArrayList throwing a "PSCustomObject doesn't contain a method for 'foreach'" error when the ArrayList contains <= 1 item, & only in PS 5?

I have the following script I wrote using PowerShell 5 that utilizes the Active Directory and Join-Object PowerShell modules to get a list of all AD Groups and their users (along with some additional properties per user like their manager and title):
$ADGroupsList = #(Get-ADGroup -Filter * -Properties * | Select-Object DistinguishedName,CN,GroupCategory,Description | Sort-Object CN)
#I'm using an ArrayList here so that later on I can use the .Add() method to avoid costly += operations.
$ADUsersList = New-Object -TypeName "System.Collections.ArrayList"
$ADUsersList = [System.Collections.ArrayList]#()
$Record = [ordered] #{
"Group Name" = ""
"Employee Name" = ""
"Title"= ""
"Manager" = ""
}
foreach ($Group in $ADGroupsList) {
$ArrayofMembers = #(Get-ADGroupMember -Identity $Group.DistinguishedName | Where-Object { $_.objectClass -eq "user" })
#Loop through each member in the list of members from above
foreach ($Member in $ArrayofMembers) {
#Get detailed user info about the current user like title and manager that aren't available from Get-ADGroupMember
$User = #(Get-ADUser -Identity $Member -Properties name,title,manager | Select-Object Name, Title, #{Label="Manager";Expression={(Get-ADUser (Get-ADUser $Member -Properties Manager).Manager).Name}})
#Specifies what values to apply to each property of the $Record object
$Record."Group Name" = $Group.CN
$Record."Employee Name" = $Member.Name
$Record."Title" = $User.Title
$Record."Manager" = $User.Manager
#Put all the stored information above in a 'copy' record
$objRecord = New-Object PSObject -property $Record
#Add that copy to the existing data in the ADUsersList object
[void]$ADUsersList.Add($objRecord)
}
#Using Join-Object here to enable me to use SQL-like JOINs
Join-Object -Left $ADUsersList -Right $ADGroupsList -LeftJoinProperty "Group Name" -RightJoinProperty "CN" -Type AllInLeft -LeftMultiMode DuplicateLines -RightMultiMode DuplicateLines -ExcludeRightProperties DistinguishedName | Export-Csv ("C:\ADReports\" + $Group.CN + " Report.csv") -NoTypeInformation
$ADUsersList.Clear()
}
Here's the output I expect (columns may be out of order, but column ordering isn't important):
My code works great for most groups, but for groups that have only one member (or none), I get an error:
Join-Object : Method invocation failed because [System.Management.Automation.PSCustomObject] does not contain a method named 'ForEach'.
At C:\GetADGroups&Users.ps1:54 char:5
+ Join-Object -Left $ADUsersList -Right $ADGroupsList -LeftJoinProp ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (ForEach:String) [Join-Object], RuntimeException
+ FullyQualifiedErrorId : MethodNotFound,Join-Object
At first, I thought it was because I read arrays/arraylists with one entry get turned into scalars. But a knee-jerk wrapping of every object I can think of in #() didn't resolve the issue. In fact, if I wrap the $objRecord assignment (New-Object PSObject -property $Record) in #() to convert it to an array, it writes the Member Properties of $ADUsersList to the Join-Object line instead of the contents of $ADUsersList, resulting in this:
Is there somewhere I've missed an array/arraylist getting converted to a scalar? Why is the code above throwing an error for groups with <= 1 entries?
Compounding my curiosity, PowerShell 7 (possibly 6, too) doesn't seem to care about this issue; it doesn't throw the error at all (instead it just outputs the appropriate single-value/blank CSV). Normally I'd just wipe my hands and say PS 7 is required, but I'd like to get this working in PowerShell 5, or at least understand what is causing the issue.
Googling led me to several related articles & questions, including:
Method Invocation .Foreach failed, System.Object doesn't contain a method named 'foreach' this one's specific to PowerShell v2 (I'm running v5)
Method invocation failed because [System.Management.Automation.PSObject] doesn't contain a method named 'op_Addition' this one seems only tangentially related. Incidentally it's where I read that arrays with one item output as scalars, as I mentioned earlier.
It does appear that scalars lack the .ForEach() & .Where() methods in 5.1. The additional of the methods is probably just an enhancement newer version, certainly 7 not sure about 6. I'm sure that's documented somewhere.
I can't really test your code but it doesn't look like there's anywhere that could be flipping to a scalar. To help guarantee ArrayList collections through out you can type constrain the variables like [Collections.ArrayList]$Var = #() This may end up being more practical than hunting for an implementing #() throughout.
Something that stands out is the error seems to come from Join-Object I only found a single invocation of .ForEach() on line 820 of Join-Object.ps1 My guess is it's this line or similar elsewhere in the module combined with the 5.1 runtime environment.
If you can manually modify that to a traditional | ForEach-Object {...} might be telling. And/or you can wrap $result like #($Result) right before the .ForEach() is invoked.
Really interested to see what you come up with. I see you've already posted an issue with the author. Please post back if you get a reply. Thanks.

Find ArrayList members and (Get-Process).id vs (1,2,3).ToString()

Let us look at the code below:
$r = New-Object System.Random
$r.GetType()
$r | Get-Member
I think I understand how it works. However, when I replace Random object with ArrayList:
$a = New-Object System.Collections.ArrayList
$a.GetType()
$a | Get-Member
[Question 1] Get-Member call produces an error: "You must specify an object for the Get-Member cmdlet". What? $a is not an object? "$a.gettype()" says it is.
[Question 2] I understand ArrayList can hold other objects, but how do I get members for ArrayList itself, such as add(), clear(), etc which I found from the documentation? ISE also knows the members.
Let us look at the following 2 similar constructs:
$p = Get-Process
$p.GetType()
$p | Get-Member
$p.id[2]
$L = #(3,1,4,1,5,9,2,6,5,3)
$L.GetType()
$L | Get-Member
$L.ToString()[2]
[Question 3] $p is an array [of process objects], however, Get-Member does not show members of the array itself, but the members of the objects the array hold, does this make sense?
[Question 4] The array $p does not have a member "id", but $p.id operates on each element of the array, generating a list of #($p[0].id, $p[1].id, ...). Let us accept this is how it works in Powweshell, then the same should apply to to array $L, however, $L.ToString() results the literal string "System.Object[]", those exact 15 chars! And $L.ToString()[2] is "s"! Why there is no consistency?
[Edit] Question 4 used a bad example because ToString() is a method for the array as well as the number element. ToString() method on an array is to return the object type "System.Object[]". A better example is:
#([byte]67, [byte]97, [byte]116).tochar($null)
C
a
t
which shows that operation on a list, when not a list operation, operates on the element (unrolls) and then re-rolls back to a list. It seems to be Powershell is pragmatic, but at least consistent.
As Mr Lee_Dailey said, $listofsomethings | get-member will apply to the actual objects in the variable. If they are all the same, powershell can figure this out and will only output the information once. However, look at this example.
$array = [int]2,[string]"doug",[float]19.83
$array | Get-Member
You'll see it gives you the details for 3 distinct types, as there are three different types in this array. On the other hand, if you do
Get-Member -InputObject $array
You will see information about the array container itself. Additionally, the following command also looks at the container as well.
$array.GetType().BaseType
To see the type of each member in the container's collection (not using the powershell auto unrolling of $array | gm) you'd have to loop over them.
$array.ForEach{$_.gettype()}
$array.ForEach{$_.gettype().basetype}
Now knowing this, it should be clear why $p.id works.. Powershell is "helping" you by effectively running this under the hood
$p | foreach id
Or written the same way as the $array foreach
$p.foreach{$_.id}
$arraylists are like arrays in that they can contain different type of objects.
$arraylist = New-Object System.Collections.ArrayList
$arraylist.Add([int]2)
$arraylist.Add([string]"Stack Overflow")
$arraylist.Add([double]32.80)
You'll notice that each time you add an item to the arraylist, powershell outputs the index of where that item was added. It's commonplace to remove this $null = $arraylist.add(...). This annoyance alone should be enough to discourage it's use, but the fact that it's deprecated in favor of the System.Collections.Generic.List lists should already lead you away from it. You specify the type of item the list is for such as [string],[int],[object],etc. It also doesn't have the side effect of spitting out index numbers when adding items.
You can create a list like this
$list = new-object System.Collections.Generic.List[string]
or
$list = [System.Collections.Generic.List[string]]::new()
And you can add/remove the same way you do with an arraylist.

Count occurrences of something in an array inside a foreach loop

I have a product CSV file that I have imported into $products
If something occurs more than once with the same name I want to populate the ParentSKU field, otherwise leave it blank.
Excuse the pseudocode but I'm imagining something like this:
foreach ($item in $products) {
if ($item.name.count -gt 1) {
$item.ParentSKU = $item.name }
else { } # do nothing
}
$item.name.count isn't correct but I hope my thinking is on the right track?
Many thanks for any advice
Powershell Object lists aren't smart enough to know that there's multiple of any one item, so you're going to have to iterate through (manually or otherwise) to find whether there's multiples here.
Since you're going to be making modifications to any duplicates, it may make sense to loop through and find duplicates manually, but it doesn't really follow the "powershell" philosophy / approach.
If you want to use powershell's built-in & powerful piping features, you might try a solution like this, which would grab all the PSObjects with duplicates using Where-Object, then sets the values for all those PSObjects.
$products |
Group-Object -Property Name |
Where-Object -FilterScript {
$_.Count -gt 1
} |
Select-Object -ExpandProperty Group |
Foreach-Object { $_.ParentSKU = $_.Name }
Since everything is passed by reference, your $products object will have the modified values!

powershell arrays add-member - looking for elegant code

I have a pretty basic PowerShell array: $TestArray with 2 text columns: Price and TimeStamp (that's the way I get them, nothing to be done about this):
Price TimeStamp
----- ----------------
0.0567 1542056680.72746
0.0567 1542056650.34414
0.0555 1542056197.46668
0.0551 1542056167.28967
I would like, in a single PowerShell line to add a rounded Time2 value
$Time2 = [math]::Round($TestArray.TimeStamp)
The code I am thinking of is:
$TestArray | Add-Member -Name Time2 -MemberType NoteProperty -Value { [math]::Round($Table.TimeStamp) }
Of course, I can do a ForEach loop; it would take care of it easily, but I would like to achieve this in this single line of code.
Any idea ?
Cheers,
Philippe
Mathias R. Jessen's answer directly solves your problem, by creating a script property (type ScriptProperty) that dynamically calculates its value from the enclosing object's .TimeStamp property.
The advantage of this approach is that even later changes to .TimeStamp will correctly be reflected in .Time2, albeit at the cost of having to calculate the value on every access.
Manuel Batsching's answer offers a Select-Object-based alternative that creates static note properties (type NoteProperty), as you've originally attempted yourself.
The advantage of this approach is that you incur the cost of calculation only once, but later changes to .TimeStamp won't be reflected in .Time2.
To offer faster PSv5+ alternatives (a single method call each, spread across multiple lines for readability):
# ScriptProperty - dynamic
$TestArray.ForEach({
$_.psobject.properties.Add(
[psscriptproperty]::new('Time2', { [math]::Round($this.TimeStamp) })
)
})
# NoteProperty - static
$TestArray.ForEach({
$_.psobject.properties.Add(
[psnoteproperty]::new('Time2', [math]::Round($_.TimeStamp))
)
})
The above solutions use the PSv4+ .ForEach() collection method and the PSv5+ static ::new() type method for calling constructors.
Finally, re one-liner:
The following foreach-loop based note-property solution would have solved your problem too, and would have been faster; while it is spread across multiple lines for readability here, it also works as a one-liner:
foreach ($el in $TestArray) {
Add-Member -InputObject $el -Name Time2 -MemberType NoteProperty `
-Value ([math]::Round($el.TimeStamp))
}
Generally, while the pipeline often enables more elegant, single-command solution, that - unfortunately - comes at the expense of performance.
Alternatively you can achieve the same with Select-Object and a custom property:
$TestArray | Select-Object *,#{ n='Time2';e={ [math]::Round($_.TimeStamp) }}
Change the member type to ScriptProperty and refer to the individual array item as $this:
$TestArray |Add-Member Time2 -Value { [math]::Round($this.Timestamp) } -MemberType ScriptProperty
Worth noting that in this example, the pipeline itself acts as a foreach loop, unravelling the array and binding each individual item to Add-Member

Powershell: How to dynamically build array or objects?

I would like to collect some information about hosts in the domain, so I am trying to write something like this:
# declare array for storing final data
$servers_list = #()
#start with a list of servers and go through collecting the info
$servers | ForEach-Object {
$sys = Get-WmiObject Win32_computersystem -ComputerName $_
# create new custom object to store information
$server_obj = New-Object –TypeName PSObject
$server_obj | Add-Member –MemberType NoteProperty –Name Domain –Value $sys.Domain
# .... add all other relevant info in the same manner
# Add server object to the array
$servers_list += $server_obj
}
The problem with this code is that I pass a reference to the object into array and not the actual object. So by the time my loop is finished I end up with an array that contains rows that look all the same :(
Any idea how to pass actual object into array and not just a reference to it?
Another thought is to dynamically declare new object instead of using $server_obj variable every time but I am not sure how to do this either...
Thanks!!!
You can build an array of objects and keep dynamically adding information to them like this:
#This will be your array of objects
#In which we will keep adding objects from each computer
$Result = #()
#start with a list of servers and go through collecting the info
$servers | ForEach-Object {
$sys = Get-WmiObject Win32_computersystem -ComputerName $_
# create new custom object to keep adding store information to it
$Result += New-Object –TypeName PSObject -Property #{Domain = $sys.Domain;
Name = $sys.Name;
SystemType = $sys.SystemType
}
}
# Get back the objects
$Result
Where Domain,Name and SystemType are the properties that you want to associate with the objects.
It sounds like it is passing a reference, but I don't think it's the object that's being passed as a reference, but the property values. There are discrete objects, but they all have the same reference for their property values, so they all look the same. If that's the case,
$server_obj | Add-Member –MemberType NoteProperty –Name Domain –Value "$($sys.Domain)"
should make the value a string, which is a value type and won't change.
You're making this a little harder than it should be. Pass the server names from a query, csv or list then iterate over them. Select what you want from the result.
$info = "server1", "server2" | ForEach-Object{Get-WmiObject -Class win32_computersystem -ComputerName $_ } | Select-Object Domain, Name, Systemtype
$info[1].Domain will output domain.com

Resources