Selecting Distinct Items within Array using PowerShell and Linq - arrays

I have been banging my head on this problem for a few hours.
I have a multi-dimensional array and I need to select the unique items based on two "columns".
Is there an efficient .Net or otherwise way to do this and achieve the desired output?
The data looks something like:
ComputerName, IPAddress, MacAddress, FirstObserved
I would like to determine unique values based on MacAddress and ComputerName and keep the unique value based on the oldest FirstObserved date.
I have tried the PowerShell ways of doing this but it's horribly slow to say the least.
$data | Group-Object -Property ComputerName,MacAddress | ForEach-Object{$_.Group | Sort-Object -Property FirstObserved | Select-Object -First 1}
In a perfect world I would have a list of items no duplicates with the oldest entry based on FirstObserved date.

You can implement the grouping manually with a hashtable:
$FirstObserved = #{}
Import-Csv .\data.csv |ForEach-Object {
$key = $_.Computername,$_.MacAddress -join ','
$date = $_.FirstObserved -as [datetime]
# Check if we already have an entry for this Name + MAC combo
if($FirstObserved.Contains($key))
{
# Make sure the current date is older than what we have already
if($FirstObserved[$key] -gt $date)
{
$FirstObserved[$key] = $date
}
}
else
{
# First instance of the Name + MAC combo, save the date
$FirstObserved[$key] = $date
}
}
Now you can eaily look up the first date for a combination with:
$FirstObserved["Computer-01,11:22:33:44:55:66"]
If you want to export the list to another csv, all you need to do is turn the key-value pairs of the hashtable into objects:
$FirstObserved |Select-Object #{Name='Identity';Expression={$_.Key}},#{Name='FirstObserved';Expression={$_.Value}} |Export-Csv -Path output.csv

Related

How does one add output from a cmdlet to an array?

I am trying to determine if specific Windows hotfixes are installed on our Windows servers. I am quite new to PowerShell scripting and this is what I have so far:
$servers = Get-ADComputer -Filter {(OperatingSystem -like "Windows Server 2019*") -and (enabled -ne $false)} -Property *
$result = #()
ForEach ($item in $servers) {
$testhotfix = Get-HotFix -Id KB4534310,KB4534314,KB4534283,KB4534288,KB4534297,KB4534309,KB4534271,KB4534273 -ComputerName $item.Name | `
select $item.Name,$item.CanonicalName,$item.OperatingSystem
$result += $testhotfix
}
$result | Export-Csv -Path C:\Users\user1\Desktop\Servers.csv -NoTypeInformation
The CSV file that is created includes one line with the information I'm looking for, followed by several lines of commas, like so:
Script Output
"SERVER1","somedomain.com/Servers/Non-Prod/New_Server_Staging/SERVER1","Windows Server 2019 Standard"
,,
,,
,,
,,
,,
,,
,,
,,
,,
,,
,,
,,
,,
,,
,,
,,
,,
,,
,,
,,
,,
We have several servers with at least one of the hotfixes installed. How do I add each server to the $result array?
Thank you
Generally speaking:
select $item.Name,$item.CanonicalName,$item.OperatingSystem
should be:
select Name, CanonicalName, OperatingSystem
That is, you need to pass the property names (e.g., Name), not the current input object's property values (e.g., $item.Name) to select (the Select-Objectcmdlet).
The net effect is that Select-Object creates custom objects whose properties are (mistakenly) named for the property values and themselves have no value, given that the input objects have no such properties.
This explains the output you saw.
However, the bigger problem is that even that won't work, given that the property names relate to the $item object, not to the objects output by Get-HotFix, which are the ones select operates on.
As it turns out, what you really need is to use the Get-HotFix call as a conditional, so as to only write a CSV row for the computer at hand if at least one of the specified hotfixes is installed:
$hotfixIds = 'KB4534310', 'KB4534314', 'KB4534283', 'KB4534288', 'KB4534297', 'KB4534309', 'KB4534271', 'KB4534273'
if (0 -ne (Get-HotFix -ErrorAction SilentlyContinue -Id $hotfixIds -ComputerName $item.Name).Count) {
$result += $item | select Name, CanonicalName, OperatingSystem
}
Note:
Note how it is now $item (the computer at hand) that is piped to select, to ensure that its properties are extracted (in the form of a custom object with these properties).
You could omit 0 -eq altogether and rely on PowerShell's implicit to-Boolean conversion, where any nonzero number evaluates to $true (see the bottom section of this answer for a summary of all rules.
If instead you want to test for all of the specified hotfixes being installed, replace 0 -ne with $hotfixIds.Count -eq.
-ErrorAction SilentlyContinue silences the errors from computers where none of the specified hotfixes are installed; you could examine the automatic $Error collection afterwards, or use -ErrorVariable err to collect all command-specific errors in variable $err.
Also, your overall command can be greatly streamlined - see the bottom section.
A solution for a different scenario, that may be of interest as well:
If you wanted to combine properties from the Get-HotFix output objects with properties from the $item objects (representing the computer at hand):
The following command:
selects all properties from the Get-HotFix output objects (-Property *)
adds the properties of interest from the current $item, using calculated properties
# Additional 'KB...' values omitted for brevity.
Get-HotFix -Id KB4534310, KB4534314 -ComputerName $item.Name |
Select-Object -Exclude Name -Property *,
#{ n = 'Name'; e = { $item.Name } },
#{ n = 'CanonicalName'; e = { $item.CanonicalName } },
#{ n = 'OperatingSystem'; e = { $item.OperatingSystem } }
Note that -Exclude Name excludes the Name property from the input objects (Get-HotFix output objects that have such a property, but it is empty), so that Name can be added as a property containing the computer name.
As for what you tried:
Aside from the Select-Object property-name problem mentioned above, your major problem was that you expected a pipeline segment as a conditional, which is not how pipelines work:
Get-HotFix ... | select ...
The above simply sends Get-HotFix's output objects to select (Select-Object), which then unconditionally processes them (and, as stated, looks for properties with the given names on these objects).
Now, if Get-HotFix produced no output, then conditional logic applies implicitly: the select command would then simply not be invoked.
Conversely, if Get-HotFix produces multiple outputs, select would be invoked on each.
That is, if we had naively tried to correct your command from:
Get-HotFix ... | select ...
to:
Get-HotFix ... | ForEach-Object { $item | select ... }
you would have potentially created multiple output objects per computer, namely whenever a given computer happens to have more than one among the given hotfixes installed.
A streamlined version of your (corrected) command:
Your command can be streamlined to use a single pipeline only, without the need for aux. variables:
Get-ADComputer -Filter '(OperatingSystem -like "Windows Server 2019*") -and (enabled -ne $false)' -Property * |
ForEach-Object {
if (0 -ne (Get-HotFix -ErrorAction SilentlyContinue -ComputerName $item.Name -Id KB4534310,KB4534314,KB4534283,KB4534288,KB4534297,KB4534309,KB4534271,KB4534273).Count) {
$item | select Name, CanonicalName, OperatingSystem
}
} | Export-Csv -Path C:\Users\user1\Desktop\Servers.csv -NoTypeInformation
Note:
If you end a line with |, you do not need a trailing ` to signal line continuation.
PowerShell [Core] v7.0+ now also allows placing | at the start of the very next line.
A single-quoted string ('...') is used instead of a script block ({ ... }) to pass the -Filter argument, because tt's best to avoid the use of script blocks ({ ... }) as -Filter arguments.
The output custom object instances created with $item | select Name, CanonicalName, OperatingSystem are sent directly to the pipeline.
I would use a PSCustomObject.
$array = foreach($item in $obj)
{
[PSCustomObject]#{
Name = $item.Name
CanonicalName = $item.CanonicalName
OS = $item.OperatingSystem
}
}

Powershell - Parse duplicates in a list

I'm working on an issue with SCCM delivered App-V connection groups. Occasionally it delivers multiple (duplicate) connection groups to the client and the apps don't function correctly.
I'm running get-appvclientconnectiongroups in user context and where duplicates exist, exporting out Name, Groupid and Version id to a CSV.
I then import this using an elevated Powershell session (as I need admin rights to remove connection groups).
So CSV headers are
Name, GroupID, VersionID
The duplication lies in the Name header only
E.g.
Name, Group ID, Version ID
Adobe_Reader, 123, 456
Adobe_Reader, 456, 789
Adobe_Reader, 111, 555
Notepad, 333,222
Notepad, 111,444
Receiver, 444,777
Receiver, 123,999
What I would like to do is, for each duplicate name grab, the Group ID and Version ID to use in a remove-appvclientconnectiongroup. HOWEVER - I don't wish to do this for each entry - I want to stop when there is one left of each name (i.e when than name becomes unique in the list).
So in the end the list would be:
Adobe_Reader, 111, 555
Notepad, 111,444
Receiver, 123,999
And these are the ones we don't want to run throught the cmdlet
Any ideas? APologies if that makes no sense!
I've been playing around with arrays but not getting anywhere fast.
Assuming you have a CSV file already, you can do the following to return the last item in a group of identical names:
Import-Csv file.csv | Group-Object Name |
Foreach-Object { $_.Group[-1] }
Explanation:
Using Group-Object, you can group objects based on a property value. Here, grouping by property Name creates collection of items with properties Count,Name,Group. Name contains the values of the property you are grouping by. Count contains the number of matching values of that grouped property. Group contains the objects that had matching property values.
Since the Group property contains your objects, you can access the objects using the member access operator .. When piping to Foreach-Object, $_.Group will return the object groupings. Then you simply need to grab the last element [-1] of the collection.
If you have that information stored in a CSV file, you can do this to remove all but the last connection group:
Import-Csv -Path 'TheGroups.csv' | Group-Object Name | Where-Object { $_.Count -gt 1 } | Foreach-Object {
$last = $_.Count - 2
# remove all but the last connection group
$_.Group[0..$last] | Remove-AppvClientConnectionGroup
}
Thanks! I managed to get it working with the code below after much messing about. As there can be multiple instances of duplicates I pushed everything into an editable array which a removes line from as the groups are removed. It then checks how many duplicates are left for any given package and stops when there's one of each left
$data = import-csv $env:ALLUSERSPROFILE\AppvDuplciateGroups.csv
#Push the data into an ArrayList so it can be edited on the fly
$dataarray = [System.Collections.ArrayList]($data)
#Get the array size, number of duplicates and target array size
$arraysize = $dataarray.Count
$dupescount = $dataarray| group name
$arraytargetsize = $dupescount.count
$i = $arraysize -1
Function RemoveDuplicates(){
#select the relevant duplicate from the array based in index number (running bottom to top)
$lineX = $dataarray | select -Index $i
 #remove the duplicate
Remove-AppvClientConnectionGroup -GroupId $lineX.groupid -VersionId $lineX.VersionId
 #remove enrty from the array
$dataarray.RemoveAt($i)
 #check to see if that was the last entry for that particular connection group
$getcount = $dataarray | group-object name| Select-Object name, count | where name -eq $lineX.name
 #if there's still more that one entry for that package, run the function again on the next line up
If ($getcount.count -gt 1){
$i = $i -1
RemoveDuplicates}
 #if the array size is still larger than the calculated target array size, move up 2 lines in the array and run the function again
Else{
    If ($dataarray.count -gt $arraytargetsize){
        $i = $i -2
        RemoveDuplicates}
 #once the array meets the calculated target size, repair all connection groups and remove .csv file         
          
          Else{
                Repair-AppvClientConnectionGroup *
                Remove-Item -Path $env:ALLUSERSPROFILE\AppvDuplicateGroups.csv}
}
}
RemoveDuplicates ```

Powershell Grouping of csv import file

Hopefully I'm just missing something that is simple...
I have a csv file similar to this :
Employee ID, Entry Date, Product Code,Amount Due
0001,20/11/2017,A001,10
0001,20/11/2017,Q003,13
0001,20/11/2017,H001,8
0002,20/11/2017,P003,12
0002,20/11/2017,A001,7
and what I want as an output is similar to this :
0001;<some header text>;200171120
A001;10
Q003;13
H001;8
0002;<some header text>;200171120
P003;12
A001;7
So that each detail section is grouped by the Employee ID it relates to
I have tried piping the group-object ("Employee ID") after using an import-csv ... but I can't get it to work correctly
Any help much appreciated!
I think this does what you need:
$CSV | Group-Object 'Employee ID' | ForEach-Object {
$EntryDate = (Get-Date ($_.Group.'Entry Date' | Select -First 1) -F 'yyyyMMdd')
"{0};<some header text>;{1}" -f $_.Name, $EntryDate
ForEach ($Item in $_.Group) {
"{0},{1}" -f $Item.'Product Code',$Item.'Amount Due'
}
}
Explanation:
Uses Group-Object to group the results by Employee ID and then ForEach-Object to iterate through the collection.
Gets the Entry date from the first entry in the group, converts it to a date object with Get-Date and then formats it as year/month/day (this part assumes you don't care if there are other/differing dates in the collection).
Outputs the header string you wanted, using {0},{1} as placeholders for the variables passed via -f (there are multiple ways to do this but this seemed tidiest for your scenario).
Uses ForEach to iterate through the Group property, which contains the grouped items. Again using string replacement to output the product code and amount due fields in the format you desired.

"Normalizing" a CSV file

I have a CSV file for help desk calls. The same ticket might have 1,2, or even 5 records based on the number of updates it has. (One field is different, all other fields are identical).
I want to take the mostly-duplicate records and create one record with the differences concatenated into it. Having programmed in the past, but being a newbie to PowerShell, I could use some help.
So, based on a previous question I asked, here's what I have so far. Assuming data like this:
ID, Date, Comment
345, 1/1/16, Moss has reported a fire in the I/T Room
345, 1/1/16, Never mind, he has sent an e-mail about it.
346, 1/2/16, Someone on the 5th floor is complaining about a man under her desk.
347, 2/1/16, Jen says she has broken the Internet.
347, 2/1/16, Douglas is very angry, we need a fix ASAP!
347, 2/1/16, Roy was playing a joke on her. Closing ticket.
I have the following code:
$FileList = Import-Csv "Call List.csv"
$incidents = $FileList | Group ID
foreach($group in $incidents)
{
# What goes here?
}
How do I take the comments from the 2nd, 3rd, etc. line in the group, concatenate it to the comment in the first, and write the file out?
The Group-Object produces an object with Name and Group, Group containing all the items in that group. You can extract them and create a new object using something like this:
$incidents = $FileList | Group-Object ID | % {
New-Object psobject -property #{
ID = $_.Name
Date = $_.Group[0].Date
Comment = ($_.Group | Select -Expandproperty Comment) -Join "`n"
}
}
(not tested as I am currently on a Mac)
I'd first get a list of the unique IDs, for example:
$Ids = $FileList | Select-Object -ExpandProperty Id -Unique
Then I'd look through the list of tickets and build up a "report" for each ID:
foreach($Id in $Ids){
# Get all incident logs for this Id:
$logs = $FileList | ?{$_.Id -eq $Id}
$report = ""
foreach($log in $logs){
$report += $log.Date + ": " + $log.Comment + "; "
}
# Now you can write the text out
$report | Out-File $Outfile -Append
}
Hope that gives you an idea.

How to Parse the Results of CMDlet into Array

I am sure there is an easy answer to this question, but I cannot find it anywhere. I would like to know how to parse the results of Get-Mailbox | select Alias into an array so each time I use the array it does not show the items as "#{Alias=username}".
I have tired this, but it seems to make the values not text:
$arrayname = Get-Mailbox | select Alias
I am sure this question has been asked before, but I cannot find it.
Essentially I would like to get the Alias' from the Get-Mailbox command into an array so that I can use the foreach cmdlet to get specific folder information from a user like so:
>> $aliases = Get-Mailbox | select Alias
>> Foreach ($username in $aliases) {Get-MailboxFolderStatistics -identity $username | select FolderPath | where {$_.FolderPath -like '*Deleted*'} | Export-CSV "C:\users\username\desktop\allusers-deletedfolder-postarchive.csv" -NoTypeInformation}
The cmdlet already produces an array, only that its elements are mailbox objects, not strings. Selecting the Alias property restricts the object properties, but still leaves you with an array of objects (hence the output you observed). You need to expand the property:
$arrayname = Get-Mailbox | select -Expand Alias
or echo it in a loop:
$arrayname = Get-Mailbox | % { $_.Alias }

Resources