I have the following dataset [Array of Hashes]:
Assume ID is always unique
$dataset = #(
#{
ID = "1234567891"
Code = "ABC1111"
},
#{
ID = "1234567892"
Code = "ABC1111"
},
#{
ID = "1234567893"
Code = "ABC1112"
},
#{
ID = "1234567894"
Code = "ABC1113"
},
#{
ID = "1234567895"
Code = "ABC1114"
},
#{
ID = "1234567896"
Code = "ABC1111"
}
)
What I am trying to do is to group the following dataset by Code key.
I already tried multiple methods such as piping Group-By, Group-Object, Sort-Object but I'm still not getting the result I want.
What result I am looking to return is a hashtable which looks like so [Or anything similar]:
$groupedDataset = #{
ABC1111 = #("1234567891","1234567892","1234567896")
ABC1112 = #("1234567893")
ABC1113 = #("1234567894")
ABC1114 = #("1234567895")
}
Convert the hash tables to a PSCustomObjects, group it, then assign it to the new hash table:
$groupedDataset = #{}
$dataset |
ForEach-Object { [PSCustomObject]$_ } |
Group-Object -Property Code |
ForEach-Object { $groupedDataset[$_.Name] = $_.Group.ID }
See Get-Help about_Object_Creation for more information about using [PSCustomObject] to create custom objects from hash tables.
To complement Bacon Bits' helpful answer:
There is no strict need to provide custom objects as input to Group-Object; [hashtable] instances can be used as-is, as long as you use a script-block argument to access the entry to group by (PSv3+ syntax):
$ht = #{}
$dataset | Group-Object { $_.Code } | ForEach-Object { $ht[$_.Name] = $_.Group.Id }
Note the use of { $_.Code } in lieu of [-Property] Code; the latter only works with bona fide properties (as opposed to hashtable entries; conversely, however, { $_.Code } works in either scenario, though Code, if applicable, is faster).
Related
I am trying to create array made by PSCustomObject elements, but having some issues with it. Here is the simplified example of the code I have:
$FineData = #()
foreach ($Session in $Sessions) {
$Job = Get-VBRTaskSession -Session $Session
$Result = [pscustomobject]#{
Id = $job.Id.Guid
}
$FineData += $Result
}
$FineData
The format it returns:
Id
{19ea68a7-f2c3-4429-bc46-a87b4a295105, 3ebf8568-10ce-4608-bbb2-c80d06874173, 2ec28852-3f5e-477d-8742-872863e41b6d}
The format I want:
Id : 19ea68a7-f2c3-4429-bc46-a87b4a295105
Id : 3ebf8568-10ce-4608-bbb2-c80d06874173
Id : 2ec28852-3f5e-477d-8742-872863e41b6d
Thanks in advance.
It seems like your Get-VBRTaskSession could be returning more than one object, hence an inner loop is required:
$FineData = foreach ($Session in $Sessions) {
foreach($guid in (Get-VBRTaskSession -Session $Session).Id.Guid) {
[pscustomobject]#{
Id = $guid
}
}
}
$FineData
You can then pipe $FineData to Format-List if you want it to be displayed as a list in your console:
$FineData | Format-List
As aside, adding to a fixed collection should be avoided, see this answer for details: Why should I avoid using the increase assignment operator (+=) to create a collection.
I'm attempting to compare two object properties called "name" and that contain the same type of data, in this case server host names. The objects are not identical AKA do not have the same properties. In short, I'm attempting to compare two lists of server names and determine where (which object) they are missing from.
I'm looking to find the items (server host names) that are missing from each object. When something is found I'm hoping to obtain all related properties for the item in the given object that it was found in. I can do the compare-object successfully, but don't know how to get the results I'm looking for.
I'm thinking two new objects could be created for each, that list the items that were not found in the other object maybe? Or do I somehow reference the previous objects with the output from compare-object and produce some formatted output?
This code currently produces a blank file.
Data Format:
$SNObject:
name,ip,class
server-place.com,10.10.10.10,windows server
$QRObject:
name,date,ip,general,device type
server-place1.com,11.11.11.11,random info,linux server
Code Example:
$compare = compare-object $SNObject $QRObject -property Name |
foreach {
if ($_.sideindicator -eq '<=')
{$_.sideindicator = $PathToQRReport }
if ($_.sideindicator -eq '=>')
{$_.sideindicator = $PathToSNReport}
}
$compare |
select #{l='Value';e={$_.InputObject}},#{l='File';e={$_.SideIndicator}} |
Out-File -FilePath C:\Temp\MissingOutputs1.txt
Ahh... just thought of an alternative that may give you exactly what you're looking for but in a slightly different way:
## Join both arrays into single array and then group it on the property name that has a shared value, 'name'
$all = #()
$group = #($SNObject + $QRObject)
$group | Group-Object -Property name | % {
## Create a custom object that contains all possible properties plus a directionality indicator ('source')
$n = New-Object PSObject -Property #{
'name' = ''
'date' = ''
'ip' = ''
'general' = ''
'platform' = ''
'source' = ''
}
if ($_.Count -eq 1) {
## Loop through the grouped results and determine their source and write properties based off of their source
foreach ($i in $_.Group) {
if (#($i | Get-Member -MemberType NoteProperty | Select-Object -ExpandProperty name) -contains 'date' ) {
## This value came from $QRObject which apparently is the only dataset that contains a date property
$n.source = 'QRObject'
$n.date = $i.date
$n.general = $i.general
$n.platform = $i.'device type'
} else {
## This object does not contain the 'date' property, therefore it came from $SNObject
$n.source = 'SNObject'
$n.platform = $i.class
}
## write out common properties
$n.name = $i.name
$n.ip = $i.ip
## add the custom PSObject back to a master array with all formatted properties
$all += $n
}
}
}
$all | out-whereever-you-want
I have a solution for this but I believe it is not the best method as it takes forever so I am looking for a faster/better/smarter way.
I have multiple pscustomObject objects pulled from .csv files. Each object has at least one common property. One is relatively small (around 200-300 items/lines in the object) but the other is sizable (around 60,000-100,000 items). The contents of one may or may not match the contents of the other.
I need to find where the two objects match on a specific property and then combine the properties of each object into one object with all or most properties.
An example snippet of the code (not exact but for this it should work - see the image for the sample data):
DataTables
Write-Verbose "Pulling basic Fruit data together"
$Purchase = import-csv "C:\Purchase.csv"
$Selling = import-csv "C:\Selling.csv"
Write-Verbose "Combining Fruit names and removing duplicates"
$Fruits = $Purchase.Fruit
$Fruits += $Selling.Fruit
$Fruits = $Fruits | Sort-Object -Unique
$compareData = #()
Foreach ($Fruit in $Fruits) {
$IndResults = #()
$IndResults = [pscustomobject]#{
#Adding Purchase and Selling data
Farmer = $Purchase.Where({$PSItem.Fruit -eq $Fruit}).Farmer
Region = $Purchase.Where({$PSItem.Fruit -eq $Fruit}).Region
Water = $Purchase.Where({$PSItem.Fruit -eq $Fruit}).Water
Market = $Selling.Where({$PSItem.Fruit -eq $Fruit}).Market
Cost = $Selling.Where({$PSItem.Fruit -eq $Fruit}).Cost
Tax = $Selling.Where({$PSItem.Fruit -eq $Fruit}).Tax
}
Write-Verbose "Loading Individual results into response"
$CompareData += $IndResults
}
Write-Output $CompareData
I believe the issue is in lines like these:
Farmer = $Purchase.Where({$PSItem.Fruit -eq $Fruit}).Farmer
If I understand this it is looking through the $Purchase object each time it goes through this line. I am looking for a way to speed that whole process up instead of having it look through the entire object for each match attempt.
Using this Join-Object:
$Purchase | Join $Selling -On Fruit | Format-Table
Result (using Simon Catlin's data):
Fruit Farmer Region Water Market Cost Tax
----- ------ ------ ----- ------ ---- ---
Apple Adam Alabama 1 MarketA 10 0.1
Cherry Charlie Cincinnati 2 MarketC 20 0.2
Damson Daniel Derby 3 MarketD 30 0.3
Elderberry Emma Eastbourne 4 MarketE 40 0.4
Fig Freda Florida 5 MarketF 50 0.5
using Join-Object
http://ramblingcookiemonster.github.io/Join-Object/
Join-Object -Left $purchase -Right $selling -LeftJoinProperty fruit -RightJoinProperty fruit -Type OnlyIfInBoth | ft
I had this very problem when trying to consolidate employee data from our HR system against employee data in our AD forest. With many thousands of rows, the process was taking an age.
I eventually walked away from custom objects and reverted to old school hash tables.
The hash tables entries themselves then held a sub-hash table with the data. In your instance, the outer hash would be keyed on $fruit, with the sub-hash containing the various attributes, e.g.: farmer, region, Etc.
Hash tables are lightning quick in comparison. It's a shame that PowerShell is slow in this regard.
Shout if you need more info.
26/01 Example code... assuming I'm correctly understanding the requirement:
PURCHASE.CSV:
Fruit,Farmer,Region,Water
Apple,Adam,Alabama,1
Cherry,Charlie,Cincinnati,2
Damson,Daniel,Derby,3
Elderberry,Emma,Eastbourne,4
Fig,Freda,Florida,5
SELLING.CSV
Fruit,Market,Cost,Tax
Apple,MarketA,10,0.1
Cherry,MarketC,20,0.2
Damson,MarketD,30,0.3
Elderberry,MarketE,40,0.4
Fig,MarketF,50,0.5
CODE
[String] $Local:strPurchaseFile = 'c:\temp\purchase.csv';
[String] $Local:strSellingFile = 'c:\temp\selling.csv';
[HashTable] $Local:objFruitHash = #{};
[System.Array] $Local:objSelectStringHit = $null;
[String] $Local:strFruit = '';
if ( (Test-Path -LiteralPath $strPurchaseFile -PathType Leaf) -and (Test-Path -LiteralPath $strSellingFile -PathType Leaf) ) {
#
# Populate data from purchase file.
#
foreach ( $objSelectStringHit in (Select-String -LiteralPath $strPurchaseFile -Pattern '^([^,]+),([^,]+),([^,]+),([^,]+)$' | Select-Object -Skip 1) ) {
$objFruitHash[ $objSelectStringHit.Matches[0].Groups[1].Value ] = #{ 'Farmer' = $objSelectStringHit.Matches[0].Groups[2].Value;
'Region' = $objSelectStringHit.Matches[0].Groups[3].Value;
'Water' = $objSelectStringHit.Matches[0].Groups[4].Value;
};
} #foreach-purchase-row
#
# Populate data from selling file.
#
foreach ( $objSelectStringHit in (Select-String -LiteralPath $strSellingFile -Pattern '^([^,]+),([^,]+),([^,]+),([^,]+)$' | Select-Object -Skip 1) ) {
$objFruitHash[ $objSelectStringHit.Matches[0].Groups[1].Value ] += #{ 'Market' = $objSelectStringHit.Matches[0].Groups[2].Value;
'Cost' = [Convert]::ToDecimal( $objSelectStringHit.Matches[0].Groups[3].Value );
'Tax' = [Convert]::ToDecimal( $objSelectStringHit.Matches[0].Groups[4].Value );
};
} #foreach-selling-row
#
# Output data. At this point, you could now build a PSCustomObject.
#
foreach ( $strFruit in ($objFruitHash.Keys | Sort-Object) ) {
Write-Host -Object ( '{0,-15}{1,-15}{2,-15}{3,-10}{4,-10}{5,10:C}{6,10:P}' -f
$strFruit,
$objFruitHash[$strFruit]['Farmer'],
$objFruitHash[$strFruit]['Region'],
$objFruitHash[$strFruit]['Water'],
$objFruitHash[$strFruit]['Market'],
$objFruitHash[$strFruit]['Cost'],
$objFruitHash[$strFruit]['Tax']
);
} #foreach
} else {
Write-Error -Message 'File error.';
} #else-if
I needed to do this myself for something similar. I wanted to take two system array objects and compare them pulling out the matches without having to manipulate the input data each time. Here's the method I used, which although I appreciate this is inefficient, it was instantaneous for the 200 or so records I had to work with.
I tried to translate what I was doing (users and their old and new home directories) into farmers, fruit and markets etc so I hope it makes sense!
$Purchase = import-csv "C:\Purchase.csv"
$Selling = import-csv "C:\Selling.csv"
$compareData = #()
foreach ($iPurch in $Purchase) {
foreach ($iSell in $Selling) {
if ($iPurch.fruit -match $iSell.fruit) {
write-host "Match found between $($iPurch.Fruit) and $($iSell.Fruit)"
$hash = #{
Fruit = $iPurch.Fruit
Farmer = $iPurch.Farmer
Region = $iPurch.Region
Water = $iPurch.Water
Market = $iSell.Market
Cost = $iSell.Cost
Tax = $iSell.Tax
}
$Build = New-Object PSObject -Property $hash
$Total = $Total + 1
$compareData += $Build
}
}
}
Write-Host "Processed $Total records"
I have an array of MailItems from Outlook. I want to search each mail item and return the Subject and a Category based on a list of search terms contained in an array - in the example called $searchArray.
For example:
$mailbox = "my.mailbox#example.com"
$outlook = New-Object -com Outlook.Application
$ns = $outlook.GetNamespace("MAPI")
$inbox = $ns.Folders.Item($mailbox).Folders.Item("Inbox")
$searchItems = $inbox.Folders.Item("MySubFolder").Folders.Item("MyNestedSubFolder").Items
$searchArray = (
"Category1", ("searchTerm1","searchTerm2","searchTerm3"),
"Category2", ("searchTerm4","searchTerm5"),
"Category3", ("searchTerm6")
)
foreach ($msg in $searchItems) {
$msg | select Subject, # <Category where email address contains one of the search terms>
}
I want to return the Subject, and then a column called Category which will look at the $msg.SenderEmailAddress and if any of the searchTerms in the $searchArray is contained within the address, return the category that had that search term in it.
For example if one of the SenderEmailAddress values was "searchTerm2#somewhere.com" then return Category1 as the Category.
I would flip that array on its head and create a hashtable from it. Then use the first matching search term as a lookup key for the category:
$searchArray = (
"Category1", ("searchTerm1","searchTerm2","searchTerm3"),
"Category2", ("searchTerm4","searchTerm5"),
"Category3", ("searchTerm6")
)
# Create hashtable
$searchTable = #{}
# Populate hash table with values from array
for($i=0;$i-lt$searchArray.Count;$i+=2){
foreach($term in $searchArray[$i+1])
{
$searchTable[$term] = $searchArray[$i]
}
}
# Select Category based on first matching search term
$msg |Select-Object Subject,#{Name="Category";Expression={
$sender = $_.SenderEmailAddress
$searchTable[$($searchTable.Keys |Where-Object{$sender -like "*$_*"} |Select -First 1)]
}
}
Still need to use a calculated expression just as Mathias did (It's really the simple way). However I wanted to show an approach where you had a custom object array for the $searchArray. If you were to tailor it from scratch it would look like this. I also converted the terms into regex pattern matches since you say they are unique. Only caveat there is you need to be sure that there are no regex meta-characters in your search terms.
$searchArray = (
[pscustomobject]#{
Category = "1"
Pattern = "searchTerm1|searchTerm2|searchTerm3"
},
[pscustomobject]#{
Category = "2"
Pattern = "searchTerm4|searchTerm5"
},
[pscustomobject]#{
Category = "3"
Pattern = "searchTerm6"}
)
foreach ($msg in $searchItems) {
$msg | select Subject, #{
Name="Category";
Expression={$searchArray | Where-Object{$msg.SenderEmailAddress -match $_.pattern } | Select-Object -ExpandProperty Category}
}
}
Solution is dependant on PowerShell 3.0 from the type accelerator [pscustomobject]. Could easily bring it back to 2.0 if need be.
To showcase similar structure using 2.0 and automatic conversion of you array to one that works with my code.
$newSearchArray = for($categoryIndex=0;$categoryIndex-lt$searchArray.Count;$categoryIndex+=2){
New-Object -TypeName pscustomobject -Property #{
Category = $searchArray[$categoryIndex]
Pattern = ($searchArray[$categoryIndex+1] | ForEach-Object{[regex]::Escape($_)}) -join "|"
}
}
Now the search terms are automatically escaped and joined into a search pattern.
Using a switch:
$mailbox = "my.mailbox#example.com"
$outlook = New-Object -com Outlook.Application
$ns = $outlook.GetNamespace("MAPI")
$inbox = $ns.Folders.Item($mailbox).Folders.Item("Inbox")
$searchItems = $inbox.Folders.Item("MySubFolder").Folders.Item("MyNestedSubFolder").Items
foreach ($msg in $searchItems) {
$object = $msg | select Subject,Category # <Category where email address contains one of the search terms>
Switch -Regex ($msg.SenderEmailAddress)
{
'searchTerm1|searchTerm2|searchTerm3' { $object.Catetory = 'Category1' ; break }
'searchTerm4|searchTerm5' { $object.Catetory = 'Category2' ; break }
'searchTerm6' { $object.Catetory = 'Category3' ; break }
}
$object
}
I've been struggling with this for a couple of days, and I'm not sure how to conquer it. I need to do the following:
Import a csv of users with the following values:
ID, Name, Region
Create an array based on the Region values that I can then use to populate with ID's and Names with that region, ie.
Array_SEA
AA_SCOM, Adam Andrews, SEA
Array_OAK
BB_SCOM, Bob Barker, OAK
Here's the code I've got right now:
$list2 = ipcsv .\TSE_Contact_List.csv | sort-object BU
$arraylist =#()
foreach ($vitem in $list2)
{
$arraylist += New-Object PsObject -Property #{'Array' = "Array_" + $vitem.bu}
}
foreach ($varray in $arraylist)
{
$arr = new-variable -Name $varray
$arr.value += $varray.array
$arr
}
This produces the following error for records with a duplicate regions:
New-Variable: A variable with name '#{Array=Array_SCA}' already exists.
I'm also getting the following when it tries to add values:
Property 'value' cannot be found on this object; make sure it exists and is settable.
I get that I'm not actually creating arrays in the second section, but I'm not sure how to pass the output of the variable to an array name without turning the variable declaration into the array name, if that makes sense.
I've tried the following with hash tables, and it gets closer:
$list2 = ipcsv .\TSE_Contact_List.csv | sort-object BU
$arraylist =#{}
foreach ($vitem in $list2){$arraylist[$vitem.bu] = #()}
foreach ($record in $list2)
{
$arraylist[$vitem.bu] += ($record.SCOMID,$record.Name,$record.BU)
Write-host "Array: "
$arraylist[$vitem.bu]
write-host ""
}
The output on this shows no errors, but it just keeps showing the added fields for all of the records for each iteration of the list, so I don't think that it's actually assigning each unique BU to the array name.
I like the hashtable-approach, but I would finetune it a little. Try:
$list2 = ipcsv .\TSE_Contact_List.csv | sort-object BU
$arraylist = #{}
foreach ($vitem in $list2){
if($arraylist.ContainsKey($vitem.BU)) {
#Array exists, add item
$arraylist[($vitem.BU)] += $vitem
} else {
#Array not found, creating it
$arraylist[($vitem.BU)] = #($vitem)
}
}
#TEST: List arrays and number of entries
$arraylist.GetEnumerator() | % {
"Array '$($_.Key)' has $($_.Value.Count) items"
}
You could also use Group-Object like:
$list2 = ipcsv .\TSE_Contact_List.csv | Group-Object BU
#TEST: List groups(regions) and number of entries
$list2 | % {
"Region '$($_.Name)' has $(#($_.Group).Count) items"
}