How to search a collection with an array in Powershell - arrays

I have an array of MailItems from Outlook. I want to search each mail item and return the Subject and a Category based on a list of search terms contained in an array - in the example called $searchArray.
For example:
$mailbox = "my.mailbox#example.com"
$outlook = New-Object -com Outlook.Application
$ns = $outlook.GetNamespace("MAPI")
$inbox = $ns.Folders.Item($mailbox).Folders.Item("Inbox")
$searchItems = $inbox.Folders.Item("MySubFolder").Folders.Item("MyNestedSubFolder").Items
$searchArray = (
"Category1", ("searchTerm1","searchTerm2","searchTerm3"),
"Category2", ("searchTerm4","searchTerm5"),
"Category3", ("searchTerm6")
)
foreach ($msg in $searchItems) {
$msg | select Subject, # <Category where email address contains one of the search terms>
}
I want to return the Subject, and then a column called Category which will look at the $msg.SenderEmailAddress and if any of the searchTerms in the $searchArray is contained within the address, return the category that had that search term in it.
For example if one of the SenderEmailAddress values was "searchTerm2#somewhere.com" then return Category1 as the Category.

I would flip that array on its head and create a hashtable from it. Then use the first matching search term as a lookup key for the category:
$searchArray = (
"Category1", ("searchTerm1","searchTerm2","searchTerm3"),
"Category2", ("searchTerm4","searchTerm5"),
"Category3", ("searchTerm6")
)
# Create hashtable
$searchTable = #{}
# Populate hash table with values from array
for($i=0;$i-lt$searchArray.Count;$i+=2){
foreach($term in $searchArray[$i+1])
{
$searchTable[$term] = $searchArray[$i]
}
}
# Select Category based on first matching search term
$msg |Select-Object Subject,#{Name="Category";Expression={
$sender = $_.SenderEmailAddress
$searchTable[$($searchTable.Keys |Where-Object{$sender -like "*$_*"} |Select -First 1)]
}
}

Still need to use a calculated expression just as Mathias did (It's really the simple way). However I wanted to show an approach where you had a custom object array for the $searchArray. If you were to tailor it from scratch it would look like this. I also converted the terms into regex pattern matches since you say they are unique. Only caveat there is you need to be sure that there are no regex meta-characters in your search terms.
$searchArray = (
[pscustomobject]#{
Category = "1"
Pattern = "searchTerm1|searchTerm2|searchTerm3"
},
[pscustomobject]#{
Category = "2"
Pattern = "searchTerm4|searchTerm5"
},
[pscustomobject]#{
Category = "3"
Pattern = "searchTerm6"}
)
foreach ($msg in $searchItems) {
$msg | select Subject, #{
Name="Category";
Expression={$searchArray | Where-Object{$msg.SenderEmailAddress -match $_.pattern } | Select-Object -ExpandProperty Category}
}
}
Solution is dependant on PowerShell 3.0 from the type accelerator [pscustomobject]. Could easily bring it back to 2.0 if need be.
To showcase similar structure using 2.0 and automatic conversion of you array to one that works with my code.
$newSearchArray = for($categoryIndex=0;$categoryIndex-lt$searchArray.Count;$categoryIndex+=2){
New-Object -TypeName pscustomobject -Property #{
Category = $searchArray[$categoryIndex]
Pattern = ($searchArray[$categoryIndex+1] | ForEach-Object{[regex]::Escape($_)}) -join "|"
}
}
Now the search terms are automatically escaped and joined into a search pattern.

Using a switch:
$mailbox = "my.mailbox#example.com"
$outlook = New-Object -com Outlook.Application
$ns = $outlook.GetNamespace("MAPI")
$inbox = $ns.Folders.Item($mailbox).Folders.Item("Inbox")
$searchItems = $inbox.Folders.Item("MySubFolder").Folders.Item("MyNestedSubFolder").Items
foreach ($msg in $searchItems) {
$object = $msg | select Subject,Category # <Category where email address contains one of the search terms>
Switch -Regex ($msg.SenderEmailAddress)
{
'searchTerm1|searchTerm2|searchTerm3' { $object.Catetory = 'Category1' ; break }
'searchTerm4|searchTerm5' { $object.Catetory = 'Category2' ; break }
'searchTerm6' { $object.Catetory = 'Category3' ; break }
}
$object
}

Related

Powershell Compare and Merge two arrays

I'm very new to Powershell and what I want to do is to compare two arrays of data and then merge them together to one large array that I can either export to Excel or POST request to a web server using restful API and json.
To do this in Python is simple by using Pandas and adding search data but in Powershell I can't really get it done.
Example: I have 2 arrays
$a1 = #(('Name1', 'Link1', 'URL1'),
('Name2', 'Link2', 'URL2'),
('Name3', 'Link3', 'URL3')
)
$a2 = #(('Name4', 'URL4', 'TEXT4'),
('Name2', 'URL2', 'TEXT1'),
('Name1', 'URL1', 'TEXT2')
)
I want to do a compare $a1 with $a2 and the other way around so I don't miss any values.
Merge them together so I will end up with a $a3 array that look something like this.
$a3 = #(('Name1', 'Link1', 'URL1', 'TEXT1),
('Name2', 'Link2', 'URL2', 'TEXT2),
('Name3', 'Link3', 'URL3', ''),
('Name4', 'URL4', '', 'TEXT4')
)
And as more I dig in to different alternatives as more confused do I get.
I'd recommend to make a collection of (at least) simple objects to work with.
This is almost classic task of combining two different sets of properties by common key. Just convert those arrays-of-arrays to arrays-of-objects-with-properties and combine them next.
Please note that powershell is primary sysadmin's language, there is almost no place for any guesses or chances when processing tasks.
Also you can use almost any external .Net-DLL library in PowerShell (search Deedle as alternative to Pandas)
$a1 = #(('Name1', 'Link1', 'URL1'),
('Name2', 'Link2', 'URL2'),
('Name3', 'Link3', 'URL3')
)
$a2 = #(('Name4', 'URL4', 'TEXT4'),
('Name2', 'URL2', 'TEXT1'),
('Name1', 'URL1', 'TEXT2')
)
# Make Hashtable K=Name;V={Name, Link, Url, Text}
$AObjects = #{}
# Fill Hashtable from A1
#($a1) |
ForEach-Object {
$name = $_[0]
$AObjects[$name] = #{
Name = $name
Link = $_[1]
Url = $_[2]
Text = ''
}
}
# Fill Hashtable from A2
#($a2) |
ForEach-Object {
$name = $_[0]
$url = $_[1]
if (-not $AObjects.ContainsKey($name)) {
Write-Warning "A2 has [$name] that was not listed in A1!"
$AObjects[$name] = #{Name = $name; Url = $url; Link = ''; Text = ''}
} elseif($AObjects[$name].Url -ne $url) {
Write-Warning "For $($name), A1's URL and A2's URL differ! Using A1's"
}
$AObjects[$name].Text = $_[2]
}
# Convert Hashtable to array of objects with props. Select is required for better output.
$AObjects = #($AObjects.Values) | % { [PSCustomObject]$_ } | Select-Object #('Name', 'Link', 'URL', 'Text')
# Convert array of objects with props back to array
$result = $AObjects |
ForEach-Object {
if ([string]::IsNullOrWhiteSpace($_.Link)) { # Didn't get your logic swapping elements in example
return #(,#($_.Name, $_.Url, $_.Link, $_.Text))
} else {
return #(,#($_.Name, $_.Link, $_.Url, $_.Text))
}
}
#Print
$result | % { "[$($_ -join ',')]" }
# [Name3,Link3,URL3,]
# [Name1,Link1,URL1,TEXT2]
# [Name2,Link2,URL2,TEXT1]
# [Name4,URL4,,TEXT4]

Powershell Group Array of Hashes with Duplicates

I have the following dataset [Array of Hashes]:
Assume ID is always unique
$dataset = #(
#{
ID = "1234567891"
Code = "ABC1111"
},
#{
ID = "1234567892"
Code = "ABC1111"
},
#{
ID = "1234567893"
Code = "ABC1112"
},
#{
ID = "1234567894"
Code = "ABC1113"
},
#{
ID = "1234567895"
Code = "ABC1114"
},
#{
ID = "1234567896"
Code = "ABC1111"
}
)
What I am trying to do is to group the following dataset by Code key.
I already tried multiple methods such as piping Group-By, Group-Object, Sort-Object but I'm still not getting the result I want.
What result I am looking to return is a hashtable which looks like so [Or anything similar]:
$groupedDataset = #{
ABC1111 = #("1234567891","1234567892","1234567896")
ABC1112 = #("1234567893")
ABC1113 = #("1234567894")
ABC1114 = #("1234567895")
}
Convert the hash tables to a PSCustomObjects, group it, then assign it to the new hash table:
$groupedDataset = #{}
$dataset |
ForEach-Object { [PSCustomObject]$_ } |
Group-Object -Property Code |
ForEach-Object { $groupedDataset[$_.Name] = $_.Group.ID }
See Get-Help about_Object_Creation for more information about using [PSCustomObject] to create custom objects from hash tables.
To complement Bacon Bits' helpful answer:
There is no strict need to provide custom objects as input to Group-Object; [hashtable] instances can be used as-is, as long as you use a script-block argument to access the entry to group by (PSv3+ syntax):
$ht = #{}
$dataset | Group-Object { $_.Code } | ForEach-Object { $ht[$_.Name] = $_.Group.Id }
Note the use of { $_.Code } in lieu of [-Property] Code; the latter only works with bona fide properties (as opposed to hashtable entries; conversely, however, { $_.Code } works in either scenario, though Code, if applicable, is faster).

Compare Two Objects Properties and Obtain Missing Items

I'm attempting to compare two object properties called "name" and that contain the same type of data, in this case server host names. The objects are not identical AKA do not have the same properties. In short, I'm attempting to compare two lists of server names and determine where (which object) they are missing from.
I'm looking to find the items (server host names) that are missing from each object. When something is found I'm hoping to obtain all related properties for the item in the given object that it was found in. I can do the compare-object successfully, but don't know how to get the results I'm looking for.
I'm thinking two new objects could be created for each, that list the items that were not found in the other object maybe? Or do I somehow reference the previous objects with the output from compare-object and produce some formatted output?
This code currently produces a blank file.
Data Format:
$SNObject:
name,ip,class
server-place.com,10.10.10.10,windows server
$QRObject:
name,date,ip,general,device type
server-place1.com,11.11.11.11,random info,linux server
Code Example:
$compare = compare-object $SNObject $QRObject -property Name |
foreach {
if ($_.sideindicator -eq '<=')
{$_.sideindicator = $PathToQRReport }
if ($_.sideindicator -eq '=>')
{$_.sideindicator = $PathToSNReport}
}
$compare |
select #{l='Value';e={$_.InputObject}},#{l='File';e={$_.SideIndicator}} |
Out-File -FilePath C:\Temp\MissingOutputs1.txt
Ahh... just thought of an alternative that may give you exactly what you're looking for but in a slightly different way:
## Join both arrays into single array and then group it on the property name that has a shared value, 'name'
$all = #()
$group = #($SNObject + $QRObject)
$group | Group-Object -Property name | % {
## Create a custom object that contains all possible properties plus a directionality indicator ('source')
$n = New-Object PSObject -Property #{
'name' = ''
'date' = ''
'ip' = ''
'general' = ''
'platform' = ''
'source' = ''
}
if ($_.Count -eq 1) {
## Loop through the grouped results and determine their source and write properties based off of their source
foreach ($i in $_.Group) {
if (#($i | Get-Member -MemberType NoteProperty | Select-Object -ExpandProperty name) -contains 'date' ) {
## This value came from $QRObject which apparently is the only dataset that contains a date property
$n.source = 'QRObject'
$n.date = $i.date
$n.general = $i.general
$n.platform = $i.'device type'
} else {
## This object does not contain the 'date' property, therefore it came from $SNObject
$n.source = 'SNObject'
$n.platform = $i.class
}
## write out common properties
$n.name = $i.name
$n.ip = $i.ip
## add the custom PSObject back to a master array with all formatted properties
$all += $n
}
}
}
$all | out-whereever-you-want

Powershell match properties and then selectively combine objects to create a third

I have a solution for this but I believe it is not the best method as it takes forever so I am looking for a faster/better/smarter way.
I have multiple pscustomObject objects pulled from .csv files. Each object has at least one common property. One is relatively small (around 200-300 items/lines in the object) but the other is sizable (around 60,000-100,000 items). The contents of one may or may not match the contents of the other.
I need to find where the two objects match on a specific property and then combine the properties of each object into one object with all or most properties.
An example snippet of the code (not exact but for this it should work - see the image for the sample data):
DataTables
Write-Verbose "Pulling basic Fruit data together"
$Purchase = import-csv "C:\Purchase.csv"
$Selling = import-csv "C:\Selling.csv"
Write-Verbose "Combining Fruit names and removing duplicates"
$Fruits = $Purchase.Fruit
$Fruits += $Selling.Fruit
$Fruits = $Fruits | Sort-Object -Unique
$compareData = #()
Foreach ($Fruit in $Fruits) {
$IndResults = #()
$IndResults = [pscustomobject]#{
#Adding Purchase and Selling data
Farmer = $Purchase.Where({$PSItem.Fruit -eq $Fruit}).Farmer
Region = $Purchase.Where({$PSItem.Fruit -eq $Fruit}).Region
Water = $Purchase.Where({$PSItem.Fruit -eq $Fruit}).Water
Market = $Selling.Where({$PSItem.Fruit -eq $Fruit}).Market
Cost = $Selling.Where({$PSItem.Fruit -eq $Fruit}).Cost
Tax = $Selling.Where({$PSItem.Fruit -eq $Fruit}).Tax
}
Write-Verbose "Loading Individual results into response"
$CompareData += $IndResults
}
Write-Output $CompareData
I believe the issue is in lines like these:
Farmer = $Purchase.Where({$PSItem.Fruit -eq $Fruit}).Farmer
If I understand this it is looking through the $Purchase object each time it goes through this line. I am looking for a way to speed that whole process up instead of having it look through the entire object for each match attempt.
Using this Join-Object:
$Purchase | Join $Selling -On Fruit | Format-Table
Result (using Simon Catlin's data):
Fruit Farmer Region Water Market Cost Tax
----- ------ ------ ----- ------ ---- ---
Apple Adam Alabama 1 MarketA 10 0.1
Cherry Charlie Cincinnati 2 MarketC 20 0.2
Damson Daniel Derby 3 MarketD 30 0.3
Elderberry Emma Eastbourne 4 MarketE 40 0.4
Fig Freda Florida 5 MarketF 50 0.5
using Join-Object
http://ramblingcookiemonster.github.io/Join-Object/
Join-Object -Left $purchase -Right $selling -LeftJoinProperty fruit -RightJoinProperty fruit -Type OnlyIfInBoth | ft
I had this very problem when trying to consolidate employee data from our HR system against employee data in our AD forest. With many thousands of rows, the process was taking an age.
I eventually walked away from custom objects and reverted to old school hash tables.
The hash tables entries themselves then held a sub-hash table with the data. In your instance, the outer hash would be keyed on $fruit, with the sub-hash containing the various attributes, e.g.: farmer, region, Etc.
Hash tables are lightning quick in comparison. It's a shame that PowerShell is slow in this regard.
Shout if you need more info.
26/01 Example code... assuming I'm correctly understanding the requirement:
PURCHASE.CSV:
Fruit,Farmer,Region,Water
Apple,Adam,Alabama,1
Cherry,Charlie,Cincinnati,2
Damson,Daniel,Derby,3
Elderberry,Emma,Eastbourne,4
Fig,Freda,Florida,5
SELLING.CSV
Fruit,Market,Cost,Tax
Apple,MarketA,10,0.1
Cherry,MarketC,20,0.2
Damson,MarketD,30,0.3
Elderberry,MarketE,40,0.4
Fig,MarketF,50,0.5
CODE
[String] $Local:strPurchaseFile = 'c:\temp\purchase.csv';
[String] $Local:strSellingFile = 'c:\temp\selling.csv';
[HashTable] $Local:objFruitHash = #{};
[System.Array] $Local:objSelectStringHit = $null;
[String] $Local:strFruit = '';
if ( (Test-Path -LiteralPath $strPurchaseFile -PathType Leaf) -and (Test-Path -LiteralPath $strSellingFile -PathType Leaf) ) {
#
# Populate data from purchase file.
#
foreach ( $objSelectStringHit in (Select-String -LiteralPath $strPurchaseFile -Pattern '^([^,]+),([^,]+),([^,]+),([^,]+)$' | Select-Object -Skip 1) ) {
$objFruitHash[ $objSelectStringHit.Matches[0].Groups[1].Value ] = #{ 'Farmer' = $objSelectStringHit.Matches[0].Groups[2].Value;
'Region' = $objSelectStringHit.Matches[0].Groups[3].Value;
'Water' = $objSelectStringHit.Matches[0].Groups[4].Value;
};
} #foreach-purchase-row
#
# Populate data from selling file.
#
foreach ( $objSelectStringHit in (Select-String -LiteralPath $strSellingFile -Pattern '^([^,]+),([^,]+),([^,]+),([^,]+)$' | Select-Object -Skip 1) ) {
$objFruitHash[ $objSelectStringHit.Matches[0].Groups[1].Value ] += #{ 'Market' = $objSelectStringHit.Matches[0].Groups[2].Value;
'Cost' = [Convert]::ToDecimal( $objSelectStringHit.Matches[0].Groups[3].Value );
'Tax' = [Convert]::ToDecimal( $objSelectStringHit.Matches[0].Groups[4].Value );
};
} #foreach-selling-row
#
# Output data. At this point, you could now build a PSCustomObject.
#
foreach ( $strFruit in ($objFruitHash.Keys | Sort-Object) ) {
Write-Host -Object ( '{0,-15}{1,-15}{2,-15}{3,-10}{4,-10}{5,10:C}{6,10:P}' -f
$strFruit,
$objFruitHash[$strFruit]['Farmer'],
$objFruitHash[$strFruit]['Region'],
$objFruitHash[$strFruit]['Water'],
$objFruitHash[$strFruit]['Market'],
$objFruitHash[$strFruit]['Cost'],
$objFruitHash[$strFruit]['Tax']
);
} #foreach
} else {
Write-Error -Message 'File error.';
} #else-if
I needed to do this myself for something similar. I wanted to take two system array objects and compare them pulling out the matches without having to manipulate the input data each time. Here's the method I used, which although I appreciate this is inefficient, it was instantaneous for the 200 or so records I had to work with.
I tried to translate what I was doing (users and their old and new home directories) into farmers, fruit and markets etc so I hope it makes sense!
$Purchase = import-csv "C:\Purchase.csv"
$Selling = import-csv "C:\Selling.csv"
$compareData = #()
foreach ($iPurch in $Purchase) {
foreach ($iSell in $Selling) {
if ($iPurch.fruit -match $iSell.fruit) {
write-host "Match found between $($iPurch.Fruit) and $($iSell.Fruit)"
$hash = #{
Fruit = $iPurch.Fruit
Farmer = $iPurch.Farmer
Region = $iPurch.Region
Water = $iPurch.Water
Market = $iSell.Market
Cost = $iSell.Cost
Tax = $iSell.Tax
}
$Build = New-Object PSObject -Property $hash
$Total = $Total + 1
$compareData += $Build
}
}
}
Write-Host "Processed $Total records"

How to create and populate an array in Powershell based on a dynamic variable?

I've been struggling with this for a couple of days, and I'm not sure how to conquer it. I need to do the following:
Import a csv of users with the following values:
ID, Name, Region
Create an array based on the Region values that I can then use to populate with ID's and Names with that region, ie.
Array_SEA
AA_SCOM, Adam Andrews, SEA
Array_OAK
BB_SCOM, Bob Barker, OAK
Here's the code I've got right now:
$list2 = ipcsv .\TSE_Contact_List.csv | sort-object BU
$arraylist =#()
foreach ($vitem in $list2)
{
$arraylist += New-Object PsObject -Property #{'Array' = "Array_" + $vitem.bu}
}
foreach ($varray in $arraylist)
{
$arr = new-variable -Name $varray
$arr.value += $varray.array
$arr
}
This produces the following error for records with a duplicate regions:
New-Variable: A variable with name '#{Array=Array_SCA}' already exists.
I'm also getting the following when it tries to add values:
Property 'value' cannot be found on this object; make sure it exists and is settable.
I get that I'm not actually creating arrays in the second section, but I'm not sure how to pass the output of the variable to an array name without turning the variable declaration into the array name, if that makes sense.
I've tried the following with hash tables, and it gets closer:
$list2 = ipcsv .\TSE_Contact_List.csv | sort-object BU
$arraylist =#{}
foreach ($vitem in $list2){$arraylist[$vitem.bu] = #()}
foreach ($record in $list2)
{
$arraylist[$vitem.bu] += ($record.SCOMID,$record.Name,$record.BU)
Write-host "Array: "
$arraylist[$vitem.bu]
write-host ""
}
The output on this shows no errors, but it just keeps showing the added fields for all of the records for each iteration of the list, so I don't think that it's actually assigning each unique BU to the array name.
I like the hashtable-approach, but I would finetune it a little. Try:
$list2 = ipcsv .\TSE_Contact_List.csv | sort-object BU
$arraylist = #{}
foreach ($vitem in $list2){
if($arraylist.ContainsKey($vitem.BU)) {
#Array exists, add item
$arraylist[($vitem.BU)] += $vitem
} else {
#Array not found, creating it
$arraylist[($vitem.BU)] = #($vitem)
}
}
#TEST: List arrays and number of entries
$arraylist.GetEnumerator() | % {
"Array '$($_.Key)' has $($_.Value.Count) items"
}
You could also use Group-Object like:
$list2 = ipcsv .\TSE_Contact_List.csv | Group-Object BU
#TEST: List groups(regions) and number of entries
$list2 | % {
"Region '$($_.Name)' has $(#($_.Group).Count) items"
}

Resources