$source = #("1234-abc","1234-xyz","34-pqr","3456-xyz","456-pp")
$name = #()
foreach($sr in $source)
{
$array = $sr -split "-"
$ID = $array[0]
if ($sr.contains($ID))
{
$name += Sr.split("-",1) +","
}
}
expected output:
1234 contains 1234-abc, 1234-xyz
34 contains 34-pqr
3456 contains 3456-xyz
456 contains 456-pp
but, as I have used contains() 34's value is coming along with 1234 and same thing is happening with 456 and 3456.
please help me out
You can use Group-Object to group the strings by their prefix:
"1234-abc","1234-xyz","34-pqr","3456-xyz","456-pp" | Group-Object {$_.Split('-')[0]}
Which gives output like this:
Count Name Group
----- ---- -----
2 1234 {1234-abc, 1234-xyz}
1 34 {34-pqr}
1 3456 {3456-xyz}
1 456 {456-pp}
# Use | to send values to next command
#('1234-abc', '1234-xyz', '34-pqr', '3456-xyz', '456-pp', '34-abcd', 'xxxx33333') |
ForEach-Object {
# ForEach-Object will loop through each value
# $_ represents the current object
# use -match operator to check if object ($_) matches pattern. Regex is used
if ($_ -match '^(?<Id>\d+?)-(?<Value>.*)') {
# If match was successful an automatic variable called $Matches is populated
# [PSCustomObject] and hashtable #{key = value } create a new object.
# Here we create an object that contains the Id and the value previously
# found by -match.
[PSCustomObject]#{
Id = $Matches.Id
Value = $Matches.Value
}
# Not assigning the object to a variable automatically sends it out down
# the pipeline to the next command
# Could have also created the object using Select-Object and calculated properties
# shown below:
# # $Matches | Select-Object #{n = 'Id'; e = {$_.Id}}, #{n = 'Value'; e = {$_.Value}}
}
else {
# If value doesn't match the regex create an object that passes this
# down the pipeline as well
[pscustomobject]#{Id = 'No Match'; Value = $_ }
}
} |
# Group objects by Id using Group-Object
Group-Object -Property Id |
# Another ForEach-Object to process each of our new groupings
# containing our pscustomobjects with Id and Value. Since each group is
# being sent 1 at a time we can use this to display each group and the values.
# Groups will contain a Name property generated based off the property being grouped: Id
ForEach-Object {
Write-Host "** $($_.Name) **" -ForegroundColor Yellow
$_.Group.Value
}
Output will look like this
** 1234 **
abc
xyz
** 34 **
pqr
abcd
** 3456 **
xyz
** 456 **
pp
** No Match **
xxxx33333
Here is the same code as above without all the comments
#('1234-abc', '1234-xyz', '34-pqr', '3456-xyz', '456-pp', '34-abcd', 'xxxx33333') |
ForEach-Object {
if ($_ -match '^(?<Id>\d+?)-(?<Value>.*)') {
[PSCustomObject]#{Id = $Matches.Id; Value = $Matches.Value}
}
else {
[pscustomobject]#{Id = 'No Match'; Value = $_ }
}
} |
Group-Object -Property Id |
ForEach-Object {
Write-Host "** $($_.Name) **" -ForegroundColor Yellow
$_.Group.Value
}
$result = [System.Collections.Generic.Dictionary[String,System.Collections.Generic.List[String]]]::new([System.StringComparer]::InvariantCultureIgnoreCase)
#("1234-abc","1234-xyz","34-pqr","3456-xyz","456-pp") |
ForEach-Object {
$indexOfDelimeter = $_.indexOf('-')
$key = $_.Substring(0, $indexOfDelimeter).ToUpperInvariant()
if (-not $result.ContainsKey($key)) {
$result[$key] = [System.Collections.Generic.List[string]]::new()
}
$result[$key].Add($_)
}
Result:
# $result
Key Value
--- -----
1234 {1234-abc, 1234-xyz}
34 {34-pqr}
3456 {3456-xyz}
456 {456-pp}
Related
I have two array of objects. I want to insert the corresponding extraProperty in the "car" array, if found. There may be several extraProperties or none. That is, I wanted to add, when an extraProperty was found for the respective car, an array with the list of extraproperties found.
Each extraProperties consists of an object with the following properties: Id, Name, Value.
Code:
param(
[Parameter(Mandatory=$false, Position=1, ValueFromPipeline=$false)]
[string]$Types,
[Parameter(Mandatory=$false, Position=2, ValueFromPipeline=$false)]
[string]$PathFile,
[Parameter(Mandatory=$false, Position=3, ValueFromPipeline=$false)]
[string]$PathPropertyFile
)
$profiles_list = Import-Csv $PathFile -Header Id, model, Type Delimiter ";"
$extraProperties_list = Import-Csv $PathPropertyFile -Header ProfileId,Name,Value -Delimiter ";" # Get-Content -Path $pathFile
foreach($p in $car_list) {
$Property = $property_list.Where({$_.Id -eq $p.Id}) | Select-Object -Property Name,Value
if(-Not (($null -eq $Property ) -And (#($Property ).Count -eq 0)) ) {
$p = $p | Add-Member -NotePropertyMembers #{Properties=$Property }
} else {
$p = $p | Add-Member -NotePropertyMembers #{Properties=#()}
}
}
Data sample:
PropertyFile.csv
Id | Name | Value
504953 | Example1 | Value1
504953 | Example2 | Value2
504955 | Example3 | Value3
CarFiles.csv
Id | Model | Type
504953 | Model1 | 3
504954 | Model1 | 0
504955 | Model3 | 3
The problem is that the code is not efficient. The car array reaches 200000 positions and where each position is an object with several properties and the properties array also reaches these values. The script takes endless hours to execute.
Any way to optimize the insertion of a new property within arrays?
You need to check if this is faster (we don't know how large the CSV files are), but you could do it like this:
For demo I'm using Here-Strings, but in real life you import the data from files:
$profiles_list = Import-Csv $PathFile -Delimiter ";"
$extraProperties_list = Import-Csv $PathPropertyFile -Delimiter ";"
Using your examples:
$profiles_list = #"
Id;Model;Type
504953;Model1;3
504954;Model1;0
504955;Model3;3
"# | ConvertFrom-Csv -Delimiter ';'
$extraProperties_list = #"
Id;Name;Value
504953;Example1;Value1
504953;Example2;Value2
504955;Example3;Value3
"# | ConvertFrom-Csv -Delimiter ';'
# get the headers from the Cars
$profileHeaders = $profiles_list[0].PsObject.Properties.Name
# get the new property names from the ExtraProperties Name column
$newHeaders = $extraProperties_list.Name | Where-Object {$profileHeaders -notcontains $_} | Select-Object -Unique
# add all these new properties to the $profiles_list, for now with $null values
$profiles_list | ForEach-Object {
foreach($prop in $newHeaders) {
$_ | Add-Member -MemberType NoteProperty -Name $prop -Value $null
}
}
# group the $extraProperties_list by the Id column and loop through these groups
$extraProperties_list | Group-Object Id | ForEach-Object {
$id = $_.Name
# get an array of profiles with matching Ids
$profiles = $profiles_list | Where-Object {$_.Id -eq $id}
# and fill in the blanks
foreach($item in $profiles) {
foreach($extra in $_.Group) {
$item.($extra.Name) = $extra.Value
}
}
}
# output on screen
$profiles_list
# output to new CSV file
$profiles_list | Export-Csv -Path 'X:\CompletedProfiles.csv' -Delimiter ';' -NoTypeInformation
Result on screen:
Id : 504953
Model : Model1
Type : 3
Example1 : Value1
Example2 : Value2
Example3 :
Id : 504954
Model : Model1
Type : 0
Example1 :
Example2 :
Example3 :
Id : 504955
Model : Model3
Type : 3
Example1 :
Example2 :
Example3 : Value3
I have a 100 column table in sql server and I want to make it so not all of the columns need to be passed in the file to load. I have assigned column names in a table that then compares the columns in a hash table to find matching columns. I then create the code based on the match for the array I want to use to insert the data from the file. The problem is, it doesn't like calling the one variable to create the custom object.
I store the following below in a array. (up to a 100 of these, few below for sample (notice sqlcolumn2 is skipped for example)).
sqlcolumn1 = if ([string]::IsNullOrEmpty($obj.P1) -eq $true) {$null} else {"$obj.P1"}
sqlcolumn3 = if ([string]::IsNullOrEmpty($obj.P2) -eq $true) {$null} else {"$obj.P2"}
sqlcolumn4 = if ([string]::IsNullOrEmpty($obj.P3) -eq $true) {$null} else {"$obj.P3"}
sqlcolumn5 = if ([string]::IsNullOrEmpty($obj.P4) -eq $true) {$null} else {"$obj.P4"}
Here is the array:
foreach($line in $Final)
{
$DataRow = "$($line."TableColumnName") = if ([string]::IsNullOrEmpty(`$obj.$($line."PName")) -eq `$true) {`$null} else {`"`$obj.$($line."PName")`"}"
$DataArray += $DataRow
}
I then try to add it to a final array where I would want this to be looped through for each row of data after which I would perform the insert from the array. Even though the "string" value in the array above is correct if it were hand coded, I can't get it to recognize the rows and run.
foreach ($obj in $data2)
{
$test = [PSCustomObject] #{
$DataArray = Invoke-Expression $DataArray
}
If I just type $DataArray, it doesn't like this because it wants the = sign which I already have built into the string.
Is what I am trying to do even possible.
I was attempting to template out various different ways we receive this data, where some people send us 30 of the 100 columns, other more or less, and no one person using the exact columns to cut down on individual scripts for everything.
Adding more code:
Function ArrayCompare() {
[CmdletBinding()]
PARAM(
[Parameter(Mandatory=$True)]$Array1,
[Parameter(Mandatory=$True)]$A1Match,
[Parameter(Mandatory=$True)]$Array2,
[Parameter(Mandatory=$True)]$A2Match)
$Hash = #{}
foreach ($Data In $Array1) {
$Hash[$Data.$A1Match] += ,$Data
}
foreach ($Data In $Array2) {
$Hash[$Data.$A2Match] += ,$Data
}
foreach ($KeyValue In $Hash.GetEnumerator()){
$Match1, $Match2 = $KeyValue.Value.Where( {$_.$A1Match}, 'Split')
[PSCustomObject]#{
MatchValue = $KeyValue.Key
A1Matches = $Match1.Count
A2Matches = $Match2.Count
TablePosition = [int]$Match2.TablePosition
TableColumnName = $Match2.TableColumnName
# PName is the P(##) that is a generic ascending column value back to import-excel module. ColumnA = P1, ColumnB = P2 etc..until no data is detected. Allows flexibility and not having to know how many columns there are
PName = $Match1.Name}
}
}
$Server = 'ServerName'
$Catalog = 'DBName'
$DestinationTable = 'ImportIntoTableName'
$FileIdentifierID = 10
$FileName = 'Test.xlsx'
$FilePath = 'C:\'
$FullFilePath = $FilePath + $FileName
$data = Import-Excel -Path $FullFilePath -NoHeader -StartRow 1 # Import-
Excel Module for working with xlsx excel files
$data2 = Import-Excel -Path $ullFilePath -NoHeader -StartRow 2 # Import-
Excel Module for working with xlsx excel files
$ExpectedHeaderArray = #()
$HeaderArray = #()
$DataArray = #()
$HeaderDetect = #()
$HeaderDetect = $data | Select-Object -First 1 # Header Row In File
$HeaderDetect |
ForEach-Object {
$ColumnValue = $_
$ColumnValue |
Get-Member -MemberType *Property |
Select-Object -ExpandProperty Name |
ForEach-Object {
$HeaderValues = [PSCustomObject]#{
Name = $_
Value = $ColumnValue.$_}
$HeaderArray += $HeaderValues
}
}
# Query below provides a list of all expected file headers and the table
column name they map to
$Query = "SELECT TableColumnName, FileHeaderName, TablePosition FROM
dbo.FileHeaders WHERE FileIdentifierID = $($FileIdentifierID)"
$ds = Invoke-Sqlcmd -ServerInstance $Server -Database $Catalog -Query $Query
-OutputAs DataSet
$ExpectedHeaderArray = foreach($Row in $ds.Tables[0].Rows)
{
new-object psObject -Property #{
TableColumnName = "$($row.TableColumnName)"
FileHeaderName = "$($row.FileHeaderName)"
TablePosition = "$($row.TablePosition)"
}
}
#Use Function Above
#Bring it together so we know what P(##) goes with which header in file/mapped to table column name
$Result = ArrayCompare -Array1 $HeaderArray -A1Match Value -Array2 $ExpectedHeaderArray -A2Match FileHeaderName
$Final = $Result | sort TablePosition
foreach($Line in $Final)
{
$DataRow = "$($Line."TableColumnName") = if ([string]::IsNullOrEmpty(`$obj.$($Line."PName")) -eq `$true) {`$null} else {`"`$obj.$($Line."PName"))`"}"
$DataArray += $DataRow
}
# The output below is what the code inside the last array would be that I would use to import into excel.
# The goal is to be dynamic and match headers in the file to the stored header value and import into a table (mapped from header column to table column name)
# The reason for this is before I was here, there were many different "versions" of a layout that was given out. In the end, it is all one in the same
# but some send all 100 columns, some only send a handful, some send 80 etc. I am trying to have everything flow through here vs. 60+ pieces of code/stored procedures/ssis packs
Write-Output $DataArray
# Output Sample -- Note how in the sample, P2 and subsequent skip SQLColumn2 because P2 maps to the header value of position 3 in the sql table and each after is one off.
# In this example, SqlColumn2 would not be populated
# SqlColumn1 = if ([string]::IsNullOrEmpty($obj.P1) -eq $true) {$null} else {"$obj.P1"}
# SqlColumn3 = if ([string]::IsNullOrEmpty($obj.P2) -eq $true) {$null} else {"$obj.P2"}
# SqlColumn4 = if ([string]::IsNullOrEmpty($obj.P3) -eq $true) {$null} else {"$obj.P3"}
# SqlColumn5 = if ([string]::IsNullOrEmpty($obj.P4) -eq $true) {$null} else {"$obj.P4"}
# I know this doesn't work. This is where I'm stuck, how to build an array now off of this output from above
foreach ($obj in $data2)
{
$test = [PSCustomObject] #{
$DataArray = Invoke-Expression $DataArray}
}
I'm gong to re-state your question first, just to make sure I understand it properly (it's possible I don't!)...
You've got an excel file that looks something like this:
+---+---------+---------+---------+
| | A | B | C |
+---+---------+---------+---------+
| 1 | HeaderA | HeaderB | HeaderC |
+---+---------+---------+---------+
| 2 | Value P | Value Q | Value R |
+---+---------+---------+---------+
| 3 | Value S | Value T | Value U |
+---+---------+---------+---------+
You've also got a database table which looks like this:
+---------+---------+---------+---------+
+ ColumnW | ColumnX | ColumnY | ColumnZ |
+---------+---------+---------+---------+
+ ....... | ....... | ....... | ....... |
+---------+---------+---------+---------+
and a column mapping table like this (note, ColumnX isn't mapped in this example):
+-----------------+----------------+---------------+
| TableColumnName | FileHeaderName | TablePosition |
+-----------------+----------------+---------------+
| ColumnW | HeaderA | 1 |
+-----------------+----------------+---------------+
| ColumnY | HeaderB | 2 |
+-----------------+----------------+---------------+
| ColumnZ | HeaderC | 3 |
+-----------------+----------------+---------------+
You want to insert the values from the spreadsheet into the database table, using the data in your mapping table so you get this:
+---------+---------+---------+---------+
+ ColumnW | ColumnX | ColumnY | ColumnZ |
+---------+---------+---------+---------+
+ Value P | null | Value Q | Value R |
+---------+---------+---------+---------+
+ Value S | null | Value T | Value U |
+---------+---------+---------+---------+
So let's load the spreadsheet (letting the header row generate meaningful property names this time):
$data = Import-Excel -Path ".\MySpreadsheet.xlsx";
write-host ($data | ft | out-string);
# HeaderA HeaderB HeaderC
# ------- ------- -------
# Value P Value Q Value R
# Value S Value T Value U
and get your column mapping data (I'm programmatically creating an in-memory dataset, but you obviously read yours from your database instead):
$mappings = new-object System.Data.DataTable;
$null = $mappings.Columns.Add("TableColumnName", [string]);
$null = $mappings.Columns.Add("FileHeaderName", [string]);
$null = $mappings.Columns.Add("TablePosition", [int]);
#(
#{ "TableColumnName"="ColumnW"; "FileHeaderName"="HeaderA"; "TablePosition"=1 },
#{ "TableColumnName"="ColumnY"; "FileHeaderName"="HeaderB"; "TablePosition"=2 },
#{ "TableColumnName"="ColumnZ"; "FileHeaderName"="HeaderC"; "TablePosition"=3 }
) | % {
$row = $mappings.NewRow();
$row.TableColumnName = $_.TableColumnName;
$row.FileHeaderName = $_.FileHeaderName;
$row.TablePosition = $_.TablePosition;
$mappings.Rows.Add($row);
}
$ds = new-object System.Data.DataSet;
$ds.Tables.Add($mappings);
write-host ($ds.Tables[0] | ft | out-string)
# TableColumnName FileHeaderName TablePosition
# --------------- -------------- -------------
# ColumnW HeaderA 1
# ColumnY HeaderB 2
# ColumnZ HeaderC 3
Now we can build the "mapped" objects:
$values = #();
foreach( $row in $data )
{
$properties = [ordered] #{};
foreach( $mapping in $mappings )
{
$properties.Add($mapping.TableColumnName, $row."$($mapping.FileHeaderName)");
}
$values += new-object PSCustomObject -Property $properties;
}
write-host ($values | ft | out-string)
# ColumnW ColumnY ColumnZ
# ------- ------- -------
# Value P Value Q Value R
# Value S Value T Value U
The tricksy bit is $properties.Add($mapping.TableColumnName, $row."$($mapping.FileHeaderName)"); - basically, you can access object properties in PowerShell using a dotted string literal or variable (I'm not sure of the exact feature name) - e.g.
PS> $myValue = new-object PSCustomObject -Property #{ "aaa"="bbb"; "ccc"="ddd" }
PS> $myValue."aaa"
bbb
PS> $myProperty = "aaa"
PS> $myValue.$myProperty
"bbb"
so $row."$($mapping.FileHeaderName)" is an expression that evaluates to the value of the property of $row named in $mapping.FileHeaderName.
And then finally you can insert the objects into your database using your existing process...
Note that I couldn't quite work out what your ArrayCompare is actually doing so it's possible the above doesn't solve your problem 100%, but it's hopefully close enough that you can either work the difference out yourself, or leave a comment with where it differs from your desired solution.
Hope this helps.
Two comma separated item added in array list and I would like to group them to count the total.
$list_distinct = [System.Collections.ArrayList]#()
$list_distinct.Add("Site A,Item A")
$list_distinct.Add("Site A,Item A")
$list_distinct.Add("Site A,Item B")
$list_distinct.Add("Site B,Item C")
$list_distinct.Add("Site B,Item D")
$list_distinct.Add("Site B,Item D")
Tried this:
$test = $list_distinct | Group-Object Values
The result shows Count (the whole total), Name(empty) and Group (the whole added items).
Any way to fix this? Or is there any better method?
Desired output example:
Site | Item | Count
Site A | Item A | 2
Site A | Item B | 1
Site B | Item C | 1
Site B | Item D | 2
Neither the ArrayList object nor its elements have a property Values. Non-existent properties are expanded to an empty result, so all of your values are grouped under the same (empty) name.
Change this
$list_distinct | Group-Object Values
into this
$list_distinct | Group-Object
and the problem will disappear.
For your desired output you will also need to split the values and create new (custom) objects:
$list_distinct | Group-Object | ForEach-Object {
$site, $item = $_.Name -split ','
New-Object -Type PSObject -Property #{
'Site' = $site
'Item' = $item
'Count' = $_.Count
}
} | Select-Object Site, Item, Count
The trailing Select-Object is to enforce field order since PowerShell hashtables aren't ordered by default.
In PowerShell v3 and newer you can simplify that to
$list_distinct | Group-Object | ForEach-Object {
$site, $item = $_.Name -split ','
[PSCustomObject]#{
'Site' = $site
'Item' = $item
'Count' = $_.Count
}
}
The trailing Select-Object isn't needed here, because the [PSCustomObject] type accelerator implicitly uses an ordered hashtable.
I am trying to iterate through emails, that have a specific subject line, and have the body of matching emails parsed and passed to a hash array. There is an issue with how I am trying to deal with multiple email bodies that is causing an issue.
$ol=New-Object -ComObject Outlook.Application
$ns=$ol.GetNameSpace('MAPI')
$inbox=$ns.GetDefaultFolder(6)
$Goodsubject='Interesting Email'
$mail = $InBox.items |
Where-Object{
$_.Unread -and $_.Subject -match $Goodsubject
#$_.Subject -match $Goodsubject
} |
Sort-Object ReceivedTime -Descending |
Select-Object Subject,ReceivedTime,Body,SenderName
# Select-Object Body
$mail | %{
$_.Body | %{$interestinghash = #{}} {
if ($_ -match "(.*): (.*)") {
$interestinghash[$matches[1]]=$matches[2];
}
}
Assuming the email has 4 line, each with a key: value, and I look at $interestinghash, I will only have the a key value pair for a email. I will not have all 4. That said, I can replace the following line:
$mail | %{
$_.Body | %{$interestinghash = #{}} {
if ($_ -match "(.*): (.*)") {
$interestinghash[$matches[1]]=$matches[2];
}
}
With:
$mail | %{$interestinghash = #{}} {
if ($_ -match "(.*): (.*)") {
$interestinghash[$matches[1]]=$matches[2];
}
}
If I do this it will give me all 4 values but will only look at 1 email. I am not sure what I am doing wrong here.
I have a CSV like below:
location,id
loc1,1234
loc1,1235
loc1,1236
Running $a = Import-CSV C:\File.csv | Group-Object "location" I get the following output:
Count Name Group
----- ---- -----
3 loc1 {#{location=loc1; id=1234}, #{location=loc1; id=1235), #{location=loc1, id=1236}}
I would like to add all ID's to a single group (Using Add-QADGroupMember) but I can't figure out how to get a group of ID's for $loc1. It seems to be be grouping them correctly but I can't seem to parse the output into a single group. E.g $loc1 = 1234,1235,1236 that I can loop through.
Any suggestions would be appreciated!
Group-Object doesn't handle hashtables well, since the keys aren't real properties.
Assuming:
$csv = Import-CSV C:\File.csv
You should be able to do, for example:
$ids = $csv | %{ $_.id }
to get an array of the ID values. You'd probably want to pipe through Get-Unique for location.
If you wanted to get the location for a single ID quickly:
$location = $csv | ?{ $_.id -eq 42 } | %{ $_.location }
If you wanted to get an array of all IDs for a single location quickly (I think this is what you want):
$loc1 = $csv | ?{ $_.location -eq 'loc1' }
For reference, if you wanted to get a hashtable mapping each location to an array of IDs:
$groups = $csv | %{ $_.location } | &{
begin
{
$hash = #{}
}
process
{
$location = $_.location
$hash[$location] = $csv | ?{ $_.location -eq $location }
}
end
{
$hash
}
}
A bit tricky, but this will do it:
Import-Csv C:\File.csv | Group-Object "location" | %{Set-Variable ($_.Name) ($_.Group | Select-Object -ExpandProperty id)}
After running that, $loc1, $loc2, etc. will be arrays of all the ids for each location.
And yet another option:
(Import-Csv c:\foo.csv | Group Location -AsHashTable).Loc1 | Foreach {$_.id}
And if you're on V3, you can do this:
(Import-Csv c:\foo.csv | Group Location -AsHashTable).Loc1.Id