Loading two arrays from a CSV - arrays

I have a CSV File that has two columns Employee and Manager and I would like to import them into two Variables for use later in my script. However when I run my code it only captures the last data item in the CSV as the previous one is being over written.
$CSVFiles = Import-CSV "C:\T2\EmployeeManager.csv"
ForEach($CSVFile in $CSVFiles)
{
$Employee = ($CSVFile.Employee); $Manager = ($CSVFile.Manager)
}

$Employee = $CSVFiles | Select -Expand Employee
$Manager = $CSVFiles | Select -Expand Manager

That's because you are overwriting the added data each time the loop runs. IN PowerShell the += appends to an object. Try this -
$Employee = #()
$Manager = #()
$CSVFiles = Import-CSV "C:\T2\EmployeeManager.csv"
ForEach($CSVFile in $CSVFiles)
{
$Employee += $CSVFile.Employee; $Manager += $CSVFile.Manager
}

Vivek Kumar Singh's helpful answer explains the problem with your approach well and offers a solution.
Here's a simpler alternative (PSv3+), given that you're loading the entire CSV file into (custom objects) into memory anyway:
$CSV = Import-CSV "C:\T2\Employee.csv"
$Employees = $CSV.Employee # collect the Employee column values across all input rows
$Managers = $CSV.Manager # ditto for Manager
This approach takes advantage of the PSv3+ member-access enumeration feature.
In PSv2, use iRon's helpful solution.
Comparing the performance of the solutions:
The member-enumeration solution in this answer is fastest,
followed by iRon's Select -Expand solution
with Vivek's foreach loop being the slowest by far, not least because use of += to (conceptually) extend an array requires creating a new instance behind the scenes in every iteration.

This ended up being my final coding to accomplish my end goal. I want to thank all the posters I used a little learning from all of you to come to the solution I wanted.
$EmployeeLists = #()
$ManagerLists = #()
$Employees = #()
$Managers = #()
$CSVFiles = Import-CSV "C:\T2\EmployeeManager.csv"
ForEach($CSVFile in $CSVFiles)
{
$EmployeeLists += ($CSVFile.Employee)
}
ForEach($CSVFile in $CSVFiles)
{
$ManagerLists += ($CSVFile.Manager)
}
$Employees += ForEach ($EmployeeList in $EmployeeLists) { Get-ADUser -Properties * -Filter { DisplayName -like $EmployeeList } | Select SamAccountName -ExpandProperty SamAccountName }
$Managers += ForEach ($ManagerList in $ManagerLists) { Get-ADUser -Properties * -Filter { DisplayName -like $ManagerList } | Select SamAccountName -ExpandProperty SamAccountName }
$EmployeeLists
"`n`n`n"
$Employees

Related

Powershell variable doesn't contain all the objects

I got the following variable $listofusers which returns the below objects in two columns:
SourceUser DestinationUser
---------- ---------------
username1#legacy.company.corp username1#modern.company.corp
username2#legacy.company.corp username2#modern.company.corp
username3#legacy.company.corp username3#modern.company.corp
username4#legacy.company.corp username4#modern.company.corp
I now need to process this list of users in a foreach loop. I have tried so far the following but without luck yet:
$Results = ForEach ($User in $listofusers) {
Write-Host "Processing SourceUser $($User.SourceUser)"
Write-Host "Processing DestinationUser $($User.DestinationUser)"
#Assign the content to variables
$SourceUsers = $User.SourceUser
$DestinationUsers = $User.DestinationUser
}
It only returns me the last line of the objects:
$SourceUsers
RETURN ONLY: username4#legacy.company.corp
$DestinationUsers
RETURN ONLY: username4#modern.company.corp
How can I add all the objects in the variable $listofusers for further processing?
UPDATE:
I am trying to achieve the following that's why I have broken the association in listofusers
$SourceUser = #()
$DestinationUser = #()
$Results = ForEach ($User in $listofusers)
{
Write-Host "Processing SourceUser $($User.SourceUser)"
Write-Host "Processing DestinationUser $($User.DestinationUser)"
#Assign the content to variables
$SourceUser += $User.SourceUser
$DestinationUser += $User.DestinationUser
#Cannot get that variables working yet
$sourceusername, $sourcedomain = $SourceUser -split ("#")
$DestinationUsername, $destinationDomain = $DestinationUser -split ("#")
$SourceAccount = Get-ADUser $sourceusername -server $sourcedomain -Properties objectSid
$TargetAccount = Get-ADUser $DestinationUsername -Server $destinationDomain
}
Is there any better way to achieve that and get those variables to that point?
NEW UPDATE:
The purpose of the script would be to achieve the following cmdlets for processing ad objects:
#get the objectSid of the source account
$objectSid = $SourceAccount.objectSid
#copy source account objectSid to target account msExchMasterAccountSid
$TargetAccount | Set-ADUser -Replace #{"msExchMasterAccountSid"=$objectSid}
#enable targetaccount
$TargetAccount | Enable-ADAccount
#disable the source account
$SourceAccount | Disable-ADAccount
#move the migrated user into prod OU
$TargetAccount | Move-ADObject -TargetPath "ou=test,dc=contoso,dc=com"
Thanks
here is a demo of the concept i was trying to get across. [grin] it keeps the association of the objects in your CSV in the original object for as long as possible. the code has NOT been tested since i have no AD access.
what it does ...
fakes reading in a CSV file
when you are ready to use real data, replace the entire "region" with a call to Import-CSV.
iterates thru the list
builds a splat of the parameters for the AD calls
see Get-Help about_Splatting for more info on that wonderfully useful idea.
calls Get-AdUser with each to the Source/Target user data sets
stores the above
uses the stored account info to ...
== replace the .objectSid of the Target account
== enable the Target account
== disable the Source account
== Move the Target account to the desired OU
the hard coded OU could be set with a variable to make this a tad more flexible. however, this seems to be a one-off operation - so there is likely no benefit.
if you want to add logging, do so in the same loop.
there is no error handling, either. that likely should be added with a try/catch around each AD call & logging of both success and failure.
the code ...
#region >>> fake reading in a CSV file
# in real life, use Import-CSV
$UserList = #'
SourceUser, DestUser
ABravo#Old.com, ABravo#NewDomain.com
BCharlie#Old.com, BCharlie#NewDomain.com
CDelta#Old.com, CDelta#NewDomain.com
DEcho#Old.com, DEcho#NewDomain.com
EFoxtrot#Old.com, EFoxtrot#NewDomain.com
'# | ConvertFrom-Csv
#endregion >>> fake reading in a CSV file
ForEach ($UL_Item in $UserList)
{
Write-Host 'Processing ...'
Write-Host (' SourceUser {0}' -f $UL_Item.SourceUser)
Write-Host (' DestinationUser {0}' -f $UL_Item.DestUser)
Write-Host '__ Source Account __'
$GADU_Params_1 = [ordered]#{
Identity = $UL_Item.SourceUser.Split('#')[0]
Server = $UL_Item.SourceUser.Split('#')[1]
Properties = 'objectSid'
}
$SourceAccount = Get-ADUser #GADU_Params_1
Write-Host '__ Target Account __'
$GADU_Params_2 = [ordered]#{
Identity = $UL_Item.DestUser.Split('#')[0]
Server = $UL_Item.DestUser.Split('#')[1]
}
$TargetAccount = Get-ADUser #GADU_Params_2
Write-Host 'Making changes ...'
# all these piped objects are slower than making _direct_ calls
# however, i don't have any way to test the code, so i can't use what likely is faster
# something like >>>
# Set-AdUser -Identity $TargetAccount -Replace #{
# 'msExchMasterAccountSid' = $objectSid
# }
# note that i also replaced the unneeded _double_ quotes with the safer _single_ quotes
$TargetAccount |
Set-AdUser -Replace #{
'msExchMasterAccountSid' = $SourceAccount.objectSid
}
$TargetAccount |
Enable-AdAccount
$SourceAccount |
Disable-AdAccount
$TargetAccount |
Move-AdObject -TargetPath 'ou=test,dc=contoso,dc=com'
Write-Host '=' * 30
Write-Host ''
}
no output shown since i can't actually run this AD stuff. [grin]
$SourceUsers and $DestinationUsers contain only the last ones becasue youa re replacing the value on each foreach iteration.
if you want it to separate the properties try this:
$SourceUsers = $User | select SourceUser -ExpandProperty SourceUser
$DestinationUsers = $User | select DestinationUser -ExpandProperty DestinationUser
That will create a collection of only those strings. you wont be able to access those values by property anymore, meaning that is a simple String[] after the -ExpandProperty.
$SourceUsers = #()
$DestinationUsers = #()
$Results = ForEach ($User in $listofusers) {
Write-Host "Processing SourceUser $($User.SourceUser)"
Write-Host "Processing DestinationUser $($User.DestinationUser)"
#Assign the content to variables
$SourceUsers += $User.SourceUser
$DestinationUsers += $User.DestinationUser
}
$SourceUsers = #() and $DestinationUsers = #() creates two empty
arrays which we will use in the loop
+= is an assignment operator which enables us to assign more than
one value to a variable. According to the documentation: Increases
the value of a variable by the specified value, or appends the
specified value to the existing value.

Generically Transpose PowerShell Array Rows into Columns

There are quite a few posts on SO that address PowerShell transposition. However, most of the code is specific to the use case or addresses data being gathered from a text/CSV file and does me no good. I'd like to see a solution that can do this work without such specifics and works with arrays directly in PS.
Example data:
Customer Name: SomeCompany
Abbreviation: SC
Company Contact: Some Person
Address: 123 Anywhere St.
ClientID: XXXX
This data is much more complicated, but I can work with it using other methods if I can just get the rows and columns to cooperate. The array things that "Name:" and "SomeCompany" are column headers. This is a byproduct of how the data is gathered and cannot be changed. I'm importing the data from an excel spreadsheet with PSExcel and the spreadsheet format is not changeable.
Desired output:
Customer Name:, Abbreviation:, Company Contact:, Address:, ClientID:
SomeCompany, SC, Some Person, 123 Anywhere St., XXXX
Example of things I've tried:
$CustInfo = Import-XLSX -Path "SomePath" -Sheet "SomeSheet" -RowStart 3 -ColumnStart 2
$b = #()
foreach ($Property in $CustInfo.Property | Select -Unique) {
$Props = [ordered]#{ Property = $Property }
foreach ($item in $CustInfo."Customer Name:" | Select -Unique){
$Value = ($CustInfo.where({ $_."Customer Name:" -eq $item -and
$_.Property -eq $Property })).Value
$Props += #{ $item = $Value }
}
$b += New-Object -TypeName PSObject -Property $Props
}
This does not work because of the "other" data I mentioned. There are many other sections in this particular workbook so the "Select -Unique" fails without error and the output is blank. If I could limit the input to only select the rows/columns I needed, this might have a shot. It appears that while there is a "RowStart" and "ColumnStart" to Import-XLSX, there are no properties for stopping either one.
I've tried methods from the above linked SO questions, but as I said, they are either too specific to the question's data or apply to importing CSV files and not working with arrays.
I was able to resolve this by doing two things:
Removed the extra columns by using the "-Header" switch on the Import-XLSX function to add fake header names and then only select those headers.
$CustInfo = Import-XLSX -Path "SomePath" -Sheet "SomeSheet" -RowStart 2 -ColumnStart 2 -Header 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18 | Select "1","2"
The downside to this is that I had to know how many columns the input data had -- Not dynamic. If anyone can provide a solution to this issue, I'd be grateful.
Flipped the columns and headers with a simple foreach loop:
$obj = [PSCustomObject]#{}
ForEach ($item in $CustInfo) {
$value = $null
$name = $null
if ($item."2") { [string]$value = $item."2" }
if ($item."1") { [string]$name = $item."1" }
if ($value -and $name) {
$obj | Add-Member -NotePropertyName $name -NotePropertyValue $value
}
}
I had to force string type on the property names and values because the zip codes and CustID was formatting as an Int32. Otherwise, this does what I need.

One element containing hashes instead of multiple elements - how to fix?

I am trying to parse robocopy log files to get file size, path, and date modified. I am getting the information via regex with no issues. However, for some reason, I am getting an array with a single element, and that element contains 3 hashes. My terminology might be off; I am still learning about hashes. What I want is a regular array with multple elements.
Output that I am getting:
FileSize FilePath DateTime
-------- -------- --------
{23040, 36864, 27136, 24064...} {\\server1\folder\Test File R... {2006/03/15 21:08:01, 2010/12...
As you can see, there is only one row, but that row contains multiple items. I want multiple rows.
Here is my code:
[regex]$Match_Regex = "^.{13}\s\d{4}/\d{2}/\d{2}\s\d{2}:\d{2}:\d{2}\s.*$"
[regex]$Replace_Regex = "^\s*([\d\.]*\s{0,1}\w{0,1})\s(\d{4}\/\d{2}\/\d{2}\s\d{2}:\d{2}:\d{2})\s(.*)$"
$MainContent = New-Object System.Collections.Generic.List[PSCustomObject]
Get-Content $Path\$InFile -ReadCount $Batch | ForEach-Object {
$FileSize = $_ -match $Match_Regex -replace $Replace_Regex,('$1').Trim()
$DateTime = $_ -match $Match_Regex -replace $Replace_Regex,('$2').Trim()
$FilePath = $_ -match $Match_Regex -replace $Replace_Regex,('$3').Trim()
$Props = #{
FileSize = $FileSize;
DateTime = $DateTime;
FilePath = $FilePath
}
$Obj = [PSCustomObject]$Props
$MainContent.Add($Obj)
}
$MainContent | % {
$_
}
What am I doing wrong? I am just not getting it. Thanks.
Note: This needs to be as fast as possible because I have to process millions of lines, which is why I am trying System.Collections.Generic.List.
I think the problem is that for what you're doing you actually need two foreach-object loops. Using Get-Content with -Readcount is going to give you an array of arrays. Use the -Match in the first Foreach-Object to filter out the records that match in each array. That's going to give you an array of the matched records. Then you need to foreach through that array to create one object for each record:
[regex]$Match_Regex = "^.{13}\s\d{4}/\d{2}/\d{2}\s\d{2}:\d{2}:\d{2}\s.*$"
[regex]$Replace_Regex = "^\s*([\d\.]*\s{0,1}\w{0,1})\s(\d{4}\/\d{2}\/\d{2}\s\d{2}:\d{2}:\d{2})\s(.*)$"
$MainContent =
Get-Content $Path\$InFile -ReadCount $Batch |
ForEach-Object {
$_ -match $Match_Regex |
ForEach-Object {
$FileSize = $_ -replace $Replace_Regex,('$1').Trim()
$DateTime = $_ -replace $Replace_Regex,('$2').Trim()
$FilePath = $_ -replace $Replace_Regex,('$3').Trim()
[PSCustomObject]#{
FileSize = $FileSize
DateTime = $DateTime
FilePath = $FilePath
}
}
}
You don't really need to use the collection as an accumulator, just output PSCustomObjects, and let them accumulate in the result variable.

Filtering Search Results Against a CSV in Powershell

Im interested in some ideas on how one would approach coding a search of a filesystem for files that match any entries contained in a master CSV file. I have a function to search the filesystem, but filtering against the CSV is proving harder than I expect. I have a csv with headers in it for Name & IPaddr:
#create CSV object
$csv = import-csv filename.csv
#create filter object containing only Name column
$filter = $csv | select-object Name
#Now run the search function
SearchSubfolders | where {$_.name -match $filter} #returns no results
I guess my question is this: Can I filter against an array within a pipeline like this???
You need a pair of loops:
#create CSV object
$csv = import-csv filename.csv
#Now run the search function
#loop through the folders
foreach ($folder in (SearchSubfolders)) {
#check that folder against each item in the csv filter list
#this sets up the loop
foreach ($Filter in $csv.Name) {
#and this does the checking and outputs anything that is matched
If ($folder.name -match $Filter) { "$filter" }
}
}
Usually CSVs are 2-dimensional data structures, so you can't use them directly for filtering. You can convert the 2-dimensional array into a 1-dimensional array, though:
$filter = Import-Csv 'C:\path\to\some.csv' | % {
$_.PSObject.Properties | % { $_.Value }
}
If the CSV has just a single column, the "mangling" can be simplified to this (replace Name with the actual column name):
$filter = Import-Csv 'C:\path\to\some.csv' | % { $_.Name }
or this:
$filter = Import-Csv 'C:\path\to\some.csv' | select -Expand Name
Of course, if the CSV has just a single column, it would've been better to make it a flat list right away, so it could've been imported like this:
$filter = Get-Content 'C:\path\to\some.txt'
Either way, with the $filter prepared, you can apply it to your input data like this:
SearchSubFolders | ? { $filter -contains $_.Name } # ARRAY -contains VALUE
The -match operator won't work, because it compares a value (left operand) against a regular expression (right operand).
See Get-Help about_Comparison_Operators for more information.
Another option is to create a regex from the filename collection and use that to filter for all the filenames at once:
$filenames = import-csv filename.csv |
foreach { $_.name }
[regex]$filename_regex = ‘(?i)^(‘ + (($filenames | foreach {[regex]::escape($_)}) –join “|”) + ‘)$’
$SearchSubfolders |
where { $_.name -match $filename_regex }
You can use Compare-Object to do this pretty easily if you are matching the actual Names of the files to names in the list. An example:
$filter = import-csv files.csv
ls | Compare-Object -ReferenceObject $filter -IncludeEqual -ExcludeDifferent -Property Name
This will print the files in the current directory that match the any Name in files.csv. You could also print only the different ones by dropping -IncludeEqual and -ExcludeDifferent flags. If you need full regex matching you will have to loop through each regex in the csv and see if it is a match.
Here's any alternate solution that uses regular expression filters. Note that we will create and cache the regex instances so we don't have to rely on the runtime's internal cache (which defaults to 15 items). First we have a useful helper function, Test-Any that will loop through an array of items and stop if any of them satisfies a criteria:
function Test-Any() {
param(
[Parameter(Mandatory=$True,ValueFromPipeline=$True)]
[object[]]$Items,
[Parameter(Mandatory=$True,Position=2)]
[ScriptBlock]$Predicate)
begin {
$any = $false
}
process {
foreach($item in $items) {
if ($predicate.Invoke($item)) {
$any = $true
break
}
}
}
end { $any }
}
With this, the implementation is relatively simple:
$filters = import-csv files.csv | foreach { [regex]$_.Name }
ls -recurse | where { $name = $_.Name; $filters | Test-Any { $_.IsMatch($name) } }
I ended up using a 'loop within a loop' construct to get this done after much trial and error:
#the SearchSubFolders function was amended to force results in a variable, SearchResults
$SearchResults2 = #()
foreach ($result in $SearchResults){
foreach ($line in $filter){
if ($result -match $line){
$SearchResults2 += $result
}
}
}
This works great after collapsing my CSV file down to a text-based array containing only the necessary column data from that CSV. Much thanks to Ansgar Wiechers for assisting me with that particular thing!!!
All of you presented viable solutions, some more complex than I cared for, nevertheless if I could mark multiple answers as correct, I would!! I chose the correct answer based on not only correctness but also simplicity.....

Powershell - Searching and Comparing Arrays with Quest CMDlets

Trying to determine if there are any user folders on the network that don’t have an associated user account. All results return "Missing" when the majority should return "Found". Any ideas?
$Dir = "\\ServerName\Share\"
$FolderList = Get-ChildItem($Dir) | where {$_.psIsContainer -eq $true}
$UserList = get-qaduser -sizelimit 0 | select LogonName
foreach ($Folder in $FolderList)
{
if ($UserList -contains $Folder.name)
{
"Found: " + $Folder.name
}
Else
{
"Missing: " + $Folder.name
}
}
How about trying a slightly different approach that uses a hashtable (which offers exceptionally fast lookups of keys):
$users = #{}
Get-QADUser -sizelimit 0 | Foreach {$users["$($_.LogonName)"] = $true}
$dir = "\\ServerName\Share\"
Get-ChildItem $dir | Where {$_.PSIsContainer -and !$users["$($_.Name)"]}
If the folder name doesn't exactly match the LogonName, then as EBGreen notes, you will need to adjust the key ($users["$($.LogonName)"]) or the folder name when you use it to index the hashtable (!$users["$($.Name)"]).
-contains will match if the item in the collection is identical to what you are testing so be sure that the $Folder.Name is exactly the same as LogonName. Usually it wouldn't be. Most companies would have the folder name be foo$ for a user named foo.

Resources