Generically Transpose PowerShell Array Rows into Columns - arrays

There are quite a few posts on SO that address PowerShell transposition. However, most of the code is specific to the use case or addresses data being gathered from a text/CSV file and does me no good. I'd like to see a solution that can do this work without such specifics and works with arrays directly in PS.
Example data:
Customer Name: SomeCompany
Abbreviation: SC
Company Contact: Some Person
Address: 123 Anywhere St.
ClientID: XXXX
This data is much more complicated, but I can work with it using other methods if I can just get the rows and columns to cooperate. The array things that "Name:" and "SomeCompany" are column headers. This is a byproduct of how the data is gathered and cannot be changed. I'm importing the data from an excel spreadsheet with PSExcel and the spreadsheet format is not changeable.
Desired output:
Customer Name:, Abbreviation:, Company Contact:, Address:, ClientID:
SomeCompany, SC, Some Person, 123 Anywhere St., XXXX
Example of things I've tried:
$CustInfo = Import-XLSX -Path "SomePath" -Sheet "SomeSheet" -RowStart 3 -ColumnStart 2
$b = #()
foreach ($Property in $CustInfo.Property | Select -Unique) {
$Props = [ordered]#{ Property = $Property }
foreach ($item in $CustInfo."Customer Name:" | Select -Unique){
$Value = ($CustInfo.where({ $_."Customer Name:" -eq $item -and
$_.Property -eq $Property })).Value
$Props += #{ $item = $Value }
}
$b += New-Object -TypeName PSObject -Property $Props
}
This does not work because of the "other" data I mentioned. There are many other sections in this particular workbook so the "Select -Unique" fails without error and the output is blank. If I could limit the input to only select the rows/columns I needed, this might have a shot. It appears that while there is a "RowStart" and "ColumnStart" to Import-XLSX, there are no properties for stopping either one.
I've tried methods from the above linked SO questions, but as I said, they are either too specific to the question's data or apply to importing CSV files and not working with arrays.

I was able to resolve this by doing two things:
Removed the extra columns by using the "-Header" switch on the Import-XLSX function to add fake header names and then only select those headers.
$CustInfo = Import-XLSX -Path "SomePath" -Sheet "SomeSheet" -RowStart 2 -ColumnStart 2 -Header 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18 | Select "1","2"
The downside to this is that I had to know how many columns the input data had -- Not dynamic. If anyone can provide a solution to this issue, I'd be grateful.
Flipped the columns and headers with a simple foreach loop:
$obj = [PSCustomObject]#{}
ForEach ($item in $CustInfo) {
$value = $null
$name = $null
if ($item."2") { [string]$value = $item."2" }
if ($item."1") { [string]$name = $item."1" }
if ($value -and $name) {
$obj | Add-Member -NotePropertyName $name -NotePropertyValue $value
}
}
I had to force string type on the property names and values because the zip codes and CustID was formatting as an Int32. Otherwise, this does what I need.

Related

Powershell variable doesn't contain all the objects

I got the following variable $listofusers which returns the below objects in two columns:
SourceUser DestinationUser
---------- ---------------
username1#legacy.company.corp username1#modern.company.corp
username2#legacy.company.corp username2#modern.company.corp
username3#legacy.company.corp username3#modern.company.corp
username4#legacy.company.corp username4#modern.company.corp
I now need to process this list of users in a foreach loop. I have tried so far the following but without luck yet:
$Results = ForEach ($User in $listofusers) {
Write-Host "Processing SourceUser $($User.SourceUser)"
Write-Host "Processing DestinationUser $($User.DestinationUser)"
#Assign the content to variables
$SourceUsers = $User.SourceUser
$DestinationUsers = $User.DestinationUser
}
It only returns me the last line of the objects:
$SourceUsers
RETURN ONLY: username4#legacy.company.corp
$DestinationUsers
RETURN ONLY: username4#modern.company.corp
How can I add all the objects in the variable $listofusers for further processing?
UPDATE:
I am trying to achieve the following that's why I have broken the association in listofusers
$SourceUser = #()
$DestinationUser = #()
$Results = ForEach ($User in $listofusers)
{
Write-Host "Processing SourceUser $($User.SourceUser)"
Write-Host "Processing DestinationUser $($User.DestinationUser)"
#Assign the content to variables
$SourceUser += $User.SourceUser
$DestinationUser += $User.DestinationUser
#Cannot get that variables working yet
$sourceusername, $sourcedomain = $SourceUser -split ("#")
$DestinationUsername, $destinationDomain = $DestinationUser -split ("#")
$SourceAccount = Get-ADUser $sourceusername -server $sourcedomain -Properties objectSid
$TargetAccount = Get-ADUser $DestinationUsername -Server $destinationDomain
}
Is there any better way to achieve that and get those variables to that point?
NEW UPDATE:
The purpose of the script would be to achieve the following cmdlets for processing ad objects:
#get the objectSid of the source account
$objectSid = $SourceAccount.objectSid
#copy source account objectSid to target account msExchMasterAccountSid
$TargetAccount | Set-ADUser -Replace #{"msExchMasterAccountSid"=$objectSid}
#enable targetaccount
$TargetAccount | Enable-ADAccount
#disable the source account
$SourceAccount | Disable-ADAccount
#move the migrated user into prod OU
$TargetAccount | Move-ADObject -TargetPath "ou=test,dc=contoso,dc=com"
Thanks
here is a demo of the concept i was trying to get across. [grin] it keeps the association of the objects in your CSV in the original object for as long as possible. the code has NOT been tested since i have no AD access.
what it does ...
fakes reading in a CSV file
when you are ready to use real data, replace the entire "region" with a call to Import-CSV.
iterates thru the list
builds a splat of the parameters for the AD calls
see Get-Help about_Splatting for more info on that wonderfully useful idea.
calls Get-AdUser with each to the Source/Target user data sets
stores the above
uses the stored account info to ...
== replace the .objectSid of the Target account
== enable the Target account
== disable the Source account
== Move the Target account to the desired OU
the hard coded OU could be set with a variable to make this a tad more flexible. however, this seems to be a one-off operation - so there is likely no benefit.
if you want to add logging, do so in the same loop.
there is no error handling, either. that likely should be added with a try/catch around each AD call & logging of both success and failure.
the code ...
#region >>> fake reading in a CSV file
# in real life, use Import-CSV
$UserList = #'
SourceUser, DestUser
ABravo#Old.com, ABravo#NewDomain.com
BCharlie#Old.com, BCharlie#NewDomain.com
CDelta#Old.com, CDelta#NewDomain.com
DEcho#Old.com, DEcho#NewDomain.com
EFoxtrot#Old.com, EFoxtrot#NewDomain.com
'# | ConvertFrom-Csv
#endregion >>> fake reading in a CSV file
ForEach ($UL_Item in $UserList)
{
Write-Host 'Processing ...'
Write-Host (' SourceUser {0}' -f $UL_Item.SourceUser)
Write-Host (' DestinationUser {0}' -f $UL_Item.DestUser)
Write-Host '__ Source Account __'
$GADU_Params_1 = [ordered]#{
Identity = $UL_Item.SourceUser.Split('#')[0]
Server = $UL_Item.SourceUser.Split('#')[1]
Properties = 'objectSid'
}
$SourceAccount = Get-ADUser #GADU_Params_1
Write-Host '__ Target Account __'
$GADU_Params_2 = [ordered]#{
Identity = $UL_Item.DestUser.Split('#')[0]
Server = $UL_Item.DestUser.Split('#')[1]
}
$TargetAccount = Get-ADUser #GADU_Params_2
Write-Host 'Making changes ...'
# all these piped objects are slower than making _direct_ calls
# however, i don't have any way to test the code, so i can't use what likely is faster
# something like >>>
# Set-AdUser -Identity $TargetAccount -Replace #{
# 'msExchMasterAccountSid' = $objectSid
# }
# note that i also replaced the unneeded _double_ quotes with the safer _single_ quotes
$TargetAccount |
Set-AdUser -Replace #{
'msExchMasterAccountSid' = $SourceAccount.objectSid
}
$TargetAccount |
Enable-AdAccount
$SourceAccount |
Disable-AdAccount
$TargetAccount |
Move-AdObject -TargetPath 'ou=test,dc=contoso,dc=com'
Write-Host '=' * 30
Write-Host ''
}
no output shown since i can't actually run this AD stuff. [grin]
$SourceUsers and $DestinationUsers contain only the last ones becasue youa re replacing the value on each foreach iteration.
if you want it to separate the properties try this:
$SourceUsers = $User | select SourceUser -ExpandProperty SourceUser
$DestinationUsers = $User | select DestinationUser -ExpandProperty DestinationUser
That will create a collection of only those strings. you wont be able to access those values by property anymore, meaning that is a simple String[] after the -ExpandProperty.
$SourceUsers = #()
$DestinationUsers = #()
$Results = ForEach ($User in $listofusers) {
Write-Host "Processing SourceUser $($User.SourceUser)"
Write-Host "Processing DestinationUser $($User.DestinationUser)"
#Assign the content to variables
$SourceUsers += $User.SourceUser
$DestinationUsers += $User.DestinationUser
}
$SourceUsers = #() and $DestinationUsers = #() creates two empty
arrays which we will use in the loop
+= is an assignment operator which enables us to assign more than
one value to a variable. According to the documentation: Increases
the value of a variable by the specified value, or appends the
specified value to the existing value.

Loading two arrays from a CSV

I have a CSV File that has two columns Employee and Manager and I would like to import them into two Variables for use later in my script. However when I run my code it only captures the last data item in the CSV as the previous one is being over written.
$CSVFiles = Import-CSV "C:\T2\EmployeeManager.csv"
ForEach($CSVFile in $CSVFiles)
{
$Employee = ($CSVFile.Employee); $Manager = ($CSVFile.Manager)
}
$Employee = $CSVFiles | Select -Expand Employee
$Manager = $CSVFiles | Select -Expand Manager
That's because you are overwriting the added data each time the loop runs. IN PowerShell the += appends to an object. Try this -
$Employee = #()
$Manager = #()
$CSVFiles = Import-CSV "C:\T2\EmployeeManager.csv"
ForEach($CSVFile in $CSVFiles)
{
$Employee += $CSVFile.Employee; $Manager += $CSVFile.Manager
}
Vivek Kumar Singh's helpful answer explains the problem with your approach well and offers a solution.
Here's a simpler alternative (PSv3+), given that you're loading the entire CSV file into (custom objects) into memory anyway:
$CSV = Import-CSV "C:\T2\Employee.csv"
$Employees = $CSV.Employee # collect the Employee column values across all input rows
$Managers = $CSV.Manager # ditto for Manager
This approach takes advantage of the PSv3+ member-access enumeration feature.
In PSv2, use iRon's helpful solution.
Comparing the performance of the solutions:
The member-enumeration solution in this answer is fastest,
followed by iRon's Select -Expand solution
with Vivek's foreach loop being the slowest by far, not least because use of += to (conceptually) extend an array requires creating a new instance behind the scenes in every iteration.
This ended up being my final coding to accomplish my end goal. I want to thank all the posters I used a little learning from all of you to come to the solution I wanted.
$EmployeeLists = #()
$ManagerLists = #()
$Employees = #()
$Managers = #()
$CSVFiles = Import-CSV "C:\T2\EmployeeManager.csv"
ForEach($CSVFile in $CSVFiles)
{
$EmployeeLists += ($CSVFile.Employee)
}
ForEach($CSVFile in $CSVFiles)
{
$ManagerLists += ($CSVFile.Manager)
}
$Employees += ForEach ($EmployeeList in $EmployeeLists) { Get-ADUser -Properties * -Filter { DisplayName -like $EmployeeList } | Select SamAccountName -ExpandProperty SamAccountName }
$Managers += ForEach ($ManagerList in $ManagerLists) { Get-ADUser -Properties * -Filter { DisplayName -like $ManagerList } | Select SamAccountName -ExpandProperty SamAccountName }
$EmployeeLists
"`n`n`n"
$Employees

Compare Two Objects Properties and Obtain Missing Items

I'm attempting to compare two object properties called "name" and that contain the same type of data, in this case server host names. The objects are not identical AKA do not have the same properties. In short, I'm attempting to compare two lists of server names and determine where (which object) they are missing from.
I'm looking to find the items (server host names) that are missing from each object. When something is found I'm hoping to obtain all related properties for the item in the given object that it was found in. I can do the compare-object successfully, but don't know how to get the results I'm looking for.
I'm thinking two new objects could be created for each, that list the items that were not found in the other object maybe? Or do I somehow reference the previous objects with the output from compare-object and produce some formatted output?
This code currently produces a blank file.
Data Format:
$SNObject:
name,ip,class
server-place.com,10.10.10.10,windows server
$QRObject:
name,date,ip,general,device type
server-place1.com,11.11.11.11,random info,linux server
Code Example:
$compare = compare-object $SNObject $QRObject -property Name |
foreach {
if ($_.sideindicator -eq '<=')
{$_.sideindicator = $PathToQRReport }
if ($_.sideindicator -eq '=>')
{$_.sideindicator = $PathToSNReport}
}
$compare |
select #{l='Value';e={$_.InputObject}},#{l='File';e={$_.SideIndicator}} |
Out-File -FilePath C:\Temp\MissingOutputs1.txt
Ahh... just thought of an alternative that may give you exactly what you're looking for but in a slightly different way:
## Join both arrays into single array and then group it on the property name that has a shared value, 'name'
$all = #()
$group = #($SNObject + $QRObject)
$group | Group-Object -Property name | % {
## Create a custom object that contains all possible properties plus a directionality indicator ('source')
$n = New-Object PSObject -Property #{
'name' = ''
'date' = ''
'ip' = ''
'general' = ''
'platform' = ''
'source' = ''
}
if ($_.Count -eq 1) {
## Loop through the grouped results and determine their source and write properties based off of their source
foreach ($i in $_.Group) {
if (#($i | Get-Member -MemberType NoteProperty | Select-Object -ExpandProperty name) -contains 'date' ) {
## This value came from $QRObject which apparently is the only dataset that contains a date property
$n.source = 'QRObject'
$n.date = $i.date
$n.general = $i.general
$n.platform = $i.'device type'
} else {
## This object does not contain the 'date' property, therefore it came from $SNObject
$n.source = 'SNObject'
$n.platform = $i.class
}
## write out common properties
$n.name = $i.name
$n.ip = $i.ip
## add the custom PSObject back to a master array with all formatted properties
$all += $n
}
}
}
$all | out-whereever-you-want

Export array with "sub-array"

I am currently trying to automate license counting in Office 365 across multiple partner tenants using PowerShell.
My current code (aquired from the internet) with some modifications gives me this output:
Column A Column B Column C
-------- -------- --------
CustA LicA,LicB 1,3
CustB LicA,LicB 7,3
CustC LicA 4
But the output I want from this code is:
Column A Column B Column C
-------- -------- --------
CustA LicA 1
LicB 3
CustB LicA 7
LicB 3
Here is my current code which is exported using Export-Csv -NoType:
$tenantID = (Get-MsolPartnerContract).tenantid
foreach($i in $tenantID){
$tenantName = Get-MsolPartnerInformation -TenantId $i
$tenantLicense = Get-MsolSubscription -TenantId $i
$properties = [ordered]#{
'Company' = ($tenantName.PartnerCompanyName -join ',')
'License' = ($tenantLicense.SkuPartNumber -join ',')
'LicenseCount' = ($tenantLicense.TotalLicenses -join ',')
}
$obj = New-Object -TypeName psobject -Property $properties
Write-Output $obj
}
I have tried this along with several other iterations of code which all fail catastophically:
$properties = [ordered]#{
'Company' = ($tenantName.PartnerCompanyName -join ','),
#{'License' = ($tenantLicense.SkuPartNumber -join ',')},
#{'LicenseCount' = ($tenantLicense.TotalLicenses -join',')}
}
I was thinking about making a "sub-array" $tenantLicense.SkuPartnumber and $tenantLicense.TotalLicenses, but I'm not quite sure how to approach this with appending it to the object or "main-array".
A second loop for each $tenantLIcense should do the trick for you. I don't have access to an environment like yours so I cannot test this.
$tenantID | ForEach-Object{
$tenantName = Get-MsolPartnerInformation -TenantId $_
$tenantLicense = Get-MsolSubscription -TenantId $_
# Make an object for each $tenantLicense
$tenantLicense | ForEach-Object{
$properties = [ordered]#{
'Company' = $tenantName.PartnerCompanyName
'License' = $_.SkuPartNumber
'LicenseCount' = $_.TotalLicenses
}
# Send the new object down the pipe.
New-Object -TypeName psobject -Property $properties
}
}
Since you have multiple $tenantLicenses that have the same company name lets just loop over those and use the same company name in the output. Assuming this worked it would not have the same output as you desired since there no logic to omit company in subsequent rows. I would argue that is it better this way since you can sort the data now with out loss of data / understanding.
Notice I change foreach() to ForEach-Object. This makes it simpler to send object down the pipe.
Without providing the code solution I can say that you need to build up the array.
In terms of programming, you will need to iterate the array ARRAY1 and populate another one ARRAY2with the extra rows. For example if columns A,B are simple value and C is an array of 3 items, then you would add 3 rows in the new table with A,B,C1, A,B,C2 and A,B,C3. On each iteration of the loop you need to calculate all the permutations, for example in your case the ones generated by columnB and columnC.
This should be also possible with pipelining using the ForEach-Object cmdlet but that is more difficult and as you mentioned your relatively new relationship with powershell I would not pursuit this path, unless of coarse you want to learn.

Filtering Search Results Against a CSV in Powershell

Im interested in some ideas on how one would approach coding a search of a filesystem for files that match any entries contained in a master CSV file. I have a function to search the filesystem, but filtering against the CSV is proving harder than I expect. I have a csv with headers in it for Name & IPaddr:
#create CSV object
$csv = import-csv filename.csv
#create filter object containing only Name column
$filter = $csv | select-object Name
#Now run the search function
SearchSubfolders | where {$_.name -match $filter} #returns no results
I guess my question is this: Can I filter against an array within a pipeline like this???
You need a pair of loops:
#create CSV object
$csv = import-csv filename.csv
#Now run the search function
#loop through the folders
foreach ($folder in (SearchSubfolders)) {
#check that folder against each item in the csv filter list
#this sets up the loop
foreach ($Filter in $csv.Name) {
#and this does the checking and outputs anything that is matched
If ($folder.name -match $Filter) { "$filter" }
}
}
Usually CSVs are 2-dimensional data structures, so you can't use them directly for filtering. You can convert the 2-dimensional array into a 1-dimensional array, though:
$filter = Import-Csv 'C:\path\to\some.csv' | % {
$_.PSObject.Properties | % { $_.Value }
}
If the CSV has just a single column, the "mangling" can be simplified to this (replace Name with the actual column name):
$filter = Import-Csv 'C:\path\to\some.csv' | % { $_.Name }
or this:
$filter = Import-Csv 'C:\path\to\some.csv' | select -Expand Name
Of course, if the CSV has just a single column, it would've been better to make it a flat list right away, so it could've been imported like this:
$filter = Get-Content 'C:\path\to\some.txt'
Either way, with the $filter prepared, you can apply it to your input data like this:
SearchSubFolders | ? { $filter -contains $_.Name } # ARRAY -contains VALUE
The -match operator won't work, because it compares a value (left operand) against a regular expression (right operand).
See Get-Help about_Comparison_Operators for more information.
Another option is to create a regex from the filename collection and use that to filter for all the filenames at once:
$filenames = import-csv filename.csv |
foreach { $_.name }
[regex]$filename_regex = ‘(?i)^(‘ + (($filenames | foreach {[regex]::escape($_)}) –join “|”) + ‘)$’
$SearchSubfolders |
where { $_.name -match $filename_regex }
You can use Compare-Object to do this pretty easily if you are matching the actual Names of the files to names in the list. An example:
$filter = import-csv files.csv
ls | Compare-Object -ReferenceObject $filter -IncludeEqual -ExcludeDifferent -Property Name
This will print the files in the current directory that match the any Name in files.csv. You could also print only the different ones by dropping -IncludeEqual and -ExcludeDifferent flags. If you need full regex matching you will have to loop through each regex in the csv and see if it is a match.
Here's any alternate solution that uses regular expression filters. Note that we will create and cache the regex instances so we don't have to rely on the runtime's internal cache (which defaults to 15 items). First we have a useful helper function, Test-Any that will loop through an array of items and stop if any of them satisfies a criteria:
function Test-Any() {
param(
[Parameter(Mandatory=$True,ValueFromPipeline=$True)]
[object[]]$Items,
[Parameter(Mandatory=$True,Position=2)]
[ScriptBlock]$Predicate)
begin {
$any = $false
}
process {
foreach($item in $items) {
if ($predicate.Invoke($item)) {
$any = $true
break
}
}
}
end { $any }
}
With this, the implementation is relatively simple:
$filters = import-csv files.csv | foreach { [regex]$_.Name }
ls -recurse | where { $name = $_.Name; $filters | Test-Any { $_.IsMatch($name) } }
I ended up using a 'loop within a loop' construct to get this done after much trial and error:
#the SearchSubFolders function was amended to force results in a variable, SearchResults
$SearchResults2 = #()
foreach ($result in $SearchResults){
foreach ($line in $filter){
if ($result -match $line){
$SearchResults2 += $result
}
}
}
This works great after collapsing my CSV file down to a text-based array containing only the necessary column data from that CSV. Much thanks to Ansgar Wiechers for assisting me with that particular thing!!!
All of you presented viable solutions, some more complex than I cared for, nevertheless if I could mark multiple answers as correct, I would!! I chose the correct answer based on not only correctness but also simplicity.....

Resources