If you have multiple arrays of different length, how can you export these to a single csv in powershell?
Array1 = 1,2,3
Array2 = Bob,smithy,Alex,Jeremy
Array3 = yes,no
Output CSV
Number Name Valid
———————————————————
1 Bob Yes
2 Smithy no
3 Alex
Jeremy
Basically each array would be in its own header column.
Tried lines like
Array1 | Select-Object Number | export-csv -Path C:\Path
This works for singular arrays to singular csv files
But if I try
Array1, Array2, Array3 | Select-Object Number, Name, Valid | export-csv -Path C:\Path
I just get the header names and no values in the columns
One way to do it is with a for loop.
$Array1 = 1, 2, 3
$Array2 = 'Joe Bloggs', 'John Doe', 'Jane Doe'
$Array3 = 'Yes', 'No'
$export = for($i = 0; $i -lt [Linq.Enumerable]::Max([int[]] ($Array1.Count, $Array2.Count, $Array3.Count)); $i++) {
[pscustomobject]#{
Number = $Array1[$i]
Name = $Array2[$i]
Valid = $Array3[$i]
}
}
$export | Export-Csv path\to\csv.csv -NoTypeInformation
Another example using a function, the logic is more or less the same except that there is more overhead involved, since this function can handle an indefinite amount of arrays coming from the pipeline.
function Join-Array {
[CmdletBinding()]
param(
[parameter(ValueFromPipeline, Mandatory)]
[object[]] $InputObject,
[parameter(Mandatory)]
[string[]] $Columns
)
begin {
$inputDict = [ordered]#{}
$index = 0
}
process {
try {
if($MyInvocation.ExpectingInput) {
return $inputDict.Add($Columns[$index++], $InputObject)
}
foreach($item in $InputObject) {
$inputDict.Add($Columns[$index++], $item)
}
}
catch {
if($_.Exception.InnerException -is [ArgumentNullException]) {
$errorRecord = [Management.Automation.ErrorRecord]::new(
[Exception] 'Different count between input arrays and Columns.',
'InputArrayLengthMismatch',
[Management.Automation.ErrorCategory]::InvalidOperation,
$InputObject
)
$PSCmdlet.ThrowTerminatingError($errorRecord)
}
$PSCmdlet.ThrowTerminatingError($_)
}
}
end {
foreach($pair in $inputDict.GetEnumerator()) {
$count = $pair.Value.Count
if($count -gt $max) {
$max = $count
}
}
for($i = 0; $i -lt $max; $i++) {
$out = [ordered]#{}
foreach($column in $inputDict.PSBase.Keys) {
$out[$column] = $inputDict[$column][$i]
}
[pscustomobject] $out
}
}
}
The usage would be pretty easy, the arrays can be passed through the pipeline:
$Array1 = 1, 2, 3
$Array2 = 'Joe Bloggs', 'John Doe', 'Jane Doe'
$Array3 = 'Yes', 'No'
$Array4 = 'hello', 'world', 123, 456
$Array1, $Array2, $Array3, $Array4 | Join-Array -Columns Number, Name, Valid, Test |
Export-Csv path\to\csv.csv -NoTypeInformation
Or via Named Parameter / Positional Binding:
Join-Array $Array1, $Array2, $Array3, $Array4 -Columns Number, Name, Valid, Test |
Export-Csv path\to\csv.csv -NoTypeInformation
For quiet some time I am maintaining a Join-Object script/Join-Object Module (see also: In Powershell, what's the best way to join two tables into one?). It's main purpose is joining tables (aka lists of objects) based on a related column (aka property) defined by the -on parameter using an user (aka scripter) friendly syntax which as much joining options as possible. This has lead to a few implicated features that could be used in your specific request:
If there is no relation defined (the -on parameter is omitted), it is assumed that you want to do a side-by-side join of the tables (aka lists)
If the tables (or lists) are not equal in size, a common (inner) join will end when either lists ends. A full join (e.g. FullJoin-Object alias FullJoin or a left - or right join) at the other hand, will join the complete respective lists
An array of scalars (e.g. strings rather then -custom- objects) is joined as being a list of objects where the (default) column (aka property) name is Value
If both sides contain a list a scalars, the output will be a [Collections.ObjectModel.Collection[psobject]] list containing each left - and right item
Join commands can be chained: ... |Join ... |Join ...
If the columns (aka properties) provided in the tables (aka lists) overlap, they will be merged in an array ( <property name> = <left property value>, <right property value>) by default, which items could also be assigned to a specific name (prefix) using the -Name parameter. This parameter can be used with every join commando in the chain.
All together this means that you might reach your requirement with the following command line:
$Array1 |FullJoin $Array2 |FullJoin $Array3 -Name Number, Name, Valid
Number Name Valid
------ ---- -----
1 Joe Bloggs Yes
2 John Doe No
3 Jane Doe
You need to restructure the data to pivot it into one array (presented here in pseudo-json):
[
{"1", "Bob", "Yes"},
{"2", "Smithy", "no"},
{"3", "Alex", ""},
{"", "Jeremy", ""}
]
Note how the missing fields still appear with empty placeholders.
Once you've pivoted your data this way, writing it to a CSV file is trivial, as are many other tasks (this is the better way to structure things in the first place).
Related
This question already has answers here:
Powershell foreach regarding multiple collections
(1 answer)
Display multiple array values in list in Powershell
(2 answers)
Closed 3 months ago.
I have 2 arrays: I need to sort them in foreach to modify 1 file.
1 array with dbs name like:
$dbs = ['Honda', 'Toyota', 'BMW', 'Opel']
2nd array with their size like:
$dbsSize = ['123', '300', '222', '143']
and I can sort names in foreach like
foreach ($db in $dbs){
# some logic..
}
How can I sort 2nd array in the same foreach so I can use it in the same loop
Like this:
foreach ($db in $dbs) {
# some logic with $db
# and here I need some logic with $dbsSize
}
but to sort out $dbsSize like $dbs because I need the result like :
Honda
123
Toyota
300
etc.
Do I need to create another loop?
It seems to me you have two arrays where the indices determine which element of the one array belongs to the element in the other array.
It is unclear how you created them and why you did not create an array of objects with both properties together in the first place, but you can still do that like this:
$dbs = 'Honda', 'Toyota', 'BMW', 'Opel'
$dbsSize = '123', '300', '222', '143'
$result = for($i = 0; $i -lt [math]::Max($dbs.Count, $dbsSize.Count); $i++) {
[PsCustomObject]#{
Car = $dbs[$i]
Size = $dbsSize[$i]
}
}
# sort the items on 'Car' for instance
$result = $result | Sort-Object Car
# show on screen
$result
# save to structured CSV file you can open in Excel
$result | Export-Csv -Path 'X:\Somewhere\cars.csv' -UseCulture -NoTypeInformation
Result on screen using the above exampe looks like
Car Size
--- ----
BMW 222
Honda 123
Opel 143
Toyota 300
I don't see any sorting in your code, I only see enumeration. Seems like you want to enumerate both collections sequentially, in that case you would need to use a for loop instead of foreach:
$dbs = "['Honda', 'Toyota', 'BMW', 'Opel']" | ConvertFrom-Json
$dbsSize = "['123', '300', '222', '143']" | ConvertFrom-Json
for($i = 0; $i -lt [math]::Max($dbs.Count, $dbsSize.Count); $i++) {
$dbs[$i]
$dbsSize[$i]
}
If you want to use a foreach loop and both collections have the same Length, then you can get the items of one of them (your choice which) via indexing [ ]:
$i = 0
foreach($item in $dbs) {
$item
$dbsSize[$i++]
}
If you're looking to merge both arrays into objects, an easy way to do it is with the function from this answer, the usage in this case would be:
$dbs, $dbsSize | Join-Array -Columns Car, Size
I have one array of hashtables like the one below:
$hashtable1 = #{}
$hashtable1.name = "aaa"
$hashtable1.surname =#()
$hashtable1.surname += "bbb"
$hashtable2 = #{}
$hashtable2.name = "aaa"
$hashtable2.surname =#()
$hashtable2.surname += "ccc"
$hashtable3 = #{}
$hashtable3.name = "bbb"
$hashtable3.surname = #()
$hashtable3.surname += "xxx"
$A = #($hashtable1; $hashtable2; $hashtable3)
I need to iterate though the array and I need to find out duplicates based on hashtable[].name
Then I need to group those hashtable.surname to hashtable[].surname so that the result will be an array of hashtables that will group all for name all the surnames:
$hashtable1.name = "aaa"
$hashtable1.surname = ("bbb","ccc")
$hashtable3.name = "bbb"
$hashtable3.surname = ("xxx")
I was looking into iterating to empty array
+
I have found this link:
powershell compare 2 arrays output if match
but I am not sure on how to reach into the elements of the hashtable.
My options:
I was wondering if -contain can do it.
I have read about compare-object but I am not sure it can be done like that.
(It looks a bit scary in the moment)
I am on PS5.
Thanks for your help,
Aster
You can group your array items by the names using a scriptblock like so.
Once grouped, you can easily build your output to do what you seek.
#In PS 7.0+ you can use Name directly but earlier version requires the use of the scriptblock when dealing with arrays of hashtables.
$Output = $A | Group-Object -Property {$_.Name} | % {
[PSCustomObject]#{
Name = $_.Name
Surname = $_.Group.Surname | Sort-Object -Unique
}
}
Here is the output variable content.
Name Surname
---- -------
aaa {bbb, ccc}
bbb xxx
Note
Improvements have been made in PS 7.0 that allows you to use simply the property name (eg: Name) in Group-Object for arrays of hashtables, just like you would do for any other arrays type. For earlier version though, these particular arrays must be accessed by passing the property in a scriptblock, like so: {$_.Name}
References
MSDN - Group_Object
SS64 - Group Object
Dr Scripto - Use a Script block to create custom groupings in PowerShell
I wish to display an array of hastables as a Table. I found several threads dealing with this issue, but so far no solution has worked for me (see here and here).
My array consists of 18 hashtables in this form:
Array = #(
#{Company=1}
#{Company=2}
#{Company=3}
#{Contact=X}
#{Contact=Y}
#{Contact=Y}
#{Country=A}
#{Country=B}
#{Country=C}
)
I would like to get the following output:
Company Contact Country
------------------------------
1 X A
2 Y B
3 Z C
I've tried the following:
$Array | ForEach-Object { New-Object -Type PSObject -Property $_ } | Format-Table
That displays the following:
Company
----------
1
2
3
A Format-List works better; I then get this:
Company: 1 2 3
Contact: X Y Z
Country: A B C
Is there any way to accomplish my desired output?
You can do the following if you want to work with CSV data and custom objects:
$Array = #(
#{Company=1}
#{Company=2}
#{Company=3}
#{Contact='X'}
#{Contact='Y'}
#{Contact='Z'}
#{Country='A'}
#{Country='B'}
#{Country='C'}
)
$keys = $Array.Keys | Get-Unique
$data = for ($i = 0; $i -lt $Array.($Keys[0]).Count; $i++) {
($Keys | Foreach-Object { $Array.$_[$i] }) -join ','
}
($Keys -join ','),$data | ConvertFrom-Csv | Format-Table
Explanation:
Since $Array is an array of hash tables, then the .Keys property will return all the keys of those hash tables. Since we only care about unique keys when building our object, Get-Unique is used to remove duplicates.
$Array.($Keys[0]).Count counts the number of items in a group. $Keys[0] here will be Company. So it returns the number of hash tables (3) that contain the key Company.
$Array.Company for example returns all values from the hash tables that contain the key Company. The Foreach-Object loops through each unique key's value at a particular index ($i). Once each key-value is read at a particular index, the values are joined by a comma.
When the loop completes, ($Keys -join ','),$data outputs the data in CSV format, which is piped into ConvertFrom-Csv to create a custom object.
Note: that if your data contains commas, you may want to consider the alternative method below.
Alernatively, you can work with hash tables and custom objects with the following:
$Array = #(
#{Company=1}
#{Company=2}
#{Company=3}
#{Contact='X'}
#{Contact='Y'}
#{Contact='Z'}
#{Country='A'}
#{Country='B'}
#{Country='C'}
)
$keys = $Array.Keys | Get-Unique
$data = for ($i = 0; $i -lt $Array.($Keys[0]).Count; $i++) {
$hash = [ordered]#{}
$Keys | Foreach-Object { $hash.Add($_,$Array.$_[$i]) }
[pscustomobject]$hash
}
$data | Format-Table
There are quite a few posts on SO that address PowerShell transposition. However, most of the code is specific to the use case or addresses data being gathered from a text/CSV file and does me no good. I'd like to see a solution that can do this work without such specifics and works with arrays directly in PS.
Example data:
Customer Name: SomeCompany
Abbreviation: SC
Company Contact: Some Person
Address: 123 Anywhere St.
ClientID: XXXX
This data is much more complicated, but I can work with it using other methods if I can just get the rows and columns to cooperate. The array things that "Name:" and "SomeCompany" are column headers. This is a byproduct of how the data is gathered and cannot be changed. I'm importing the data from an excel spreadsheet with PSExcel and the spreadsheet format is not changeable.
Desired output:
Customer Name:, Abbreviation:, Company Contact:, Address:, ClientID:
SomeCompany, SC, Some Person, 123 Anywhere St., XXXX
Example of things I've tried:
$CustInfo = Import-XLSX -Path "SomePath" -Sheet "SomeSheet" -RowStart 3 -ColumnStart 2
$b = #()
foreach ($Property in $CustInfo.Property | Select -Unique) {
$Props = [ordered]#{ Property = $Property }
foreach ($item in $CustInfo."Customer Name:" | Select -Unique){
$Value = ($CustInfo.where({ $_."Customer Name:" -eq $item -and
$_.Property -eq $Property })).Value
$Props += #{ $item = $Value }
}
$b += New-Object -TypeName PSObject -Property $Props
}
This does not work because of the "other" data I mentioned. There are many other sections in this particular workbook so the "Select -Unique" fails without error and the output is blank. If I could limit the input to only select the rows/columns I needed, this might have a shot. It appears that while there is a "RowStart" and "ColumnStart" to Import-XLSX, there are no properties for stopping either one.
I've tried methods from the above linked SO questions, but as I said, they are either too specific to the question's data or apply to importing CSV files and not working with arrays.
I was able to resolve this by doing two things:
Removed the extra columns by using the "-Header" switch on the Import-XLSX function to add fake header names and then only select those headers.
$CustInfo = Import-XLSX -Path "SomePath" -Sheet "SomeSheet" -RowStart 2 -ColumnStart 2 -Header 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18 | Select "1","2"
The downside to this is that I had to know how many columns the input data had -- Not dynamic. If anyone can provide a solution to this issue, I'd be grateful.
Flipped the columns and headers with a simple foreach loop:
$obj = [PSCustomObject]#{}
ForEach ($item in $CustInfo) {
$value = $null
$name = $null
if ($item."2") { [string]$value = $item."2" }
if ($item."1") { [string]$name = $item."1" }
if ($value -and $name) {
$obj | Add-Member -NotePropertyName $name -NotePropertyValue $value
}
}
I had to force string type on the property names and values because the zip codes and CustID was formatting as an Int32. Otherwise, this does what I need.
I have two arrays imported from csv files, which from here on i will refer to as the master array and update array.
The master array has three extra columns on the front, and three extra columnns on the back.
Each day i get a new update array that i need to do two things with.
A) Remove any rows on the master that do not appear in the update
B) Add any rows that appear in the update but not the master to the master
I am still fairly new to powershell, and scripting in general(mostly self taught) and can't figure out how to approach this. I know there's a compare-object command, so I can get a list of which rows match pretty easily, but I'm not sure how to combine them the way I want.
Edit:
The master array entries have this information:
ef: true
ea: true
rem: true
accountname: example1
Enabled: True
PasswordLastSet: 01/08/2002 13:14:19
whencreated: 01/08/2002 13:14:19
Description:
Owner Email: johnsmith#email.com
givenname: John
sn: Smith
manager: John Doe
Level2: Person 1
Level3: Person 2
Level4: Person 3
While the updates only have:
accountname: example1
Enabled: True
PasswordLastSet: 01/08/2002 13:14:19
whencreated: 01/08/2002 13:14:19
Description:
Owner Email: johnsmith#email.com
givenname: John
sn: Smith
manager: John Doe
Assuming the accountname column can be used as the unique key that ties the two arrays together, you could use something like the below script. It creates a third array and then overwrites the master array csv once completed.
$arrmaster = import-csv c:\temp\arrmaster.csv
$arrupdate = import-csv c:\temp\arrupdate.csv
$arrworking = #()
foreach ($rowupdate in $arrupdate){
$rowmaster = #($arrmaster | where {$_.accountname -eq $rowupdate.accountname})
if ($rowmaster.Count -lt 1){
Write-Debug "Could not find record for $($row.accountname)"
}
if ($rowmaster.Count -gt 1){
Write-Debug "Found duplicate records for $($row.accountname)"
}
if ($rowmaster.Count -eq 1){
$rowworking = "" | select ef,ea,rem,accountname,Enabled,PasswordLastSet,whencreated,Description,"Owner Email",givenname,sn,manager,Level2,Level3,Level4
$rowworking.ef = $rowmaster.ef
$rowworking.ea = $rowmaster.ea
$rowworking.rem = $rowmaster.rem
$rowworking.accountname = $rowupdate.accountname
$rowworking.Enabled = $rowupdate.Enabled
$rowworking.PasswordLastSet = $rowupdate.PasswordLastSet
$rowworking.whencreated = $rowupdate.whencreated
$rowworking.Description = $rowupdate.Description
$rowworking."Owner Email" = $rowupdate."Owner Email"
$rowworking.givenname = $rowupdate.givenname
$rowworking.sn = $rowupdate.sn
$rowworking.manager = $rowupdate.manager
$rowworking.Level2 = $rowmaster.Level2
$rowworking.Level3 = $rowmaster.Level3
$rowworking.Level4 = $rowmaster.Level4
$arrworking += $rowworking
}
}
$arrworking | Export-Csv -Force -NoTypeInformation c:\temp\arrmaster.csv
Not tested, but I think this should work:
$MasterFile = 'c:\somedir\master.csv'
$UpdateFile = 'c:\somedir\update.csv'
$master= #{}
$update = #{}
import-csv $MasterFile |
ForEach-Object { $master[$_.accountname] = $_ }
import-csv $update |
ForEach-Object { $update[$_.accountname] = $_ }
#Get Master entries contained in Update
[array]$NewArray = $master.keys |
Where-Object { $update.keys -contains $_ } |
ForEach-Object { $master[$_] }
#Get Updates not in Master
$NewArray += $update.keys |
Where-Object { $master.keys -notcontains $_ } |
ForEach-Object { $update[$_] }
$NewArray | Export-Csv 'c:\somedir\new_master.csv' -NoTypeInformation
That starts by loading each of your arrays into a hash table, indexed by the accountname. Then the keys are used to extract the master entries that have an accountname that appears in the update keys and load that into a new array. Then the process is reversed and the update keys compared to the master keys, and any entries that do not have a matching key in the master are added to the array. Then the array is exported to csv.
The CSV export will create it's header row from the first entry, and add the necessary commas for any objects in the array afterward that are missing properties. You don't have to worry about adding the missing properties to the update entries as long as they're added after the master entries.
Ok, again based off the assumption that AccountName is a unique identifier that both lists would have in common you can run this:
$Master = Import-CSV Master.csv
$Update = Import-CSV Update.csv
$T2Keys = $Master|gm|?{$_.MemberType -match "Property"}|Select -ExpandProperty Name
$T1Keys = $Update|gm|?{$_.MemberType -match "Property"}|Select -ExpandProperty Name
$KeysToAdd = $T2Keys|?{$T1Keys -notcontains $_}
$NewMaster = #()
$NewMaster += $Update | ?{!($Master.accountname -contains $_.accountname)}
$KeysToAdd|%{$NewMaster|Add-Member $_ ""}
$NewMaster += $Master | ?{$Update.accountname -contains $_.accountname}
$Newmaster| Select ef,ea,rem,accountname,enabled,passwordlastset,whencreated,description,'owner email',givenname,sn,manager,level2,level3,level4|Export-CSV NewMaster.csv -notype
Ok, that will import a CSV for the master list and the updates list. If you already have those as objects then skip the import-csv lines. Then it gets all properties from both, and figures out which ones to add to the updates (the 6 that the master has that the updates doesn't). It then creates an empty array and adds all records from the Updates list to it that aren't in the master list. Then it adds the missing fields, and adds all the records from the master list that are in the updates list. Then it exports it to a CSV. So it does what you asked:
Gets all records from the master list that are in both lists.
Adds
records from the update list that are missing from the master list.
Edit: The reason I had asked if you had searched is that 95% of my answer there was almost copied and pasted from this question that I answered just under a month ago. But hey, it's all good, not that hard for me to copy and paste to get you an answer, and I kinda knew what I was looking for anyway. I don't know that the other question's title would have been indicative that it had what you needed.