My goal is to take two objects (created by importing CSVs) that have host names and compare one list to the other and show what's missing from each.
Before I can do the comparison I need to manipulate the host names stored within the object(s). The first step is to use regular expressions to remove (-replace) unnecessary text and then set all host names to lowercase (ToLower()).
I'm not very proficient with modifying existing objects and keeping them "intact", so I'm hoping someone could help me with this.
Here's an example of the data stored within the CSV. The header is on line 7 and each line of data is stored like:
...
7 "name","IP","OSType"
8 "WCSMserver.com","10.10.10.10","OSX"
9 "SERVER2.com","11.11.11.11","Windows"
10 "windowsserver # SERVER2.com","11.11.11.13","Windows"
11 "winner.comSERVER2.com","11.11.11.12","Windows"
...
Here's an example of what I'm trying to do so far (just replacing the name property values):
function ReadExcelReport() {
$global:ConvertedQReportTest = $PSScriptRoot + "\" + "AllSources.csv"
$global:QReportObject = Get-Content -Path $global:ConvertedQReportTest |
Select-Object -Skip 7 |
ConvertFrom-Csv
}
ReadExcelReport
$global:QReportObject.name = $global:QReportObject.name | ForEach-Object {
#($global:QReportObject.name)
$_ -replace 'WCSM \- ' `
-replace '.*?# '`
-replace '.*?#'`
-replace '.*?\:\:.*?'`
-replace '\.cooper\.winner\.com'`
-replace '\.winner\.com'
}
By doing $global:QReportObject.name | ForEach-Object you loop the names of the objects and not the objects.
I've simplified your script a bit (for readability):
$csv = #"
"name"
"WCSMserver-remove this-com"
"SERVER2.com","11.11.11.11"
"windowsserver-remove this-"
"winner.comSERVER2.com"
"#
$global:QReportObject = $csv | ConvertFrom-Csv
$global:QReportObject | Out-Default
$global:QReportObject | ForEach-Object {
$_.name = $_.name -replace '-remove this-'
$_.name = $_.name.ToLower()
}
$global:QReportObject | Out-Default
This will output:
name
----
WCSMserver-remove this-com
SERVER2.com
windowsserver-remove this-
winner.comSERVER2.com
name
----
wcsmservercom
server2.com
windowsserver
winner.comserver2.com
Related
I'm creating a PowerShell script for DPM which outputs offsite ready tapes. The below code takes a few seconds to complete the DPM connection and to return the query, which is fine.
# Connect to local DPM server
$DPMServer = $env:COMPUTERNAME
Connect-DPMServer -DPMServerName $DPMServer | Out-Null
# Get the DPM libary
$DPMLib = Get-DPMLibrary
#Format output
$Formatting = #{Expression={$_.Barcode}; Label="Barcode "; Width=12},
#{Expression={"{0:MM/dd/yyyy HH:mm tt}" -f $_.CreationDate}; Label="Creation Date "; Width=23},
#{Expression={$_.DisplayString}; Label="Tape Label "; Width=28},
#{Expression={"{0,15:N0}" -f $_.DataWrittenDisplayString}; Label="Data Written "}
#Calculate Monday at midnight of this week
$Monday = (Get-Date).AddDays((-1 * (Get-Date).DayOfWeek.Value__) + 1).Date
# Get specific tapes
$Tapes = Get-DPMTape -DPMLibrary $DPMLib |
Where-Object {$_.Barcode -notlike "CLN*"} |
Where-Object {$_.DisplayString -notlike "Free"} |
Where-Object {$_.CreationDate -gt $Monday} |
Sort-Object Barcode |
Format-Table -Property $Formatting
Write-Host "`nOffsite Ready Tapes:" -ForegroundColor Cyan
#Output tapes
$Tapes
This outputs like so:
My question is how do I now select just the barcodes listed within the $Tapes array, and output them (below what I already have) as a comma delimited list, without running the DPM query again. I've tried all sorts of things with no luck, I'm missing something obvious.
If I do a second connection to DPM, I can do it like this, but I'm trying to avoid doubling the time it takes to run. This is part of my original newbie script that I'm trying to better.
# Define DPM server to connect to
$DPMServer = $env:COMPUTERNAME
Connect-DPMServer -DPMServerName $DPMServer | Out-Null
# Get the DPM libary
$DPMLib = Get-DPMLibrary
# Get tape display strings and barcodes, sort by barcode
$Tapes = Get-DPMTape -DPMLibrary $DPMLib | Select-Object CreationDate, DisplayString, Barcode | Sort-Object Barcode
# Create empty array
$DPMTapesForOffsite = #()
Write-Host "`nOffsite Ready Tapes:`n" -ForegroundColor Cyan
foreach ($_ in $Tapes) {
# Exclude cleaning tapes
if ($_.Barcode -notlike "CLN*") {
# Exclude marked as free
if ($_.DisplayString -notlike "Free") {
$TimeStamp = Get-Date $_.CreationDate
# Timestamp is from this week
if ($Timestamp -gt $Monday) {
$DPMTapesForOffsite = $DPMTapesForOffsite + $_.barcode
Write-Host $_.barcode
}
}
}
}
# Format tape list as comma delimited
$DPMTapesForOffsite = $DPMTapesForOffsite -join ","
I'm missing some obvious, any help would greatly be appreciated.
# Omit Format-Table initially, so as to store actual *data* in $tapes,
# not *formatting instructions*, which is what Format-* cmdlets return.
$tapes = Get-DPMTape -DPMLibrary $DPMLib |
Where-Object {$_.Barcode -notlike "CLN*"} |
Where-Object {$_.DisplayString -notlike "Free"} |
Where-Object {$_.CreationDate -gt $Monday} |
Sort-Object Barcode
# Now you can apply the desired formatting.
Write-Host "`nOffsite Ready Tapes:" -ForegroundColor Cyan
$tapes | Format-Table -Property $Formatting
# Thanks to PowerShell's member-access enumeration feature,
# accessing property .Barcode on the entire $tapes *collection*
# conveniently returns its *elements'* property values.
# -join ',' joins them with commas.
$tapes.Barcode -join ','
Format-* cmdlets output objects whose sole purpose is to provide formatting instructions to PowerShell's output-formatting system - see this answer.
In short: only ever use Format-* cmdlets to format data for display, never for subsequent programmatic processing.
This answer explains member-access enumeration.
I am working with two CSV files. One holds the name of users and the other one holds their corresponding email address. What I want to do is to combine them both so that users is column 1 and email is column 2 and output it to one file. So far, I've managed to add a second column from the email csv file to the user csv file, but with blank row data. Below is the code that I am using:
$emailCol= import-csv "C:\files\temp\emailOnly.csv" | Select-Object -skip 1
$emailArr=#{}
$i=0
$nameCol = import-csv "C:\files\temp\nameOnly.csv"
foreach ($item in $emailCol){
$nameCol | Select *, #{
Name="email";Expression=
{$emailArr[$i]}
} | Export-Csv -path
C:\files\temp\revised.csv -NoTypeInformation
}
Updated: Below is what worked for me. Thanks BenH!
function combineData {
#This function will combine the user CSV file and
#email CSV file into a single file
$emailCol = Get-Content "C:\files\temp\emailOnly.csv"
| Select-Object -skip 1
$nameCol = Get-Content "C:\files\temp\nameOnly.csv" |
Select-Object -skip 1
# Max function to find the larger count of the two
#csvs to use as the boundary for the counter.
$count = [math]::Max($emailCol.count,$nameCol.count)
$CombinedArray = for ($i = 0; $i -lt $count; $i++) {
[PSCustomObject]#{
fullName = $nameCol[$i]
email = $emailCol[$i]
}
}
$CombinedArray | Export-Csv C:\files\temp\revised.csv
-NoTypeInformation
}
To prevent some additional questions about this theme let me show you alternative approach. If your both CSV files have same number of lines and each line of the first file corresponds to the first line of the second file and etc. then you can do next. For example, users.csv:
User
Name1
Name2
Name3
Name4
Name5
and email.csv:
Email
mail1#gmail.com
mail2#gmail.com
mail3#gmail.com
mail5#gmail.com
Our purpose:
"User","Email"
"Name1","mail1#gmail.com"
"Name2","mail2#gmail.com"
"Name3","mail3#gmail.com"
"Name4",
"Name5","mail5#gmail.com"
What we do?
$c1 = 'C:\path\to\user.csv'
$c2 = 'C:\path\to\email.csv'
[Linq.Enumerable]::Zip(
(Get-Content $c1), (Get-Content $c2),[Func[Object, Object, Object[]]]{$args -join ','}
) | ConvertFrom-Csv | Export-Csv C:\path\to\output.csv
If our purpose is:
"User","Email"
"Name1","mail1#gmail.com"
"Name2","mail2#gmail.com"
"Name3","mail3#gmail.com"
"Name5","mail5#gmail.com"
then:
$c1 = 'C:\path\to\user.csv'
$c2 = 'C:\path\to\email.csv'
([Linq.Enumerable]::Zip(
(Get-Content $c1), (Get-Content $c2),[Func[Object, Object, Object[]]]{$args -join ','}
) | ConvertFrom-Csv).Where{$_.Email} | Export-Csv C:\path\to\output.csv
Hope this helps you in the future.
A for loop would be better suited for your loop. Then use the counter as the index for each of the arrays to build your new object.
$emailCol = Get-Content "C:\files\temp\emailOnly.csv" | Select-Object -Skip 2
$nameCol = Get-Content "C:\files\temp\nameOnly.csv" | Select-Object -Skip 1
# Max function to find the larger count of the two csvs to use as the boundary for the counter.
$count = [math]::Max($emailCol.count,$nameCol.count)
$CombinedArray = for ($i = 0; $i -lt $count; $i++) {
[PSCustomObject]#{
Name = $nameCol[$i]
Email = $emailCol[$i]
}
}
$CombinedArray | Export-Csv C:\files\temp\revised.csv -NoTypeInformation
Answer edited to use Get-Content with an extra skip added to skip the header line in order to handle blank lines.
Bond.out file example (looking to replace what is highlighted):
Out.csv file (data to be used):
Code:
#set paths up
$filepath= 'C:\folder\path\bond.out'
$filepath2= 'C:\folder\path\temp.txt'
$Ticklist='C:\folder\path\tick.txt'
$ratelist='C:\folder\path\rate.txt'
#Import needed data from an excel file which creates and array
$csv = Import-CSV C:\folder\path\RateIDTable.csv | Where { $_.'Rate' -ne "" } | Export-Csv C:\folder\path\out.csv -NoTypeInformation
$bond = Import-CSV C:\folder\path\out.csv | select -Property TickerID, Rate
#Put array from Excel file into two text files
$Tick = $bond | foreach-object {$_.TickerID} | set-content $Ticklist
$replace = $bond | foreach-object {$_.rate} | set-content $Ratelist
#Create two separate arrays from the new text files
$Tickdata = (Get-content $Ticklist ) -join ','
foreach ($t in $Tickdata)
{
$t = $t -split(",")
$First = $t[0]}
$Ratedata = (Get-content $Ratelist ) -join ','
foreach ($r in $Ratedata)
{
$r = $r -split(",")
$First = $r[0]}
#Get main file to search (bond.out) and search for the word that is in the first line from "t" array file
$data = Select-String $filepath -pattern $t[0] | Select-Object -ExpandProperty Line
$data
#Once found, split the line, replace the rate on the 3rd line with the rate in the first line from the "r" array file, the put the line back to together
$split=$data.split("{|}")
$split[3]=$r[0]
$join = $split -join "|"
$join
#Put the updated line back into the "bond.out" file from whence it came
(get-content $filepath) -replace($data,$join) | set-content $filepath
#computer says no :(
Output:
As you can see, it actually replaces the rate and puts it all back like I need it to. But that last line doesn't seem to work. Instead I get the file back like so:
It appears as though it is repeating the same line from the $join parameter and adding letters to the beginning of each iteration.
I believe it has something to do with the '|' at the end of the line, and remember reading something about marking the beginning and end of lines some time ago, but can't find it anywhere.
Here's an idea. Instead of using regular expressions ...
The Import-Csv command has a -Delimiter parameter. Can you just import bond.out as a "CSV" (but with a pipe delimiter), and update it just like you would a CSV file?
Pseudo-code
### Convert bond.out to objects
$BondOut = Import-Csv -Delimiter '|' -Path $FilePath
### Get the line you want to update
$LineToUpdate = $BondOut.Where({ $PSItem.TickerID -eq 'BBG0019K2QZ5' })
### Update the Rate property from your source (out.csv)
$LineToUpdate.Rate = $SomeSource.Rate
### Export the modified objects to a new bond.out.modified file
$BondOut | Export-Csv -Delimiter '|' -Path 'bond.out.modified' -NoTypeInformation
As per PetSerAI's clue:
#set paths up
$filepath= 'C:\folder\path\bond.out'
$filepath2= 'C:\folder\path\temp.txt'
$Ticklist='C:\folder\path\tick.txt'
$ratelist='C:\folder\path\rate.txt'
#Import needed data from an excel file which creates and array
$csv = Import-CSV C:\folder\path\RateIDTable.csv | Where { $_.'Rate' -ne "" } | Export-Csv C:\folder\path\out.csv -NoTypeInformation
$bond = Import-CSV C:\folder\path\out.csv | select -Property TickerID, Rate
#Put array from Excel file into two text files
$Tick = $bond | foreach-object {$_.TickerID} | set-content $Ticklist
$replace = $bond | foreach-object {$_.rate} | set-content $Ratelist
#Create two separate arrays from the new text files
$Tickdata = (Get-content $Ticklist ) -join ','
foreach ($t in $Tickdata)
{
$t = $t -split(",")
}
$Ratedata = (Get-content $Ratelist ) -join ','
foreach ($r in $Ratedata)
{
$r = $r -split(",")
}
#Get main file to search (bond.out) and search for the word that is in the first line from "t" array file
###Replace all pipes with a comma
(get-content $filepath) -replace('\|', ',') | set-content $filepath
$data = Select-String $filepath -pattern $t[0] | Select-Object -ExpandProperty Line
$data
#Once found, split the line, replace the rate on the 3rd line with the rate in the first line from the "r" array file, the put the line back to together
$split=$data.split("{,}")
$split[3]=$r[0]
$join = $split -join ","
#Put the updated line back into the "bond.out" file from whence it came
###change all commas back to pipes
(get-content $filepath) -replace($data,$j) | set-content $filepath
(get-content $filepath) -replace(',', '|') | set-content $filepath
#computer says yay :D
Here is my dilemma. I have a csv file with two columns
ID,FullFileName
1,Value1
1,Value2
1,Value3
2,Value1
2,Value2
3,Value1
4,Value1
5,Value1
5,Value2
The output I'm looking for is to get an exported csv with two columns in it ID, and FullFilename. The value in FullFileName will contain the matching joined values separated by a pipe delimiter.
But my output i'm trying to get the following:
ID,FullFilename
1,Value1|Value2|Value3
2,Value1|Value2
3,Value1
4,Value1
5,Value1|Value2
I'm not sure how to make powershell search the value in ID and take all of the results and yield them into a single concatenated value with a pipe separation. Any assistance on trying to search the array or join / concatenate array values would be greatly appreciated.
Group-Object is the useful cmdlet that can help you. Grouping the data by ID turns it into:
PS D:\> ipcsv .\t.csv | group id
Count Name Group
----- ---- -----
3 1 {#{ID=1; FullFileName=Value1}, #{ID=1; FullFileName=Value2}, #{ID=1; FullFileName=Value3}}
2 2 {#{ID=2; FullFileName=Value1}, #{ID=2; FullFileName=Value2}}
1 3 {#{ID=3; FullFileName=Value1}}
1 4 {#{ID=4; FullFileName=Value1}}
2 5 {#{ID=5; FullFileName=Value1}, #{ID=5; FullFileName=Value2}}
So you want the Name (= ID) and the Group property, just the FullFileName, joined up:
Import-Csv -Path c:\path\data.csv |
Group-Object -Property ID |
Select-Object #{Name='ID'; Expression={$_.Name}},
#{Name='FullFilename'; Expression={$_.Group.FullFileName -join '|'}} |
Export-Csv -Path C:\Path\out.csv -NoTypeInformation
$InFile = '.\Sample.csv'
$OutFile= '.\New.csv'
$Csv = Import-Csv $InFile | Group-Object ID | ForEach-Object{
[pscustomobject]#{
ID=$_.Name
FullFileName=$_.Group.FullFileName -join '|'
}
}
$Csv
"----------"
$Csv | Export-Csv $OutFile -NoTypeInformation
Get-Content $OutFile
Sample output:
ID FullFileName
-- ------------
1 Value1|Value2|Value3
2 Value1|Value2
3 Value1
4 Value1
5 Value1|Value2
----------
"ID","FullFileName"
"1","Value1|Value2|Value3"
"2","Value1|Value2"
"3","Value1"
"4","Value1"
"5","Value1|Value2"
Edit Just saw you wanted the pipe as delimiter.
Did my best but was to late, anyway here is the code
Instead of Group-Object i am using a HashTable, i find it easy to work with when my data comes from multiple sources.
$CSV = Import-Csv -Delimiter ',' -Path "$env:TEMP\testfolder\csv.txt" #can be .csv or whatever.
$HastTable = #{}
Foreach ($Line in $CSV) {
if (!$HastTable["$($Line.ID)"]) {
$HastTable["$($Line.ID)"] = $Line
}
else {
$HastTable["$($Line.ID)"].FullFileName += "|$($Line.FullFileName)"
}
}
$HastTable.Values | Export-Csv -Delimiter ',' -NoTypeInformation -Path "$env:TEMP\testfolder\newcsv.txt" #can be .csv or whatever.
I am iterating through a directory full of sub directories, looking for the newest file at each level.
The code below does this, but I need to be able to add each line/loop of the iterator to an array so that at the end I can output all the data in tabular format for use in Excel.
Any advice on how I can do this?
$arr = get-childItem -Path "\\network location\directory" | select FullName
$res = #()
foreach($fp in $arr)
{
get-childItem -Path $fp.FullName | sort LastWriteTime | select -last 1 Directory, FullName, Name, LastWriteTime
}
Here's a one-liner for you, split onto multiple lines for readability with the backtick escape character. You can copy paste this and it will run as is. The csv file will be created in the folder where you run this from.
dir -rec -directory | `
foreach {
dir $_.fullname -file | `
sort -Descending lastwritetime | `
select -first 1
} | `
export-csv newestfiles.csv
dir is an alias for get-childitem. foreach is an alias for foreach-object. %, gci and ls are even shorter aliases for get-childitem. Note that I am avoiding storing things in arrays, as this is doubling the work required. There is no need to enumerate the folders, and then enumerate the array afterwards as two separate operations.
Hope this helps.
If I understand you correctly, you just need to pipe the results into $res. So adding | %{$res += $_} should do the trick
$arr = get-childItem -Path "\\network location\directory" | select FullName
$res = #()
foreach($fp in $arr)
{
get-childItem -Path $fp.FullName | sort LastWriteTime | select -last 1 Directory, FullName, Name, LastWriteTime | % {$res += $_}
}
$res | % {write-host $_}