$groups = Get-Content c:\devices.csv | Group {$_.Substring(0,3)}| %{$_.Group; ""}
AAAGroup1,192.168.1.1
AAAGroup1,192.168.1.2
BBBGroup2,192.168.2.1
BBBGroup2,192.168.2.2
CCCGroup3,192.168.3.1
CCCGroup3,192.168.3.2
I have searched far and wide and can only find solutions based on column selections To output the above data to either a separate variable per group or separate CSV file. Technically, there's a space in between each group in terms of a spare row, that's it as far as i can see.
$groups = Get-Content c:\devices.csv | Group {$_.Substring(0,3)}| %{$_.Group; ""}
ForEach ($Group in $Groups)...
Provided you have a file devices.csv with headers Device,IPv4
The following script will add a calculated property named Group you can use to sort, output to a table and -GroupBy or whatever.
## Q:\Test\2018\07\10\SO_51267179.ps1
$data = Import-Csv '.\devices.csv' |
Select-Object Device,IPv4,#{n='Group';e={$_.Device.Substring(0,3)}}
$data | Format-Table -GroupBy Group
Sample output
Group: AAA
Device IPv4 Group
------ ---- -----
AAAGroup1 192.168.1.1 AAA
AAAGroup1 192.168.1.2 AAA
...snip...
To output each group to it's own .csv
$data = Import-Csv '.\devices.csv' |
| Group-Object {$_.Device.Substring(0,3)}| ForEach-Object {
$_.Group | Export-Csv "$($_.Name).csv" -NoTypeInformation
}
Sample output:
> gc .\AAA.csv
"Device","IPv4"
"AAAGroup1","192.168.1.1"
"AAAGroup1","192.168.1.2"
Related
I'm writing a windows powershell script to open/edit a text file.
It has many records and each record is sorted of comma-separated values (csv):
Steps I want to achieve:
Open the text files in a directory on Server A.
Edit the "Date" field with "Current Date" or other.
Save the same text file at the same location(folder).
Copying all the files to a new folder in different Server B.
I've just written this code snippet:
$path = "C:\PSFiles\Rec_File_001.txt"
$Filedata = Get-Content $path
$Record01 = $Filedata[0].split(",")
$Record01Date = $Record01[3]
$Record01CurrentDate = Get-Date -format yyMMdd
$Record01 -replace $Record01Date, $Record01CurrentDate
Set-Content $path
Please, any help on this?
You are having multiple questions here. I'll address the one that is presented in the title - replacing text in a text file.
The script:
# current date in a new format
$CurrentDate = Get-Date -format yyMMdd
#replace old format
Get-Content -ReadCount 500 -Path C:\PSFiles\Rec_File_001.txt | % {$_ -replace "(0[1-9]|1[012])\d{1,2}(0[1-9]|[12][0-9]|3[01])", "$CurrentDate"} | Set-Content -Path C:\PSFiles_output\Rec_File_001.txt
This takes regexp for date format Date(mmYYdd) and exchanges it for a new one. An option -ReadCount limits the number of lines that go via pipe at one time.
Import-CSV $Path -header text1, text2, text3, date, text5 |
Select text1, text2, text3, #{Name="Date"; Expression={Get-Date -format yyMMdd}}, text5 |
ConvertTo-Csv -NoTypeInformation |
Select-Object -Skip 1 |
Set-Content $Path
Or:
$data = Import-CSV $Path -header text1, text2, text3, date, text5
$data | ForEach {"Date" = Get-Date -format yyMMdd}
$data | ConvertTo-Csv -NoTypeInformation | Select-Object -Skip 1 | Set-Content $Path
Hopefully this helps. Run the following commands in powershell:
$(Get-Item ath/filename.extensionfilename).creationtime=$(get-date "2019-10-15T15:45:12.2723844+01:00")
$(Get-Item C:\temp\log\txt.log).creationtime=$(get-date "2019-10-15T15:45:12.2723844+01:00")
Here is an article from Microsoft that explains how to modify the timestamps of a file https://devblogs.microsoft.com/scripting/use-powershell-to-modify-file-access-time-stamps/
Below is a table from that article showing the attributes that involve time that you can modify.
+----------------+----------+-----------------+---------------------------+
| Name | Member | Type | Definition |
+----------------+----------+-----------------+---------------------------+
| CreationTime | Property | System.DateTime | CreationTime {get;set;} |
| LastAccessTime | Property | System.DateTime | LastAccessTime {get;set;} |
| LastWriteTime | Property | System.DateTime | LastWriteTime {get;set;} |
+----------------+----------+-----------------+---------------------------+
Here is my dilemma. I have a csv file with two columns
ID,FullFileName
1,Value1
1,Value2
1,Value3
2,Value1
2,Value2
3,Value1
4,Value1
5,Value1
5,Value2
The output I'm looking for is to get an exported csv with two columns in it ID, and FullFilename. The value in FullFileName will contain the matching joined values separated by a pipe delimiter.
But my output i'm trying to get the following:
ID,FullFilename
1,Value1|Value2|Value3
2,Value1|Value2
3,Value1
4,Value1
5,Value1|Value2
I'm not sure how to make powershell search the value in ID and take all of the results and yield them into a single concatenated value with a pipe separation. Any assistance on trying to search the array or join / concatenate array values would be greatly appreciated.
Group-Object is the useful cmdlet that can help you. Grouping the data by ID turns it into:
PS D:\> ipcsv .\t.csv | group id
Count Name Group
----- ---- -----
3 1 {#{ID=1; FullFileName=Value1}, #{ID=1; FullFileName=Value2}, #{ID=1; FullFileName=Value3}}
2 2 {#{ID=2; FullFileName=Value1}, #{ID=2; FullFileName=Value2}}
1 3 {#{ID=3; FullFileName=Value1}}
1 4 {#{ID=4; FullFileName=Value1}}
2 5 {#{ID=5; FullFileName=Value1}, #{ID=5; FullFileName=Value2}}
So you want the Name (= ID) and the Group property, just the FullFileName, joined up:
Import-Csv -Path c:\path\data.csv |
Group-Object -Property ID |
Select-Object #{Name='ID'; Expression={$_.Name}},
#{Name='FullFilename'; Expression={$_.Group.FullFileName -join '|'}} |
Export-Csv -Path C:\Path\out.csv -NoTypeInformation
$InFile = '.\Sample.csv'
$OutFile= '.\New.csv'
$Csv = Import-Csv $InFile | Group-Object ID | ForEach-Object{
[pscustomobject]#{
ID=$_.Name
FullFileName=$_.Group.FullFileName -join '|'
}
}
$Csv
"----------"
$Csv | Export-Csv $OutFile -NoTypeInformation
Get-Content $OutFile
Sample output:
ID FullFileName
-- ------------
1 Value1|Value2|Value3
2 Value1|Value2
3 Value1
4 Value1
5 Value1|Value2
----------
"ID","FullFileName"
"1","Value1|Value2|Value3"
"2","Value1|Value2"
"3","Value1"
"4","Value1"
"5","Value1|Value2"
Edit Just saw you wanted the pipe as delimiter.
Did my best but was to late, anyway here is the code
Instead of Group-Object i am using a HashTable, i find it easy to work with when my data comes from multiple sources.
$CSV = Import-Csv -Delimiter ',' -Path "$env:TEMP\testfolder\csv.txt" #can be .csv or whatever.
$HastTable = #{}
Foreach ($Line in $CSV) {
if (!$HastTable["$($Line.ID)"]) {
$HastTable["$($Line.ID)"] = $Line
}
else {
$HastTable["$($Line.ID)"].FullFileName += "|$($Line.FullFileName)"
}
}
$HastTable.Values | Export-Csv -Delimiter ',' -NoTypeInformation -Path "$env:TEMP\testfolder\newcsv.txt" #can be .csv or whatever.
I am trying to locate discrepancies in BIND DNS records. I would like to output a CSV file that only has those discrepancies. I have a CSV file that has all records from all locations in BIND (ns.prvt, ns.pub, common, includes). What I'm trying to figure out is how to output a CSV that only shows the discrepancies. For 2 records to be considered a discrepancy, they must meet the following criteria:
Both records have the same RecordName and RecordType.
Both records have different Data or TTL.
Both records come from different locations.
I am almost there with the following script but it keeps showing me a couple of rows that don't necessarily meet the above criteria.
$Records = Import-Csv C:\Temp\Domain_ALL.csv | Select * | Sort Data,Location
$RecordsRev = #()
$Records | % {
$Record = $_
$Records | % {
$DataFE = $_
If (
([string]($Record | ? {($_.RecordName -eq $DataFE.RecordName)}).RecordName -eq $DataFE.RecordName) -and
([string]($Record | ? {($_.RecordName -eq $DataFE.RecordName)}).RecordType -eq $DataFE.RecordType) -and
([string]($Record | ? {($_.RecordName -eq $DataFE.RecordName)}).Location -ne $DataFE.Location) -and
(([string]($Record | ? {($_.RecordName -eq $DataFE.RecordName)}).Data -ne $DataFE.Data) -or
([string]($Record | ? {($_.RecordName -eq $DataFE.RecordName)}).TTL -ne $DataFE.TTL))
) {
$RecordsRev += $_
}
}
}
$RecordsRev | Export-Csv C:\Temp\Domain_Discrepancies.csv -NoType
The results that I get are:
RecordName RecordType Data TTL Location
---------- ---------- ---- --- --------
domain.com TXT "MS=abc1234566" 600 Includes
domain.com TXT "MS=abc1234566" 600 Common
domain.com TXT "site-verification=abcd1234" 600 Includes
domain.com TXT "site-verification=abcd1234" 600 Common
www CNAME somedomain.com.test. 600 Includes
www CNAME somedomain.com. 600 Common
The results that I expect are:
RecordName RecordType Data TTL Location
---------- ---------- ---- --- --------
www CNAME somedomain.com.test. 600 Includes
www CNAME somedomain.com. 600 Common
How do I delete all duplicated rows in the array? This is different from "Select * -unique" as I don't want to keep any row that contains the duplicated information.
EDIT: I think the main problem is that, since the script checks each record against every record in the CSV, it technically is a discrepancy. For example, in the below table, record 1 meets the criteria to be a discrepancy because it differs from record 4. However, since record 1 is the same as record 2, it should actually be omitted from the results.
RecordNumber RecordName RecordType Data TTL Location
------------ ---------- ---------- ---- --- --------
1 domain.com TXT "MS=abc1234566" 600 Includes
2 domain.com TXT "MS=abc1234566" 600 Common
3 domain.com TXT "site-verification=abcd1234" 600 Includes
4 domain.com TXT "site-verification=abcd1234" 600 Common
5 www CNAME somedomain.com.test. 600 Includes
6 www CNAME somedomain.com. 600 Common
Any help would be greatly appreciated.
Kyle
I was able to figure this out with the help of someone who deleted their post... Here is the script that I am using now to find all records that meet ALL of the following criteria:
Both records have the same RecordName and RecordType. -AND
Both records have different Data or TTL. -AND
Both records come from different locations.
$Records = Import-Csv C:\Temp\Domain_ALL.csv | Select * | Sort Data,Location
$Discrepancies = #()
$GoodRecords = #()
$BadRecords = #()
$Records | ForEach-Object {
# for each record $_, compare it against every other record..
foreach ($R in $Records) {
# if Both records have the same RecordName and RecordType..
if (($_.RecordName -eq $R.RecordName) -and ($_.RecordType -eq $R.RecordType)) {
# and if Both records come from different locations..
if ($_.Location -ne $R.Location) {
# if Both records have the same Data and TTL then they are considered good:
if (($_.Data -eq $R.Data) -and ($_.TTL -eq $R.TTL)) {
$GoodRecords += $_
}
Else{
# if Both records have different Data or TTL then they are considered bad:
$BadRecords += $_
}
}
}
}
}
ForEach ($BadRecord in $BadRecords){
If (($GoodRecords -notcontains $BadRecord)){
$Discrepancies += $BadRecord
}
}
$Discrepancies | Select * -Unique | Sort RecordName,Location,Data | ft
I have the following powershell script:
$grouped_TPR_Test1=Import-Csv c:\TPR.csv | group UPC -AsHashTable -AsString
Import-Csv c:\HQ.csv | foreach{
$tpr_Sales=($grouped_TPR_Test1."$($_.UPC)" | foreach {$_.TPR_Sales}) -join ","
$_ | Add-Member -MemberType NoteProperty -Name TPR_SALES -Value $tpr_Sales -PassThru
} | Export-Csv -NoTypeInformation c:\HQ_TPR_sales.csv
It finds/matches the UPC value in file c:\TPR.csv with the same value in this file. c:\HQ.csv and outputs the corresponding sales data from to a 3rd file that includes all fields in c:\HQ.csv as well as the additional ones that match on UPC from c:\TPR.csv
This works.
However, I am not sure how to add a second field to check ("Zone", to narrow down the results that are sent to the 3rd output file. Both files have the zone field as well.
I read a bit on this and an array seems better suited for multiple criteria, rather than a hashtable, but I'm not having much luck.
c:\HQ.csv looks essentially like this:
UPC ZONE column1 column2 column3
1234567890123 3 blah1 blah2 blah3
c:\TPR.csv looks essentially like this:
UPC ZONE sales
1234567890123 3 5.00
1234567890123 2 4.00
3210987654321 2 3.00
Any help is appreciated.
Thanks!
You could simply just use WHERE-OBJECT on the resulting file to just pull out the zone(s) you are interested in:
Import-CSV c:\HQ_TPR_sales.csv | WHERE-OBJECT {$_.Zone -eq 2}
Or put the WHERE-OBJECT into your pipe before you export-csv:
... | ? {$_.Zone -eq 2} | Export-CSV ...
I have a CSV like below:
location,id
loc1,1234
loc1,1235
loc1,1236
Running $a = Import-CSV C:\File.csv | Group-Object "location" I get the following output:
Count Name Group
----- ---- -----
3 loc1 {#{location=loc1; id=1234}, #{location=loc1; id=1235), #{location=loc1, id=1236}}
I would like to add all ID's to a single group (Using Add-QADGroupMember) but I can't figure out how to get a group of ID's for $loc1. It seems to be be grouping them correctly but I can't seem to parse the output into a single group. E.g $loc1 = 1234,1235,1236 that I can loop through.
Any suggestions would be appreciated!
Group-Object doesn't handle hashtables well, since the keys aren't real properties.
Assuming:
$csv = Import-CSV C:\File.csv
You should be able to do, for example:
$ids = $csv | %{ $_.id }
to get an array of the ID values. You'd probably want to pipe through Get-Unique for location.
If you wanted to get the location for a single ID quickly:
$location = $csv | ?{ $_.id -eq 42 } | %{ $_.location }
If you wanted to get an array of all IDs for a single location quickly (I think this is what you want):
$loc1 = $csv | ?{ $_.location -eq 'loc1' }
For reference, if you wanted to get a hashtable mapping each location to an array of IDs:
$groups = $csv | %{ $_.location } | &{
begin
{
$hash = #{}
}
process
{
$location = $_.location
$hash[$location] = $csv | ?{ $_.location -eq $location }
}
end
{
$hash
}
}
A bit tricky, but this will do it:
Import-Csv C:\File.csv | Group-Object "location" | %{Set-Variable ($_.Name) ($_.Group | Select-Object -ExpandProperty id)}
After running that, $loc1, $loc2, etc. will be arrays of all the ids for each location.
And yet another option:
(Import-Csv c:\foo.csv | Group Location -AsHashTable).Loc1 | Foreach {$_.id}
And if you're on V3, you can do this:
(Import-Csv c:\foo.csv | Group Location -AsHashTable).Loc1.Id