Input:
1116559 P1303251287 20130325225906CD 13013822 1 0000
1104220 P1303250282 20130325070119CD 1 0000
1064743 P1303251094 20130325191600CD 0 0000
1100819 P1303250369 20130325091722CD 0 0000
1101405 P1303250051 20130325010740CD 2 0000
What I get from my attempt:
$lines = Get-Content "filenamehere.txt"
ForEach ($x in $lines) {
$y = "$($x[0..9] -join '')|$($x[10..23] -join '')|$($x[24..42] -join '')|
$($x[43..53] - join '')|$($x[54..57] -join '')|$($x[58..61] -join '')|
$($x[126..138] -join '')"
$z = $y -join '|'
Write-Output $z | Out-File -FilePath "foo.txt" -Append}
I get:
1116559 |P1303251287 |20130325225906CD |13013822 |1 |0000|
1104220 |P1303250282 |20130325070119CD | |1 |0000|
1064743 |P1303251094 |20130325191600CD | |0 |0000|
1100819 |P1303250369 |20130325091722CD | |0 |0000|
1101405 |P1303250051 |20130325010740CD | |2 |0000|
I don't mind the trailing spaces as long as I can get into this format. But "Get-Content" parse my data into an array and importing to SQL gives me an error. Question is, how can I convert this into CSV?
Output should be:
1116559|P1303251287|20130325225906CD|13013822|1|0000
1104220|P1303250282|20130325070119CD| |1|0000
1064743|P1303251094|20130325191600CD| |0|0000
1100819|P1303250369|20130325091722CD| |0|0000
1101405|P1303250051|20130325010740CD| |2|0000
I'd do that with -replace
$Regex = '(.{7})\s{3}(.{11})\s{3}(.{16})\s{3}(.{8})\s{3}(.{1})\s{3}(.{4})'
$Replace = '$1|$2|$3|$4|$5|$6'
(Get-Content "filenamehere.txt") -replace $Regex,$Replace |
Set-Content "foo.txt"
Using your sample you can use trim()
$lines = Get-Content "c:\temp\filenamehere.txt"
ForEach ($x in $lines)
{
$y = "$(($($x[0..9] -join '')).trim())|$(($($x[10..23] -join '')).trim())|$(($($x[24..42] -join '')).trim())|$(($($x[43..53] -join '')).trim())|$(($($x[54..57] -join '')).trim())|$(($($x[58..61] -join '')).trim())|$(($($x[126..138] -join '')).trim())"
$z = $y -join '|'
Write-Output $z | Out-File -FilePath "c:\temp\foo.txt" -Append
}
Perhaps it remove too much spaces.
I output
1116559|P1303251287|20130325225906CD|13013822|1|0000|
1104220|P1303250282|20130325070119CD||1|0000|
1064743|P1303251094|20130325191600CD||0|0000|
1100819|P1303250369|20130325091722CD||0|0000|
1101405|P1303250051|20130325010740CD||2|0000|
Which should be better on a CSV point of view.
#echo off
for /F "tokens=1-6" %%a in (input.txt) do (
if "%%f" neq "" (
echo %%a^|%%b^|%%c^|%%d^|%%e^|%%f
) else (
echo %%a^|%%b^|%%c^| ^|%%d^|%%e
)
)
Output:
C:\> test.bat
1116559|P1303251287|20130325225906CD|13013822|1|0000
1104220|P1303250282|20130325070119CD| |1|0000
1064743|P1303251094|20130325191600CD| |0|0000
1100819|P1303250369|20130325091722CD| |0|0000
1101405|P1303250051|20130325010740CD| |2|0000
Using a ConvertFrom-FixedLengths function you could just do:
Get-Content "C:\input.txt" |
ConvertFrom-FixedLengths 10,14,19,11,4,4 -Trim |
Foreach { #($_.Column1, $_.Column2, $_.Column3, $_.Column4.PadLeft(8, ' '), $_.Column5, $_.Column6) -Join "|" } |
Out-File -FilePath "c:\output.txt"
Or, of course, if you want to create a csv-file with the | character as delimiter you could just do:
Get-Content "C:\input.txt" |
ConvertFrom-FixedLengths 10,14,19,11,4,4 -Trim |
Select Column1, Column2, Column3, #{ N = "Column4"; E = { $_.Column4.PadLeft(8) } }, Column5, Column6 |
Export-Csv -Path "C:\Output.csv" -NoTypeInformation -Delimiter "|"
Or, to make it even simpler, if you want a csv-file and don't need to pad the fourth column with spaces, you could just skip the `Select´ line in the last sample, making it into:
Get-Content "C:\input.txt" |
ConvertFrom-FixedLengths 10,14,19,11,4,4 -Trim |
Export-Csv -Path "C:\Output.csv" -NoTypeInformation -Delimiter "|"
For a decent solution, you need to handle the content as fixed length fields, the other answers here do that.
If you know only column 4 might be blank, you can bodge it for a one-off script by replacing an 11 character space with a comma (which will do nothing on rows where column 4 has content), then replacing spaces with commas:
Get-Content "data.txt" | % { ($_ -replace "\s{11}", ",") -replace "\s+", "," } > out.txt
Sample output:
1116559,P1303251287,20130325225906CD,13013822,1,0000
1104220,P1303250282,20130325070119CD,,1,0000
1064743,P1303251094,20130325191600CD,,0,0000
1100819,P1303250369,20130325091722CD,,0,0000
1101405,P1303250051,20130325010740CD,,2,0000
Working code..
CD 'C:\\FOLDERPATH\'
$filter = "FILE_NAME_*.txt"
$columns = 11,22,32,42,54
## DO NOT NEED TO REVERSE [array]::Reverse($columns) #too lazy to re-write array after finding out I need to iterate in reverse
$files = get-childitem ./ |where-object {$_.Name -like $filter}
$newDelimiter = '|'
foreach($file in $files)
{
$file
$csvFile = 'C:\\FOLDERPATH\NEW_' + $file.BaseName + '.txt'
if (!(get-childitem ./ |where-object {$_.Name -like $csvFile})) #check whether file has been processed
{
$content | ForEach {
$line = $_
$counter = 0
$columns | ForEach {
$line = $line.Insert($_+$counter, $newDelimiter)
$counter = $counter + 1
}
$line = $line.Trim($newDelimiter)
$line
} | set-content $csvFile
}
}
Related
I am trying to pull substrings out of text files in bulk saving these substrings to an array. I have tried variations of the following. This outputs all of the selected strings to the screen but only saves the final output to the variable. Is there a way to mimic the functionality of a =+ operator in outvariable so that all items get stored in an array?
$FILES = ls "*.txt"
foreach($f in $FILES){
$in=Get-Content $f
$in | Foreach { Select-String -Path "$f" -Pattern "Ad ID" -outvariable
array1 }}
In the event that my strategy is misguided, the overall purpose of pulling substrings into an array is to have several arrays of separate substrings of these text files. Then I will concatenate the values into a csv. I'm attempting to pull elements out rather than re-arrange the text files as substrings within the text files are in different order. Example:
Txt File One:
Ad Id: xxxx
Ad Text: blah blah
Ad placement: spaceship
Txt File Two:
Ad Id: yyyy
Ad placement: zoo
Ad Text: blah blah
Final desired result (this part is working except for the order of the elements)
CSV file
xxxx, spaceship, blah blah
yyyy, zoo, blah blah
Here is a way to build the array you are talking about. I do not think this is the best way to solve this problem. This does nothing about the order of results, nor does it create a .csv file.
$FILES = Get-ChildItem -File -Filter "*.txt"
$array1 = $()
foreach($f in $FILES) {
Get-Content -Path $f |
Select-String -Pattern "Ad Id.*" |
ForEach-Object { $array1 += #($_.Matches.Value) }
}
$FILES.Count
$array1.Count
$array1
Try this one:
$files = ls "*.txt"
$dictionary = #{}
foreach($f in $files) {
$in = Get-Content $f
$in.Split([Environment]::NewLine) | ForEach-Object {
$key,$value = $_.Split(':')
$dictionary[$key] = $value
}
$dictionary['Ad Id'] + ', ' + $dictionary['Ad placement'] + ', ' + $dictionary['Ad Text'] | Out-File -FilePath '.\results.csv' -Append
}
Sorted output:
$files = ls "fil*.txt"
$dictionary = #{}
[System.Collections.Generic.List[String]]$list = #()
foreach($f in $files) {
$in = Get-Content $f
$in.Split([Environment]::NewLine) | ForEach-Object {
$key,$value = $_.Split(':')
$dictionary[$key] = $value
}
[void]$list.Add( $dictionary['Ad Id'] + ', ' + $dictionary['Ad placement'] + ', ' + $dictionary['Ad Text'] )
}
[void]$list.Sort()
$list | Out-File -FilePath '.\results.csv' -Append
Another slightly different approach.
A RegEx parses $Line and creates a variable with the name before the colon (without Ad) and value what is behind
After each processed file the vars are output as a custom object
$Data = ForEach ($File in (Get-ChildItem File*.txt)){
$Id,$Text,$Placement="","",""
ForEach ($Line in (Get-Content $File)){
If ($Line -Match "AD (?<Label>.*?): (?<Value>.*)"){
Set-Variable -Name "$($Matches.Label)" -Value $Matches.Value
}
}
[PSCustomObject]#{ID = $Id
Placement = $placement
Text = $Text}
}
$Data
$Data | Export-CSv ".\Result.csv" -NoTypeInformation
Sample output:
ID Placement Text
-- --------- ----
xxxx spaceship blah blah
yyyy zoo blah blah
Bond.out file example (looking to replace what is highlighted):
Out.csv file (data to be used):
Code:
#set paths up
$filepath= 'C:\folder\path\bond.out'
$filepath2= 'C:\folder\path\temp.txt'
$Ticklist='C:\folder\path\tick.txt'
$ratelist='C:\folder\path\rate.txt'
#Import needed data from an excel file which creates and array
$csv = Import-CSV C:\folder\path\RateIDTable.csv | Where { $_.'Rate' -ne "" } | Export-Csv C:\folder\path\out.csv -NoTypeInformation
$bond = Import-CSV C:\folder\path\out.csv | select -Property TickerID, Rate
#Put array from Excel file into two text files
$Tick = $bond | foreach-object {$_.TickerID} | set-content $Ticklist
$replace = $bond | foreach-object {$_.rate} | set-content $Ratelist
#Create two separate arrays from the new text files
$Tickdata = (Get-content $Ticklist ) -join ','
foreach ($t in $Tickdata)
{
$t = $t -split(",")
$First = $t[0]}
$Ratedata = (Get-content $Ratelist ) -join ','
foreach ($r in $Ratedata)
{
$r = $r -split(",")
$First = $r[0]}
#Get main file to search (bond.out) and search for the word that is in the first line from "t" array file
$data = Select-String $filepath -pattern $t[0] | Select-Object -ExpandProperty Line
$data
#Once found, split the line, replace the rate on the 3rd line with the rate in the first line from the "r" array file, the put the line back to together
$split=$data.split("{|}")
$split[3]=$r[0]
$join = $split -join "|"
$join
#Put the updated line back into the "bond.out" file from whence it came
(get-content $filepath) -replace($data,$join) | set-content $filepath
#computer says no :(
Output:
As you can see, it actually replaces the rate and puts it all back like I need it to. But that last line doesn't seem to work. Instead I get the file back like so:
It appears as though it is repeating the same line from the $join parameter and adding letters to the beginning of each iteration.
I believe it has something to do with the '|' at the end of the line, and remember reading something about marking the beginning and end of lines some time ago, but can't find it anywhere.
Here's an idea. Instead of using regular expressions ...
The Import-Csv command has a -Delimiter parameter. Can you just import bond.out as a "CSV" (but with a pipe delimiter), and update it just like you would a CSV file?
Pseudo-code
### Convert bond.out to objects
$BondOut = Import-Csv -Delimiter '|' -Path $FilePath
### Get the line you want to update
$LineToUpdate = $BondOut.Where({ $PSItem.TickerID -eq 'BBG0019K2QZ5' })
### Update the Rate property from your source (out.csv)
$LineToUpdate.Rate = $SomeSource.Rate
### Export the modified objects to a new bond.out.modified file
$BondOut | Export-Csv -Delimiter '|' -Path 'bond.out.modified' -NoTypeInformation
As per PetSerAI's clue:
#set paths up
$filepath= 'C:\folder\path\bond.out'
$filepath2= 'C:\folder\path\temp.txt'
$Ticklist='C:\folder\path\tick.txt'
$ratelist='C:\folder\path\rate.txt'
#Import needed data from an excel file which creates and array
$csv = Import-CSV C:\folder\path\RateIDTable.csv | Where { $_.'Rate' -ne "" } | Export-Csv C:\folder\path\out.csv -NoTypeInformation
$bond = Import-CSV C:\folder\path\out.csv | select -Property TickerID, Rate
#Put array from Excel file into two text files
$Tick = $bond | foreach-object {$_.TickerID} | set-content $Ticklist
$replace = $bond | foreach-object {$_.rate} | set-content $Ratelist
#Create two separate arrays from the new text files
$Tickdata = (Get-content $Ticklist ) -join ','
foreach ($t in $Tickdata)
{
$t = $t -split(",")
}
$Ratedata = (Get-content $Ratelist ) -join ','
foreach ($r in $Ratedata)
{
$r = $r -split(",")
}
#Get main file to search (bond.out) and search for the word that is in the first line from "t" array file
###Replace all pipes with a comma
(get-content $filepath) -replace('\|', ',') | set-content $filepath
$data = Select-String $filepath -pattern $t[0] | Select-Object -ExpandProperty Line
$data
#Once found, split the line, replace the rate on the 3rd line with the rate in the first line from the "r" array file, the put the line back to together
$split=$data.split("{,}")
$split[3]=$r[0]
$join = $split -join ","
#Put the updated line back into the "bond.out" file from whence it came
###change all commas back to pipes
(get-content $filepath) -replace($data,$j) | set-content $filepath
(get-content $filepath) -replace(',', '|') | set-content $filepath
#computer says yay :D
Hi I need some help getting this to work :)
I have a file with names in a list.
FILE A.
SomeNameA
SomeNameB
SomeNameC
etc
Now I need to get a set of lines from FILE B:
ddrObjId_1=SomeNameA
adrrObjType_1=8
adrrObjZone_1=ZONE_A
addrObjIP_1=0.0.0.0
addrObjIP2_1=0.0.0.0
addrObjId_2=SomeNameB
adrrObjType_2=8
adrrObjZone_2=ZONE_B
addrObjIP_2=0.0.0.0
addrObjIP2_2=0.0.0.0
starting with the line with a value which matches one of the names in FILE A, and all subsequent lines until either another match is found of the end of the file. Hope this explains it better :)
So when I'm finished I need a new FILE C to be generated with the data:
address-object SomeNameA
type 1
host 0.0.0.0
mask 0.0.0.0
zone ZONE_A
address-object SomeNameB
type 8
host 0.0.0.0
mask 0.0.0.0
zone ZONE_B
etc
So I have been trying some different scripts found her on StackO, and I can now scan the file for a word from my file and it gets the data, so i need it to get the values and then make a new file witch holds the commands. etc.. Here is the code not Working but it's a start. Hope someone can make sens of things for me..
Clear-Host
$array = #('SomeNameA')
$found = #{}
Get-Content 'D:\Scripts\FileB.txt'| % {
$line = $_
foreach ($item in $array) {
if ($line.Split('=')[1] -like $item) { $found[$item] = $true }
foreach ($item in $line | Where-Object{$_ -like 'addrObjId_*=*' -or $_ -like 'addrObjType_*=*'}) {
write-host $item
}
}
}
$found.Keys | Out-File "C:\results.txt"
this should do the job:
$names = cat '.\fileA.txt'
$data = cat '.\fileB.txt'
$outfile = '.\fileC.txt'
rm -Force $outfile
foreach($d in ([string]$data -split 'addrObjId_\d+='))
{
if ($d -eq "") { continue }
$d = $d -split ' '
$obj = $d[0].Trim()
$type = ($d[1] -split "=")[1]
$zone = ($d[2] -split "=")[1]
$hostip = ($d[3] -split "=")[1]
$maskip = ($d[4] -split "=")[1]
if ($names.Contains($obj))
{
"address-object $obj" | Out-File -Append -FilePath $outfile
"type $type" | Out-File -Append -FilePath $outfile
"host $hostip" | Out-File -Append -FilePath $outfile
"mask $maskip" | Out-File -Append -FilePath $outfile
"zone $zone" | Out-File -Append -FilePath $outfile
}
}
Her is the code I used in the end :)
Thx
metix
Clear-Host
$names = cat 'FileA.txt'
$data = cat 'FileB.txt'
$outfile = 'C:\Temp\output.txt'
if(test-path $outfile){Remove-Item $outfile}
foreach($d in ([string]$data -split 'addrObjId_\d+='))
{
if ($d -eq "") { continue }
$d = $d -split ' '
$obj = $d[0].Trim()
$type = ($d[2] -split "=")[1]
$zone = ($d[3] -split "=")[1]
$hostip1 = ($d[5] -split "=")[1]
$hostip2 = ($d[6] -split "=")[1]
if ($names.Contains($obj))
{
"address-object ipv4 $obj" | Out-File -Append -FilePath $outfile
if ($type -eq '4'){"network $hostip1 $hostip2" | Out-File -Append -FilePath $outfile}elseif($type -eq '1'){"host $hostip1" | Out-File -Append -FilePath $outfile}elseif($type -eq '2'){"range $hostip1 $hostip2" | Out-File -Append -FilePath $outfile}
"zone $zone" | Out-File -Append -FilePath $outfile
}
}
I am trying to build an array with the results of the following powershell script. I would then like to only output the unique results in that array (I am expecting duplicates). I tried but it does not work. One thing I noticed is that my array is being converted to a character array. So if I call a single object in the array ($array[1]) it only displays one character, which may be why the select -unique is not working.
Please help.
$servers="Server1","server2"
$regex="(\.\d\d\d.*\()"
$path = "SomePath"
$yesterday = (get-date).Date.AddDays(-1)
Foreach ($server in $servers) {Get-ChildItem -Path $path -recurse | where { $_.LastWriteTime -le $yesterday } | ForEach-Object { Get-Content -Path "$_" -TotalCount 2 | select-string $regex | % { $_.Matches } | % { $array += $_.Value.Substring(5,6) } }}
$array|select -Unique
If you initialize the $array value as an array, then your code will work. Without this knowledge, PowerShell is treating $array += like concatenating strings together.
$array = #()
$servers="Server1","server2"
$regex="(\.\d\d\d.*\()"
$path = "SomePath"
$yesterday = (get-date).Date.AddDays(-1)
Foreach ($server in $servers) {Get-ChildItem -Path $path -recurse | where { $_.LastWriteTime -le $yesterday } | ForEach-Object { Get-Content -Path "$_" -TotalCount 2 | select-string $regex | % { $_.Matches } | % { $array += $_.Value.Substring(5,6) } }}
$array|select -Unique
I have many textfiles containing extended ascii characters. I would like to build a deduplicated list of these characters, for instance:
á
ö
¿
I am able to read files and strip out the basic ascii characters. However, since I am new to Powershell, how can I break each line into characters, compare them to an existing list of characters found and output a deduplicated list of non-basic ascii chars?
$files = Get-ChildItem "C:\Users\me\Desktop\ascii" -filter "*.txt"
Foreach ($file in $files) {
$newfile = #()
Get-Content $file.fullname | Foreach-Object {
$newfile += [string]([char[]]$_ | where-object {[int]$_ -lt 127})
}
Write-Host $newfile
}
[edit 1] Getting there...
$files = Get-ChildItem "C:\Users\me\Desktop\ascii" -filter "*.txt"
$array = #()
Foreach ($file in $files) {
Get-Content $file.fullname | Foreach-Object {
$line = [string]([char[]]$_ | where-object {[int]$_ -lt 127}) -split '\s+' | Foreach {
If ($array -notcontains $_) {
$array.Add($_)
}
}
}
}
[edit 2] This ^^^ works in PS 4 but not in 2 (which I have on my server)? Any help in writing the PS2 compatible version of this?
[edit 3] I found out that using $array = #() works fine on V2 and V4 :-)
Perhaps I'm missing something, but shouldn't you check for values greater than 127 if you're building a list of extended ASCII characters?
This should give you a list of extended ASCII characters from all files without duplicates:
Get-ChildItem 'C:\Users\me\Desktop\ascii' -filter '*.txt' |
Get-Content |
% { [char[]]$_ } |
? { [int]$_ -gt 127 } |
select -Unique
Any help? (requires V3)
$string = 'áTestáöö¿'
$ht = #{}
$basic,$extended = ([char[]]$string).Where({[int]$_ -lt 127},'Split')
$extended | foreach {$ht[$_] += $null}
[string]$basic
$ht.keys
T e s t
¿
ö
á