I am trying to replace some text in a file. Currently I am replacing IP addresses with:
(Get-Content $editfile) | ForEach-Object { $_ -replace "10.10.37.*<", "10.10.37.$BusNet<" } | Set-Content $editfile
This code works well here. However I can't get the code to work with another line:
<dbserver>SVRNAME</dbserver>
Here is the code I have written for this line:
(Get-Content $editfile) | ForEach-Object { $_ -replace "<dbserver>*</dbserver>", "$DbSVRName" } | Set-Content $editfile
The code above should replace SVRNAME with the DbSVRName. Yet it does not. I know it's simple, and I know I am going to feel dumb afterwards. What am I missing?
While debugging trying to find a solution I found that for some reason it can't see the *
(Get-Content $editfile) | ForEach-Object { $_ -match "<dbserver>*</dbserver>" }
This code reveals all falses results.
* doesn't capture stuff in regex, you need .* and specifically (.*?)
$str = 'text <dbserver>SVRNAME</dbserver> text'
$replace = 'foo'
$str -replace '(<dbserver>)(.*?)(</dbserver>)', ('$1'+$replace+'$3')
Output
text <dbserver>foo</dbserver> text
Related
I am trying to remove duplicates and leave only unique entries from the output of 2 queries.
I am pulling a list of installed Windows Updates using the following (also stripping 12 chars of whitespace and dropping to lower case:
$A = #(Get-HotFix | select-object #{Expression={$_.HotFixID.ToLower()}} | ft -hidetableheaders | Out-String) -replace '\s{12}',''
I am then querying a list of available files in a folder and stripping 3 trailing whitespace chars using:
$B = #(Get-ChildItem D:\y | select-object 'Name' | ft -hidetableheaders | Out-String) -replace '\s{3}',''
The problem I have is that the first query ($A) returns output like:
kb4040981
kb4041693
kb2345678
kb8765432
While the second query ($B) returns output like:
windows8.1-kb4040981-x64_d1eb05bc8c55c7632779086079c7759f40d7386f.cab
windows8.1-kb4041687-x64_3bdf264bcfc0dda01c2eaf2135e322d2d6ce6c64.cab
windows8.1-kb4041693-x64_359b7ac71a48e5af003d67e3e4b80120a2f5b570.cab
windows8.1-kb4049179-x64_e6ec21d5d16fa6d8ff890c0c6042c2ba38a1f7c4.cab
I need to compare the 2 outputs using wildcards around each entry in the $A array (I think), and where it exists in $B remove the entire line from $B array.
I cannot truncate the output of $B as I need to use the full filenames in a subsequent process.
IE in the example output above, the entire FIRST and THIRD lines would be remove from the $B array and other lines left intact.
I have tried numerous methods from online searches, and used foreach loops, all to no avail.
Thank you in advance for any assistance.
What did you try with foreach loops that didn't work? Unless your output is huge, this method is pretty striaght forward.
$a = "kb4040981","kb4041693","kb2345678","kb8765432","test"
[System.Collections.ArrayList]$b = "windows8.1-kb4040981-x64_d1eb05bc8c55c7632779086079c7759f40d7386f.cab","windows8.1-kb4041687-x64_3bdf264bcfc0dda01c2eaf2135e322d2d6ce6c64.cab","windows8.1-kb4041693-x64_359b7ac71a48e5af003d67e3e4b80120a2f5b570.cab","windows8.1-kb4049179-x64_e6ec21d5d16fa6d8ff890c0c6042c2ba38a1f7c4.cab"
$toRemove = New-Object System.Collections.ArrayList
foreach($kb in $a)
{
foreach($line in $b)
{
if($line -match $kb)
{
write-host "$kb found in: $line" -ForegroundColor Green
$toRemove.add($line) | out-null
}
}
}
foreach($line in $toRemove)
{
$b.Remove($line)
}
$b
Hope it helps.
I would recommend for you to take a little time to learn the very basics of Powershell. When you use format cmdlets and text files instead of objects you cut yourself of the good stuff. ;-)
Here is how I would start the task:
$A = Get-HotFix
$B = Get-ChildItem D:\y | Select-Object -Property Name,#{Name='HotFixID';Expression={($_.BaseName -split '-')[1]}}
Compare-Object -ReferenceObject $A -DifferenceObject $B -Property 'HotFixID' -PassThru
Sincere thanks to sambardo for his patience and input! The final working solution based on his excellent recommendation is:
$a = (Get-Hotfix).hotfixID
$b = (Get-ChildItem D:\y\ -file *.cab).name
$toRemove = New-Object System.Collections.ArrayList
foreach($kb in $a)
{
foreach($line in $b)
{
if($line -match $kb)
{
# write-host "$kb found in: $line" -ForegroundColor Green
$toRemove.add($line) | out-null
}
}
}
foreach($line in $toRemove)
{
$b.Remove($line)
}
$b
I've got a script that searches for a string ("End program" in this case). It then goes through each file within the folder and outputs any files not containing the string.
It works perfectly when the phrase is hard coded, but I want to make it more dynamic by creating a text file to hold the string. In the future, I want to be able to add to the list of string in the text file. I can't find this online anywhere, so any help is appreciated.
Current code:
$Folder = "\\test path"
$Files = Get-ChildItem $Folder -Filter "*.log" |
? {$_.LastWriteTime -gt (Get-Date).AddDays(-31)}
# String to search for within the file
$SearchTerm = "*End program*"
foreach ($File in $Files) {
$Text = Get-Content "$Folder\$File" | select -Last 1
if ($Text | WHERE {$Text -inotlike $SearchTerm}) {
$Arr += $File
}
}
if ($Arr.Count -eq 0) {
break
}
This is a simplified version of the code displaying only the problematic area. I'd like to put "End program" and another string "End" in a text file.
The following is what the contents of the file look like:
*End program*,*Start*
If you want to check whether a file contains (or doesn't contain) a number of given terms you're better off using a regular expression. Read the terms from a file, escape them, and join them to an alternation:
$terms = Get-Content 'C:\path\to\terms.txt' |
ForEach-Object { [regex]::Escape($_) }
$pattern = $terms -join '|'
Each term in the file should be in a separate line with no leading or trailing wildcard characters. Like this:
End program
Start
With that you can check if the files in a folder don't contain any of the terms like this:
Get-ChildItem $folder | Where-Object {
-not $_.PSIsContainer -and
(Get-Content $_.FullName | Select-Object -Last 1) -notmatch $pattern
}
If you want to check the entire files instead of just their last line change
Get-Content $_.FullName | Select-Object -Last 1
to
Get-Content $_.FullName | Out-String
I have hundreds of text files in a folder which can often reference each other, and go serveral levels deep. Not sure if I am explaining this well, so I will explain with an example.
Let's say folder "A" contains 500 .txt files. The first one could be called A.txt and somewhere in there it mentions B.txt, which in turn mentions C.txt and so on. I believe the number of levels down is no more than 10.
Now, I want to find a certain text strings which relate to A.txt by programmitically going through that file, then if it sees references to other .txt files go through them as well and so on. The resulting output would be something like A_out.txt which contains everything it found based on a regex.
I started out with this using Powershell but am now a little stuck:
$files = Get-ChildItem "C:\TEST\" -Filter *.txt
$regex = ‘PCB.*;’
for ($i=0; $i -lt $files.Count; $i++) {
$infile = $files[$i].FullName
$outfile = $files[$i].BaseName + "_out.txt"
select-string $infile -Pattern $regex -AllMatches | % { $_.Matches } | % { $_.Value } > $outfile
}
It goes through every .txt file and outputs everything that matches the PCB.*; expression to its corresponding _out.txt file.
I have absolutely no idea how to now expand this to include references to the other files. I'm not even sure if this is possible in PowerShell or whether I need to use another language to achieve what I want.
I could get some office monkey's to do all this manually but if this is relatively simple to code then it would save us a lot of time. Any help would be greatly appreciated :)
/Edit
Whilst running through this in my head, I thought I could build up an array for every time another one of the files is mentioned, and then repeat the process for those as well. However, back to my original problem, I have no idea how I would go about this.
/Edit 2:
Sorry, had been away for a few days and am only just picking this up. I have been using what I've learnt from this question and a few others to come up with the following:
function Get-FileReference
{
Param($FileName, $OutputFileName='')
if ($OutputFileName -eq '')
{
Get-FileReference $FileName ($FileName -replace '.xml$', '_out.xml')
}
else
{
Select-String $FileName -Pattern 'BusinessObject.[^"rns][w.]*' -AllMatches | % { $_.Matches } | % { $_.Value } | Add-Content $OutputFileName
Set-Location C:\TEST
$References = (Select-String -Pattern '(?<=resid=")d+' -AllMatches -path $FileName | % { $_.Matches } | % { $_.Value })
Write "SC References: $References" | Out-File OUTPUT.txt -Append
foreach ($Ref in $References)
{
$count
Write "$count" | Out-File OUTPUT.txt -Append
$count++
Write "SC Reference: $Ref" | Out-File OUTPUT.txt -Append
$xml = [xml](Get-Content 'C:\TEST\package.xml')
$res = $xml.SelectSingleNode('//res[#id = $Ref]/child::resver[last()]')
$resource = $res.id + ".xml"
Write "File to Check $resource" | Out-File OUTPUT.txt -Append
Get-FileReference $resource $OutputFileName
}
}
}
$files = gci "C:\TEST" *.xml
ForEach ($file in $files) {
Get-FileReference $file.FullName
}
Following my original question, I realised that this was a little bit more extensive than I originally thought and therefore had to tinker.
These are the noteable points:
All the parent files are .xml and code that matches on
"BusinessObject" etc works as expected.
The references to other
files are not simply .txt but require a pattern match of
'(?<=resid=")d+'.
This pattern match needs to be cross referenced with another file package.xml and based on the value
it returns, the file it next needs to look into is [newname].xml
As before, those child .xml files could reference some of the
other .xml files
The code I have pasted above seems to be getting stuck in endless loops (hence why I have debugging in there at the moment) and it is not liking the use of $Ref in:
$res = $xml.SelectSingleNode('//res[#id = $Ref]/child::resver[last()]')
That results in the following error:
Exception calling "SelectSingleNode" with "1" argument(s): "Namespace Manager or XsltContext needed. This query has a prefix, variable, or user-defined function."
Since there could be hundreds of files it dies when it gets over 1000+.
A recursive function which tries to do what you want.
function Get-FileReference
{
Param($FileName, $OutputFileName='')
if ($OutputFileName -eq '')
{
Get-FileReference $FileName ($FileName -replace '\.txt$', '_out.txt')
}
else
{
Select-String -Pattern 'PCB.*;' -Path $FileName -AllMatches | Add-Content $OutputFileName
$References = (Select-String -Pattern '^.*\.txt' -AllMatches -path $FileName).Matches.Value
foreach ($Ref in $References)
{
Get-FileReference $Ref $OutputFileName
}
}
}
$files = gci *.txt
ForEach ($file in $files) { Get-FileReference $file.FullName }
It takes two parameters - a filename and an output filename. If called without an output filename, it assumes it's at the top of a new recursion tree and generates an output filename to append to.
If called with an output filename (i.e. by itself) it searches for PCB patterns, appends to the output, then calls itself on any file references, with the same output filename.
Assuming that file references are lines on their own with no spaces xyz.txt.
I have a pipe-delimited file containing 5 columns. I need to append a sixth (pipe-delimited) column to the end of each row.
Old data:
a|b|c|d|e
p|q|r|s|t
New Data:
a|b|c|d|e|x
p|q|r|s|t|x
The sixth column (x) is a value which read from a text-file.
I am wondering if there is a quick way to append this data into existing data-file using powershell? The file contains variable number of rows (between 10 to 100,000)
Any help is appreciated
Simple text operations should work:
$replace = 'x'
(Get-Content file.txt) -replace '$',"|$replace"
a|b|c|d|e|x
p|q|r|s|t|x
For large files, you can do this:
$replace = 'x'
filter add-data {$_ -replace '$',"|$replace"}
Get-Content file.txt -ReadCount 1000 | add-data | add-content newfile.txt
That should produce very good performance with large files.
Assuming that your data does not have any headers in the CSV already, then you'll have to define the headers with the -Headers parameter of the Import-Csv cmdlet. To run the example below, put your data into a file called c:\test\test.csv. Then, run the script in PowerShell or PowerShell ISE.
# 1. Import the data
$Data = Import-Csv -Delimiter '|' -Path c:\test\test.csv -Header prop1,prop2,prop3,prop4,prop5;
# 2. Add a new member to each row
foreach ($Item in $Data) {
Add-Member -InputObject $Item -MemberType NoteProperty -Name prop6 -Value x;
}
# 3. Export the data to a new CSV file
$Data | Export-Csv -Delimiter '|' -Path c:\test\test.new.csv -NoTypeInformation;
# 4. Remove the double quotes around values
(Get-Content -Path c:\test\test.new.csv -Raw) -replace '"','' | Set-Content -Path c:\test\test.new.csv;
Original Data
The source data in c:\test\test.csv should look like this (according to your original post):
a|b|c|d|e
p|q|r|s|t
Resulting Data
After executing the script, your resulting data in c:\test\test.new.csv will look like this:
prop1|prop2|prop3|prop4|prop5|prop6
a|b|c|d|e|x
p|q|r|s|t|x
Random Sample Data Generation
Here is a short script that will generate a 10,000-line, randomized sample file to c:\test\test.csv:
$Random = { [System.Text.ASCIIEncoding]::ASCII.GetString((1..5 | % { [byte](Get-Random -Minimum 97 -Maximum 122); })).ToCharArray(); };
1..10000 | % { #('{0}|{1}|{2}|{3}|{4}' -f (& $Random)) } | Set-Content -Path c:\test\test.csv;
After running my first script against this sample data (10,000 lines), the result took: 1,729 milliseconds to execute. I would say that's pretty fast. Not that this is a race or anything.
I ran the sample file generator again, to generate 100,000 lines of data. After running the same script against that data, it took 19,784 milliseconds to run. It's roughly proportional to the 10,000 line test, but all in all, still doesn't take all that long. Is this a one-time thing, or does it need to be run on a schedule?
You could loop through the file line for line and just append the value in the loop:
Edit full sample code:
function append{
process{
foreach-object {$_ + "|x"}}}
$a = get-content yourcsv.csv
$a | append | set-content yourcsv.csv
I've got the following script that works and I'm trying to rewrite in a more efficient manner. The following lines of code below work and accomplish what I want:
Get-Content "C:\Documents and Settings\a411882\My Documents\Scripts\printserveroutput.txt" | Select-String -SimpleMatch -AllMatches -Pattern "/GPR" | % {
$_.Line -replace '(?:.*)(/GPR)(?:.*)(?<=on\s)(\w+)(?:.*)', '$1,$2'
}
Get-Content "C:\Documents and Settings\a411882\My Documents\Scripts\printserveroutput.txt" | Select-String -SimpleMatch -AllMatches -Pattern "DV6" | % {
$_.Line -replace '(?:.*)(DV6)(?:.*)(?<=on\s)(\w+)(?:.*)', '$1,$2'
}
I repeat the same exact lines of code seven times with slightly altering what I'm looking for. The output that I get is the following, which I'd want (note: this is just a small output):
/GPR,R3556
/GPR,R3556
While this works, I really don't like how cluttered the code is and I decided to try re-writing it in a more effective method. I've re-written the code like this:
$My_Arr = "/GPR", "DV6", "DV7", "RT3", "DEV", "TST", "PRE"
$low = $My_Arr.getlowerbound(0)
$upper = $My_Arr.getupperbound(0)
for ($temp=$low; $temp -le $upper; $temp++){
$Test = $My_Arr[$Temp]
Get-Content "C:\Documents and Settings\a411882\My Documents\Scripts\printserveroutput.txt" | Select-String -SimpleMatch -AllMatches -Pattern $My_Arr[$temp] | % {
$_.Line -replace '(?:.*)($Test)(?:.*)(?<=on\s)(\w+)(?:.*)', '$1,$2'
}
}
The output that this gives me is the following:
10 Document 81, A361058/GPR0000151814_1: owned by A361058 was printed on R3556 via port IP_***.***.***.***. Size in bytes: 53704; pages printed: 2 20130219123105.000000-300
10 Document 80, A361058/GPR0000151802_1: owned by A361058 was printed on R3556 via port IP_***.***.***.***. Size in bytes: 53700; pages printed: 2 20130219123037.000000-300
This is almost correct however the -replace line is where my error is occurring since the code is expecting a string instead of variable when it reaches ($Test). It is posting the entire line of the text file that I'm parsing each time that I find /GPR in this example, rather than the desired output shown above. Would anyone know of a method to fix this line and get the same output as the original code I was using?
EDIT: the output that I'm getting right now with the newer code is also the exact text that is in the .txt file I'm trying to parse through. There's more lines than that in the .txt but for the most part it is identical to that. I'm only concerned with getting the /GPR or any of the other possible strings in the array and then the server name which comes after the word "on" each time.
I'd say this is caused by the simple quotes, which prevent variable expansion.
PS is trying to replace the exact string '(?:.*)($Test)(?:.*)(?<=on\s)(\w+)(?:.*)' without replacing the $test variable by its value.
Try replacing them with quotes, but keep simple quotes on the second string, as follows:
Get-Content "C:\Documents and Settings\a411882\My Documents\Scripts\printserveroutput.txt" | Select-String -SimpleMatch -AllMatches -Pattern $My_Arr[$temp] | % {
$_.Line -replace "(?:.*)($Test)(?:.*)(?<=on\s)(\w+)(?:.*)", '$1,$2'
Does this work on your data?
$file = 'C:\Documents and Settings\a411882\My Documents\Scripts\printserveroutput.txt'
$regex = '(?:.*)(/GPR|DV6|DV7|RT3|DEV|TST|PRE)(?:.*)(?<=on\s)(\w+)(?:.*)'
(Get-Content $file) -match $regex -replace $regex,'$1,$2'