I saw that in batch, to delete a specific line from a text file, you need to do it with findstrthat allow you to find your line and then delete it. But is there a way to do it when you don't know the line ? I got another program that fills the file and the batch must delete the first line. Does anyone know how to do it ?
I tried with something thet reads a line from an index and then use what I got with findstr, but it doesn't work:
#echo off
setlocal EnableDelayedExpansion
set count=1
for /f "tokens=*" %%a in (test.txt) do (
if !count! equ 1 (set "TIMER=%%a")
if !count! equ 1 (type test.txt | findstr /v %TIMER%)
set /a count+=1
)
echo %TIMER%
timeout %TIMER%
for /f "tokens=*" %%a in (test.txt) do (
echo %%a
)
pause
It tells me : FINDSTR : wrong command line
(the piece of code for the loop on the lines of the file and find a specific line was found on the internet)
So where is the problem ? Or maybe does someone know something like delete(x) and it deletes the line ? Just something that takes an INDEX... ^^'
(The last for loop is used to check if the line was removed btw)
Thanks by advance for any help !
To remove the first n lines from a file without converting tabs into spaces:
#echo off
>RemoveFirstLine.txt <test.txt (
FOR /L %%N in (1 1 1) do set/p"=" SKIP N lines
%__APPDIR__%findstr.exe /R "^"
)
This works as FINDSTR moves the pointer from STDIN, while FIND and MORE do not.
Your code mainly doesn't work because the lack of delayed expansion.
But there are easier ways to achieve your goal. If Compo's suggestion (more.com" +1 "test.txt" > "modifiedtest.txt") doesn't work for you (need to keep TABs or need to remove another line than the first one), the following might work for you:
#echo off
setlocal
set file=test.txt
set lineToDelete=1
(for /f "tokens=1* delims=:" %%a in ('findstr /N "^" "%file%" ^|findstr /bv "%lineToDelete%:"') do echo/%%b) > "%file%.tmp
move /y "%file.tmp%" "%file%"
Related
I am writing a .bat program that will find and replace text in a file. The problem that I am having is that it is removing blank lines and left justifying the other lines. I need the blank lines to remain and the new text to remain in the same location. Here is what I have wrote, and also the result. Can anybody please help.
program:
#ECHO OFF
cls
cd\
c:
setLocal EnableDelayedExpansion
For /f "tokens=* delims= " %%a in (samplefile.tx) do (
Set str=%%a
set str=!str:day=night!
set str=!str:winter=summer!
echo !str!>>samplefile2.txt)
ENDLOCAL
cls
exit
samle File:
this line is the first line in my file that I am using as an example.This is made up text
the cat in the hat
day
winter
below is the result:
this line is the first line in my file that I am using as an example.This is made up text
the cat in the hat
night
summer
I need the lines, spaces and new text to remain in the same position while making the text replacement. Please help
Your use of "tokens=* delims= " will trim leading spaces. Instead, use "delims=" to preserve leading spaces.
FOR /F always skips empty lines. The trick is to insert something before each line. Typically FIND or FINDSTR is used to insert the line number at the front of each line.
You can use !var:*:=! to delete the the line number prefix from FINDSTR.
Use echo(!str! to prevent ECHO is off message when line is empty
It is more efficient (faster) to redirect only once.
#echo off
setlocal enableDelayedExpansion
>samplefile2.txt (
for /f "delims=" %%A in ('findstr /n "^" samplefile.txt') do (
set "str=%%A"
set "str=!str:*:=!"
set "str=!str:day=night!"
set "str=!str:winter=summer!"
echo(!str!
)
)
This still has a potential problem. It will corrupt lines that contain ! when %%A is expanded because of the delayed expansion. The trick is to toggle delayed expansion on and off within the loop.
#echo off
setlocal disableDelayedExpansion
>samplefile2.txt (
for /f "delims=" %%A in ('findstr /n "^" samplefile.txt') do (
set "str=%%A"
setlocal enableDelayedExpansion
set "str=!str:*:=!"
set "str=!str:day=night!"
set "str=!str:winter=summer!"
echo(!str!
endlocal
)
)
Or you could forget custom batch entirely and get a much simpler and faster solution using my JREPL.BAT utility that performs regular expression search and replace on text. There are options to specify multiple literal search/replace pairs.
jrepl "day winter" "night summer" /t " " /l /i /f sampleFile.txt /o sampleFile2.txt
I used the /I option to make the search case insensitive. But you can drop that option to make it case sensitive if you prefer. That cannot be done easily using pure batch.
#ECHO Off
SETLOCAL
(
FOR /f "tokens=1*delims=]" %%a IN ('find /n /v "" q27459813.txt') DO (
SET "line=%%b"
IF DEFINED line (CALL :subs) ELSE (ECHO()
)
)>newfile.txt
GOTO :EOF
:subs
SET "line=%line:day=night%"
SET "line=%line:winter=summer%"
ECHO(%line%
GOTO :eof
Thi should work for you. I used a file named q27459813.txt containing your data for my testing.
Produces newfile.txt
Will not work correctly if the datafile lines start ].
Revised to allow leading ]
#ECHO Off
SETLOCAL
(
FOR /f "delims=" %%a IN ('type q27459813.txt^|find /n /v "" ') DO (
SET "line=%%a"
CALL :subs
)
)>newfile.txt
GOTO :EOF
:subs
SET "line=%line:*]=%"
IF NOT DEFINED line ECHO(&GOTO :EOF
SET "line=%line:day=night%"
SET "line=%line:winter=summer%"
ECHO(%line%
GOTO :eof
I am creating a code that strips through different MAC addresses randomly, but cannot figure out how to do this. My thought on how to approach this is to randomize or rearrange the order of the MAC address in the text file with this script, but I cannot figure out how to do this with a batch file. How this will work is that it will read "maclist.txt", then create a new temp file with the random order "maclist_temp.txt", that will be the rearranged file. Then, it will pull this randomized file in order.
I have tried Google and searching the web, but I haven't found anything too useful. I'm still actively looking, but any advice would be extremely useful.
Something as simple as extracting and deleting a random line and then adding to the bottom might work. Randomization would be better though, but I want to keep the original list. Something like:
Make a temp copy of maclist.txt called maclist_temp.txt
Take one random MAC address, remove it from maclist_temp.txt
Readd it to the bottom
That is all I want, but any suggestions are welcome.
You may try this batch file to help you to shuffle your maclist.txt. The usage of the batch code is
C:\> type list.txt | shuffle.bat > maclist_temp.txt
Here are the contents of shuffle.bat:
#ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION
SET TmpFile=tmp%RANDOM%%RANDOM%.tmp
TYPE NUL >%Tmpfile%
FOR /F "tokens=*" %%i IN ('MORE') DO SET Key=!RANDOM!!RANDOM!!RANDOM!000000000000& ECHO !Key:~0,15!%%i>> %TmpFile%
FOR /F "tokens=*" %%i IN ('TYPE %TmpFile% ^| SORT') DO SET Line=%%i&ECHO.!Line:~15!
::DEL %TmpFile%
ENDLOCAL
After issuing the above command, maclist_temp.txt will contain a randomized list of MAC addresses.
Hope this helps.
Here is a simpler method to randomize/randomise a file, no temp files needed. You can even reuse the same input filename.
Limitations are: blank lines and line starting with ; will be skipped, and lines starting with = will have all leading = signs stripped and ^ characters are doubled.
#echo off
setlocal
for /f "delims=" %%a in (maclist.txt) do call set "$$%%random%%=%%a"
(for /f "tokens=1,* delims==" %%a in ('set $$') do echo(%%b)>newmaclist.txt
endlocal
I really like foxidrive's approach. Nevertheless I want to provide a solution with all the listed limitations eliminated (although cmd-related restrictions like file sizes < 2 GiB and line lengths < ~ 8 KiB remain).
The key is delayed expansion which needs to be toggled to not lose explamation marks. This solves all the potential problems with special characters like ^, &, %, !, (, ), <, >, | and ".
The counter index has been implemented in order not to lose a single line of the original text file, which could happen without, because random may return duplicate values; with index appended, the resulting variable names $$!random!.!index! are unique.
The findstr /N /R "^" command precedes every line of the original file with a line number followed by a colon. So no line appears empty to the for /F loop which would ignore such. The line number also implicitly solves the issue with leading semicolons, the default eol character of for /F.
Finally, everything up to and including the first colon (remember the said prefix added by findstr) is removed from every line before being output, hence no more leading equal-to signs are dismissed.
So here is the code:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set /A "index=0"
for /f "delims=" %%a in ('findstr /N /R "^" "%~dpn0.lst"') do (
setlocal EnableDelayedExpansion
for /F %%b in ("$$!random!.!index!") do (
endlocal
set "%%b=%%a"
)
set /A "index+=1"
)
> "%~dpn0.new" (
for /f "delims=" %%a in ('set $$') do (
set "item=%%a"
setlocal EnableDelayedExpansion
echo(!item:*:=!
endlocal
)
)
endlocal
exit /B
This seems to work. Feed it a command line parameter of the file to randomize.
#echo off
setlocal EnableDelayedExpansion
rem read the number of lines in the file
rem the find prepends the line number so we catch blank lines
for /f "delims=" %%n in ('find /c /v "" %1') do set "len=%%n"
set len=!len:*: =!
rem echo %1 has %len% lines
rem Relocate as many lines as there are lines in the file
for /l %%j in (1 1 !len!) do (
rem echo starting round %%j
rem geta random number between 1 and the number of lines in the file
set /a var=!random! %% !len! + 1
rem echo relocating line !var!
rem make sure there is no temp file
if exist %1.temp del %1.temp
rem read each line of the file, write any that don't match and then write the one that does
<%1 (
for /l %%i in (1 1 !len!) do (
rem if it is the target line then save it
if %%i == !var! (
set /p found=
rem echo saving !found!
)
rem if it is the target line then write it
if not %%i == !var! (
set /p other=
rem echo writing !other!
echo !other!>> %1.temp
)
)
rem now write the target line at the end
rem echo appending !found!
echo !found!>> %1.temp
)
rem replace the original with the temp version
move %1.temp %1>nul
)
rem print the result
type %1
Place in cmd file
for /f "tokens=2 delims=/" %%m in ('cmd /e:on /v:on /c "for /f %%f in (maclist.txt) do #echo !random!/%%f" ^| sort') do echo %%m
It spawns a cmd which reads the mac list in the inner for, prefixes a random value and a slash to the mac and sorts the list. Then this list is splitted in the outter for using the slash as delimiter and printing the mac address.
I'd like to print each line of 2 separate txt files alternately using a for loop in a batch file, I tried using an AND but was given: "AND was unexpected at this time" in cmd.exe when I ran my batch. Any ideas?
FOR /F "tokens=*" %%F in (!logPath!) AND for /f "tokens=*" %%H in (%%refLogPath) DO (
REM print each line of log file and refLog file sequentially
echo %%F
echo %%H
REM set logLine=%%F
REM check 'each line' of log file against ENG-REF.log
)
There isn't a keyword like AND, normally you couldn't solve this with two FOR loops.
But there is an alternative way to read a file with set /p.
setlocal EnableDelayedExpansion
<file2.txt (
FOR /F "delims=" %%A in (file1.txt) DO (
set /p lineFromFile2=
echo file1=%%A, file2=!lineFromFile2!
)
)
I believe this is as robust as a batch solution can get.
It handles blank lines in both files
It can read up to approximately 8k bytes on each line
The number of lines in the files does not have to match
A line can begin with any character (avoiding a FOR /F EOL issue)
A line can contain ! without getting corrupted (avoiding a problem of expanding a FOR
variable while delayed expansion is enabled)
Lines can be either Unix or Windows style.
Control characters will not be stripped from end of line
But this solution will get progressively slower as it reads a large file because it must rescan the 2nd file from the beginning for every line.
#echo off
setlocal disableDelayedExpansion
set "file1=file1.txt"
set "file2=file2.txt"
for /f %%N in ('find /c /v "" ^<"%file2%"') do set file2Cnt=%%N
findstr /n "^" "%file1%" >"%file1%.tmp"
findstr /n "^" "%file2%" >"%file2%.tmp"
set "skip=0"
set "skipStr="
for /f "usebackq delims=" %%A in ("%file1%.tmp") do (
set "ln1=%%A"
call :readFile2
set /a "skip+=1"
)
if %file2Cnt% gtr %skip% (
for /f "usebackq skip=%skip% delims=" %%B in ("%file2%.tmp") do (
set "ln2=%%B"
setlocal enableDelayedExpansion
set "ln2=!ln2:*:=!"
(echo()
(echo(!ln2!)
)
)
del "%file1%.tmp" 2>nul
del "%file2%.tmp" 2>nul
exit /b
:readFile2
if %skip% gtr 0 set "skipStr=skip=%skip% "
if %file2Cnt% gtr %skip% (
for /f "usebackq %skipStr%delims=" %%B in ("%file2%.tmp") do (
set "ln2=%%B"
goto :break
)
) else set "ln2="
:break
setlocal enableDelayedExpansion
set "ln1=!ln1:*:=!"
if defined ln2 set "ln2=!ln2:*:=!"
(echo(!ln1!)
(echo(!ln2!)
exit /b
Much better to use jeb's approach if that solution's limitations are not a concern with your files. It currently has the following limitations that could be removed with fairly minor modifications:
Files must have same number of lines
Files must not have blank lines
File1 must not contain ! character
No line in File1 can start with ;
In addition it has the following limitations when reading File2 that are inherent to the SET /P limitations
Lines must be Windows style, ending in carriageReturn lineFeed
Lines cannot exceed 1021 characters (bytes) excluding the line terminators
Control characters will be stripped off the end of each line
An even better solution would be to use something other than batch. There are many possibilities: VBS, JScript, PowerShell, perl ... the list goes on and on.
This DOS batch script is stripping out the blank lines and not showing the blank lines in the file even though I am using the TYPE.exe command to convert the file to make sure the file is ASCII so that the FIND command is compatible with the file. Can anyone tell me how to make this script include blank lines?
#ECHO off
FOR /F "USEBACKQ tokens=*" %%A IN (`TYPE.exe "build.properties" ^| FIND.exe /V ""`) DO (
ECHO --%%A--
)
pause
That is the designed behavior of FOR /F - it never returns blank lines. The work around is to use FIND or FINDSTR to prefix the line with the line number. If you can guarantee no lines start with the line number delimiter, then you simply set the appropriate delimiter and keep tokens 1* but use only the 2nd token.
::preserve blank lines using FIND, assume no line starts with ]
::long lines are truncated
for /f "tokens=1* delims=]" %%A in ('type "file.txt" ^| find /n /v ""') do echo %%B
::preserve blank lines using FINDSTR, assume no line starts with :
::long lines > 8191 bytes are lost
for /f "tokens=1* delims=:" %%A in ('type "file.txt" ^| findstr /n "^"') do echo %%B
::FINDSTR variant that preserves long lines
type "file.txt" > "file.txt.tmp"
for /f "tokens=1* delims=:" %%A in ('findstr /n "^" "file.txt.tmp"') do echo %%B
del "file.txt.tmp"
I prefer FINDSTR - it is more reliable. For example, FIND can truncate long lines - FINDSTR does not as long as it reads directly from a file. FINDSTR does drop long lines when reading from stdin via pipe or redirection.
If the file may contain lines that start with the delimiter, then you need to preserve the entire line with the line number prefix, and then use search and replace to remove the line prefix. You probably want delayed expansion off when transferring the %%A to an environment variable, otherwise any ! will be corrupted. But later within the loop you need delayed expansion to do the search and replace.
::preserve blank lines using FIND, even if a line may start with ]
::long lines are truncated
for /f "delims=" %%A in ('type "file.txt" ^| find /n /v ""') do (
set "ln=%%A"
setlocal enableDelayedExpansion
set "ln=!ln:*]=!"
echo(!ln!
endlocal
)
::preserve blank lines using FINDSTR, even if a line may start with :
::long lines >8191 bytes are truncated
for /f "delims=*" %%A in ('type "file.txt" ^| findstr /n "^"') do (
set "ln=%%A"
setlocal enableDelayedExpansion
set "ln=!ln:*:=!"
echo(!ln!
endlocal
)
::FINDSTR variant that preserves long lines
type "file.txt" >"file.txt.tmp"
for /f "delims=*" %%A in ('findstr /n "^" "file.txt.tmp"') do (
set "ln=%%A"
setlocal enableDelayedExpansion
set "ln=!ln:*:=!"
echo(!ln!
endlocal
)
del "file.txt.tmp"
If you don't need to worry about converting the file to ASCII, then it is more efficient to drop the pipe and let FIND or FINDSTR open the file specified as an argument, or via redirection.
There is another work around that completely bypasses FOR /F during the read process. It looks odd, but it is more efficient. There are no restrictions with using delayed expansion, but unfortunately it has other limitations.
1) lines must be terminated by <CR><LF> (this will not be a problem if you do the TYPE file conversion)
2) lines must be <= 1021 bytes long (disregarding the <CR><LF>)
3) any trailing control characters are stripped from each line.
4) it must read from a file - you can't use a pipe. So in your case you will need to use a temp file to do your to ASCII conversion.
setlocal enableDelayedExpansion
type "file.txt">"file.txt.tmp"
for /f %%N in ('find /c /v "" ^<"file.txt.tmp"') do set cnt=%%N
<"file.txt.tmp" (
for /l %%N in (1 1 %cnt%) do(
set "ln="
set /p "ln="
echo(!ln!
)
)
del "file.txt.tmp"
I wrote a very simple program that may serve as replacement for FIND and FINDSTR commands when they are used for this purpose. My program is called PIPE.COM and it just insert a blank space in empty lines, so all the lines may be directly processed by FOR command with no further adjustments (as long as the inserted space don't cares). Here it is:
#ECHO off
if not exist pipe.com call :DefinePipe
FOR /F "USEBACKQ delims=" %%A IN (`pipe ^< "build.properties"`) DO (
ECHO(--%%A--
)
pause
goto :EOF
:DefinePipe
setlocal DisableDelayedExpansion
set pipe=´)€ì!Í!ŠÐŠà€Ä!€ü.t2€ü+u!:æu8²A€ê!´#€ì!Í!².€ê!´#€ì!Í!²+€ê!´#€ì!Í!Šò€Æ!´,€ì!Í!"Àu°´LÍ!ëÒ
setlocal EnableDelayedExpansion
echo !pipe!>pipe.com
exit /B
EDIT: Addendum as answer to new comment
The code at :DefinePipe subroutine create a 88 bytes program called pipe.com, that basically do a process equivalent to this pseudo-Batch code:
set "space= "
set line=
:nextChar
rem Read just ONE character
set /PC char=
if %char% neq %NewLine% (
rem Join new char to current line
set line=%line%%char%
) else (
rem End of line detected
if defined line (
rem Show current line
echo %line%
set line=
) else (
rem Empty line: change it by one space
echo %space%
)
)
goto nextChar
This way, empty lines in the input file are changed by lines with one space, so FOR /F command not longer omit they. This works "as long as the inserted space don't cares" as I said in my answer.
Note that the pipe.com program does not work in 64-bits Windows versions.
Antonio
Output lines including blank lines
Here's a method I developed for my own use.
Save the code as a batch file say, SHOWALL.BAT and pass the source file as a command line parameter.
Output can be redirected or piped.
#echo off
for /f "tokens=1,* delims=]" %%a in ('find /n /v "" ^< "%~1"') do echo.%%ba
exit /b
EXAMPLES:
showall source.txt
showall source.txt >destination.txt
showall source.txt | FIND "string"
An oddity is the inclusion of the '^<' (redirection) as opposed to just doing the following:
for /f "tokens=1,* delims=]" %%a in ('find /n /v "" "%~1"') do echo.%%ba
By omitting the redirection, a leading blank line is output.
Thanks to dbenham, this works, although it is slightly different than his suggestion:
::preserve blank lines using FIND, no limitations
for /f "USEBACKQ delims=" %%A in (`type "file.properties" ^| find /V /N ""`) do (
set "ln=%%A"
setlocal enableDelayedExpansion
set "ln=!ln:*]=!"
echo(!ln!
endlocal
)
As mentioned in this answer to the above question, it doesn't seem that lines are skipped by default using for /f in (at least) Windows XP (Community - Please update this answer by testing the below batch commands on your version & service pack of Windows).
EDIT: Per Jeb's comment below, it seems that the ping command, in at least Windows XP, is
causing for /f to produce <CR>'s instead of blank lines (If someone knows specifically why, would
appreciate it if they could update this answer or comment).
As a workaround, it seems that the second default delimited token (<space> / %%b in the example)
returns as blank, which worked for my situation of eliminating the blank lines by way of an "parent"
if conditional on the second token at the start of the for /f, like this:
for /f "tokens=1,2*" %%a in ('ping -n 1 google.com') do (
if not "x%%b"=="x" (
{do things with non-blank lines}
)
)
Using the below code:
#echo off
systeminfo | findstr /b /c:"OS Name" /c:"OS Version"
echo.&echo.
ping -n 1 google.com
echo.&echo.
for /f %%a in ('ping -n 1 google.com') do ( echo "%%a" )
echo.&echo.&echo --------------&echo.&echo.
find /?
echo.&echo.
for /f %%a in ('find /?') do ( echo "%%a" )
echo.&echo.
pause
.... the following is what I see on Windows XP, Windows 7 and Windows 2008, being the only three versions & service packs of Windows I have ready access to:
I've got a somewhat weird case, where a for-loop is incredibly slow when I use findstr as the string for DO.
Its worth mentioning that the file (old-file.xml) that I'm processing contains about 200 000 lines.
This part is blazing fast, but can be rendered slower if I remove | find /c ":"
rem find total number of lines in xml-file
findstr /n ^^ old-file.xml | find /c ":" > "temp-count.txt"
set /p lines=< "temp-count.txt"
The code which is slow looks like this and I can't use the pipe trick above. It seems like the slow part is the for itself, as i'm not seeing any progress in the title bar until after 10 min.
setlocal DisableDelayedExpansion
rem start replacing wrong dates with correct date
for /f "usebackq Tokens=1* Delims=:" %%i in (`"findstr /n ^^ old-file.xml"`) do (
rem cache the value of each line in a variable
set read-line=%%j
set line=%%i
rem restore delayed expansion
setlocal EnableDelayedExpansion
rem write progress in title bar
title Processing line: !line!/%lines%
rem remove trailing line number
rem set read-line=!read-line:*:=!
for /f "usebackq" %%i in ("%tmpfile%") do (
rem replace all wrong dates with correct dates
set read-line=!read-line:%%i=%correctdate%!
)
rem write results to new file
echo(!read-line!>>"Updated-file.xml"
rem end local
endlocal
)
EDIT:
Further investigation showed me that using this single line that should display the current line number being looped takes about 10 minutes on my 8MB file of 200 000 lines. That's just for getting it to start displaying the lines.
for /f "usebackq Tokens=1* Delims=:" %%i in (`"findstr /n ^^ old-file.xml"`) do echo %%i
So it seems like findstr is writing screen output hidden for the user, but visible for the for-loop. How can I prevent that from happening while still getting the same results?
EDIT 2: Solution
The solution as proposed by Aacini and finally revised by me.
This is a snippet from a much bigger script. Wrong dates are retrieved in another loop. And total number of lines are also retrieved from another loop.
setlocal enabledelayedexpansion
rem this part is for snippet only, dates are generated from another loop in final script
echo 2069-04-29 > dates-tmp.txt
echo 2069-04-30 >> dates-tmp.txt
findstr /n ^^ Super-Large-File.xml > out.tmp
set tmpfile=dates-tmp.txt
set correctdate=2011-11-25
set wrong-dates=
rem hardcoded total number of lines
set lines=186442
for /F %%i in (%tmpfile%) do (
set wrong-dates=!wrong-dates! %%i
)
rem process each line in out.tmp and loop them through :ProcessLines
call :ProcessLines < out.tmp
rem when finished with above call for each line in out.tmp, goto exit
goto ProcessLinesEnd
:ProcessLines
for /L %%l in (1,1,%lines%) do (
set /P read-line=
rem write progress in title bar
title Processing line: %%l/%lines%
for %%i in (%wrong-dates%) do (
rem replace all wrong dates with correct dates
set read-line=!read-line:%%i=%correctdate%!
)
rem write results to new file
echo(!read-line:*:=!>>"out2.tmp"
)
rem end here and continue below
goto :eof
:ProcessLinesEnd
echo this should not be printed until call has ended
:exit
exit /b
Two points here:
1- The setlocal EnableDelayedExpansion command is executed with every line of the file. This means that about 200000 times the complete environment must be copied to a new local memory area. This may cause several problems.
2- I suggest you to start with the most basic part. How much time takes the findstr to execute? Run findstr /n ^^ old-file.xml alone and check this before trying to fix any other part. If this process is fast, then add a single step to it and test again until you discover the cause of the slow down. I suggest you not use pipes nor for /f over the execution of findstr, but over the file generated by a previous redirection.
EDIT A faster solution
There is another way to do this. You may pipe findstr output into a Batch subroutine, so the lines can be read with SET /P command. This method allows to process the lines entirely via delayed expansions and not via the command-line susbtitution of FOR /F, so the pair of setlocal EnableDelayedExpansion and endlocal commands are no longer necessary. However, if you still want to display the line number it is necessary to calculate it again.
Also, it is faster to load the wrong dates in a variable instead of process the %tmpfile% with every line of the big file.
setlocal EnableDelayedExpansion
rem load wrong dates from tmpfile
set wrong-dates=
for /F %%i in (%tmpfile%) do (
set wrong-dates=!wrong-dates! %%i
)
echo creating findstr output, please wait...
findstr /n ^^ old-file.xml > findstr.txt
echo :EOF>> findstr.txt
rem start replacing wrong dates with correct date
call :ProcessLines < findstr.txt
goto :eof
.
:ProcessLines
set line=0
:read-next-line
set /P read-line=
rem check if the input file ends
if !read-line! == :EOF goto :eof
rem write progress in title bar
set /A line+=1
title Processing line: %line%/%lines%
for %%i in (%wrong-dates%) do (
rem replace all wrong dates with correct dates
set read-line=!read-line:%%i=%correctdate%!
)
rem write results to new file
echo(!read-line:*:=!>>"Updated-file.xml"
rem go back for next line
goto read-next-line
SECOND EDIT An even faster modification
Previous method may be slighlty speeded up if the loop is achieved via for /L command instead of via a goto.
:ProcessLines
for /L %%l in (1,1,%lines%) do (
set /P read-line=
rem write progress in title bar
title Processing line: %%l/%lines%
for %%i in (%wrong-dates%) do (
rem replace all wrong dates with correct dates
set read-line=!read-line:%%i=%correctdate%!
)
rem write results to new file
echo(!read-line:*:=!>>"Updated-file.xml"
)
This modification also omit the :EOF comparison and the calculation of line number, so the time gain may be significative after repeated it 200000 times. If you use this method, don't forget to delete the echo :EOF>> findstr.txt line in first part.
A FOR /F expression will always executed/read/evaluated complete before the inner loop starts.
You can try it with
(
echo line1
echo line2
) > myFile.txt
FOR /F "delims=" %%a in (myFile.txt) DO (
echo %%a
del myFile.txt 2> nul >nul
)
It will display
line1
line2
In your case the complete ('"findstr /n ^^ old-file.xml"') will executed and cached before the loop can start
EDIT: Added Solution
I measured with a file ~20MB with 370.000 lines
type testFile.txt > nul
findstr /n ^^ testFile.txt > nul
for /F "delims=" %%a in (testFile.txt) do (
rem Nothing
)
for /f "usebackq delims=" %%a in (`"findstr /n ^^ testFile.txt"`) do ...
findstr /n ^^ testFile.txt > out.tmp
type_nul ~10000ms
findstr_nul ~30000ms
for_file ~ 1600ms
for_findstr cancled after 10 minutes
findstr_tmp ~ 500ms !!!
I would recommend to use a temporary file, it's extremly fast.
findstr /n ^^ myFile.txt > out.tmp
set lineNr=0
(
for /f "usebackq delims=" %%a in ("out.tmp") do (
set /a lineNr+=1
set "num_line=%%a"
setlocal EnableDelayedExpansion
set "line=!num_line:*:=!"
echo(!line!
endlocal
)
) > out2.tmp
Btw. Your for/F splitting can fail, if the original line begins with a colon
for /f "usebackq Tokens=1* Delims=:"
Sample: :ThisIsALabel
:ThisIsALabel
Findstr /n prepends a line number
17::ThisIsALabel
the delims=: will split the first token and handles all colons as only one seperator
ThisIsALabel