The following is the batch script i have written
#echo off
setlocal enabledelayedexpansion
set finalcontent=
For /F "tokens=1-2* delims= " %%I in (abc.txt) do (
IF %%J EQU MAJORVER (
set currentline=%%I %%J %1
set finalcontent=!finalcontent!!currentline!
) ELSE IF %%J EQU MINORVER (
set currentline=%%I %%J %2
set finalcontent=!finalcontent!!currentline!
) ELSE IF %%J EQU BUILDNUM (
set currentline=%%I %%J %3
set finalcontent=!finalcontent!!currentline!
) ELSE (
set currentline=%%I %%J %%K%NL%
set finalcontent=!finalcontent!!currentline!
)
)
echo %finalcontent%>>xyz.txt
I want a newline character appended at the end of every occurence of the variable currentline. Can anyone guide me on this?
You can create a real newline character and assign it to a variable.
setlocal EnableDelayedExpansion
set LF=^
rem TWO empty lines are required
echo This text!LF!uses two lines
The newline best works with delayed expansion, you can also use it with the percent expansion, but then it's a bit more complex.
set LF=^
rem TWO empty lines are required
echo This text^%LF%%LF%uses two lines
echo This also^
uses two lines
How it works?
The caret is an escape character, it escapes the next character and itself is removed.
But if the next character is a linefeed the linefeed is also removed and only the next character is effectivly escaped (even if this is also an linefeed).
Therefore, two empty lines are required, LF1 is ignored LF2 is escaped and LF3 is neccessary to finish the "line".
set myLinefeed=^<LF1>
<LF2>
<LF3>
Hints:
It's often better to use a quite different format of the newline variable definition,
to avoid an inadvertently deletion of the required empty lines.
(SET LF=^
%=this line is empty=%
)
I have removed to often one of the empty lines and then I searched forever why my program didn't work anymore.
And the paranoid version checks also the newline variable for whitespaces or other garbage.
if "!LF!" NEQ "!LF:~0,1!" echo Error "Linefeed definition is defect, probably multiple invisble whitespaces at the line end in the definition of LF"
FOR /F "delims=" %%n in ("!LF!") do (
echo Error "Linefeed definition is defect, probably invisble whitespaces at the line end in the definition of LF"
)
You can define the newline as follows:
set newline=^& echo.
Then, you can use the newline in the echo-statement, like this: (not applicable in the outlined situation, however)
echo %finalcontent%%newline%
or you can use the newline in each set-statement, like this: (mind the extra caret)
set finalcontent=!finalcontent!!currentline!^!newline!
or similarly:
set currentline=%%I %%J %1^%newline%
Related
I need to parse a text file.
I want to find the firstline in the text file
: the first line to find
set firstLine=------------------------------------------------------------------------------------------------------------------
and find the last line
:: the last line to find
set lastLine=*******************************************************************************************************************
Then I need to export to a new file everything between those two line.
echo >> M:\TESTING\Output.txt
I'm a beginner with this and I've searched for days, but am not finding how to do this.
I looked at for loops and if statements, but I'm still puzzled.
for /f "tokens=1 delims= " %%f in (M:\TESTING\*.txt) do (
:: sets then the line variable to the line just read
set line=%%f
:: the first line to find
set firstLine=------------------------------------------------------------------------------------------------------------------
:: the last line to find
set lastLine=*******************************************************************************************************************
Then if %line% = %fistLine% start the export.....
Any direction will be appreciated. thanks.
#DennisvanGils' approach is a good start and will do well in many cases.
However, it will not produce an exact copy of the text file content between the given lines:
leading whitespaces (SPACE and TAB) will be removed (due to tokens=* option),
lines starting with ; will be skipped (due to the default option eol=; of for /F), and
empty lines will be skipped as well (as for /F always skips such).
To get an exact copy of the text file portion, you could use the following code snippet:
#echo off
set "INFILE=M:\TESTING\input.txt"
set "OUTFILE=M:\TESTING\output.txt"
set "FIRSTLINE=------------------------------------------------------------------------------------------------------------------"
set "LASTLINE=*******************************************************************************************************************"
setlocal EnableExtensions DisableDelayedExpansion
set "FLAG="
> "%OUTFILE%" (
for /F "delims=" %%L in ('findstr /N "^" "%INFILE%"') do (
set "LINE=%%L"
setlocal EnableDelayedExpansion
set "LINE=!LINE:*:=!"
if "!LINE!"=="%FIRSTLINE%" (
endlocal
set "FLAG=TRUE"
) else if "!LINE!"=="%LASTLINE%" (
endlocal
goto :CONTINUE
) else if defined FLAG (
echo(!LINE!
endlocal
) else (
endlocal
)
)
)
:CONTINUE
endlocal
Core function here is findstr, which is configured so that every line in the file is returned, prefixed with a line number and a colon :. Its output is then parsed by a for /F loop. Because of the prefix, no line appears to be empty and therefore every one is iterated by the loop. In the body of the loop, the prefix is removed by the set "LINE=!LINE:*:=!" command for each line.
The variable FLAG is used to decide whether or not the current line is to be output; if it is defined, that is, a value is assigned, the command echo !LINE! is executed; otherwise it is not. FLAG is set if the current line matches the string in %FIRSTLINE%; if the line matches the string in %LASTLINE%, a goto command is executed which breaks the loop. This means also that only the first block between %FIRSTLINE% and %LASTLINE% matches is output.
If there might occur multiple %FIRSTLINE% and %LASTLINE% matches within the text file and you want to output every block, replace the goto command line by set "FLAG=".
Note that this approach does not check whether %FIRSTLINE% occurs before %LASTLINE%, nor does it even check for existence of %LASTLINE% (all remaining lines to the end of file are output in case). If all this is important, the logic need to be improved and even a second loop will be required most likely.
What you should do in this case is use a variable like a boolean to know if you encountered the startline and endline yet, and to know if you have to output the lines.
Also, you should use setlocal ENABLEDELAYEDEXPANSION with ! instead of % so you can change variables in loops and ifs (for more information about that, see this. The usage of () after if is not needed in this case, since the if is on one line, but they make things easier to read in my opinion. If you want to output the start and endline too, switch the checks for the start and endlines.
#echo off & setlocal ENABLEDELAYEDEXPANSION
set start=0
:: the first line to find
set firstLine=------------------------------------------------------------------------------------------------------------------
:: the last line to find
set lastLine=*******************************************************************************************************************
for /F "tokens=*" %%A in (TEST.txt) do (
:: sets then the line variable to the line just read
set line=%%A
if "!line!"=="!lastLine!" (set start=0)
if !start! equ 1 (echo !line!>>TESTOUTPUT.txt)
if "!line!"=="!firstLine!" (set start=1)
)
This should do what you want. Note that when you encounter a startline a second time it starts reading again.
I have a script in which I read html files which I want to edit. Here I paste the code which calls :remove_redundant_columns subroutine.
It should remove the spaces/white spaces from begin of each line and remove from html file. Only problem is that it adds extra text like = to lines which are almost empty, just have few tabs.
The html file which I downloaded is from hidemyass.com/proxy-list/1
call parse_proxy.bat remove_redundant_columns !FILENAME!
exit /b
:remove_redundant_columns
REM Remove whitespaces from begin of lines and <span></span>
FOR /f "tokens=*" %%t in (%1) do (
SET S=%%t
SET S=!S:^<span^>^</span^>=!
if NOT "!S!"=="" >>$tmp$ echo !S!
)
del %1
REN $tmp$ %1
exit /b
If you believe, that's your only problem... You need to check, if your variable S contains content.
That's required, as substitution on an undefined variable will not produce an undefined/empty variable, the new content will be the substitution text.
:remove_redundant_columns
REM Remove whitespaces from begin of lines and <span></span>
FOR /f "tokens=*" %%t in (%1) do (
SET S=%%t
if defined S (
SET S=!S:^<span^>^</span^>=!
>>$tmp$ echo !S!
)
)
As dbenham stated, you got many other problems,
and one additional problem is the echo !S! command itself.
ECHO has some nasty side effects on different content.
If the content is empty (or only spaces) then it will print it's currently state
ECHO IS OFF
If the content is OFF or ON it will NOT be echoed, it will only change the state.
And if the content is /? it will echo the help instead of /?.
To solve this you could simply change ECHO !S! to ECHO(!S! and all problems are gone.
jeb already solved your = problem (once the extra IF DEFINED check is added to his answer). But you may have at least one other problem.
I agree with Joey that you should not be using batch to manipulate HTML like this. But, if you really want to...
Your potential problem is that HTML usually has ! characters sprinkled within. Your code uses delayed expansion, but that causes corruption of FOR variable expansion when it contains ! character(s). The solution is to toggle delayed expansion on and off within your loop.
:remove_redundant_columns
setlocal disableDelayedExpansion
REM Remove whitespaces from begin of lines and <span></span>
(
FOR /f "usebackq eol= tokens=*" %%t in ("%~1") do (
SET S=%%t
setlocal enableDelayedExpansion
if defined S SET "S=!S:<span></span>=!"
for /f "eol= tokens=*" %%S in ("!S!") do if "%%S" neq "" echo %%S
endlocal
)
) >>$tmp$
move /y $tmp$ "%~1"
exit /b
Other minor changes that were made to the code:
The search and replace can be simplified by using quotes so that special chars don't need to be escaped.
You can replace DEL and REN with a single MOVE.
Redirection is more efficient (faster) if you redirect once using an outer set of parentheses
You may need to search a file name that has spaces and or special characters, in which case you will need to quote the name. But that requires the FOR /F "USEBACKQ" option.
EDIT
Modified code to strip leading spaces after <span></span> has been replaced to eliminate potential of a line containing nothing but spaces and/or tabs.
Also set EOL to space to prevent stripping of lines beginning with ;
i have a file called file.txt which contains:
this is line one ^
this is line two ^
this is the last line
how can i put that into an env var?
i can do this from a batch file:
test.bat
set LF=^
[blank line]
[blank line]
rem two blank lines needed above
set multi=Line 1!LF!Line 2!LF!Last line
echo !multi!
this outputs three lines:
Line 1
Line 2
Last line
so how can i get file.txt into envvar inside a batch file?
As dbenham said, it can be done also with for/f but it's a bit more complicated.
The simple 80% solution is
setlocal EnableDelayedExpansion
set "var="
set LF=^
rem *** Two empty lines are required for the linefeed
FOR /F "delims=" %%a in (myFile.txt) do (
set "var=!var!!LF!%%a"
)
echo !var!
But it fails with:
- If a line is blank it will be skipped
- If a line begins with ; the EOL-character
- If a line contains ! (and carets)
But then you could use a bit more complex solution
#echo off
SETLOCAL DisableDelayedExpansion
set "all="
FOR /F "usebackq delims=" %%a in (`"findstr /n ^^ aux1.txt"`) do (
set "line=%%a"
SETLOCAL EnableDelayedExpansion
set "line=!line:#=#S!"
set "line=!line:*:=!"
for /F "delims=" %%p in ("!all!#L!line!") do (
ENDLOCAL
set "all=%%p"
)
)
SETLOCAL EnableDelayedExpansion
if defined all (
set "all=!all:~2!"
set ^"all=!all:#L=^
!"
set "all=!all:#S=#!"
)
echo !all!
What the code do?
First, the findstr /n ^^ will prepend each line with a line number and a colon, like
1:My first Line
2:; beginning with a semicolon
3:
4:there was an empty line
This solves the problem of empty lines and also the standard EOL-character ; can be ignored.
To get the content of the line, the value is set to a variable while delayed expansion is disabled, this solves the problem with ! and ^ characters.
To remove the line number and the colon, the delayed expansion will be enabled (no, a delim of : can't solve it).
Then all # are replaced with #S, this will be done first, as after the prefix removing the line could be empty and the replacement would fail.
But why I replace it?
That's because I can't insert the linefeeds here, as the following FOR/F would fail with embedded linefeeds,
so I only add linefeed marker (in this case I use #L), but the content of the file could contain also a #L, but by replacing all # with #S all markers are unique.
After the marker, there is the problem to close/disable the delayed expansion with an endlocal, but preserve the content of the modified all and line variable.
This is done with the FOR/F-endlocal trick, as the %%p can transport content behind the endlocal barrier.
Then after reading the complete file, I check if the all is defined, as it would be empty for an empty file.
Then the first linefeed marker #L will be removed, and all other markers are replaced with a real linefeed character.
Then the sharp safer #S will be reverted to #.
That's all, so even this solution is obviously...
You were almost there. You need to read each line of text and then append the line plus a line feed to the variable.
FOR /F could be used, but it doesn't play well with delayed expansion if the content contains ! characters. It is also awkward to preserve blank lines and awkward to disable the EOL option.
A simpler solution is to use SET /P to read the lines. The limitations with this technique are:
1) It trims trailing control characters from each line
2) The file must use Windows standard line terminators of carriage return, line feed. It will not work with Unix style line feed.
3) It is limited to reading 1023 bytes per line (not including the line terminator characters)
Bear in mind that an environment variable can only hold a little less than 8 kbytes of data. So you are limited to only loading a very small file into a variable with this technique.
#echo off
setlocal enableDelayedExpansion
set LF=^
:: Two blank lines above needed for definition of LF - do not remove
set file="test.txt"
set "var="
for /f %%N in ('find /c /v "" ^<%file%') do set lineCnt=%%N
<%file% (
for /l %%N in (1 1 %lineCnt%) do (
set "ln="
set /p "ln="
set "var=!var!!ln!!lf!"
)
)
set var
I have this script, to read xml file. The file contains coordinates and I want to list the coordinates:
#echo off
setlocal EnableDelayedExpansion
FOR %%K IN (*.xml) DO (
SET K=%%K
SET K=!K:~0,-4!
SET "prep=0"
REM READ DATA
FOR /F "tokens=*" %%X IN (!K!.kml) DO (
if !prep! == 1 (
echo %%X
pause
FOR /F %%L IN ("%%X") DO (
SET L=%%L
IF NOT "!L:~0,1!" == "<" (
echo %%L
)
)
SET "prep=0"
)
if "%%X" == "<coordinates>" ( SET "prep=1" )
)
)
I got these result:
14.63778004128814,49.50141683426452,0 14.63696238385996,49.48348965654706,0 14.6
8840586504191,49.47901033971912,0 14.68589371304878,49.49939179836829,0 14.63778
004128814,49.50141683426452,0 </coordinates>
Press and key to continue...
14.63778004128814,49.50141683426452,0
Press and key to continue...
First you see the line with coordinates. Second, in the 3rd loop, there are coordinates printed. But I have only one pair of coordinates printed... If I will press a key again, the batch finishes without printing next columns. Can you help?
Edit
After the answer has been posted, I have question 1) could we use this:
SET LF=^
setlocal EnableDelayedExpansion
... (next code) ...
set "var=!var: =%LF%!"
So when there is no delayed LF variable, we could embed it. Or not?
And 2) why in your code
for %%L in ("!LF!") do set "X=!X: =%%~L!"
Did you use %%~L and not just %%L
Your immediate problem is that FOR /F does not iterate the tokens in a line. It simply parses each token that you ask for. If you don't specify a "tokens" option, then it defaults to "tokens=1" - it only parses the first token in the line.
However, FOR /F will treat a string as multiple lines if the string contains linefeed characters. It will then iterate each line like you want. The trick is to replace your space delimiter with a line feed character. There are multiple methods that can do the job, but I will show what I think is the easiest to work with.
First define a variable containing a single linefeed
set LF=^
::The two blank lines above are critical for the definition of the line feed
The next trick is to replace spaces in your variable with linefeeds. Normally substituion using a variable for the replacement would look something like set "var=!var:search=%replaceVar%!". But that won't work for the LF variable - it is difficult to work with the LF variable using normal expansion. It is much easier to use delayed expansion. We can't embed delayed expansion within delayed expansion, but we can transfer the value of LF to a simple FOR variable and use for %%L in ("!LF!") do set "var=!var: =%%~L!"
One thing about your code I do not understand - your initial FOR loop is iterating accross all the .KML files. You strip off the extension using a substring operation. There is a much easier way to do that without using an environment variable: %%~nK will give the base name of the file without the extension. But why do that at all when you turn around and append the extension again?
I used the %%K value directly - I added the USEBACKQ option and added quotes to allow for spaces in the file name.
Here is code that should do what you are expecting.
#echo off
setlocal EnableDelayedExpansion
::define a variable containing a linefeed character
set LF=^
::Above 2 blank lines are part of the LF definition, do not remove
for %%K in (*.kml) do (
set "prep=0"
for /f "usebackq tokens=*" %%X in ("%%K") do (
if !prep! == 1 (
echo %%X
pause
set "ln=%%X"
for %%L in ("!LF!") do set "ln=!ln: =%%~L!"
for /f %%L in ("!ln!") do (
set L=%%L
if not "!L:~0,1!" == "<" (
echo %%L
)
)
set "prep=0"
)
if "%%X" == "<coordinates>" ( set "prep=1" )
)
)
BUT - I think you have a bigger problem. I am worried that you are setting yourself up for a world of pain by using batch to parse XML. You are assuming the XML will always be layed out the same way. There are countless valid ways of adding or subtracting linefeeds and white space into the XML document that would break your algorithm. Can you be sure all your input files came from the same source and will always be formatted like you expect? I think you really should be using XSLT to parse and transform your XML document into a naked list of coordinates.
Answsers to additional questions
1) set "var=!var: =%LF%!" will not work - Regular expansion of LF requires escape sequences and multiple expansions. This will work: set "var=!var: =^%LF%LF%!"
The escape sequences for %LF% can get very tricky, so I try to avoid them.
2) Regarding for %%L in ("!LF!") do set "X=!X: =%%~L!", note that it is a simple FOR, not FOR /F. The !LF! must be quoted or else FOR will not read it. But the FOR statement preserves the quotes (unlike FOR /F), so I need %%~L to remove the enclosing quotes.
There is a very important distinction between FOR and FOR /F with regard to linefeeds. FOR will preserve quoted linefeeds, whereas FOR /F treats the linefeed as a line delimiter and iterates each line, so the linefeeds are not preserved.
Can someone please explain how this works?
#echo off
REM Creating a Newline variable (the two blank lines are required!)
set NLM=^
set NL=^^^%NLM%%NLM%^%NLM%%NLM%
REM Example Usage:
echo There should be a newline%NL%inserted here.
emits:
There should be a newline
inserted here.
from How can you echo a newline in batch files?
The trick uses the behaviour of the caret.
Also explained at Long commands split over multiple lines in Windows Vista batch (.bat) file
The caret is an escape character for the next character, or at the line end it is used as multiline character, but this is nearly the same.
At the line end it simply escapes the next character, in this case the <Linefeed>, but there is a hidden feature, so if the escaped character is a <LF> it is ignored and the next character is read and escaped, but this charater will be always escaped, even if it is also a <LF>.
Now you can understand
set NLM=^
rem Two empty lines are required here
The NLM-Variable contains exactly one <LF> character.
But if you try to use it with echo Line1%NLM%Line2 it fails, as the parser stops parsing at a single <LF>.
But this works
echo Line1^
Line2
So you need to add an escaped linefeed into the line and that is the NL-Variable.
The NL-Variable consists of only three characters.
NL=^<LF><LF>
And if this is expanded, it creates only one escaped <LF> as the first <LF> after the caret will be ignored.
Btw. In my opinion, it is much easier to use linefeeds with delayed expansion, as there is no need to escape anything.
In this example I use %=EMPTY=% instead of an empty line (for self commenting), but as the variable =EMPTY= can't exists it will be expanded to an empty line.
setlocal EnableDelayedExpansion
(set NLM=^
%=EMPTY=%
)
echo Line1!NLM!Line2
EDIT: Append some hints for useful using the <LF>
1) Use it as newline in an echo
setlocal EnableDelayedExpansion
(set LF=^
%=EMPTY=%
)
echo Line1!LF!Line2
2) Use it to split commands in a parenthesis block
setlocal EnableDelayedExpansion
(set LF=^
%=EMPTY=%
)
(
echo Line1%LF%set x=4%LF%echo !x!%LF%
)
3) Create a (nearly) empty EOL-chararcter in a FOR /F loop,
as <LF> is the line delimiter an EOL of <LF> is the same than an empty one.
FOR /F ^"eol^=^
delims^=^" %%a in (myFile.php) do echo %%a
4) Use LF for splitting text in a FOR /F loop
setlocal EnableDelayedExpansion
(set LF=^
%=EMPTY=%
)
set "var=Content1;Content2"
FOR /F "delims=" %%a in ("%var:;=!LF!%") do (
echo %%a
)
There seems a way that also works with pipe:
(echo 1st line^
&echo 2nd line) | sort