CMD substitute phrase containing special characters - batch-file

I need to do a very simple thing: substitute one frase with another. This must be done by CMD batch file (for Windows 7). The frase to be subsitituted can be in any position in txt file line (and in many lines).
The problem is the frase to be substituted contains ":" and "!" characters. I am not very skilled in batch files (to put it mildly), though I spend some hours especially to learn about this specific problem. It looks very complicated for me. At last, by chance, I overided the issue, but... I feel it is a barbarity how I did it.
The real line with the frase which should be substituted is e.g.:
"21:12:45 WARNING: No video signal present!"
The frase which should be substituted is:
"WARNING: No video signal present!"
The frase, which it should be substituted with is:
"Recognition suspended"
I have found this code: https://www.computerhope.com/forum/index.php?topic=41188.0
It works fine, except cannot work with "!" as I see, and escape char "^" never works. But I noticed that although it doesn't work properly - it trims the exclamation mark. Here are real strings before (b) and after (a):
(b)20:42:18 WARNING: No video signal present!
(a)20:42:18 WARNING: No video signal present
So I add 2 other lines to the code and this does the thing. The whole code is now:
#echo off
setlocal enabledelayedexpansion
set txtfile=D:\wfc\testlib\test.txt
set newfile=D:\wfc\testlib\new_test.txt
if exist "%newfile%" del /f /q "%newfile%"
for /f "tokens=*" %%a in (%txtfile%) do (
set newline=%%a
set newline=!newline:No video signal present!=!
set newline=!newline:No video signal present=!
set newline=!newline:WARNING:=Suspend recognition!
echo !newline! >> %newfile%
)
First crucial line cuts "!",
second line substitutes "No video signal present" with nothing (trims it),
third line substitutes the rest "Warning:" with desirable "Suspend recognition".
And at the end I have:
(b)20:42:18 WARNING: No video signal present!
(a)20:42:18 Suspend recognition
I feel this could be done elegantly. Besides I am not sure, if my way is not dangerous for some reason (data damage etc.). Please help.

Yes, this can be done easier.
First of all, it is not the line set newline=!newline:No video signal present!=! that removes the ! character, but it is the line set newline=%%a, because there is delayed variable expansion enabled when the for variable reference %%a is expanded, which happens before delayed expansion, hence an orphaned exclamation mark appears that is simply removed.
The key to success is to disable delayed expansion during expansion of %%a and to enable it afterwards. In the sub-string replacement syntax the : does not cause any problems; neither does the ! in the search string, given it is not the first character. So the following code should work:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set "txtfile=D:\wfc\testlib\test.txt"
set "newfile=D:\wfc\testlib\new_test.txt"
> "%newfile%" (
for /F usebackq^ delims^=^ eol^= %%a in ("%txtfile%") do (
set "newline=%%a"
setlocal EnableDelayedExpansion
echo(!newline:WARNING: No video signal present!=Recognition suspended!
endlocal
)
)
endlocal
exit /B
I also changed the following items:
improved the syntax of set by using quotes to avoid trouble with some special characters;
avoided interim variable newline, so did the string manipulation directly in the echo line;
quoted file path/name to avoid problems with white-spaces, hence using usebackq option;
replaced the tokens=* by the delims= option in order to keep leading white-spaces; to avoid loss of lines that begin with ; due to the default eol option, I used the odd-looking unquoted option string syntax as this is the only way (!) to have no delims and eol defined at the same time (you cannot write "delims= eol=" as this would define " as the eol character; "eol= delims=" would define the SPACE, and "eol=delims=" the d);
redirected whole for loop, so no initial file deletion necessary, and the performance is a lot better, because the output file has to be opened and closed once only, not in every iteration;
echo Text > "file.ext" returns a trailing SPACE (actually the one in front of >); this is avoided by the above item; in general, this can be avoided when using the reversed redirection syntax > "file.ext" echo Text;
echo !VAR! returns ECHO is on|off. in case variable VAR is empty; to avoid this or other unexpected output, the odd-looking but safe syntax echo(!VAR! is used;
avoided immediate (%) expansion when delayed expansion is enabled to avoid loss of !;
To keep empty lines that appear in the original file, change the code to this:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set "txtfile=D:\wfc\testlib\test.txt"
set "newfile=D:\wfc\testlib\new_test.txt"
> "%newfile%" (
for /F "delims=" %%a in ('findstr /N /R "^" "%txtfile%"') do (
set "newline=%%a"
setlocal EnableDelayedExpansion
set "newline=!newline:WARNING: No video signal present!=Recognition suspended!"
echo(!newline:*:=!
endlocal
)
)
endlocal
exit /B
The findstr command is used to prefix every line of the input file by a line number plus a colon (/N option). Then the sub-string replacement is done. Finally, everything up to and including the first :, hence the line number prefix becomes removed.

Substituting a phrase is pretty easy to do in a .bat file script.
powershell -NoLogo -NoProfile -Command ^
"Get-Content -Path '.\subfrase-in.txt' |" ^
"ForEach-Object { $_ -replace 'WARNING: No video signal present!', 'Recognition suspended' }"

Related

How remove the | character from variable

I've a code like below:
FOR /F "tokens=*" %%B IN ('%*') DO (
ECHO %%B
SET data=%%B
ECHO %data%
)
The %%B contains one or more than one | character, for example the value of %%B is First | Second | Third. The SET data=%%B or SET data="%%B" function doesn't work. When %data% is called back, it shows Echo is off.
I need to remove the | character from %%B and save it to a %data% variable. But I don't know how?
I will be grateful if any solution has been provided from anyone....
The first solution is using:
#echo off
setlocal EnableExtensions EnableDelayedExpansion
for /F delims^=^ eol^= %%B in ('%*') do (
set "data=%%B"
set "data=!data:|=!"
echo(!data!
)
endlocal
tokens=* results in getting assigned to loop variable B the entire line with leading spaces/tabs removed while delims= results in getting assigned to the loop variable B the entire line including leading spaces/tabs. So the definition of an empty list of delimiters is better in general.
FOR ignores by default empty lines and lines starting with a semicolon because of ; is interpreted by default as end of line character. For that reason the uncommon, not double quoted option string delims^=^ eol^= is used here to define an empty list of delimiters and no end of line character. The caret character ^ is used to escape the next character to get it interpreted as literal character and not as argument separator although not enclosed in a double quoted argument string.
But there is one problem with this solution: A line containing one or more exclamation marks ! is not processed correct because Windows command processor interprets on command line set "data=%%B" each ! as begin/end of a delayed environment variable reference and replaces the string between two exclamation marks by the current value of the environment variable with that name or nothing on no variable existing with such a name and removes everything after ! if there is no more !.
There are at least three solutions.
The first one is enabling and disabling delayed expansion within the loop.
#echo off
setlocal EnableExtensions DisableDelayedExpansion
for /F delims^=^ eol^= %%B in ('%*') do (
set "data=%%B"
setlocal EnableDelayedExpansion
set "data=!data:|=!"
echo(!data!
endlocal
)
endlocal
Please read this answer for details about the commands SETLOCAL and ENDLOCAL as they do much more than just toggling delayed expansion on/off in the loop.
( between echo and value of environment variable data is used in case of a line contains only | without or with additionally just spaces/tabs resulting in data becoming either undefined or a string consisting only of spaces/tabs. A space between echo and !data! would be on execution just the command echo  with an ignored space and 0 or more spaces/tabs resulting in getting output ECHO is off. instead of an empty line or a line with just spaces/tabs. The opening round bracket prevents that and is interpreted by cmd.exe as separator between command echo and its argument string which begins in this case always with (. ECHO ignores the first character of the argument string on output as long as the argument string is not /? and so ( is never output.
The second solution is using a subroutine:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
for /F delims^=^ eol^= %%B in ('%*') do (
set "data=%%B"
call :ProcessLine
)
endlocal
goto :EOF
:ProcessLine
set "data=%data:|=%"
echo(%data%
goto :EOF
Note: echo %data% can result in unexpected behavior if the line contains characters like the redirection operators < and > or the operator & as Windows command processor executes echo(%data% after substituting %data% with current string of environment variable data. Even set "data=%data:|=%" can be problematic depending on line containing " and <>&. So this solution is really not safe.
See also Where does GOTO :EOF return to?
The third solution is using a "double percent" environment variable reference and using command CALL to force a double parsing of the command line containing %%variable%% instead of just %variable% as usual.
#echo off
setlocal EnableExtensions DisableDelayedExpansion
for /F delims^=^ eol^= %%B in ('%*') do (
set "data=%%B"
call set "data=%%data:|=%%"
call echo(%%data%%
)
endlocal
Note Also this solution is not really safe depending on the data as the second solution.
See also How does the Windows Command Interpreter (CMD.EXE) parse scripts?
Windows command processor is designed for executing commands and applications and not for reformatting or processing CSV files using vertical bar as delimiter/separator.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
call /?
echo /?
endlocal /?
for /?
goto /?
set /?
setlocal /?
And read also answer on Why is no string output with 'echo %var%' after using 'set var = text' on command line? for the reason using the syntax set "variable=value" instead of set variable=value or set variable="value".

CMD file - delayed variable expansion

I am writing a CMD/batch file (running under Win-7 cmd.exe) and I seem to be getting hung up on delayed variable expansion.
I am using text file input that is in the form:
9 .CN=ISRJX.OU=Linc.OU=thisco.O=UCOM.T=UCOM. 8-20-13
10 .CN=FXXLISHER.OU=Linc.OU=thisco.O=UCOM.T=UCOM. 10-13-13
11 .CN=QXX004F.OU=Linc.OU=thisco.O=UCOM.T=UCOM. 10-14-13
12 *.CN=QXX1001OB.OU=Linc.OU=thisco.O=UCOM.T=UCOM. 10-15-13
as contents in "inputfile.txt". Purpose is to extract the first word after ".CN=", at this point in the process.
Note that I can't strip based on number of chars before "CN=" because the number of chars will vary.
My script is:
setlocal enableextensions enableDelayedExpansion
REM #echo off
for /f "tokens=3 delims==." %%a in (inputfile.txt) do (
set "acct =%%a"
echo. %%a,%acct%
)
endlocal
I've tried every set of combination of quote, !, % etc. and both enabled & disabled delayed expansion, and still can't get this to work. For the most part, when I use ! I end up echoing the actual !. i.e. "echo !acct!" will echo the actual text "!acct!".
I have read many examples and answers, here and elsewhere, about delayed variable expansion. I just can't figure out how to work around it in this example, where I want "acct" to expand within the loop.
Suspect this is a simple punctuation problem, but I'm out of ideas. When I run this, I see acct set to the value for %%a, but when it echoes, clearly it doesn't expand to the new value -- instead it will echo whatever it was set to previously, or blank.
I have also tried disabledelayedexpansion, which makes no difference in my results (including when I use !acct! instead of %acct%.)
Remove the space after the acct variable name and use the ! marks.
#echo off
setlocal EnableExtensions EnableDelayedExpansion
for /f "tokens=3 delims==." %%A in (inputfile.txt) do (
set "acct=%%A"
echo. %%A,!acct!
)
endlocal

How to split string without for loop in batch file

I want to split a string in two parts, without using any for loop.
For example, I have the string in a variable:
str=45:abc
I want to get 45 in a variable and abc in another variable. Is it possible in batch file?
pattern is like somenumber:somestring
You could split the str with different ways.
The for loop, you don't want use it.
The trailing part is easy with the * (match anything until ...)
set "var2=%str:*:=%"
The leading part can be done with a nasty trick
set "var1=%str::="^&REM #%
The caret is needed to escape the ampersand,
so effectivly the colon will be replaced by "&REM #
So in your case you got the line after replacing
set "var1=4567"&REM #abcde
And this is splitted into two commands
set "var1=4567"
REM #abcde`
And the complete code is here:
set "str=4567:abcde"
echo %str%
set "var1=%str::="^&REM #%
set "var2=%str:*:=%"
echo var1=%var1% var2=%var2%
Edit 2: More stable leading part
Thanks Dave for the idea to use a linefeed.
The REM technic isn't very stable against content with quotes and special characters.
But with a linefeed trick there exists a more stable version which also works when the split argument is longer than a single character.
#echo off
setlocal enableDelayedExpansion
set ^"str=456789#$#abc"
for /F "delims=" %%a in (^"!str:#$#^=^
!^") do (
set "lead=%%a"
goto :break
)
:break
echo !lead!
Solution 3: Adpated dbenhams answer
Dbenham uses in his solution a linefeed with a pipe.
This seems a bit over complicated.
As the solution uses the fact, that the parser removes the rest of the line after an unescaped linefeed (when this is found before or in the special character phase).
At first the colon character is replaced to a linefeed with delayed expansion replacement.
That is allowed and the linefeed is now part of the variable.
Then the line set lead=%lead% strips the trailing part.
It's better not to use the extended syntax here, as set "lead=%lead%" would break if a quote is part of the string.
setlocal enableDelayedExpansion
set "str=45:abc"
set ^"lead=!str::=^
!"
set lead=%lead%
echo "!lead!"
You can try this . If its fixed that numbers to left of the colon will be always 2 & to the right will be 3. Then following code should work assuming your str has the value.
set "str=45:abc"
echo %str%
set var1=%str:~0,2%
set var2=%str:~3,3%
echo %var1% %var2%
Keep me posted. :)
It seems pointless to avoid using a FOR loop, but it does make the problem interesting.
As jeb has pointed out, getting the trailing part is easy using !str:*:=!.
The tricky bit is the leading part. Here is an alternative to jeb's solution.
You can insert a linefeed into a variable in place of the : using the following syntax
setlocal enableDelayedExpansion
set "str=45:abc"
echo !str::=^
!
--OUTPUT--
45
abc
The empty line above the last ! is critical.
I'm not sure why, but when the output of the above is piped to a command, only the first line is preserved. So the output can be piped to a FINDSTR that matches any line, and that result directed to a file that can then be read into a variable using SET /P.
The 2nd line must be eliminated prior to using SET /P because SET /P does not recognize <LF> as a line terminator - it only recognizes <CR><LF>.
Here is a complete solution:
#echo off
setlocal enableDelayedExpansion
set "str=45:abc"
echo(!str::=^
!|findstr "^" >test.tmp
<test.tmp set /p "var1="
del test.tmp
set "var2=!str:*:=!"
echo var1=!var1! var2=!var2!
Update
I believe I've mostly figured out why the 2nd line is stripped from the output :)
It has to do with how pipes are handled by Windows cmd.exe with each side being processed by a new CMD.EXE thread. See Why does delayed expansion fail when inside a piped block of code? for a related question with a great answer from jeb.
Just looking at the left side of the piped command, I believe it is parsed (in memory) into a statement that looks like
C:\Windows\system32\cmd.exe /S /D /c" echo {delayedExpansionExpression}"
I use {delayedExpansionExpression} to represent the multi-line search and replace expansion that has not yet occurred.
Next, I think the variable expression is actually expanded and the line is broken in two by the search and replace:
C:\Windows\system32\cmd.exe /S /D /c" echo 43
abc"
Only then is the command executed, and by normal cmd.exe rules, the command ends at the linefeed. The quoted command string is missing the end quote, but the parser doesn't care about that.
The part I am still puzzled by is what happens to the abc"? I would have thought that an attempt would be made to execute it, resulting in an error message like 'abc"' is not recognized as an internal or external command, operable program or batch file. But instead it appears to simply get lost in the ether.
note - jeb's 3rd comment explains why :)
Safe version without FOR
My original solution will not work with a string like this & that:cats & dogs. Here is a variation without FOR that should work with nearly any string, except for string length limits and trailing control chars will be stripped from leading part.
#echo off
setlocal enableDelayedExpansion
set "str=this & that:cats & dogs"
set ^"str2=!str::=^
!^"
cmd /v:on /c echo ^^!str2^^!|findstr /v "$" >test.tmp
<test.tmp set /p "var1="
del test.tmp
set "var2=!str:*:=!"
echo var1=!var1! var2=!var2!
I delay the expansion until the new CMD thread, and I use a quirk of FINDSTR regex that $ only matches lines that end with <cr>. The first line doesn't have it and the second does. The /v option inverts the result.
Yes, I know this is a very old topic, but I just discovered it and I can't resist the temptation of post my solution:
#echo off
setlocal
set "str=45:abc"
set "var1=%str::=" & set "var2=%"
echo var1="%var1%" var2="%var2%"
You may read full details of this method here.
In the Light of people posting all sorts of methots for splitting variables here i might as well post my own method, allowing for not only one but several splits out of a variable, indicated by the same symbol, which is not possible with the REM-Method (which i used for some time, thanks #jeb).
With the method below, the string defined in the second line is split into three parts:
setlocal EnableDelayedExpansion
set fulline=one/two/three or/more
set fulline=%fulline%//
REM above line prevents unexpected results when input string has less than two /
set line2=%fulline:*/=%
set line3=%line2:*/=%
set line1=!fulline:/%line2%=!
set line2=!line2:/%line3%=!
setlocal DisableDelayedExpansion
echo."%line1%"
echo."%line2%"
echo."%line3%"
OUTPUT:
"one"
"two"
"three or/more//"
i recommend using the last so-created partition of the string as a "bin" for the remaining "safety" split-characters.
Here's a solution without nasty tricks for leading piece
REM accepts userID#host
setlocal enableDelayedExpansion
set "str=%1"
set "host=%str:*#=%"
for /F "tokens=1 delims=#" %%F IN ("%str%") do set "user=%%F"
echo user#host = %user%#%host%
endlocal

Edit html file in batch file

I have a script in which I read html files which I want to edit. Here I paste the code which calls :remove_redundant_columns subroutine.
It should remove the spaces/white spaces from begin of each line and remove from html file. Only problem is that it adds extra text like = to lines which are almost empty, just have few tabs.
The html file which I downloaded is from hidemyass.com/proxy-list/1
call parse_proxy.bat remove_redundant_columns !FILENAME!
exit /b
:remove_redundant_columns
REM Remove whitespaces from begin of lines and <span></span>
FOR /f "tokens=*" %%t in (%1) do (
SET S=%%t
SET S=!S:^<span^>^</span^>=!
if NOT "!S!"=="" >>$tmp$ echo !S!
)
del %1
REN $tmp$ %1
exit /b
If you believe, that's your only problem... You need to check, if your variable S contains content.
That's required, as substitution on an undefined variable will not produce an undefined/empty variable, the new content will be the substitution text.
:remove_redundant_columns
REM Remove whitespaces from begin of lines and <span></span>
FOR /f "tokens=*" %%t in (%1) do (
SET S=%%t
if defined S (
SET S=!S:^<span^>^</span^>=!
>>$tmp$ echo !S!
)
)
As dbenham stated, you got many other problems,
and one additional problem is the echo !S! command itself.
ECHO has some nasty side effects on different content.
If the content is empty (or only spaces) then it will print it's currently state
ECHO IS OFF
If the content is OFF or ON it will NOT be echoed, it will only change the state.
And if the content is /? it will echo the help instead of /?.
To solve this you could simply change ECHO !S! to ECHO(!S! and all problems are gone.
jeb already solved your = problem (once the extra IF DEFINED check is added to his answer). But you may have at least one other problem.
I agree with Joey that you should not be using batch to manipulate HTML like this. But, if you really want to...
Your potential problem is that HTML usually has ! characters sprinkled within. Your code uses delayed expansion, but that causes corruption of FOR variable expansion when it contains ! character(s). The solution is to toggle delayed expansion on and off within your loop.
:remove_redundant_columns
setlocal disableDelayedExpansion
REM Remove whitespaces from begin of lines and <span></span>
(
FOR /f "usebackq eol= tokens=*" %%t in ("%~1") do (
SET S=%%t
setlocal enableDelayedExpansion
if defined S SET "S=!S:<span></span>=!"
for /f "eol= tokens=*" %%S in ("!S!") do if "%%S" neq "" echo %%S
endlocal
)
) >>$tmp$
move /y $tmp$ "%~1"
exit /b
Other minor changes that were made to the code:
The search and replace can be simplified by using quotes so that special chars don't need to be escaped.
You can replace DEL and REN with a single MOVE.
Redirection is more efficient (faster) if you redirect once using an outer set of parentheses
You may need to search a file name that has spaces and or special characters, in which case you will need to quote the name. But that requires the FOR /F "USEBACKQ" option.
EDIT
Modified code to strip leading spaces after <span></span> has been replaced to eliminate potential of a line containing nothing but spaces and/or tabs.
Also set EOL to space to prevent stripping of lines beginning with ;

How to extract all instances of a specific XML tag attribute using Windows batch

I have an XML file and I need to extract
testname
from all the instances of
<con:testSuite name="testname"
within the XML file.
I am not quite sure how to approach this, or whether this is possible in batch.
Here is what I have thought so far:
1) Use FINDSTR and store every line that has
<con:testSuite name=
in a variable or a temporary file, like this:
FINDSTR /C:"<con:testSuite name=" file.xml > tests.txt
2) Somehow use that file or variable to extract the strings
Note that there might be more than one instance of the matching string in the same line.
I am a novice at batch and any help is appreciated.
Parsing XML is very painful with batch. Batch is not a good text processor to begin with. However, with some amount of effort you can usually extract the data you want from a given XML file. But the input file could easily be rearranged into an equivalent valid XML form that will break your parser.
With that disclaimer out of the way...
Here is a native batch solution
#echo off
setlocal disableDelayedExpansion
set input="test.xml"
set output="names.txt"
if exist %output% del %output%
for /f "delims=" %%A in ('findstr /n /c:"<con:testSuite name=" %input%') do (
set "ln=%%A"
setlocal enableDelayedExpansion
call :parseLine
endlocal
)
type %output%
exit /b
:parseLine
set "ln2=!ln:*<con:testSuite name=!"
if "!ln2!"=="!ln!" exit /b
for /f tokens^=2^ delims^=^" %%B in ("!ln2!") do (
setlocal disableDelayedExpansion
>>%output% echo(%%B
endlocal
)
set "ln=!ln2!"
goto :parseLine
The FINDSTR /N option is only there to guarantee that no line begins with a ; so that we don't have to worry about the pesky default FOR "EOL" option.
The toggling of delayed expansion on and off is to protect any ! characters that may be in the input file. If you know that ! never appears in the input, then you can simply setlocal enableDelayedExpansion at the top and remove all other setlocal and endlocal commands.
The last FOR /F uses special escape sequences to enable the specification of a double quote as a DELIM character.
Answer to additional question in comment
You cannot simply put the additional constraint in the existing FINDSTR command because it will return the entire line that has a match. Remember you said yourself, "there might be more than one instance of the matching string in the same line". The first name might start with the correct prefix, and the 2nd name on the same line might not. You only want to keep the one that starts appropriately.
One solution is to simply change the echo(%%B >>%output% line as follows:
echo(%%B|findstr "^lp_" >>%output%
The FINDSTR is using a regular expression meta-character ^ to specify that the string must start with lp_. The quotes have already been removed at this point, so we don't have to worry about them.
However, you may run into a situation in the future where you must include " in your search string. Plus it might be marginally faster to include the lp_ screen in the initial FINDSTR so that :parseLine is not called unnecessarily.
FINDSTR requires that search string double quotes are escaped with a back slash. But the Windows CMD processor also has its own rules for escaping. Special characters like > need to be either quoted or escaped. The original code used quotes, but you want to include a quote in the string, and that creates unbalanced quotes in your command. Windows batch generally likes quotes in pairs. At least one of the quotes must be escaped for CMD as ^". If the quote needs to be escaped for both CMD and FINDSTR, then it looks like \^".
But any special characters within the string that are no longer functionally quoted from a CMD perspective must be escaped using ^ as well.
Here is one solution that escapes all special characters. It looks awful and is very confusing.
for /f "delims=" %%A in ('findstr /n /c:^"^<con:testSuite^ name^=\^"lp_^" %input%') do (
Here is another solution that looks much better, but it is still confusing to keep track of what is escaped for CMD and what is escaped for FINDSTR.
for /f "delims=" %%A in ('findstr /n /c:"<con:testSuite name=\"lp_^" %input%') do (
One way to keep things a bit simpler is to convert the search into a regular expression. A single double quote can be searched using [\"\"]. It is a character class expression that matches either a quote or a quote - silly I know. But it keeps quotes paired so that CMD is happy. Now you don't have to worry about escaping any characters for CMD, and you can concentrate on the regex search string.
for /f "delims=" %%A in ('findstr /nr /c:"<con:testSuite name=[\"\"]lp_" %input%') do (

Resources