I'm trying to d a simple batch replace of doublequotes to singlequotes.
The teststring must containg special characters, at most: "<LF>"
I cannot replace the double quotes there, as the batch just exists with Syntaxerror. Do you know why, or how to overcome this?
SET TEST="<LF>","<HT>"
SET modified=%TEST:"='% <-- Syntaxerror
ECHO %modified%
Use delayed expansion:
setlocal enabledelayedexpansion
SET TEST="<LF>","<HT>"
SET modified=!TEST:"='! <-- Syntaxerror
ECHO !modified!
As Mr Fuzzy Button notes, the problem is that the shell interprets < and > as redirection. Delayed expansion (using ! instead of %) expands variables after parsing and thus does not affect redirection.
You can solve the SET without delayed expansion by enclosing the argument in quotes:
SET "modified=!TEST:"='!"
But the ECHO would still be problematic, then.
While delayed expansion is a good thing, it's not necessary to answer this question. There are also times when setlocal isn't available. I ran into one.
In the windows shell, string matching for quotes works on outermost match pairs (in most cases, but not all), not innermost.
SET TEST="<LF>","<HT>"
echo %TEST%
SET "modified=%TEST:"='%"
echo.|set /p "___=%modified%"
Workaround: If echo can't print something, set can. Go figure.
This works for me...
SET TEST="LF","HT"
echo %TEST%
SET modified=%TEST:"='%
ECHO %modified%
The problem lies in your <s and >s being interpreted as input/output pipes in the Echo
Related
i have a request for theorical knowledge purpose only in Windows Batch scripting.
Today my question is about preparing the value of a local string loc_str INSIDE A BLOCK OF PARENTHESIS and WITHOUT ENABLING DELAYED EXPANSION, in order to return it to a local scope that enables variable expasion (or eventually to prepare it for an incoming local for loop with in("%loc_str%") if nasty characters in loc_str have been properly escaped before).
We assume that loc_str contains at least one ^ and one ! and do not contain double quotes. Lets consider the following code:
#echo off
setlocal disabledelayedexpansion
set "flag_dde_prev=%flag_dde%" & set "flag_dde=!"
(
set "loc_str=Hello^^^ planet!!!! ^^Earth^"
if not defined flag_dde_prev (
call set "out_str=%%loc_str:^=^^%%"
set out_str
call set "out_str=%%out_str:^=^^%%"
set out_str
)
)
endlocal & set "str=%out_str%"
set str
As you know the local flags flag_{dde, dde_prev} are used to test the type of the calling and current scopes. Typically they're both defined at the beginning of a block setlocal..endlocal. The flag_dde is equal to ! if the current scope disables variable expansion, or is undefined
if the current scope enables it. The local value of flag_dde_prev is the inherited value of flag_dde in the calling scope.
What we must do here is to escape all ^ and ! to prepare the return of out_str with %..% to the calling scope when this latter enable variable expansion (ie when flag_dde_prev is defined). Two substitutions ^=^^ then !=^! with a simple set would be enough, but being inside a block (..) forces us to use the call set statement. Unfortunatly the caret ^ is not replaced in the same way they are in a scope that enables variable expansion.
Precisely, the first substitution ^=^^ doubles even sequences of carets, but does not double odd ones (it doubles them minus one).
Then, the second substitution !=^! replace each ! with ^^!.
To sump up,
loc_str=Hello^^^ planet!!!! ^^Earth^ ;init
out_str=Hello^^^^^ planet!!!! ^^^^Earth^ ;1st substitution ^=^^
out_str=Hello^^^^^ planet^^!^^!^^!^^! ^^^^Earth^ :2nd substitution !=^!
^=^^ seems to be reversible with ^^=^ if done BEFORE !=^! only.
If ^^=^ is done after !=^! then sequences of ^ not preceeding ! are modified only.
If ^^=^ is done after !=^! i didn't find a way to replace the sequences of ^^ before each ".
It behaves like a sequence ^^ before ! is the atomic unreplacable one, longer caret sequences can be replaced but i couldn't obtain ^! whatever i tried.
The same problem is quite easy to solve when variable expansion is enabled, even with nasty strings containing quotes by replacing "" by " at first (jeb already talked about this in another thread). For example if locs_tr doesn't contain quotes, the substitutions would be the following:
set "out_str=!loc_str:^=^^^^!" ;1st substitution, multiplicates the number of ^ by 4 as expected
call set "out_str=%%out_str:^!=^^^!%%" ! ;2nd substitution, replaces each ! by ^^!
set "out_str=!out_str:^^=^!" ;3rd substitution required, replaces each ^^ by ^ to divide by 2 the total number of ^
So my question is simple:
Is there a way using substitution IN THE EXACT SAME CONTEXT (ie no call) to obtain the desired prepared output value for out_str, so it can be returned safely to a scope that enables delayed expansion ?
The prepared output value should double the number of ^ and add ^ before each ! ie:
out_str=Hello^^^^^^ planet^!^!^!^! ^^^^Earth^^
Note that's my question is for theorical knowledge purpose ONLY. Indeed calling a label to do the two substitutions with a simple set is the reasonable way to proceed.
Your assumptions are wrong!
What we must do here is to escape all ^ and ! to prepare the return ...
Two substitutions ^=^^ then !=^! with a simple set would be enough ...
The problem is here the fact that caret folding is different if there is an exclamation mark in the line or not!
setlocal EnableDelayedExpansion
echo "one caret^"
echo "no caret^ but a bang^!"
echo "caret ^^ and a bang^!"
echo one caret ^^
echo no caret^^ but a bang^^!
echo caret ^^^^ and a bang^^!
call and carets and exclamation marks are really funny.
Because call always double all carets before any other rule, but you don't see it with quotes, because in the next parser phase the doubled carets are folded to single carets again.
Parse flow for a single line like
Starting with:
call echo my caret^^^^ "caret^^" and a bang !
1. Caret escaping outside quotes
call echo my caret^^ "caret^^" and a bang !
2. delayed expansion caret escaping (outside and **inside** quotes), the bang itself is lost here
call echo my caret^ "caret^" and a bang
3. The `CALL` caret doubling
call echo my caret^^ "caret^^" and a bang
4. Caret escaping outside quotes
call echo my caret^ "caret^^" and a bang
The problem itself was solved by dbenham and me a few years ago.
See SO: Macro to preserve variables when leaving setlocal scope
or
SO: Make an environment variable survive ENDLOCAL.
In your case it's much simpler, because you don't support quotes nor line feeds in the variable.
And you know, that you leave the current scope and will return into a scope with delayed expansion enabled, this simplifies the problem, too.
#echo off
setlocal EnableDelayedExpansion
setlocal disabledelayedexpansion
(
set "loc_str=Hello^^^ planet!!!! ^^Earth^"
set "loc_str"
setlocal EnableDelayedExpansion
set "out_str=!loc_str:^=^^^^!" # 1st substitution, multiply all carets by four
set "out_str"
call set "out_str=%%out_str:^!=^^^!%%" ! # 2nd substitution, replace all exclamation marks to two carets with exclam
set "out_str"
set "out_str=!out_str:^^=^!" # 3rd substitution reduce all carets by factor of two
set "out_str"
for /F "delims=" %%V in ("!out_str!") do (
endlocal
endlocal
set "str=%%V" ! # The trailing exclamation mark is intentionally, but will not part of the result
)
)
echo str = '!str!'
Ok jeb i just executed your macro. The code below can be copied and run. If i understood properly, the general principle is the following (correct me if i didn't).
#echo off
setlocal enableextensions disabledelayedexpansion
set "_(echo=echo: & echo"
set "_(set=echo: & set"
:: Initialize the DDE flags of the current/previous scopes.
set "flag_dde_prev=" & set "flag_dde=!"
:: Base string ; one of your nasty strings defined in another post ;p
set "str_base=caret ^ bang ! and some (other ) <>&| %% \"
setlocal enabledelayedexpansion
set "flag_dde_prev=!flag_dde!" & set "flag_dde=!"
setlocal enabledelayedexpansion
set "flag_dde_prev=!flag_dde!" & set "flag_dde=!"
:: Nasty local string to be returned ; still one of yours !
set "loc_str_nasty=!str_base! and also quoted stuff "!str_base!""
set loc_str & echo:
if defined flag_dde_prev (
set "out_str=!loc_str_nasty!"
) else (
rem STEP1: Each " is replaced by ""q so everything is consider outside quotes
set "out_str=!loc_str_nasty:"=""q!"
set out_str
rem STEP2: Each ^ is replaced with ^^
set "out_str=!out_str:^=^^!"
set out_str
rem STEP3: Each ! is replaced by ""e!
rem WARN A simple `set` must be avoided because of special characters.
call set "out_str=%%out_str:^!=""e^!%%"
set out_str
rem STEP4/5: Initial " and ^! were marked differently with letters q and e so we can restore them by replacing ""q and ""e! by " and ! respectively
set "out_str=!out_str:""e=^!"
set out_str
set "out_str=!out_str:""q="!"
set out_str
)
rem Return the value of out_str to the calling DDE/EDE scope by using a safe `for`.
for /F "delims=" %%# in ("!out_str!") do endlocal & set "str_pro=%%#"
:: Echo safely the protected string str_pro in the current DDE/EDE scope
%_(set% str_pro
if defined flag_dde (
for /F "tokens=1* delims==" %%v in ('set str_pro') do echo %%str_pro%%=%%w
) else (
%_(echo% ^^^!str_pro^^^!=!str_pro!
)
endlocal
Thanks for your remarks, indeed i talked too fast there is no problem at all with ""q and ""e in the input value of loc_str. You're very right about avoiding special characters with rem, i totally forgot. I did typing mistake also by assigning flag_dde_prev, it must be done with !..! inside an EDE scope of course. I struggle with the code editor here, i'm not used to such stuffs for now (yes i know i sound like a cavern man lol).
I edited my previous post, the protection block (..) is skipped when returning to a DDE scope (i forgot also to assign out_str when flag_dde_prev when defined to !). The returned protected string to DDE contains nasty characters so can't be echoed with %..%, so i used a for to echo it.
Is there a way to deal with " and &|<> to protect a such nasty loc_str to echo it in the calling DDE scope with a brutal echo %..% or call set %..%, or at the contrary is an undirect echoing can't be avoided ?
By undirect i mean without a slow echo only ie with a for like above, or with a very fast set str_pro only by deleting the leading string "str_pro=" by adding a dynamic leading string of backspace in the out_str value.
OUTPUT
Run twice, first when the calling scope is DDE, second when it's EDE (written as EDE in the code above).
Related: Using batch echo with special characters
How do I deal with using escape characters on text that might or mightn't be a special character?
Suppose we have user input:
Set /p var=prompt:
Now, I need to make sure that the text gets interpreted as text even if the user enters something like a special character. But I cannot simply add ^ before the variable...because that'd cancel the variable. The ^^%var% and ^%%var% options don't seem to work either.
How do I go about doing this?
You should realize that the escapes are required in the source code of your program or when you expand a variable via %variable% or in the replaceable parameter of a for command. This is not required if you expand a variable via !delayed! expansion. So, your problem may be easily solved this way:
setlocal EnableDelayedExpansion
Set /p var=prompt:
echo !var!
The standard method to avoid the problem when you read a file that may have special characters is enclosing the value of the replaceable parameter in quotes when the value is asigned with Delayed Expansion disabled, and then Enable Delayed Expansion in order to manage the value. Of course, this method forces to insert an endlocal command inside the loop:
for /F "delims=" %%a in (anyFile.txt) do (
set "line=%%a"
setlocal EnableDelayedExpansion
echo Line with special characters: !line!
endlocal
)
Not possible. The shell processes the user input before your script does. Your script won't even know the user typed an escape character.
Why does string-manipulation work inline in an if statement without using delayed expansion variables - but fails otherwise. For example:
set test=testString
if %test:~0,4%==test echo Success
This works correctly; returning Success. However if I do the following:
setLocal enableDelayedExpansion
set test=testString
if !test:~0,4!==test echo Success
I receive the error - 4!==test was unexpected at this time.
Obviously you can get around this by doing something like set comp=!test:~0,4! then just using the !comp! variable in the if statement.
npocmaka correctly diagnosed the problem that the IF command gets its own special parsing phase, and token delimiters within the variable expansion are causing problems.
Substitution also causes a problem:
:: This fails
if !test:String=!==test echo Success
The reason why normal expansion works is because that occurs before the IF parsing, but delayed expansion occurs after IF parsing.
An alternative to using quotes is to escape the problem characters.
:: Both of these work
if !test:~0^,4!==test echo Success
if !test:String^=!==test echo Success
,;=<tab> and <space> are delimiters for cmd.exe and in many cases are ignored if they are not in quotes and act like empty space.Probably in this case , is taken as the end of the first operand and IF is surprised that there is not valid comparison operator e.g. this will print yep :
if a ;==;,,=a echo yep
(but will not work if there's a equal sign in the first part of the operands)
But this will not:
if "a ;" == ";,,=a" echo yep
so to make a valid IF expression when you use comma you need quotes.And this will work
setLocal enableDelayedExpansion
set test=testString
if "!test:~0,4!" == "test" echo Success
Without delayed expansion substitution is made immediately and this will work without quotes:
set test=testString
setlocal disableDelayedExpansion
if %test:~0,4% == test echo Succes
endlocal
For the same reason this will be taken as a wrong syntax expression (see jeb's comment):
set "test="
setlocal disableDelayedExpansion
if %test% == test echo Succes
endlocal
May be is not the complete answer , but should be close.As both echo !test:~0,4! and echo %test:~0,4% will work without quotes stays the question why exactly IF fails - may because IF command uses its own parser
As a conclusion - it's always good to use quotes when you compare strings with IF :
with delayed expansion commas and semicolons will cause troubles.
without delayed expansion undefined variables will cause troubles.
I have three variables: string, search, replace. I wish to substitute the %search% in %string% with %replace%.
This works but needs hard characters.
SET modified=%string:morning=evening%
This seems to be answer in the forums but does not work. It simply stores the entire line at %modified%
SET modified=!string:%search%=%replace%!
The ! format is doing delayed expansion--the % variables get expanded immediately, but the ! variable gets expanded only when it's needed. I believe that only works in a batch file, so if you're experimenting directly at the command line you won't get the same behaviour as if you are running a batch file.
Make sure to enable delayed expansion in your batch file before using the ! notation, like this:
SETLOCAL ENABLEDELAYEDEXPANSION
SET string=This morning
SET search=morning
SET replace=evening
SET modified=!string:%search%=%replace%!
ECHO %modified%
ENDLOCAL
This will echo This evening.
I want to split a string in two parts, without using any for loop.
For example, I have the string in a variable:
str=45:abc
I want to get 45 in a variable and abc in another variable. Is it possible in batch file?
pattern is like somenumber:somestring
You could split the str with different ways.
The for loop, you don't want use it.
The trailing part is easy with the * (match anything until ...)
set "var2=%str:*:=%"
The leading part can be done with a nasty trick
set "var1=%str::="^&REM #%
The caret is needed to escape the ampersand,
so effectivly the colon will be replaced by "&REM #
So in your case you got the line after replacing
set "var1=4567"&REM #abcde
And this is splitted into two commands
set "var1=4567"
REM #abcde`
And the complete code is here:
set "str=4567:abcde"
echo %str%
set "var1=%str::="^&REM #%
set "var2=%str:*:=%"
echo var1=%var1% var2=%var2%
Edit 2: More stable leading part
Thanks Dave for the idea to use a linefeed.
The REM technic isn't very stable against content with quotes and special characters.
But with a linefeed trick there exists a more stable version which also works when the split argument is longer than a single character.
#echo off
setlocal enableDelayedExpansion
set ^"str=456789#$#abc"
for /F "delims=" %%a in (^"!str:#$#^=^
!^") do (
set "lead=%%a"
goto :break
)
:break
echo !lead!
Solution 3: Adpated dbenhams answer
Dbenham uses in his solution a linefeed with a pipe.
This seems a bit over complicated.
As the solution uses the fact, that the parser removes the rest of the line after an unescaped linefeed (when this is found before or in the special character phase).
At first the colon character is replaced to a linefeed with delayed expansion replacement.
That is allowed and the linefeed is now part of the variable.
Then the line set lead=%lead% strips the trailing part.
It's better not to use the extended syntax here, as set "lead=%lead%" would break if a quote is part of the string.
setlocal enableDelayedExpansion
set "str=45:abc"
set ^"lead=!str::=^
!"
set lead=%lead%
echo "!lead!"
You can try this . If its fixed that numbers to left of the colon will be always 2 & to the right will be 3. Then following code should work assuming your str has the value.
set "str=45:abc"
echo %str%
set var1=%str:~0,2%
set var2=%str:~3,3%
echo %var1% %var2%
Keep me posted. :)
It seems pointless to avoid using a FOR loop, but it does make the problem interesting.
As jeb has pointed out, getting the trailing part is easy using !str:*:=!.
The tricky bit is the leading part. Here is an alternative to jeb's solution.
You can insert a linefeed into a variable in place of the : using the following syntax
setlocal enableDelayedExpansion
set "str=45:abc"
echo !str::=^
!
--OUTPUT--
45
abc
The empty line above the last ! is critical.
I'm not sure why, but when the output of the above is piped to a command, only the first line is preserved. So the output can be piped to a FINDSTR that matches any line, and that result directed to a file that can then be read into a variable using SET /P.
The 2nd line must be eliminated prior to using SET /P because SET /P does not recognize <LF> as a line terminator - it only recognizes <CR><LF>.
Here is a complete solution:
#echo off
setlocal enableDelayedExpansion
set "str=45:abc"
echo(!str::=^
!|findstr "^" >test.tmp
<test.tmp set /p "var1="
del test.tmp
set "var2=!str:*:=!"
echo var1=!var1! var2=!var2!
Update
I believe I've mostly figured out why the 2nd line is stripped from the output :)
It has to do with how pipes are handled by Windows cmd.exe with each side being processed by a new CMD.EXE thread. See Why does delayed expansion fail when inside a piped block of code? for a related question with a great answer from jeb.
Just looking at the left side of the piped command, I believe it is parsed (in memory) into a statement that looks like
C:\Windows\system32\cmd.exe /S /D /c" echo {delayedExpansionExpression}"
I use {delayedExpansionExpression} to represent the multi-line search and replace expansion that has not yet occurred.
Next, I think the variable expression is actually expanded and the line is broken in two by the search and replace:
C:\Windows\system32\cmd.exe /S /D /c" echo 43
abc"
Only then is the command executed, and by normal cmd.exe rules, the command ends at the linefeed. The quoted command string is missing the end quote, but the parser doesn't care about that.
The part I am still puzzled by is what happens to the abc"? I would have thought that an attempt would be made to execute it, resulting in an error message like 'abc"' is not recognized as an internal or external command, operable program or batch file. But instead it appears to simply get lost in the ether.
note - jeb's 3rd comment explains why :)
Safe version without FOR
My original solution will not work with a string like this & that:cats & dogs. Here is a variation without FOR that should work with nearly any string, except for string length limits and trailing control chars will be stripped from leading part.
#echo off
setlocal enableDelayedExpansion
set "str=this & that:cats & dogs"
set ^"str2=!str::=^
!^"
cmd /v:on /c echo ^^!str2^^!|findstr /v "$" >test.tmp
<test.tmp set /p "var1="
del test.tmp
set "var2=!str:*:=!"
echo var1=!var1! var2=!var2!
I delay the expansion until the new CMD thread, and I use a quirk of FINDSTR regex that $ only matches lines that end with <cr>. The first line doesn't have it and the second does. The /v option inverts the result.
Yes, I know this is a very old topic, but I just discovered it and I can't resist the temptation of post my solution:
#echo off
setlocal
set "str=45:abc"
set "var1=%str::=" & set "var2=%"
echo var1="%var1%" var2="%var2%"
You may read full details of this method here.
In the Light of people posting all sorts of methots for splitting variables here i might as well post my own method, allowing for not only one but several splits out of a variable, indicated by the same symbol, which is not possible with the REM-Method (which i used for some time, thanks #jeb).
With the method below, the string defined in the second line is split into three parts:
setlocal EnableDelayedExpansion
set fulline=one/two/three or/more
set fulline=%fulline%//
REM above line prevents unexpected results when input string has less than two /
set line2=%fulline:*/=%
set line3=%line2:*/=%
set line1=!fulline:/%line2%=!
set line2=!line2:/%line3%=!
setlocal DisableDelayedExpansion
echo."%line1%"
echo."%line2%"
echo."%line3%"
OUTPUT:
"one"
"two"
"three or/more//"
i recommend using the last so-created partition of the string as a "bin" for the remaining "safety" split-characters.
Here's a solution without nasty tricks for leading piece
REM accepts userID#host
setlocal enableDelayedExpansion
set "str=%1"
set "host=%str:*#=%"
for /F "tokens=1 delims=#" %%F IN ("%str%") do set "user=%%F"
echo user#host = %user%#%host%
endlocal