Why are spaces being inserted into this batch file at runtime? - batch-file

I've got this simple test script:
echo start
falsey ^
&& echo nope
echo end
falsey is a tiny C program I made to help test, it simply returns 1:
int main(int argc, const char** argv) {
return 1;
}
The problem is, when I open up cmd and run this batch script, it inserts a couple spaces between the two &s, which naturally causes it to incorrectly echo nope:
C:\jenkins-foo-test>foo.bat
C:\jenkins-foo-test>echo start
start
C:\jenkins-foo-test>falsey & & echo nope
nope
C:\jenkins-foo-test>echo end
end
I have figured out a few workarounds for this, such as putting quotes around the &&:
falsey ^
"&&" echo nope
Quotes don't actually work in this case, see #dbenham's answer for why.
dropping the ^ and merging the two lines:
falsey && echo nope
or adding a space in front of the &&:
falsey ^
&& echo nope
Each of those three options correctly do not print nope. But they do all still have extra spaces when cmd.exe prints the commands it's running.
So why are these extra spaces being inserted and how can I stop cmd.exe from breaking things without having to contort my code into mildly unnatural shapes all the time?

The line continuation ^ escapes the first character on the next line, so the first & is treated as a literal and is passed as an argument to falsey. The second & is treated as simple command concatenation.
One simple solution to get && to be treated as conditional command concatenation is to put the && on the line before:
falsey &&^
echo nope
or you could put a space before the &&
falsey ^
&& echo nope
The other option is to use the redirection trick that jeb points out in his linked answer, but I never use that trick.
Regarding your "solution" with quoted "&&" - you are fooling yourself. It appears to work when falsey fails. But if your command succeeds, then you have trouble:
echo ONE ^
"&&" echo TWO
-- OUTPUT --
C:\test>echo ONE " && " echo TWO
ONE "
'" echo TWO' is not recognized as an internal or external command,
operable program or batch file.
Note that the && works because the first " is escaped, so the && is not quoted. If you add a space before the "&&"
echo ONE^
"&&" echo TWO
then you get the following
C:\test>echo ONE "&&" echo TWO
ONE "&&" echo TWO
because now the space is escaped and the "&&" is quoted.
Regarding your comment, you can ignore those extra spaces - they are an artifact of how the parser displays the line when ECHO is ON. The parser often rearranges the line significantly, but normally it does not affect the outcome.
For example
< nul set test=OK
echo [%test%]
--OUTPUT--
C:\test>set test=OK 0<nul
C:\test>echo [OK]
[OK]
Note how the echoed SET line is completely rearranged. The redirection is compressed and moved to the end of the statement, with a space before the redirection. You might think the space would be included in the assignment, but you can see that it is not :-)
The only time you have might have to worry about the rearrangement is if the command is used in a pipe.
For example:
(set test=OK&call echo [%%^^test%%])|findstr "^"
--OUTPUT--
C:\test>(set test=OK & call echo [%^test%] ) | findstr "^"
[OK ]
You can see that there is a single unwanted extra space that is included in the SET value. This is an artifact of how pipes are implemented - each side is executed in a new CMD.EXE process, and the line gets parsed multiple times. You can see where the space comes from by using %CMDCMDLINE% to display the command line passed to the left side's cmd.exe.
(set test=OK&call echo [%%^^test%%] %%^^cmdcmdline%%)|findstr "^"
--OUTPUT--
C:\test>(set test=OK & call echo [%^test%] %^cmdcmdline% ) | findstr "^"
[OK ] C:\WINDOWS\system32\cmd.exe /S /D /c" ( set test=OK & call echo [%^test%] %^cmdcmdline% )"
See Why does delayed expansion fail when inside a piped block of code? for more information about many quirks with pipes. In particular, pay attention to the selected answer for a good explanation of what is going on.

Related

CMD: How to echo special command symbol to clipboard?

I need to put this string >>> (it's just some handy copypaste) to clipboard.
Since > is a special cmd-character, I'm using ^ before it to mean all special characters literally.
So far, my Batch code looks like this
(&& pause here is used to see debug messages):
echo ^>^>^> && pause
echo ^>^>^>>"%~dp0foo.txt" && pause
echo foo|clip && pause
echo ^>^>^>|clip && pause
1st line works perfectly (not affecting clipboard though).
2nd line works perfectly (not affecting clipboard either though).
3rd line works perfectly (not using the symbols I need though).
4th line returns >> was unexpected at this time error.
Obviously, I need some syntax tips.
It's a bit tricky, because the pipe creates two new cmd instances (like #aschipf mentioned).
You could use a variable and delayed expansion
set "var=>>>"
cmd /v:on /c "echo(!var!"| clip
Or you can use FOR-variable expansion
set "var=>>>"
( FOR %%X in ("%%var%%") DO #(echo(%%~X^) ) | clip
Okay, I figured an almost decent workaround:
echo ^>^>^>>"%~dp0foo.txt"
type "%~dp0foo.txt"|clip
del "%~dp0foo.txt"
puts >>> into foo.txt right next to your Batch
(it also accounts for spaces in path to the file via ").
returns >>> as a content from foo.txt and puts it into clipboard.
deletes foo.txt right away.
Still hoping to meet a proper-syntax-based solution.

Space and " is illegal input while I am echo-ing the variable %input%

I have a weird problem. The program below is supposed to checks user's input and echoes it on the screen, unless the input is some kind of "commands" I have defined in "if" statements like the /h or /r or /x. IF statement will treat the string as a single item and not as several separate strings. So when I input something with a delimiter like
Comma (,)
Semicolon (;)
Equals (=)
Space ( )
Tab ( )
Quote (")
it will search for a second item, thus throws an error and exits.
That's my problem. Please test this program if you can't understand me.
I want somehow to make checks for those "commands" first, before the input is echoed.
Please enlighten me! :/
Thanks.
#echo off
:again
set /p input= ^>
if /i {%input%}=={/h} (
cls
echo /h
echo /r
echo /x
goto again
)
if /i {%input%}=={/r} (
cls
type blablafile.txt
goto again
)
if /i {%input%}=={/x} (
exit
)
echo %input%
goto again
Try changing all the curly braces to double quotes. That fixes the delimiter issue you mentioned for some simple cases, such as a space in the input string. For example,
if /i {%input%}=={/h}
can be changed to
if /i "%input%"=="/h"
Note that this does not handle some other cases, such as a double-quote in the input string. See the other answer for a solution to this.
The IF statement has the general form of IF Value1 CompareOperator Value2 COMMAND
The line has to be parsed into the various components before the IF is actually executed. There are many scenarios that can lead to invalid syntax.
Some obvious ones are special characters like &, |, <, >: These must be escaped like ^& or quoted like "&".
Not as obvious a problem are token delimiters like <space>, <tab>, ;, ,, and =. These also must be escaped or quoted. Take for example a line from your program with <space> as the user input. After expansion of `%input%, the line looks
if /i { }=={/h} (
The parser terminates Value1 at the 1st unescaped/unquoted token delimiter, so it thinks Value1 is { and it thinks that } is part of the CompareOperator. That yields a syntax error.
Another problem is a value with an odd number of quotes, such as your user input example of ".
if /i {"}=={/h} (
Quotes are a state machine. Once quoting is enabled, the parser thinks all characters up until the next quote are part of the same string. So the parser thinks Value1 is {"}=={/h} (, and the command is incomplete - another syntax error.
Partial Solution
You could do something like
if /i "%input%"=="/h" ...
and it will work with input like ; or <space>, or =, etc.
But there are still lots of inputs that will fail. For example: ", "&".
Full Solution
The only sure way to do the test is to enable delayed expansion and use !input! instead of %input%. This works because the parser detects the various components of the IF statement before delayed expansion takes place.
if /i !input!==/h (
The parser identifies Value1 to be !input!, then just before execution, the value is expanded into the user input. At this point there is no more parsing to be done - the IF statement "knows" the entire expanded value is the value to be compared.
Here is a complete working version of your code with the delayed expansion fix:
#echo off
setlocal enableDelayedExpansion
:again
set /p input= ^>
if /i !input!==/h (
cls
echo /h
echo /r
echo /x
goto again
)
if /i !input!==/r (
cls
echo type blablafile.txt
goto again
)
if /i !input!==/x (
exit /b
)
echo !input!

Parenthesis in variables inside IF blocks

In one of my scripts, I need to use variables that contain parenthesis inside IF statements, but either the string is missing a closing parenthesis or the script exits prematurely with * was unexpected at this time (not actually an asterisk), depending on the scenario.
Example
#echo off
SET path=%programFiles(x86)%
echo Perfect output: %path%
IF NOT "%path%" == "" (
REM Variable is defined
echo Broken output: %path%
)
pause >nul
Output
Perfect output: C:\Program Files (x86)
Broken output: C:\Program Files (x86
I think/know that this is because it thinks the closing parenthesis in C:\Program Files (x86) is the end of the IF statement and it exits before the echo is complete.
Is there a simple way to cirumvent this? Preferably without resorting to
single-line IF statements, as I need to run more than one line of code within them,
copious amounts of GOTOs, as it's not practical,
SETLOCAL EnableDelayedExpansion and using !path! instead of %path%, as I recall reading somewhere that that method doesn't work consistently across OSs.
If not, I'll happily accept the most reliable solution offered, whatever it is.
(The scenario isn't up for debate. This is just a refined, concentrated example of the problem. The structure needs to be like this, as it is in my actual script, for reasons I won't go into. It's besides the point and it'll just confuse things and distract from the actual issue.)
First off - you should never use the PATH variable for your own use. It is a reserved environment variable. Using it for your own purposes can break your scripts.
The simplest solution really is to use delayed expansion. As long as your platform uses CMD.EXE then you have access to delayed expansion.
But there is a relatively easy way to make it work without delayed expansion. You can use disappearing quotes. The quote exists at parse time as the name of a FOR variable while the command is parsed. It expands to nothing before execution time.
#echo off
SET mypath=%programFiles(x86)%
echo Perfect output: %mypath%
IF NOT "%mypath%" == "" (
REM Variable is defined
for %%^" in ("") do echo fixed output: %%~"%mypath%%%~"
)
pause >nul
EDIT - When to use delayed expansion: Response to comment
I generally only use delayed expansion when it is needed (or more precisely, when it is advantageous). That being said, I usually find it advantageous in some portion of my batch code.
Major Advantages
Inside a code block in order to see changes to a variable within the block
When dereferencing the name of a variable. If a variable name is passed in as a parameter, the value of the variable can be gotten via delayed expansion: echo !%1!
When using variables as arguments to search and replace or substring operations: echo !var:%search%=%replace%!, echo !var:%start%,%len%!.
Whenever I need to expand the value and not worry about special characters within it needing escaping or quoting: set "var=A&B" & echo !var!
There are other methods to do the above (except the last), but delayed expansion is the easiest, most efficient (fastest to execute), and most reliable option.
Major Disadvantage
Any FOR variable that contains ! in its value will be corrupted when it is expanded if delayed expansion is enabled. I frequently toggle delayed expansion on and off within a FOR loop to get around the problem.
It is not good for executing a "macro" (executing code contained within a variable value) because many important phases of command parsing take place prior to the delayed expansion. So many batch features are unavailable to "macros" that are executed via delayed expansion.
my suggestion is :
if (condition_TRUE) goto goodbye_parenthesis_BEGIN
goto goodbye_parenthesis_END ----- line when previous condition is FALSE ----
:goodbye_parenthesis_BEGIN ----- line when previous condition is TRUE ----
...
variable treatment
...
:goodbye_parenthesis_END
The ) from the resolved variable in your echo statement is prematurely closing the IF block.
Ordinarily, you could fix that by escaping the ) with ^), but you can't modify the environment variable to resolve to C:\Program Files (x86^).
You can prevent this issue by surrounding the variable with quotes.
As a simpler example:
> SET bad=a)b
> IF 1 == 1 ( ECHO %bad% )
b was unexpected at this time.
> IF 1 == 1 ( ECHO "%bad%" )
"a)b"
As others already pointed out, the unescaped and unquoted closing parenthesis ) unintentionally ends the parenthesised if block.
Besides escaping, quotation, delayed expansion and "disappearing quotes", there are the following further options:
Use a for meta-variable on the quoted value and remove the quotes by the ~-modifier:
#echo off
set "PATH=%ProgramFiles(x86)%"
echo Perfect output: %PATH%
if not "%PATH%" == "" (
rem Variable is defined
for %%P in ("%PATH%") do echo Unbroken output: %%~P
)
pause > nul
Use the call command to initiate another variable expansion phase, together with doubled (escaped) %-symbols:
#echo off
set "PATH=%ProgramFiles(x86)%"
echo Perfect output: %PATH%
if not "%PATH%" == "" (
rem Variable is defined
call echo Unbroken output: %%PATH%%
)
pause > nul
Do escaping by sub-string substitution, which happens before ^-escaping is detected:
#echo off
set "PATH=%ProgramFiles(x86)%"
echo Perfect output: %PATH%
if not "%PATH%" == "" (
rem Variable is defined
echo Unbroken output: %PATH:)=^)%
)
pause > nul
Forgive me if I'm reading this wrong, but isn't the "NOT" causing control to enter the bracketed if and run the broken output?
what about:
#echo off
echo Perfect output: %programFiles(x86)%
IF NOT "%programFiles(x86^)%" == "" (
REM Variable is defined
echo Broken output: %programFiles(x86)%
)
pause >nul
?

How to split double quoted line into multiple lines in Windows batch file?

Long commands in Windows batch files can be split to multiple lines by using the ^ as mentioned in Long commands split over multiple lines in Windows Vista batch (.bat) file.
However, if the caret is inside a double quoted string, it won't work. For example:
echo "A very long line I want to ^
split into two lines"
This will print "A very long line I want to ^ and tell me split is an unknown command.
Is there a way to get around this?
I see three possible workarounds.
1) Building the line combining multiple for-parameters.
#echo off
SETLOCAL EnableDelayedExpansion
set "line="
for %%a in ("line1"
"line2"
"line3"
"line4"
) do set line=!line!%%~a
echo !line!
Drawback: It drops lines, when there is a ? or * in the text.
2) Leaving the "quote" at the end of each line
#echo on
SETLOCAL EnableDelayedExpansion
set "line=line1 & x#"^
"line2 & a#"^
"line3 & b #"^
"line4 & c "
set "line=!line:#" "=!"
echo !line!
The first space in each line is important, because the caret works as multiline character but it also escapes the first character, so also a quote would be escaped.
So I replace the unnessary #" " after building the line.
EDIT Added: 3) Disappearing quotes
setlocal EnableDelayedExpansion
echo "A very long line I want to !="!^
split into two lines"
In my opinion this is the best way, it works as the parser first see the quotes and therefore the last caret will work, as it seems to be outside of the quotes.
But this !="! expression will expand the variable named =", but such a variable name can't exists (an equal sign can't occur as first character) so it expands to nothing.
You can also create safe expressions, they will always escape out of quotes, independent if there is a quote or not in the line.
!="^"!
echo This multiline works !="^"!^
as expected
echo "This multiline works !="^"!^
too"
If you want avoid delayed expansion, you could also use a -FOR-Loop like
for %%^" in ("") do (
echo "This multiline works %%~"^
too"
)
The most straight forward answer is to escape the quotes. They will be printed, but they will not functionally quote the contents as far as CMD.EXE is concerned.
#echo off
echo ^"A very long line I want to ^
split into two lines^"
Any special characters that appear in the line must also be escaped since they are no longer functionally quoted.
#echo off
echo ^"A very long line I want to ^
split into two lines ^
that also contains special characters ^^ ^& ^| ^> ^< ^"
As jeb said, the line continuation escapes the first character of the next line. So if the first character of the next line happens to be a special character, it should not be escaped a second time. The code below will successfully escape the & while it also introduces a new line.
#echo off
echo ^"A very long line I want to ^
split into two lines ^
that also contains special characters ^^ ^
& ^| ^> ^< ^"
I came up with my own method for this today, and I've never seen it suggested anywhere so I thought I'd post it here. Certainly, it's not elegant in an objective sense. I would say it has it's own unique kind of ugly and some limitations, but I find it far more "ergonomic" when compared to the alternatives.
Specifically, this method has the unique characteristic of not needing to escape anything, while still performing typical variable substitution. This enables me to to take some string that looks exactly how I want it, spread it across multiple lines, and add a prefix to each line without changing anything else in the string. Thus, there's no trial-and-error needed to figure out how special characters and escape characters might interact. Overall, I think it reduces the cognitive load needed to deterministically produce a complicated string variable over multiple lines, which is good for me because I don't want to have to think hard and get frustrated with incidental complexity like the nuances of the windows command interpreter. Also, a similar technique can be used on linux bash, making it somewhat portable.
Note: I've not thoroughly tested it, it might not work for some use cases, I don't know. However, the example has a fair number of seemingly troublesome characters an it works, so it seems somewhat robust.
set IMAGE_NAME=android-arm64
set CMD=docker run --rm
set CMD=%CMD% --platform=linux
set CMD=%CMD% -v %CD%:/work
set CMD=%CMD% dockcross/%IMAGE_NAME%
set CMD=%CMD% /bin/sh -c
set CMD=%CMD% "mkdir build-%IMAGE_NAME% &&
set CMD=%CMD% cd build-%IMAGE_NAME% &&
set CMD=%CMD% cmake .. &&
set CMD=%CMD% cmake --build ."
echo %CMD%
In my case, it's a command, so I can run it with:
%CMD%
I'd be open to any feedback or suggestions about this method. Perhaps it can even be improved.
The clear advantage of #solvingJ's method over the variants of Jeb is that it is quite obvious what happens and that there is a spatial seperation of on the left hand the ugly cmd stuff and on the right the actual content. And it can be improved in readability by putting more spaces between the left and the right part:
set CMD= docker run --rm
set CMD=%CMD% --platform=linux
set CMD=%CMD% -v %CD%:/work
set CMD=%CMD% dockcross/%IMAGE_NAME%
set CMD=%CMD% /bin/sh -c
set CMD=%CMD% "mkdir build-%IMAGE_NAME% &&
set CMD=%CMD% cd build-%IMAGE_NAME% &&
set CMD=%CMD% cmake .. &&
set CMD=%CMD% cmake --build ."
Unfortunately it is not possible to demonstrate this in a comment because multiple spaces are reduced to one there. So I do in an answer what is actually just a comment to #solvingJ's answer, which I consider the best.
If you want to add blank lines then ensure you have a space BEFORE the ^ on the 2nd line
echo Hello world^
^
Have a good day !

Split long commands in multiple lines through Windows batch file

How can I split long commands over multiple lines in a batch file?
You can break up long lines with the caret ^ as long as you remember that the caret and the newline following it are completely removed. So, if there should be a space where you're breaking the line, include a space. (More on that below.)
Example:
copy file1.txt file2.txt
would be written as:
copy file1.txt^
file2.txt
The rule for the caret is:
A caret at the line end, appends the next line, the first character of the appended line will be escaped.
You can use the caret multiple times, but the complete line must not exceed the maximum line length of ~8192 characters (Windows XP, Windows Vista, and Windows 7).
echo Test1
echo one ^
two ^
three ^
four^
*
--- Output ---
Test1
one two three four*
echo Test2
echo one & echo two
--- Output ---
Test2
one
two
echo Test3
echo one & ^
echo two
--- Output ---
Test3
one
two
echo Test4
echo one ^
& echo two
--- Output ---
Test4
one & echo two
To suppress the escaping of the next character you can use a redirection.
The redirection has to be just before the caret.
But there exist one curiosity with redirection before the caret.
If you place a token at the caret the token is removed.
echo Test5
echo one <nul ^
& echo two
--- Output ---
Test5
one
two
echo Test6
echo one <nul ThisTokenIsLost^
& echo two
--- Output ---
Test6
one
two
And it is also possible to embed line feeds into the string:
setlocal EnableDelayedExpansion
set text=This creates ^
a line feed
echo Test7: %text%
echo Test8: !text!
--- Output ---
Test7: This creates
Test8: This creates
a line feed
The empty line is important for the success. This works only with delayed expansion, else the rest of the line is ignored after the line feed.
It works, because the caret at the line end ignores the next line feed and escapes the next character, even if the next character is also a line feed (carriage returns are always ignored in this phase).
(This is basically a rewrite of Wayne's answer but with the confusion around the caret cleared up. So I've posted it as a CW. I'm not shy about editing answers, but completely rewriting them seems inappropriate.)
You can break up long lines with the caret (^), just remember that the caret and the newline that follows it are removed entirely from the command, so if you put it where a space would be required (such as between parameters), be sure to include the space as well (either before the ^, or at the beginning of the next line — that latter choice may help make it clearer it's a continuation).
⚠ Note: The first character of the next line is escaped. So if it carries any special meaning (like & or |), that meaning will be lost and it will be interpreted as a pure text character (see last example at bottom).
Examples: (all tested on Windows XP and Windows 7)
xcopy file1.txt file2.txt
can be written as:
xcopy^
file1.txt^
file2.txt
or
xcopy ^
file1.txt ^
file2.txt
or even
xc^
opy ^
file1.txt ^
file2.txt
(That last works because there are no spaces betwen the xc and the ^, and no spaces at the beginning of the next line. So when you remove the ^ and the newline, you get...xcopy.)
For readability and sanity, it's probably best breaking only between parameters (be sure to include the space).
Be sure that the ^ is not the last thing in a batch file, as there appears to be a major issue with that.
Here's an example of character escaped at the start of the next line:
xcopy file1.txt file2.txt ^
& echo copied successfully
This will not work because & will be escaped and lose its special meaning, thus sending all of "file1.txt file2.txt & echo copied successfully" as parameters to xcopy, causing an error (in this example).
To circumvent, add a space at the beginning of the next line.
Multiple commands can be put in parenthesis and spread over numerous lines; so something like echo hi && echo hello can be put like this:
( echo hi
echo hello )
Also variables can help:
set AFILEPATH="C:\SOME\LONG\PATH\TO\A\FILE"
if exist %AFILEPATH% (
start "" /b %AFILEPATH% -option C:\PATH\TO\SETTING...
) else (
...
Also I noticed with carets (^) that the if conditionals liked them to follow only if a space was present:
if exist ^
It seems however that splitting in the middle of the values of a for loop doesn't need a caret(and actually trying to use one will be considered a syntax error). For example,
for %n in (hello
bye) do echo %n
Note that no space is even needed after hello or before bye.
One thing I did not find when searching for 'how to split a long DOS batch file line' was how to split something containing long quoted text.
In fact it IS covered in the answers above, but is not obvious. Use Caret to escape them.
e.g.
myprog "needs this to be quoted"
can be written as:
myprog ^"needs this ^
to be quoted^"
but beware of starting a line with Caret after ending a line with caret - because it will come out as caret..?:
echo ^"^
needs this ^
to be quoted^
^"
-> "needs this to be quoted^"
Though the carret will be preferable way to do this here's one more approach using macro that constructs a command by the passed arguments:
#echo off
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
set "{{=setlocal enableDelayedExpansion&for %%a in (" & set "}}="::end::" ) do if "%%~a" neq "::end::" (set command=!command! %%a) else (call !command! & endlocal)"
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
%{{%
echo
"command"
written
on a
few lines
%}}%
command is easier to read without the carets but using special symbols e.g. brackets,redirection and so on will break it. So you can this for more simpler cases. Though you can still enclose parameters in double quotes

Resources