I have a Windows CMD script that accepts a number of parameters and executes an EXE, passing first some hard-coded arguments and then all of the parameters from the user. The CMD script looks like this:
launcher.exe paramX paramY %*
The user would execute the CMD script from the Windows shell as follows:
launcher.cmd param1 param2 param3 [...]
The problem I have is that if the parameters to the CMD script contain shell special characters like < > and ^, the user is forced to escape these by preceding each with 3 caret ^ shell escape characters.
Two Examples
1) To pass the argument ten>one to the EXE, the user must launch the CMD as follows:
launcher.cmd ten^^^>one
The reason for this is that the shell special characters ^ and > are interpreted by the command shell at two levels, first on the command line and second inside the CMD script. So, the shell escaping with the caret ^ shell escape character must be applied twice. The problem is that this is non-obvious to the user and looks ugly.
For this example, a nicer solution is to surround the argument with double quotes. However, this breaks down for more complex examples that include a literal double quote in the argument.
2) To pass the argument "^ to the EXE, the user must launch the CMD as follows:
launcher.cmd "\"^^^^"
In my case I want to support arguments that contain any sequence of low ASCII characters, excluding control characters, i.e. code points 0x20 to 0x7E. I understand that there will be examples where the user will have to escape certain shell special characters with a caret. However, I don't want the user to have to use 3 carets every time in these cases just because they happen to be calling a CMD script instead of an EXE.
I can solve this problem by replacing the CMD script with an EXE that does the same. However, is there any way to alter the CMD script so that it passes its parameters through to the EXE without interpreting the shell special characters?
One way is to work with delayed expansion inside of the batch, because then the special characters lose there "special" meanings.
The only problem is to get the parameters into a variable.
Something like this could help
#echo off
setlocal DisableDelayedExpansion
rem ** At this point the delayedExpansion should be disabled
rem ** otherwise an exclamation mark in %1 can remove carets
set "param1=%~1"
setlocal EnableDelayedExpansion
rem ** Now you can use the param1, independent of the content, even with carets or quotes
rem ** but be careful with call's, because they start a second round of expansion
echo !param1!
set "tmp=!param1:~1,4!"
Now the parameters can be surround by quotation marks, so there the carets aren't neccessary anymore.
Example
launcher.bat "abc>def&geh%ijk|lmn^opq!"
The only remaining problematic special character seems to be the quotation mark.
[Edit/Improve]
I create another way to retrieve a parameter, I assume it can accept any string also your second example.
Even really hard strings like
launcher "^
launcher ten^>one
launcher "&"^&
#echo off
setlocal DisableDelayedExpansion
set "prompt=X"
for %%a in (1 ) do (
#echo on
for %%b in (4) do (
rem #%1#
)
) > XY.txt
#echo off
for /F "delims=" %%a in (xy.txt) DO (
set "param=%%a"
)
setlocal EnableDelayedExpansion
set param=!param:~7,-4!
echo param='!param!'
How it works?
The only way I have found to expand %1 without expanding the special characters like " or ^ is in a REM statement (For REM that's not completly true, but that is an other story)
Ok, the only problem is that a REM is a remark and has no effect :-)
But if you use echo on also rem lines are echoed before they are executed (execute for rem is a nice word).
The next problem is that it is displayed and you can not redirect this debug output with the normal > debug.txt.
This is also true if you use a for-loop.
Ok, you can redirect the echo on output with a call like
echo on
call :myFunc > debug.txt
But if you call a function you can't access the %1 of the batch file anymore.
But with a double for-loop, it is possible to activate the redirection for the debug output and it's still possible to access %1.
I change the prompt to "X", so I know it is always only one character long.
The only thing left is to explain why I append a # to %1.
That's because, some special characters are recognized in some situations even in a REM line, obviously ;-)
rem This is a remark^
rem This_is_a_multiline^
rem "This is also a multiline"^
So the # suppress a possible multiline situation.
Does this help:
EscapPipes.Cmd:
#echo off
:Start
If [%1]==[] goto :eof
#Echo %1
shift
goto :Start
When started thus:
EscapPipes.Cmd Andy Pandy "Pudding | > < and pie"
gives
Andy
Pandy
"Pudding | > < and pie"
As soon as you strip the quotes the pipe symbols will become live.
Related
I want to put a user inputted string which contains an ampersand into the clipboard using batch. I can modify the string, and I can get it to print to the terminal using setlocal EnableExtensions EnableDelayedExpansion
but I can't pipe it to the clipboard.
There is an in depth discussion here which talks about why pipes can break things, but I couldn't understand it well enough to get around my problem.
https://www.robvanderwoude.com/battech_inputvalidation_setp.php
setlocal EnableExtensions EnableDelayedExpansion
set /P "INPUT=Paste the stuff in the terminal please"
set "SEARCHTEXT=+"
set REPLACETEXT=%%2B
for /F "delims=" %%A in ("%INPUT%") do (
set "string=%%A"
set "modified=!string:%SEARCHTEXT%=%REPLACETEXT%!"
echo !modified! | clip
)
Because the string I'm trying to modify contains "&username" in it, the output I get is:
'username' is not recognized as an internal or external command,
operable program or batch file.
If I only echo !modified!, there are no errors. How can I get an arbitrary un-sanitized string into the clipboard?
The major problem in your code is the following line:
echo !modified! | clip
A pipe (|) creates a new cmd instance for either side. You have got delayed expansion enabled in your script, so the variable !modified! becomes expanded when the whole command line is parsed, then the pipe is executed, and then the new cmd instance for the left side receives the variable already expanded, including all potential poisonous characters, like &, for example.
To prevent !modified! to be expanded immediately, we need to escape the exclamation marks like ^^! (^^ becomes first escaped to a single ^, so ^! is left during the delayed expansion phase), which lets the ! be treated as a literal character and no variable expansion happens at first.
The new cmd instance (for the left side of the pipe in our situation) now has got delayed expansion disabled, so we need to explicitly instantiate another (nested) one with delayed expansion enabled (by cmd /V):
cmd /V /C echo(^^!modified^^!| clip
With this technique we force the variable !modified! to be expanded as late as possible, hence by the inner-most cmd instance, which avoids the expanded string to be received by any other instance, and therefore, poinsonous characters become hidden from the parser.
In addition, I used the safe echo variant echo(..., because echo ... might fail under certain circumstances (imagine ... is the literal string /?). Moreover, I removed the SPACE in front of |, because such would become echoed as well, unintentionally.
When I try to make a variable, sometimes it won't recall when surrounded with %. An example is
#echo off
set firstvar= <o
set secondvar= ^. .^
echo.
echo %firstvar%
echo %secondvar%
pause
I'm wanting it to show up as
<o
^. .^
but I get
The system cannot find the file specified.
ECHO is off.
. .echo.
Press any key to continue . . .
I'm really confused by this and I can't seem to fix it. I've narrowed down the problem to be something about the rules of assigning variables, and I'm guessing that the <o is being viewed as HTML or a website or something. I'm really confused with the second line though, why does it say ECHO is off? Why . .echo.? Any help would be appreciated, thank you.
First, read the answer on Why is no string output with 'echo %var%' after using 'set var = text' on command line? for an explanation why it is advisable to use the syntax set "variable=value" on assigning something to an environment variable.
Second, read the Microsoft article about Using Command Redirection Operators for an explanation of meaning of < in a Windows command line.
Third, caret character ^ is interpreted by Windows command interpreter as escape character like the backslash character \ by many programming and scripting languages.
So the batch code with immediate environment variable expansion should be coded as:
#echo off
set "firstvar= ^<o"
set "secondvar= ^^. .^^"
echo/
echo %firstvar%
echo %secondvar%
echo Okay!
The strings to output are easier to code on using delayed expansion:
#echo off
setlocal EnableExtensions EnableDelayedExpansion
set "firstvar= <o"
set "secondvar= ^. .^"
echo/
echo !firstvar!
echo !secondvar!
echo Okay^^!
endlocal
^ and < must not be escaped inside a double quoted string and delayed expansion is used on outputting the strings of the environment variables.
Note 1: The two environment variables firstvar and secondvar do not exist anymore after the line with endlocal which is usually the last line in a batch file on using setlocal. Read this answer for details about the commands SETLOCAL and ENDLOCAL.
Note 2: A percent sign % must be escaped with another % and not with ^ to be interpreted as literal percent sign and not as begin/end of an environment variable reference expanded during preprocessing of a command line or an entire command block starting with ( and ending with matching ) before running a command line.
Note 3: Any string including directory/file names containing an exclamation mark ! is interpreted different on having delayed expansion enabled. Then the exclamation mark is interpreted as begin/end of an environment variable reference expanded delayed after preprocessing a command line/block. ! must be escaped with two ^ on delayed expansion being enabled to be interpreted as literal character. The command line with ^^! changes to just ^! on first parsing it by Windows command processor. The command line changes to literally interpreted ! on second parsing of the command line because of enabled delayed expansion.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
cmd /? ... Windows command interpreter
echo /?
endlocal /?
set /?
setlocal /?
And read also DosTips forum topic ECHO. FAILS to give text or blank line - Instead use ECHO/ why it is better to use echo/ instead of echo. to output an empty line.
Last read also Debugging a batch file for instructions how to see what the command interpreter really executes after preprocessing each command line/block.
See also: How does the Windows Command Interpreter (CMD.EXE) parse scripts?
The caret ^ is the escape symbol, to echo it literally you have to double it ^^ once for every processing step (setting/echoing).
The redirecting/piping symbols <|> have to be escaped with a ^ caret.
If a string/variable to echo evaluates to nothing echo returns the status.
To avoid this use a different delimiter sign instead of a space \/(:; etc.
#echo off
set firstvar= ^^^<o
set secondvar= ^^^^. .^^^^
echo.
echo:%firstvar%
echo:%secondvar%
pause
So I'm currently writing a tutorial about security and for that reason I have to write a vbe file (encoded script written in VBScript) using a batch file.
So, I just have to write this to a file:
##~^mgAAAA==6 P3MDKDP"+k;:PH+XY~###&fks~D;EdO{6k^+SPhnk/Co8WX~~AMkYnm6ks+B~T+O|wmYtBPDn:a{2lDtS~6kxms{alY4~###&s+k/Con8K6~',h/T4GavJKndDJ~~8BPEwlDlV,2M.WMJbP###&2zEAAA==^#~#
(Note: There are some characters that cannot be printed above).
But the problem is that I never managed to write it successfully, I tried escaping all the characters using instructions from http://www.robvanderwoude.com/escapechars.php and it didn't work.
I tried using DelayedExpansion like this:
SET "foo=##~^mgAAAA==6 P3MDKDP"+k;:PH+XY~###^&fks~D;EdO{6k^+SPhnk/Co8WX~~AMkYnm6ks+B~T+O|wmYtBPDn:a{2lDtS~6kxms{alY4~###&s+k/Con8K6~',h/T4GavJKndDJ~~8BPEwlDlV,2M.WMJbP###^&2zEAAA==^#~# "
setlocal EnableDelayedExpansion
(
echo !foo!
) > test.vbe
And it did not work either, I have problems with characters that are not escaped.
Any ideas?? Thanks!!
The reason is obvious, that is a quotation mark after [...P3MDKDP]. Since you assign the variable "foo" to jumble characters with a open and a close quotation mark, like so SET "foo=...", batch think you stop assigning "foo" after [...P3MDKDP]. This leaves [+k;:PH+XY~.....] alone, without assigning to a variable or working with commands. Batch can't recognize it, and so the command prompt quit automatically.
What you can do is, assign the part after the quotation mark to another variable, I named it "foo2" in the following example:
#echo off
setlocal enabledelayedexpansion
SET "foo=##~^mgAAAA==6 P3MDKDP""
SET "foo2=+k;:PH+XY~###^&fks~D;EdO{6k^+SPhnk/Co8WX~~AMkYnm6ks+B~T+O|wmYtBPDn:a{2lDtS~6kxms{alY4~###&s+k/Con8K6~',h/T4GavJKndDJ~~8BPEwlDlV,2M.WMJbP###^&2zEAAA==^#~# "
echo !foo!!foo2!>test.vbe
pause >nul
And also, if you add another quotation mark before / after the quotation mark, like so [P3MDKDP ""], even though you did not assign the second part to a new variable, it still work, but it output an extra quotation mark in the string.
maybe this little trick helps you:
#echo off
for /f "delims=[]" %%n in ('find /n "REM DATA:" "%~dpnx0"') do set /a n=%%n
more +%n% "%~dpnx0">test.vbe
REM rest or your batchfile
goto :eof
REM DATA:
##~^mgAAAA==6 P3MDKDP"+k;:PH+XY~###^&fks~D;EdO{6k^+SPhnk/Co8WX~~AMkYnm6ks+B~T+O|wmYtBPDn:a{2lDtS~6kxms{alY4~###&s+k/Con8K6~',h/T4GavJKndDJ~~8BPEwlDlV,2M.WMJbP###^&2zEAAA==^#~#
(this trick avoids any character escaping or splitting the string. Can also be used to write a multiline text)
Related: Using batch echo with special characters
How do I deal with using escape characters on text that might or mightn't be a special character?
Suppose we have user input:
Set /p var=prompt:
Now, I need to make sure that the text gets interpreted as text even if the user enters something like a special character. But I cannot simply add ^ before the variable...because that'd cancel the variable. The ^^%var% and ^%%var% options don't seem to work either.
How do I go about doing this?
You should realize that the escapes are required in the source code of your program or when you expand a variable via %variable% or in the replaceable parameter of a for command. This is not required if you expand a variable via !delayed! expansion. So, your problem may be easily solved this way:
setlocal EnableDelayedExpansion
Set /p var=prompt:
echo !var!
The standard method to avoid the problem when you read a file that may have special characters is enclosing the value of the replaceable parameter in quotes when the value is asigned with Delayed Expansion disabled, and then Enable Delayed Expansion in order to manage the value. Of course, this method forces to insert an endlocal command inside the loop:
for /F "delims=" %%a in (anyFile.txt) do (
set "line=%%a"
setlocal EnableDelayedExpansion
echo Line with special characters: !line!
endlocal
)
Not possible. The shell processes the user input before your script does. Your script won't even know the user typed an escape character.
I am trying to create a batch file to input 3 pieces of data and use that data to create another batch file. Just create it, and stop. The batch maps several network drives for users that haaven't a clue as to how to do it.
I have a "master.bat" and using notepad I am using "replace" to fill in the "username" "Password" and "drive path". I thought I would try to get it down to entering the variables into the "master.bat" creating a "custom.bat" for that user.
I got a lot of help here getting to the final step. Everything is working except the final part. Now that I have all the variables as well as a template to put them in, how do I get that first batch file to create the cuctomized output as a workable file that I can send the user where all they do is run it.
One way would be to use your template in file form and replace placeholders in there by your actual values:
setlocal enabledelayedexpansion
for /f %%L in (template.cmd) (
set "Line=%%L"
set "Line=!Line:[username]=!username!"
...
>network-drives.cmd echo !Line!
)
This assumes placeholders like [username] in the template and corresponding variables defined.
However, I always get a little anxious if I use data read from a file in a batch. When I recently had to create a batch file from another I went the following route:
(
echo #echo off
echo net use !drivepath! \\server\share "/user:!username!" "!password!"
echo net use !drivepath2! \\server\share2 "/user:!username!" "!password!"
) > network_drives.cmd
Care has to be taken with things like closing parentheses and several characters reserved for the syntax you may need in the generated batch file. But this approach is entirely self-contained, albeit a little harder to maintain.
It is simple to embed the template within your batch file. There are multiple ways to do this. One is to simply prefix each template line with :::. I chose that sequence because : is already used as a batch label and :: is frequently used as a batch comment.
Delayed expansion can be used to do your search and replace automatically!
There are just 3 special characters you need to worry about if you want to include them in your output. These special characters are probably not needed for the original question. But it is good to know how to handle them in a general sense.
An exclamation literal ! must be either escaped or substituted
A caret literal ^ can be escaped or substituted if it appears on a line with an exclamation. But it must not be escaped if there is not an exclamation on the line. Caret substitution is always safe.
Use substitution to start a line with :
#echo off
setlocal
::The following would be set by your existing script code
set drivePath=x:
set username=rumpelstiltskin
set password=gold
::This is only needed if you want to include ! literals using substitution
set "X=!"
::This is only needed if you want to include ^ literal on same line
::containing ! literal
set "C=^"
::This is only needed if you want to start a line with :
set ":=:"
::This is all that is needed to write your output
setlocal enableDelayedExpansion
>mapDrive.bat (
for /f "tokens=* delims=:" %%A in ('findstr "^:::" "%~f0"') do #echo(%%A
)
::----------- Here begins the template -----------
:::#echo off
:::net use !drivePath! \\server\share "/user:!username!" "!password!"
:::!:!: Use substitution to start a line with a :
:::!:!: The following blank line will be preserved
:::
:::echo Exclamation literal must be escaped ^! or substituted !X!
:::echo Caret with exclamation must be escaped ^^ or substituted !C!
:::echo Caret ^ without exclamation must not be escaped