I have a batch file which moves files from one folder to another. The batch file is generated by another process.
Some of the files I need to move have the string "%20" in them:
move /y "\\myserver\myfolder\file%20name.txt" "\\myserver\otherfolder"
This fails as it tries to find a file with the name:
\\myserver\myfolder\file0name.txt
Is there any way to ignore %? I'm not able to alter the file generated to escape this, such as by doubling percent signs (%%), escaping with / or ^ (caret), etc.
You need to use %% in this case. Normally using a ^ (caret) would work, but for % signs you need to double up.
In the case of %%1 or %%i or echo.%%~dp1, because % indicates input either from a command or from a variable (when surrounded with %; %variable%)
To achieve what you need:
move /y "\\myserver\myfolder\file%%20name.txt" "\\myserver\otherfolder"
I hope this helps!
The question's title is very generic, which inevitably draws many readers looking for a generic solution.
By contrast, the OP's problem is exotic: needing to deal with an auto-generated batch file that is ill-formed and cannot be modified: % signs are not properly escaped in it.
The accepted answer provides a clever solution to the specific - and exotic - problem, but is bound to create confusion with respect to the generic question.
If we focus on the generic question:
How do you use % as a literal character in a batch file / on the command line?
Inside a batch file, always escape % as %%, whether in unquoted strings or not; the following yields My %USERNAME% is jdoe, for instance:
echo My %%USERNAME%% is %USERNAME%
echo "My %%USERNAME%% is %USERNAME%"
On the command line (interactively) - as well as when using the shell-invoking functions of scripting languages - the behavior fundamentally differs from that inside batch files: technically, % cannot be escaped there and there is no single workaround that works in all situations:
In unquoted strings, you can use the "^ name-disrupter" trick: for simplicity, place a ^ before every % char, but note that you're not technically escaping % that way (see below for more); e.g., the following again yields something like My %USERNAME% is jdoe:
echo My ^%USERNAME^% is %USERNAME%
In double-quoted strings, you cannot escape % at all, but there are workarounds:
You can use unquoted strings as above, which then requires you to additionally ^-escape all other shell metacharacters, which is cumbersome; these metacharacters are: <space> & | < > "
Alternatively, unless you're invoking a batch file, , you can individually double-quote % chars as part of a compound argument (most external programs and scripting engines parse a compound argument such as "%"USERNAME"%" as verbatim string %USERNAME%):
some_exe My "%"USERNAME"%" is %USERNAME%
From scripting languages, if you know you're calling a binary executable, you may be able to avoid the whole problem by forgoing the shell-invoking functions in favor of the "shell-free" variants, such as using execFileSync instead of execSync in Node.js.
Optional background information re command-line (interactive) use:
Tip of the hat to jeb for his help with this section.
On the command line (interactively), % can technically not be escaped at all; while ^ is generally cmd.exe's escape character, it does not apply to %.
As stated, there is no solution for double-quoted strings, but there are workarounds for unquoted strings:
The reason that "^ name-disrupter" trick (something like ^%USERNAME^%) works is:
It "disrupts" the variable name; that is, in the example above cmd.exe looks for a variable named USERNAME^, which (hopefully) doesn't exist.
On the command line - unlike in batch files - references to undefined variables are retained as-is.
Technically, a single ^ inside the variable name - anywhere inside it, as long as it's not next to another ^ - is sufficient, so that %USERNAME^%, for instance, would be sufficient, but I suggest adopting the convention of methodically placing ^ before each and every % for simplicity, because it also works for cases such as up 20^%, where the disruption isn't even necessary, but is benign, so you can apply it methodically, without having to think about the specifics of the input string.
A ^ before an opening %, while not necessary, is benign, because ^ escapes the very next character, whether that character needs escaping - or, in this case, can be escaped - or not. The net effect is that such ^ instances are ultimately removed from unquoted strings.
Largely hypothetical caveat: ^ is actually a legal character in variable names (see jeb's example in the comments); if your variable name ends with ^, simply place the "disruptive" ^ somewhere else in the variable name, as long as it's not directly next to another ^ (as that would cause a ^ to appear in the resulting string).
That said, in the (very unlikely) event that your variable has a name such as ^b^, you're out of luck.
In batch files, the percent sign may be "escaped" by using a double percent sign ( %% ).
That way, a single percent sign will be used within the command line. from http://www.robvanderwoude.com/escapechars.php
I think I've got a partial solution working. If you're only looking to transfer files that have the "%20" string in their name and not looking for a broader solution, you can make a second batch file call the first with %%2 as the second parameter. This way, when your program tries to fetch the second parameter when it hits the %2 in the text name, it will replace the %2 with an escaped %2, leaving the file name unchanged.
Hope this works!
How to "escape" inside a batch file withoput modify the file**
The original question is about a generated file, that can't be modified, but contains lines like:
move /y "\\myserver\myfolder\file%20name.txt" "\\myserver\otherfolder"
That can be partly solved by calling the script with proper arguments (%1, %2, ...)
#echo off
set "per=%%"
call generated_file.bat %%per%%1 %%per%%2 %%per%%3 %%per%%4
This simply sets the arguments to:
arg1="%1"
arg2="%2"
...
How to add a literal percent sign on the command line
mklement0 describes the problem, that escaping the percent sign on the command line is tricky, and inside quotes it seems to be impossible.
But as always it can be solved with a little trick.
for %Q in ("%") do echo "file%~Q20name.txt"
%Q contains "%" and %~Q expands to only %, independent of quotes.
Or to avoid the %~ use
for /F %Q in ("%") do echo "file%Q20name.txt"
You should be able to use a caret (^) to escape a percent sign.
Editor's note: The link is dead now; either way: It is % itself that escapes %, but only in batch files, not at the command prompt; ^ never escapes %, but at the command prompt it can be used indirectly to prevent variable expansion, in unquoted strings only.
The reason %2 is disappearing is that the batch file is substituting the second argument passed in, and your seem to not have a second argument. One way to work around that would be to actually try foo.bat ^%1 ^%2... so that when a %2 is encountered in a command, it is actually substituted with a literal %2.
Related
I'm using windows 10, running batch files through the command prompt window.
I can make things work, but I don't know why it works or why I can't do certain things:
set "file_list=a1 a2"
for %%a in (%file_list%) do (
echo %%a.py
)
This little piece of code works. I can build on it, BUT
Q1: I want to change the variable %%a to %%filename... but that doesn't work! I wondered if maybe filename were reserved, so I tried %%fname .
In this case I get the error:
%fname was unexpected at this time.
I can do a set fromm the command line and use a descriptive variable name, but it doesn't seem to work when looping. (I did it with the %file_list% variable above!) So how come I can only use a single character for a loop variable? Is there some way around that?
Q1a. This makes me think that the loop index variable is a different kind of variable that the ones in set commands. Is that correct? If so, is there a link that clearly and concisely explains the difference?
Q2. I notice the loop index variable is %%a, instead of a or %a or %a% . I never would have guessed this. The web sites I've looked at have just said, do this. But I can't see any explanation of why, except that the first percent is an escape. Okay. That doesn't really explain anything. It just means "this is how you do it." The error message when I use one percent sign is interesting.
set "file_list=a1 a2"
for %a in (%file_list%) do (
echo %a.py
)
"file_list) was unexpected at this time."
So I can vaguely see that maybe something isn't being escaped correctly. Why does that % in the %a need to be escaped, so it becomes %%a ?
A for meta-variable must consist of % (in Command Prompt) or %% (in a batch file) and a single character (case-sensitive letter), like %a or %%a. You cannot define %filename or %%filename.
Loop variables only exist within the respective for loop. Do not confuse such loop variables with normal environment variables, like %TEMP%, for example, which are available globally.
There are these things marked by %-signs:
%-escaping (only applicable for batch files), so %% denotes one literal %-sign;
command line arguments/parameters (only applicable for batch files, obviously), like %1;
immediately expanded environment variables*, like %TEMP%;
for meta-variables, like %a (in Command Prompt) or %%a (in batch files), which are specific to the for command, so they do not exist outside of the related loop context;
%-escaping (1.) happens before expanding for meta-variables (4.), hence actually the for command receives a loop variable like %a.
Then environment variables (3.) are treated differently in Command Prompt and in batch files: the former keeps undefined variables literally, the latter removes them.
The detailed parsing rules can be found in this post, which have been implemented by Microsoft (or IBM?) that way in order to be able to distinguish between the different %-things, so at the end it was their decision, therefore you have to ask them for the exact reason…
*) There is also something like delayed environment variable expansion, but this uses !-signs to mark the variables, like !TEMP!, and this is something that happens after all the %-sign expressions have been parsed.
A command needs to use a sequence looking like this one: Z;[%&.:mn=WO. The following call works:
D:\Some_folder> some_command -s "Z;[%&.:mn=WO"
The aim is to embed this call in a batch file, whose behavior isn't than the interactive mode one.
As you can see, there are several special characters, so it has to be protected before use. The batch file looks like the sample below:
set SEQUENCE=<sequence>
call %SOMEREP%\command -s %SEQUENCE% REM blah blah blah
Using set SEQUENCE="Z;[%&.:mn=WO" will fail, and surprisingly, with echo on, the displayed line before execution is exactly the same as the one in interactive mode, which works. Actually, the command line complains that the sequence should only contains ASCII characters in a range of 33-256, and be exactly 12 characters length. As is the quotes became a part of the value.
The attempt to protect each characters fails, with or without quotes:
With set SEQUENCE=Z^;[^^%%^&.:mn^=WO, the line will be:
D:\Some_folder> some_command -s Z;[% & .:mn=WO
It produces the error "The system cannot find the drive" (sounds like .: is interpreted as a drive switching)
With set SEQUENCE="Z^;[^^%%^&.:mn^=WO", the line will be:
D:\Some_folder> some_command -s "Z^;[^^%^&.:mn^=WO"
It complains about the sequence size, as well as the first attempt
Other character escaping attempt were tried without success, like quote escaping inside quoted string (""). What point is missed here? Asking before rewriting it in Bash.
Update: I noticed this morning that the reproduced attempts show a double caret (^^). I don't recall why it was done. I let it in place.
Within a batch-file, cmd will consume the first % so just double up on it:
set "SEQUENCE=Z;[%%&.:mn=WO"
some_command -s "%SEQUENCE%"
or if you want to retain the double quotes as part of the value, which is not preferred though:
set SEQUENCE="Z;[%%&.:mn=WO"
some_command -s %SEQUENCE%
If needed to use it without the surrounding double quotes, directly from the command, you need to double up on % and escape the & else the ampersand will be seen as a chaining operator:
some_command -s Z;[%%^&.:mn=WO
a Good source, in table form, for all of the escape sequences can be found on Robvanderwoude
EDIT
To explicitly call it, simply double on the variable % again:
set SEQUENCE="Z;[%%&.:mn=WO"
call some_command -s "%%SEQUENCE%%"
Calling the command is generally unnecessary unless you have variables set in a loop and delayed expansion isn't enabled (I used to explicitly avoid delayed expansion and use calls then I needed it instead.)
Assuming that %SOMEREP% was not set in a code block (and therefore no need to Call it) then the following should do.
Essentially just double the % once on the Variable, unless you really do need to call, in which case you quadruple it.
No Call:
Set "_CMD=%SOMEREP%\command"
Set "_SEQUENCE=Z;[%%&.:mn=WO"
"%_CMD%" -s "%_SEQUENCE%"
REM blah blah
Assuming you have actual need to call the command, the following escapes the % twice by using the %%%%
Set "_CMD=%SOMEREP%\command"
Set "_SEQUENCE=Z;[%%%%&.:mn=WO"
call "%_CMD%" -s "%_SEQUENCE%"
REM blah blah
Ignore the original bit about the carrots, I wrote this as I was waking this AM and apparently my brain was still thinking about some commands I was runnign the day prior and got confused, as at the CMD CLI you need to use ^ to escape % to force the delayed expansion of some variables.
I have a simple batch test file test.bat with following lines:
#echo off
REM IF "%~version_info" == "" echo No version information found
echo test
When I run it I expected to get test instead I get:
The following usage of the path operator in batch-parameter
substitution is invalid: %~version_info" == "" echo No version information found
For valid formats type CALL /? or FOR /?
The syntax of the command is incorrect.
Why does batch try to interpret the comment? Or what is happening here? If I take the comment out, the script prints out test as expected.
Also the documentation doesn't mention anything of this.
I believe it's a consequence of the parsing sequence. In this case, it's a problem, but suppose you code (as I have done):
set "debug=rem"
%debug% echo some debug data
First, we replace the values in %vars%, then we interpret the line, using the first token as the command to execute. The above construct allows the command to be varied.
So there is a method to the madness...
The reason for this is the sequence of batch scripts.
The very first thing that happens is the (poor1) %-sign handling, that is the normal variable (%VAR%) and the command line argument (%1, %2, etc., and %*) expansion. Commands, and therefore even rem, are recognised in a later parsing phase.
The string %~ is an invalid argument reference, because there is neither a valid modifier or a combination of such (f, d, p, n, x, s, a, t, z, $PATH:), or a numeric digit following.
Refer to this thread: How does the Windows Command Interpreter (CMD.EXE) parse scripts?
1... The % expansion is faulty in my opinion, because %~ or %VAR:=, %VAR:*=, in case variable VAR is defined, result in an error, and variable expansion like %VAR:[*]search=[replace]% or %VAR:~[position][,[length]]% becomes aborted in case VAR is not defined (so %VAR:~%STR% becomes expanded to ~text when STR is set to text).
It is not ignoring it.
In batch files you need to add %% and not % so it is simply warning you that the substitution is invalid. cmdline still reads comment lines and seeing as you have a valid command in it, but incorrect method, it will warn you.
By doing this, you will not get the warning:
#echo off
REM IF "%%~version_info" == "" echo No version information found
echo test
Why batch interpret comments?
It doesn't interpret comments, but it has to parse lines, that's the problem.
First, the parser reads a line.
Then it expands all percent expressions and then it takes a look at the first token in the line.
If the first token is REM then the remaining stuff will not be interpreted anymore (redirection, delayed expansion, pipes, ampersands, ..., are all ignored)
The problem is, that the parser first expands all percent expresssions, when there is a expression like %~ then the parser throws an error message.
If you do not want to modify the commented code, you can use:
REM %= IF "%~version_info" == "" echo No version information found
Though this is slightly ugly, it will avoid interpreting the %~.
You can find more information on comments in batch in this answer
As to why this is the case, the whole batch thing sounds a bit broken.
The Problem
In a main batch file, values are pulled from a .txt file (and SET as values of variables within this batch file). These values may each contain % characters.
These are read from the .txt file with no issues. However, when a variable with a value containing a % character is passed to a second batch file, the second batch file interprets any % characters as a variable expansion. (Note: There is no control over the second batch file.)
Example
echo %PERCENTVARIABLE%
Output: I%LOVE%PERCENT%CHARACTERS%
When passed to a second file and then echo'ed, would (probably) become IPERCENT, as it interprets %LOVE% and %CHARACTERS% as unset variables.
Research
I found the syntax to find and replace elements within a string in a batch file, as I thought I could potentially replace a % character with %% in order to escape it. However I cannot get it to work.
The syntax is -
set string=This is my string to work with.
set string=%string:work=play%
echo %string%
Where the output would then be This is my string to play with..
Questions
Is it possible to escape % characters using the find and replace syntax
in a variable? (If not, is there another way?)
Is it advisable to do so? (Could using these escape characters cause any issue in the second batch file which (as mentioned above) we would have no control over?)
Is there another way to handle this issue, if the above is not possible?
There are no simple rules that can be applied in all situations.
There are a few issues that make working with string literals in parameters difficult:
Poison characters like &, |, etc. must be escaped or quoted. Escaping is difficult because it can be confusing as to how many times to escape. So the recommendation is to usually quote the string.
Token delimiters like <space>, <tab>, =, ; and , cannot be included in a parameter value unless it is quoted.
A CALL to a script will double any quoted % characters, and there is no way to prevent this. Executing a script without CALL will not double the % characters. But if a script calls another script and expects control to be returned, then CALL must be used.
So we have a catch-22: On the one hand, we want to quote parameters to protect against poison characters and spaces (token delimiters). But to protect percents we don't want to quote.
The only reliable method to reliably pass string literals without concern of value corruption is to pass them by reference via environment variables.
The value to be passed should be stored in an environment value. Quotes and/or escapes and/or percent doubling is used to get the necessary characters in the value, but it is very manageable.
The name of the variable is passed in as a parameter.
The script accesses the value via delayed expansion. For example, if the first parameter is the name of a variable containing the value, then it is accessed as !%1!. Delayed expansion must be enabled before that syntax can be used - simply issue setlocal enableDelayedExpansion.
The beauty of delayed expansion is you never have to worry about corruption of poison characters, spaces, or percents when the variable is expanded.
Here is an example that shows how the following string literal can be passed to a subroutine
"<%|,;^> This & that!" & the other thing! <%|,;^>
#echo off
setlocal enableDelayedExpansion
set "parm1="^<%%^|,;^^^^^> This ^& that^^!" & the other thing^! <%%|,;^^^>"
echo The value before CALL is !parm1!
call :test parm1
exit /b
:test
echo The value after CALL is !%1!
-- OUTPUT --
The value before CALL is "<%|,;^> This & that!" & the other thing! <%|,;^>
The value after CALL is "<%|,;^> This & that!" & the other thing! <%|,;^>
But you state that you have no control over the 2nd called script. So the above elegant solution won't work for you.
If you were to show the code of the 2nd script, and show exactly what value you were trying to pass, then I might be able to give a solution that would work in that isolated situation. But there are some values that simply cannot be passed unless delayed expansion is used with variable names. (Actually, another option is to put the value in a file and read the value from the file, but that also requires change to your 2nd script)
may be...?
input.txt
I%LOVE%PERCENT%CHARACTERS%
batch1.bat
#echo off
setlocal enableDelayedExpansion
set/P var=<input.txt
echo(In batch 1 var content: %var%
set "var=!var:%%=%%%%!"
call batch2.bat "%var%"
endlocal
exit/B
batch2.bat
#echo off
set "var=%~1"
echo(In batch 2 var content: %var%
exit/B
I have a batch file in which the 2 first lines read:
set DIRwhereRUN=C:\UNIVERSITY\testSTABLEunstable(WITHrandomBED)
PUSHD %DIRwhereRUN%
but the batch does not work.
If I create a directory named testSTABLEunstable_WITHrandomBED and copy my stuff there everything works smoothly. Is there a way to make it work with the brackets? I don't want to rename for at least 2 reasons.
It's very difficult - and misleading - to isolate two lines as you have. There's nothing wrong with those two lines.
The difficulty that you are having is with "block" statements like
IF ... (something
something
somethinginvolving DIRWHERERUN
)
This is because batch substitutes the value of any %var% with its the-current value before executing the command(s) and hence misinterprets the ) in %Dirwhererun% as the closing-parenthesis of the IF (or ELSE or FOR.)
The way to overcome this is to "escape" %dirwhererun%'s ) (ie. temporarily suspend its special meaning) - this is done with the caret ^ - which itself is a special character (with the special meaning "the following character is just a character, not a special character".)
So - how to do this?
Here's a demonstration:
#ECHO OFF
setlocal
set "DIRwhereRUN=U:\UNIVERSITY\testSTABLEunstable(WITHrandomBED^)"
SET dir
set DIRwhereRUN=U:\THISWILLBEWRONG(WITHrandomBED^)
SET dir
set DIRwhereRUN=U:\UNIVERSITY\testSTABLEunstable(WITHrandomBED^^)
SET dir
set DIRwhereRUN=U:\UNIVERSITY\testSTABLEunstable(WITHrandomBED)
SET dir
MD %DIRwhereRUN%
PUSHD "%DIRwhereRUN%"
DIR
POPD
SET dirwhererun=%dirwhererun:)=^^)%
SET dir
(Note that I use U: as a temporary drive. I'm creating the directory using your original SET deliberately to show that it's not the SET or normal operations that are causing the problem)
Note that where the set uses quotes around the parameters, the value is applied literally. This form is often used to ensure that stray trailing spaces in lines are not included in the value set into the variable.
Note that the ^ seems ineffectual in the next set - because all it is doing is escaping the ) - and ) is NOT a special character in an ordinary SET
With the third version, the caret is included - but only one, because ^ escapes ^ and ) is an ordinary character.
Then we do all of the operations using the ) unadorned. Obviously attempting to re-create a directory is going to cause an error - but it's because the directory already exists, not because there's anything wrong with the command itself.
As demonstrated, the directory will be listed, so the PUSHD works correctly.
Finally, there's a method of setting a variable dynamically - possibly better set into another variable-name. This is useful where the variable may be read from a file or input from a user - that is, not specified literally.
Well - not quite finally. Two further quirks: First, % is not escaped by ^ but by %, and second, ECHO( appears to be the most flexible form of ECHO (where the character immediately following ECHO, normally space, but may be a number of others) - and doesn't participate in the statement-blocking mechanism.
( and ) are special characters to the shell, so you need to escape them. Cmd.exe's escape character is ^. So you can do the following:
set DIRWHERERUN=C:\UNIVERSITY\testSTABLEunstable^(WITHrandomBED^)
Bill
Use double quotes.
set "DIRwhereRUN=C:\UNIVERSITY\testSTABLEunstable(WITHrandomBED)"
PUSHD "%DIRwhereRUN%"