I am trying to assign the output of the command python --version to a variable in batch script but it does not work.
Here is the sample code I am using
FOR /F "tokens=* USEBACKQ" %%F IN (`python --version`) DO (
SET message=%%F
)
ECHO %message%
Although it prints the version for python, it does not assign to the variable var.
If I use another command instead of python --version then the output is correctly assigned. e.g. using dir works.
What could be wrong?
NOTE -
#echo off
set message=Hello
FOR /F "tokens=* USEBACKQ" %%F IN (`python --version`) DO (
SET message=%%F
)
ECHO %message%
PAUSE
OUTPUT -
C:\Users\nik>extra.bat
Python 2.7.9
Hello
Press any key to continue . . .
The installed version of Python outputs obviously the version information to handle STDERR (standard error) instead of handle STDOUT (standard output). For that reason it is necessary to redirect the standard error stream with the version information to standard output stream using 2>&1 as described by Microsoft in the documentation about Using command redirection operators.
The command line to use is:
for /F "delims=" %%I in ('python.exe --version 2^>^&1') do set "message=%%I"
The redirection operators > and & must be escaped with caret character ^ on FOR command line to be interpreted as literal characters when Windows command interpreter processes this command line before executing command FOR.
FOR respectively cmd.exe processing the batch file with this command line starts in background one more command process with %ComSpec% /c and the command line specified within ' appended as additional arguments. Therefore is executed with Windows installed into C:\Windows in the background:
C:\Windows\System32\cmd.exe /c python.exe --version 2>&1
The command process in background searches now for an executable python.exe as described at What is the reason for "X is not recognized as an internal or external command, operable program or batch file"? When the file python.exe could be found by cmd.exe running in background without a visible console window, it executes it and Python outputs the version information to standard error stream which is redirected to standard output stream of the background command process.
The started cmd.exe closes itself after python.exe terminated itself after printing the version information.
The cmd.exe instance processing the batch file captures everything written to handle STDOUT of started background command process and FOR processes the captured lines after started cmd.exe closed itself.
There is split up by default a non-empty captured line into substrings using normal space and horizontal tab as string delimiters. If the first substring after removal of 0 or more leading spaces/tabs starts with a semicolon, the captured line is ignored, otherwise the first substring is assigned to the specified loop variable I.
The default line handling behavior is not wanted here. For that reason the for /F option delims= defines an empty list of string delimiters to disable the line splitting behavior to get the entire captured line assigned to the specified loop variable I. The implicit default eol=; (semicolon is end of line) can be kept in this case as the version information output by Python starts never with a semicolon.
Related
This is my Windows script which is started every x minutes:
cd C:\
cd Speedtest
speedtest.exe -s 18571 -f csv>>Speedtest.log
How to add a timestamp to every log line?
Let us assume the application speedtest runs just one test run on each execution and outputs one or more lines without empty lines and on each line the current time on starting this application should be written first into the CSV file with , as list separator (default in most countries for CSV files). In this case the following batch file could be used for this task:
#cd /D "C:\Speedtest" 2>nul && #for /F delims^=^ eol^= %%I in ('speedtest.exe -s 18571 -f csv') do >>"Speedtest.log" #echo %TIME%,%%I
The first command CD changes the current directory to C:\Speedtest. This command fails if the directory does not exist. The error message output on changing directory failed is redirected with 2>nul from handle STDERR (standard error) to device NUL to suppress it. There is nothing output by CD if the current directory could be successfully changed to the specified directory.
The operator && results in executing the next command FOR only if CD exited with 0 indicating a successful change of the current directory which means the specified directory C:\Speedtest really exists.
FOR with option /F and a set specified between two ' results in execution of one more command process in background with %ComSpec% /c and the strings between the two ' appended as further arguments. So executed is on Windows installed to C:\Windows:
C:\Windows\System32\cmd.exe /c speedtest.exe -s 18571 -f csv
The started Windows command process executes speedtest.exe with the four specified arguments and then closes itself because of option /c.
FOR captures all output written to handle STDOUT (standard output) and processes it after started cmd.exe terminated itself.
FOR ignores all empty lines on processing the captured lines.
FOR would split up by default each line into substrings using normal space and horizontal tab as string delimiters and would assign just first space/tab separated string of a non-empty line to specified loop variable I. This string splitting behavior is not wanted here. For that reason option delims= is used to define an empty list of string delimiters which disables line splitting behavior.
FOR would ignore by default all lines on which first substring starts with a semicolon because of eol=; is the default for end of line option. It is unknown if speedtest outputs the lines with a semicolon at beginning. For that reason the option eol= is used to define no end of line character which means all lines except empty lines are assigned completely to specified loop variable I.
The two options delims= and eol= are specified usually in a double quoted argument string to get the two equal signs and the space character between interpreted as literal characters and not as argument separators by Windows command processor. But "delims= eol=" cannot be used here as FOR would interpret in this case " as end of line character. Therefore the two options must be written without " which requires escaping the two equal signs and the space character with ^ to be interpreted as literal character instead of argument separators by Windows command processor.
Windows command processor parses the entire line before execution of command CD and replaces during this parsing process %TIME% by the current value of the local time in format defined by the country configured for the account used to run this batch file. So the time written into the CSV file is the time on which the entire command line was parsed by cmd.exe before executing the command CD and on success next the other commands and executables.
The redirection operator >> with the file name Speedtest.log is specified left to command ECHO to be 100% safe that the line is written correct into the file without a trailing space even on ending with number 1 to 9. It can be seen on running this batch file from within a command prompt window without the three # that cmd.exe moved >>"Speedtest.log" to end of the command line with inserting left to this string a space and 1 before executing command CD. It must be always taken into account how a command line looks like after processing it by Windows command processor before executing the command and not how it is written in the batch file.
Note: If the configured country is Germany, Austria, Switzerland, Luxembourg or Liechtenstein with time format HH:mm:ss,ms and list separator ;, it is necessary to use a semicolon instead of a comma between %TIME% and %%I to produce a valid CSV file. speedtest outputs the data in this case hopefully also with ; instead of ,. Otherwise if speedtest outputs the data independent on list separator of the configured country always with a comma, it would be necessary to use on command ECHO either "%TIME%",%%I or %TIME:,=.%,%%I to get a valid CSV file using comma as separator.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
cd /?
echo /?
for /?
set /?
See also:
How does the Windows Command Interpreter (CMD.EXE) parse scripts?
Single line with multiple commands using Windows batch file
Microsoft article about Using command redirection operators
There's something I'm not understanding about the way that the batch for command works, when used with /f and a command.
I was trying to loop on the output of an executable. The executable turned out to not exist, which is fine.
But instead of getting the expected error about a bad path, the script seemed to spontaneously incorrectly tokenize the string. This led me down a rabbit hole of thinking that I had formatted the for loop and/or used quotes or back-ticks incorrectly.
Placing an executable at the location fixes the issue, making it appear like the path string tokenizing is dependent on the existence of an actual file at that path.
Why does the following batch file
#echo off
for /f "usebackq delims=" %%i in ( `"C:\Program Files\BOGUS_PATH\FAKE.exe"`) do (
rem
)
output 'C:\Program' is not recognized as an internal or external command, operable program or batch file. instead of The system cannot find the path specified. ?
First off, a pet peeve of mine, the usebackq is not needed. The simpler and functionally equivalent command is
#echo off
for /f "delims=" %%i in ('"C:\Program Files\BOGUS_PATH\FAKE.exe"') do (
rem
)
The only time usebackq is required is when you are trying to read a file, and the file path contains token delimiters like space.
for /f "usebackq" %%A in ('"some file.txt"')
All other situations can use the "normal" forms
for /f %%A in (fileName) do...
for /f %%A in ("string") do...
for /f %%A in ('someCommand') do...
Now to answer your actual question :-)
The presence or absence of the file does not actually alter the parsing.
First you need to understand the possible error messages when you try to execute a non-existent program from the command line. There are three possibilities:
1) If the "command" includes a colon anywhere other than the 2nd position, then you may get the following
c:\test\>abc:fake.exe
The filename, directory name, or volume label syntax is incorrect.
c:\test\>abc:\test\fake.exe
The filename, directory name, or volume label syntax is incorrect.
Or sometimes you get the next possible error. The rules as to which message you get are not obvious to me, and I don't feel like it is important enough to figure out the exact rules.
2) If the "command" includes a backslash that indicates a path, and the path is not valid, then you get
c:\test\>c:bogus\fake.exe
The system cannot find the path specified.
c:\test\>\bogus\fake.exe
The system cannot find the path specified.
c:\test\>abc:bogus\fake.exe
The system cannot find the path specified.
3) If the executable file cannot be found, and either the "command" does not include path info or the provided path is valid, then you get
C:\test\>fake.exe
'fake.exe' is not recognized as an internal or external command,
operable program or batch file.
C:\test>c:\test\fake.exe
'c:\test\fake.exe' is not recognized as an internal or external command,
operable program or batch file.
The last conundrum is why "C:\Program Files\BOGUS_PATH\FAKE.exe" gives error 3) instead of 2)
If you execute from the command line with quotes then you get the expected result:
C:\test>"C:\Program Files\BOGUS_PATH\FAKE.exe"
The system cannot find the path specified.
If you execute from the command line without quotes, then you get this expected result:
C:\test>C:\Program Files\BOGUS_PATH\FAKE.exe
'C:\Program' is not recognized as an internal or external command,
operable program or batch file.
Your script includes quotes, so you would expect the former, but you get the latter.
There are two parts to understanding this.
First off, FOR /F executes the command by executing
C:\WINDOWS\system32\cmd.exe /c "C:\Program Files\BOGUS_PATH\FAKE.exe"
You might think your "command" is quoted, but cmd.exe plays games with quotes, which is the 2nd part to the explanation. The cmd.exe quote rules are described in the help:
c:\test\>help cmd
Starts a new instance of the Windows command interpreter
...
If /C or /K is specified, then the remainder of the command line after
the switch is processed as a command line, where the following logic is
used to process quote (") characters:
1. If all of the following conditions are met, then quote characters
on the command line are preserved:
- no /S switch
- exactly two quote characters
- no special characters between the two quote characters,
where special is one of: &<>()#^|
- there are one or more whitespace characters between the
two quote characters
- the string between the two quote characters is the name
of an executable file.
2. Otherwise, old behavior is to see if the first character is
a quote character and if so, strip the leading character and
remove the last quote character on the command line, preserving
any text after the last quote character.
Most of the conditions under 1. are met except for the last one - the string does not point to a valid executable. So the 2. rules are used, thus the command becomes
C:\Program Files\BOGUS_PATH\FAKE.exe
And now the result makes perfect sense - the "command" breaks at the space.
'C:\Program' is not recognized as an internal or external command,
operable program or batch file.
If you are trying to execute a command with FOR /F, and the command must be enclosed within quotes, then you must put an extra set of quotes around the entire command.
for /f "delims=" %%A in ('""c:\some path\file.exe" "arg 1" arg2"') do ...
But the above will have problems if there are poison characters in the path or quoted argument, because the extra set of quotes mess up the quoting. You could escape the poison characters like
for /f "delims=" %%A in ('""c:\some path\file.exe" "this^&that" arg2"') do ...
But I find that awkward because the command you use on the command line does not match the command you use in a FOR /F. I prefer to escape the extra set of enclosing quotes so that whatever works on the command line also works within FOR /F.
for /f "delims=" %%A in ('^""c:\some path\file.exe" "this&that" arg2^"') do ...
FOR is an internal command of Windows command processor cmd.exe. The usage of command FOR with option /F and a command to execute results in starting with %ComSpec% /c and the specified command line as further parameter(s) one more command process running in background.
Everything output to handle STDOUT is captured by current command process executing the batch file with command FOR and is processed by FOR after termination of additionally started cmd.exe line by line.
The output to handle STDERR of additionally started command process is redirected to handle STDERR of current command process resulting in getting it displayed in console window in case of STDERR is neither in additionally started command process nor in current command process redirected to a different handle, file or device NUL.
This can be seen on using free Sysinternals (Microsoft) tool Process Monitor.
Download the ZIP file containing Process Monitor.
Extract the ZIP file into any local directory.
Start Procmon.exe with Run as administrator which is required to run Process Monitor.
In Process Monitor Filter dialog displayed first Add the entry Process Name is cmd.exe and close the dialog with button OK.
Toggle off the last five options on toolbar with the exception of the filling cabinet symbol with the tooltip Show File System Activity.
Press Ctrl+X to clear the list.
Run the batch file.
Switch back to Process Monitor and press Ctrl+E to stop capturing.
Make sure to see among the column Process Name also the column PID. Right click on list header and left click on context menu item Select Columns... if the PID column is not already displayed, check Process ID and close dialog window with OK. I recommend to move the column PID using drag and drop of column header PID right to column Process Name.
Scroll up to begin of list of captured file system activities and look for the line where a cmd.exe with a different process identifier is displayed than the first one. That second cmd.exe with different PID is the Windows command processor instance started by FOR.
Right click on this second cmd.exe, left click in context menu on Properties, select tab Process and look on Command Line showing:
C:\Windows\system32\cmd.exe /c "C:\Program Files\BOGUS_PATH\FAKE.exe"
Okay. So we know now what FOR respectively cmd.exe does on running a command to capture the output of this command. Close the properties window.
Look on list and it can be seen which file system accesses are made by second cmd.exe to find the specified executable which does not exist in specified and existing directory. It searches next for C:\Program Files\BOGUS_PATH\FAKE.exe.* also with no such file as result.
The first argument string is split up now on argument separators to which belongs the space character among some others like comma or equal sign. So the first argument string C:\Program Files\BOGUS_PATH\FAKE.exe is argument separated once more resulting in interpreting C:\Program now as first argument string.
cmd.exe checks if the directory path being now just C:\ is existing which is true like it was before for directory path C:\Program Files\BOGUS_PATH.
Next is checked if there is a file matching the pattern C:\Program.* which returns no such file. Then cmd.exe checks if there is the file C:\Program which again results in no such file.
Now second cmd.exe gives up finding a file which could be executed and outputs the error message.
It is indeed confusing on having specified the executable with full path enclosed in double quotes as required because of the space to get not output
'C:\Program Files\BOGUS_PATH\FAKE.exe' is not recognized as an internal or external command
operable program or batch file.
This would be expected, but output is the error message:
'C:\Program' is not recognized as an internal or external command
operable program or batch file.
I think, parsing the first argument string once again using the argument separators on not finding a file to execute is done for example if a user runs cmd.exe with the command (argument)
"C:\Programs\MyApplication.exe C:\Temp\FileToProcess.txt"
instead of
"C:\Programs\MyApplication.exe" "C:\Temp\FileToProcess.txt"
So the question is:
Is this automatic argument parsing good or not good?
Well, this is difficult to answer. It is at least somehow documented in help of cmd.exe as described by dbenham in his answer.
What you probably need to do is to enclose your executable command in doublequotes:
#Echo Off
For /F Delims^=^ EOL^= %%A In ('""C:\Program Files\BOGUS_PATH\FAKE.exe""'
)Do Echo(%%A
Pause
Where the inner set of doublequotes is to protect the known space in the path, and the outer set to protect the inner ones when the content is passed to the cmd.exe instance, that the parenthesised command is run in.
Then to prevent any error message from the command, you should redirect the STDERR output to NUL:
#Echo Off
For /F Delims^=^ EOL^= %%A In ('""C:\Program Files\BOGUS_PATH\FAKE.exe" 2>Nul"'
)Do Echo(%%A
Pause
If the executable does not exist, this will supress the error message, The system cannot find the path specified. and the Do command will not run, (as there is no STDOUT to process).
If I understand correctly this batch script
for /f "usebackq tokens=*" %%i in (`time /t`) do (
echo "%%i"
)
echo "done"
should just print the time and then "done"
Instead though on my machine, Windows 10 Pro x64 it prints
>test.bat
>for /F "usebackq tokens=*" %i in (`time /t`) do (echo "%i" )
>(echo "Active code page: 65001" )
"Active code page: 65001"
>(echo "17:48" )
"17:48"
>echo "done"
"done"
Why is it doing this and how do I fix it so it just gets the time and not this "Active code page: 65001" result. This is breaking build scripts as it doesn't matter what command goes in the FOR command the first line is always "Active code page: 65001"
To be clear I'm looking for an OS setting fix, not a fix to the batch file. The real batch files I'm trying to run are from build scripts in open source projects and they are failing because of this issue.
When we run a command and we get the correct output, but it is preceded by the output of another command, somehow this unexpected command is run before our intended command.
The usual suspect is a batch file created with the same name that the command we are calling, but in this case, being time an internal command, this can not be the case, internal commands have precedence over external commands (default behaviour).
If time is an internal command, how can another command being called?
When for /f needs to process the output of a command a separate cmd instance is created to execute that command. The output generated in this separate instance is processed by the code in the do clause of the for /f command.
The creation of this separate instance can be the source of the unexpected output.
Reading the help information shown by cmd /? we can see that there are two registry keys
HKEY_LOCAL_MACHINE\Software\Microsoft\Command Processor\AutoRun
HKEY_CURRENT_USER\Software\Microsoft\Command Processor\AutoRun
that can be configured to run commands every time a new cmd instance is created (unless the /D switch is used when starting the cmd instance).
This is the most probable place where to find why you get the Active code page: 65001 output.
Instead of changing the registry you can control the code page simply by creating a file %HOMEPATH%\init.cmd.
Mine says:
#ECHO OFF
CHCP 65001 > nul
You need to suppress the output from CHCP to avoid the undesired output.
I am trying to create a list of files (about 4000) with batch file but without the extensions, some of the file names have - or _ or spaces which I would like to keep them the same, and then I am going to copy and paste the list inside Excel and run a macro that I found on this website to create individual files with different extension using the names on that list. I hope I make sense.
Here is my attempt so far:
for /f %%a in ('dir /b *.dsg') do #echo %%~na >txt1.txt
But this one just creates a list with only one name.
The batch code for this simple task is:
#echo off
del txt1.txt 2>nul
for /F "delims=" %%I in ('dir *.dsg /A-D /B 2^>nul') do echo %%~nI>>txt1.txt
The command DEL is used to delete a probably already existing file txt1.txt with redirecting the error message output to handle STDERR to device NUL to suppress it in case of the file does not exist in current directory.
Next the command DIR is executed by FOR in a separate command process. DIR lists only the names of all files matching the pattern *.dsg because of /A-D (not directory attribute) in bare format because of /B.
The error message output by DIR in case of no *.dsg file in current directory is suppressed by redirecting it to device NUL. The redirection operator > must be escaped here with caret character ^ to be interpreted as literal character when Windows command interpreter parses the FOR command line. Later on execution of DIR command line > is interpreted as redirection operator.
The list output by DIR is parsed by FOR line by line. The option delims= disables splitting up each line using spaces/tabs as delimiter. So assigned to loop variable I is always the entire line read from output of DIR.
Output by ECHO to handle STDOUT is the string left of last dot, i.e. the file name without file extension. This output is redirected to a file with appending the line to already existing file contents.
A space between %%~nI and redirection operator >> would be also output by ECHO and therefore also written as trailing space into the file.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
del /?
echo /?
for /?
Read also the Microsoft article about Using Command Redirection Operators.
#user8033911, based upon your latest comment, there seems no reason why you cannot replace the file extension with another as part of the same process.
The code below entered directly in the Command prompt window should show you, it replaces, .dsg with .new.
For /F "EOL=/Delims=/" %A In ('Where/F .:*.dsg') Do #Echo %A to "%~dpnA.new"
If you still cannot understand just comment below explaining where you are having difficulty.
What am I missing?
#echo off
rem - processfiles.bat - Processes all text files in the "source" folder:
rem - Runs %executable% with each text file as parameter
rem - Move the text file to the target folder after processing
set SourceFolder=C:\Users\Chris\Desktop\Yield_Files
set TargetFolder=C:\Users\Chris\Desktop\YieldCleanFiles
set fin=default.set
if not exist "%TargetFolder%" md "%TargetFolder%"
echo Processing text files in %SourceFolder%:
for %%f in (%SourceFolder%\*.txt) do call "C:\Program Files\Yield_Editor\yieldeditor.exe/csvin="(%SourceFolder%\*.txt)"/auto="y"/hide/fin=%fin%"%%f
pause
I have to have the file name I am working on each time I call the .exe
when ran it says it cannot find the file specified, but I am not sure which one it is talking about.
The line
for %%f in (%SourceFolder%\*.txt) do call "C:\Program Files\Yield_Editor\yieldeditor.exe/csvin="(%SourceFolder%\*.txt)"/auto="y"/hide/fin=%fin%"%%f
must be written most likely
for %%f in ("%SourceFolder%\*.txt") do "C:\Program Files\Yield_Editor\yieldeditor.exe" /csvin="%SourceFolder%\*.txt" /auto=y /hide /fin=%fin% "%%~f"
or
for %%f in ("%SourceFolder%\*.txt") do "%ProgramFiles%\Yield_Editor\yieldeditor.exe" "/csvin=%SourceFolder%\*.txt" /auto=y /hide /fin=%fin% "%%~f"
which means your executable must be run with syntax
"Path to Application\Application.exe" "/First Option" /Option2 /Option3 /Option4 "File Name As Fifth Parameter"
File name of application with file extension and path must be in quotes if there is at least 1 space or another character with a special meaning which are output in a command prompt window on last help page displayed when running in command prompt window cmd /?. This is argument 0 for the application.
The first parameter/option is argument 1 for the application. It must be in quotes if the parameter string contains a space or another special character. Options not containing a space must not be quoted, but can be nevertheless also quoted.
Some applications support quoting inside an option with a variable string. An example for such a syntax would be /csvin="%SourceFolder%\*.txt".
Read the help/documentation of the application for details on using it from command line. Most Windows console applications can be executed from within a command prompt window with just /? as parameter for printing command line help to console.
It can be seen here why argument strings with spaces must be enclosed in quotes. The space is the separator for the arguments.