Does Windows batch support exception handling? - batch-file

Does Windows batch programming support exception handling? If not, is there any way to effectively emulate exception handling within batch files?
I would like to be able to "throw an exception" anywhere within a batch script, at any CALL level, and have the CALL stack popped repeatedly until it finds an active "TRY block", whereupon a "CATCH block" can handle the exception fully and carry on, or do some cleanup and continue popping the CALL stack. If the exception is never handled, then batch processing is terminated and control returns to the command line context with an error message.
There are already couple posted ways to terminate batch processing at any CALL depth, but none of those techniques allow for any structured cleanup activity that would normally be provided within other languages via exception handling.
Note: This is a case where I already know a good answer that has only recently been discovered, and I want to share the info

Windows batch scripting certainly does not have any formal exception handling - hardly surprising considering how primitive the language is. Never in my wildest dreams did I ever think effective exception handling could be hacked up.
But then some amazing discoveries were made on a Russian site concerning the behavior of an erroneous GOTO statement (I have no idea what is said, I can't read Russian). An English summary was posted at DosTips, and the behavior was further investigated.
It turns out that (GOTO) 2>NUL behaves almost identically to EXIT /B, except concatenated commands within an already parsed block of code are still executed after the effective return, within the context of the CALLer!
Here is a short example that demonstrates most of the salient points.
#echo off
setlocal enableDelayedExpansion
set "var=Parent Value"
(
call :test
echo This and the following line are not executed
exit /b
)
:break
echo How did I get here^^!^^!^^!^^!
exit /b
:test
setlocal disableDelayedExpansion
set "var=Child Value"
(goto) 2>nul & echo var=!var! & goto :break
echo This line is not executed
:break
echo This line is not executed
-- OUTPUT --
var=Parent Value
How did I get here!!!!
This feature is totally unexpected, and incredibly powerful and useful. It has been used to:
Create PrintHere.bat - an emulation of the 'nix here document feature
Create a RETURN.BAT utility that any batch "function" can conveniently CALL to return any value across the ENDLOCAL barrier, with virtually no limitations. The code is a fleshed out version of jeb's original idea.
Now I can also add exception handling to the list :-)
The technique relies on a batch utility called EXCEPTION.BAT to define environment variable "macros" that are used to specify TRY/CATCH blocks, as well as to throw exceptions.
Before a TRY/CATCH block can be implemented, the macros must be defined using:
call exception init
Then TRY/CATCH blocks are defined with the following syntax:
:calledRoutine
setlocal
%#Try%
REM normal code goes here
%#EndTry%
:#Catch
REM Exception handling code goes here
:#EndCatch
Exceptions can be thrown at any time via:
call exception throw errorNumber "messageString" "locationString"
When an exception is thrown, it pops the CALL stack iteratively using (GOTO) 2>NUL until it finds an active TRY/CATCH, whereupon it branches to the CATCH block and executes that code. A series of exception attribute variables are available to the CATCH block:
exception.Code - The numeric exception code
exception.Msg - The exception message string
exception.Loc - The string describing the location where the exception was thrown
exception.Stack - A string that traces the call stack from the CATCH block (or command line if not caught), all the way to the exception origin.
If the exception is fully handled, then the exception should be cleared via call exception clear, and the script carries on normally. If the exception is not fully handled, then a new exception can be thrown with a brand new exception.Stack, or the old stack can be preserved with
call exception rethrow errorNumber "messageString" "locationString"
If an exception is not handled, then an "unhandled exception" message is printed, including the four exception attributes, all batch processing is terminated, and control is returned to the command line context.
Here is the code that makes all this possible - full documentation is embedded within the script and available from the command line via exception help or exception /?.
EXCEPTION.BAT
::EXCEPTION.BAT Version 1.4
::
:: Provides exception handling for Windows batch scripts.
::
:: Designed and written by Dave Benham, with important contributions from
:: DosTips users jeb and siberia-man
::
:: Full documentation is at the bottom of this script
::
:: History:
:: v1.4 2016-08-16 Improved detection of command line delayed expansion
:: using an original idea by jeb
:: v1.3 2015-12-12 Added paged help option via MORE
:: v1.2 2015-07-16 Use COMSPEC instead of OS to detect delayed expansion
:: v1.1 2015-07-03 Preserve ! in exception attributes when delayed expansion enabled
:: v1.0 2015-06-26 Initial versioned release with embedded documentation
::
#echo off
if "%~1" equ "/??" goto pagedHelp
if "%~1" equ "/?" goto help
if "%~1" equ "" goto help
shift /1 & goto %1
:throw errCode errMsg errLoc
set "exception.Stack="
:: Fall through to :rethrow
:rethrow errCode errMsg errLoc
setlocal disableDelayedExpansion
if not defined exception.Restart set "exception.Stack=[%~1:%~2] %exception.Stack%"
for /f "delims=" %%1 in ("%~1") do for /f "delims=" %%2 in ("%~2") do for /f "delims=" %%3 in ("%~3") do (
setlocal enableDelayedExpansion
for /l %%# in (1 1 10) do for /f "delims=" %%S in (" !exception.Stack!") do (
(goto) 2>NUL
setlocal enableDelayedExpansion
if "!!" equ "" (
endlocal
setlocal disableDelayedExpansion
call set "funcName=%%~0"
call set "batName=%%~f0"
if defined exception.Restart (set "exception.Restart=") else call set "exception.Stack=%%funcName%%%%S"
setlocal EnableDelayedExpansion
if !exception.Try! == !batName!:!funcName! (
endlocal
endlocal
set "exception.Code=%%1"
if "!!" equ "" (
call "%~f0" setDelayed
) else (
set "exception.Msg=%%2"
set "exception.Loc=%%3"
set "exception.Stack=%%S"
)
set "exception.Try="
(CALL )
goto :#Catch
)
) else (
for %%V in (Code Msg Loc Stack Try Restart) do set "exception.%%V="
if "^!^" equ "^!" (
call "%~f0" showDelayed
) else (
echo(
echo Unhandled batch exception:
echo Code = %%1
echo Msg = %%2
echo Loc = %%3
echo Stack=%%S
)
echo on
call "%~f0" Kill
)>&2
)
set exception.Restart=1
setlocal disableDelayedExpansion
call "%~f0" rethrow %1 %2 %3
)
:: Never reaches here
:init
set "#Try=call set exception.Try=%%~f0:%%~0"
set "#EndTry=set "exception.Try=" & goto :#endCatch"
:: Fall through to :clear
:clear
for %%V in (Code Msg Loc Stack Restart Try) do set "exception.%%V="
exit /b
:Kill - Cease all processing, ignoring any remaining cached commands
setlocal disableDelayedExpansion
if not exist "%temp%\Kill.Yes" call :buildYes
call :CtrlC <"%temp%\Kill.Yes" 1>nul 2>&1
:CtrlC
#cmd /c exit -1073741510
:buildYes - Establish a Yes file for the language used by the OS
pushd "%temp%"
set "yes="
copy nul Kill.Yes >nul
for /f "delims=(/ tokens=2" %%Y in (
'"copy /-y nul Kill.Yes <nul"'
) do if not defined yes set "yes=%%Y"
echo %yes%>Kill.Yes
popd
exit /b
:setDelayed
setLocal disableDelayedExpansion
for %%. in (.) do (
set "v2=%%2"
set "v3=%%3"
set "vS=%%S"
)
(
endlocal
set "exception.Msg=%v2:!=^!%"
set "exception.Loc=%v3:!=^!%"
set "exception.Stack=%vS:!=^!%"
)
exit /b
:showDelayed -
setLocal disableDelayedExpansion
for %%. in (.) do (
set "v2=%%2"
set "v3=%%3"
set "vS=%%S"
)
for /f "delims=" %%2 in ("%v2:!=^!%") do for /f "delims=" %%3 in ("%v3:!=^!%") do for /f "delims=" %%S in ("%vS:!=^!%") do (
endlocal
echo(
echo Unhandled batch exception:
echo Code = %%1
echo Msg = %%2
echo Loc = %%3
echo Stack=%%S
)
exit /b
:-?
:help
setlocal disableDelayedExpansion
for /f "delims=:" %%N in ('findstr /rbn ":::DOCUMENTATION:::" "%~f0"') do set "skip=%%N"
for /f "skip=%skip% tokens=1* delims=:" %%A in ('findstr /n "^" "%~f0"') do echo(%%B
exit /b
:-??
:pagedHelp
setlocal disableDelayedExpansion
for /f "delims=:" %%N in ('findstr /rbn ":::DOCUMENTATION:::" "%~f0"') do set "skip=%%N"
((for /f "skip=%skip% tokens=1* delims=:" %%A in ('findstr /n "^" "%~f0"') do #echo(%%B)|more /e) 2>nul
exit /b
:-v
:/v
:version
echo(
for /f "delims=:" %%A in ('findstr "^::EXCEPTION.BAT" "%~f0"') do echo %%A
exit /b
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:::DOCUMENTATION:::
EXCEPTION.BAT is a pure batch script utility that provides robust exception
handling within batch scripts. It enables code to be placed in TRY/CATCH blocks.
If no exception is thrown, then only code within the TRY block is executed.
If an exception is thrown, the batch CALL stack is popped repeatedly until it
reaches an active TRY block, at which point control is passed to the associated
CATCH block and normal processing resumes from that point. Code within a CATCH
block is ignored unless an exception is thrown.
An exception may be caught in a different script from where it was thrown.
If no active TRY is found after throwing an exception, then an unhandled
exception message is printed to stderr, all processing is terminated within the
current CMD shell, and control is returned to the shell command line.
TRY blocks are specified using macros. Obviously the macros must be defined
before they can be used. The TRY macros are defined using the following CALL
call exception init
Besides defining #Try and #EndTry, the init routine also explicitly clears any
residual exception that may have been left by prior processing.
A TRY/CATCH block is structured as follows:
%#Try%
REM any normal code goes here
%#EndTry%
:#Catch
REM exception handling code goes here
:#EndCatch
- Every TRY must have an associated CATCH.
- TRY/CATCH blocks cannot be nested.
- Any script or :labeled routine that uses TRY/CATCH must have at least one
SETLOCAL prior to the appearance of the first TRY.
- TRY/CATCH blocks use labels, so they should not be placed within parentheses.
It can be done, but the parentheses block is broken when control is passed to
the :#Catch or :#EndCatch label, and the code becomes difficult to interpret
and maintain.
- Any valid code can be used within a TRY or CATCH block, including CALL, GOTO,
:labels, and balanced parentheses. However, GOTO cannot be used to leave a
TRY block. GOTO can only be used within a TRY block if the label appears
within the same TRY block.
- GOTO must never transfer control from outside TRY/CATCH to within a TRY or
CATCH block.
- CALL should not be used to call a label within a TRY or CATCH block.
- CALLed routines containing TRY/CATCH must have labels that are unique within
the script. This is generally good batch programming practice anyway.
It is OK for different scripts to share :label names.
- If a script or routine recursively CALLs itself and contains TRY/CATCH, then
it must not throw an exception until after execution of the first %#Try%
Exceptions are thrown by using
call exception throw Code Message Location
where
Code = The numeric code value for the exception.
Message = A description of the exception.
Location = A string that helps identify where the exception occurred.
Any value may be used. A good generic value is "%~f0[%~0]",
which expands to the full path of the currently executing
script, followed by the currently executing routine name
within square brackets.
The Message and Location values must be quoted if they contain spaces or poison
characters like & | < >. The values must not contain additional internal quotes,
and they must not contain a caret ^.
The following variables will be defined for use by the CATCH block:
exception.Code = the Code value
exception.Msg = the Message value
exception.Loc = the Location value
exception.Stack = traces the call stack from the CATCH block (or command line
if not caught), all the way to the exception.
If the exception is not caught, then all four values are printed as part of the
"unhandled exception" message, and the exception variables are not defined.
A CATCH block should always do ONE of the following at the end:
- If the exception has been handled and processing can continue, then clear the
exception definition by using
call exception clear
Clear should never be used within a Try block.
- If the exception has not been fully handled, then a new exception should be
thrown which can be caught by a higher level CATCH. You can throw a new
exception using the normal THROW, which will clear exception.Stack and any
higher CATCH will have no awareness of the original exception.
Alternatively, you may rethrow an exception and preserve the exeption stack
all the way to the original exception:
call exception rethrow Code Message Location
It is your choice as to whether you want to pass the original Code and/or
Message and/or Location. Either way, the stack will preserve all exceptions
if rethrow is used.
Rethrow should only be used within a CATCH block.
One last restriction - the full path to EXCEPTION.BAT must not include ! or ^.
This documentation can be accessed via the following commands
constant stream: exception /? OR exception help
paged via MORE: exception /?? OR exception pagedHelp
The version of this utility can be accessed via
exception /v OR exception version
EXCEPTION.BAT was designed and written by Dave Benham, with important
contributions from DosTips users jeb and siberia-man.
Development history can be traced at:
http://www.dostips.com/forum/viewtopic.php?f=3&t=6497
Below is script to test the capabilities of EXCEPTION.BAT. The script recursively calls itself 7 times. Each iteration has two CALLs, one to a :label that demonstrates normal exception propagation, and the other to a script that demonstrates exception propagation across script CALLs.
While returning from a recursive call, it throws an exception if the iteration count is a multiple of 3 (iterations 3 and 6).
Each CALL has its own exception handler that normally reports the exception and then rethrows a modified exception. But if the iteration count is 5, then the exception is handled and normal processing resumes.
#echo off
:: Main
setlocal enableDelayedExpansion
if not defined #Try call exception init
set /a cnt+=1
echo Main Iteration %cnt% - Calling :Sub
%#Try%
(
call :Sub
call echo Main Iteration %cnt% - :Sub returned %%errorlevel%%
)
%#EndTry%
:#Catch
setlocal enableDelayedExpansion
echo(
echo Main Iteration %cnt% - Exception detected:
echo Code = !exception.code!
echo Message = !exception.msg!
echo Location = !exception.loc!
echo Rethrowing modified exception
echo(
endlocal
call exception rethrow -%cnt% "Main Exception^!" "%~f0<%~0>"
:#EndCatch
echo Main Iteration %cnt% - Exit
exit /b %cnt%
:Sub
setlocal
echo :Sub Iteration %cnt% - Start
%#Try%
if %cnt% lss 7 (
echo :Sub Iteration %cnt% - Calling "%~f0"
call "%~f0"
%= Show any non-exception return code (demonstrate ERRORLEVEL is preserved if no exception) =%
call echo :Sub Iteration %cnt% - testException returned %%errorlevel%%
)
%= Throw an exception if the iteration count is a multiple of 3 =%
set /a "1/(cnt%%3)" 2>nul || (
echo Throwing exception
call exception throw -%cnt% "Divide by 0 exception^!" "%~f0<%~0>"
)
%#EndTry%
:#Catch
setlocal enableDelayedExpansion
echo(
echo :Sub Iteration %cnt% - Exception detected:
echo Code = !exception.code!
echo Message = !exception.msg!
echo Location = !exception.loc!
endlocal
%= Handle the exception if iteration count is a multiple of 5, else rethrow it with new properties =%
set /a "1/(cnt%%5)" 2>nul && (
echo Rethrowing modified exception
echo(
call exception rethrow -%cnt% ":Sub Exception^!" "%~f0<%~0>"
) || (
call exception clear
echo Exception handled
echo(
)
:#EndCatch
echo :Sub Iteration %cnt% - Exit
exit /b %cnt%
-- OUTPUT --
Main Iteration 1 - Calling :Sub
:Sub Iteration 1 - Start
:Sub Iteration 1 - Calling "C:\test\testException.bat"
Main Iteration 2 - Calling :Sub
:Sub Iteration 2 - Start
:Sub Iteration 2 - Calling "C:\test\testException.bat"
Main Iteration 3 - Calling :Sub
:Sub Iteration 3 - Start
:Sub Iteration 3 - Calling "C:\test\testException.bat"
Main Iteration 4 - Calling :Sub
:Sub Iteration 4 - Start
:Sub Iteration 4 - Calling "C:\test\testException.bat"
Main Iteration 5 - Calling :Sub
:Sub Iteration 5 - Start
:Sub Iteration 5 - Calling "C:\test\testException.bat"
Main Iteration 6 - Calling :Sub
:Sub Iteration 6 - Start
:Sub Iteration 6 - Calling "C:\test\testException.bat"
Main Iteration 7 - Calling :Sub
:Sub Iteration 7 - Start
:Sub Iteration 7 - Exit
Main Iteration 7 - :Sub returned 7
Main Iteration 7 - Exit
:Sub Iteration 6 - testException returned 7
Throwing exception
:Sub Iteration 6 - Exception detected:
Code = -6
Message = Divide by 0 exception!
Location = C:\test\testException.bat<:Sub>
Rethrowing modified exception
Main Iteration 6 - Exception detected:
Code = -6
Message = :Sub Exception!
Location = C:\test\testException.bat<:Sub>
Rethrowing modified exception
:Sub Iteration 5 - Exception detected:
Code = -6
Message = Main Exception!
Location = C:\test\testException.bat<C:\test\testException.bat>
Exception handled
:Sub Iteration 5 - Exit
Main Iteration 5 - :Sub returned 5
Main Iteration 5 - Exit
:Sub Iteration 4 - testException returned 5
:Sub Iteration 4 - Exit
Main Iteration 4 - :Sub returned 4
Main Iteration 4 - Exit
:Sub Iteration 3 - testException returned 4
Throwing exception
:Sub Iteration 3 - Exception detected:
Code = -3
Message = Divide by 0 exception!
Location = C:\test\testException.bat<:Sub>
Rethrowing modified exception
Main Iteration 3 - Exception detected:
Code = -3
Message = :Sub Exception!
Location = C:\test\testException.bat<:Sub>
Rethrowing modified exception
:Sub Iteration 2 - Exception detected:
Code = -3
Message = Main Exception!
Location = C:\test\testException.bat<C:\test\testException.bat>
Rethrowing modified exception
Main Iteration 2 - Exception detected:
Code = -2
Message = :Sub Exception!
Location = C:\test\testException.bat<:Sub>
Rethrowing modified exception
:Sub Iteration 1 - Exception detected:
Code = -2
Message = Main Exception!
Location = C:\test\testException.bat<C:\test\testException.bat>
Rethrowing modified exception
Main Iteration 1 - Exception detected:
Code = -1
Message = :Sub Exception!
Location = C:\test\testException.bat<:Sub>
Rethrowing modified exception
Unhandled batch exception:
Code = -1
Msg = Main Exception!
Loc = C:\test\testException.bat<testException>
Stack= testException [-1:Main Exception!] :Sub [-1::Sub Exception!] C:\test\testException.bat [-2:Main Exception!] :Sub [-2::Sub Exception!] C:\test\testException.bat [-3:Main Exception!] :Sub [-3::Sub Exception!] [-3:Divide by 0 exception!]
Finally, here are a series of trivial scripts that show how exceptions can be used effectively even when intermediate scripts know nothing about them!
Start off with a simple division script utility that divides two numbers and prints the result:
divide.bat
:: divide.bat numerator divisor
#echo off
setlocal
set /a result=%1 / %2 2>nul || call exception throw -100 "Division exception" "divide.bat"
echo %1 / %2 = %result%
exit /b
Note how the script throws an exception if it detects an error, but it does nothing to catch the exception.
Now I'll write a divide test harness that is totally naive about batch exceptions.
testDivide.bat
#echo off
for /l %%N in (4 -1 0) do call divide 12 %%N
echo Finished successfully!
--OUTPUT--
C:\test>testDivide
12 / 4 = 3
12 / 3 = 4
12 / 2 = 6
12 / 1 = 12
Unhandled batch exception:
Code = -100
Msg = Division exception
Loc = divide.bat
Stack= testDivide divide [-100:Division exception]
Note how the final ECHO never executes because the exception raised by divide.bat was not handled.
Finally I'll write a master script that calls the naive testDivide and properly handles the exception:
master.bat
#echo off
setlocal
call exception init
%#Try%
call testDivide
%#EndTry%
:#Catch
echo %exception.Msg% detected and handled
call exception clear
:#EndCatch
echo Finished Successfully!
-- OUTPUT --
C:\test>master
12 / 4 = 3
12 / 3 = 4
12 / 2 = 6
12 / 1 = 12
Division exception detected and handled
Finished Successfully!
The master script was able to successfully catch an exception raised by divide.bat, even though it had to pass through testDivide.bat, which knows nothing about exceptions. Very cool :-)
Now this is certainly not a panacea for all things related to error handling:
There are a number of syntactical and code layout limitations that are fully described in the built in documentation. But nothing too egregious.
There is no way to automatically treat all errors as an exceptions. All exceptions must be explicitly thrown by code. This is probably a good thing, given that error reporting is handled by convention - there are no strict rules. Some programs do not follow the convention. For example, HELP ValidCommand returns ERRORLEVEL 1, which by convention implies an error, while HELP InvalidCommand returns ERRORLEVEL 0, which implies success.
This batch exception technique cannot catch and handle fatal run-time errors. For example GOTO :NonExistentLabel will still immediately terminate all batch processing, without any opportunity to catch the error.
You can follow the development of EXCEPTION.BAT at http://www.dostips.com/forum/viewtopic.php?f=3&t=6497. Any future developments will be posted there. I likely will not update this StackOverflow post.

Well, if the term "exception handling" is taken in the same sense of other programming languages, I think the answer is: "NO".
In all standard programming languages the term "exception handling" refers to "the occurrence, during computation, of exceptions – anomalous or exceptional conditions requiring special processing", like run-time errors that may be managed in a way different than the standard error handling performed by the system.
For example, in C++: "Exceptions are run-time anomalies, such as division by zero, that require immediate handling when encountered by your program".
The .NET Framework specify: "Exceptions represents errors that occur during application execution".
In Visual Basic 6: "Visual Basic supports exception (error) handling, which allows the program to detect and possibly recover from errors during execution."
The JScript description indicate: "The try...catch...finally statement provides a way to handle some or all of the possible errors that may occur in a given block of code, while still running code".
In all these languages "exception handling" means manage a run-time error that otherwise would cause the interruption of the program with an error message. The way to do that is via the "try...catch" statement this way:
try {
*any* code
that may cause a *run-time ERROR*
}
catch (exception) {
code that allows to *identify the error*
testing specific values of "exception"
}
Now the differences vs. the proposed Batch file code emulation.
In a Batch file there is no way to "manage" a run-time error: all run-time errors cause the Batch file stop execution with an error message. In Batch files, in a different way than other languages, there are several situations that are not reported as "errors", but just as the manageable result of a command. For example, if find command can not find the search string, it returns an errorlevel greater than zero, and in an entirely equivalent way, if set /A command produce a "run-time error" (like division by zero), it returns an errorlevel greater than zero and the execution continue normally. This way, any program may manage any possible error situation that is reported this way via standard Batch code, with no need of "exception handling".
In the standard "try...catch" feature, any code that may produce any run-time error can be placed in the "try" part with no further testing; the exception is automatically thrown by the system. The particular error that caused the exception can be identified via individual tests in the "catch" part. The proposed Batch emulation is entirely different. In this case, each particular "error" situation must be individually inspected in the "try" part in order to explicitly throw the corresponding "exception"; the "catch" part must also process each one of the given exceptions.
This mechanism looks more like another standard programming languages feature: the "event management" mechanism of languages like C++ that is also supported via Windows Exception Handling Functions. In this scheme an exception/event is explicitly raised via RaiseException function, that cause that the execution thread jump to the function previously registered via AddExceptionHandler.
Please, don't misunderstand me. I think this method is a valuable tool that may facilitate the management of "errors" in Batch code in a simple and powerful way. However, I disagree with the proposed scheme of using this feature via the "try...catch" construct of standard programming languages, that gives the false impression that is possible to emulate the standard exception handling mechanism in Windows Batch files in order to catch run-time errors. In my humble opinion, the method would become close to the standards if it would be based on the "RegisterExceptionHandler" and "RaiseException" scheme instead...

using the successCmd && ( failingCmd & (call ) ) || ( excHandlingCmd ) syntax is quite nice for a lot of cases and does not require any additional files as also described here: https://stackoverflow.com/a/17085933/1915920
(the dummy (call ) is only for the case the 2nd (last) cmd fails)

Related

Working around 32 window limit in MinGW (Windows batch files)

I'm a computational biologist and I'm trying to run large batches of similar code with a single command, but my implementation has hit a brick wall.
I'm using the NEURON simulation environment, which uses MinGW for its Windows interface, which is where my research has shown my problem arises.
Currently, I am using a batch file to run all of these similar pieces of code, to iterate across the "collection" subfolders:
#echo off
for /D %%a in ("%cd%\all_cells\cell_*.*") do cd "%%a\sim1\" & START neuron sim.hoc
The problem arises when I have more than 32 subfolders; the additional instances won't run and will error with a "console device allocation failure: too many consoles" error.
My research has shown me that this is a known problem with Cygwin/MinGW.
However, working around this manually (ensuring that there is no more than 32 "collection" folders) is extremely time consuming when I am now dealing with hundreds of instances (each refers to a simulated cell and I want to gather statistics on hundreds of them), so I am trying to find a solution.
That said, I am terrible at writing batch files (I'm a terrible programmer who is used to scientific languages) and I can't figure out how to code around this.
It would be great if someone could help me either find a way around the 32 limit, or failing that, help me write a batch file that would do this:
-iterate over up to 32 folders
-wait for the instances to finish
-do it again for the next 32, until I reach the end of the folder.
I have tried using the /wait command to do them one at a time, but it still opens all 32. (And this wouldn't be ideal as I'd like to use all 16 cores I have.
The following is adapted from https://stackoverflow.com/a/11715437/1012053, which shows how to run any number of processes while limiting the total number run simultaneously in parallel. See that post for some explanation, though the code below is fairly well documented with comments.
I've eliminated the /O option and the code to work with PSEXEC from the original script.
The script below runs everything in one window - the output of each process is captured to a temporary lock file, and when finished, the full output of each process is typed to the screen, without any interleaving of process output. The start and end times of each process are also displayed. Of course you can redirect the output of the master script if you want to capture everything to a single file.
I've limited the total number of parallel processes to 16 - of course you can easily modify that limit.
The code will not work as written if any of your folder paths include the ! character. This could be fixed with a bit of extra code.
Other than that, the code should work, provided I haven't made any silly mistakes. I did not test this script, although the script it was derived from has been thoroughly tested.
#echo off
setlocal enableDelayedExpansion
:: Define the maximum number of parallel processes to run.
set "maxProc=16"
:: Get a unique base lock name for this particular instantiation.
:: Incorporate a timestamp from WMIC if possible, but don't fail if
:: WMIC not available. Also incorporate a random number.
set "lock="
for /f "skip=1 delims=-+ " %%T in ('2^>nul wmic os get localdatetime') do (
set "lock=%%T"
goto :break
)
:break
set "lock=%temp%\lock%lock%_%random%_"
:: Initialize the counters
set /a "startCount=0, endCount=0"
:: Clear any existing end flags
for /l %%N in (1 1 %maxProc%) do set "endProc%%N="
:: Launch the commands in a loop
set launch=1
for /D %%A in ("%cd%\all_cells\cell_*.*") do (
if !startCount! lss %maxProc% (
set /a "startCount+=1, nextProc=startCount"
) else (
call :wait
)
set "cmd!nextProc!=%%A"
echo -------------------------------------------------------------------------------
echo !time! - proc!nextProc!: starting %%A
2>nul del %lock%!nextProc!
cd "%%A\sim1\"
%= Redirect the output to the lock file and execute the command. The CMD process =%
%= will maintain an exclusive lock on the lock file until the process ends. =%
start /b "" cmd /c 1^>"%lock%!nextProc!" 2^>^&1 neuron sim.hoc
)
set "launch="
:wait
:: Wait for procs to finish in a loop
:: If still launching then return as soon as a proc ends
:: else wait for all procs to finish
:: redirect stderr to null to suppress any error message if redirection
:: within the loop fails.
for /l %%N in (1 1 %startCount%) do 2>nul (
%= Redirect an unused file handle to the lock file. If the process is =%
%= still running then redirection will fail and the IF body will not run =%
if not defined endProc%%N if exist "%lock%%%N" 9>>"%lock%%%N" (
%= Made it inside the IF body so the process must have finished =%
echo ===============================================================================
echo !time! - proc%%N: finished !cmd%%N!
type "%lock%%%N"
if defined launch (
set nextProc=%%N
exit /b
)
set /a "endCount+=1, endProc%%N=1"
)
)
if %endCount% lss %startCount% (
timeout 1 /nobreak >nul
goto :wait
)
2>nul del %lock%*
echo ===============================================================================
echo Thats all folks^^!
You could install screen or tmux in cygwin.
Then you can start all neuron instances in a screen/tmux session.
They will not open a new window, so there is no limit anymore.

Difficulty using set/endlocal in Batch across files

I'm trying to call one bat file from another, while keeping the target variables local to itself(mostly). The code below is my failed attempt, the var i want is missing, but the local is still around.
I've found some info on using setlocal here, which is how I think I have it. Also the info I'm using to push a var past setlocal is here. It's very possible I'm missing something from these. Any help is appreciated!
The calling bat:
#SET errmsg=Old Message
#CALL Target.bat TEST_JOB errmsg
#ECHO.jobname = %jobname%, but should be undefined
#IF DEFINED errmsg ECHO.Error occurred: %ERRORLEVEL% %errmsg%
The target bat:
#SETLOCAL
#ECHO OFF
:START
SET jobname=%~1
SET errmsgvar=%~2
:MAIN
CALL:ERROR "New Error Message"
ECHO.Should have never returned here
ENDLOCAL
:EXIT
#ECHO ON
#EXIT /b %ERRORLEVEL%
:ERROR "errorMesage"
ENDLOCAL & SET "%errmsgvar%=%~1"
ECHO.%errmsg% ^<-- Testing only, we don't know what the actual var will be
(GOTO) 2>NUL & GOTO:EXIT
The result:
C:\Projects\Batch>Caller.bat
New Error Message <-- Testing only, we don't know what the actual var will be
jobname = TEST_JOB, but should be undefined
Error occurred: 0 Old Message
Edit for clarity:
I'm trying to accomplish several things here...
Allow the caller to specify what var the error message should be stored in | 2nd param in call
On an error exit immediately, saving the error message to the provided var | :ERROR block
Keep all vars in target local, push error message through by setting it on the endlocal line
It seems like my endlocal is being ignored, and its just doing endlocal at the end of the file, which breaks things
As Magoo said, your TEST_JOB value is cleared properly. Your test must have been looking at a holdover result from a prior run. Best to explicitly clear the value in your calling bat, prior to the CALL, so you can never get a false result..
Your errmsg is not being set because of a flaw in your logic. More on that after I provide a little introduction to (goto) 2>nul.
You are attempting to use a relatively new (goto) 2>nul technique that is not widely known. It is effectively an exit /b except additional concatenated commands still execute, but in the context of the caller. I believe the first discovery was published at http://forum.script-coding.com/viewtopic.php?id=9050 (which I cannot read), and the first known English posting is at http://www.dostips.com/forum/viewtopic.php?f=3&t=6491.
Since that DosTips post, many useful tools have resulted:
RETURN.BAT - a function to safely return any value across the ENDLOCAL barrier. This function is a derivative of jeb's original idea
PrintHere.bat - a partial emulation of the unix here doc functionality
EXCEPTION.BAT - Effective exception handling within batch scripts
Within your code, you try to ENDLOCAL in your error routine, but that only affects SETLOCAL that were issued while in your routine. You need to use the (goto) 2>nul trick prior to issuing the ENDLOCAL so that ENDLOCAL works properly on the parent context.
:error "errormesage"
(
(goto) 2>nul
endlocal
set "%errmsgvar%=%~1"
goto :exit
)
I'm worried about your returned ERRORLEVEL. I don't think it is returning the value you want when there is an error.
You must remember that all external commands always set the ERRORLEVEL whether there was an error or not. Almost all internal commands set the ERRORLEVEL to non-zero upon error, and some internal commands set the ERRORLEVEL to 0 upon success (but some preserve the existing ERRORLEVEL if there was no error).
It is safest to explicitly pass your desired error number into your :error routine, and then explicitly set the value upon exit. Here is one way to do it:
:error "errMsg" errNum
(
(goto) 2>nul
endlocal
set "%errmsgvar%=%~1"
cmd /c exit %2
goto :exit
)
But I would modify things so that you always exit the same way regardless whether there was an error or not. The :exit routine below is called with the error message and error number if there was an error, and the values are set appropriately. Upon normal exit, neither value is passed in, so the errmsg variable gets cleared, and the routine returns with the current ERRORLEVEL (presumably 0).
Calling script
#setlocal
#set errmsg=Old Message
#set "jobname="
#call target.bat TEST_JOB errmsg
#if defined jobname (
echo ERROR - jobname = %jobname%
) else (
echo SUCCESS - jobname is undefined
)
#if defined errmsg (
echo.Error occurred: %errorlevel% %errmsg%
) else (
echo No error occurred
)
Target.bat
#echo off
setlocal enableDelayedExpansion
:start
set jobname=%~1
set errmsgvar=%~2
:main
set /a "1/(%random% %% 3)" || call :exit !errorlevel! "Random error"
echo Success
call :exit
:exit [errNum] ["errMsg"]
(
(goto) 2>nul
endlocal
set "%errmsgvar%=%~2"
echo on
exit /b %1
)
The :main code randomly raises an error 33% of the time when it attempts to divide by zero.
Here is some sample output, showing both possible outcomes:
C:\test>test
Success
SUCCESS - jobname is undefined
No error occurred
C:\test>test
Divide by zero error.
SUCCESS - jobname is undefined
Error occurred: 1073750993 Random error
Worked perfectly for me.
I believe your test set jobname at some stage and hence your setlocal/endlocal frames simply restore the original environment.
I'd suggest you execute
set "jobname="
as the first line of your calling bat to force jobname to be clear before the action starts.
Change your second script to the following. If accounts for the ENDLOCAL at the end of the script. I believe the ENDLOCAL in the :ERROR block is for that block only and not the entire file.
#SETLOCAL
#ECHO OFF
:START
SET jobname=%~1
SET errmsgvar=%~2
:MAIN
CALL:ERROR "New Error Message"
ECHO.Should have never returned here
ENDLOCAL
:EXIT
REM case insensitive test for errmsgvar.
IF DEFINED errmsgvar (
IF /I NOT "%errmsgvar%" == "" (
ENDLOCAL & SET "errmsg=%errmsg%"
)
)
#ECHO ON
#EXIT /b %ERRORLEVEL%
:ERROR "errorMesage"
ENDLOCAL & SET "errmsgvar=%~1"
SET "errmsg=%errmsgvar%"
ECHO.%errmsg% ^<-- Testing only, we don't know what the actual var will be
(GOTO) 2>NUL & GOTO:EXIT
Output becomes (extra line breaks for clarity):
Caller.bat
New Error Message <-- Testing only, we don't know what the actual var will be
jobname = TEST_JOB
Error occurred: 0 New Error Message
Change 2nd last line to
ECHO.%errmsgvar%.....
to match the 3rd to last line
By way of explanation...
ENDLOCAL & SET "errmsgvar=%~1"
1. The entire line loaded and %~1 is expanded.
2. ENDLOCAL is executed and ends localization.
3. SET "errmsgvar=%~1" is executed as...
SET "errmsgvar=WhateverWasIn%~1"
This technique is tunneling and is a way to pass variables across locales.

Batch File conditional statement not working correctly

So i am attempting to make a simple script to check if an application is running using a external text file (using 1 and 0 for if running or is not). However i cannot seem to get the statement working correctly..
setlocal enabledelayedexpansion
set /p Running=<IsRunning.txt
IF %Running% EQU 0(GOTO ProgramNotRunning)
IF %Running% EQU 1(GOTO ProgramRunning)
:ProgramNotRunning
echo program starting
echo 0 >IsRunning.txt
echo 1 >IsRunning.txt
GOTO:EOF
:ProgramRunning
echo program already running
GOTO:EOF
The issue is no matter what value it is, it always only ever runs the ProgramNotRunning code block and not the other.
Prior to using EQU, i was simply using == to check for equivilance.
Much thanks for any assistance given!
1 - Missing spaces
If %Running% EQU 0 (...
^ This space is needed
2 - In batch files lines of code are executed one after the other unless one command changes this behaviour. You can iterate with for, jump with goto, call subroutines with call, leave the batch, the subroutine or the console with exit, ... but a label will not break the execution. After your if %Running% EQU 1 ... there isn't anything that prevent execution to continue into the code following code it none of the if tests find a coincidence. So, if the set /p does not retrieve 0 or 1 the code after :ProgramNotRunning will be always executed.
3 - Missing/empty file. If IsRunning.txt can not be found or it is empty (or at least the first line is empty) or if it does contain an unexpected value, the if lines will fail. The code executed is
file missing : if EQU 0 (
line/file empty : if EQU 0 (
bad data : if this is a test EQU 0 (
All this cases will cause the line to be considered an error and the execution will be canceled.
#echo off
setlocal enableextensions disabledelayedexpansion
rem Retrieve running state
set "Running="
if exist "IsRunning.txt" (
set /p "Running=" < "IsRunning.txt"
)
IF "%Running%" EQU "0" goto ProgramNotRunning
IF "%Running%" EQU "1" goto ProgramRunning
rem As there is no valid data, assume the program is not running
:ProgramNotRunning
echo program starting
>"IsRunning.txt" (echo 1)
goto :eof
:ProgramRunning
echo program already running
goto :eof
Why >"IsRunning.txt" (echo 1)? Just to be sure there are no aditional spaces after the 1 that will be included in the output (as happens with your code), retrieved when the line is readed from the file and causing the if to fail
if "1 " EQU "1" ( ... This will be evaluated to false
And this still leaves cases where the data retrieved can make the code fail. For a 0/1 test, it is easier to not read the file, just test for presence of the file. If the file exist, the program is running, else it is not running.

Batch ErrorLevel: In For loop, calling another batch, calling SQL

I've done a thorough search on this, but I can't find all of the pieces together;
I have:
BatchA runs for loop
Calls BatchB
BatchB runs a SQL script
SQL Script raises error
BatchB gets error, does (exit /b %ErrorLevel%)
Batch A checks for error
I can't make the last step work, in which BatchA responds to the exit error of batch B.
I've simplified it, and I can't even get the following to work. By not work, I mean that the GOTO statement doesn't execute. I hate for loops in batch!!!!
BatchA
setlocal enabledelayedexpansion
FOR /L %%i IN (0,1,0) DO (
:: Define variables
CALL SET TEST_NAME=%%TESTS[%%i]%%
CALL SET D_TEST=%%D_TESTS%%\%%TEST_NAME%%
CALL SET D_DB_RAW=%%D_TEST%%
CALL SET P_TEST=%%D_TEST%%\%%TEST_NAME%%.bat
:: Run preparation script
CALL ECHO. && ECHO.
CALL ECHO [INFO] TEST: %%TEST_NAME%%
CALL ECHO ----------------------------------------------------------
CALL ECHO [INFO] PREPARING THE DATABASE
CALL PUSHD %%D_PRE_TEST%%
CALL %%P_PRE_TEST_BAT%% %%D_DB%% %%D_DB_RAW%%
CALL POPD
CALL SET ERR_MSG="Error preparing the database before test: %%TEST_NAME%%"
if !ERRORLEVEL! NEQ 0 GOTO L_ERROR
BATCH B
EXIT /b 1

How can I exit a batch file from within a function?

I have a simple function written to check for directories:
:direxist
if not exist %~1 (
echo %~1 could not be found, check to make sure your location is correct.
goto:end
) else (
echo %~1 is a real directory
goto:eof
)
:end is written as
:end
endlocal
I don't understand why the program would not stop after goto:end has been called. I have another function that uses the same method to stop the program and it work fine.
:PRINT_USAGE
echo Usage:
echo ------
echo <file usage information>
goto:end
In this instance, the program is stopped after calling :end; why would this not work in :direxist? Thank you for your help!
I suppose you are mixing call and goto statements here.
A label in a batch file can be used with a call or a goto, but the behaviour is different.
If you call such a function it will return when the function reached the end of the file or an explicit exit /b or goto :eof (like your goto :end).
Therefore you can't cancel your batch if you use a label as a function.
However, goto to a label, will not return to the caller.
Using a synatx error:
But there is also a way to exit the batch from a function.
You can create a syntax error, this forces the batch to stop.
But it has the side effect, that the local (setlocal) variables will not be removed.
#echo off
call :label hello
call :label stop
echo Never returns
exit /b
:label
echo %1
if "%1"=="stop" goto :halt
exit /b
:halt
call :haltHelper 2> nul
:haltHelper
()
exit /b
Using CTRL-C:
Creating an errorcode similar to the CTRL-C errorcode stops also the batch processing.
After the exit, the setlocal state is clean!
See #dbenham's answer Exit batch script from inside a function
Using advanced exception handling:
This is the most powerful solutions, as it's able to remove an arbitrary amount of stack levels, it can be used to exit only the current batch file and also to show the stack trace.
It uses the fact, that (goto), without arguments, removes one element from the stack.
See Does Windows batch support exception handling?
jeb's solution works great. But it may not be appropriate in all circumstances. It has 2 potential drawbacks:
1) The syntax error will halt all batch processing. So if a batch script called your script, and your script is halted with the syntax error, then control is not returned to the caller. That might be bad.
2) Normally there is an implicit ENDLOCAL for every SETLOCAL when batch processing terminates. But the fatal syntax error terminates batch processing without the implicit ENDLOCAL! This can have nasty consequences :-( See my DosTips post SETLOCAL continues after batch termination! for more information.
Update 2015-03-20 See https://stackoverflow.com/a/25474648/1012053 for a clean way to immediately terminate all batch processing.
The other way to halt a batch file within a function is to use the EXIT command, which will exit the command shell entirely. But a little creative use of CMD can make it useful for solving the problem.
#echo off
if "%~1" equ "_GO_" goto :main
cmd /c ^""%~f0" _GO_ %*^"
exit /b
:main
call :label hello
call :label stop
echo Never returns
exit /b
:label
echo %1
if "%1"=="stop" exit
exit /b
I've got both my version named "daveExit.bat" and jeb's version named "jebExit.bat" on my PC.
I then test them using this batch script
#echo off
echo before calling %1
call %1
echo returned from %1
And here are the results
>test jebExit
before calling jebExit
hello
stop
>test daveExit
before calling daveExit
hello
stop
returned from daveExit
>
One potential disadvantage of the EXIT solution is that changes to the environment are not preserved. That can be partially solved by writing the environent to a temporary file before exiting, and then reading it back in.
#echo off
if "%~1" equ "_GO_" goto :main
cmd /c ^""%~f0" _GO_ %*^"
for /f "eol== delims=" %%A in (env.tmp) do set %%A
del env.tmp
exit /b
:main
call :label hello
set junk=saved
call :label stop
echo Never returns
exit /b
:label
echo %1
if "%1"=="stop" goto :saveEnvAndExit
exit /b
:saveEnvAndExit
set >env.tmp
exit
But variables with newline character (0x0A) in the value will not be preserved properly.
If you use exit /b X to exit from the function then it will set ERRORLEVEL to the value of X. You can then use the || conditional processing symbol to execute a command if ERRORLEVEL is non zero.
#echo off
setlocal
call :myfunction PASS || goto :eof
call :myfunction FAIL || goto :eof
echo Execution never gets here
goto :eof
:myfunction
if "%1"=="FAIL" (
echo myfunction: got a FAIL. Will exit.
exit /b 1
)
echo myfunction: Everything is good.
exit /b 0
Output from this script is:
myfunction: Everything is good.
myfunction: got a FAIL. Will exit.
Here's my solution that will support nested routines if all are checked for errorlevel
I add the test for errolevel at all my calls (internal or external)
#echo off
call :error message&if errorlevel 1 exit /b %errorlevel%<
#echo continuing
exit /b 0
:error
#echo in %0
#echo message: %1
set yes=
set /p yes=[no]^|yes to continue
if /i "%yes%" == "yes" exit /b 0
exit /b 1

Resources