I have to maintain a batch script of about 3500 lines littered with GOTO. Seems that the original "developer" hasn't heard of this famous paper and modular programming.
What the script does?
The script deals with the (silent) installation/uninstallation/reinstallation of several programs using different options. It could be split in several files that deal with each program in part. The problem is that if you're trying to take a part in another file that part will still GOTO another section that needs to be in the original script.
Refactoring?
Normally you wouldn't do a refactoring without having automated tests (so you can be sure you didn't break anything), but I don't know how to do it. There's no testing framework for that.
Partial Solution
I have come up with a partial "solution" that is some kind of adaptation of characterization tests (from Working Effectively with Legacy Code by Michael Feathers) and approval tests:
- create another script: test.py that replaces all commands (like copy or msiexec) with echo,
- redirect the output to a text file (good.txt),
- change the original batch script,
- run the test.py script again and save the output to another text file (current.txt),
- diff good.txt and current.txt -> if there are no differences then I didn't break anything, but if they are different I need to check if I broke something.
Problem with partial solution
How can I capture and replace all the commands? I could make a list of commands to replace, but there are also a lot of string concatenations to get the name and path of the program to be installed.
CMD level capture/hook?
Is there any way I can hook into the command line interpreter (CMD.exe) so I can replace on the fly all the calls to installers with echo?
Other suggestions?
Do I approach the problem in the wrong way? Can I do it better somehow? Do you have some advice I could use?
You could replace all COPY, DEL or CALL with %COPY%, %DEL% ,...
So you can use the same file for production and also for the tests.
#echo off
if not defined UNITTEST (
set "COPY=COPY"
set "DEL=DEL"
set "CALL=CALL"
)
%COPY% src dest
%DEL% somefile.txt
%CALL% installer.exe
And from your unittest.bat, you could start it via
#echo off
set "COPY=>>trace.log ECHO COPY"
set "DEL=>>trace.log ECHO DEL"
set "CALL=>>trace.log CALL ECHO "
del trace.log
set "unittest=Active"
call production.bat
fc good.txt trace.log
I'm not an expert in Batch, but I have done my fair share of it. With that said, I can offer a few tips.
Forget trying to do it all at once. Batch is very hard to debug. Echoing out to a log file helps a lot, but it will not capture everything you need if something goes wrong.
Work on breaking out the exe and msiexec calls into self-contained scripts. It is much easier to test the small script for the functionality you desire. Once you have that working, it is simple to call that script from the "Master" script.
Establish a good protocol for passing args to, and return codes from the smaller scripts. If there are common settings needed to be used for all the scripts consider using a central settings file.
GOTOs are not the devil, unless they pass control all over the place. Normally there are two good reasons that I know of to use GOTO’s.
Skip past a block of code that does not need to run.
To SET values into variables. Note there is a bug that can prevent variables from having their value set from within an 'IF' statement block. That little bug caused a big headache for me at one time.
Calls to a label might be better option at times.
Depending on how far back the legacy support is required, consider using Powershell when possible. The power and debugging capabilities of Powershell far out way the benefits of simple scripting of Batch. Which at 3500 lines simplicity has already been lost. You are already looking at Python, so maybe that could be used instead.
If you need a break point, use Pause. ECHO all the settings you need to examine right before the pause. This is as close to a break point I have found for batch.
Echo the command you intend to run to a log file and actually run it.
Write small verification scripts to be used independently or with the “Master” script to confirm you are getting the results you are expecting.
Use the right tool for the job. I like to use EditPadPro, RegexBuddy, and BeyondCompare for batch editing and comparing differences. There free tools that can be used too NotePad++ and Windiff. Making many edits in a file of that size is best handled by a good editor. IE inserting an echo at the beginning of a line that calls a cmd.exe.
Remember it is scripting not programming. While there is a lot of overlap of the two, the same exact approach to a problem may not be viable between the two.
Always make a backup copy of the scripts as a whole before mucking around. A fallback position is greatly appreciated when there is one small bug that you can’t find.
If it ain't broke... well you wouldn't be working on it if everything was working just fine.
Always test changes. And when you are done test it again. After that have someone else test it.
Just my .02. I’m sure someone else can chime in with more advanced advice. My knowledge on Batch has been acquired from the school of hard knocks, supplemented by ss64.com
Related
I have experienced (random) weird corruption of environment variables from consecutive calls of multiple (3d party) batch scripts. These individual scripts call #echo off.
Is it possible to force echo on by environment variable/other means or I need to go to the every script and comment/remove the line to see what has been actually executed?
I'm quite sure that this is not possible. You can't force your system to simply ignore a command like ECHO OFF. The only way to I can think of as a workaround is to write a script which replaces all ECHO OFF commands in you bat files before execution and restoring them afterwards.
Take a look at this: https://superuser.com/questions/351661/how-do-i-force-echo-on-for-a-batch-file-without-editing-the-file
I am trying to use the tool here business objects query builder output
And there are virtually no examples, so I'm struggling to make it work. It produces no errors, but outputs no file in the directory where the batch file is, that I can see.
Here is the code inside querybuilder.bat:
set lib=c:\Program Files\Business Objects\Common\4.0\java\lib
java -cp "querybuilder.jar;poi-3.8-20120326.jar;%lib%\*" org.bukhantsov.querybuilder.Program %*
Here is the code inside what I am running, which I've named RunQuery_ALLACTIVE.bat, except of course with my Server, Username, and Password changed for the purpose of this post.
I have this all on one line, with no line breaks.
querybuilder.bat -cms:SERVER -username:OURUSERNAME -password:OURPASSWORD -query:"SELECT * FROM CI_INFOOBJECTS where SI_SCHEDULE_STATUS = 9 order by SI_NAME" -auth:windowsad -excel "Output.xls"
Can't tell if the - options go on different LINES ?
Can't tell if I'm supposed to put output file in quotes, or if it should be an existing file or not?
can't tell if for Windows AD (which we use), I would put "Windows AD" or WindowsAD, I'm assuming no spaces obviously.
Tons of unanswered questions on this tool - it LOOKS cool, but has anyone actually successfully used it? Can't really find comments or history on the 'net..
To answer your questions:
The options go on the same line, not on different ones
As Joe said, you'll need to specify the output file as -excel:"Output.xls"
If you want to use Windows AD, you'll probably need to specify secWinAD (case-sensitive).
If you're not sure about the command line options, I suggest you build up gradually: first only specify the required options, then add the optional ones one by one so you know which one is giving you problems.
Also, I noticed that the download page contains a version compiled for XI3.x and BI4. Make sure you use the correct version, corresponding to the version of BusinessObjects you're using. Also, verify the path in the batch file to see if it points to a valid folder containing the JAR files for the BusinessObjects environment.
Update:
I just noticed that the same author/developer created another application (GUI, not command line) that might be a bit easier to use. Have a look here.
I have a batch file that encounters several errors. These errors require the command prompt to be forcefully closed. Which causes me to have to open the file again to fix the issue.
Due to the nature of this application it is required to run all the time.
I'm looking for a way to automate the file to restart when it encounters an error. Is there a command I can do this with?
Could you please describe in detail and why the command accomplishes such a resolution?
Update:
What I would recommend to accomplish your goal, would be to turn your batch into a Service. (Documentation here) By converting your batch into a Service it no longer becomes subject to users being logged in, permission issues, it will run as a SYSTEM ACCOUNT. This in itself can alleviate a lot of anger for the process.
After you've completed that, you can write a batch file that ensures that your Service is indeed running. Your current issue, is it doesn't automatically restart. Well, a Service always runs- Even if it has an error it will still attempt to run.
Which means unless it has a Fatal Exception your Service should always work- But for certainty you can create a batch that will ensure your Service is running.
An example:
:START
timeout 3600
for /F "tokens=3 delims: " %%H in ('sc query "MyServiceName" ^| findstr "
if /I "%%H" NEQ "RUNNING" (
NET START "MyServiceName"
REM Service has Started...
)
)
GOTO START
So in theory every 3,600 seconds it will test if your Service is running, if it isn't it will start the Service for you.
Important:
This is more the proper way to resolve your issue, rather then circumvent it. However, as I noted your batch should still implore Exception Handling to ensure your application doesn't fall into an unusable state. This still isn't the best way, as it should implore Exception Handling and Verification to test against it's state.
As I mentioned before, you have a lot of methods to solve your issue. However, your thinking in a Linear Mindset. Which means:
Execute Command, Goal Guaranteed.
If I do this, this happens.
Essentially based on the minimal example I saw, it looks like you've created an infinite loop to continually execute your command. My question to you: When your loop has an error, how can it continue to run?
You've already stated that it happens in random areas- Nothing is random, those are more then likely areas that require some verification / testing to ensure it remains in a proper state. The faster your identify the potential problems, the more effective your program can run with no errors.
Hopefully that helps-
What exactly does your batch application do?
The reason I ask is because you can circumvent the issue with Windows Task Scheduler which allows you to configure some parameters to auto start and auto open particular applications based on your specified criteria. Will it be ideal? Will it truly automate to your needs- More then likely not.
As mentioned above by GolezTrol, the cause of your error will be the more important aspect to resolve your issue. Based on your remark
The errors are different each time, to be honest.
That could be an indicator that the batch script doesn't adhere to testing but rather assuming it successfully completed. Without any underlining information such as:
Function
Code Example
Where an error occurred, and during what task.
It makes it relatively difficult to point you in the proper direction. One thing that I would consider is IF. This is a fundamentally basic task but is quite important-
if(Directory.Exists(dirName))
{
// Do This
}
else
{
// Do This
}
I find the C# outline an easier method to understand the purpose of the IF. You can actually implement something similar in your batch. You would accomplish it like this:
if exist { insert file name } (
rem file exists
) else (
rem file doesn't exists
)
or you can accomplish it like this:
if exists c:\myFile.bat notepad c:\myFile.bat
If C:\myFile.bat exists, then open notepad. The reason this is an important is because if the variable doesn't exists, then it can not be affected. This allows your application to essentially make decisions in a very primitive manner.
You have quite a bit of flexibility- There are a lot of examples on this topic because batch programming has been around for a very, very long time. Another alternative would be to eventually move to Powershell. It will have access to the Windows Management Interface (WMI).
Hopefully this points you in the right direction, without more information our answers may not be much help.
I believe, that easiest way is to create one more .bat file with GOTO statement:
#echo off
:startover
echo (%time%) App started.
call "c:\app.bat"
echo (%time%) WARNING: App closed or crashed, restarting.
goto startover
Possibly this will fix your problem:
http://nssm.cc/usage
Basically what it does is you adding some bat file to nssm and making it a service.
In "Action on exit" part it says:
To configure the action which nssm should take when the application exits, edit the default value of the key HKLM\System\CurrentControlSet\Services\servicename\Parameters\AppExit. If the key does not exist in the registry when nssm runs it will create it and set the value to Restart.
If from inside a bat file you called another batch file but still had a few remaining operations to complete, how can you make sure that the call to first bat file will after completion or error, will return to the file that called it in the first instance?
Example:
CD:\MyFolder\MyFiles
Mybatfile.bat
Copy afile toHere
or
CD:\MyFolder\MyFiles
CALL Mybatfile.bat
COPY afile toHere
What is the difference between using CALL or START or none of them at all? Would this have any impact on whether it would return for the results of the copy command or not?
As others have said, CALL is the normal way to call another bat file within a .bat and return to the caller.
However, all batch file processing will cease (control will not return to the caller) if the CALLed batch file has a fatal syntax error, or if the CALLed script terminates with EXIT without the /B option.
You can guarantee control will return to the caller (as long as the console window remains open of course) if you execute the 2nd script via the CMD command.
cmd /c "calledFile.bat"
But this has a limitation that the environment variables set by the called batch will not be preserved upon return.
I'm not aware of a good solution to guarantee return in all cases and preserve environment changes.
If you really need to preserve variables while using CMD, then you can have the "called" script write the variable changes to a temp file, and then have the caller read the temp file and re-establish the variables.
call is necessary for .bat or .cmd files, else the control will not return to the caller.
For exe files it isn't required.
Start isn't the same as call, it creates a new cmd.exe instance, so it can run a called batch file asynchronosly
The `CALL' statement was introduced in MS-DOS 3.3
It is used to call other batch files within a batch file, without aborting the execution of the calling batch file, and using the same environment for both batch files.
So in your case the solution is to use CALL
Okay, I actually didn't even really think about the fact that if you call a batch (regardless of the 'type', i.e. '.bat', or '.cmd') that it won't return if you don't use call.
I've been using call myself though for a different reason that I am actually pretty surprised that no one else has brought up. Maybe I missed it. MAYBE I'M THE ONLY ONE IN THE WORLD WHO KNOWS!! :O
Probably not, but I'm going to drop this knowledge off here because it's super useful.
If you use call you can use binary logic operators to decide how to proceed based on the ERRORLEVEL result. In fact, I always was flabbergasted on how && and || existed in DOS and COULDN'T be used this way. Well, that's why.
The easiest way to test this is to create a return.cmd with notepad, or from the command prompt like so:
c:\> type con >return.cmd
You will now notice the cursor goes down to the next line and hangs. Enter:
#exit /B %1
And then hit ENTER, and then CTRL-Z and that file will be created. Good! You may now feel free to try the following two examples:
call return.cmd 0 && echo Huzzah! A Complete Success! (Or cover up...)
call return.cmd 123 || echo Oops! Something happened. You can check ERRORLEVEL if you want the tinest amount of additional information possible.
So what? Well, run them again with the 0 and the 123 swapped and you should see that the messages DON'T print.
Maybe this multi-line example will make more sense. I use this all the time:
call return.cmd 0 && #(
echo Batch says it completed successfully^^!
) || #(
echo Batch completed, but returned a 'falsey' value of sort.
call echo The specific value returned was: %ERRORLEVEL%
)
(Note the 'call' in the || section before the second 'echo'. I believe this is how people got around not having delayed expansion back in the day. If you DO have delayed expansion enabled (via. setlocal EnableDelayedExpansion inside a batch OR launch a command prompt with cmd /v:on then you can just do !ERRORLEVEL!.)
... This is where I have to apologize and say if you have if ERRORLEVEL trauma in your past you should stop reading. I get it. Trust me. I thought about paying someone on fiverr to remotely type this for me, but for completeness sake I'm just going to take one for the team and mention that you can also do the following to check errorlevel:
if ERRORLEVEL 123 #echo QUICK! MOTHERS, COVER YOUR CHILDREN'S EYES! FINGERS ARE BEING UNDONE! :'(
If you've never typed that before then GOOD! You will live longer without having to read up why exactly you aren't getting the results you expect. Cruel is the word you're looking for, not 'quirky'.
The important part that I really want to get across however is that if you try this and DON'T use 'call' it will ALWAYS execute the 'true' branch. Try it for yourself!
If I'm missing something, or you know a better way to do this, please let me know. I love learning stuff like this!
Additional information I mentioned:
I have known for quite some time that you can put redirects BEFORE commands like so:
>nul echo. This won't be displayed!
But I accidentally discovered the other day by being a dumdum that you can apparently also do:
echo A B>file.txt C
And was REALLY surprised to find a file.txt which consisted of "A B C". It appears yo can place them ANYWHERE, even inside the command. I've never seen anyone do this, nor mention it, but I HAVE seen people mention that you can prefix a line with them.
Maybe it's a bug exclusive to Windows 10 or something. If you have another version and wanna try it out and let me know I'd be interested in what you find out.
Stay nerdy!
Hullo, I was just wondering whether a plain SET parameter could be used for obfuscations or any other protection from antivirus in Windows batch files.
I ask because I recently obtained the two "famous" batch virus scripts "PiFv," and "meLT." For study of course. They kept getting detected by antivirus, so I would disable it, study for a while and it would be deleted again. Lucky I have internet ;) Anyway, I tried to think of a way to hide the virus'.
I thought maybe because they are analyzed by string, merely a text change would help. So I used inconspicuous words like so
SET hello="# Echo off"
%hello%
And so on. Unfortunately, Norton was clever enough to suss it (surprisingly) though it's status was lowered from "Infectious" to "Heuristic." I figured if it had gone from yes to maybe this could work.
Next I went on single letter changes making words, example, if I was to print "infect;"
set z=i
set h=n
set j=f
set x=e
set d=c
set q=t &echo %z%%h%%j%%x%%d%%q%
That has half worked. Although a lengthy process I think if a set alphabet was established then changed it might. Anyway, the half working. It is classed heuristic when off a USB, but not classed if off a floppy. Why? Also, do you think this is a good idea?
Half way through reading your question I thought the same as your second idea, splitting the words.
I can't test as different AV's will detect different things, but maybe try splitting them into half's instead of single letters, so it doesn't take as long to do.
set z=inf
set h=ect
echo %z%%h%
Another option would be to read in the complete commands from a text file
set /p z= <command.txt
echo %z%