Difference between !var! and %var% in batch files [duplicate] - batch-file
for /f "tokens=*" %%a in ('find /v ":" "%appdata%\gamelauncher\options.txt" ^| find "menu=a"') do ( set usemenu=a )
for /f "tokens=*" %%a in ('find /v ":" "%appdata%\gamelauncher\options.txt" ^| find "menu=b"') do ( set usemenu=b )
for /f "tokens=*" %%a in ('find /v ":" "%appdata%\gamelauncher\options.txt" ^| find "menu=c"') do ( set usemenu=c )
Right, in this code (which may not work, that what i'm trying to find out) we have this "%%a" in that 'for' command.
First, whats the difference between %variable% and %%a?
Second, can someone explain the 'for' command to me? I have Google'd it way too much and all the explanations seem way to complicated...
What I am trying to do is pull a variable from options.txt, so i can change the menu style of my game launcher. there are 3 styles (a, b and c), so if the options.txt reads "menu=a" how can i get it to set a variable like %usemenu% to the value of a?
Thanks for any help in advance!
%variable% are environment variables. They are set with set and can be accessed with %foo% or !foo! (with delayed expansion if enabled). %%a are special variables created by the for command to represent the current loop item or a token of a current line.
for is probably about the most complicated and powerful part of batch files. If you need loop, then in most cases for has you covered. help for has a summary.
You can
iterate over files: for %x in (*.txt) do ...
repeat something n times: for /l %x in (1, 1, 15) do... (the arguments are start, step and end)
iterate over a set of values: for %x in (a, b, c) do ...
iterate over the lines of a file: for /f %x in (foo.txt) do ...
tokenize lines of a file: for /f "tokens=2,3* delims=," %x in (foo.txt) do ...
iterate over the output of a command: for /f %x in ('somecommand.exe') do ...
That's just a short overview. It gets more complex but please read the help for that.
Variables of the form %%a (or %a if for is used outside of batch files) are very similar to arguments to batch files and subroutines (%1, %2, ...). Some kinds of expansions can be applied to them, for example to get just the file name and extension if the variable represents a file name with path you can use %%~nxa. A complete overview of those is given in help for.
On the other hand, environment variables have other kinds of special things. You can perform replacements in them via %foo:a=b% would result in %foo% except that every a is replaced by a b. Also you can take substrings: %foo:~4,2%. Descriptions of those things can be found in help set.
As to why %variables% and %%a are different things that's a bit hard to answer and probably just a historical oddity. As outlined above there is a third kind of variable, %1, etc. which are very similar to those used in for and have existed for longer, I guess. Since environment variables are a bit unwieldy to use in for due to blocks and thus heavy reliance on delayed expansion the decision probably was made to use the same mechanisms as for arguments instead of environment variables.
Also environment variables could be more expensive, given that the process has a special “environment” block of memory where they are stored in variable=value␀ pairs, so updating environment variables involves potentially copying around a bit of memory while the other kind of variables could be more lightweight. This is speculation, though.
As for your problem, you don't really need for here:
find /v ":" "%appdata%\gamelauncher\options.txt" | find "menu=a" && set usemenu=a
This will only run the set if the preceding command was successful, i.e. menu=a was found. This should be considerably easier than for. From what I read you're trying to look whether menu=a exists in a line that does not contain a colon and in that case usemenu should be set to a, right? (And likewise for b and c. You could try coaxing for into doing that by looping over the lines of the file or output and tokenizing appropriately to figure out the value of menu but depending on the format of the lines this can be tricky. If what you have there works in theory then you should simply stick to that. You can however use a loop around it to avoid having to repeat the same line three times for a, b and c:
for %%X in (a b c) do (
find /v ":" "%appdata%\gamelauncher\options.txt" | find "menu=%%X" && set usemenu=%%X
)
If the file you are parsing is simple, however, with just name=value pairs in each line where : foo would be a comment, then you could use for as well:
for /f "tokens=1,* eol=: delims==" %%A in (%appdata%\gamelauncher\options.txt) do (
if "%%A"=="menu" set usemenu=%%B
)
But that depends a little on the exact format of the file. Above snippet would now read the file line by line and for each line would discard everything after a colon (the eol=: option), use the equals sign as a token delimiter and capture two tokens: The part before the first = and everything after it. The tokens are named starting with %%A so the second one is implicitly %%B (again, this is explained in help for). Now, for each line we examine the first token and look whether it's menu and if so, assign its value to the usemenu variable. If you have a lot of possible options to support this is certainly easier to maintain :-).
Related
Batch loopvariable manipulation
I have a small problem with a .bat file that I have to build to manipulate a specific .csv. I want the .bat to read the line of the file, and then check for the first three letters of that line. At the end there should be n-files where file xxx.csv contains the lines of the input.csv with xxx as the first three letters of line. First things first, I don't even know if it is possible to do it this job in a batch-file, because the file has >85000 lines and may even get much bigger. So if it is impossible you can directly tell me that. for /f "delims=" %%a in (input.CSV) DO ( echo %%~a:~0,3 pause ) I want to "output" the first three letters of %%a. It would be great if you could help me. Phil
Substring substitution only works with environment variables (%var%), but not with metavariables (%%a) (as Mofi already commented). And because you are setting and using a variable within the same command block, you need delayed expansion: setlocal enabledelayedexpansion for /f "delims=" %%a in (input.CSV) DO ( set "var=%%~a" echo !var:~0,3! pause ) (there are methods without delayed expansion, but they make use of call, which slows things down)
Batch script to grab lines with findstr without filepath
I've got a log file that monitors a large system including requests and acknowledgements. My objective is to be able to:
1. Loop through the script and get the lines where requests & their acknowledgements happen
2. Pull the entire lines of importance as strings and store them as variables for string modifying to output somewhere else.
Here's what I have so far:
#ECHO off
setlocal
setlocal enabledelayedexpansion
setlocal enableextensions
:: Lets get today's date, formatted the way the ABCD File is named
for /f "tokens=1-5 delims=/ " %%d in ("%date%") do set targetDate=%%f-%%d-%%e
:: Now we set the targetFile name
SET ABCDLogsFile=C:\Users\me\Documents\monitoring_file_for_jim\ABCDFIX*%targetDate%.log
::****Scrapped original approach*****
set "ackFoundCount=0"
set "reqFoundCount=0"
::Get lines with acks
for /f delims^=^ eol^= %%a in ('findstr /c:"\<ACK\>" "%ABCDLogsFile%"') do (
set /a "ackFoundCount+=1"
setlocal enabledelayedexpansion
for %%N in (!ackFoundCount!) do (
endlocal
set "ackFound%%N=%%a"
)
)
::Get lines with requests
for /f delims^=^ eol^= %%b in ('findstr /c:"ReqSingle" "%ABCDLogsFile%"') do (
set /a "reqFoundCount+=1"
setlocal enabledelayedexpansion
for %%N in (!reqFoundCount!) do (
endlocal
set "reqFound%%N=%%b"
)
)
setlocal enabledelayedexpansion
for /l %%N in (1,1,2 %reqFoundCount%) do echo REQ %%N FOUND= !reqFound%%N!
pause
for /l %%N in (1,1,2 %ackFoundCount%) do echo ACK %%N FOUND= !ackfound%%N!
endlocal
EDIT 2 dbenham
The roundabout way I was trying to accomplish this before was totally unnecessary.
Thanks to the questions and answer here:
'findstr' with multiple search results (BATCH)
I've got my script working similarly. However, I'm curious if its possible to get findstr output without the filepath at the beginning. I only need to substring out the timestamp in the log, which would always be the first 12 characters of each line (without the filepath). My output currently is prefixed with the path, and while I could get the path where the log would eventually be in production, it would be safer to try and do it another way. At the time that this script would eventually be run, there would only be 1 or 2 reqs and acks each, that is why I store all which are found. It's not necessary but I think it would be reassuring to see two if there are two. Here is what the output looks like for acks and reqs alike:
C:\Users\me\Documents\monitoring_file_for_jim\ABCDFIX 2015-04-01.log:2015-03-26 07:00:11,028 INFO etc...
I'm thinking that if I could strip the filepath off the start, then all I'd need to do to get just the timestamps of the events would be
for /l %%N in (1,1,1 %reqFoundCount%) do echo Req %%N occurred at: !reqFound%%N:~0,12! >> MorningAckChecks.txt
for /l %%N in (1,1,1 %ackFoundCount%) do echo ACK %%N occurred at: !ackfound%%N:~0,12! >> MorningAckChecks.txt
I suspect you could not get SKIP to work because you you were iterating the delimited list of line numbers with a FOR statement, which means the number is in a FOR variable. Problem is, you cannot include FOR variables or (delayed expansion) when specifying a SKIP value, or any other FOR option. The batch parser evaluates the FOR options before FOR variables are expanded, so it couldn't possibly work. Only normal expansion can be used when including a variable as part of FOR options. But I don't understand why you think you need the line numbers at all. FINDSTR is already able to parse out the lines you want. Simply use FOR /F to iterate each matching line. For each line, define a variable containing the line content, and then use substring operations to parse out your desired values. But I can offer an alternative that I think could make your life much easier. JREPL.BAT is a sophisticated regular expression text processor that could identify the lines and parse out and transform your desired values, all in one pass. JREPL.BAT is a hybrid JScript/batch script that runs natively on any Windows machine from XP onward. If I knew what your input looked like, and what your desired output is, then I could probably knock up a simple solution using JREPL.BAT. Or you could read the extensive built in documentation and figure it out for yourself. Documentation is accessed from the command line via jrepl /?. You might want to pipe the output through MORE so you get one screen of help at a time. But I never do because my command line console is configured with a large output buffer, so I can simply scroll up to see past output. EDIT - In response to comment and updated question Here are the relevant snippets of your code that are causing the problem. SET ABCDLogsFile=C:\Users\me\Documents\monitoring_file_for_jim\ABCDFIX*%targetDate%.log findstr /c:"\<ACK\>" "%ABCDLogsFile%" findstr /c:"ReqSingle" "%ABCDLogsFile% The issue is your ABCDLogsFile definition includes a wildcard, which causes FINDSTR to prefix each matching line with the full path to the file name where the match occurred. I have a simple solution for you - Just change the definition of ABCDLogsFile as follows: SET "ABCDLogsFile=C:\Users\me\Documents\monitoring_file_for_jim\ABCDFIX<%targetDate%.log" Explanation My solution relies on two undocumented features 1) Undocumented file mask wildcards. < - Very similar to * > - Very similar to ? These symbols are normally used for redirection, so they must be either quoted or escaped if you want to use them as file mask wildcards. We discuss the undocumented feature at DosTips - Dir undocumented wildcards. Sprinkled throughout the thread (and a link) are some example use cases. I document my understanding of exactly how the non-standard wildcards work at http://www.dostips.com/forum/viewtopic.php?p=39420#p39420 2) FINDSTR works with the non-standard wildcards FINDSTR will prefix each matching line with the file name (and possibly path) if any of the following conditions occur The /M option is used The /F option is used Multiple input files are explicitly listed on the command line Multiple input files are implied via a file mask with at least one * or ? wildcard on the command line Your are getting the file path prefix because of the last trigger - the * in your file mask. But you can use < instead to get the same result, except the non-standard wildcards do not trigger the file prefix in the output. Problem solved :-) I talk about this FINDSTR feature at http://www.dostips.com/forum/viewtopic.php?p=39464#p39464. Some day I hope to update my What are the undocumented features and limitations of the Windows FINDSTR command? post with this tasty little tidbit.
This post has become a bit cluttered. It would be very helpful if you posted the lines of input that correspond to the output you are getting. If you can't do that then add this statement before your FOR. I am sure you will find that testReqSkip is blank. echo.testReqSkip=%testReqSkip%
Removing some columns and rows from csv file via batch
I am trying to create a batch file that will edit a .csv and remove the first column, and any summary lines contained in the file. I am, however, fairly new to programming batch files, so I am not sure the best way to start this, and it would be great if you could include a basic explanation of how the code works so I can be self-sustaining in the future! ,Type,Date,Num,Name,Memo,Member,Clr,Split,Alias,Value,Balance ABB - Egypt,,,,,,,,,,, ElAin EL-Sokhna,,,,,,,,,,, ,Invoice,09-06-10,12005,ABB - EL-Sokhna,,Accounts Receivable,,Training Income,15000,,15000 ,Invoice,09-14-11,12005,ABB - EL-Sokhna,“ElAin EL-Sokhna“ Trainer for OTS Application: First two weeks,Training Income,,Accounts,,150001,0 Total ElAin EL-Sokhna,,,,,,,,,241194,210400,301794 ABB - Egypt - Other,,,,,,,,,,, There are various iterations of this file, as they come from a monthly report, I need to remove the first (empty) column, and any rows that look like ABB - Egypt,,,,,,,,,,, or Total ElAin EL-Sokhna,,,,,,,,,241194,210400,301794 So the output should be: Type,Date,Num,Name,Memo,Member,Clr,Split,Alias,Value,Balance Invoice,09-06-10,12005,ABB - EL-Sokhna,,Accounts,,Training Income,15000,,15000 Invoice,09-14-11,13002,ABB - EL-Sokhna,“ElAin EL-Sokhna“ Trainer for OTS Application: First two weeks,Training Income,,Accounts,,150001,0 Thanks for the input! EDIT: It seems I wasn't clear enough in my OP (Sorry, first time here). There are two processes that need to happen here, in every file the first column must be deleted, and any lines that are either title lines ABB - Egypt,,,,,,,,,,, or summary lines Total ElAin EL-Sokhna,,,,,,,,,241194,210400,301794 need to be removed. All lines that need to be kept will be mostly filled in, such as ,Type,Date,Num,Name,Memo,Member,Clr,Split,Alias,Value,Balance or ,Invoice,09-06-10,12005,ABB - EL-Sokhna,,Accounts Receivable,,Training Income,15000,,15000 Notice that, as in the second line, it is possible for there to be some missing values in them, so doing a search for something like ",," will not work.
Batch is a terrible language for modifying text files. There are a great many special cases that require arcane knowledge to work around the problem. You may have a script that seems to do what you want, and then some wrinkle appears in your data, and the entire script may have to be redesigned.
With regard to your specific problem, it appears to me that you only want to preserve rows that begin with a comma, meaning the first column is empty. Of those remaining rows, you want to remove the first (empty) column.
Assuming none of the rows you want to keep have an empty value for the second column, then there is a really trivial solution:
#echo off
>"%~1.new" (for /f "delims=, tokens=*" %%A in ('findstr "^," %1') do echo %%A)
move /y "%~1.new" %1 >nul
The script expects the file to be passed as the first and only argument. So if your script is named "fixCSV.bat", and the file to be modified is "c:\test\file.csv", then you would use:
fixCSV "c:\test\file.csv"
The %1 expands to the value of the first argument, and %~1 is the same, except it also strips any enclosing quotes that may or may not be present.
The FINDSTR command reads the file and writes out only lines that begin with a comma. The FOR /F command iterates each line of output. The "delims=, tokens=*" options effectively strip all leading commas from each line, and the result is in variable %%A, which is then ECHOed. The entire construct is enclosed in parentheses and stdout is redirected to a temporary file. Finally, the temporary file is moved over top of the original file, thus replacing it.
If the 2nd column may be empty, then the result will be corrupted because it removes all leading commas (both columns 1 and 2 in this case). The script must be more complicated to compensate. You would need to set a variable and then use delayed expansion to get the sub-string, skipping the first character. But delayed expansion will corrupt expansion of the %%A variable if it contains the ! character. So delayed expansion must be toggled on and off. You are beginning to see what I mean by lots of special cases.
#echo off
setlocal disableDelayedExpansion
>"%~1.new" (
for /f "delims=" %%A in ('findstr "^," %1') do (
set "ln=%%A"
setlocal enableDelayedExpansion
echo !ln:~1!
endlocal
)
)
move /y "%~1.new" %1 >nul
As the batch scripts become more complicated, they become slower and slower. It may not be an issue for most files, but if the file is really large (say hundreds of megabytes) then it can become an issue.
I almost never use pure batch to modify text files anymore. Instead, I use a hybrid JScript/batch utility that I wrote called JREPL.BAT. The utility is pure script that runs natively on any Windows machine from XP onward. JREPL.BAT is able to efficiently modify text files using regular expression replacement. Regular expressions can appear to be mysterious, but they are well worth the investment in learning.
Assuming you have JREPL.BAT somewhere within your PATH, then the following command is all that you would need:
jrepl "^,(.*)" "$1" /jmatch /f "yourFile.csv" /o -
The /F option specifies the file to read.
The /O option with value of - specifies that the output should replace the original file.
The /JMATCH option specifies that each replacement value is written out to a new line. All other text is dropped.
The first argument is the search expression. It matches any line that begins with a comma, and everything after that is captured in a variable named $1.
The second argument specifies the replacement value, which is simply the captured value in variable $1.
A way will be to define all your rules in a variable which will be used against findstr. The rules must be defined like this : /c:"String which exclude the line" /c:"Another string which exclude the Line" /c: "etc.." This rules must be exact (That they can't be found in a line who must stay). For the empty first colonne you can use a substitution the way i made it in the code with ,Type=Type ,Invoice=Invoice Test.bat : #echo off&cls setlocal enabledelayedexpansion Rem The rules set $String_To_Search=/c:"ABB - Egypt," /c:"Total ElAin El-Sokhna," /c:"ElAin EL-Sokhna," /c:"ABB - Egypt - Other," for /f "delims=" %%a in (test.csv) do ( set $line=%%a Rem the substitutions for the first Column set $Line=!$Line:,Type=Type! set $line=!$Line:,Invoice=Invoice! Rem the test and the ouput if nothing was found echo !$Line! | findstr /i %$String_To_Search% >nul || echo !$Line! ))>Output.csv I used a file test.csv for my test. The ouput is redirected to Output.csv
Perhaps is this what you want? #echo off setlocal EnableDelayedExpansion for /F "delims=" %%a in (input.csv) do ( set "line=%%a" if "!line:~0,1!" equ "," echo !line:~1! ) When a problem is not enough explained we can only guess the missing details. In this case, I assumed that you just want the lines that start with comma, deleting it. The output is the same as your output example... EDIT: Output example added Type,Date,Num,Name,Memo,Member,Clr,Split,Alias,Value,Balance Invoice,09-06-10,12005,ABB - EL-Sokhna,,Accounts Receivable,,Training Income,15000,,15000 Invoice,09-14-11,12005,ABB - EL-Sokhna,“ElAin EL-Sokhna“ Trainer for OTS Application: First two weeks,Training Income,,Accounts,,150001,0
I would start here to learn this: How can you find and replace text in a file using the Windows command-line environment? It covers many details of substitution from Windows command line and many ways to do it, some requiring only what's built into Windows, and some requiring other downloadable software. Magoo is right, more criteria is needed, but there might be enough information in the linked page for you to get past the main hurdles.
#ECHO OFF
SETLOCAL
(FOR /f "tokens=*delims=," %%a IN ('findstr /b /l "," q28079306.txt') DO ECHO %%a)>newfile.txt
GOTO :EOF
I used a file named q28079306.txt containing your data for my testing.
Produces newfile.txt
I cannot seem to get this string to loop through in batch, and I only can get a single line of output.
My code so far: for /F "tokens=*" %%A in (#opm.txt) do set str=%%A set str=%str:~0,3% #echo.%str% > #tags.txt Basically, I have a huge list of things that I am attempting to take the first three letters from, however, I only end up getting one of those things from the list right now. For example: The word 'eagle' is on the list. The end result should be EAG (and preferably in caps). Any tips?
If you want to do more than one statement in a for loop you need to use brackets: for /F "tokens=*" %%A in (#opm.txt) do ( set str=%%A set str=%str:~0,3% #echo.%str% > #tags.txt ) Note that opening bracket is in the same line as for ... do ... statement. In this case, you need to use delayed expansion to manipulate variables inside a loop (that makes variables enclosed with exclamation marks to be resolved in place): #setlocal ENABLEDELAYEDEXPANSION for /F "tokens=*" %%A in (#opm.txt) do ( set str=%%A set str=!str:~0,3! #echo.!str! >> #tags.txt ) #endlocal Please also note that redirection syntax was changed from > to >>. Single bracket redirection overwrites #tags.txt every times statement is executed. Double bracket redirection appends lines at the end of the file (you may want to assure that the file is empty before the loop). Replacing capitalization of a letters in batch is fairly complicated without external tools. This is not a text-processing-friendly technology. Have you considered Python instead?
How to extract all instances of a specific XML tag attribute using Windows batch
I have an XML file and I need to extract testname from all the instances of <con:testSuite name="testname" within the XML file. I am not quite sure how to approach this, or whether this is possible in batch. Here is what I have thought so far: 1) Use FINDSTR and store every line that has <con:testSuite name= in a variable or a temporary file, like this: FINDSTR /C:"<con:testSuite name=" file.xml > tests.txt 2) Somehow use that file or variable to extract the strings Note that there might be more than one instance of the matching string in the same line. I am a novice at batch and any help is appreciated.
Parsing XML is very painful with batch. Batch is not a good text processor to begin with. However, with some amount of effort you can usually extract the data you want from a given XML file. But the input file could easily be rearranged into an equivalent valid XML form that will break your parser.
With that disclaimer out of the way...
Here is a native batch solution
#echo off
setlocal disableDelayedExpansion
set input="test.xml"
set output="names.txt"
if exist %output% del %output%
for /f "delims=" %%A in ('findstr /n /c:"<con:testSuite name=" %input%') do (
set "ln=%%A"
setlocal enableDelayedExpansion
call :parseLine
endlocal
)
type %output%
exit /b
:parseLine
set "ln2=!ln:*<con:testSuite name=!"
if "!ln2!"=="!ln!" exit /b
for /f tokens^=2^ delims^=^" %%B in ("!ln2!") do (
setlocal disableDelayedExpansion
>>%output% echo(%%B
endlocal
)
set "ln=!ln2!"
goto :parseLine
The FINDSTR /N option is only there to guarantee that no line begins with a ; so that we don't have to worry about the pesky default FOR "EOL" option.
The toggling of delayed expansion on and off is to protect any ! characters that may be in the input file. If you know that ! never appears in the input, then you can simply setlocal enableDelayedExpansion at the top and remove all other setlocal and endlocal commands.
The last FOR /F uses special escape sequences to enable the specification of a double quote as a DELIM character.
Answer to additional question in comment
You cannot simply put the additional constraint in the existing FINDSTR command because it will return the entire line that has a match. Remember you said yourself, "there might be more than one instance of the matching string in the same line". The first name might start with the correct prefix, and the 2nd name on the same line might not. You only want to keep the one that starts appropriately.
One solution is to simply change the echo(%%B >>%output% line as follows:
echo(%%B|findstr "^lp_" >>%output%
The FINDSTR is using a regular expression meta-character ^ to specify that the string must start with lp_. The quotes have already been removed at this point, so we don't have to worry about them.
However, you may run into a situation in the future where you must include " in your search string. Plus it might be marginally faster to include the lp_ screen in the initial FINDSTR so that :parseLine is not called unnecessarily.
FINDSTR requires that search string double quotes are escaped with a back slash. But the Windows CMD processor also has its own rules for escaping. Special characters like > need to be either quoted or escaped. The original code used quotes, but you want to include a quote in the string, and that creates unbalanced quotes in your command. Windows batch generally likes quotes in pairs. At least one of the quotes must be escaped for CMD as ^". If the quote needs to be escaped for both CMD and FINDSTR, then it looks like \^".
But any special characters within the string that are no longer functionally quoted from a CMD perspective must be escaped using ^ as well.
Here is one solution that escapes all special characters. It looks awful and is very confusing.
for /f "delims=" %%A in ('findstr /n /c:^"^<con:testSuite^ name^=\^"lp_^" %input%') do (
Here is another solution that looks much better, but it is still confusing to keep track of what is escaped for CMD and what is escaped for FINDSTR.
for /f "delims=" %%A in ('findstr /n /c:"<con:testSuite name=\"lp_^" %input%') do (
One way to keep things a bit simpler is to convert the search into a regular expression. A single double quote can be searched using [\"\"]. It is a character class expression that matches either a quote or a quote - silly I know. But it keeps quotes paired so that CMD is happy. Now you don't have to worry about escaping any characters for CMD, and you can concentrate on the regex search string.
for /f "delims=" %%A in ('findstr /nr /c:"<con:testSuite name=[\"\"]lp_" %input%') do (