Tokenize strings in a batch file - batch-file

Suppose I have a file, and I don't know how many words there are in it. I would like to go through this file and echo every word, one at a time. Is there a way to do that? The delim is space and I don't know how many words I will be going through because the file content may change.
For example, if the file contains:
"Hello world
I hope winter should be over soon"
Then I would like the output to be:
"Hello
world
I
hope
winter
should
be
over
soon"
Thanks in advance!

Here is how to do it in batch. I picked this up here awhile back but I don't remember who wrote it. Probably Jeb or dbenham. It looks like their code ;)
#echo off
setlocal disableDelayedExpansion
:: Define LF to contain a newline character
set LF=^
:: Above 2 blank lines are critical - DO NOT REMOVE
for /f "eol= tokens=*" %%A in (hw.txt) do (
set "ln=%%A"
setlocal enableDelayedExpansion
for %%L in ("!LF!") do (
for /f "eol= " %%W in ("!ln: =%%~L!") do echo %%W
)
endlocal
)

The following will work, unless there are double quotes in the text.txt file. Are you sure you are going to have double quotes in your text file? If you don't care about the double quotes, this can be further tuned to strip out the quotes before splitting it
#ECHO OFF
:: Read file text.txt line by line
FOR /F "tokens=* delims=" %%X IN (text.txt) DO (
CALL :SPLIT %%X
)
EXIT /B 0
:SPLIT
:: Verify parameter is not blank, else quit
IF ["%*"]==[""] EXIT /B 0
:: Extract first token, recursively :SPLIT the remainder
FOR /F "tokens=1,* delims= " %%A IN ("%*") DO (
ECHO %%A
CALL :SPLIT %%B
)
The first FOR loop will read the text.txt file, line by line.
The :SPLIT recursive function will take a line, and split it by delimiter of space until there is nothing left in the line.

This uses a helper batch file called repl.bat - download from: https://www.dropbox.com/s/qidqwztmetbvklt/repl.bat
Place repl.bat in the same folder as the batch file or in a folder that is on the path.
type "file.txt"|repl " " "\r\n" x

Related

Batch script to extract part of a string

I need a batch script that will read in another batch script (batch2) and:
look for the string "configout:" OR "configin:"
If it encounters one of these two strings, extract what's after it until the string ".xml"
copy paste it in a new text file.
and do this for each line of batch2.
For exemple:
If this is my first line in the batch script
/configin:%faxml%fm_sellin_in.xml /configout:%faxml%transco_fm_sellin_out%col_transco%.xml /inputfile:
I should have this in my text file:
%faxml%fa_sellin_in.xml
%faxml%transco_fm_sellin_out%col_transco%.xml
I have seen a good code in Here:
for /f "tokens=1-2 delims=~" %%b in ("yourfile.txt") do (
echo %%b >> newfile.txt
echo removed %%a)
but i don't know how to adapt it to my specific case.
Why not replace all the /configin and /configout with newlines? -
(Replace string with a new line in Batch)
For example
setlocal EnableDelayedExpansion
set "str=/configin:%%faxml%%fm_sellin_in.xml /configout:%%faxml%%transco_fm_sellin_out%%col_transco%%.xml /inputfile:"
set str=!str:/configin^:=^
!
set str=!str:/configout^:=^
!
Now, !str! would contain
fm_sellin_in.xml
transco_fm_sellin_out.xml /inputfile:
Then, you could use the for loop to extract the strings
for /f "tokens=1,2 delims=. " %%a in ("!str!") do ()
this for loop iterates through each line and splits each line with the and . characters.
So %%a is your file name and %%b is the extension.
then
if [%%b]==[xml] (echo %%a.%%b>>mytextfile.txt)
We will do this for all the lines of batch2.
And the finished code is
setlocal EnableDelayedExpansion
for /f "delims=" %%c in (batch2.txt) do (
set str=%%c
set str=!str:/configin^:=^
!
set str=!str:/configout^:=^
!
for /f "tokens=1,2 delims=. " %%a in ("!str!") do (if [%%b]==[xml] (echo %%a.%%b>>mytextfile.txt))
)

How to seprate space and more than one space delimiter in a batch For loop?

I have a problem to parse a text as a result of "net user" command
ABC ABCDE ABCDEFG
ABDCS HFJ ATi CObdnsen
to single users in a separate lines in batch script.
my code is this :
FOR /F "tokens=1,2,3 delims= " %%A in (test.txt) do echo %%A & echo %%B & echo %%C
the problem is the second line of my text that contains a username with space in the middle.
Note that i want this result :
ABC
ABCDE
ABCDEFG
ABDCS HFJ
ATi
CObdnsen
so what is the solution?
That looks to be fixed width columns, 25 characters per column. You could load each line into a variable and use substring operations to get the values.
#echo off
setlocal enableDelayedExpansion
for /f delims^=^ eol^= %%A in (test.txt) do (
set "ln=%%A"
echo col1=[!ln:~0,25!]
echo col2=[!ln:~25,25!]
echo col3=[!ln:~50!]
echo(
)
But that leaves you with the problem of removing trailing spaces from the end of each value. It is doable, but not easy with batch. My output from the above script encloses the values within square brackets so you can easily see the trailing space issue.
Instead of messing with the issue of removing trailing spaces, I would use my REPL.BAT utility to transform the data from fixed width to delimited format. REPL.BAT is a hybrid JScript/batch utility that performs a regular expression search/replace operation on stdin and writes the result to stdout. It is pure script that will run on any modern Windows machine from XP onward without needing any 3rd party executables. Full documentation is embedded within the script.
I would use REPL.BAT to insert a delimiter after the 25th and 50th characters. I would then use another REPL.BAT to strip out any spaces that precede the delimiter, and then a normal FOR /F can safely parse the values. I chose to use a pipe (|) as the delimiter.
#echo off
for /f "eol=| delims=| tokens=1-3" %%A in (
'type test.txt ^| repl "^(.{25})(.{25})" "$1|$2|" ^| repl " *\|" "|"'
) do (
echo col1=[%%A]
echo col2=[%%B]
echo col3=[%%C]
echo(
)
If you know that no value contains consecutive spaces, and all values are separated by at least 2 spaces, then you can get by with a single REPL that replaces 2 or more spaces with a delimiter
#echo off
for /f "eol=| delims=| tokens=1-3" %%A in (
'type test.txt ^| repl " {2,}" "|"'
) do (
echo col1=[%%A]
echo col2=[%%B]
echo col3=[%%C]
echo(
)
#echo off
setlocal enableextensions
for /f "tokens=*" %%a in ('net user^|find " "') do (
set "u=%%a"
setlocal enabledelayedexpansion
set "u=!u: =/!"
set "u=!u:/ =/!"
for /f "tokens=1 delims=/" %%b in ("!u!") do echo %%b
endlocal
)
endlocal
Not bullet proof. Will fail for user names with two spaces in it or (probably, not tested) with names of 24 characters or more (just to name two).
Thanks MC ND for your great idea... But your Answer should be edited like below for the best result :
#echo off
setlocal enableextensions
for /f "tokens=*" %%a in ('net user^|find " "') do (
set "u=%%a"
setlocal enabledelayedexpansion
set "u=!u: =/!"
set "u=!u:/ =/!"
for /f "tokens=1,2,3 delims=/" %%b in ("!u!") do echo %%b & echo %%c & echo %%d
endlocal
)
endlocal
Note that the delimiter is "5 spaces". because the most allowed user length is 20 character in windows. and the distance between the columns are 25 character.. so we have at least 5 spaces as a good delimiter. and we are not worry about spaces in the middle of a username.
and at last your should parse three tokens . Not only 1 token.
Thanks for your attention guys.

Read words separated by space in a batch script

I need to read the content of a text file word by word, in a batch script. The words are space separated. I tried this but it doesn't work, it only takes the first word:
for /f "delims= " %%i in (%OUTPUT_FILE%) do echo %%i
I also tried this but doesn't work also:
for /f "tokens=* delims= " %%i in (%OUTPUT_FILE%) do echo %%i
for /f "delims=" %%a in (file.txt) do for %%b in (%%a) do echo %%b
Delimiters: space, tab, 0xFF, , ; =.
Here is a pure batch solution that should be able to handle any content in your file. It reads each line with an outer FOR /F, and replaces each space with a newline character. It reads each resulting line (word) with an inner FOR /F.
#echo off
setlocal disableDelayedExpansion
:: Define LF to contain a newline character
set LF=^
:: Above 2 blank lines are critical - DO NOT REMOVE
for /f "eol= tokens=*" %%A in (%OUTPUT_FILE%) do (
set "ln=%%A"
setlocal enableDelayedExpansion
for %%L in ("!LF!") do for /f "eol= " %%W in ("!ln: =%%~L!") do echo(%%W
endlocal
)
Life is much simpler if you use a hybrid JScript/batch utility called REPL.BAT that performs regex search and replace on stdin and writes the result to stdout.
Assuming REPL.BAT is in your current folder, or better yet, somewhere within your PATH:
for /f "eol= " %%W in ('repl " " "\n" x ^<%OUTPUT_FILE%') do echo(%%W
Tokens control the elements split from delimiters that are available. So depending on what your overall goal is, it might be worth pursuing tokens to retrieve the desired variables. If you know how many elements will be on each line of your text file, such as a config file or key value pairs on each line, you could leverage 'tokens=' to get what you need.
for /f "tokens=1,2 delims= " %%a in (%OUTPUT_FILE) do echo %%a %%b
If the items per line is unknown, Endoro's word loop inside the line loop is definitely the way to go.

How can we remove the leading and traling spaces from a string in batch script?

I have a first.properties file and I am reading that file, skipping the first 5 lines and creating a new file with the remaining lines of first.properties. This is working fine. But I am getting spaces for each line in the newly created file. I want to remove these spaces.
I have used the following piece of code:
#echo off
set new=0
setlocal ENABLEDELAYEDEXPANSION
for /F "skip=5" %%L in (first.properties) do (
echo %%L>>NewProjectsInfo.properties
)
endlocal
pause
first.properties is as follows:
name=abcd
number=bcde
address=cdef
mobile=efgh
worklocation=ghij
Please help me out to remove the spaces.
Try this. There can be issues when numbers directly precede the >> characters.
You are not using ENABLEDELAYEDEXPANSION so I removed it too.
#echo off
set new=0
setlocal
for /F "skip=5 delims=" %%L in (first.properties) do (
>>NewProjectsInfo.properties echo %%L
)
endlocal
pause
There is a space after NewProjectsInfo.properties in your code. if that is removed, it works as you expect.
It's sad, but looks like, batch interprets command as entire line & separates out the redirection part. In bash, for example, the command terminates at > symbol itself.
e.g. echo hello >somefile.txt world will put hello world in somefile.txt.
#ECHO OFF
SETLOCAL
(
FOR /f "skip=5 delims=" %%i IN (rltsf.txt) DO CALL :unpad %%i
)>output.txt
FC rltsf.txt output.txt
GOTO :eof
:unpad
ECHO %*
GOTO :eof
This approach seemst to work, but I fancy it may be a little sensitive to line-content. Easier to test if we have representative data...

batch script - read line by line

I have a log file which I need to read in, line by line and pipe the line to a next loop.
Firstly I grep the logfile for the "main" word (like "error") in a separate file - to keep it small. Now I need to take the seperate file and read it in line by line - each line needs to go to another loop (in these loop I grep the logs and divide it in blocks) but I stuck here.
The log looks like
xx.xx.xx.xx - - "http://www.blub.com/something/id=?searchword-yes-no" 200 - "something_else"
with a for /f loop I just get the IP instead of the complete line.
How can I pipe/write/buffer the whole line? (doesn't matter what is written per line)
Try this:
#echo off
for /f "tokens=*" %%a in (input.txt) do (
echo line=%%a
)
pause
because of the tokens=* everything is captured into %a
edit:
to reply to your comment, you would have to do that this way:
#echo off
for /f "tokens=*" %%a in (input.txt) do call :processline %%a
pause
goto :eof
:processline
echo line=%*
goto :eof
:eof
Because of the spaces, you can't use %1, because that would only contain the part until the first space. And because the line contains quotes, you can also not use :processline "%%a" in combination with %~1. So you need to use %* which gets %1 %2 %3 ..., so the whole line.
The "call" solution has some problems.
It fails with many different contents, as the parameters of a CALL are parsed twice by the parser.
These lines will produce more or less strange problems
one
two%222
three & 333
four=444
five"555"555"
six"&666
seven!777^!
the next line is empty
the end
Therefore you shouldn't use the value of %%a with a call, better move it to a variable and then call a function with only the name of the variable.
#echo off
SETLOCAL DisableDelayedExpansion
FOR /F "usebackq delims=" %%a in (`"findstr /n ^^ t.txt"`) do (
set "myVar=%%a"
call :processLine myVar
)
goto :eof
:processLine
SETLOCAL EnableDelayedExpansion
set "line=!%1!"
set "line=!line:*:=!"
echo(!line!
ENDLOCAL
goto :eof
This has worked for me in the past and it will even expand environment variables in the file if it can.
for /F "delims=" %%a in (LogName.txt) do (
echo %%a>>MyDestination.txt
)
For those with spaces in the path, you are going to want something like this:
n.b. It expands out to an absolute path, rather than relative, so if your running directory path has spaces in, these count too.
set SOURCE=path\with spaces\to\my.log
FOR /F "usebackq delims=" %%A IN ("%SOURCE%") DO (
ECHO %%A
)
To explain:
(path\with spaces\to\my.log)
Will not parse, because spaces.
If it becomes:
("path\with spaces\to\my.log")
It will be handled as a string rather than a file path.
"usebackq delims="
See docs will allow the path to be used as a path (thanks to Stephan).

Resources