Reading large file from CMD and cutting information - batch-file

Question

I'm writing batch script on Windows, with it's help I would like to sort out information from many files to smaller files.
I got ~3000 long lines in log files, from whom I need get few things, basically there are name and value (example ",INC_LIMI=050,ISO_LIMI=050,INC_MLIM=074,"), and everything is separated with "," symbol. My question how you can read long string line and just read values like:
String LineString[]
LineString = Line.split(,)
String s = "INC_MLIM"
For elem in LineString
if elem.exist(s)
NewLine.append(elem)
and latter on just save to new file.
EDIT:
There is service.log file which contains multiple lines with same variable names, but I don't need all of them so the thing I'm trying to do is
From line :
",INC_MLIM=074,ISO_MLIM=074,LOC_LI_P=050,LOC_LI_L=050,TRI_LI_P=074,TRI_LI_L=074,"
Transform to new line structure with less variables and separate with tabs instead of comma. New line should look something like this:
"INC_MLIM=074 ISO_MLIM=074 LOC_LI_L=050 TRI_LI_L=074"

Answer 1

You don't state which values you want. I'll arbitrarily assume you want INC_LIMI and INC_MLIM.
Like most any text file manipulation, this is a pain to do with pure batch. But it is possible.
I'm assuming your lines are all <8192 characters long. If you have lines that are longer than that, then a pure batch solution is not possible, and you should skip right down to the bottom of this answer for a JREPL solution
Batch does not have a convenient split function that allows splitting at a specific user defined character. The FOR command almost works, but it also splits at ;, =, <tab>, and <space>. So it is not a good choice.
With the correct arcane syntax, you can use variable expansion find/replace to substitute a newline (0x0A) for every comma. This will generate one name=value pair per line, which is very convenient for letting FINDSTR filter out the values that you want.
Here is a solution that relies on a temporary table. This iterates all *.log files, and for each one, it creates output in *.log.new.
#echo off
setlocal enableDelayedExpansion
(set LF=^
%= This creates a newline 0x0A character =%
)
for %%N in ("!LF!") do for %%F in (*.log) do (
(
for /f "usebackq delims=" %%A in ("%%F") do (
set "ln=%%A"
echo(!ln:,=%%~N!
)
)>"%%F.temp"
findstr /b "INC_LIMI= INC_MLIM=" "%%F.temp" >"%%F.new"
del "%%F.temp"
)
type *.log.new
exit /b
Note that the above can fail if your log files contain !. This could be solved by toggling delayed expansion on and off as needed.
Some people don't like to use temp files. In this case, getting rid of the temp file introduces even more arcane batch constructs. But it does eliminate the ! delayed expansion issue, and the code is shorter. This version can also be significantly slower if the source files are very large.
#echo off
setlocal disableDelayedExpansion
(set LF=^
%= This creates a newline 0x0A character =%
)
for %%F in (*.log) do (
for /f "usebackq delims=" %%A in ("%%F") do (
set "ln=%%A"
cmd /v:on /c "for %%N in ("!LF!") do #echo(!ln:,=%%~N!"|findstr /b "INC_LIMI= INC_MLIM="
)
) >"%%F.new"
type *.log.new
exit /b
It is also possible to solve this without using FINDSTR. But this solution assumes the same name never appears more than once on any given line, and all found names have a value:
#echo off
setlocal disableDelayedExpansion
for %%F in (*.log) do (
for /f "usebackq delims=" %%A in ("%%F") do (
set "ln=,%%A"
for %%N in (INC_LIMI INC_MLIM) do call :findName %%N
)
) >"%%F.new"
type *.log.new
exit /b
:findName Name
setlocal enableDelayedExpansion
set "test=!ln!"
:loop
set "test2=!test:*,%1=!"
if "!test2!" equ "!test!" return
if not defined test2 return
if "!test2:~0,1!" neq "=" set "test=,!test2:*,=!" & goto :loop
for /f "delims=," %%V in ("!test2:~1!") do (
endlocal
echo(%1=%%V
)
exit /b
Here is a variation that handles empty values, but can break if a value contains quotes or poison characters:
#echo off
setlocal disableDelayedExpansion
for %%F in (*.log) do (
for /f "usebackq delims=" %%A in ("%%F") do (
set "ln=,%%A"
for %%N in (INC_LIMI INC_MLIM) do call :findName %%N
)
) >"%%F.new"
type *.log.new
exit /b
:findName Name
setlocal enableDelayedExpansion
set "test=!ln!"
:loop
set "test2=!test:*,%1=!"
if "!test2!" equ "!test!" return
if not defined test2 return
if "!test2:~0,1!" neq "=" set "test=,!test2:*,=!" & goto :loop
set "test2=%1!test2!
endlocal&echo(%test2:,=&rem %
exit /b
But I wouldn't use any of the above. In fact, I would never restrict myself to pure batch because text file manipulation is so darn inefficient and inscrutable.
Instead, I would use JREPL.BAT - a regular expression command line text processing utility. JREPL.BAT is pure script (hybrid JScript/batch) that runs natively on any Windows machine from XP onward - no 3rd party exe file required.
With JREPL, the solution is as simple as
#echo off
for %%F in (*.log) do call jrepl "(?:^|,)((?:INC_LIMI|INC_MLIM)=[^,]*)" "$txt=$1" /jmatchq /f "%%F" /o "%%F.new"
type *.log.new
Not only is the code nice and clean, it is much faster than any pure batch solution.