Reading a file with a batch with n tokens including empty ones - batch-file

Question

I want to write a batch script which reads each line of a file. Sounds easy but the problem is that not every line has the same amount of tokens. The next thing is that every token is seperated from each other with a delimiter which I don't want to have in my output.
Here is a example of the file:
string1|string2|string3|string4
string1|||string4
When I'm reading that file, every token has to be on its right place. What I mean is when I'm reading the second line the output must be:
string1 empty empty string4
thanks for help

Answer 1

gawk for Windows can do the job:
awk -F"|" "{for (i=1;i<=NF;i++) $i=$i?$i:\"empty\"};7" file.in>file.out

Answer 2

#ECHO OFF
SETLOCAL
(
FOR /f "delims=" %%a IN (q21551753.txt) DO (
SET line=%%a
SETLOCAL ENABLEDELAYEDEXPANSION
SET line=!line:^|= ^|!
ECHO(!line!
endlocal
)
)>tempfile.txt
SET "spaces= "
FOR /f "tokens=1-7delims=|" %%a IN (tempfile.txt) DO (
SET "$a=%%a%spaces%"
SET "$b=%%b%spaces%"
SET "$c=%%c%spaces%"
SET "$d=%%d%spaces%"
SET "$e=%%e%spaces%"
SET "$f=%%f%spaces%"
SET "$g=%%g%spaces%"
SETLOCAL ENABLEDELAYEDEXPANSION
ECHO(!$a:~0,8!!$b:~0,9!!$c:~0,8!!$d:~0,8!!$e:~0,9!!$f:~0,5!!$g:~0,6!
endlocal
)
GOTO :EOF
I used a file named q21551753.txt for my testing.
string1|string2|string3|string4
string1|||string4
|string2||string4|string5|string6
%one%|!2!||"three"|^four^|string6
Response:
string1 string2 string3 string4
string1 string4
string2 string4 string5 strin
%one% !2! "three" ^four^ strin
Note that the column-widths are defined in the ECHO(!$a... statement - your choice of the number and width of each column. Note that spaces is set to a bunch of spaces.
tempfile.txt is created. Whether you delete this, where it is created and what you call it are all up to you.

Answer 3

The following assumes data does not contain !, and no line begins with ;. These restrictions can be eliminated with more code.
The key trick is to use variable expansion search and replace to convert str1|str2|str3|str4 into "str1"|"str2"|"str3"|"str4". Then a second FOR /F can safely parse the tokens, even when the value is missing. The values you want are in %%~A, %%~B, %%~C, and %%~D. The line I ECHO is just to demonstrate the values of each token.
#echo off
setlocal enableDelayedExpansion
for /f "delims=" %%L in (test.txt) do (
set "ln=%%L"
set "ln="!ln:^|="|"!""
for /f "tokens=1-4 delims=|" %%A in ("!ln!") do (
echo A=%%~A B=%%~B C=%%~C D=%%~D
)
)

Answer 4

Thanks for your answers.
But I found out that I had another problem. The count of the tokens are not always the same and the count is often about 20-40 tokens at one line.
I created a solution with a sub-procedure:
#echo off
setlocal enabledelayedexpansion
for /f "tokens=*" %%a in (Book1.csv) DO (
set "work=%%a"
for /l %%i in (1,1,2) do set "work=!work:||=|NULL|!"
set "line=!work!"
call :processToken
)
goto eof
:processToken
for /F "tokens=1* delims=|" %%x in ("!line!") do (
echo %%x
set line=%%y
)
if not "!line!" == "" goto :processToken
goto :eof
:eof