Remove a number of delimiters from each line of file - batch-file

Question

I have a file which I need to load into a database. It has a delimiter of pipe (|) however each line contains different number of pipes. Using a batch script, how can I remove pipes from each line so the same number of pipes are on each line?
Example of file:
1|2|3||||||
4|5|6|||
7|8||||||
Let's say I'd like 5 pipes on each line only so it looks like:
1|2|3|||
4|5|6|||
7|8||||

Answer 1

Update See second solution and limitation updates.
Example file.txt contents
A|B|C|D|E|F|G
1|2|3|4|5|6|7
!|#|#|$|%|^|&
]1|]2|]3|]4|]5|]6|]7
|Two||Four||||Eight
!#$%^&%^*(){}|[]';/.,<>/|
Lonely||||||||||||||||||
Sep|er|ate| From| Th|e |W||orld | |
First Solution
Here is a simple way to do what you want. It should not have any problems with special characters.
Limitations
It only supports up to 24 25 columns as it is currently written. %%A to %%Y
The first value may not begin with ]. Replaced for /F "tokens=1,* delims=]" %%Y in ('type file.txt ^| find /v /n ""') do ( with for /F "delims=" %%Z in ('type file.txt') do (.
"Empty" fields may only appear at the end of every line. See second solution.
Does not preserve blank lines in the file. (This can be fixed if desired.)
Just specify how many and which columns you want to keep. For example tokens=3-5,12,48-50 will select only columns 3,4,5,12,48,49,50. Make sure you add on or remove the variables to match the output you want. echo %%A^|%%B^|%%D^|%%C^|%%G^|%%E^|%%F. Note that the columns can be reordered as well in the echo statement.
#echo off
setlocal DisableDelayedExpansion
for /F "delims=" %%Z in ('type file.txt') do (
for /F "tokens=1-5 delims=|" %%A in ("%%Z") do (
echo %%A^|%%B^|%%C^|%%D^|%%E
)
)
endlocal
pause >nul
You can either redirect the output of the .bat file into a new file Script.bat>output.txt or output the echo command to a file by appending >>output.txt to the echo line.
Example Output:
A|B|C|D|E
1|2|3|4|5
!|#|#|$|%
]1|]2|]3|]4|]5
Two|Four|Eight|| <-- Note that this line exhibits limit 3.
!#$%^&%^*(){}|[]';/.,<>/|||
Lonely||||
Sep|er|ate| From| Th
Second Solution
Shares only limitations 1 and 4. Currently adds spaces into existing blank columns to preserve all columns. They can be removed with a further code change, but will not add unless desired by the OP.
#echo off
setlocal EnableExtensions DisableDelayedExpansion
for /F "delims=" %%Z in ('type file.txt') do (
set "xLine=|%%Z"
call :Parse xLine
)
endlocal
pause >nul
goto :eof
:Parse
call set "xLine=%%%~1:||=| |%%"
for /F "tokens=1-5 delims=|" %%A in ("%xLine%") do (
echo %%A^|%%B^|%%C^|%%D^|%%E
)
goto :eof
Example Output:
A|B|C|D|E
1|2|3|4|5
!|#|#|$|%
]1|]2|]3|]4|]5
|Two| |Four|
!#$%^&%^*(){}|[]';/.,<>/|||
Lonely| | | |
Sep|er|ate| From| Th

Answer 2

There is no direct way to achieve this process, so each character must be revised in order to count the number of pipes in each line. It works, but it is somewhat slow.
#echo off
setlocal EnableDelayedExpansion
rem Number of desired pipes
set limit=5
for /F "delims=" %%a in (input.txt) do (
set "line=%%a"
rem Get position of last character
set last=0
for /L %%b in (12,-1,0) do (
set /A "last|=1<<%%b"
for %%c in (!last!) do if "!line:~%%c,1!" equ "" set /A "last&=~1<<%%b"
)
rem Copy each character to result, but just %limit% number of pipes
set pipes=0
set result=
for /L %%c in (0,1,!last!) do (
if "!line:~%%c,1!" neq "|" (
set "result=!result!!line:~%%c,1!"
) else (
set /A pipes+=1
if !pipes! leq %limit% set "result=!result!|"
)
)
echo !result!
)
Previous program will fail if the input line contain exclamation marks.
Output:
1|2|3|||
4|5|6|||
7|8||||
Antonio