I've been working on a batch script, to change some .xml files format/type.
It's almost done, but I'm facing a problem changing the time codes.
Here is an example
<Events>
<Event In="00:00:20.650" Out="00:00:22.970"
<Event In="00:00:23.050" Out="00:00:26.300"
This Time Format hh:mm:ss.ms should be changed to hh:mm:ss:ff
which means, changing the Milliseconds to Frames.
The Formula for doing so goes like this: ms*25/1000 or for simplicity ms*0.025
Output should be
<Events>
<Event In="00:00:20:16" Out="00:00:22:24"
<Event In="00:00:23:01" Out="00:00:26:08"
Notes
0.025 is the number of frames per Milliseconds.
The number of frames MUST be an integer w/o any fractional component.
I'm Using GNUWin and Cygwin library Tools, so it's okay to use bc expr , tr , awk sed or whatever solution to get the job done.
You can do it with an awk script:
script.awk
{
re = "[0-9]{2}:[0-9]{2}:[0-9]{2}.([0-9]{3})"
while( match( $0, re, grps) ) {
frames = sprintf("%02.0f",( grps[1] *0.025 ) )
gsub( "." grps[1], ":" frames)
}
print
}
Run it like this awk -f script.awk yourfile.
It tries to match such a timestamp and captures the last part (the milliseconds) into grps[1]. Then the calculation is done and sprintf is used to format the frames.
After that gsub replaces the milliseconds with the frames.
PowerShell:
[xml]$xml = gc events.xml
function fix([timespan]$ts) {
"{0}:{1:D2}" -f $ts.ToString("hh\:mm\:ss"),[int]($ts.Milliseconds * 0.025)
}
$xml.SelectNodes('//Event') | %{
$_.In = fix $_.In
$_.Out = fix $_.Out
}
$xml.Save("events_fixed.xml")
If you'd rather have it as a batch script, you can convert this to a Batch + PowerShell polyglot by inserting this at the top and saving it with a .bat extension:
<# : batch portion
#echo off & setlocal
powershell "iex (${%~f0} | out-string)"
goto :EOF
: end Batch / begin PowerShell hybrid code #>
Here is a pure batch-file solution. This script is secure against all characters that have special meanings to cmd. Basically, it searches the given input file for keywords or attribute names In and Out, followed by = and a time code enclosed in quotation marks "". If found, the milliseconds portion is split off, converted to frames and appended to the remaining time code, separated by :. There can be an arbitrary number of time codes per line, all of them are recognised and converted as long as their format is like given in your sample data and one of the given keywords is preceded.
The script supports rounding of the converted values -- see the remarks (rem) at variable definition FORMULA at the beginning of the file (look it up in the block with the remark Define constants here:).
This is the code:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
rem // Define constants here:
set KEYWORDS="In","Out" & rem // (provide a list of attribute names)
set "PATTERN=[0-9][0-9]:[0-9][0-9]:[0-9][0-9].[0-9][0-9][0-9]"
set /A PLENGTH=12 & rem // (char. length of resolved `PATTERN` string)
set "FORMULA=(MS*25+0)/1000" & rem /* (`+0` means to round frames down;
rem use `+999` instead to round up, or `+500` to round to nearest) */
set /A PADDING=2 & rem // (number of digits of resulting frame value)
set "FILE=%~1"
if not defined FILE exit /B 1
for /F "delims=" %%L in ('findstr /N /R "^" "%FILE%"') do (
set "LINE=%%L"
setlocal EnableDelayedExpansion
set "LINE=!LINE:*:=!"
if defined LINE (
for /F delims^=^ eol^= %%E in ("!LINE!") do (
endlocal
set "LINE=%%E"
setlocal EnableDelayedExpansion
)
) else (
endlocal
set "LINE="
setlocal EnableDelayedExpansion
)
for %%P in (!KEYWORDS!) do (
endlocal
set "KEYWORD=%%~P"
call :PROCESS LINE LINE KEYWORD
setlocal EnableDelayedExpansion
)
echo(!LINE!
endlocal
)
endlocal
exit /B
:PROCESS rtn_string ref_string ref_key
setlocal EnableDelayedExpansion
set "COLL="
set "LINE=!%~2!"
set "KEYW=!%~3!"
:LOOP
if defined LINE (
call :LEN LLENGTH LINE
call :LEN KLENGTH KEYW
for /F delims^=^ eol^= %%T in ("!KEYW!") do (
set "SEEK=!LINE:*%%T=!"
)
if defined SEEK (
if not "!SEEK!"=="!LINE!" (
call :LEN SLENGTH SEEK
set /A TLENGTH=LLENGTH-^(SLENGTH+KLENGTH^)
for /F "tokens=1,2" %%M in ("!TLENGTH! !KLENGTH!") do (
set "DONE= !LINE:~,%%M!"
set "TEST=!LINE:~%%M,%%N!"
)
set "CHAR=!DONE:~-1!" & set "CHAR=!CHAR: = !"
set "DONE=!DONE:~1!"
if "!TEST!"=="!KEYW!" (
if "!CHAR!"==" " (
set "DONE=!DONE!!KEYW!"
set "TEST=!SEEK!"
set /A TLENGTH=PLENGTH+3
for /F %%N in ("!TLENGTH!") do (
set "TEST=!SEEK:~,%%N!" & set "SEEK=!SEEK:~%%N!"
)
echo(!TEST! | > nul findstr /R /C:^^^"^^^^^^^=\"!PATTERN!\"^^^ $^^^"
if not ErrorLevel 1 (
for /F "tokens=1,2 delims=." %%S in ("!TEST:~1!") do (
set "HHMMSS=%%~S" & set "MS=%%~T"
set /A "FF=%FORMULA%"
set "FF=0000!FF!" & set "FF=!FF:~-%PADDING%!"
)
set "COLL=!COLL!!DONE!="!HHMMSS!:!FF!"" & set "LINE=!SEEK!"
) else (
set "COLL=!COLL!!DONE!!TEST!" & set "LINE=!SEEK!"
)
) else (
set "COLL=!COLL!!DONE!!TEST!" & set "LINE=!SEEK!"
)
) else (
set "COLL=!COLL!!DONE!!TEST!" & set "LINE=!SEEK!"
)
) else (
set "COLL=!COLL!!LINE!" & set "LINE="
)
) else (
set "COLL=!COLL!!LINE!" & set "LINE="
)
goto :LOOP
)
if defined COLL (
for /F delims^=^ eol^= %%E in ("!COLL!") do (
endlocal
set "%~1=%%E"
setlocal EnableDelayedExpansion
)
) else (
endlocal
set "%~1="
setlocal EnableDelayedExpansion
)
endlocal
exit /B
:LEN rtn_length ref_string
setlocal EnableDelayedExpansion
set "STR=!%~2!"
if not defined STR (set /A LEN=0) else (set /A LEN=1)
for %%L in (4096 2048 1024 512 256 128 64 32 16 8 4 2 1) do (
if defined STR (
set "INT=!STR:~%%L!"
if not "!INT!"=="" set /A LEN+=%%L & set "STR=!INT!"
)
)
endlocal & set "%~1=%LEN%"
exit /B
Supposing the script is saved as convert-ms-to-frames.bat, the input file is called sample.xml and the output file is called return.xml, use the following command line:
convert-ms-to-frames.bat "sample.xml" > "return.xml"
Simply remove the > "return.xml" part to display the output data on the console for testing.
Supposing the input file sample.xml contains the following data...:
<?xml version="1.0"?>
<Events>
<Event In="00:00:20.650" Out="00:00:22.970" />
<Event In="00:00:23.050" Out="00:00:26.300" />
</Events>
...the output file return.xml will contain the data below:
<?xml version="1.0"?>
<Events>
<Event In="00:00:20:16" Out="00:00:22:24" />
<Event In="00:00:23:01" Out="00:00:26:07" />
</Events>
Using bc
$ ms="650"
$ ff=$(echo ${ms} "*0.025/1" |bc)
$ echo $ff
16
I leave to you when the result is less then 10 to add the additonal "0"
Related
i have to search a string from a txt like Pippo.K=5 and replace it with Pippo.K=1. I need to search the entire string. What i did is:
set "search=Pippo.K=5"
set "replace=Pippo.K=1"
set "textFile=%SettingFile%.txt"
for /f "delims=" %%i in ('type "%textFile%" ^& break ^> "%textFile%" ') do (
set "line=%%i"
setlocal enabledelayedexpansion
set "line=!line:%search%=%replace%!"
>>"%textFile%" echo(!line!
endlocal
)
but what i returned is
5=Pippo.K=1=5
How can i fix this error?
The following script constitutes a pure batch-file solution. Supposing it is stored as repl-str.bat, you need to call it like this for your application:
repl-str.bat "%SettingFile%.txt" "Pippo.K=5" "Pippo.K=1" "%SettingFile%.txt"
This specifies the input file %SettingFile%.txt, the literal and case-sensitive search string Pippo.K=5, the replacement string Pippo.K=1 and the output file %SettingFile%.txt that is the same as the input file (the related technique has been taken from this answer: Batch script to find and replace a string in text file without creating an extra output file for storing the modified file). If no output file is given, the result is output to the console (useful for testing). If a fifth command line argument is given (arbitrary value), the search is done in a case-sensitive manner.
Here is the code of the script repl-str.bat:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set "FILE_I=%~1"
set "SEARCH=%~2"
set "REPLAC=%~3"
set "FILE_O=%~4"
set "CASE=%~5"
set "FLAG=%~6"
if not defined FILE_I exit /B 1
if not defined SEARCH exit /B 1
if not defined FILE_O set "FILE_O=con"
if defined CASE set "CASE=#"
if defined FLAG set "FLAG=#"
for /F "delims=" %%L in ('
findstr /N /R "^" "%FILE_I%" ^& break ^> "%FILE_O%"
') do (
set "STRING=%%L"
setlocal EnableDelayedExpansion
set "STRING=!STRING:*:=!"
call :REPL RETURN STRING SEARCH REPLAC "%CASE%" "%FLAG%"
>> "%FILE_O%" echo(!RETURN!
endlocal
)
endlocal
exit /B
:REPL rtn_string ref_string ref_search ref_replac case flag
setlocal EnableDelayedExpansion
set "STR=!%~2!"
set "SCH=!%~3!"
set "RPL=!%~4!"
if "%~5"=="" (set "OPT=/I") else (set "OPT=")
if not defined SCH endlocal & set "%~1=" & exit /B 1
set "SCH_CHR=!SCH:~,1!"
if not "%~6"=="" set "SCH_CHR="
if "!SCH_CHR!"=="=" set "SCH_CHR=" & rem = terminates search string
if "!SCH_CHR!"==""^" set "SCH_CHR=" & rem " could derange syntax
if "!SCH_CHR!"=="%%" set "SCH_CHR=" & rem % ends variable expansion
if "!SCH_CHR!"=="^!" set "SCH_CHR=" & rem ! ends variable expansion
call :LEN SCH_LEN SCH
call :LEN RPL_LEN RPL
set /A RED_LEN=SCH_LEN-1
set "RES="
:LOOP
call :LEN STR_LEN STR
if not defined STR goto :END
if defined SCH_CHR (
set "WRK=!STR:*%SCH_CHR%=!"
if %OPT% "!WRK!"=="!STR!" (
set "RES=!RES!!STR!"
set "STR="
) else (
call :LEN WRK_LEN WRK
set /A DFF_LEN=STR_LEN-WRK_LEN-1,INC_LEN=DFF_LEN+1,MOR_LEN=DFF_LEN+SCH_LEN
for /F "tokens=1,2,3 delims=," %%M in ("!DFF_LEN!,!INC_LEN!,!MOR_LEN!") do (
rem set "RES=!RES!!STR:~,%%M!"
if defined WRK set "WRK=!WRK:~,%RED_LEN%!"
if %OPT% "!STR:~%%M,1!!WRK!"=="!SCH!" (
set "RES=!RES!!STR:~,%%M!!RPL!"
set "STR=!STR:~%%O!"
) else (
set "RES=!RES!!STR:~,%%N!"
set "STR=!STR:~%%N!"
)
)
)
) else (
if %OPT% "!STR:~,%SCH_LEN%!"=="!SCH!" (
set "RES=!RES!!RPL!"
set "STR=!STR:~%SCH_LEN%!"
) else (
set "RES=!RES!!STR:~,1!"
set "STR=!STR:~1!"
)
)
goto :LOOP
:END
if defined RES (
for /F delims^=^ eol^= %%S in ("!RES!") do (
endlocal
set "%~1=%%S"
)
) else endlocal & set "%~1="
exit /B
:LEN rtn_length ref_string
setlocal EnableDelayedExpansion
set "STR=!%~2!"
if not defined STR (set /A LEN=0) else (set /A LEN=1)
for %%L in (4096 2048 1024 512 256 128 64 32 16 8 4 2 1) do (
if defined STR (
set "INT=!STR:~%%L!"
if not "!INT!"=="" set /A LEN+=%%L & set "STR=!INT!"
)
)
endlocal & set "%~1=%LEN%"
exit /B
Basically, this approach takes the first character of the search string and looks it up in the input text. At each match, it is checked whether the whole search string occurs. If so, it is replaced by the replacement string by removing as many characters as the search string consists of, hence avoiding sub-string replacement syntax which fails in case the search string contains =, or the search or the replacement string contains % or !.
However, if the first character of the search string is =, ", % or !, the approach is different, the script checks every single character position for occurrence of the search string then, with the disadvantage of reduced overall performance. If a sixth command line argument is given (arbitrary value), this (slow) mode is forced.
Batch variable substring substitution does have limitations. Dealing with literal equal signs is one of them.
powershell "(gc \"%textFile%\") -replace '%search%','%replace%'"
would work. That PowerShell one-liner is a simple alternative to your for /f loop without that limitation.
If you prefer a for /F loop, if your text file is an ini-style file, try this:
#echo off & setlocal
set "searchItem=Pippo.K"
set "searchVal=5"
set "newVal=1"
set "textFile=test.txt"
>"outfile.txt" (
for /f "eol=; usebackq tokens=1* delims==" %%I in ("%textFile%") do (
if /I "%%~I"=="%searchItem%" (
if "%%~J"=="%searchVal%" (
echo %%I=%newVal%
) else echo %%I=%%J
) else (
if not "%%~J"=="" (echo %%I=%%J) else echo %%I
)
)
)
move /y "outfile.txt" "%textFile%"
Be advised that if any of the items in your file has a blank value (e.g. valuename=), the equal sign will be stripped unless you add some additional logic.
You might also consider using ini.bat from this answer.
I am trying to search through a text file for keywords, then insert a number of lines after a specific line/keyword (not end of file).
My code can find the keywords, however I am struggling to add the lines. My code adds the line to the end of the file, so the bit I need help with is after :ADD THE TEXT.
myfile.text looks like:
QFU;
text2;
LastUpdate=20180323;
text3;
I would like to add a list of static lines after LastUpdate, which makes the file look like:
QFU;
text2;
LastUpdate=20180323;
Inserted text1
Inserted text2
text3;
This is my code:
#echo
SET /A COND1=0
for /F "tokens=*" %%i in (myfile.txt) do call :process %%i
goto thenextstep
:process
set VAR1=%1
IF "%VAR1%"=="QFU" SET /A COND1=1
IF "%VAR1%"=="QFU" (
msg * "QFU line found !!"
)
:If QFU line is found then look for Last update
IF "%COND1%"=="1" IF "%VAR1%"=="LastUpdate" (
msg * "LastUpdate line found !!"
:ADD THE TEXT
echo. text to be added>>myfile.txt
:reset COND1 to 0
set /A COND1=0
)
#echo off
setlocal enabledelayedexpansion
call :get_insert_index
if not defined index (
>&2 echo index not defined.
exit /b 1
)
set "i=0"
(
for /f "tokens=*" %%A in (myfile.txt) do (
set /a "i+=1"
echo %%A
for %%B in (%index%) do if !i! equ %%B (
echo --- INSERT
)
)
) > myupdate.txt
exit /b
:get_insert_index
setlocal enabledelayedexpansion
set "i=0"
set "qfu="
set "total="
for /f "tokens=*" %%A in (myfile.txt) do (
set /a i+=1
set "line=%%~A"
if "%%~A" == "QFU;" (
set /a "qfu=!i! + 1"
) else if "!line:~,11!" == "LastUpdate=" (
if defined qfu (
if !i! gtr !qfu! (
if defined total (set total=!total! !i!) else set total=!i!
set "qfu="
)
)
)
)
endlocal & set "index=%total%"
exit /b
This will insert text after the 1st line starting with LastUpdate=,
after the line of QFU;, but not the line starting with LastUpdate=
which is the next line after QFU;.
The label :get_insert_index is called and uses a for loop
to read myfile.txt to get the line index of LastUpdate=
mentioned in the above paragraph.
The variable qfu stores the line index + 1 of QFU; so
LastUpdate= cannot be matched on the next line.
If gfu and LastUpdate= is found and the line index is
greater then gfu, then the line index is appended to total.
qfu is undefined to avoid further matches to LastUpdate=
until QFU; is matched again.
The loop will end and the global variable index is set the
value of total. The label returns control back to the caller.
index is checked if defined at the top of the script after
the call of the label.
The top for loop reads myfile.txt and echoes each line read.
The nested for loop checks the index variable to match the
current line index and if equal, will echo the new text.
The echoes are redirected to myupdate.txt.
Used substitution of "!line:~,11!" so view set /? for help.
Used enabledelayedexpansion so view setlocal /? for help.
Text using ! may find ! being interpreted as a variable
so avoid using !.
Used gtr which can be viewed in if /?. gtr is
"Greater than".
Alternative to avoid creation of an index:
#echo off
setlocal enabledelayedexpansion
set "i=0"
set "gfu="
for /f "tokens=*" %%A in (myfile.txt) do (
set /a i+=1
set "line=%%~A"
>> myupdate.txt echo(%%A
if "%%~A" == "QFU;" (
set /a "qfu=!i! + 1"
) else if "!line:~,11!" == "LastUpdate=" (
if defined qfu (
if !i! gtr !qfu! (
>> myupdate.txt echo --- INSERT
set "qfu="
)
)
)
)
exit /b
>> myupdate.txt echo(%%A writes each line.
>> myupdate.txt echo --- INSERT writes new line to insert.
If system memory permits based on file size, this is much faster:
#echo off
setlocal enabledelayedexpansion
set "i=0"
set "gfu="
(
for /f "tokens=*" %%A in (myfile.txt) do (
set /a i+=1
set "line=%%~A"
echo(%%A
if "%%~A" == "QFU;" (
set /a "qfu=!i! + 1"
) else if "!line:~,11!" == "LastUpdate=" (
if defined qfu (
if !i! gtr !qfu! (
echo --- INSERT
set "qfu="
)
)
)
)
) > myupdate.txt
exit /b
Used on 2.74 MB file, Time reduced from 70s to 21s. The write handle to myupdate.txt remains open for the entire loop, thus the write is cached.
I would like write a batch file to count the number of occurrences of a specific character in each line of a text file.
For example, the count of \ in the string "aa\bb\cc\dd\" would be 4.
The find and the findstr show only the number of lines which is contains the exact character.
You might try the following script, providing the input string as (quoted) command line argument:
set "STRING=%~1$"
set STRING="%STRING:\=" "%"
set /A "COUNT=-1"
for %%E in (%STRING%) do set /A "COUNT+=1"
echo Count of `\`: %COUNT%
This replaces every character to be counted by " + SPACE + " and encloses the entire string in between "", so the input string aa\bb\cc\dd\ becomes "aa" "bb" "cc" "dd" "". The resulting string is fed into a for loop that recognises individual items to iterate through -- five in this case. The counter variable COUNT is initialised with a value of -1, so the result is not the number of iterated items but the separators, namely the \ characters present in the original string.
This approach fails if the string contains ? or * characters. It would also fail in case the character to count is one of the following: ", %, =, *, ~.
#echo off
setlocal
set "string=aa\bb\cc\dd\"
set "count=-1"
for %%a in ("%string:\=" "%") do set /A count+=1
echo %count%
This method works correctly as long as the string don't include wild-card characters: *?; if this is required, I would use the same npocmaka's method, but written in a simpler way:
#echo off
setlocal EnableDelayedExpansion
set "string=aa\bb\cc\dd\"
set "str=A%string%Z"
set "count=-1"
for /F "delims=" %%a in (^"!str:\^=^"^
% Do NOT remove this line %
^"!^") do (
set /A count+=1
)
echo %count%
While slow, you can try with this
#echo off
setlocal enableextensions disabledelayedexpansion
set "inputFile=input.txt"
set "searchChar=\"
for /f "delims=" %%a in ('
findstr /n "^" "%inputFile%"
') do for /f "delims=:" %%b in ("%%~a") do (
set "line=%%a"
for /f %%c in ('
cmd /u /v /e /q /c"(echo(!line:*:=!)"^|find /c "%searchChar%"
') do echo Line %%b has %%c characters
)
The input file is readed using findstr /n to get all the lines in the file with a number prefix (both for output "decoration" and to ensure all the lines in the file are processed). Each line is processed inside a pipe, from cmd to find. The cmd instance is started with unicode output (/u) so when the readed line is echoed, the output will be a two bytes sequence for each input character, one of them a 0x0 ASCII character. The find command sees the 0 as a line terminator, so we get each character in the input line as one separated line. Now, the find command counts in how many lines the searched character happens.
#ECHO OFF
SETLOCAL
SET "String=a\b\c\\\\d"
CALL :count "%string%" \
ECHO %tally%
GOTO :EOF
:count
SETLOCAL enabledelayedexpansion
SET /a tally=0
SET "$2=%~1"
:cloop
SET "$1=%$2%"
SET "$2=!$1:*%2=!"
IF "%$1%" neq "%$2%" SET /a tally+=1&GOTO cloop
endlocal&SET tally=%tally%
GOTO :eof
Here's a way to count particular characters in a string. It won't work for the usual suspects.
here's one way:
#echo off
:checkCountOf string countOf [rtnrVar]
:: checks count of a substring in a string
setlocal EnableDelayedExpansion
set "string=aa"
set "string=%~1"
set "checkCountOf=%~2"
if "%~1" equ "" (
if "%~3" neq "" (
endlocal & (
echo 0
set "%~3=0"
exit /b 0
)
) else (
endlocal & (
echo 0
exit /b 0
)
)
)
if "!checkCountOf!" equ "$" (
set "string=#%string%#"
set "string=!string:%checkCountOf%%checkCountOf%=#%checkCountOf%#%checkCountOf%#!"
) else (
set "string=$%string%$"
set "string=!string:%checkCountOf%%checkCountOf%=$%checkCountOf%$%checkCountOf%$!"
)
set LF=^
rem ** Two empty lines are required
set /a counter=0
for %%L in ("!LF!") DO (
for /f "delims=" %%R in ("!checkCountOf!") do (
set "var=!string:%%~R%%~R=%%~L!"
set "var=!var:%%~R=%%~L!"
for /f "tokens=* delims=" %%# in ("!var!") do (
set /a counter=counter+1
)
)
)
if !counter! gtr 0 (
set /a counter=counter-1
)
if "%~3" neq "" (
endlocal & (
echo %counter%
set "%~3=%counter%"
)
) else (
endlocal & (
echo %counter%
)
)
you can call it like:
call ::checkCountOf "/aa/b/c/" "/" slashes
echo %slashes%
exit /b %errorlevel%
wont work with some special characters (",~ and !)
You can also use replacement and the :strlen function
Not tested extensively but works with your example.
#ECHO OFF
SETLOCAL disabledelayedexpansion
SET "String=\a\b\c\\\\d\\"
set "previous=%string%"
set /a count=0
:loop
set "newstg=%previous:*\=%"
IF NOT "%previous%"=="%newstg%" (
set /a count+=1
set "previous=%newstg%"
IF DEFINED previous goto loop
)
echo %count%
pause
GOTO :eof
Here is one more option. I don't think this is bullet proof with poison characters.
#ECHO OFF
SETLOCAL disabledelayedexpansion
SET "String=\\a\b\c\\\\d\\"
set i=0
set "x=%string%"
set "x=%x:\=" & set /A i+=1 & set "x=%"
echo %i%
pause
I have 2 CSV files that has
File1:
Column1,column2
data1, data2
File2:
Column3,column4, column5,column6
data3,data4,data5,data6
I have to create a new CSV file that combines both columns from file 1 with the 1st and 3rd columns from file 2 (4 total columns).
Column1,column2,column3,column5
data1,data2,data3,data5
I am looking to do this using batch file commands. Any suggestions?
Code i am using helps me copy one file.
#ECHO OFF
SETLOCAL
(
FOR /f "tokens=1-3delims=," %%a IN (file1.csv) DO (
ECHO(%%a,%%c
)
)>new.csv
GOTO :EOF
How about following script?
File1.csv :
column1,column2
data1,data2
data3,data4
data5,data6
File2.csv :
column3,column4,column5,column6
data3,data4,data5,data6
data7,data8,data9,data10
Script :
#echo off
setlocal enabledelayedexpansion
set ct1=0
for /f "tokens=*" %%i in (File1.csv) do (
set /a ct1+=1
set ar1[!ct1!]=%%i
)
set ct2=0
for /f "tokens=*" %%i in (File2.csv) do (
set /a ct2+=1
set ar2[!ct2!]=%%i
)
if !ct1! lss !ct2! (
set ct=!ct2!
) else (
set ct=!ct1!
)
for /l %%i in (1,1,!ct!) do (
echo !ar1[%%i]!,!ar2[%%i]!>> new.csv
)
new.csv :
column1,column2,column3,column4,column5,column6
data1,data2,data3,data4,data5,data6
data3,data4,data7,data8,data9,data10
data5,data6,
Here is a pure batch solution that works, but with the following limitations and or assumptions:
File 1 lines are terminated by carriage return and linefeed (Windows style)
File 1 lines are no longer than 1021 bytes
File 2 must have a value for each column (no consecutive commas)
File 2 line lengths never exceed ~8191 bytes
File 2 does not have any quoted column values that include commas.
Files 1 and 2 have the same number of lines
Neither file has quoted data values that include new lines (rare, but possible within a CSV).
#echo off
setlocal disableDelayedExpansion
<"file1.csv" >"merged.csv" (
for /f "usebackq eol=, delims=, tokens=1,3" %%A in ("file2.csv") do (
set /p "part1="
set "part2=%%A,%%B"
setlocal enableDelayedExpansion
echo !part1!,!part2!
endlocal
)
)
A much more robust and faster solution is possible if you use PowerShell, JScript, or VBS.
You can also implement an efficient and robust solution using JREPL.BAT - a regular expression text processing utility. JREPL.BAT is pure script (hybrid batch/JScript) that runs natively on any Windows machine from XP onward. Full documentation is available from the command line via jrepl /?, or jrepl /?? for paged help.
This JREPL solution has only the following reasonable limitations:
Files 1 and 2 must have the same number of lines
Neither file has quoted data values that include new lines
#echo off
setlocal
set "col=\q(?:\q\q|[^,\q])*\q|[^,]*"
call jrepl "^(%col%),(?:%col%),(%col%)(?:,.*|$)" ^
"$txt=stdin.ReadLine()+','+$1+','+$2" ^
/jq /x /f "file2.csv" /o "merged.csv" <"file1.csv"
This flexible script does what you want, given that the following restrictions are not violated:
both files must contain the same number of lines;
the number of columns per line/row must be equal per each file;
lines are no longer than 1023 bytes, including the terminating line-break;
field/cell values must not contain line-breaks;
each line/row must be terminated by Windows-style line-breaks (CR+LF);
the given column numbers to copy must be sorted in ascending order;
So here is the code:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
rem // Define constants here:
set "_FILE1=%~dp0File1.csv" & rem // (1st input CSV file; state `%~1` to use 1st arg.)
set "_FILE2=%~dp0File2.csv" & rem // (2nd input CSV file; state `%~2` to use 2nd arg.)
set "_COLS1=1,2" & rem // (ascending list of columns to copy from 1st file)
set "_COLS2=1,3" & rem // (ascending list of columns to copy from 2nd file)
set "_SEPAR=," & rem // (separator character, usually `,`)
rem // Main routine:
4< "%_FILE1%" 3< "%_FILE2%" (
call :READLINE
)
endlocal
exit /B
:READLINE
rem // Read a line of both files:
set "LINE1=" & set "LINE2="
<&4 set /P LINE1=""
<&3 set /P LINE2=""
rem // Terminate sub-routine in case both lines are empty:
if not defined LINE1 if not defined LINE2 exit /B
rem // Process lines:
call :PROCESS LINE1 LINE2
rem // Repeat reading:
goto :READLINE
exit /B
:PROCESS ref_string1 ref_string2
setlocal DisableDelayedExpansion
set "BUF=%_SEPAR%"
setlocal EnableDelayedExpansion
rem // Test both strings against global wild-card characters:
set "STR1=!%~1!" & set "STR2=!%~2!"
if "!STR1:**=!!STR2:**=!"=="!STR1!!STR2!" goto :PROCESS_CONT
if "!STR1:*?=!!STR2:*?=!"=="!STR1!!STR2!" goto :PROCESS_CONT
if "!STR1:*<=!!STR2:*<=!"=="!STR1!!STR2!" goto :PROCESS_CONT
if "!STR1:*>=!!STR2:*>=!"=="!STR1!!STR2!" goto :PROCESS_CONT
>&2 echo(ERROR: Illegal character encountered^^!
exit /B 1
:PROCESS_CONT
rem // Prepare line strings for being processed by a standard `for` loop:
set "STR1=!STR1:"=""!^"
set "STR2=!STR2:"=""!^"
set "STR1="!STR1:%_SEPAR%=","!""
set "STR2="!STR2:%_SEPAR%=","!""
rem // `for /F` loops to transport prepared line strings beyond `endlocal`:
for /F "delims=" %%E in (^""!STR1!"^") do (
for /F "delims=" %%F in (^""!STR2!"^") do (
endlocal
rem // Process 1st line string:
set /A "IDX=0"
for %%I in (%%~E) do (
rem // Compare column index of current item with given column list:
set /A "IDX+=1" & set "FND="
for %%J in (%_COLS1%) do (
setlocal EnableDelayedExpansion
if !IDX! EQU %%J (
endlocal & set "FND=#"
) else endlocal
)
rem // Matching column index encountered, so assemble output line:
if defined FND (
set "NEW=%%~I%_SEPAR%"
setlocal EnableDelayedExpansion
for /F "delims=" %%K in (^""!BUF!!NEW!"^") do (
endlocal
set "BUF=%%~K"
)
)
)
rem // Process 1st line string:
set /A "IDX=0"
for %%I in (%%~F) do (
rem // Compare column index of current item with given column list:
set /A "IDX+=1" & set "FND="
for %%J in (%_COLS2%) do (
setlocal EnableDelayedExpansion
if !IDX! EQU %%J (
endlocal & set "FND=#"
) else endlocal
)
rem // Matching column index encountered, so assemble output line:
if defined FND (
set "NEW=%%~I%_SEPAR%"
setlocal EnableDelayedExpansion
for /F "delims=" %%K in (^""!BUF!!NEW!"^") do (
endlocal
set "BUF=%%~K"
)
)
)
setlocal EnableDelayedExpansion
)
)
rem // Return output line buffer:
echo(!BUF:~1,-1!
endlocal
endlocal
exit /B
i have to search a string from a txt like Pippo.K=5 and replace it with Pippo.K=1. I need to search the entire string. What i did is:
set "search=Pippo.K=5"
set "replace=Pippo.K=1"
set "textFile=%SettingFile%.txt"
for /f "delims=" %%i in ('type "%textFile%" ^& break ^> "%textFile%" ') do (
set "line=%%i"
setlocal enabledelayedexpansion
set "line=!line:%search%=%replace%!"
>>"%textFile%" echo(!line!
endlocal
)
but what i returned is
5=Pippo.K=1=5
How can i fix this error?
The following script constitutes a pure batch-file solution. Supposing it is stored as repl-str.bat, you need to call it like this for your application:
repl-str.bat "%SettingFile%.txt" "Pippo.K=5" "Pippo.K=1" "%SettingFile%.txt"
This specifies the input file %SettingFile%.txt, the literal and case-sensitive search string Pippo.K=5, the replacement string Pippo.K=1 and the output file %SettingFile%.txt that is the same as the input file (the related technique has been taken from this answer: Batch script to find and replace a string in text file without creating an extra output file for storing the modified file). If no output file is given, the result is output to the console (useful for testing). If a fifth command line argument is given (arbitrary value), the search is done in a case-sensitive manner.
Here is the code of the script repl-str.bat:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set "FILE_I=%~1"
set "SEARCH=%~2"
set "REPLAC=%~3"
set "FILE_O=%~4"
set "CASE=%~5"
set "FLAG=%~6"
if not defined FILE_I exit /B 1
if not defined SEARCH exit /B 1
if not defined FILE_O set "FILE_O=con"
if defined CASE set "CASE=#"
if defined FLAG set "FLAG=#"
for /F "delims=" %%L in ('
findstr /N /R "^" "%FILE_I%" ^& break ^> "%FILE_O%"
') do (
set "STRING=%%L"
setlocal EnableDelayedExpansion
set "STRING=!STRING:*:=!"
call :REPL RETURN STRING SEARCH REPLAC "%CASE%" "%FLAG%"
>> "%FILE_O%" echo(!RETURN!
endlocal
)
endlocal
exit /B
:REPL rtn_string ref_string ref_search ref_replac case flag
setlocal EnableDelayedExpansion
set "STR=!%~2!"
set "SCH=!%~3!"
set "RPL=!%~4!"
if "%~5"=="" (set "OPT=/I") else (set "OPT=")
if not defined SCH endlocal & set "%~1=" & exit /B 1
set "SCH_CHR=!SCH:~,1!"
if not "%~6"=="" set "SCH_CHR="
if "!SCH_CHR!"=="=" set "SCH_CHR=" & rem = terminates search string
if "!SCH_CHR!"==""^" set "SCH_CHR=" & rem " could derange syntax
if "!SCH_CHR!"=="%%" set "SCH_CHR=" & rem % ends variable expansion
if "!SCH_CHR!"=="^!" set "SCH_CHR=" & rem ! ends variable expansion
call :LEN SCH_LEN SCH
call :LEN RPL_LEN RPL
set /A RED_LEN=SCH_LEN-1
set "RES="
:LOOP
call :LEN STR_LEN STR
if not defined STR goto :END
if defined SCH_CHR (
set "WRK=!STR:*%SCH_CHR%=!"
if %OPT% "!WRK!"=="!STR!" (
set "RES=!RES!!STR!"
set "STR="
) else (
call :LEN WRK_LEN WRK
set /A DFF_LEN=STR_LEN-WRK_LEN-1,INC_LEN=DFF_LEN+1,MOR_LEN=DFF_LEN+SCH_LEN
for /F "tokens=1,2,3 delims=," %%M in ("!DFF_LEN!,!INC_LEN!,!MOR_LEN!") do (
rem set "RES=!RES!!STR:~,%%M!"
if defined WRK set "WRK=!WRK:~,%RED_LEN%!"
if %OPT% "!STR:~%%M,1!!WRK!"=="!SCH!" (
set "RES=!RES!!STR:~,%%M!!RPL!"
set "STR=!STR:~%%O!"
) else (
set "RES=!RES!!STR:~,%%N!"
set "STR=!STR:~%%N!"
)
)
)
) else (
if %OPT% "!STR:~,%SCH_LEN%!"=="!SCH!" (
set "RES=!RES!!RPL!"
set "STR=!STR:~%SCH_LEN%!"
) else (
set "RES=!RES!!STR:~,1!"
set "STR=!STR:~1!"
)
)
goto :LOOP
:END
if defined RES (
for /F delims^=^ eol^= %%S in ("!RES!") do (
endlocal
set "%~1=%%S"
)
) else endlocal & set "%~1="
exit /B
:LEN rtn_length ref_string
setlocal EnableDelayedExpansion
set "STR=!%~2!"
if not defined STR (set /A LEN=0) else (set /A LEN=1)
for %%L in (4096 2048 1024 512 256 128 64 32 16 8 4 2 1) do (
if defined STR (
set "INT=!STR:~%%L!"
if not "!INT!"=="" set /A LEN+=%%L & set "STR=!INT!"
)
)
endlocal & set "%~1=%LEN%"
exit /B
Basically, this approach takes the first character of the search string and looks it up in the input text. At each match, it is checked whether the whole search string occurs. If so, it is replaced by the replacement string by removing as many characters as the search string consists of, hence avoiding sub-string replacement syntax which fails in case the search string contains =, or the search or the replacement string contains % or !.
However, if the first character of the search string is =, ", % or !, the approach is different, the script checks every single character position for occurrence of the search string then, with the disadvantage of reduced overall performance. If a sixth command line argument is given (arbitrary value), this (slow) mode is forced.
Batch variable substring substitution does have limitations. Dealing with literal equal signs is one of them.
powershell "(gc \"%textFile%\") -replace '%search%','%replace%'"
would work. That PowerShell one-liner is a simple alternative to your for /f loop without that limitation.
If you prefer a for /F loop, if your text file is an ini-style file, try this:
#echo off & setlocal
set "searchItem=Pippo.K"
set "searchVal=5"
set "newVal=1"
set "textFile=test.txt"
>"outfile.txt" (
for /f "eol=; usebackq tokens=1* delims==" %%I in ("%textFile%") do (
if /I "%%~I"=="%searchItem%" (
if "%%~J"=="%searchVal%" (
echo %%I=%newVal%
) else echo %%I=%%J
) else (
if not "%%~J"=="" (echo %%I=%%J) else echo %%I
)
)
)
move /y "outfile.txt" "%textFile%"
Be advised that if any of the items in your file has a blank value (e.g. valuename=), the equal sign will be stripped unless you add some additional logic.
You might also consider using ini.bat from this answer.