I have a batch script that will combine all .txt files with a few caveats, such as adding a comma in between each file and adding a square bracket at the start and end of the output file.
echo [ >> output.txt
for %f in (*.txt) do type "%f" >> output.txt & echo. >> output.txt & echo , >> output.txt
echo ] >> output.txt
What I would like to do it limit the output.txt to 10,000 txt files, whilst creating a new output.txt for the next 10,000 files. So for 25,000 records i will end up with;
Output1.txt (10,000 txt files)
Output2.txt (10,000 txt files)
Output3.txt (5,000 txt files)
How can I change my script to accommodate this?
Also if possible, I don't really want a comma at the end of the very last record that it combines. Is there a way to achieve this?
There are issues with your existing code:
A simple FOR loop will likely include your Output file(s). Obviously you don't want that. That could be prevented by writing the file list to a temp file, before any output is created. Easily done via DIR /B /A-D *.txt >tempFile
It takes significant time to repeatedly open the same output file thousands of times. Better (faster) to open it once, if possible.
Ideally, the final code should do most of the processing in some kind of FOR loop, with delayed expansion to enable working with changing values within the loop. Reading file names with a FOR loop can cause problems with delayed expansion because it will corrupt any name that happens to contain !. It takes a bit more code, but using SET /P to read the file is significantly faster, and delayed expansion doesn't cause problems.
Typically you must know the total number of lines in a file to detect the end when using SET /P. But in this case, the absence of data in a row indicates the end of file - There can never be any empty lines in DIR /B output.
The following code is untested, but if it does not work, then any fix should be minor.
#echo off
setlocal enableDelayedExpansion
dir /b /a-d *.txt >files.temp
set /a cnt=0
call :read <files.temp
del files.temp
exit /b
:read
set "file="
set /p "file="
if not defined file exit /b
set /a cnt+=1
call :write >output%cnt%.txt
goto :read
:write
echo [
type "!file!"
echo(
for /l %%N in (2 1 10000) do (
set "file="
set /p "file="
if not defined file goto :end
echo ,
type "!file!"
echo(
)
:end
echo ]
exit /b
Related
I am a complete novice when it comes to scripting, but am attempting to write a batch script which runs a command to output a png file to a printer. The script I have works fine for one file, but when there are multiple files it does not.
Can anyone point me in the right direction please?
#echo off
REM ___Change Directory to where Label Is Stored___
pushd C:\AFP\to
REM ___Create Variable to capture filename of any png file___
for /F %%a in ('dir /b *.png') do set FileName=%%~na.png
REM ___Now we have the filename as a variable, send it to printer using Zebra SSDAL___
\\172.16.100.2\nDrive\Prime_DPD_Label_Print\ssdal.exe /p "TSC DA200" send %FileName% >> C:\AFP\Log\Label_Printing_Log.txt
REM ___Copy PNG File to Backup Folder___
XCOPY /y /q /c C:\AFP\to\*.png C:\AFP\backup\
REM ___Delete PNF File from To Folder___
DEL C:\AFP\to\*.png
When the script runs, the first file prints fine. The subsequent files then do not print, I get "File does not exist" back from the ssdal.exe command. Why would the first one work but not the subsequent prints? I would have expected the for to loop through.
#ECHO Off
SETLOCAL
REM ___Change Directory to where Label Is Stored___
pushd C:\AFP\to
REM ___Process all png files___
for /F "delims=" %%a in ('dir /b *.png') do (
REM ___Now we have the filename as "%%a", send it to printer using Zebra SSDAL___
\\172.16.100.2\nDrive\Prime_DPD_Label_Print\ssdal.exe /p "TSC DA200" send "%%a" >> C:\AFP\Log\Label_Printing_Log.txt
IF ERRORLEVEL 1 (
CALL ECHO SSDAL returned ERRORLEVEL %%errorlevel%% FOR "%%a"
) ELSE (
REM ___Move PNG File to Backup Folder___
IF EXIST "c:\afp\backup\%%a" (
ECHO MOVE "%%a" to backup skipped as file already exists IN backup
) ELSE (
MOVE "%%a" C:\AFP\backup\
)
)
REM Two-second delay
TIMEOUT /t 2 >nul 2>nul
)
POPD
GOTO :EOF
Ah! using Zebra printers. Sensible lad!
This replacement script should do what you want.
The setlocal command is used to ensure that any variation made by this batch to the cmd environment is discarded when the batch ends.
The delims= sets "no delimiters" so for/f will set %%a to the entire filename, even if it contains spaces or other delimiters. Quoting %%a ensures such filenames are kept together as a single unit, not interpreted as separate tokens.
I'm assuming that ssdal acts responsibly and returns errorlevel non-zero in the case of errors. The if errorlevel 1 means if the errorlevel is currently 1 or greater than 1 and in that case, the error message is generated. We need to call echo ... %%varname%% ... in order to display the current value of the variable, if we're not using delayed expansion (many SO articles explain this)
Otherwise, if ssdal was successful, check for the existence of the filename in the backup directory, and either move it there or report that it already exists.
Of course, there are many ways in which this could be manipulated if features I've added are not desired. I'm happy to adjust this script to comply.
timeout is a standard utility to wait for a keypress. The redirection takes care of its prompting (it will provide a countdown unless gagged).
This question already has answers here:
Interleave files with CMD using echo
(1 answer)
How can two text files be read in parallel by a batch file?
(1 answer)
Closed 3 years ago.
I want to merge two .txt files following a pattern that uses one line from file1, adds a break and follows with another from file2, then repeat.
I've tried to solve this in notepad++' replace function, but it seems out of its scope completely. I've had mixed success using batch (only useful tool I'm familiar with) managing to make the .bat file past the lines separately with linebreaks, but I can't find information on how to select one line from each file alternating instead of the whole file at once.
This is the .bat file I have so far:
#Echo off
( for /f "delims=" %%A in (
'Type 1.txt^&Echo:^&Type 2.txt'
) Do #Echo:%%A
) > 3.txt
Current output format:
entire file1.txt
entire file2.txt
Intended output format:
single line from file1.txt
single line from file2.txt
repeat
I continue researching but I'm not seeing much that can point me in the right direction, all help appreciated.
Windows command processor cmd.exe interpreting and executing a batch file is not designed for text file processing. It is designed for execution of commands and executables. There are lots of other scripting languages which would be better for this task than cmd.exe.
However, this small batch file works for this task with some limitations.
#echo off
setlocal EnableExtensions EnableDelayedExpansion
set "LineNumber=0"
for /F usebackq^ delims^=^ eol^= %%I in ("file1.txt") do (
set /A LineNumber+=1
set "Line[!LineNumber!]=%%I"
)
set "LineNumber=0"
(for /F usebackq^ delims^=^ eol^= %%I in ("file2.txt") do (
set /A LineNumber+=1
call echo(%%Line[!LineNumber!]%%
echo(%%I
))>"file3.txt"
endlocal
The limitations are:
The number of non-empty lines must be equal in both input files.
The two input files do not contain empty lines which should be also written into output file as empty lines are ignored by FOR.
The lines in both input files do not contain exclamation marks as otherwise those lines are corrupted by enabled delayed expansion on double parsing of the command lines with %%I.
The first input file is not too large to be loaded into memory space for environment variables.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
call /?
echo /?
endlocal /?
for /?
set /?
setlocal /?
This will do it if the files aren't the same size. The batch file counts the lines in each file and compares the line counts. If one file has more lines than the other, the shorter one will output empty lines for missing lines in file with less lines.
Personally I would go this route then just fix the blank lines with PowerShell if I needed this script for myself.
#echo off
setlocal EnableExtensions DisableDelayedExpansion
pushd "\\server\folder"
set "InputFile1=File1.txt"
set "InputFile2=File2.txt"
set "OutputFile=mix.txt"
(set Newline=^
%==%
)
if not exist "%InputFile1%" goto EndBatch
if not exist "%InputFile2%" goto EndBatch
del "%OutputFile%" 2>nul
for /F "delims=:" %%I in ('%SystemRoot%\System32\findstr.exe /R /N "^" "%InputFile1%"') do set "LineCount1=%%I"
for /F "delims=:" %%I in ('%SystemRoot%\System32\findstr.exe /R /N "^" "%InputFile2%"') do set "LineCount2=%%I"
rem Exchange input file 1 and 2 if input file 2 has more lines than input file 1.
if %LineCount2% GTR %LineCount1% set "InputFile1=%InputFile2%" & set "InputFile2=%InputFile1%"
(
for /F usebackq^ delims^=^ eol^= %%I in ("%InputFile1%") do (
set "LineFile1=%%I"
setlocal EnableDelayedExpansion
set /P LineFile2=
echo(!LineFile1!!Newline!!LineFile2!>>"%OutputFile%"
endlocal
)
)<"%InputFile2%"
:EndBatch
popd
endlocal
pause
Lines starting with a semicolon and containing an exclamation mark are also processed well by this script. Just empty lines in the two input files are ignored and cause unexpected content in output file.
[Mofi] made some edits to this version.
If you wanted to blank the lines with PowerShell like I suggested then save this as clearlines.ps1 and then run it from your batch file at the end.
$pathToFile = "\\server\folder"
(Get-Content -Path $pathToFile) -notmatch '(^[\s,-]*$)| (rows\s*affected)' | Set-Content -Path $pathToFile
On Windows 10 I have this .bat:
#echo off
for /f "delims=" %%i in (filelist.txt) do (
echo %%~nxi >> output.txt
type "%%~ni*" >> output.txt
echo. >> output.txt
echo. >> output.txt
)
Exit
Now what this does is:
reads filelist.txt, which contains names of .txt files like:
20180808173105 (without ".txt"
searches for those files: 20180808173105.txt
copies name of files (without ".txt") into output.txt
inserts content of files
inserts two blank lines
repeats whole process for all files named in filelist.txt
--> It works fine! (or do you see any exception where this might malfunction?)
This inserts the full contents of a text file according to a list.
Can I modify it, so
not the whole content of a .txt, but only a part of it is inserted?
For example, everything from just after "title:" to just before "<!--"
if the filelist had a hierarchical structure (outline), it could
be preserved, like so:
#201508081213
###201609101219
to
#201508081213
TEXT
###201609101219
TEXT
I am using this to convert Outlines (using only the file names) to a rough first draft of text for writing articles and blogs
#echo off
2> output.txt echo.
#>&3 (
echo Debug Information
echo -----------------
)
for /f "delims=" %%A in (filelist.txt) do (
for %%B in ("%%~nA*") do call :read "%%~B"
) >> output.txt
exit /b
:read
setlocal
set "line="
echo(%~nx1
for /f "usebackq delims=" %%C in ("%~1") do (
set "line=%%C"
for /f "tokens=*" %%D in ('call echo "%%line:~0,4%%"') do (
#>&3 echo File: "%~1" Test: %%D == "<!--"
if %%D == "<!--" (
#>&3 echo Found: "<!--"
echo.
echo.
exit /b 0
)
)
call echo(%%line%%
)
echo.
echo.
exit /b 0
Note
Not the best language for this task. Had to avoid
enabledelayedexpansion as to known use of ! in <!--.
Used call even though < and some other characters
could cause issue.
for /f loops do not normally process empty lines so
hope that is not a problem.
Hierarchical structure depends on the order in filelist.txt.
The structure of a document can vary and I cannot consider
what the correct order might be. The use of a wildcard gives
some doubt. A filename #a will find #a1 and #a2 so
placement of 3 headings is unknown.
I have left in the std stream 3 messages for your
understanding of operation with the <!-- test.
Std stream 1 is the output that goes to file.
Operation
The file output.txt is erased before echoing text to the file.
The 1st for loop reads each line of the filelist.txt file.
The nested for loop gets the filenames with a wildcard.
Each call to label :read passes the argument of a filename.
In the label of :read, the filename is echoed.
A for loop reads lines from the filename.
Each line is stored in variable named line.
The nested for loop will use call echo to expand
the variable line and echoes the 1st 4 characters.
A comparison is done to test if it is <!-- and if so,
echo 2 newlines and then the label is exited.
If not, echo each line. echo 2 newlines is done at
the end of the label before exiting.
I have a list of files named:
file.txt
file (1).txt
file (2).txt
etc.
Where the greater (number) is the last file updated.
I want a .bat script that allows get the content of file (maxnumer).txt to file.txt.
dir /B /OD /TW file*.txt lists file names in sort order of last written time (cf. dir /?).
Next commented batch script could do the job for you:
#echo OFF
SETLOCAL EnableExtensions
rem delete empty `_lastfile` variable
set "_lastfile="
rem store file name of last written time into `_lastfile` variable
for /F "delims=" %%G in ('dir /B /OD /TW file*.txt 2^>NUL') do set "_lastfile=%%~G"
rem ↑↑↑↑↑↑ suppress errors
rem check the `_lastfile` variable
rem is defined?
rem AND is different from "file.txt"?
if defined _lastfile if /I "%_lastfile%" NEQ "file.txt" copy /Y "%_lastfile%" "file.txt"
Resources (required reading):
(command reference) An A-Z Index of the Windows CMD command line
(additional particularities) Windows CMD Shell Command Line Syntax
(%~G etc. special page) Command Line arguments (Parameters)
(2>NUL etc. special page) Redirection
Here is a bit of a hacky answer.
This script will move all files from file (1).txt up to file (10).txt to file.txt , leaving only file.txt which now contains the text that was in file (10).txt
Doing it in ascending order will ensure the highest number that exists is the last one to be moved.
#echo off
set /P name=Enter name of file without extension:
echo enter extension:
set /P ext=.
echo. & echo.
echo these actions will be performed: & echo.
FOR /L %%A IN (1,1,10) DO #echo move /y "%name% (%%A).%ext%" "%name%.%ext%"
echo. & pause & echo.
FOR /L %%A IN (1,1,10) DO move /y "%name% (%%A).%ext%" "%name%.%ext%"
pause
You could use IF EXIST %name% (%%A).%ext% To stop the script from trying to move files that don't exist, but it doesn't really affect anything to do that, so I didn't bother.
The script above will do it for a specific named file. To do it for all files in a directory will be possible, here are some hints to get you going
use a dir /b >filenames.txt to get all files in a directory listed in a text file
to perform an action for every line in a textfile do
for /f "usebackq delims= tokens=* %%a in (`filenames.txt`)" DO (
::some stuff here
)
The way I would go about it would be to get filenames.txt , manually delete all the (n) files so you just have a list of the "non-duplicate" filenames, and use that as your input. You
There are probably more elegant solutions out there, but with all the peculiarities of batch I wouldn't be surprised to find they are pages and pages long.
If you want to keep all the original files not just end up with the final file.txt with no copies, then I you want to use COPY
If you want to keep all the original files, then you would want to use COPY not MOVE.
In that case, to remove all superfluous operations (i.e. only copy the highest numbered file, not copy all the files in order) then something like IF NOT EXIST %name% (!B!).%ext% where !B!=%%A+1 within your (now multiline) for loop and use Setlocal EnableDelayedExpansion on to make the arithmetic work properly. But it's not really necessary, copying 1, then 2, then 3, then 4 does the same thing if a little slower than skipping 1 2 and 3 and just doing 4.
I hope this helps enough to point you in the right direction, feel free to ask questions if this isn't clear.
I have a requirement to, within a windows batch file, read the first available line from a text file, pass it to a variable and mark the name\line as used
An example of the file is below.
apple
pear
orange
The script would start with 'apple', pass 'apple' to a variable to be used later in the script (I know how to do that bit), and then write back that line to read &apple, the '&' works as a marker to say it's been used.
The file would then look like:
&apple
pear
orange
the next time the batch file is run it would take 'pear', pass it to a variable and mark it with a & making it look like:
&apple
&pear
orange
I started by trying to find '&' and then trying to move to the next line, but I'm failing after about 12 hours of trying. This is what I got so far .. not much:
for /f "tokens=1" %l in ('name.txt') do (Find /v "&" /v "^---- ^$") (For /F %n in (%l) do (set NewName=%n))
Thanks
Running this on the.file would modify each line in turn;
#echo off
setlocal enabledelayedexpansion
type nul > the.file.temp
set last=
for /F "tokens=*" %%A in (the.file) do (
set line=%%A
if "!line:~0,1!" neq "&" if "!last!" equ "" (
set last=!line!
set line=^&!line!
)
echo !line! >> the.file.temp
)
echo last value is !last!
type the.file.temp > the.file
(If the line does not begin with & and the variable last is empty, put the line in last & modify line with a leading &. Always append line to a temp file, renaming when done)
Alex k. has a good answer that is probably fine for most situations. (I upvoted.)
However, it will corrupt any text containing !. That limitation can be fixed by toggling delayed expansion on and off within the loop.
The solution is likely to be fast enough for most reasonably sized files. But a FOR loop can become quite slow for large files.
I tested a 190kb file containing 2817 lines, and the Alex K. solution took 20 seconds for one run.
Here is a completely different solution without using any loops that processes the same 190kb file in 0.07 seconds - 285 times faster :)
#echo off
setlocal enableDelayedExpansion
set "file=test.txt"
findstr /bv "$ &" "%file%" >"%file%.available"
set "var="
<"%file%.available" set /p "var="
if defined var (
>"%file%.new" (
findstr /b "&" "%file%"
<nul set /p "=&"
type "%file%.available"
)
move /y "%file%.new" "%file%" >nul
)
del "%file%.available"
echo var=!var!
Update: As requested in comment, here is a heavily commented version of the code.
#echo off
setlocal enableDelayedExpansion
:: Define the file to process
set "file=test.txt"
:: Write the unused lines to a temporary "available" file. We don't want any
:: empty lines, so I strip them out here. There are two regex search strings;
:: the first looks for empty lines, the second for lines starting with &.
:: The /v option means only write lines that don't match either search string.
findstr /bv "$ &" "%file%" >"%file%.available"
:: Read the first available line into a variable
set "var="
<"%file%.available" set /p "var="
:: If var defined, then continue, else we are done
if defined var (
REM Redirect output to a "new" file. It is more efficient to redirect
REM the entire block once than it is to redirect each command individulally
>"%file%.new" (
REM Write the already used lines to the "new" file
findstr /b "&" "%file%"
REM Append the & without a new line
<nul set /p "=&"
REM Append the unused lines from the "available" file. The first appended
REM line is marked as used because of the previously written &
type "%file%.available"
)
REM Replace the original file with the "new" content
move /y "%file%.new" "%file%" >nul
)
:: Delete the temp "available" file
del "%file%.available"
:: Display the result
echo var=!var!
I haven't tested this, but I just realized I could have written the line that writes the available lines to look for lines that start with a character other than &:
findstr "^[^&]" "%file%" >"%file%.available"