Concatenating files in multiple directories from batch file - batch-file

I'm trying to concatenate files from multiple directories. From a single directory, I know that you can execute
copy /B *.blah all.blah
to concatenate the files with extension .blah to a single file named all.blah. What my structure looks like is:
level 1/
level 2_1/
file_1.blah
file_2.blah
level 2_2/
...
level 2_3/
...
do_not_include_this_directory/
...
What I'm looking to do is create a single all.blah file in the top level directory which is a concatenation of all the .blah files in the level* sub-directories, without including any files from the do_not_include_this_directory directory.
My aim is to do this in a batch file (there will be other file concatenation logic for different directories included in this batch file), but I've spent the hour past playing around with cmd for logic to no avail (some of my directories have spaces in the names). Maybe this is something that I should just do using a python script? I'm thinking that this can be done relatively easily though using some for loops with copy, but my skills with these things are lacking to say the least (just came across cmd for about 2 hours ago).
Does anyone know how to do this, or would you just recommend that I buck up and write something using Python? Any help or suggestions would be appreciated.

It can be done easily from the command line without a batch script :)
copy nul all.blah >nul&for /d %F in (level*) do #copy /b all.blah + "%F\*.blah" >nul
As a batch script
#echo off
copy nul all.blah >nul
for /d %F in (level*) do copy /b all.blah + "%F\*.blah" >nul
I'm not sure if the /B switch is exactly correct. It has different meaning depending on where it appears: before any file, after a source, or after the destination.

This seems suitably horrific for a Windows batch file. Tested under Windows 7; YMMV, etc.
#rem get all pathnames, even in excluded directories
#rem EDIT THIS COMMAND to change wildcard to match
dir /b/s *.c >files.tmp
#rem get rid of things with prefix we want to exclude
#rem EDIT THIS COMMAND to change prefix
findstr /V "C:\temp\fee" files.tmp >files2.tmp
del copy.tmp
#rem append things one at a time the hard way!
For /f tokens^=*^ delims^=^ eol^= %%a in (files2.tmp) do (
copy "%%a" + copy.tmp copy2.tmp
del copy.tmp
rename copy2.tmp copy.tmp
echo.%%a)
#rem clean up
del copy2.tmp
del files.tmp
del files2.tmp

Related

Setting up Source and Destdir in a batch

for /L %%f in (1,1,10) do copy File1.txt %%f.txt
this code does the job very well, but I'm trying to understand how to change it to make it read
subfolders, so that I don't have to keep moving the batch file to every folder
I saw this, but not really sure how to put it together
#echo off
SET "sourcedir=C:\Users\user\Desktop\Main\Original"
SET "destdir=C:\Users\user\Desktop\Main\Copied"
for /L %%f in ('dir /s 1,1,10) do copy *.txt %%f.txt
in the section - copy *.txt %%f - I put a * so that it can only look for .txt files, but this action
slows down the coping and then stops working.
I know my code is a mess, just trying to put something together
I have many Subfolders and each folder has 1 txt file in it with all random names
and I need to make multiple copies of each file.txt in each folder
I have so many subfolders that it would literally take me months of time to get all files copied
and by then I would have many more new files to work on
so getting this copier to read Subfolders is like top priority for me.
I would like help putting this together and then explaining how it links
because I'm interested in applying the Set and dir to other batch file I have
Please any details on this will be much appreciated
I was told to look into xcopy and robocopy, but I have no idea were to add a counter
#echo off
for /1 %f in (1,1,10) do xcopy "C:\Sources" "C:\Target" /c /d /i /y
exit
So I have this that reads from source and dumps in main folder where the batch is
Option 1)
for /L %%f in (1,1,10) do xcopy "C:\Source Address\*.txt" %%f.txt
Option 2)
for /L %%f in (1,1,10) do xcopy "C:\Source Address\" "C:\Destination Address\ %%f.txt"
The thing I don't like is that is asks me a question
and I have over 10,000 txt files, I can't sit here and press F for Filename
10,000 times, can we disable that
OK, so I got this working
all I need help with is were to add this /c /d /i /y
I am still trying to get it to read Subfolders with the batch sitting
in the main folder and me not having to move files back and forth
Option 3)
for /L %%f in (1,1,110) do copy "C:\Source Address\*.txt" %%f.txt`
This works well with the Source and the wild card #magoo told me to add
the Source before the .txt file
But with this code I would still have to open hundreds of folders and move
the file to the source run the copier and then move back all copied files
Still need help with Subfolders
for /L %%f in (1,1,10) do copy File1.txt %%f.txt
will vary %%f from 1 (the first number in the parenthesised list) to 10 (the last) in steps of 1 (the middle) and will therefore copy file1.txt to 10 separate files, 1.txt to 10.txt.
Since there are no paths specified, the copy will be performed from file1.txt in the current directory to 1.txt .. 10.txt in the current directory.
If you were to put your batch file in any directory that is mentioned in the path variable (just execute path at the prompt to show it) then no matter where the current directory is, you could just use that batch filename to execute the batch, and the files would be created by copying file1.txt in the now-current directory to 1.txt .. 10.txt in the now-current directory - if file1.txt exists in the now-current directory, and if not, it will generate error messages reporting that the source file is missing.
If you were to replace file1.txt in the batch with "x:\wherever\file1.txt" then the file would be copied specifically from the file "x:\wherever\file1.txt" regardless of whether file1.txt exists in the now-current directory. (And please get used to "quoting file or pathnames" as it will avoid a whole slough of problems when you tackle names containing spaces and some other special characters).
I have no idea how for /L %%f in ('dir /s 1,1,10) do is supposed to work since according to Hoyle, the first element in the parenthesised list should be a number. I'd suggest that we have a small transcription problem here.
Th slower and slower problem - yes, understandable. You are creating more and more .txt files, and copying all of them to a new destination...
Perhaps referring to This response and question might show how to solve your subdirectory-scan issue.

Delete semi duplicate files

I'm wondering if there is a way to remove semi-duplicate files (name based) using a batch file or any other means (freeware utility) in Windows?
To give an example I have following files in a directory:
fileNameXXX(aaa).ext
fileNameXXX(bbb).ext
In this case, I only want to keep the fileNameXXX(bbb).ext
it's a single line in batch:
for /f "delims=" %%f in ('dir /b "*(*).ext" ^| find /v "(ddd)"') do ECHO del "%%f"
For every file matching the filemask excluding files with (ddd) do: delete it.
Remove the ECHO if the output fits your needs.
Note: if you want to use it directly on command line (instead in a batch file), replace every %%f with %f.
Tip: think about using some more code to check, if there is a Dutch version, and if not, keep the English one (or whatever you prefer).

Batch: Preserve (creation-)date on copying files to another (flatten) folder structure, incl. renaming files to avoid doublettes

this is my first question, so I apologize beforehand if I write not as you are used to...
fact:
I've a deep folder structure with tons of files (images,videos and so on) so I want to copy that files to a flat structure for a better overview.
I want to keep (at least) the original date attributes: creation-date and last-modified date.
Problem 1) there are files with same name 00001.jpg in different folders which I want to have in same folder, so I want to add creation date/time to filename on copy process.
00001.jpg becomes 2015-11-17_11-23-35_00001.jpg
So far so good. Or not good...
Copy and XCopy doesn't give me an option to do that, without loosing at least the creation date information (I didn`t find a solution with both).
Now I try to copy the files (file by file) with robocopy to new folder and use ren on the copied file to "pre-set" the date/time information before the filename.
Here is a simple test.bat example:
setlocal enableextensions enabledelayedexpansion
robocopy . ./zzz original.txt /copy:DATSO
pause
rem :: formatted creation date of original file will be here, in real life
set "myDate=2015-11-17-_11-23-35"
rem rename "./zzz/original.txt" "!myDate!_renamed.txt" ::doesnt work: why? relative path??
rem :: this will do what I want - original creation date is kept on copy file
FOR %%A IN (zzz/original.txt) DO REN "%%~fA" "!myDate!_%%~nxA"
[possibly] Problem2) Is there a better way to do this, or could I run into thread problems (asynchronous execution). Could it be, that I try to rename a file before the robocopy has finished the copy process (e.g. for large files)?
Sorry I'm a totally batch newbie (also as poster in SO ;).
ThanX in advance for each tip and also for critics on my solution approach. Maybe I have the horse-blinkers on my head and dont see the easy solution?!
[edit: formatting of post]
[edit: content of post -> date/time in front of filename for better sorting]
It is possible to use command DIR to get recursive listed all files in the specified folder and its subfolders with creation date/time.
The format of the date/time depends on Windows Region and Language settings.
Example output for F:\Temp\Test on my machine with English Windows 7 and region is configured to Germany on running the command line dir F:\Temp\Test\* /A-D /S /TC:
Volume in drive F is TEMP
Volume Serial Number is 1911-26A4
Directory of F:\Temp\Test
25.09.2017 17:26 465.950 SimpleFile.ccl
1 File(s) 465.950 bytes
Directory of F:\Temp\Test\Test Folder
25.09.2017 17:26 360.546 Test File.tmp
1 File(s) 360.546 bytes
Total Files Listed:
2 File(s) 826.496 bytes
0 Dir(s) 58.279.460.864 bytes free
This output is filtered with findstr /R /C:"^ Directory of" /C:"^[0123][0123456789]" to get only lines starting with  Directory of (note the space at begin) or with a number in range 00 to 39.
Directory of F:\Temp\Test
25.09.2017 17:26 465.950 SimpleFile.ccl
Directory of F:\Temp\Test\Test Folder
25.09.2017 17:26 360.546 Test File.tmp
And this filtered output is processed by FOR loop and the commands executed by FOR.
#echo off
for /F "tokens=1-2*" %%A in ('dir "F:\Temp\Test\*" /A-D /S /TC ^| %SystemRoot%\System32\findstr.exe /R /C:"^ Directory of" /C:"^[0123][0123456789]" 2^>nul') do (
if "%%A %%B" == "Directory of" (
set "FilePath=%%C"
) else (
set "CreationDate=%%A"
set "CreationTime=%%B"
for /F "tokens=1*" %%I in ("%%C") do set "FileName=%%J"
call :RenameFile
)
)
goto :EOF
:RenameFile
set "NewName=%CreationDate:~-4%-%CreationDate:~3,2%-%CreationDate:~0,2%_%CreationTime:~0,2%-%CreationTime:~3,2%_%FileName%"
ren "%FilePath%\%FileName%" "%NewName%"
goto :EOF
It would be advisable to put command echo before command ren in last but one line to first verify the expected new file names.
ren "F:\Temp\Test\SimpleFile.ccl" "2017-09-25_17-26_SimpleFile.ccl"
ren "F:\Temp\Test\Test Folder\Test File.tmp" "2017-09-25_17-26_Test File.tmp"
Note: The batch file must be in a folder not processed by this batch file as otherwise the batch file itself would be renamed while running which breaks the processing of the batch file.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
call /?
dir /?
echo /?
for /?
goto /?
if /?
ren /?
set /?
By the way: WinRAR can add files into a RAR archive with creation and last access time in addition to last modification time and extract the files to a different directory with restoring also creation and last access time using SetFileTime function of Windows kernel.
currently I use Locale independent date. I use tokens from that for currrent date/time.
for /f "tokens=2 delims==" %%I in ('wmic os get localdatetime /format:list') do set datetime=%%I
rem :: format it to YYYY-MM-DD_hh-mm-ss
set myDateTime=%datetime:~0,4%-%datetime:~4,2%-%datetime:~6,2%_%datetime:~8,2%-%datetime:~10,2%-%datetime:~12,2%
Thats not the problem.
To clarify:
The listing is also not the problem.
I loop throw all related files without a prob (except batch files and output dir and its sub tree ;).
My problem is:
I call robocopy for each file, then I rename the file to save the original creation date. My fear is that it makes problems (Multi-Threading?) for large files and for the number of calls (many thousend times)
Is batch execution really serialized? Is the process waiting for robocopy, that it has finished, before I try to rename file. Could I run into dead-locks for vry large files? I'll test it with some fake gigabyte-files.
Your suggestion, to use winrar sounds interesting.
If I could add all that files to a big archive (with structure) and at the end extract it to target dir... I'll try it ;)
If it doesn't work I will program it in java!
There I know what to do, thats my playground :D
I thought it would be easy to write a simple batch file, to do this for me, but it seems it's not as easy as I thought!
ThanX anyway

Windows batch way to replace all files in subdirectories with singular file (copy, rename all files)

I have a good command over cmd commands, but this may require a variable or a loop which is where I fail in batch commands. Please help if you can!
-- Have about 100 subdirectories each has 1-20 HTML files in it. There are about 100 HTML files in the root directory too.
-- Need to replace all HTML files in the above directories with the same HTML source (copy over existing file and keep the name the same). Basically trying to replace all existing files with a redirect script to a new server for direct bookmarked people. We are running a plain webserver without access to server-side redirects so trying to do this just by renaming the files (locked down corp environment).
Seems pretty simple. I can't get it to work with wildcards by copying the same file over to replace. I only get the first file replaced, but the rest of the files will fail. Any one with any advice?
This should do it from the command prompt. Replace % with %% for use in a batch file.
for /r "c:\base\folder" %a in (*.html) do copy /y "d:\redirect.html" "%a"
Without knowing more precisely how you want to update the file content I suggest the following rough approach.
To re-create your example, I had to create some folders. Run this command to do that:
for /l %i in (1,1,20) do mkdir fold%i
I then used this script to create some example files:
#echo off
set number=0
for /d %%i in (c:\Logs\htmltest\*) do call :makefiles %%i
goto :EOF
:makefiles
set /a number+=1
touch %1\file%number%.txt
echo %number% >%1\file%number%.txt
I then used this script to append the text changed to the file. Not sure if that is what you wanted - probably you need something more sophisticated.
#echo off
set number=0
for /d %%i in (c:\Logs\htmltest\*) do #for %%f in ("%%i\*.txt") do call :changetext %%f
goto :EOF
:changetext
echo changing file contents to ^"changed^" for file: %1
echo changed>>%1

How do I copy a directory that has a date stamp

I'm trying to copy the contents of a directory using a DOS batch file that begins with the computer name followed by an underscore and a date stamp. My first impulse was some variation of:
copy D:\%Computername%_\*\\*.* C:\WhateverPath
Of course I could not get this to work. Seems like a simple problem but I don't have much experience with batch files or DOS.
Try:
FOR /d %d IN (D:\%COMPUTERNAME%_*) DO xcopy %d C:\WhateverPath /E
This iterates over all directories (hence the /d) with the pattern %COMPUTERNAME%_* under D:\, and copies the contents of these directories into C:\WhateverPath. /Eis for copying all files and directories, also the empty ones.
For documentation of xcopy, type xcopy /? in a DOS shell (cmd).
Note: If you put this in a batch-file (something.bat), you must replace %d with %%d in the code above.
If you have multiple folders labeled C:\%computername%_%random_time_stamp%\ and you need to access each of them then move all of their contents to a single folder, you can do this:
Given the only underscore in the path is the one between %computername% and your timestamp
FOR /F "USEBACKQ tokens=*" %%F IN (`DIR /b /a:d "C:\" ^| FIND /I "%computername%_"`) DO (
COPY /y "%%~fF\*" "C:\WhateverPath\"
)
That states for every result that comes from the command DIR, /b switch meaning no header information, /a:d meaning only returning directories, I want to find only folders with the computername_ in it, and I want to copy the contents of each of those folders to C:\WhateverPath\ folder.

Resources