Batch Script that checks a directory for duplicate files - file

Question

Trying to look through a file for example,
C:\Users\Admin\Desktop\Test\12345
C:\Users\Admin\Desktop\Test\45635
C:\Users\Admin\Desktop\Test\12345-2018-04-21
and create a subfolder named "12345" and will store the two files (12345 and 12345-2018-04-21) in subfolder "12345". and this will loop through the whole folder to make sure there is no copies
:: ---------------------------------------------------------------
:: - This is a program that searches for files with the same 5
:: digit number and puts them in a subfolder.
:: ---------------------------------------------------------------
#ECHO off
SETLOCAL
SET "sourcedir=C:\Users\Admin\Desktop\Test"
SET "destdir=C:\Users\Admin\Desktop\Test"
SET /a lastnum=200000
if "%first5%" equ "%lastnum%"(
SET "destdir=%destdir%\%first5%-sub"
md %destdir%
)
FOR /f "delims=" %%a IN (
'dir /b /a-d /on "%sourcedir%\*" '
) DO (
CALL :detect "%%a"
IF DEFINED dupnum (ECHO(MOVE "%sourcedir%\%%a" "%destdir%\")
)
GOTO end
:: Routine to detect whether the first 5 characters of the filename "%1"
:: are all numeric and if so, whether they match the previous 5-digit number
:: found.
:detect
SET "dupnum="
:: Get the first 5 characters of the filename; prefix with a `1`
SET "fullfilename=%~1"
SET "first5=1%fullfilename:~0,5%"
IF "%first5%" neq "%lastnum%" (
SET "lastnum=%first5%"
GOTO end
)
:: First 5 chars of this filename = first 5 of previous filename
:: Check to see whether numeric
SET /a dummy=first5 + 0
IF "%dummy%" neq "%first5%" (
SET "lastnum=%first5%"
GOTO end
)
SET "dupnum=Y"
GOTO end
ENDLOCAL
:end
PAUSE
Not sure why i can not get it to make a folder and store the duplicated files into it. It will read through my test folder and notice the duplicates and prompt to move them but i am not sure what i need to do to move it to a new folder.

Answer 1

It's difficult to determine where to start if you've not encountered the language before.
The first issue is to properly specify what your terms mean. A "duplicate file" for instance would normally mean duplicate contents or a duplicate name (but since a directory can only hold one file with any particular name, this would imply different directories). In your case, your individual definition is "files whose names start with the same 5 digits".
Since this leads to a batch which uses many techniques (and there can be many approaches) then this is one that could be used:
#ECHO OFF
SETLOCAL
SET "sourcedir=U:\sourcedir"
SET "destdir=U:\destdir"
SET /a lastnum=200000
md %destdir% 2>nul
FOR /f "delims=" %%a IN (
'dir /b /a-d /on "%sourcedir%\*" '
) DO (
CALL :detect "%%a"
IF DEFINED dupnum (ECHO(MOVE "%sourcedir%\%%a" "%destdir%\")
)
GOTO :EOF
:: Routine to detect whether the first 5 characters of the filename "%1"
:: are all numeric and if so, whether they match the previous 5-digit number
:: found.
:detect
SET "dupnum="
:: Get the first 5 characters of the filename; prefix with a `1`
SET "fullfilename=%~1"
SET "first5=1%fullfilename:~0,5%"
IF "%first5%" neq "%lastnum%" (
SET "lastnum=%first5%"
GOTO :eof
)
:: First 5 chars of this filename = first 5 of previous filename
:: Check to see whether numeric
SET /a dummy=first5 + 0
IF "%dummy%" neq "%first5%" (
SET "lastnum=%first5%"
GOTO :eof
)
SET "dupnum=Y"
GOTO :eof
You would need to change the settings of sourcedir and destdir to suit your circumstances.
The required MOVE commands are merely ECHOed for testing purposes. After you've verified that the commands are correct, change ECHO(MOVE to MOVE to actually move the files. Append >nul to suppress report messages (eg. 1 file moved)
Fundamentally, batch code is case-insensitive with the notable exception of the metavariable in a for statement (%%a above)
Batch is however quite layout-sensitive, so best to cut-and-paste rather than retyping and attempting to re-format to suite some more-pleasing-to-the-eye style.
It's normally assumed that batch code is run from the prompt, not by 'clicking'.
The syntax SET "var=value" (where value may be empty) is used to ensure that any stray trailing spaces are NOT included in the value assigned. set /a can normally be used "quoteless".
The #echo off statement turns command-echoing off so that the command is not echoed to the console before being executed.
The setlocal statement invokes a 'local environment' so that any alterations or additions to the environment are disposed of when the batch finishes.
The main loop first executes a dir command to produce a directory list. The options chosen are /b - basic (no header/footer - names only) /a-d - no directorynames and /on - order-by-name (ie alphabetical)
The output of the dir command is then processed by a for /f with the "delims=" option. This assigns the entire line to the metavariable %%a.
For each filename found, the subroutine :detect is executed. Since that rouine is located within this batch file, a : is used in the call statement. Without the :, the routine called would be a batchfile named detect (either detect.bat or detect.cmd, the former being more popular)
After the routine :detect has been executed, a variable named dupnum may or may not be set. If it's set, echo the string. If not, don't.
The routine :detect first "set"s dupnum (the result of the routine) to an empty string, which removes the variable from the environment if it exists.
The variable fullfilename is then set to the value passed from the call statement. %1 means "first parameter". %~1 means "and remove enclosing quotes". The quotes are required to allow the filename being passed to contain spaces.
We then get the first 5 characters from the full filename (from "character 0" for 5 characters) and prefix this with a 1. Batch regards a string which starts with a 0 as an octal string if used in a mathematical operation, so the 1 prefix ensures the now 6 characters are interpreted as decimal.
Now - if the first5 is not equal to the lastnum, then we record the new lastnum and goto :eof which exits the subroutine. dupnum has not be set, so the calling loop will not show this filename.
Next, we only want to do the move if the first 5 (well, now 6) chars are all numeric. We can add 0 to the string in mathematical mode using a set /a which will either result in the same number (if these chars are all-numeric) or with a different number (up to the first non-numeric) if the "first5" contains a non-numeric.
So - test the result - if different, "first5" contains a non-numeric, so bail out with dupnum not set. If the same, then set dupnum so the filename is reported.
Please also note that the official remark instruction is rem, but :: (which is a broken label - one which can't be reached) is often used instead for remarks as it is easier to type and less intrusive to the eye. Note however that the :: style can't be used within code-blocks (statements grouped between parentheses)
I've added a new line, md ... which should create the destination directory. The 2>nul suppresses error messages should that directory already exist. If you want this directory to be a subdirectory of the source directory, then set destdir thus:
set "destdir=%sourcedir%\subdirectorynamedesired"