How to stop after the script created a number of folders and choose files randomly? - batch-file

Question

#Mofi helped me a lot on this answer of mine
How to organize files in folders?
Now I tried myself to add a feature, it's working, but for the respect of this wonderful job Mofi did, I think, there is a more professional way to do it.
I need the batch to stop after it created a number of folders.
Let's say, I have 100000 .mp4 files in the main directory. The script processes now them all together.
For example, I need to use only 1200 files and group them into 120 folders. I can do that now with FoldersInFolders=120 and FilesInFolder=10. The issue is that after finishing Canal-1 it continues with Canal-2 to Canal-10, etc. until all 100000 MP4 files are moved into directories and I don't want that. I want to stop after 120 folders overall and process only 1200 from 100000 MP4 files which I have in my main folder.
I added the following two lines:
set "MaxTotalFolders=120"
if "!VideosIndex!"=="%MaxTotalFolders%" exit /b 0
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set "FilesInFolder=10"
set "FoldersInFolders=120"
set "MaxTotalFolders=120"
for %%# in ("%USERPROFILE%\Videos" "D:\Videos" "F:\Temp\Videos" "\\MyNAS\Videos") do (
pushd %%# 2>nul
if not errorlevel 1 (
if not exist "*!*.mp4" (
setlocal EnableDelayedExpansion
set "FileCount=%FilesInFolder%"
set "CanalIndex=0"
set "VideosIndex=%FoldersInFolders%
for /F "eol=| delims=" %%I in ('dir *.mp4 /A-D /B /ON 2^>nul') do (
if !FileCount! == %FilesInFolder% (
set FileCount=0
if !VideosIndex! == %FoldersInFolders% (
set /A CanalIndex+=1
set VideosIndex=1
) else set /A VideosIndex+=1
set "TargetFolder=Canal-!CanalIndex!\Videos-!VideosIndex!"
md "!TargetFolder!" 2>nul
if "!VideosIndex!"=="%MaxTotalFolders%" exit /b 0
)
move /Y "%%I" "!TargetFolder!\" >nul
set /A FileCount+=1
)
endlocal
) else (
echo/
echo ERROR: Moving video files not possible because of file names with ! in name.
echo/
echo Please rename first the following files(s^) in: %%#
echo/
dir *!*.mp4 /A-D /B
echo/
pause
)
popd
) else (
echo/
echo ERROR: Failed to change to directory: %%#
echo/
pause
)
)
endlocal
Another feature that would be very helpful for me is to choose the MP4 files randomly every time I run the batch file.

Answer 1

The first batch file is without random selection of the MP4 files.
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set "FilesInFolder=10"
set "FoldersInFolders=50"
set "MaxTotalFolders=120"
for %%# in ("%USERPROFILE%\Videos" "D:\Videos" "F:\Temp\Videos" "\\MyNAS\Videos") do call :MoveVideos %%#
goto EndBatch
:MoveVideos
pushd %1 2>nul
if errorlevel 1 (
echo/
echo ERROR: Failed to change to directory: %1
echo/
pause
goto :EOF
)
if exist "*!*.mp4" (
echo/
echo ERROR: Moving video files not possible because of file names with ! in name.
echo/
echo Please rename first the following files(s^) in: %1
echo/
dir *!*.mp4 /A-D /B
echo/
popd
pause
goto :EOF
)
setlocal EnableDelayedExpansion
set "FileCount=%FilesInFolder%"
set "CanalIndex=0"
set "FolderCount=-1"
set "VideosIndex=%FoldersInFolders%
for /F "eol=| delims=" %%I in ('dir *.mp4 /A-D /B /ON 2^>nul') do (
if !FileCount! == %FilesInFolder% (
set FileCount=0
if !VideosIndex! == %FoldersInFolders% (
set /A CanalIndex+=1
set VideosIndex=1
) else set /A VideosIndex+=1
set /A FolderCount+=1
if !FolderCount! == %MaxTotalFolders% goto EndMove
set "TargetFolder=Canal-!CanalIndex!\Videos-!VideosIndex!"
md "!TargetFolder!" 2>nul
)
move /Y "%%I" "!TargetFolder!\" >nul
set /A FileCount+=1
)
:EndMove
endlocal
popd
goto :EOF
:EndBatch
endlocal
There is used a subroutine with name MoveVideos to be able to use goto :EOF to stop moving files in the current directory into subdirectories on reaching the maximum number of total folders as defined with MaxTotalFolders and counted with FolderCount inside the second FOR loop.
This batch file creates with the three values defined at top for a directory with the MP4 files Video001.mp4 to Video1203.mp4 a directory structure with following files:
Canal-1
Videos-1
Video0001.mp4
:
Video0010.mp4
:
Videos-50
Video0491.mp4
:
Video0500.mp4
Canal-2
Videos-1
Video0501.mp4
:
Video0510.mp4
:
Videos-50
Video0991.mp4
:
Video1000.mp4
Canal-3
Videos-1
Video1001.mp4
:
Video1010.mp4
:
Videos-20
Video1191.mp4
:
Video1200.mp4
Video1201.mp4
Video1202.mp4
Video1203.mp4
This first batch file took 1.58 seconds for the file moving task on my PC.
The second batch file is with random selection of the MP4 files.
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set "FilesInFolder=10"
set "FoldersInFolders=50"
set "MaxTotalFolders=120"
for %%# in ("%USERPROFILE%\Videos" "D:\Videos" "F:\Temp\Videos" "\\MyNAS\Videos") do call :MoveVideos %%#
goto EndBatch
:MoveVideos
pushd %1 2>nul
if errorlevel 1 (
echo/
echo ERROR: Failed to change to directory: %1
echo/
pause
goto :EOF
)
if exist "*!*.mp4" (
echo/
echo ERROR: Moving video files not possible because of file names with ! in name.
echo/
echo Please rename first the following files(s^) in: %1
echo/
dir *!*.mp4 /A-D /B
echo/
popd
pause
goto :EOF
)
setlocal EnableDelayedExpansion
set "FileCount=%FilesInFolder%"
set "CanalIndex=0"
set "FolderCount=-1"
set "VideosIndex=%FoldersInFolders%
set "TotalFileCount=0"
for /F "eol=| delims=" %%I in ('dir *.mp4 /A-D /B /ON 2^>nul') do set /A TotalFileCount+=1
:NextFile
if %TotalFileCount% == 0 goto EndMove
if %TotalFileCount% LEQ 32768 set /A "FileIndex=%RANDOM% %% TotalFileCount" & goto MoveFile
set /A FileGroups=TotalFileCount / 32768
set /A LastFileCount=TotalFileCount %% 32768
set "LastGroupIndex=%FileGroups%"
if not %LastFileCount% == 0 set /A FileGroups+=1
set /A GroupMultiplier=%RANDOM% %% FileGroups
if not %GroupMultiplier% == %LastGroupIndex% (set /A "FileIndex=GroupMultiplier * 32768 + %RANDOM%") else set /A "FileIndex=(FileGroups - 1) * 32768 + (%RANDOM% %% LastFileCount)"
:MoveFile
if not %FileIndex% == 0 (set "SkipValue=skip=%FileIndex% ") else set "SkipValue="
for /F "%SkipValue%eol=| delims=" %%I in ('dir *.mp4 /A-D /B 2^>nul') do (
if !FileCount! == %FilesInFolder% (
set FileCount=0
if !VideosIndex! == %FoldersInFolders% (
set /A CanalIndex+=1
set VideosIndex=1
) else set /A VideosIndex+=1
set /A FolderCount+=1
if !FolderCount! == %MaxTotalFolders% goto EndMove
set "TargetFolder=Canal-!CanalIndex!\Videos-!VideosIndex!"
md "!TargetFolder!" 2>nul
)
move /Y "%%I" "!TargetFolder!\" >nul
set /A FileCount+=1
set /A TotalFileCount-=1
goto NextFile
)
:EndMove
endlocal
popd
goto :EOF
:EndBatch
endlocal
This variant is much slower because of the randomization. It took 18.93 seconds to do the job on my PC.
It produces the same directory tree as the first batch file on current directory containing the MP4 files Video001.mp4 to Video1203.mp4, but which file is moved into which directory is really random.
Here is a third variant also with random selection of the next video file to move.
#echo off
echo %TIME%
setlocal EnableExtensions DisableDelayedExpansion
set "FilesInFolder=10"
set "FoldersInFolders=50"
set "MaxTotalFolders=120"
for /F "delims==" %%I in ('set # 2^>nul') do set "%%I="
for %%# in ("%USERPROFILE%\Videos" "D:\Videos" "F:\Temp\Videos" "\\MyNAS\Videos") do call :MoveVideos %%#
goto EndBatch
:MoveVideos
pushd %1 2>nul
if errorlevel 1 (
echo/
echo ERROR: Failed to change to directory: %1
echo/
pause
goto :EOF
)
if exist "*!*.mp4" (
echo/
echo ERROR: Moving video files not possible because of file names with ! in name.
echo/
echo Please rename first the following files(s^) in: %1
echo/
dir *!*.mp4 /A-D /B
echo/
popd
pause
goto :EOF
)
setlocal EnableDelayedExpansion
set "FileCount=%FilesInFolder%"
set "CanalIndex=0"
set "FolderCount=-1"
set "VideosIndex=%FoldersInFolders%
set "TotalFileCount=0"
for /F "eol=| delims=" %%I in ('dir *.mp4 /A-D /B /ON 2^>nul') do set "#!TotalFileCount!=%%I" & set /A TotalFileCount+=1
:NextFile
if %TotalFileCount% == 0 goto EndMove
if %TotalFileCount% LEQ 32768 set /A "FileIndex=%RANDOM% %% TotalFileCount" & goto MoveFile
set /A FileGroups=TotalFileCount / 32768
set /A LastFileCount=TotalFileCount %% 32768
set "LastGroupIndex=%FileGroups%"
if not %LastFileCount% == 0 set /A FileGroups+=1
set /A GroupMultiplier=%RANDOM% %% FileGroups
if not %GroupMultiplier% == %LastGroupIndex% (set /A "FileIndex=GroupMultiplier * 32768 + %RANDOM%") else set /A "FileIndex=(FileGroups - 1) * 32768 + (%RANDOM% %% LastFileCount)"
:MoveFile
if not %FileIndex% == 0 (set "SkipValue=skip=%FileIndex% ") else set "SkipValue="
for /F "%SkipValue%tokens=1* delims==" %%I in ('set #') do (
if !FileCount! == %FilesInFolder% (
set FileCount=0
if !VideosIndex! == %FoldersInFolders% (
set /A CanalIndex+=1
set VideosIndex=1
) else set /A VideosIndex+=1
set /A FolderCount+=1
if !FolderCount! == %MaxTotalFolders% goto EndMove
set "TargetFolder=Canal-!CanalIndex!\Videos-!VideosIndex!"
md "!TargetFolder!" 2>nul
)
set "%%I="
move /Y "%%J" "!TargetFolder!\" >nul
set /A FileCount+=1
set /A TotalFileCount-=1
goto NextFile
)
:EndMove
endlocal
popd
goto :EOF
:EndBatch
endlocal
It defines for each MP4 video file name an environment variable with a name beginning with # and a number incremented on each file and assigns the current file name to the environment variable. It processes this list of environment variables reduced by one environment variable after each file move.
It causes less file system accesses as the second batch file. But it took 20.54 seconds in my test on my PC to complete the video file movement task.
The main cause of the much longer time with random file selection is the fact that for each file to move one more cmd.exe must be started in background to output the new list of video file names in current directory, capture that file names list, skip all file names up to current random file index number and move the randomly selected file.
It would be much better if the list of environment variables with the video file names in current directory could be updated with removal of the environment variable of a just moved file and check if an environment variable with current randomly determined file index number still exists in the list to move that file and otherwise determine a new random file number index as often as needed. But such an approach can easily result in determining again and again randomly a file index number of video files which were moved already before. The result could be a nearly endless running loop on number of remaining files in environment variables list becomes small. The Windows Command Processor is not designed for such a task.
The randomization is tricky in case of the number of files in a directory is greater than 32768 because of %RANDOM% expands to a random decimal number between 0 and 32767. A cascaded randomized file index number determination is added for that reason to the code of the batch file.
The randomization of the selection of next video file to move causes also a lot of batch file open, seeking to next line to process, processing that line and perhaps read more lines and process them to, remember current position in batch file, batch file close. The first batch file is for that reason much more efficient. A more modern and powerful script interpreter like PowerShell would be much better for that task if randomized movement of video files is an important requirement.
To understand the commands used and how they work, open a command prompt window, execute there the following commands, and read the displayed help pages for each command, entirely and carefully.
call /?
dir /?
echo /?
endlocal /?
for /?
goto /?
if /?
md /?
move /?
pause /?
popd /?
pushd /?
set /?
setlocal /?