Bat file to delete files only when younger files are present - batch-file

Question

Our backup system creates .bak files everyday which we can use to restore files if we ever run into issues. If left alone these would fill up our storage so I found a batch file that I can run to delete the old batch files everyday after new ones are created.
forfiles -p "c:\xxx\yyy" -s -m *.bak /D -2 /C "cmd /c del #path"
This works fine but I want to create a safety net so that if for whatever reason our backup system fails and the new .bak files aren't created the old .bak files will stay there instead of being deleted, otherwise we would be left with no backup files in the event of an incident. So ideally I want something that will check for .bak files younger than one day and if those files are not present it won't run the above line but if those younger files are present it will run the above line and delete the older files. Not sure if this is possible with batch files or not. Thanks in advance for your help on this.
EDIT: Some more info on what I need. Everyday at around 10pm around 50 backup .bak files are created and put into folder c:\xxx\yyy
These files are quite large so I have set up a batch file to run automatically every day that removes all .bak files that are older than 1 day. This is fine for everyday use but the scenario I have in my head is what if the backup system doesnt create the .bak files for whatever reason. I want the batch file to check to make sure the new .bak files have been created before it deletes the old ones. Basically using a batch file is there a way to check if there is a certain file type in a folder that is newer than 1 day old and can we change what the batch file does depending on the outcome.
These are examples of the files created for the 18th and 19th.
2004 Apr_backup_2017_12_18_210001_2986007.bak
2004 Apr_backup_2017_12_19_210001_3168635.bak
Subscribers_backup_2017_12_19_210003_3012893.bak
model_backup_2017_12_19_210003_2544131.bak
They all seem to follow the below format:
[DESC]_backup_[YEAR]_[MONTH]_[DAY]_21000[1/2/3]_[7 DIGIT NO.].bak

Answer 1

I think, an unknown list of [DESC] strings in all the backup file names is most difficult to handle in batch file. The code could be very simple on knowing this list as it can be seen below, or at least on knowing if those strings do not contain characters being critical on batch file processing like !%=.
But the coding challenge for unknown list of [DESCR] strings with special characters in file names was interesting for me and so I developed first following commented batch file:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set "BackupFolder=C:\xxx\yyy"
rem Search for files matching the wildcard pattern *_backup_*.bak in backup
rem folder, assign each file name without file extension to environment
rem variable FileName and call the subroutine GetUniqueDescs to get the
rem file description at beginning of each file name into a list in memory.
for /F "delims=" %%I in ('dir "%BackupFolder%\*_backup_*.bak" /A-D /B /ON 2^>nul') do (
set "FileName=%%~nI"
call :GetUniqueDescs
)
rem Run command SET with FileDesc: to output all environment variables
rem starting with that string in name and sorted by name and process
rem this list whereby each line ends with =1 as value 1 is assigned
rem to each of these environment variables.
rem For each unique file description in output list assign the file
rem description with =1 appended to environment variable FileDesc
rem and run subroutine DeleteFiles.
for /F "tokens=2 delims=:" %%I in ('set FileDesc: 2^>nul') do (
set "FileDesc=%%I"
call :DeleteFiles
)
rem Restore initial environment on starting this batch file and exit it.
endlocal
goto :EOF
rem The subroutine GetUniqueDescs first runs a string substitution which
rem gets the backup pattern part from file name, i.e. everything in file
rem name from _backup_ to end of file name.
rem Then another string substitution is used to remove this string from
rem current file name to get just the description and define an environment
rem variable of which name starts with FileDesc: and ends with the file
rem description. The value assigned to this environment variable is 1.
:GetUniqueDescs
set "BackupPart=%FileName:*_backup_=_backup_%"
call set "FileDesc:%%FileName:%BackupPart%=%%=1"
goto :EOF
rem The subroutine DeleteFiles removes first from passed file description
rem the last two characters being always =1 from list of environment
rem variables starting with FileDesc: and appends the backup wildcard
rem pattern.
rem Command DIR is used to find all files in backup folder starting
rem with current file description and _backup_ and output the found
rem files sorted by last modification date with newest modified file
rem first and oldest modified file last.
rem The command FOR processing this list skips the first file name
rem output by DIR which means the newest file. All other, older
rem files perhaps also found by DIR are deleted one after the other.
:DeleteFiles
set "FilePattern=%FileDesc:~0,-2%_backup_*.bak"
for /F "skip=1 delims=" %%J in ('dir "%BackupFolder%\%FilePattern%" /A-D /B /O-D /TW') do ECHO del "%BackupFolder%\%%J"
goto :EOF
The command ECHO in last but one line before command del results in just getting displayed which files would be deleted instead of really deleting them.
The option skip=1 in last but one line determines how many backup files are always kept.
For example using skip=5 results in keeping the newest five files according to last modification date being usually on backup files also the creation date and deleting all others.
The advantage of such a backup deletion strategy is that it does not matter:
how often a specific backup is created – daily, weekly or monthly;
if last backup creation was successful at all;
if some or even all backup files were deleted manually;
how old each individual backup file is;
how often the batch file for backup files deletion is executed.
What really matters on deletion of backups is the storage size needed for each backup and how much free storage space remains after deletion process. The file date of a backup file is not limiting the free storage size. The file sizes of all remaining backup files and the total storage size on backup media are the factors which really matter. That's why I do not understand all those "delete older than" questions. Who has to care about age of a file as long as there is enough free space for new files?
The file creation date could be also used by using /TC instead of /TW in last but one line. But the file creation date is the date on which the file was created in that directory and not on which the file itself was created. For that reason the file creation date is only useful when the file was never copied or moved to another directory since first time creation.
I tested this batch file on following files:
C:\xxx\yyy\2004 !Apr_backup_2017_12_18_210001_2986007.bak
C:\xxx\yyy\2004 !Apr_backup_2017_12_19_210001_3168635.bak
C:\xxx\yyy\model%_backup_2017_12_19_210003_2544131.bak
C:\xxx\yyy\model%_backup_2017_12_20_210003_2544131.bak
C:\xxx\yyy\Subscribers=_backup_2017_12_19_210003_3012893.bak
C:\xxx\yyy\Subscribers=_backup_2017_12_20_210003_3012893.bak
The last modification date of each file matched the date in file name.
The output of the batch file was:
del "C:\xxx\yyy\2004 !Apr_backup_2017_12_18_210001_2986007.bak"
del "C:\xxx\yyy\model%_backup_2017_12_19_210003_2544131.bak"
del "C:\xxx\yyy\Subscribers=_backup_2017_12_19_210003_3012893.bak"
That is the expected result. The older file of each file pair would be deleted.
Then I thought getting [DESC] part of file name could be done easier as the remaining part of file name without file extension has a fixed length of 33 characters.
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set "BackupFolder=C:\xxx\yyy"
rem Search for files matching the long wildcard pattern
rem *_backup_????_??_??_??????_???????.bak
rem in backup folder and assign each file name without
rem file extension to environment variable.
rem The last 33 characters are removed from each file name to get the
rem file description part at beginning of each file name. Then define
rem an environment variable of which name starts with FileDesc: and
rem ends with the file description. The value assigned to this
rem environment variable is 1.
for /F "delims=" %%I in ('dir "%BackupFolder%\*_backup_????_??_??_??????_???????.bak" /A-D /B /ON 2^>nul') do (
set "FileName=%%~nI"
call set "FileDesc:%%FileName:~0,-33%%=1"
)
rem Run command SET with FileDesc: to output all environment variables
rem starting with that string in name and sorted by name and process
rem this list whereby each line ends with =1 as value 1 is assigned
rem to each of these environment variables.
rem For each unique file description in output list assign the file
rem description with =1 appended to environment variable FileDesc
rem and run subroutine DeleteFiles.
for /F "tokens=2 delims=:" %%I in ('set FileDesc: 2^>nul') do (
set "FileDesc=%%I"
call :DeleteFiles
)
rem Restore initial environment on starting this batch file and exit it.
endlocal
goto :EOF
rem The subroutine DeleteFiles removes first from passed file description
rem the last two characters being always =1 from list of environment
rem variables starting with FileDesc: and appends the backup wildcard
rem pattern.
rem Command DIR is used to find all files in backup folder starting
rem with current file description and _backup_ and output the found
rem files sorted by last modification date with newest modified file
rem first and oldest modified file last.
rem The command FOR processing this list skips the first file name
rem output by DIR which means the newest file. All other, older
rem files perhaps also found by DIR are deleted one after the other.
:DeleteFiles
set "FilePattern=%FileDesc:~0,-2%_backup_*.bak"
for /F "skip=1 delims=" %%J in ('dir "%BackupFolder%\%FilePattern%" /A-D /B /O-D /TW') do ECHO del "%BackupFolder%\%%J"
goto :EOF
That batch file containing also ECHO left to command del in last but one line produces the same result on the six files in the backup folder.
I don't know if the batch file could be even more optimized without knowing which characters could exist in [DESC] part of the file names. I did not think about a possible further optimization.
Let us assume the list of unique [DESC] strings is well known and can be hard coded in the batch file, for example 2004 !Apr, model% and Subscribers= for the six files in my test case:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set "BackupFolder=C:\xxx\yyy"
for %%I in ("2004 !Apr" "model%%" "Subscribers=") do for /F "skip=1 delims=" %%J in ('dir "%BackupFolder%\%%~I*_backup_*.bak" /A-D /B /O-D /TW 2^>nul') do del "%BackupFolder%\%%J"
endlocal
This batch file really deletes files because there is no ECHO in last but one line.
Oh yes, knowing the individual backup file names makes everything much easier.
The batch file can be even optimized to a single command line:
#for %%I in ("2004 !Apr" "model%%" "Subscribers=") do #for /F "skip=1 delims=" %%J in ('dir "C:\xxx\yyy\%%~I*_backup_*.bak" /A-D /B /O-D /TW 2^>nul') do #del "C:\xxx\yyy\%%J"
Best and simple backup deletion concept
On a backup storage media is created:
a backup of an entire machine with file name ComputerName_backup_YYYY_MM.tib every three months which is huge as taking 200 GiB and where it is enough to have only last backup on the backup storage media;
a backup of a folder with files not often updated with file name Folder_backup_YYYY_MM_DD.zip every Saturday which takes about 400 MiB on storage media where it is enough to be able to restore the last four weeks;
a backup of a database file with file name Database_backup_YYYY_MM_DD.bak every day which takes at the moment 20 MiB per backup, but is growing more or less constant as typical for database files and on where it should be possible to restore data entries of the last seven days.
The required minimum storage media size is:
(1+1) × 200 GiB + (4+1) × 400 MiB + (7+1) × (20×3) MiB
A storage media size of 1 TiB is really enough for approximately the next three years depending on growing rate of database backup on which an increase by a factor of three is included already in calculation.
It would be best to delete all backup files no longer needed on creating the daily database backup to keep the backup files management simple by using a single and simple batch file.
#echo off
set "BackupFolder=C:\xxx\yyy"
call :DeleteBackups 1 "ComputerName"
call :DeleteBackups 4 "Folder"
call :DeleteBackups 7 "Database"
goto :EOF
:DeleteBackups
for /F "skip=%1 delims=" %%I in ('dir "%BackupFolder%\%~2*_backup_*" /A-D /B /O-D /TW 2^>nul') do del "%BackupFolder%\%%I"
goto :EOF
Deletion of no longer needed backups can be really so easy on thinking about right strategy.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
call /?
del /?
dir /?
echo /?
endlocal /?
for /?
goto /?
rem /?
set /?
setlocal /?
Read also the Microsoft article about Using Command Redirection Operators for an explanation of 2>nul. The redirection operator > must be escaped with caret character ^ on FOR command lines to be interpreted as literal character when Windows command interpreter processes this command line before executing command FOR which executes the embedded dir command line with using a separate command process started in background.

Answer 2

Here's an untested example script which should work as long as you don't have file names where [DESC] contains _, = or other problematic characters.
#Echo Off
SetLocal DisableDelayedExpansion
For /F "Delims==" %%A In ('Set _[ 2^>Nul') Do Set "%%A="
If /I Not "%CD%"=="C:\xxx\yyy" (Set "_[:]=T"
PushD "C:\xxx\yyy" 2>Nul||Exit /B)
For /F "Tokens=1* Delims=_" %%A In ('Dir /B /O-N *_backup_*_*_*_*_*.bak'
) Do If Defined _[%%A] (Del /A /F "%%A_%%B") Else Set "_[%%A]=T"
If Defined _[:] PopD
EndLocal
Exit /B