FTP Delete local files if they no longer exist on remote server - batch-file

I is it possible using the script below to delete the local files if they no longer exist on the remote directory? Currently this downloads all new files but wont delete files that no longer exist on the ftp server?
#Echo Off
REM -- Define File Filter, i.e. files with extension .txt
set FindStrArgs=/x "..*\...*"
REM -- Extract Ftp Script to create List of Files
Set "FtpCommand=ls"
Call:extractFileSection "[Ftp Script 1]" "-">"%temp%\%~n0.ftp"
Rem Notepad "%temp%\%~n0.ftp"
REM -- Execute Ftp Script, collect File Names
Set "FileList="
For /F "Delims=" %%A In ('"Ftp -v -i -s:"%temp%\%~n0.ftp"|Findstr %FindStrArgs%"') Do (
Call Set "FileList=%%FileList%% "%%A""
)
REM -- Extract Ftp Script to download files that don't exist in local folder
Set "FtpCommand=mget"
For %%A In (%FileList%) Do If Not Exist "%%~A" Call Set "FtpCommand=%%FtpCommand%% "%%~A""
Call:extractFileSection "[Ftp Script 1]" "-">"%temp%\%~n0.ftp"
Rem Notepad "%temp%\%~n0.ftp"
For %%A In (%FtpCommand%) Do Echo.%%A
REM -- Execute Ftp Script, download files
ftp -i -s:"%temp%\%~n0.ftp"
Del "%temp%\%~n0.ftp"
GOTO:EOF
:extractFileSection StartMark EndMark FileName -- extract a section of file that is defined by a start and end mark
:: -- [IN] StartMark - start mark, use '...:S' mark to allow variable substitution
:: -- [IN,OPT] EndMark - optional end mark, default is first empty line
:: -- [IN,OPT] FileName - optional source file, default is THIS file
:$created 20080219 :$changed 20100205 :$categories ReadFile
:$source http://www.dostips.com
SETLOCAL Disabledelayedexpansion
set "bmk=%~1"
set "emk=%~2"
set "src=%~3"
set "bExtr="
set "bSubs="
if "%src%"=="" set src=%~f0& rem if no source file then assume THIS file
for /f "tokens=1,* delims=]" %%A in ('find /n /v "" "%src%"') do (
if /i "%%B"=="%emk%" set "bExtr="&set "bSubs="
if defined bExtr if defined bSubs (call echo.%%B) ELSE (echo.%%B)
if /i "%%B"=="%bmk%" set "bExtr=Y"
if /i "%%B"=="%bmk%:S" set "bExtr=Y"&set "bSubs=Y"
)
EXIT /b

If possible, use an ftp client like lftp. It will give you options to mirror two ftp locations in one line of code (see mirror option). Also check out the flags:
--only-newer
--parallel=10
--delete

Related

Creating a batch file to prompt for response and replace lines of text in a text or .ini file

I have gotten the batch file to work for the first line, however when I try to add an additional FOR loop to get the next line of text it actually copies or duplicates the text instead of replacing it.
currently the batch file will not even change the .txt file.
Code below --
::Find and Replace script allows the user to
::define a file path, file name and a string
::to find and replace so as to create a new file.
::
::Original file is backed up with an added extension of *.bak, in case
::the user finds the need to go back to the original.
#echo off
::Use the path from whence the script was executed as
::the Current Working Directory
set CWD=%~dp0
::***BEGIN MODIFY BLOCK***
::The variables below should be modified to the
::files to be changed and the strings to find/replace
::Include trailing backslash in _FilePath
set _FilePath=C:\
set _FileName=Password1.txt
::_WrkFile is the file on which the script will make
::modifications.
set /P Password="What is the admin Password?"
set /P AuditPassword="What is the Audit Password?"
set oldPass="cmspassword=Password1"
set OldAudit="existingauditingdbpassword=Password1"
set CMSAdmin="cmspassword="
set CMSAudit="existingauditingdbpassword="
set OldStr=%oldPass%
set OldStr2=%OldAudit%
set NewStr="%CMSAdmin%%Password%"
set NewStr2="%CMSAudit%%AuditPassword%"
::***END MODIFY BLOCK***
::Set a variable which is used by the
::search and replace section to let us
::know if the string to be modified was
::found or not.
set _Found=Not found
SETLOCAL
SETLOCAL ENABLEDELAYEDEXPANSION
if not exist "%_FilePath%%_FileName%" goto :NotFound
echo Searching for %OldStr% string...
echo.
for /f "usebackq tokens=*" %%a in ("%_FilePath%%_FileName%") do (
set _LineChk=%%a
if "!_LineChk!"==%OldStr% (
SET _Found=Found
SET NewStr=!NewStr:^"=!
echo !NewStr!
) else (echo %%a)
)>>"%_FilePath%%_FileName%" 2>&1
for /f "usebackq tokens=*" %%b in ("%_FilePath%%_FileName%") do (
set _LineChk=%%b
if "!_LineChk!"==%OldStr2% (
SET _Found=Found
SET NewStr2=!NewStr2:^"=!
echo !NewStr2!
)
)
echo.
:Exit
exit /b

Batch File - Create Zip file with Command Line using the first part of the first file name

I have a folder of PDF files that have a consistent naming convention. I want to create a zip file of these PDF files but named the zip file using the portion of the file that is before the # -- all of the files are the same in the front (it is the NTID of the user that created the pdf files).
As an example these are what the files might look like in the PDF output folder (there could be 100 files all that start with the same UserID before the #:
UserID#Carlos+Alberto+Mafra-+bribery-2019-05-16
UserID#MAJELA+HOSPITALAR+LTDA-+bribery-2019-05-16
(Ideally, I would also want the current date appended to the zip file)
The zip should be called UserID-2019-05-16.zip based on the example above.
This is the code I am trying to use but not having success...
I created a batch script using others suggestions for each step. but can't get it to work end to end.
FOR %%F IN ("C:\Users\SA-JJC-HCC_Ops\OneDrive - JNJ\workflows\TPIGoogle\pdf\*.pdf") DO (
set filename=%%F
goto next
)
:next
echo "%filename%"
set zipfile=%filename%
for /f "tokens=1 delims=#" %%a in ("%zipfile%") do (
)
cd "C:\Program Files\7-Zip\"
7z.exe" a "C:\Users\SA-JJC-HCC_Ops\JNJ\HCC&P Alteryx - Documents\EPiC\GoogleSearches\zip\" && %zipfile% && ".zip" "C:\Users\SA-JJC-HCC_Ops\OneDrive - JNJ\workflows\TPIGoogle\pdf\*.pdf"
One zip file with all the PDFs that are using the first part of the string from the file names in the PDF folder.
#ECHO OFF
SETLOCAL enabledelayedexpansion
SET "sourcedir=U:\sourcedir\t w o"
SET "destdir=U:\destdir"
FOR /f "tokens=1*delims=#" %%a IN (
'dir /b /a-d "%sourcedir%\*#*-????-??-??.pdf" '
) DO (
SET "pre=%%~a"
SET "post=%%~nb"
SET "post=!post:~-10!"
IF DEFINED post ECHO "C:\Program Files\7-Zip\7z" a "%destdir%\!pre!-!post!.zip" "%sourcedir%\%%~a#%%~b"
IF NOT DEFINED post ECHO SKIP "%%~a#%%~b"
)
GOTO :EOF
You would need to change the settings of sourcedir and destdir to suit your circumstances.
I used the variablenames pre and post to ensure that the names used in this process are not keywords like the more logical date.
Read a list of all filenames matching the pattern *#*-????-??-??.pdf in the source directory, tokenising on #. Assign the userid to %%a and thence to pre and the "name" part of the dregs of the actual filename to post, then select only the last 10 characters of post using delayed-expansion.
There is an opportunity here to further process post to check whether it truly fits the pattern for a date, if that is required. That routine may return post either unmolested or empty. If it's not empty then construct the required 7z command (you may wish to add -tzip) and echo this for verification - remove the echo to actuate the 7z compression. If post is emptied by a pattern-checking routine, then the filename will simply be reported as having been skipped.
If it is indeed current date you want to append to the end of the zip file, then we need to get the non locale dependent date and time. This will then copy each of the pdf files that starts with UserID to a zip with a date of the day you run the script UserID-2019-05-17 :
#echo off
set "outDir=C:\Users\SA-JJC-HCC_Ops\JNJ\HCC&P Alteryx - Documents\EPiC\GoogleSearches\zip\"
set "inDir=C:\Users\SA-JJC-HCC_Ops\OneDrive - JNJ\workflows\TPIGoogle\pdf\"
for /f "tokens=1,2 delims==" %%i in ('wmic os get LocalDateTime /VALUE') do (
if ".%%i."==".LocalDateTime." set mydate=%%j
)
set mydate=%mydate:~0,4%-%mydate:~4,2%-%mydate:~6,2%
for %%a in (*.pdf) do for /f "delims=#" %%i in ('dir /b /a-d %%a') do (
"C:\Program Files\7-Zip\7z" a "%outDir%%%i-%mydate%.zip" "%inDir%%%~a"
)
If in fact you want to append the date of the filename instead (In other words create Zip files for each file with a different date as well as matching userid):
#echo off
setlocal enabledelayedexpansion
set "outDir=C:\Users\SA-JJC-HCC_Ops\JNJ\HCC&P Alteryx - Documents\EPiC\GoogleSearches\zip\"
set "inDir=C:\Users\SA-JJC-HCC_Ops\OneDrive - JNJ\workflows\TPIGoogle\pdf\"
for %%a in (*.pdf) do for /f "tokens=1,* delims=#" %%i in ('dir /b /a-d %%a') do (
set fdate=%%~nj
set fdate=!fdate:~-10!
echo "C:\Program Files\7-Zip\7z" a "%outDir%%%i-!fdate!.zip "%inDir%%%~a"
)

How to set environment variable with &

So I have a bat file that I wrote, and I am having issues with setting variables that contain an & in the file path.
Right now this works, but I can't use the %SOURCE% or %DEST% variables in the copy command. I get an error. Hence the reason I have the full path written out in the copy command.
#echo off
echo Backing Up Build Files ^& Templates
echo.
:: do a dump of repo
echo Creating Dump File
svnadmin dump "Z:\Subversion\Build Files "^&" Templates" > "Z:\Subversion\Build Files "^&" Templates_repository-backup.dmp"
:: copy dump file to backup location
set YEAR=%date:~10,4%
set MONTH=%date:~4,2%
set DAY=%date:~7,2%
set THE_DATE=%MONTH%_%DAY%_%YEAR%
set SPACE_CHECK=%time:~0,1%
if "%SPACE_CHECK%"==" " goto handlehourspace
set HOURS=%time:~0,2%
goto hourdone
:handlehourspace
set HOURS=0%time:~1,1%
:hourdone
set MINUTES=%time:~3,2%
set SECONDS=%time:~6,2%
set THE_TIME=%HOURS%_%MINUTES%_%SECONDS%
set SOURCE="Z:\Subversion\Build Files "^&" Templates_repository-backup.dmp"
set DEST="K:\IETM_Repo_Backup\Build Files "^&" Templates\Build Files "^&" Templates_repository-backup.dmp"_%THE_DATE%_%THE_TIME%
echo Copying Dump File
copy /Z "Z:\Subversion\Build Files "^&" Templates_repository-backup.dmp" "K:\IETM_Repo_Backup\Build Files "^&" Templates\Build Files "^&" Templates_repository-backup.dmp"_%THE_DATE%_%THE_TIME%
echo Deleting Original Dump File
del "Z:\Subversion\Build Files "^&" Templates_repository-backup.dmp"
use this syntax: set "var=value" to be able to use special chars in a value:
set "x=this&that"
echo "%x%"
REM create a file:
break>"%x%"
dir this*
REM remove the file:
del "%x%"
With your style set DEST="K:\...." the quotes are part of the content. With the extended SET-syntax set "DEST=K:\...." they escape the string but aren't part of the string.
In the most cases you should use the extended syntax
set "SOURCE=Z:\Subversion\Build Files & Templates_repository-backup.dmp"
set "DEST=K:\IETM_Repo_Backup\Build Files & Templates\Build Files & Templates_repository-backup.dmp_%THE_DATE%_%THE_TIME%"
copy /Z "%SOURCE%" "%DEST%"
In the copy "%SOURCE%" ... line the quotes are necessary to escape again the ampersands and spaces.
To have a locale independent way to format the date/time I suggest using WMI
and the backup of the dump file stays better usable when the extension remains the last part of the file name, not the date/time stamp.
#echo off
echo Backing Up Build Files ^& Templates
echo.
:: get datetime string of the format yyyyMMddhhnnss.
for /f "tokens=1-3 delims=.+-" %%A in (
'wmic os get LocalDateTime^|findstr ^^[0-9]'
) do Set _DT=%%A
set THE_DATE=%_DT:~4,2%_%_DT:~6,2%_%_DT:~0,4%
set THE_TIME=%_DT:~8,2%_%_DT:~10,2%_%_DT:~12,2%
:: do a dump of repo
echo Creating Dump File
set "BFT=Build Files & Templates"
set "svnBFT=Z:\Subversion\%BFT%"
set "svnSRC=%svnBFT%_repository-backup.dmp"
svnadmin dump "%svnBFT%" > "%svnSRC%"
:: copy dump file to backup location
echo Copying Dump File
set "DEST=K:\IETM_Repo_Backup\%BFT%\%BFT%_repository-backup_%THE_DATE%_%THE_TIME%.dmp"
copy /Z "%svnSRC%" "%Dest%"
echo Deleting Original Dump File
del "%svnSRC%"

Batch: search for files with certain extension, owner, fullpath and last write access and output in CSV

I'm trying to create a CSV with fullpath\filename, file owner and last write access (modification date) of all txt and html files from all hard drives of a data server.
Here's what I got so far:
set pgm=%~n0
set log=%~dpn0.log
set host=%COMPUTERNAME%
set csv=%host%.csv
set dir=D:\BME
if not exist "%csv%" type nul>"%csv%"
for /f "delims=;" %%a in ('dir /b/s %dir%\*.txt, %dir%\*.html') do (
>>%csv% echo "%%a"
)
That outputs the path + filename of all found txt and html files of a certain folder in a CSV. I tried this command to get the hard drives:
wmic logicaldisk where drivetype=3 get caption
But I can't get my head around how to store that in a variable or file and loop through it and also retrieve the owner and last modification date and put it into a new column of the csv file.
I can't get my head around how to store that in a variable
Use the following batch file.
GetDrives.cmd:
#echo off
setlocal enabledelayedexpansion
rem skip=1 to remove the header
rem findstr to remove blank lines
for /f "skip=1" %%d in ('wmic logicaldisk where drivetype^=3 get caption ^| findstr /r /v "^$"') do (
set _drive=%%d
echo !_drive!
)
endlocal
Notes:
Be careful when using drivetype=3 as I have a removable drive of type 3. In the below output C: is a fixed hard disk and F: is a removable external USB drive.
Replace echo !_drive! as appropriate with a modified version of your existing code.
Example Output:
F:\test>GetDrives
C:
F:
F:\test>
Further Reading
An A-Z Index of the Windows CMD command line - An excellent reference for all things Windows cmd line related.
enabledelayedexpansion - Delayed Expansion will cause variables to be expanded at execution time rather than at parse time.
for /f - Loop command against the results of another command.
wmic - Windows Management Instrumentation Command.
DavidPostill answered how-to store wmic logicaldisk … output in a variable;
to retrieve file last modification date: use echo "%%a","%%~ta" in your script using %~t Parameter Extension;
to retrieve file owner: echo "%%a","%%~ta","!_owner!" where _owner variable comes from getRealOwner subroutine based on modified schletti2000's answer Get ownership information from command line by using wmic.
The script:
#ECHO OFF
SETLOCAL EnableExtensions DisableDelayedExpansion
set "pgm=%~n0"
set "log=%~dpn0.log"
set "host=%COMPUTERNAME%"
set "csv=%host%.csv"
set "dir=D:\BME"
set "dirmask=%dir%\*.txt, %dir%\*.html"
rem if not exist "%csv%" type nul>"%csv%"
>"%csv%" (
for /f "delims=;" %%a in ('dir /b/s %dirmask% 2^>NUL') do (
set "_fFullPath=%%~a"
set "_fLastWrite=%%~ta"
set "_fOwner="
call :getRealOwner
SETLOCAL EnableDelayedExpansion
echo "!_fFullPath!","!_fOwner!","!_fLastWrite!"
ENDLOCAL
)
)
)
type "%csv%"
goto :continue
:getRealOwner
SET "ESCAPED=%_fFullPath:\=\\%"
SET "UNDELIMITED="
for /F "skip=2 delims=" %%g in ('
wmic path Win32_LogicalFileSecuritySetting where Path^="%ESCAPED%" ^
ASSOC /RESULTROLE:Owner /ASSOCCLASS:Win32_LogicalFileOwner ^
/RESULTCLASS:Win32_SID 2^>NUL
') do (
SET "UNDELIMITED=%%g"
call :process_wmioutput
)
if NOT defined UNDELIMITED set "_fOwner=???"
exit /B
:process_wmioutput
SET "DELIMITED=%UNDELIMITED: =•%"
FOR /F "delims=• tokens=10,12" %%G in ("%DELIMITED%") DO set "_fOwner=%%H\%%G"
exit /B
:continue
I used next settings to demonstrate various output:
set "dir=D:"
set "dirmask=%dir%\loc*.vbs %dir%\bcd*.log %dir%\act*.xsl %dir%\diag*.xml %dir%\chec*.csv"
Output - non-privileged cmd window:
==> D:\bat\SO\39034430.bat
"D:\odds and ends\tempx\links\testDJ\LocUsers.vbs","mypc\user","25.12.2014 00:13"
"D:\tempWin\ActivityLog.xsl","NT AUTHORITY\SYSTEM","24.02.2016 13:12"
"D:\tempWin\CompatTelemetryLogs\diagerr.xml","???","12.08.2015 03:17"
"D:\tempWin\CompatTelemetryLogs\diagwrn.xml","???","12.08.2015 03:17"
"D:\test\check_acl.csv","BUILTIN\Administrators","06.03.2016 14:28"
Output - privileged (run as administrator) cmd window:
=ADMIN=> D:\bat\SO\39034430.bat
"D:\odds and ends\tempx\links\testDJ\LocUsers.vbs","mypc\user","25.12.2014 00:13"
"D:\tempWin\ActivityLog.xsl","NT AUTHORITY\SYSTEM","24.02.2016 13:12"
"D:\tempWin\CompatTelemetryLogs\diagerr.xml","NT AUTHORITY\SYSTEM","12.08.2015 03:17"
"D:\tempWin\CompatTelemetryLogs\diagwrn.xml","NT AUTHORITY\SYSTEM","12.08.2015 03:17"
"D:\test\check_acl.csv","BUILTIN\Administrators","06.03.2016 14:28"

How to use batch job to add the file "create date" into all the files in a directory?

I have a folder that gets a new file added everyday to the folder with the same file name but incremental extension such as .001, .002, .003, etc. However, if there's no file within the folder it starts at .001 again.
The problem is they are all named the same and if I move them to another folder to archive them it would just overwrite the same file over and over again. I could create a folder each day with the date with only one file in it, but that seems a bit redundant.
Is there a way to look at the create date of each file and rename it to the create date?
I've gotten this far, but it looks like for this situation I have to use a static file name, how to loop through the entire directory?
SET filename = C:\test.001
FOR %%f IN (%filename%) DO SET filedatetime=%%~tf
rename c:\test.001 C:\test_%filedatetime%.txt
move C:\*.txt C:\archive\
this provides the correct sort order:
#echo off &setlocal disableDelayedExpansion
set "startfolder=%userprofile%\test"
cd /d "%startfolder%"
for %%a in (*) do (
for /f "delims=." %%b in ('wmic datafile where "name='%startfolder:\=\\%\\%%~a'" get lastmodified^|find "."') do (
echo(ren "%startfolder%\%%~a" "%%~b.txt"
)
)
Remove echo to get it working.
#ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION
SET "targetdir=c:\sourcedir"
SET "destdir=c:\destdir"
PUSHD "%targetdir%"
FOR %%a IN (*.*) DO (
SET "timestamp=%%~ta"
SET "timestamp=!timestamp:/=_!
SET "timestamp=!timestamp::=_!
SET "timestamp=!timestamp:.=_!
SET "timestamp=!timestamp:,=_!
SET "timestamp=!timestamp: =_!
ECHO MOVE "%%a" "%destdir%\%%~na.!timestamp!"
)
GOTO :EOF
This should work with any file in the nominated target directory where the name does not include ! or ^.
The required MOVE commands are merely ECHOed for testing purposes. After you've verified that the commands are correct, change ECHO MOVE to MOVE to actually move the files. Append >nul to suppress report messages (eg. 1 file moved)
The gymnastics around timestamp are intende to replace /: with _ since these are illegal filename characters. Space., are similarly replaced - they're legal but often painful.
If you want the destination filename to be name.003.timestamp, remove the ~na from the destination name.
Try like this :
SET $path=The_path_who_contain_the_FILES
FOR /F "DELIMS=" %%f IN ('dir "%$path%" /a-d/b') DO (
SET filedatetime=%%~tf
move "%%~dpnxf" "C:\archive\test_%filedatetime%.txt")

Resources