I currently have batch scripts on different servers that transfer a csv file to an FTP server at a different location. My script looks similar to this:
echo user ftp_user> ftpcmd.dat
echo password>> ftpcmd.dat
echo put c:\directory\%1-export-%date%.csv>> ftpcmd.dat
echo quit>> ftpcmd.dat
ftp -n -s:ftpcmd.dat ftp.MyFTPSite.com
del ftpcmd.dat
If I wanted to require a secure transmission, is how would my script be updated?
Thanks.
First, make sure you understand, if you need to use Secure FTP (=FTPS, as per your text) or SFTP (as per tag you have used).
Neither is supported by Windows command-line ftp.exe. As you have suggested, you can use WinSCP. It supports both FTPS and SFTP.
Using WinSCP, your batch file would look like (for SFTP):
echo open sftp://ftp_user:password#ftp.MyFTPSite.com -hostkey="..." >> ftpcmd.dat
echo put c:\directory\%1-export-%date%.csv >> ftpcmd.dat
echo exit >> ftpcmd.dat
winscp.com /script=ftpcmd.dat
del ftpcmd.dat
And the batch file:
winscp.com /log=ftpcmd.log /script=ftpcmd.dat /parameter %1 %date%
Though using all capabilities of WinSCP (particularly providing commands directly on command-line and the %TIMESTAMP% syntax), the batch file simplifies to:
winscp.com /log=ftpcmd.log /command ^
"open sftp://ftp_user:password#ftp.MyFTPSite.com -hostkey=""...""" ^
"put c:\directory\%1-export-%%TIMESTAMP#yyyymmdd%%.csv" ^
"exit"
For the purpose of -hostkey switch, see verifying the host key in script.
Easier than assembling the script/batch file manually is to setup and test the connection settings in WinSCP GUI and then have it generate the script or batch file for you:
All you need to tweak is the source file name (use the %TIMESTAMP% syntax as shown previously) and the path to the log file.
For FTPS, replace the sftp:// in the open command with ftpes:// (explicit TLS/SSL) or ftps:// (implicit TLS/SSL). And remove the -hostkey switch.
winscp.com /log=ftpcmd.log /command ^
"open ftps://ftp_user:password#ftp.MyFTPSite.com -explicit" ^
"put c:\directory\%1-export-%%TIMESTAMP#yyyymmdd%%.csv" ^
"exit"
You may need to add the -certificate switch, if your server's certificate is not issued by a trusted authority.
Again, as with the SFTP, easier is to setup and test the connection settings in WinSCP GUI and then have it generate the script or batch file for you.
See a complete conversion guide from ftp.exe to WinSCP.
You should also read the Guide to automating file transfers to FTP server or SFTP server.
Note to using %TIMESTAMP#yyyymmdd% instead of %date%: A format of %date% variable value is locale-specific. So make sure you test the script on the same locale you are actually going to use the script on. For example on my Czech locale the %date% resolves to čt 06. 11. 2014, what might be problematic when used as a part of a file name.
For this reason WinSCP supports (locale-neutral) timestamp formatting natively. For example %TIMESTAMP#yyyymmdd% resolves to 20170515 on any locale.
(I'm the author of WinSCP)
The built in FTP command doesn't have a facility for security. Use cUrl instead. It's scriptable, far more robust and has FTP security.
ftps -a -z -e:on -pfxfile:"S-PID.p12" -pfxpwfile:"S-PID.p12.pwd" -user:<S-PID number> -s:script <RemoteServerName> 2121
S-PID.p12 => certificate file name ;
S-PID.p12.pwd => certificate password file name ;
RemoteServerName => abcd123 ;
2121 => port number ;
ftps => command is part of ftps client software ;
Related
I have an issue where the files I am uploading to a vendor are being consumed before they are finished uploading. To get around it the vendor has provided me with a location underneath where I can upload the files first then requested I move them once the upload is done.
Stripping out sensitive information, I am referencing this file in my sftp command in batch myfile.scr
cd /from_client/payments/upload
lcd E:/uploadfiles
mput uploadfiles.xml.*
quit
That would upload the files to the remote directory upload. I know need to get them into the level above. The theory is this will then process as normal and the files will already exist so the application won't consume them before they are done uploading.
I would assume I need to do a job to build a list of the files and then a 2nd to loop through them to do a rename on the remote folder to get them to go up a level.
Any ideas on how this could work? The vendor is unable to do anything here.
already tried various solutions to see if I could use mv but thats not an option as its not using Winscp. We can only use batch scripts and then native sftp commands.
You can generate put and rename commands for each file using a batch file like this:
set SOURCE=E:\uploadfiles
set SCRIPT=myfile.scr
(
echo cd /from_client/payments/upload
echo lcd %SOURCE%
for %%F in (%SOURCE%\*) do (
echo put %%~nF
echo rename %%~nF /final/path/%%~nF
)
echo quit
) > %SCRIPT%
sftp user#host -b %SCRIPT%
It should generate a script like this:
cd /from_client/payments/upload
lcd E:\uploadfiles
put one.txt
rename one.txt /final/path/one.txt
put two.txt
rename two.txt /final/path/two.txt
put three.txt
rename three.txt /final/path/three.txt
quit
Though with a more capable client, it would be easier. For example my WinSCP supports wildcards with mv command, so you can simply do:
WinSCP.com /ini=nul /log=winscp.log /command ^
"open sftp://user:password#host/ -hostkey=""...""" ^
"put E:\uploadfiles\* /from_client/payments/upload/" ^
"mv /from_client/payments/upload/* /final/path/" ^
"exit"
I'm aware that you wrote that you cannot use other clients, because:
"It's not using WinSCP" – What "it"? It's you! Use the best tool for your task.
"We can only use ... native sftp commands" – There are no "native sftp commands". There are only commands of the client you are using. Currently you are using OpenSSH sftp client. But that's not set in stone. It's easy to switch (you do not even to read it, everything is already in my code above).
I am trying to automate a call to a remote server using WinSCP which is a redundant everyday task. I want the batch file to call remote machine - provide username and password, and later check for if files exist.
So far I am able to start a WinSCP server but still the batch file does not consume username and password - it is still asking for them and/or providing an error regarding too many arguments.
chdir /d D:\Program Files\WinSCP\
winscp.com 172.18.186.39 username password
Your syntax does not even remotely resemble WinSCP command-line syntax for scripting.
See WinSCP article on Checking file existence using scripting:
#echo off
set REMOTE_PATH=/home/user/test.txt
winscp.com /command ^
"open ftp://username:password#ftp.example.com/" ^
"stat %REMOTE_PATH%" ^
"exit"
if %ERRORLEVEL% neq 0 goto error
echo File %REMOTE_PATH% exists
rem Do something
exit /b 0
:error
echo Error or file %REMOTE_PATH% not exists
exit /b 1
The above example is for FTP protocol. You didn't tell use what protocol are you using. If not FTP, you have to change the open command accordingly.
You can have WinSCP GUI generate the open command for you.
Once you have the batch file ready, use Windows Scheduler to call it regularly.
I can't try to reconnect with a different password after authentication failed. ERRORLEVEL seems to not be working in my code. I have no clue what's going wrong. The code is in batch file on Windows 10.
#echo off
echo use USERNAME> ftpcmd.dat
echo PASSWORD>> ftpcmd.dat
ftp -n -s:ftpcmd.dat files.000webhost.com
if %ERRORLEVEL% neq 0 (
goto TEST
)
:TEST
echo user USERNAME> ftpcmd.dat
echo DIFFERENTPASSWORD>> ftpcmd.dat
echo prompt>>ftpcmd.dat
echo cd Test>>ftpcmd.dat
echo mput FILE PATH>> ftpcmd.dat
ftp -n -s:ftpcmd.dat FTPTHING
The Windows ftp.exe always returns 0, no matter what.
All you can possibly do, is to parse its output and look for errors. But that's pretty unreliable.
If you want to take this approach anyway, see
how to capture error conditions in windows ftp scripts? or
Batch command to check if FTP connection is successful.
Though, you better use some better FTP client. Sooner or later you will run into troubles with the ftp.exe anyway, as it does not support an encryption, the passive mode or recursive transfers, among many other limitations.
If you are familiar with PowerShell, use the FtpWebRequest or WebClient from a Powershell script. There are numerous examples here, for example Upload files with FTP using PowerShell.
If not, use some 3rd party command-line FTP client.
For example with WinSCP FTP client, your batch file may look like:
#echo off
winscp.com /ini=nul /log=script.log /command ^
"open ftp://USERNAME:PASSWORD#ftp.example.com/" ^
"put C:\LOCAL\FILE /REMOTE/PATH/" ^
"exit"
if ERRORLEVEL 1 (
winscp.com /ini=nul /log=script2.log /command ^
"open ftp://USERNAME:DIFFERENTPASSWORD#ftp.example.com/" ^
"put C:\LOCAL\FILE /REMOTE/PATH/" ^
"exit"
)
References:
Automate file transfers to FTP server;
How do I know that script completed successfully?;
Guide to converting Windows FTP script to WinSCP SFTP script.
WinSCP can also be used from PowerShell, if you want even more robust implementation.
(I'm the author of WinSCP)
I searched from several posts to find the solution for this problem but couldn't find it. I want to build a Batch file that goes to a folder on a FTP and Download all the records then Delete all of them.
So far I tried to Use the command "mdel .", please see the code below:
#echo off
echo user ftpUser> ftpcmd.dat
echo Password>> ftpcmd.dat
echo cd /tst/>>ftpcmd.dat
echo binary>> ftpcmd.dat
echo prompt n>> ftpcmd.dat
echo mget *.*>> ftpcmd.dat
echo mdel *.*>> ftpcmd.dat
echo quit>> ftpcmd.dat
ftp -n -s:ftpcmd.dat 77.99.999.999
del ftpcmd.dat
For this solution I noticed that if i use the command "delete" and specify the file it works. Like this:
echo delete 312312312.csv >> ftpcmd.dat
But for some reason using mdel(like the example) it doesn't.
Do you know a better way to make it works?
Thanks
The mdel command needs to list files in the remote directory. The problem is that the Windows built-in ftp.exe supports an FTP active mode only. In this mode the server needs to open an incoming connection to your machine, to transfer the listing, what gets by default blocked by Windows firewall or any other firewall/proxy/NAT on the way between you and the server. That makes the ftp.exe most useless nowadays.
See my article about network configuration for FTP active mode for details.
Use any other 3rd party FTP client. Most support the passive mode that does not have these kinds of problems.
As you have found out yourself, one such client is WinSCP.
Using WinSCP scripting, your batch file would be like:
#echo off
winscp.com /log=ftp.log /command ^
"open ftp://ftpUser:Password#77.99.999.999/" ^
"cd /tst" ^
"get *" ^
"rm *" ^
"exit"
Though, if supported by the server, consider using a secure FTP over TLS/SSL, by replacing the ftp:// with the ftpes://.
See also a guide for converting Windows FTP script to WinSCP script.
(I'm the author of WinSCP)
I found a workaround for this problem:
http://winscp.net/eng/docs/scriptcommand_rm#examples
Using this software I was able to get and delete several files at the same time, and the code now is like this:
option batch abort
option confirm off
open ftp://user:Password#77.XX.XXX.XXX
cd /tst/
get *.* d:\www\*.bak
rm *.*
EXIT
Thanks for the help anyway.
#Martin Prikryl Thanks for the software and the explanation.
Regards!
I am writing a batch file that will do the following:
Connect to remote FTP site
Push all files from local director to remote FTP site
Disconnet
This works fine, but I also want to empty the local directory after upload (otherwise they will be transferred everytime). Unfortunately I have a problem where the connection to side might not always be possible, and thus if I add a crude del . to the end of my batch file, it will delete the files even if they havent been uploaded.
Can anyone think of a way around this? i.e. If file uploaded then delete.
#echo off
cd \
cd c:\temp
echo user bacon> ftpcmd.dat
echo eggs>> ftpcmd.dat
echo bin>> ftpcmd.dat
echo put *.xls>> ftpcmd.dat
echo bye>> ftpcmd.dat
ftp -n -s:ftpcmd.dat <Server>
del ftpcmd.dat
It seems ftp command does not return an errorlevel on which to operate, so best option is to redirect output of ftp command to a file and use findstr to check for errors in output.
The Windows FTP command does not provide a method to guarantee the file is complete.
A command line FTP transfer could fail and you will have a truncated file and FTP doesn't provide a method to detect this.