I have an ETL (SSIS) job which creates a file in a folder named as TEST_20170505.csv before it sends the file to an SFTP server.
There would be multiple file in the folder such as, TEST_20170504.csv, TEST_20170503.csv. Currently I am using following sftp script file (File.txt) in a batch file.
lcd E:\localpath\
cd \sftpserverpath\
ascii
put *.csv
bye
This is my .bat file.
sftp -oIdentityFile=E:\sftp\filepath\ssh.ppk -B E:\sftp\filepath\File.txt username#ipaddress
But this will upload all the files in the local path to SFTP server instead of taking the latest/last modified/current date file.
OpenSSH sftp cannot do this on its own. But you can use some fancy batch file construct to select the latest file and then generate an ad-hoc sftp upload script.
Some references:
How do I write a Windows batch script to copy the newest file from a directory?
Windows batch file - Upload only latest file to FTP
Or use some more advanced Windows command-line SFTP client.
For example with WinSCP scripting, it's as easy as using -latest switch in the put command:
open sftp://username#example.com/ -privatekey=ssh.ppk
lcd E:\localpath\
cd \sftpserverpath\
ascii
put -transfer=ascii -latest *.csv
exit
Run the script (upload.txt) like:
winscp.com /script=upload.txt /ini=nul /log=upload.log
You can even have WinSCP generate the script/batch file for you (you just need to add the -latest switch manually).
References:
Uploading the most recent file with WinSCP
Converting OpenSSH SFTP script to WinSCP script
Note that WinSCP uses .ppk format of private key files. While OpenSSH sftp uses PEM format. Even though your key file is named .ppk, it cannot be real .ppk file, otherwise the sftp would reject it. You have probably converted original .ppk file to PEM, but incorrectly kept an original extension. You have to use the original .ppk file with WinSCP (if you do not have it, you can convert the PEM back to .ppk using PuTTYgen).
Also note that WinSCP actually supports the text/ascii mode (-transfer=ascii), while sftp does not (there's no ascii command).
(I'm the author of WinSCP)
The solution is instead of "put *.csv" in your script file, you specify the actual filename you want to upload
Related
I have a batch file that receives an argument then passes that argument to a text scrip file that invokes WinSCP to download a file.
My goal is to download today's file.
I am able to download the file if in the text script file I don't have >=today.
What do I need to change to download today's file, if it exists?
Batch File
set arg1=%1
set CurrentPath=C:\Temp\
rem drive
c:
rem folder do WinSCP
cd C:\Program Files (x86)\WinSCP
rem Download file
winscp.exe /console /script=%CurrentPath%sftp.txt /parameter %arg1%
Text script file
option batch abort
option confirm off
# Connect
open <.....>
# Download file to
get %1%>=today "C:\Temp\"
# Disconnect
close
The error I'm getting:
Mask is invalid near '=today'
I do not think this is about passing parameters.
But the today keyword is a "recent" feature (WinSCP 5.14 from October 2018). So you probably have an old version of WinSCP that does not support it.
If you do not have a version of WinSCP that supports today, you can use %TIMESTAMP% pattern:
get %1%>=%TIMESTAMP#yyyy-mm-dd% "C:\Temp\"
Though you should always use the latest version of WinSCP, if you want to stay safe.
Replace
As-Is
# Download file to
get %1%>=today "C:\Temp\"
To-Be
# Download file to
get -filemask= %1%"*>1D" "C:\Temp\"
I have WinSCP script that downloads a file from the SFTP server to the local machine. How can I rename the file from within the WinSCP script?
The script is being run from Windows machine.
The WinSCP script:
option batch off
option confirm off
open sftp://username:password#servername -hostkey=""
option transfer binary
get -latest "oldname.csv" c:\localfolder
mv c:\localfolder\oldname.csv latestname.csv
exit
I tried renaming using the commands:
rename InitialFileName FinalFileName
then tried using:
mv InitialFileName FinalFileName
But the script is throwing the error:
File not found, Language: en
Tried stuff from this blog
If you want to download the file to a different name, specify the new name directly in the get command:
get "oldname.csv" c:\localfolder\latestname.csv
(Note that when downloading one specific file, the -latest switch is pointless. The latest file out of one file[s] is that one file.)
I have gone through the following guide to set up an SSIS package to retrieve a text file located on an SFTP server:
https://www.mssqltips.com/sqlservertip/3435/using-sftp-with-sql-server-integration-services/
To summarize, the SSIS package executes PSFTP.exe (A PuTTY tool) which takes the necessary credentials to connect to the server. It also takes a batch file that it executes after connecting. This batch file contains the commands to retrieve the desired text file. To start from the guide, it simply contains a cmd command to change directory, and a get command to retrieve the file:
cmd DataDump
get TeleMarketingResults.txt
All of this works fine.
The issue arises when I try to make this batch file logic more complex as it does not seem to recognize basic keywords. For instance, I would like to modify it to retrieve the most recent file, so I tried adding this:
for /f %%i in ('dir /b/a-d/od/t:c') do set LAST=%%i
echo The most recently created file is %LAST%
but then I get these errors:
psftp: unknown command "for"
psftp: unknown command "echo"
If I execute the batch file manually in a local directory, it works. The issue only occurs when passing it as a parameter to PSFTP.exe. Why is this?
psftp script file can contain psftp commands only. for, set or dir are not psftp commands.
There's hardly any reasonable way to retrieve latest file using psftp. You would have to do it in two steps. First to retrieve listing and store it to a file. Then parse that file using some smart batch file commands to find the latest files. And then run psftp again to download that file. It is cumbersome and ineffective as it requires two connections.
You better use a more powerful SFTP client. For example it's trivial with my WinSCP SFTP client. See
Question WinSCP select most recent file or
WinSCP article Downloading the most recent file.
I am new to batch and trying to execute a script I wrote, essentially I want to go to a remote server copy the files there and transfer it to a folder in my local directory. I am executing the bactch file but nothing is being copied any suggestions would be great. This is my script
open sftp://site:#ftp.site.net -hostkey="server finger print"
synchronize local C:\Users\localdirectory\Desktop\test2 /Home/folderA/NewFiles
exit
I am positive all the information is correct because that's how I login with WinSCP. I got this script from https://www.youtube.com/watch?v=ndvEYOQLc4c
This is WinSCP script. There's no batch file in your question.
To run WinSCP script, use for example:
"C:\Program Files (x86)\WinSCP\WinSCP.com" /script="C:\path\to\winscp.txt"
(assuming your WinSCP script is in C:\path\to\winscp.txt).
You can put the above command to a batch file.
See guide to scripting with WinSCP.
You can also have WinSCP GUI generate both the script and batch file for you (or even a batch file that directly contains the WinSCP commands).
I'm using a PSFTP script to upload files to a remote server for processing and there are specific steps that I need to take with file in order for the file to be processed correctly by their system. Specifically, I have to upload the file with a ".u01" in the extension, then rename it to a ".r01". after it has been uploaded.
Is there a way to replace just the extension of the uploaded file? The rest of the filename can't be modified because the filename is corresponds with the contents of the file.
And no, I can't just upload it with a ".r" extension. Tried that already :P
The script is simple. It uploads all files from the "to_upload" directory into "ul" on the remote server:
cd ul
put -r to_upload/