SQL Server agent for scheduling SFTP using WinSCP under SSIS - batch-file

Question

I have a batch script which generates a WinSCP upload script to upload a file to an SFTP location. Now when I run the batch file via command prompt - it runs successfully and loads it. I called the same thru SSIS Execute process task - it runs successfully and loads it. Now when I put the same on SQL Agent - I tried the following two options:
Using Operating System (CmdExec) - cmd.exe /c "\.bat"
Added the SSIS package to SSISDB and added it as a job step.
With both the above options the job showed a successful run. However the file is not uploaded! Any ideas on what is happening?
Here's my batch script:
echo off
SET winscp=C:\"Program Files (x86)"\WinSCP\WinSCP.com
SET stagingDirectory=\\<staging path>\
SET scriptPath=\\<ScriptPath>\UploadScript.txt
SET ftpHost=xx.xx.xx.xx
SET ftpUser=user
SET ftpPass=password
SET fileName=Test.xlsx
SET ftpFlags=
#REM ftpFlags: -explicit
echo deleting uploadScript if it already exists
IF EXIST %scriptPath% del /F %scriptPath%
IF EXIST %scriptPath% exit 1
echo Generating WINSCP Upload Script
>>%scriptPath% echo option batch abort
>>%scriptPath% echo option confirm off
>>%scriptPath% echo open sftp://%ftpUser%:%ftpPass%#%ftpHost% %ftpFlags%
>>%scriptPath% echo option transfer binary
>>%scriptPath% echo put %stagingDirectory%%fileName% /
>>%scriptPath% echo close
>>%scriptPath% echo exit
echo Launching WINSCP upload
start /wait %winscp% /console /script=%scriptPath%

Answer 1

As you start the WinSCP via the start (why?), the exit code is not propagated to the SSIS. So, you never learn, if the script fails. And it most probably fails.
You also should enable logging, so that you can see what's wrong.
You should use this code to propagate the WinSCP exit code to SSIS and to enable logging:
%winscp% /log=\\<ScriptPath>\UploadScript.log /script=%scriptPath%
exit /b %ERRORLEVEL%
(Note that the winscp.com does not have the /console parameter)
Anyway, one clear problem is that you do not specify an expected SSH host key in the script. When you run the script manually, you probably have the key cached in the registry of your Windows account. But under the SSIS a different account is used, and its host key cache is likely empty. You should add the -hostkey switch to the open command in the script to make the script independent on the cache. See Where do I get SSH host key fingerprint to authorize the server?
When testing the script, add the /ini=nul parameter to isolate the script from your configuration.
For this and other hints, when debugging WinSCP running under SSIS, see My script works fine when executed manually, but fails or hangs when run by Windows Scheduler, SSIS or other automation service. What am I doing wrong?
And finally, see WinSCP SFTP Task for SSIS.

Answer 2

Your variable seems set incorrectly. To manage with a space in the path and into the variable you have to put in quotes the whole path or the whole variable.
i.e.
set "winscp=C:\Program Files (x86)\WinSCP\WinSCP.com"
echo start "%winscp%"
:: output: start "C:\Program Files (x86)\WinSCP\WinSCP.com"
or
set winscp="C:\Program Files (x86)\WinSCP\WinSCP.com"
echo start %winscp%
:: output: start "C:\Program Files (x86)\WinSCP\WinSCP.com"
Another point, you have to check this file: UploadScript.txt because your script adds new lines rather than remake the file.
change this line to >%scriptPath% echo option batch abort instead of >>%...
Ah, I did not pay attention to the IF EXIST.