I'm using an SSIS package to unzip a file that's on another server. I've looked around for solutions and can only see answers for files that are on the same server. I found this answer but couldn't get it working - how to unzip rar files which are in remote server using ssis task
I thought referencing the server like this would work in the arguments field on an execute process task, but it didn't:
x backup.gz "-o\\server\E$\folder\backup.gz"
I can use a batch file but it still doesn't work, I feel like I'm referencing the server wrong?
Related
I have an ogr2ogr batch file that reprojects SQL data into a new SQL Server table.
It works fine when I run the bat file manually but it fails if I run the bat file via a SQL Server stored procedure. I have given the gdal folders SQL Service permissions and xp_CommandShell is also enabled. I'm using
EXECUTE xp_CMDShell 'blah'
in the T-SQL script.
For some reason the ogr_MSSQLSpatial.dll causes it to fail.
ERROR 1: Can't load requested DLL: Z:\BroadSpectrumSQLTreeExtract\ogr2ogr\gdalplugins\ogr_MSSQLSpatial.dll
If I remove this dll the script runs via SQL but it means I need to add extra commands that the dll must take care of, such as setting source coordinate system. I haven't managed to get it working 100%. The furthest I got to was producing the reprojected table but the geometry field is empty.
The DLL does contain SQL commands to the system tables. Could this be a SQL Server security issue stopping it from working?
I again had this problem with another ogr2ogr bat while executing with SQL. If I put the bat in the same folder as the dll's it works fine.
I have CSV files stored in SFTP Server which I am accessing using winscp.
I have to import these CSVs into SQL Server tables.
Currently I am downloading files from SFTP and saving them in O Drive and created SSIS packages to import in SQL Server(Using O Drive as source)
How can I import data directly from SFTP server to SQL Server?
I would like to use SSIS for the same but would like to know other methods as well.
Thanks,
AP
There's a free tool called WinSCP which provides a command line utility to communicate with SFTP sites. You just need to understand the scripting language for the command line and execute the executable from SSIS.
Here you can find about this method: https://www.codeproject.com/Articles/23740/SFTP-with-SSIS-Packages
Another method to load data from SFTP to SQL Server is to use ETL tools, like Skyvia, Alooma, etc.
I use a Powershell script to copy files to an accessible location & then copy contents into a SQL table, not sure if you can skip a step & read the contents directly from the secure location (seem to remember I couldn't get it to work though).
It's a modified version of the powershell example here, followed by Import-Csv combined with Out-Datatable and Write-DataTable
There's further WinSCP code examples here
I'm new to SSIS and stuck with a problem and hope some of them would have already gone through any of this.
Task:
Copying files from a remote server to a local machine folder using File System task and For each loop container.
Problem:
The job executes i.e. files are getting copied successfully when I execute from the SSIS designer but when deployed the project on the SQL server instance it isn't copying any files in fact the target folder is totally empty.
I'm not understanding this strange behavior. Any inputs would be of great help!
Regards-
Santosh G.
The For each loop will not error out if it doesn't find any files.
The SQL Agent account may not have access to read the directory contents.
Check is your path a variable - is it being set by a config or /SET statement?
Can you log the path before starting the for loop?
Can you copy a dummy file and see can SSIS see this file?
How are you running the job - cmd_exec() can give spurious results with file I/O tasks
The issue was related to the user authorizarions of the SQL Server agent service.
When I execute the job from SQL Server it uses agent service and for that agent service you need to assign a service user who has access rights to the desired file path.
I recently had to move my files to a new SSIS server. Everything seems to be working except when I try to execute a bulk insert it tells me
(Cannot bulk load because the file "E:\FlatFiles\SSG\apmast.txt" could
not be opened. Operating system error code 21(The device is not
ready.).".
It does this for all my flat files. I found an article saying you need to give the MSSQLSERVER user full control of the files, which I did but this does not seem to fix it. Any other ideas? Do I need to give other files the same permissions? I really don't want to just throw full control around if I don't have to. Thanks
I figured it out, turns out that a bulk insert tells the server to look locally for the text file. I was trying to get SSIS to do a bulk insert of flat files from one server into another sql server on the network. As soon as I put the flatfiles on the remote server it grabbed and used them. This seems like a very odd way for it to work. I would expect it to push the files to the sql server instead of asking the sql server to look for the files locally via a hard path.
I need to import a flat file daily. The file changes its name every day. After the file is processed, it needs to be moved to another folder.
I noticed I can schedule jobs in the SQL Server Agent, and that I can tell it to run every hour or so and that I am able to add CMD commands to it.
The solution I found was to run a script to check if the file exists, since the folder should be empty or have at least one file.
If the file exists, the script renames the file to one used in the SSIS package and then it runs the SSIS package.
After the whole thing is done, it should rename the file again based on today's date and move it to another folder.
If the file does not exist, then it should do nothing and wait another hour or so to run again.
What's the best solution to this scenario? Is the script a good idea? Maybe is it possible to add the if/else -for the file exists- into the SSIS package? Or even make the script run from the SSIS package itself instead of adding it to the Server Agent?
EDIT:
It seems I was a little naïve, it's possible to run VB scripts from the server. Would that be the recommended solution? It does solve my problem, but I'm just wondering if it's a good idea.
This solves all my questions:
http://www.sqlservercentral.com/articles/Integration+Services+%28SSIS%29/90571/