this is one that has me stumped and Ive been doing this a long while.
Migrating to SQL server 2016, large number of ETL. Easy enough.
One of the ETL packages has a simple script task to take a table of files, run a file exists foreach loop.
it uses a project parameter to create the unc ( \servername\share) and then binds that to the file name in the script task.
use an environment config setup in SSISDB
execute in SSDT works fine, deploy to catalog and it cant see the file. i know youll say permissions, but ive permissioned everyone group to share and drive in case its that. SSISDB execution means it should be running under my security context and im domain admin, local admin and creator owner of the share.
even strangeR, i have created simple package to grab the contents of one of the files and import into a dump table in case permissions or pathway were duff ( even though they work in SSDT might be the enviroNment config in SSISDB). THIS WORKS FINE, therefore it cant be the envrionment setup of SSISDB being referenced.
please note this is not running from an agent job yet so wont be due to agent server account issue. need to get it running from ssisdb first then ill create an agent job
So -- script task cant see unc share, built from two variables, that works in ssdt and its running under same credentials...
Go
For what its work the script task code is
Dts.Variables("BolFileExists").Value = File.Exists(Dts.Variables("StrLoadFileLocation").Value.ToString & Dts.Variables("StrCurrentFile").Value.ToString)
This is a slightly different answer as it shows a different approach and removes the script task. I use a foreach to check if the file exists using GUI tools provided by SSIS:
Well I found the answer and I deserve to punch myself in the face.
Tried everything, it was a file variable and path variable being pulled together in the script task so tried concatenation that before the script task, pumped this into a table to ensure it was going to write table.
Literally everything was fine and still didn’t work.
The issue....
Building it as a 2017 package onto a 2016 Sql server.
I’ve not found what was missing dll wise but it must have been one of those that meant the script task couldn’t find the files but weird it didn’t break and just said the files weren’t there!
Thanks all for input, I’m going to go put my head in the door and slam it
Related
I have designed a SSIS project and deployed it to SQL server and also created the job to run on daily basis but its giving me this error when executing this as job (doesnt give any error within VS):
There is this CLSid in this error message but there is no application associated to it in
--> Component Services -> Computers -> My Computer -> DCOM Config
But this CLSid is registered inside registry editor
About this particular task on which this error is occurring: This is a script task which is modifying and deleting the un-wanted rows from the excel file in which I am trying to write SQL table data.
Script task code looks like this:
I have been working for hours now trying to fix this problem but no success. Kindly guide me how can I fix this issue. If any other information is required related to this project, please let me know....
Doing Excel automation in a SQL Server agent job is totally unsupported and probably won't work.
To have even a ghost of a chance of making this work you'll need to run a real desktop session on the server and automate Excel in that. Excel expects a real user to be logged in with a full profile. And Excel has failure conditions where it displays a popup window, which you'll need to be able to access via remote desktop.
You can read and write Excel files on a server with the OpenXML SDK, without actually having to run Excel. There's also a wrapper library called ClosedXML which you may find easier to use than using OpenXML directly.
tl;dr;
You need to install Office (Excel) on the server AND ensure that you install it in a manner that mirrors the SQL Agent's expected bit-edness. Default for Agent is going to be 64bit, default for Office is still 32 :(
Error guessing
You have a script task that uses the Office interop libraries to delete some rows (2 through 11?) out of a spreadsheet.
You have Office installed on your machine and therefore you have the libraries installed. Excel still has COM based "stuff" in it, thus the interop and errors shrieking about the CLSid, registry, etc but that's likely just secondary errors because there is no base "application is not installed" exception to be thrown.
If Office is installed, then ensure your agent execution model matches the version of Office. If 32 bit Excel is already installed, don't potentially break everyone else's stuff by uninstalling and reinstalling as 64 bit, just got the Advanced section of the SQL Agent Job Step and check the 32bit box.
Once all that's done, then if you're still getting errors but new ones, then the existing comments mentioning permissions may come into play - it depends on where the Excel document actually exists (on the computer where SQL agent can access vs on the computer where it cannot vs networked drive)
Good luck in not finding people on the sanctions lists.
I've seen this question asked several times but never with a satisfacory answer. I have read through all the posts I can related to this and tried as much as I possibly can.
I have an SSIS package that loops thru a network folder of Excel files, I won't explain what it does inseide the loop container as that is not relevant.
I refernce the folder via a UNC \servername\folder
The package works fine from within Visual Studio.
I deployed the package to the Integration Services Catalogs on the
server
After I deploy, I connect to the server from my local pc via SSMS and then I execute it from SSMS via "Integration Services Catalogs"... Execute - This fails.
However If I remote desktop onto the SQL Server box, then start SSMS, connect to the SQL Server using my own credentials and execute the package using exactly the same method as above it works fine.
When I look in the logs of the failed attempts I see a warning that "The For Each File enumerator is empty". I'm not sure if this is telling the full story as, if I rename the network folder, I get the same message, (I expected to see an error that the folder was not found) - this may or may not be relevant.
I've sketched up a quick overview of what works and what doesn't
NOTE The "script execution" method is just a t-sql script for running these packages. This is the method I will use eventually but at the moment I'm focusing on the simple right-click execute method as this essentially does the same thing.
Since I did this I have tried a few more things...
I've tried accessing the folder as a UNC, a mapped drive and also UNC using IP address instead of server name.
I recreated the issue on our development server so I could change
service accounts etc.
I tried the default accounts on both SSIS and SQL Agent services.
I tried changing these to domain accounts and network service accounts.
I get the package to log the folder name which is and expression - this always looks correct. I do the same with the user name which always shows me as the exection user.
I can change most things as I can test on the development server with the excpetion of testing with a domain admin account so any suggestions would be greatly appreciated.
Let me start by saying I've read just about everything on 'Unexpected Termination' errors the last few days and am none the wiser!
I have a number of XLSX files in a network folder, and have 2 packages in a project to read them and load to SQL server (2017) using Excel connector (ACE.16) on dev and prod.
One package loops through about 35 workbooks and appends them all to a single SQL table.
The second package simply reads a single (~5mb) file and writes to a SQL table (with a single data transformation along the way - its not that, I removed it and it still failed)
Both packages use same source folder.
Both packages write to same destination DB.
Both packages run fine on my local machine.
Both packages can be executed correctly from the SSISDB catalog on the server (Right-click-->Execute).
When scheduled in Agent (we have a proxy account with correct folder permissions) the looping package runs just fine, whilst the simpler package fails with 'Unexpected Termination'
Verbose logging reveals nothing, as does Event Viewer on the SQL box.
I started to look into other options such as converting to CSV first using Script Task and Excel Interop but we are a 365 site now so we'd need Office client installs on dev and production server (and not knowing much C# won't help!)
My next route is to see if I can get it to write to CSV destination successfully. If so then I might be able to go XLSX-->CSV-->SQL without having to use Interop or external libraries.
Unfortunately I've not been able to turn up anything further in my searches, so wondered if someone more enlightened than me might be able to suggest where to look next.
We've run into a strange issue. We have some SSIS 2008 packages that had been working fine for quite a while. As part of the work flow they copy files to a backup folder, and then delete the original file. (No, I don't know why they didn't just make it a File Move task.) Suddenly, the delete task has stopped working. It's not just our packages either, it's the packages of a different group on a different SSIS server. Same target folders though.
The server team has gone over the folder permissions, the account the jobs run under have full access. And, if it was an access issue, how could we move the file but not delete it?
We finally found the answer. The file server we're using is Linux based. It is case sensitive, but doesn't throw an error if you use the wrong case. Was a very tough issue for us to track down.
I have to say that I hate myself for such general question as "What I am doing wrong?" but I simply have no idea what can be the problem:
I've created SSIS package that takes the data from flat files (CSV), counts the average on one of the columns, groups by date and writes it to the database and deletes the original file. All works fine when executed within SSIS, but when I am scheduling it within Server Agent it simply doesn't work - log reports success but there is no new data in the database and the .csv file exists in its original location.
I know the problem with protection level set up in SSIS, so I've changed it to "EncryptAllWithPassword" and I use the same password with Server Agent.
Here is a link to the Server Agent Job script (created as "script job as DROP and CREATE")
Edit: Just to make things weirder, using
dtexcec /f {filepath} /de {password}
executes program without problem. I know I can shedule such command in the Windows itself, but i'd like to keep all scheduled jobs in one place - in the Server Agent
EDIT: Solved by changing the path to UNC
There are two important things to remember when setting up packages to run via a SQL Server Agent job.
Use UNC paths for all file locations, no matter how simple. There is a high probability that the server will have a different view of the file structure than your development machine, so UNC paths ensure that both machines are referencing the same paths.
Use a proxy account to execute that package, as described here http://www.mssqltips.com/sqlservertip/2163/running-a-ssis-package-from-sql-server-agent-using-a-proxy-account/.
The proxy account must have access to the physical paths and the server objects.
This also allows for security stratification on your various packages (not all packages need access to everything).