I have an SSIS solution that inserts content of .csv files into my database. The happy path scenario works great. However, some of the database fields are required. If I, for example, let my solution load an empty .csv file with not nullable fields in the database, the solution will fail, ofcourse. I have used the SSIS failure flow to put these files into a seperate folder. This works fine. When the package finishes, the package finishes and says it completed with errors. This translates to a failed job in the SQL Server agent. The job however did exactly what I wanted: process the valid files and archive the invalid files.
My question is, is there any way I can tell SSIS it's done the right thing and it shouldn't return an error on finish?
Thanks in advance. I hope someone can help me.
From Agent Job you can try, In the Job Step Properties dialog box, select the Advanced page
OR
You can ignore the failure from the OnError property of the task:
You can also go to the task that is causing the package to fail, and change its FailPackageOnFailure property to "False".
Related
I'm trying to load multiple files from a location into DB using Foreach Loop Container & DataFlow task in SSIS.
It's getting crashed while I try to execute the package. It's not giving any error message, whenever I execute the package it crashes and closes the visual studio app immediately. I have to kill the debug task in the task manager for the next execution of the package.
So I tried the below steps:
I used a FileSystem task instead of DataFlow task to just
move all the files from the source to the archive directory, which ran
fine without any issues.
Ran the DataFlow task individually to load a single file into DB,
which was also executed successfully.
I couldn't figure out what was going wrong here. Any help would be appreciated! Thanks!
Screenshots
All screenshots look fine to me. I will give some tips to try to figure out the issue.
Since the File System Task is executed without any problem, there is no problem with the ForEach Loop Container. You can try to remove the OLE DB Destination and replace it with a dummy task to check if it causing the issue. If the issue remains, it means that the Flat File Source could be the cause.
Things to try
Make sure that the TargetServerVersion is accurate. You can learn more about this property in the following article: How to change TargetServerVersion of my SSIS Project
Try running the package in 32-bit mode. You can do this by changing the Run64bitRuntime property to False. You can learn more about this property in the following article: Run64bitRunTime debugging property
Running Visual Studio in safe mode. You can use the following command devenv.exe /safemode.
Workaround - Using Bulk Insert
Since you are inserting flat files into the SQL database without performing any transformation. Why not use the SSIS Bulk Insert Task. You can refer to the following step-by-step guide for more information:
SSIS Basics: Bulk-Import various text files into a table
As mentioned in the official documentation, make sure that the following requirements are met:
The server must have permission to access both the file and the destination database.
The server runs the Bulk Insert task. Therefore, any format file that the task uses must be located on the server.
The source file that the Bulk Insert task loads can be on the same server as the SQL Server database into which data is inserted, or on a remote server. If the file is on a remote server, you must specify the file name using the Universal Naming Convention (UNC) name in the path.
I'm maintaining a project using Microsoft SQL Server 2016 (SP1) (according to this script) which heavily depends on recurring jobs (mirroring certain external db's and so on).
Especially the mirroring jobs are essentially based on SSIS packages which define a datasource, then execute a hardcoded SQL query and afterwards store the results in the specified destination.
Unfortunately the source databases where moved to a different domain and thus aren't accessible via the previous url.
My issue right now is that I simply have to change the source destination url but I'm not able to do that. There are plenty of ways to 'modify' SSIS packages but none of them seem to work with me.
What I managed (and seems the most promising) to do is to open the 'Integration Services...' part of my db, export the jobs to my desktop, modify them with Notepad and reimport them. And they seem to work if I execute them separately. But as soon as I try to execute the packages via SQL Server Agent it fails screaming:
Description: Failed to decrypt protected XML node "DTS:Password" with error 0x8009000B "Key not valid for use in specified state."
Does somebody know whats going on here and how I'm able to solve this? No password or username changed, only the connection string.
Is it even possible to manage a package like that?
Thank you for your help!
After further investigation I detected that even a newly created job didn't run properly. It was kind of strange that a package would run without any issues while directly executed but not via the SQL Server Agent, so I assumed it may be a rights issue and it was!
Somehow the Server Agent wasn't allowed to decrypt (although I never changed the executing user of a step) the password anymore.
I was able to work around my issue by simply creating each SSIS package again (some click hell but ok) but this time I secured 'sensitive data' with a password instead of the users key.
Afterwards I had to change each job step with a reference to to the damaged ssis packages and obviously type in the new passwords.
Seems to work again.
Thanks anyway
I have a SSIS package that was copied from somewhere else and I had delete most of the connection string it has and only kept 3 of them. I checked all the steps and made sure none of the step is using a connection string other than the 3 that I have. But When I run the package it will throw me some errors saying:
Error
The connection "{96C72D18-B8B3-4FCB-96A0-4B5CC0461C10}" is not found. This error is thrown by Connections collection when the specific connection element is not found.
It doesn't matter when I run the package in BIDS but it will fail my job when I run it using SQL Agent Job. What can I do to find out this connection string and delete it? I have checked everywhere.
The connection manager must be referenced somewhere or it wouldn't be throwing that error. In my experience this should result in a red X being shown somewhere in the solution either during the design or when you try to execute it. A commonly missed place is in event handler tasks since they don't jump into view when you run the solution.
If you know which package is causing the error you can right click the package in the solution explorer and select "View code". Then do a text search for the offending GUID. DTSX files are just XML. I wouldn't recommend manually deleting the text but you should be able to identify which element is referencing the deleted connection manager.
Can you go to the Job step properties data sources tab and uncheck the offending connection?
Posted as an answer instead of a comment because I was unable to leave a comment.
I have a SSIS package which look for 8 files in a pre-defined location. Using script task I am checking if any files are missing? If any files are missing then I send an email stating files are missing. Now I want to stop the current package if any files are missing after sending email task.
From the Microsoft link:
RunningPackage.Stop Method
I can see I can stop the ssis package by stopping the ssis service (from the SQL Server which is running SSIS Service) what I do not want to do as I am not sure if it will start the ssis service automatically again. Also I do not have permission to see and run the packages in the ssis server and test this way.
I am not sure about how to stop using DTEXEC tool either. I would appreciate any kind of help.
Just fail the container by adding code "dts.TaskResult=ScriptResults.Failure".
After that, you can add an "Failure" container (just add a container and change the green arrow to a red one" to send out the email.
Once you do this, you must force a "fail" on the email container and go to the container's properties (you can just right-click and choose "Properties"). Look for "FailPackageOnFailure" and change that to "True". Hope this helps.
I created an sql server agent job that consists of steps that are calls to ssis packages.
Let's say that the job is being executed and one of the steps (that is a ssis package) has a file system task/sql task etc that fails. Is there any way that can I retry/rerun this particular package step (file system task/sql task etc) and then go on with my execution of the rest of the job steps after I fix my error? I know that you can retry certain sql server agent jobs and steps but I can't find any way to retry this step and resume my execution in case something inside the package fails.
And I also want to know if there is any way to disable certain package steps from the sql server agent "level" - without having to open Data tools.
Thank you.
Go into the job properties, and into the step that contains the SSIS package. Go to the Advanced tab, and you can modify the number of Retry Attempts for that step.
You could always put some error handling into the package in question. If an exception occurs, it will be handled accordingly allowing the other packages to continue. I always handle exceptions within the packages and log exceptions to a table.
In relation to your other question:
"And I also want to know if there is any way to disable certain package steps from the sql server agent "level" - without having to open Data tools. "
You could always go down the PowerShell route, create a script that takes a parameter. The parameter specifying whether or not you want a specific package to run.
If you want to skip certain steps within the package then you would have to create some sort of precedent constraint that acts on a variable you have set.