I am trying to run the below procedure from SSIS using SQL Task.
ALTER PROCEDURE [dbo].[sp_Previous_Load_Dt_Tm]
AS
BEGIN
SELECT Extract_Dt_Tm = ISNULL(CONVERT(VARCHAR(20), MAX([Previous_Load_Date_Time]),120),'')
FROM [Test_table] WITH (NOLOCK)
END
I am returning as below
When I use the variable type as String in SSIS :
If I update the variable type to date it gives me the value:
Question ) I am not sure why its returning a date when I am converting the date to varchar in the procedure.
Is this happening at run time?
If you change the variable type to datetime at design time, and the string returned from the stored procedure is written to it during execution, this would most likely be due to SSIS performing an implicit conversion, i.e. successfully parsing a datetime value from the string.
I imagine the images you posted are at design time, with the string not having been written yet, and the datetime having defaulted to the current date and time?
I just removed and re added the variable. And it worked!!!! Not sure what the issue was.
I have an execute sql task ( Task 1 ) which runs sql to returns a column called Note from Table A and stores it as a String SSIS variable type. In Table A, Note is defined as varchar(2000).
I then have an execute sql task ( Task 2) to run a stored procedure. The input parameter is Note varchar(max).
I run these 2 task in SSIS and get the following error:
DECLARE #..." failed with the following error: "The text, ntext, and image data types are invalid for local variables.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
I have tried several solutions with no success. How can you get round this error and get SSIS to store the variable and feed it to the SP ?
On sql server 2012 SSIS and hitting an old 2008 database where the SP resides.
You are somehow mapping a text type column in your tsql code. Before you return from your tsql proc convert the text column to a varchar column with select cast (textval as varchar(max)) and make sure your output variables are defined as varchars.
You need to make sure the Task 1 should only return one row.
example capture
Otherwise you need to use an Object variable. And use a For each loop container to loop thru that object.
I fetch a result set with an execute SQL task. It has only one column NullTime varchar. It has three rows, first one is NULL. I want to simply iterate and display the value of these rows. If I do it only by C# script, then there is no problem. The NULL is displayed as a blank. The same thing can also be done with a foreach loop.
How to do it with foreach - use that loop to read each row and set the value of each row to SSIS string User::STR_WORD. Then, simply display User::STR_WORD with a C# script task.
In the pure C# method, I can even assign the blank value (actually a NULL) to the SSIS string. But with foreach loop method, I get an error because of the NULL value.
The error is -
Error: The type of the value being assigned to variable "User::STR_WORD" differs from the current variable type. Variables may not change type during execution. Variable types are strict, except for variables of type Object.
How do I fix this error ? Is script the only alternative to what seems to be a flawed for
loop ?
A workaround for this is the Coalesce function which will convert NULLs to the value you specify.
SELECT
COALESCE([YourColumn], 'EnterValueHere') AS [YourColumn]
FROM YourTable
So, this will replace a null with the value you want to use instead. It will
prevent the weird FOR loop from suddenly crashing.
Create a temporary table to see this working -
create table ##tester
(names varchar(25))
insert into ##tester
values(Null)
insert into ##tester
values('Not a null value')
select names from ##tester
select coalesce(names, 'Null has been obliterated!') from ##tester
I have a stored procedure that gets executed through SQL SSIS using a Execute SQL Task.
The task has the following:
USE [OPPY_DWUSD]
GO
DECLARE #return_value int
EXEC #return_value = [dbo].[generate_merge_scdbk]
#Schema = N'dim',
#Dimension = N'VARIETY',
#ETLSchema = N'stg',
#ETLTable = N'vw_VARIETY',
#Execute = 1
SELECT 'Return Value' = #return_value
GO
Right now the way I have this setup, I have multiple Execute SQL Tasks with the same code but different values, about 20 Execute SQL Tasks.
Is there a more cleaner way to pull this off?
Here is one way of doing this. The example uses SSIS 2008 R2 with SQL Server 2012 backend.
Create a table to store your parameter values. Let's say the table name is dbo.SProcValues. Based on your stored procedure definition, the table schema would look like this.
CREATE TABLE dbo.SProcValues(
Id int IDENTITY(1,1) NOT NULL,
SProcName nvarchar(40) NOT NULL,
SchemaName nvarchar(20) NOT NULL,
Dimension nvarchar(40) NOT NULL,
ETLSchema nvarchar(20) NOT NULL,
ETLTable nvarchar(40) NOT NULL,
IsExecute bit NOT NULL
)
GO
Let's insert some sample data using the following script.
INSERT INTO dbo.SProcValues
(SProcName, SchemaName, Dimension, ETLSchema, ETLTable, IsExecute) VALUES
('dbo.sp_generate_merge', 'dim1', 'dimension1', 'stg1', 'table1', 1),
('dbo.sp_generate_merge_scdbk', 'dim2', 'dimension2', 'stg2', 'table2', 1),
('dbo.sp_generate_merge_scdbk', 'dim3', 'dimension3', 'stg3', 'table3', 0),
('dbo.sp_generate_merge', 'dim4', 'dimension4', 'stg4', 'table4', 0);
GO
On the SSIS package, assuming that you have the data source and connection manager already established. Create the following variables. Variable SProcValues will hold the parameter set that we stored in the above-mentioned table. Variable SQLInnerQuery will hold the query that will be used later in the inner Execute SQL Task. Other variables relate to each column available in the table so we can loop through each row and hold it in a variable.
Paste the following query in the value of the variable SQLGetParameters
SELECT SProcName, SchemaName, Dimension, ETLSchema, ETLTable, IsExecute FROM dbo.SProcValues
Select the variable SQLInnerQuery and press F4 to view the properties. Set the property EvaluateAsExpression to True and then click the Ellipsis button against the Expression property.
We need to set an expression that will evaluate to the EXEC stored procedure statement that can be later supplied to the inner Execute SQL Task. Set the following expression.
"EXEC " + #[User::SProcName] + " #Schema = ?, #Dimension = ?, #ETLSchema = ?, #ETLTable = ?, #IsExecute = ?"
If you click Evaluate Expression button on the editor, you can see what the expression will evaluate to. You will also notice that there is no stored procedure name in the below screenshot that is because the package variable SProcName currently does not have any value. During runtime, the SProcName will be assigned with the value from the table and this expression will automatically resolve itself.
On the SSIS package, drag and drop an Execute SQL Task. This task will run the following query to fetch the list of parameter values that are stored in the table dbo.SProcValues. Configure the General page on the Execute SQL Task as shown below. The example uses OLEDB connection and the connection manager/data source is named as Practice.
Configure the Result Set page of Execute SQL Task to store the result set from the query to an object variable.
Now that the first Execute SQL Task is configured to get the list of parameter values that should be passed to the stored procedure, you need to loop through the records.
Drag and drop a Foreach Loop container. Connect the Execute SQL Task's precedence container to the Foreach Loop container. Configure the Collection page of the Foreach Loop container as shown below. We are looping through the result set using the ADO enumerator.
Configure the Variable Mappings page on Foreach Loop container as shown below. As we loop through each row, we are storing the column values in respective variables so we can pass it to the next Execute SQL Task to run the stored procedure.
Drag and drop an Execute SQL Task inside the Foreach Loop container so that this task is executed each time we loop through a row in the result set. Configure the Execute SQL Task as shown below.
NOTE
You might want to configure the ResultSet property on this second Execute SQL Task according to your requirements. If you choose ResultSet, then you need to configure an appropriate object variable to accept the result set. I left it as None for this example.
Configure the values to be passed as parameters to the stored procedure.
Finally, the control flow would look something like this.
When the package runs, the loop will execute the stored procedure for as many records are returned by the SELECT query mentioned above, provided that you have all the stored procedures defined in the table rows are available in the database. I had created the stored procedures dbo.sp_generate_merge_scdbk and dbo.sp_generate_merge with the same parameters definition. That's the reason the package executed successfully.
You have the right concept, just need to use some concepts like variables, a foreach loop and parameters on the Execute SQL Task.
Control Flow
Your Control Flow would look something like this
Variables
I have 6 variables defined in SSIS
Dimension | String | VARIETY
ETLSchema | String | stg
ETLTable | String | vw_VARIETY
Execute | Int32 | 1
RecordSet | Object | System.Object
Schema | String | dim
The first Execute SQL Task will be a query or something enumerable like it. Currently, have a hard coded query to produce the supplied query values. Your solution could just be a chain of SELECT's UNIONed together. The goal of this step is to populate the RecordSet variable.
My Execute SQL Task returns a full result set
and I push that into my object thusly
ForEach Loop Container (ADO Recordset)
The ForEach Loop Container is going to consume that enumerable thing we established beforehand. It will go through each row and we will pop the values out of the object and assign them into local variables.
Change the Enumerator to Foreach ADO Enumerator. Select the object we populated with results User::RecordSet and then use an enumeration mode of Rows in first table
In the Variable Mappings tab, we will identify the ordinal based location for the values (column 0 maps to variable X). The only trick here is to ensure your SSIS Variable data types match the data type in the result set from your source query. Do note it's a zero based ordinal system.
At this point, if you click run you see it enumerate through all the rows you have sent into the RecordSet variable. I find it helpful to run it at this point to make sure I have all of my data types aligned.
Inner Execute SQL Task
I have taken your query and replaced the hard coded values with place holder. An OLEDB connection will use ? while an ADO.NET will use named #varname.
In the Parameter Mapping tab, simply map those local variables to the place holders.
Now you have a nice template for running the same proc with varying values.
Here's a simplified version of my sproc:
CREATE PROCEDURE usp_AddSomething
#i_input1 int,
#o_output1 int = 1 OUTPUT,
#o_output2 int = 2 OUTPUT
AS
-- Do stuff
If I don't explicitly set values for #o_output1 or #o_output2 within the sproc then I expect to see the default output values of 1 and 2. It works this way when I call EXEC usp_AddSomething ... from a normal SQL script. However, when I run the same statement in an SSIS SQL Task like so:
EXEC ? = usp_AddSomething
#i_input1 = ?,
#o_output1 = ? OUTPUT,
#o_output2 = ? OUTPUT
I get seemingly random values (e.g. 15305391 and 69085360) instead of the default 1 and 2.
But if the values are explicitly set somewhere in the sproc like so:
...
SELECT #o_output1 = 1
SELECT #o_output2 = 2
RETURN 0
Then I get the expected values of 1 and 2 in SSIS.
My SSIS vars are Int32, the Parameter Mapping on the SQL task uses LONG and the ordinal positions (i.e. 0, 1, 2) are setup correctly under Parameter Name.
It's easy enough to redundantly initialize the values of the output parameters to match their defaults, but I'd really like to know WHY it's not working. Maybe another quirky SSIS thing? SQL Server 2005 SP4.
A good way to see what's going on is to start a trace with profiler and see the t-sql that ssis is sending to the database engine. Then try executing that from ssms and see if you get the same results.