SSIS: passing derived column variable to execute sql task - sql-server

I followed the post from bilinkc as mentioned here
how to load extracted file name into sql server table in SSIS
I use this extracted file name in a variable called FileName in a data task within a For Each container. But when I want to use the same variable in a consecutive Execute SQL task it is throwing up the error
[Execute SQL Task] Error: Executing the query "
INSERT INTO HOURLY_DATA
..." failed with the following error: "Multiple-step OLE DB operation generated errors. Check each OLE DB status value, if available. No work was done.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
Parameter settings : Data Type is VARCHAR Parameter name is 0 and Parameter length is 1000.
Is there anyway to get around this error. WHen i checked on other posts they say it is related to a TEXT length error but I am not able to get my head around this solution for my scenario.
The full query is as given below:
This is the full query :
INSERT INTO HOURLY_DATA
(UNIQUE_NUM, NAME, FILE_NAME, USER_NAME)
SELECT DISTINCT UNIQUE_NUM, 'MANUS',?, 'MENON'
FROM MY_TABLE
WHERE UNIQUE_NUM IS NOT NULL
For the parameter variable, I have a package level variable defined which depends upon this expression:
REPLACE(REPLACE(REPLACE(REPLACE(#[User::varFilePath],#[User::varSourceFolder],""),"\",""),"My File",""),".xlsx","")
But when I evaluate this expression nothing happens. Could this variable be the issue?
Thank you in advance for your time and help.

Related

SSIS Access DB - Multiple-step OLE DB operation generated errors while executing SQL Procedure

When I am executing a Procedure with output parameters using SQL Task, I am getting the below mentioned error. There is no problem in db connection, number of passing params, sequence of params and data type.
Command for execution
EXEC [dbo].[spArchiveTablesCleanup] ? OUTPUT, ? OUTPUT, ? OUTPUT
Error:
[Execute SQL Task] Error: Executing the query "EXEC [dbo].[spArchiveTablesCleanup] ? OUTPUT, ? OU..." failed with the following error: "Multiple-step OLE DB operation generated errors. Check each OLE DB status value, if available. No work was done.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
SSIS Parameter Mapping:
enter image description here
SSIS Variable Declaration:
enter image description here
SQL Server procedure params definition:  
enter image description here
Remove 'OUTPUT' from your EXEC statement. I don't believe you actually need to have that there.
https://www.sqlshack.com/execute-sql-tasks-in-ssis-output-parameters-vs-result-sets/
I have changed the Output parameter Data Type from NUMERIC to LONG, it worked perfectly.
enter image description here
I also had this problem and I was able to solve it by checking the variables that I had assigned in the SSIS task that I no longer used.

Dynamically Create Views from parameters/variables SQL SSIS

I have a table that contains just under a million rows. I'm building a form using SSIS that asks for user input and uses the values as parameters to build a view from source data. I'm having trouble getting SSIS to create the view from a variable.
The purpose of this 'tool' is to provide a dialogue that programmatically builds a view and later an update statement based upon parameters defined via a form that will execute an SSIS package. A number of the ppl on my team know 0 SQL. Therefore this circumvents any SQL knowledge. Creating an entirely standalone app is not ideal as it would require too much additional overhead on my side and would deviate from a number of our existing processes that currently use SSIS/SQL to achieve similar results.
With that here is what I've tried/trying.
I have an SSIS package that contains 'Execute SQL Task'
This task brings up a form with 5 inputs (variables)
var1,var2,var3,var4,var5.
some vars are strings others are doubles, ints etc... (they all vary)
You populate the fields and hit okay.
These variables are passed to an 'Execute Package Task'.
Inside this package (Package B)
the vars are used in an 'Execute SQL Task'.
This task is attempting to take the users input and create a view with a where clause containing 4 other variables.
example:
Create View ? AS Select col1,col2,col3,col4 WHERE
col1 = ?
AND col2 =?
AND col3 =?.........
First it appears that using ? in the create view is invalid.
The error being:
Error: 0x0 at Build_Query: Incorrect syntax near '#P1'.
Error: 0xC002F210 at Build_Query, Execute SQL Task: Executing the query "CREATE VIEW ? as Select * from S_t_equip_template
..." failed with the following error: "The batch could not be analyzed because of compile errors.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established
correctly.
Task failed: Build_Query
If I use the create view variable as an expression and remove the variables/paramerters for the where statements, I can create the view no problem.
However the where statements throw errors once I add them back in. I've tried evaluating these as an expression in the 'Execute SQL Task' but as these are of various types I get the error:
[Execute SQL Task] Error: Executing the query "
CREATE VIEW testing AS SELECT
P.label,P.uniq..." failed with the following error: "The metadata could not
be determined because statement 'CREATE VIEW testingagain AS SELECT
P.label,P.uniqueid,C.label as Child_Label,C.uniqueid as Child_uni' does not
support metadata discovery.". Possible failure reasons: Problems with the
query, "ResultSet" property not set correctly, parameters not set correctly,
or connection not established correctly.
No idea what is going on. Any help would be appreciated
I've googled the error and found some info but the other use-cases are so different that it's difficult to understand the actual cause, or another work around.
AS requested (simplified example):
I've created a package variable (datatype string) called: View_Name
Execute SQL Task:
CREATE VIEW #[User::View_Name] AS SELECT
* from table1
where col1 = 100;
Specifically it does not like that I use a variable here.
If I set the View name everything works until: I move on to my Where clauses that contain variables.
Create a variable called type (datatype int)
I map the variable/parameter in my sql task
Example:
CREATE VIEW tempTable AS SELECT
* from table1
where col1 = ?;
This won't work, same error.
If i attempt to do the above via an expression or expressions I get the following error
[Execute SQL Task] Error: Executing the query "CREATE VIEW test_45678 AS SELECT P.label,P.uniquei..." failed with the following error: "Must declare the scalar variable "#".". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
Generally having to due with the variables cannot be evaluated this way. I'm guessing I'd need to evaluate each piece individually and build the expression piece by piece. That's fine but very inefficient and not maintainable.
I had some fundamental misunderstanding in my attempt to evaluate my expressions. Specifically syntax issues and having to cast each variable to a string.
My SQLstatement variable
ON C.pid = P.ID
where
C.width >="+(DT_WSTR, 8)#[User::Width] +"-"+ (DT_WSTR, 8)#[User::Range]+........
The final expression looks like so:
"Create View "+#[User::View_Name] + " AS SELECT " + #[User::SQLStatement]

SSIS Data Types

I have an execute sql task ( Task 1 ) which runs sql to returns a column called Note from Table A and stores it as a String SSIS variable type. In Table A, Note is defined as varchar(2000).
I then have an execute sql task ( Task 2) to run a stored procedure. The input parameter is Note varchar(max).
I run these 2 task in SSIS and get the following error:
DECLARE #..." failed with the following error: "The text, ntext, and image data types are invalid for local variables.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
I have tried several solutions with no success. How can you get round this error and get SSIS to store the variable and feed it to the SP ?
On sql server 2012 SSIS and hitting an old 2008 database where the SP resides.
You are somehow mapping a text type column in your tsql code. Before you return from your tsql proc convert the text column to a varchar column with select cast (textval as varchar(max)) and make sure your output variables are defined as varchars.
You need to make sure the Task 1 should only return one row.
example capture
Otherwise you need to use an Object variable. And use a For each loop container to loop thru that object.

Storing System::StartTime in SSIS to a datetime column [duplicate]

I am working in SQL Server 2008 and BIDS (SSIS). I am trying to generate a "load ID" for when a package is executed and store that ID in a load history table (which then populates subsequent tables).
My basic SSIS control flow is the following:
Execute SQL Task, Data Flow Task
The load table is created via the following:
CREATE TABLE dbo.LoadHistory
(
LoadHistoryId int identity(1,1) NOT NULL PRIMARY KEY,
LoadDate datetime NOT NULL
);
The editor for the Execute SQL Task is as follows:
General:
ResultSet = None
ConnectionType = OLE DB
SQLStatement:
INSERT INTO dbo.LoadHistory (LoadDate) VALUES(#[System::StartTime]);
SELECT ? = SCOPE_IDENTITY()
Parameter Mapping:
Variable Name = User::LoadID
Direction = Output
Data Type = LONG
Parameter Name = 0
Parameter Size = -1
SSIS is throwing the following error:
[Execute SQL Task] Error: Executing the query "INSERT INTO dbo.LoadHistory
..." failed with the following error: "Multiple-step OLE DB operation generated errors. Check each OLE DB status value, if available. No work was done.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
This error message doesn't really help me find the problem. My best guess is that it's due to the parameter mapping, but I don't see my mistake. Can anybody point out my problem and provide the fix?
I figured out my problem. System::StartTime needs to have DATE as its data type, not DBTIMESTAMP.
I was passing three parameters.
In the Parameter Name property I had:
0
1
3
Corrected it to:
0
1
2
It works now, no multiple-step operation generated errors message.

Referencing a variable in an SSIS package

I have an SSIS package which iterates over a table, and for each row in the table, passes the values from two different columns as parameters to a stored procedure. At least, that's what it is supposed to do. What it actually does is give me this error:
[Execute SQL Task] Error: Executing the query "EXEC sproc_AccessImport #mAccessLocation (SELECT M..." failed with the following error: "Must declare the scalar variable "#mAccessLocation".". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
I followed this tutorial in order to make the part that grabs the info, and the code that is supposed to execute the stored procedure is:
EXEC sproc_AccessImport #mAccessLocation (SELECT MVID FROM tbl_AccessMailDataFiles WHERE FileLocation = #mAccessLocation)
I've been looking around online, but haven't found anything useful. This is the first time I've used SSIS, so I suspect I'm simply doing something wrong. Does anyone know what exactly that is? Thanks.
Edit: I tried mapping the variable as a parameter, with the options:
Variable Name: User::mAccessLocation
Direction: Input
Data Type: VARCHAR
Parameter Name: Path
Parameter Size: 200
And changed the SQL query to:
EXEC sproc_AccessImport #Path (SELECT MVID FROM tbl_AccessMailDataFiles WHERE FileLocation = #Path)
It now gives me the error:
[Execute SQL Task] Error: Executing the query "EXEC sproc_AccessImport #Path (SELECT MVID FROM tb..." failed with the following error: "Must declare the scalar variable "#Path".". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
So, really, the same error.
Depending on how your package is set up you may need to map the SSIS variables to the parameters within the Execute SQL Task. the specific error you are seeing is most liklely because you are using an OLEDB connection and the statment should probably read
EXEC sproc_AccessImport ? (SELECT MVID FROM tbl_AccessMailDataFiles WHERE FileLocation = ?)
If you are following the example you linked to then you would map the User::mAccessLocation to the first and second parameter. I hope that makes sense, but I'm not 100% on how you have this set up.

Resources