I'm working on a datawarehouse.
I wrote a script to loada dimention table by creating date entries (such as the SSAS wizard but directly in my SSIS ETL process). It works well in SSMS and directly in a T-SQL task in SSIS without using parameters).
This script doesn't provide any ResultSet, this is just a loop to insert data in a table.
Here is a quick look of my query.
USE [MySQLServerDatabase]
GO
-- Some parameters used by the script.
DECLARE #PREFIX_YEAR_NAME [nvarchar](50) = 'Year ';
DECLARE #PREFIX_QUARTER_NAME [nvarchar](50) = 'Q';
DECLARE #PREFIX_WEEK_NAME [nvarchar](50) = 'W';
DECLARE #PREFIX_MONTH_NAME [nvarchar](50) = 'M';
DECLARE #DefaultBeginDate [datetime] = '01/01/2000';
DECLARE #Date [datetime];
SET #Date = #BeginDate
WHILE #Date <= #EndDate
BEGIN
-- ...
-- INSERT INTO ...
END
-- ...
However, to get something more easy to use and to maintain, I would like to use SSIS variables directly in the script.
Here is my params (Project.params file).
My script needs all of them to work :
Then, I added a "Execute SQL Task component" containing my SQL query (OLEDB connection, direct input method). I created my 5 parameters :
How to use parameters in the SQL query ?
I tried to use the same name as the variables in the script, it doesn't work.
I tried to use indexes (0,1,2 etc.) names and to use '?' in the script, I doesn't work.
Here is the error when using '?' as parameters :
Error: 0xC002F210 at Execute SQL Task, Execute SQL Task: Executing the
query " DECLARE #PREFIX_YEAR_NAME nvarchar = ?; D..." failed
with the following error: "Multiple-step OLE DB operation generated
errors. Check each OLE DB status value, if available. No work was
done.". Possible failure reasons: Problems with the query, "ResultSet"
property not set correctly, parameters not set correctly, or
connection not established correctly. Task failed: Execute SQL Task
Any idea to solve that ?
Thanks.
you can put this query into stored procedure. In SQL Task Editor in SQLStatement you can execute it by exec [sch].[sp_name] ?,?,?...,? (each ? is one parameter).
In your 'Parameter Mapping' tab in Parameter Name column you can use numbers 0, 1, 2... n (it's order of params in your stored procedure).
Related
I'm trying to create a program that upload an excel table to SQL server and run a procedure of that takes the table as an input. The procedure is pretty complex with declare statement using injected variable, and dynamic variable (e.g. derived from injected variable, or select the latest table in the database). I tried several things but the proc were not executed.
Server='IP address'
Database='database name'
Driver='ODBC Driver 17 for SQL Server'
date = '20221220'
MONTH = '202211'
db = pyodbc.connect(driver=Driver, server=Server, Trusted_Connection='yes')
dbc = db.cursor()
stored_proc = f"exec db..proc #date = '{date}',#Month='{MONTH}'"
dbc.execute(stored_proc)
#I also tried this, but it did not worked either
#dbc.execute(f"db..proc {date}, {MONTH}")
#dbc.execute("db..proc ?,?", [date,MONTH])
I expected that the procedure was successfully executed and data are updated. However, no error were shown in python terminal, but data in the needed SQL table were not updated either. I suspect that pyodbc can run select, insert into, update, ... statements, but not the declare statement.
Can any one help me with this?
Many thanks
I wanted to create package in SSIS.
At one of the first steps I would like to set the 'bookmark' variable (data type in SQL is datetime) using SQL Server stored procedure. Than I would like to pass that variable to other stored procedure which would prepare the table to the export.
Unfortunately I got stuck with the issue which I guess is related to datatypes in SSIS, could you please have a look?
So I created an variable time for that:
Than I used execute SQL task which have as SQL Statement:
exec MyProcedure ?,?,? out
And Parameter mapping as per below:
During the execution I can see that it is assigned as an expected (the only thing which is odd at that place is that SSIS uses MM/DD/YYYY format, while I have everywhere YYYY/MM/DD)
After that I have the SQL task which is calling the 2nd procedure:
exec MyProcedure2 #var1 = ?, #var2 = ?, #bookmark =?
And that component is failing all the time with the errors as per below:
[Execute SQL Task] Error: Executing the query "exec MyProcedure2
#var1 = ?, #var2 = ?, ..." failed with the following error: "The type
is not supported.DBTYPE_DBTIME". Possible failure reasons: Problems
with the query, "ResultSet" property not set correctly, parameters not
set correctly, or connection not established correctly.
Please note that the procedure runs without any issues when I trigger that directly from SQL with the same values.
My understanding/guess is that I messed up something with data types in SSIS... but I was trying change DBTIME to DBTIME2, DBTIMESTAMP, DBDATE, DATE.. and each time I was getting similar error messages and at this moment I am out of ideas.
If you're using an OLEDB connection, I think User::time must be type NVARCHAR because it's an input variable. I think it's different for output variables.
https://learn.microsoft.com/en-us/sql/integration-services/control-flow/execute-sql-task?view=sql-server-ver15#use-date-and-time-parameters-with-ole-db-connection-managers
I am using SSIS 2016. I need to execute a stored procedure that returns 4 result sets. I only need to keep the first result set and write this to a table. I can not modify the stored procedure. I do not care about any of the data returned in the other result sets. The stored procedure is in a SQL Server 2016 database. Results will also reside in SQL Server 2016.
I currently have this process running in SSIS 2008 using the "SQL Command" data access mode in an OLE DB Source like below. I have this in a For Each Loop Container to pass a series of param values to the stored procedure as I execute it multiple times for different param values on a daily basis.
SET FMTONLY OFF;
EXEC myProc
#Param1 = ?,
#Param2 =?,
#Param3 = ?;
By default SSIS 2008 is only returning the first result set, which has worked for me as I only care about the first result set.
I am using the Native OLEDB SQL Server client. From what I have read, it has changed the way it handles multiple result sets. I have used the WITH RESULT SETS to define the first result set but if I execute SSIS will fail indicating other result sets need to be defined.
In short, what is the best approach to duplicate what works in SSIS 2008 in SSIS 2016?
Solution Overview
I made 2 Experiments on that issue, the first experiments showed that in case of stored procedures with no parameters, nothing changed in SQL Server 2016 and SSIS 2016, the first Result Set is returned and others are ignored.
The second experiment showed that when using parameters, this will throw an exception, so you have to define metadata using WITH RESULT SETS option, then remove this option.
Detailed Solution
Experiment 1
The following experiment are made using SQL Server 2016 and Visual Studio 2015 with SSDT 2016
First i created this stored procedure
CREATE PROCEDURE sp_Test
AS
BEGIN
SET NOCOUNT ON;
SELECT TOP 10 PersonType,NameStyle,Title
FROM [AdventureWorks2016CTP3].[Person].[Person]
SELECT TOP 10 PersonType,Firstname,Lastname
FROM [AdventureWorks2016CTP3].[Person].[Person_json]
END
GO
Then i added a Data flow task to SSIS package
Added an OLEDB Source, Recordset destination
In OLEDB source i select the Data access mode to SQL command
an use the following commnad
EXEC sp_Test
When clicking on Columns Tab it shows the first ResultSet structure
And we i executed the package it runs succesfully
Experiment 2
I changed the stored procedures to the following:
ALTER PROCEDURE [dbo].[sp_Test]
#param1 varchar(10),
#param2 varchar(10),
#param3 varchar(10)
AS
BEGIN
SET NOCOUNT ON;
SELECT TOP 10 PersonType,NameStyle,Title ,#param2 as 'Param'
FROM [AdventureWorks2016CTP3].[Person].[Person]
SELECT TOP 10 PersonType,Firstname,Lastname,#param3 as 'Param'
FROM [AdventureWorks2016CTP3].[Person].[Person_json]
END
And i used the following SQL Command in the OLEDB Source:
EXEC sp_Test ?,?,?
WITH RESULT SETS (
(
PersonType NVarchar(10),
NameStyle NVarchar(10),
Title NVarchar(10),
Param Varchar(10)
)
)
And i mapped the parameters correctly.
When running the package it throws the following exception.
[OLE DB Source 2] Error: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040E14.
An OLE DB record is available. Source: "Microsoft SQL Server Native Client 11.0" Hresult: 0x80040E14 Description: "EXECUTE statement failed because its WITH RESULT SETS clause specified 1 result set(s), and the statement tried to send more result sets than this.".
After that i tried to remove the With RESULT SETS option, so the command is :
EXEC sp_Test ?,?,?
I tried to execute the package again, so it is executed with no errors.
Conclusion
Try to use the WITH RESULT SETs option to define the OLEDB Source metadata, after that the metadata is defined, just remove this option and run the package, so it will just take the first Result Set succesfully.
I know it's not exactly what you are asking for, but maybe you could create another stored proc that will return only the first result set for you? (Without touching the first stored proc.) Or insert the data into a table for you and then you just read the data?
I've got a database that is replicated on two servers, a live server and a test server, so that whenever it's needed the 'test' database gets overwritten by the 'live' database (so that I can reset everything if I've made a mess.)
I want an Stored Procedure in the 'test' database, that will only run in the 'test' database, but to do this I need to have it in the 'live' database as well, so that it can be copied over when 'test' is overwritten.
The procedure starts:
if ##SERVERNAME<>'TEST'
begin
raiserror ('NOT ON TEST! This SP must only be run on TEST.',16,1)
return
end
So that if it runs in live, it immediately exits.
Unfortunately the "Live" database server uses an older version of SQL, and doesn't seem to understand the lead/lag/over statements in the script, and refuses to create the procedure because of these "Incorrect syntax" errors.
The SP definitely works in the test server.
Is there a way to disregard the error messages when creating a stored procedure?
I've found a prior question that explained how to make a stored procedure with the same name, but I need the stored procedure to contain the script that the server thinks is incorrect.
The only way to not get the stored-procedure validated when created, is to run a dynamic-sql query within.
Example:
CREATE PROCEDURE dbo.YourStoredProcedure AS
BEGIN
IF ##SERVERNAME<>'TEST'
BEGIN
RAISERROR ('NOT ON TEST! This SP must only be run on TEST.',16,1)
RETURN
END
DECLARE #SQL NVARCHAR = N'
SELECT rowOne
, rowTwo
, valueOne
, LEAD(valueOne) OVER (PARTITION BY rowOne ORDER BY rowTwo DESC) AS preValue
FROM dbo.YourTable
'
EXEC(#SQL)
END
Notes:
On the long term, try to find a better strategy than DB replication for different systems. Check Continuous Deployment
Make sure to check concatenated dynamic-sql for potential issues (sql injection). Check QUOTENAME()
I using ssis package.I want insert flat file source (text file) to sql.
Addres of text file is dynamic so i define variable for path.I have the sp like this:
CREATE PROCEDURE [dbo].[Insert_FileMaster]
#FILE_PATH nVARCHAR(MAX)
,#id int OUTPUT
AS
BEGIN
insert into [dbo].[FileMaster] ([FM_Name])
values(#FILE_PATH))
set #id = ##IDENTITY
END
I want exec this sp With variable parameter.
this is my package:
Which ssis tool should i use?and how to get output from sp (return parametert must be use in another sp in package)?
You will want to add an Execute SQL Task before your Data Flow Task (this will be at the Control Flow level).
You will need to configure the Execute SQL task as described in this answer.
Insert a single row and return its primary key