I have ssis package in that I'm taking values from flat file and insert it into table.
I have taken one Execute SQL Task in that creating one temptable
CREATE TABLE [tempdb].dbo.##temptable
(
date datetime,
companyname nvarchar(50),
price decimal(10,0),
PortfolioId int,
stype nvarchar(50)
)
Insert into [tempdb].dbo.##temptable (date,companyname,price,PortfolioId,stype)
SELECT date,companyname,price,PortfolioId,stype
FROM ProgressNAV
WHERE (Date = '2011-09-30') AND (PortfolioId = 5) AND (stype in ('Index'))
ORDER BY CompanyName
Now in above query I need to pass (Date = '2011-09-30') AND (PortfolioId = 5) AND (stype in ('Index'))
these 3 parameter using variable name I have created variables in package so that I become dynamic.
In your Execute SQL Task, make sure SQLSourceType is set to Direct Input, then your SQL Statement is the name of the stored proc, with questionmarks for each paramter of the proc, like so:
Click the parameter mapping in the left column and add each paramter from your stored proc and map it to your SSIS variable:
Now when this task runs it will pass the SSIS variables to the stored proc.
The EXCEL and OLED DB connection managers use the parameter names 0 and 1.
I was using a oledb connection and wasted couple of hours trying to figure out the reason why the query was not working or taking the parameters. the above explanation helped a lot
Thanks a lot.
Along with #PaulStock's answer, Depending on your connection type, your variable names and SQLStatement/SQLStatementSource Changes
https://learn.microsoft.com/en-us/sql/integration-services/control-flow/execute-sql-task
SELECT, INSERT, UPDATE, and DELETE commands frequently include WHERE clauses to specify filters that define the conditions each row in the source tables must meet to qualify for an SQL command. Parameters provide the filter values in the WHERE clauses.
You can use parameter markers to dynamically provide parameter values. The rules for which parameter markers and parameter names can be used in the SQL statement depend on the type of connection manager that the Execute SQL uses.
The following table lists examples of the SELECT command by connection manager type. The INSERT, UPDATE, and DELETE statements are similar. The examples use SELECT to return products from the Product table in AdventureWorks2012 that have a ProductID greater than and less than the values specified by two parameters.
EXCEL, ODBC, and OLEDB
SELECT* FROM Production.Product WHERE ProductId > ? AND ProductID < ?
ADO
SELECT * FROM Production.Product WHERE ProductId > ? AND ProductID < ?
ADO.NET
SELECT* FROM Production.Product WHERE ProductId > #parmMinProductID
AND ProductID < #parmMaxProductID
The examples would require parameters that have the following names:
The EXCEL and OLED DB connection managers use the parameter names 0 and 1. The ODBC connection type uses 1 and 2.
The ADO connection type could use any two parameter names, such as Param1 and Param2, but the parameters must be mapped by their ordinal position in the parameter list.
The ADO.NET connection type uses the parameter names #parmMinProductID and #parmMaxProductID.
A little late to the party, but this is how I did it for an insert:
DECLARE #ManagerID AS Varchar (25) = 'NA'
DECLARE #ManagerEmail AS Varchar (50) = 'NA'
Declare #RecordCount AS int = 0
SET #ManagerID = ?
SET #ManagerEmail = ?
SET #RecordCount = ?
INSERT INTO...
Related
I want to set local variables or pass parameters from Excel to SQL. I've found similar questions, but all referred to old versions of Excel and/or the answers showed how to filter or manipulate output from a generic SQL query in the Power Query Editor, rather than pass a parameter or modify the SQL, so that the SQL Server supplies data in the needed form.
I'm building a large Microsoft Excel spreadsheet that depends on ten different SQL queries, all against a common SQL Server database. Excel and SQL Server are installed on my laptop and are current versions (as of 16 Mar 2022). All ten queries share a common date restriction, imposed in the WHERE clauses of the queries. The tables accessed and the form of output are very different, so there is no easy way to combine the ten queries into a single query. The queries contain multiple levels of aggregation (e.g. SUM(...)) so I need to restrict the records accessed prior to aggregation and passing results from the query back to Excel.
As currently written, each query begins by setting two date values in local variables. For example,
DECLARE #BEGIN_DATE AS smalldatetime;
DECLARE #END_DATE AS smalldatetime;
#BEGIN_DATE = CAST('2021-03-01 00:00' AS smalldatetime);
#END_DATE = CAST('2021-03-02 23:59' AS smalldatetime);
Every one of the ten queries includes a line in the WHERE clause similar to
WHERE
PickUpDate BETWEEN #BEGIN_DATE AND #END_DATE
Every query will use the same pair of dates. However, the column filtered (PickUpDate above) changes from one query to the next.
As it is, I have to manually edit each of the ten queries to change the two dates--twenty edits in all. This is time-consuming and error-prone. Ideally, I'd like to set the date range in the spreadsheet, in a pop-up dialog box, or any other convenient way and pass the dates to the SQL queries. Then by selecting Data > Refresh All in Excel, update all my tables at once.
Is this possible, and if so, how does one do it?
The answer from David Browne is generally on-target. But I found some difficulties reading data from an Excel table directly into the SQL, given security restrictions in the latest version of Excel/Power Query. Also, since this was the first time I worked directly in M-code and the advanced editor, it was challenging to fill-in the gaps.
I finally got a nice solution running; here is what worked for me.
First, I stored the parameter values in a two-column table. My table is named "ParameterTable" with column headers named "Parameter_Name" and "Value". The value(s) to pass to SQL Server are stored in the Value column. My table has two rows with row entries labeled "Begin_DateTime" and "End_DateTime".
Secondly I created a callable function named “ftnGetParameter.” Select Data > Get Data > From Other Sources > Blank Query. Then select “Advanced Editor.” Delete any boilerplate added by Excel, and enter and save this function
let theParameter=(TableName,ParameterLabel) =>
let
Source=Excel.CurrentWorkbook(){[Name=TableName]}[Content],
value = Source{[Parameter_Name=ParameterLabel]}[Value]
in
value
in
theParameter
Thirdly, code-up your SQL statement as usual. I was trying to pass dates to SQL, so I initially coded with string literals. Enter the query in the usual way. I used Data > Get Data > From Database > From SQL Server Database. Then pasted in the SQL. The two relevant lines in my query looked like this:
DECLARE #BEGIN_DATE AS SMALLDATETIME='2021-01-01 00:00';
DECLARE #END_DATE AS SMALLDATETIME='2021-12-31 23:59';
You could skip this step, but it allowed me to get complex SQL code entered, formatted, and running before I invoked the function to pass the parameters.
Finally, simply replace the string literals in the SQL with code to call the function. My first few lines of M-code looks like this:
let
Source = Sql.Database("DESKTOP-04P8E8C", "nfbdata",
[Query=
"
DECLARE #BEGIN_DATE AS SMALLDATETIME= '" & ftnGetParameter("ParameterTable","Begin_DateTime") & "';
DECLARE #END_DATE AS SMALLDATETIME='" & ftnGetParameter("ParameterTable","End_DateTime") & "' (… the query continues )
Excel will issue some warnings about running the query and prompt you to edit permissions. Once permission has been granted, the function reads the text from the parameter table and passes it into the SQL.
I found that the function call was not optional. Apparently, importing the code directly into a native call to SQL Server is considered an unacceptable security risk.
Many thanks to Mr. David Browne. His post definitely points in the right direction.
You can reference a table on a sheet from Power Query and integrate values from that table into your other queries. Eg if ParameterTable is a single-row table on some worksheet with a column called "StartDate", something like
let
theDate = Date.From( Record.Field(Table.First(ParameterTable),"StartDate") ),
Source = Sql.Databases("localhost"),
AdventureWorksDW2017 = Source{[Name="AdventureWorksDW2017"]}[Data],
dbo_DimDate = AdventureWorksDW2017{[Schema="dbo",Item="DimDate"]}[Data],
#"Filtered Rows" = Table.SelectRows(dbo_DimDate, each [FullDateAlternateKey] = theDate )
in
#"Filtered Rows"
for M query folding, or
let
theDate = Date.From( Record.Field(Table.First(ParameterTable),"StartDate") ),
sql = "
select *
from dimDate
where FullDateAlternateKey = '" & Text.From(theDate) & "'
",
Source = Sql.Database("localhost", "adventureworksdw2017", [Query=sql])
in
Source
for dynamic SQL.
I'm attempting to edit an ETL package(SSIS) that queries a SQL table and outputs csv files for every StationID and I'm having trouble understanding how the question mark is being used in the query definition below. I understand ? is used a parameter but I don't understand how it's used in the date function below:
SELECT TimeSeriesIdentifier, StationID, ParameterID FROM dbo.EtlView WHERE
LastModified > DATEADD(hour, ?*-1, GETDATE())
AND StationID LIKE
CASE WHEN ? = 0 THEN
StationID
ELSE
?
END
The parameterization available in SSIS is dependent upon the connection manager used.
OLE DB and ODBC based connection managers use ? as the variable place holder, whereas ADO.NET uses a named parameter, #myVariable.
OLE DB begins counting at 0 whereas ODBC used a 1 based counting system. They are both however ordinal based systems so in your CASE expression the two ? are for the same variable. But, you'll have to list that SSIS Variable twice in the parameter mapping dialog because it's ordinal based - i.e. (param, name) => #HoursBack, 0; #MyVar, 1; and #MyVar, 2;
A "dumb trick" I would employ if I had to deal with repeated ordinal based parameters or if I was troubleshooting packages is to make the supplied query use local variables in the query itself.
DECLARE
#HoursBack int = ?
, #MyVariable int = ?;
SELECT
TimeSeriesIdentifier
, StationID
, ParameterID
FROM
dbo.EtlView
WHERE
LastModified > DATEADD(HOUR, #HoursBack * -1, GETDATE())
AND StationID LIKE
CASE
WHEN #MyVariable = 0 THEN StationID
ELSE #MyVariable
END;
Now I only have to map the SSIS Variable #MyVar once into my script as the "normal" TSQL parameterization takes over. The other benefit is that I can copy and paste that into a query tool and sub in the ?s with actual values to inspect the results directly from the source. This can be helpful if you're running into situations where the strong typing in SSIS prevents you from getting the results into a data viewer.
SSIS is building a parameterized query.
You can get more information about this here (MySQL-specific):
What is the question mark's significance in MySQL at "WHERE column = ?"?
Or you can get a more generally-applicable response here: What does a question mark represent in SQL queries?
At a very "nuts and bolts" level, those are parameters being passed into the SQL statement by the package. With the Execute SQL task open, click on the tab that says Parameter Mapping. There will be a list of variables that are being sent into the query, and they are consumed in the order that they're listed.
Here's a logger for an archiving package I'm working on:
The query on the General tab just writes those five values to a table:
INSERT INTO dbo.ArchiveRowCounts (
TableName,
ServerName,
ReportYear,
BaseTblCnt,
ArchiveTblCnt)
VALUES (?,?,?,?,?);
I have a requirement where I need to run a update script over multiple partitions of a table . I have written a script for it as below:
but it gives
ORA-14108: illegal partition-extended table name syntax
Cause: Partition to be accessed may only be specified using its name. User attempted to use a partition number or a bind variable.
Action: Modify statement to refer to a partition using its name
Any idea how can I circumvent this error?
DECLARE
TYPE partition_names IS varray(1) OF varchar2(20);
curr_partition partition_names;
LENGTH integer;
BEGIN
curr_partition :=partition_names('SM_20090731');
LENGTH := curr_partition.count;
FOR i IN 1 .. LENGTH LOOP
dbms_output.put_line('Current Partition name is: '||curr_partition(i));
UPDATE TABLE_Y PARTITION (curr_partition(i))
SET PARTITION_KEY=TO_DATE('2017-08-21','YYYY-MM-DD')
WHERE ORDER_ID IN
(SELECT ORDER_ID
FROM TABLE_X);
END LOOP;
END;
/
You will have to concatenate the partition name in and use dynamic SQL, i.e.
EXECUTE IMMEDIATE
'UPDATE TABLE_Y PARTITION (' || curr_partition(i) || ')
SET PARTITION_KEY=TO_DATE(''2017-08-21'',''YYYY-MM-DD'')
WHERE ORDER_ID IN
(SELECT ORDER_ID
FROM TABLE_X)';
Whenever you run a SQL SELECT query or an INSERT, UPDATE or DELETE statement from PL/SQL, bind variables are used to pass into the SQL engine the values of any PL/SQL expressions. In particular, a bind parameter will be used for curr_partition(i). However, it seems the PARTITION clause of such queries and statements doesn't support bind parameters. I guess that this is because Oracle tries to create an execution plan for the query or statement before it has the bind parameter values, but if the query or statement specifies a partition, that information is a critical part of the plan and hence cannot be provided in a bind parameter.
I want to create a report in MS SQL Server BIDS (SSMS and Visual Studio). The user would enter a list of email addresses as a parameter. So #pEmails would be 'foo#bluh.com', 'bar#meh.org', etc. These email addresses may or may not be in a table.
I can simply do:
and Table.Email in (#pEmails)
and that works, except I need to return the email address if it's NOT found as well. So the results would be something like:
|email |found in table|
|------------|--------------|
|foo#bluh.com| Y |
|bar#meh.org | N |
I was thinking I could take the list of values entered as the #pEmails parameter and create a temp table with them, which I could then left join with, but my attempts to do so have not worked out.
declare #pEmails table (EmailAddress varchar(255));
insert into #pEmails values (#ReportParameter1);
select
*
from
#pEmails
The above works if only a single value is put into #ReportParameter1, but not if multiples are in it.
I am using SQL Server 2008. Any suggestions on how best to proceed?
As has been stated, you need some kind of split function, for analysis on the performance of various methods Split strings the right way – or the next best way is an excellent read. Once you have your function, you then need to define your query parameter as a string, rather than a table:
So your query would actually become:
DECLARE #pEmails TABLE (EmailAddress varchar(255));
INSERT #pEmails (EmailAddress)
SELECT Value
FROM dbo.Split(#pEmallString);
Then go to your dataset properties, and instead of passing the multivalue parameter #pEmails to the dataset, instead create a new one #pEmailString, and set the value as an expression, which should be:
=Join(Parameters!pEmails.Value, ",")
This turns your multivalue parameter into a single comma delimited string. It seems pretty backwards that you need to convert it to a delimited string, only to then split it in SQL, unfortunately I don't know of a better way.
Here are some learnings on this topic (standing on the shoulders of the information elsewhere in this thread).
Set a parameter (select 'multiple values' checkbox):
InputList
Establish dataset query:
SELECT *
INTO #InputTemp
FROM STRING_SPLIT(#InputListJoin, ',')
SELECT value as ValueName
FROM #InputTemp T2
WHERE NOT EXISTS (
SELECT MyValue
FROM MyTable T1
WHERE T1.MyValue = T2.value
)
Establish dataset parameters:
Name: #InputList | Value: [#InputList]
Name: #InputListJoin | Value(expression): =Join(Parameters!InputList.Value,",")
The element names can be changed as needed.
Somewhat on topic, other details that might be helpful:
[#InputList.IsMultiValue] --> true/false whether your parameter is multi-value (not whether there are multiple values)
[#InputList.Count] --> count of items in input list (excludes blank lines)
=Parameters!InputList.Value(2) --> return third value from list (counting from zero)
OK this seems like it should be insanely easy, but I cannot figure it out. Every where I look online says to create temp tables and VB scripts and I cannot believe I have to do that. My goal is to insert all the records in a table with a date later than the max date in that destination table.
UPDATE The 2 tables are in two different non linked SQL databases
So:
Select #[User::Dated] = MAX(Dateof) from Table2
Insert into Table2
Select *
From Table1
Where DateOf > #[User::Dated]
I am trying to do this in SSIS. I declared a variable, the SQL execution step looks like it is assigning the single row output to it. But when I got go into the data flow it give me no parameters to choose, when I force the known parameter which is in the project scope it says no parameter exists
Create two OLE DB data sources each pointing at you two databases.
Create a variable called max_date and make its data type String.
Place an Execute SQL Task on the Control Flow, change its connection type to OLE DB and for the connection select the name of the data source that contains Table2. Set the ResultSet to Single Row. Add the following for the SQLStatement:
SELECT CAST(MAX(Dateof) AS VARCHAR) AS max_date FROM Table2
Go to the Result Set pane, click Add and enter the following:
Result Name: max_date
Variable Name: User::max_date
You can now use the max_date variable in an expression to create a SQL statement, for example you could use it in another Execute SQL Task which would use the second Data Connection like so:
"INSERT INTO Table2
SELECT *
FROM Table1
WHERE DateOf > '" + #[User::max_date] + "'"
Or in an OLE DB Source in a data flow like so:
"SELECT *
FROM Table1
WHERE DateOf > '" + #[User::max_date] + "'"
You can do this in a single SQL Task if you want:
Insert into Table2
Select *
From Table1
Where DateOf > (Select MAX(Dateof) from Table2)
If you want to use multiple Execute SQL Task items in the control flow, or want to make use of the parameter in a data flow instead, you have to change the General > Result Set option for your MAX() query to Single Row, then move from General to Result Set and Add a new variable for your result set to occupy.
To use that variable in your INSERT INTO.... query via Execute SQL Task, you'll construct your query with a ? for each parameter and map them in the parameter mapping section. If a variable is used multiple times in a query it's easiest to use a stored procedure, so you can simply pass the relevant parameters in SSIS.