Dynamic query with variable value from a table in SSIS 2015 - sql-server

I have Table A that has only one row :
CODE | DATE
202211 | 2022-11
this table will update it self automatically every end of the month (eg: Next month it will change to 202212 and 2022-12)
I want to use 'CODE' and 'DATE' to make my query dynamic, using variable and Execute SQL Task in SSIS.
My original query look like this :
SELECT * FROM X
WHERE PERIOD = '202211', EXPDATE > '2022-11'
I want to make it so that whenever Table A change, I don't have to change the Query too.
This is what I tried already :
DECLARE #Period varchar(50)
DECLARE #Expdate varchar(50)
SET #Period = ?
SET #Expdate = ?
SELECT * FROM X
WHERE PERIOD = #Period, EXPDATE > #Expdate
When I try to run using '?' just as the documentation say, it doesn't work, but it run when I change the '?' into hardcode, so I'm pretty sure at least my query works. Am I missing something, or I'm setting the Variable wrong.
This is my variable settings
Name | Scope | Data type | Value | Expression
position | MyDtsx | String | |
date | MyDtsx | String | |
This is my SQL Task setting
General
__________________________
Result Set = Single Row
SQLSourceType = Direct Input
SQLStatement = 'SELECT CODE as position, DATE as date FROM A'
Result Set
__________________________
Result Name | Variable Name
position | User::position
date | User::date

You have invalid syntax
SELECT *
FROM X
WHERE PERIOD = #Period, EXPDATE > #Expdate
Make that
SELECT *
FROM X
WHERE PERIOD = #Period AND EXPDATE > #Expdate
However, if table A (the source of Period and ExpDate) is in the same database as table X, skip the extraneous Execute SQL Task and the variables and just make the query
SELECT X.*
FROM X
WHERE EXISTS (SELECT *
FROM A
WHERE X.PERIOD = A.Code AND X.EXPDATE > P.[Date]);
The comment indicates
Table A and Table X is from a different database. Sadly the query is actually fine since when I change it to' #Period = '202211'' and '#Expdate = '2022-11'' it run.
A working repo.
My working notes for those following along at home as the name changes messed me a few times.
Table A's Code = SSIS Variable position = Table X's Period
Table A's Date = SSIS Variable date = Table X's ExpDate
SQL Setup
Execute sql task to ensure I have data
drop table if exists dbo.so_74394641;
create table dbo.so_74394641
(
Col1 bigint, Period varchar(50), ExpDate varchar(50)
);
insert into dbo.so_74394641
SELECT row_number() over (order by (SELECT NULL)) AS col1, '202211', '2022-12'
FROM sys.all_objects;
SQL Get Values
Execute sql task. Hard coded as I didn't want to create another table in a different database
SELECT '202211' AS Code, '2022-11' AS [Date];
DFT Get Data
A data flow. OLE DB Source component using the following query
DECLARE #Period varchar(50)
DECLARE #Expdate varchar(50)
SET #Period = ?
SET #Expdate = ?
SELECT * FROM dbo.SO_74394641 AS X
WHERE PERIOD = #Period and EXPDATE > #Expdate;
Parameters mapped
0 User::dosition Input
1 User::date Input
Control flow
Data Flow

Related

Insert values in temp table form same table and different variables

I have this code, that works, but I want to insert in the temp table the same values (DateTime and Value) from another variable (UBB_PreT_Line_LA.If_TotalInFeddWeight) present in the same table ([Runtime].[dbo].[History]). Then, I show the result in SQL Report Builder 3.0 in a table.
SET NOCOUNT ON
DECLARE #fechaItem DATETIME;
DECLARE #fechaFinTotal DATETIME;
SET #fechaItem = DateAdd(hh,7,#Fecha)
SET #fechaFinTotal = DateAdd(hh,23,#Fecha)
SET NOCOUNT OFF
DECLARE #tblTotales TABLE
(
VALOR_FECHA DATETIME,
VALOR_VALUE float
)
WHILE #fechaItem < #fechaFinTotal
BEGIN
DECLARE #fechaFin DATETIME;
SET #fechaFin = DATEADD(minute, 15, #fechaItem );
INSERT INTO #tblTotales
SELECT
MAX( [DateTime] ),
MAX( [Value] )
FROM [Runtime].[dbo].[History]
WHERE
[DateTime] >= #fechaItem
AND [DateTime] <= #fechaFin
AND (History.TagName='UBB_PreT_Belt_PF101A.Time_Running')
SET #fechaItem = #fechaFin;
END
SELECT TOP 64 VALOR_FECHA as Fecha,VALOR_VALUE as Valor
FROM #tblTotales
order by Valor ASC
What I want, is to join in a single query the result I get in these two tables, with the same query in which only the variable that is queried changes.
The purpose is to create a unique Dataset in Report Builder to display in a single table, the data of the two tables of the image. The 15 minute interval is because I just want to show the variation of the values every 15 minutes.
enter image description here
I have modified the code (Image_02), and with the Query Designer of the Report Builder I have obtained what is shown in the Image_03. The final goal would be to have the data of the second variable, in two more columns on the right (Fecha_Ton and Valor_Ton). How can I do it?
enter image description here
enter image description here
If I've understood your question correctly, I think that this query replaces your code entirely (and adds the second value):
declare #sample table (Datetime datetime not null, Value int not null,
TagName varchar(50) not null)
insert into #sample (DateTime, Value, TagName) values
('2018-08-16T10:14:00',6,'UBB_PreT_Belt_PF101A.Time_Running'),
('2018-08-16T10:08:00',8,'UBB_PreT_Belt_PF101A.Time_Running'),
('2018-08-16T10:23:00',7,'UBB_PreT_Belt_PF101A.Time_Running'),
('2018-08-16T10:07:00',7,'UBB_PreT_Line_LA.If_TotalInFeddWeight')
declare #Fecha datetime
set #Fecha = '20180816'
select
MAX(DateTime),
MAX(CASE WHEN TagName='UBB_PreT_Line_LA.If_TotalInFeddWeight' THEN Value END) as Fed,
MAX(CASE WHEN TagName='UBB_PreT_Belt_PF101A.Time_Running' THEN Value END) as Running
from
#sample
where
DateTime >= DATEADD(hour,7,#Fecha) and
DateTime < DATEADD(hour,23,#Fecha) and
TagName in ('UBB_PreT_Line_LA.If_TotalInFeddWeight',
'UBB_PreT_Belt_PF101A.Time_Running')
group by DATEADD(minute,((DATEDIFF(minute,0,DateTime)/15)*15),0)
order by MAX(DateTime) asc
Results:
Fed Running
----------------------- ----------- -----------
2018-08-16 10:14:00.000 7 8
2018-08-16 10:23:00.000 NULL 7
(You may want two separate dates following the same pattern using CASE as the values)
You shouldn't be building your data up row by agonising row1, you should find as way (such as that above) to express what the entire result set should look like as a single query. Let SQL Server itself decide whether it's going to do that by searching through the rows in date order, etc.
1There may be circumstances where you end up having to do this, but first exhaust any likely set-based options first.

Performance issue with larger resultsets MSSQL

I currently have a stored procedure in MSSQL where I execute a SELECT-statement multiple times based on the variables I give the stored procedure. The stored procedure counts how many results are going to be returned for every filter a user can enable.
The stored procedure isn't the issue, I transformed the select statement from te stored procedure to a regular select statement which looks like:
DECLARE #contentRootId int = 900589
DECLARE #RealtorIdList varchar(2000) = ';880;884;1000;881;885;'
DECLARE #publishSoldOrRentedSinceDate int = 8
DECLARE #isForSale BIT= 1
DECLARE #isForRent BIT= 0
DECLARE #isResidential BIT= 1
--...(another 55 variables)...
--Table to be returned
DECLARE #resultTable TABLE
(
variableName varchar(100),
[value] varchar(200)
)
-- Create table based of inputvariable. Example: turns ';18;118;' to a table containing two ints 18 AND 118
DECLARE #RealtorIdTable table(RealtorId int)
INSERT INTO #RealtorIdTable SELECT * FROM dbo.Split(#RealtorIdList,';') option (maxrecursion 150)
INSERT INTO #resultTable ([value], variableName)
SELECT [Value], VariableName FROM(
Select count(*) as TotalCount,
ISNULL(SUM(CASE WHEN reps.ForRecreation = 1 THEN 1 else 0 end), 0) as ForRecreation,
ISNULL(SUM(CASE WHEN reps.IsQualifiedForSeniors = 1 THEN 1 else 0 end), 0) as IsQualifiedForSeniors,
--...(A whole bunch more SUM(CASE)...
FROM TABLE1 reps
LEFT JOIN temp t on
t.ContentRootID = #contentRootId
AND t.RealEstatePropertyID = reps.ID
WHERE
(EXISTS(select 1 from #RealtorIdTable where RealtorId = reps.RealtorID))
AND (#SelectedGroupIds IS NULL OR EXISTS(select 1 from #SelectedGroupIdtable where GroupId = t.RealEstatePropertyGroupID))
AND (ISNULL(reps.IsForSale,0) = ISNULL(#isForSale,0))
AND (ISNULL(reps.IsForRent, 0) = ISNULL(#isForRent,0))
AND (ISNULL(reps.IsResidential, 0) = ISNULL(#isResidential,0))
AND (ISNULL(reps.IsCommercial, 0) = ISNULL(#isCommercial,0))
AND (ISNULL(reps.IsInvestment, 0) = ISNULL(#isInvestment,0))
AND (ISNULL(reps.IsAgricultural, 0) = ISNULL(#isAgricultural,0))
--...(Around 50 more of these WHERE-statements)...
) as tbl
UNPIVOT (
[Value]
FOR [VariableName] IN(
[TotalCount],
[ForRecreation],
[IsQualifiedForSeniors],
--...(All the other things i selected in above query)...
)
) as d
select * from #resultTable
The combination of a Realtor- and contentID gives me a set default set of X amount of records. When I choose a Combination which gives me ~4600 records, the execution time is around 250ms. When I execute the sattement with a combination that gives me ~600 record, the execution time is about 20ms.
I would like to know why this is happening. I tried removing all SUM(CASE in the select, I tried removing almost everything from the WHERE-clause, and I tried removing the JOIN. But I keep seeing the huge difference between the resultset of 4600 and 600.
Table variables can perform worse when the number of records is large. Consider using a temporary table instead. See When should I use a table variable vs temporary table in sql server?
Also, consider replacing the UNPIVOT by alternative SQL code. Writing your own TSQL code will give you more control and even increase performance. See for example PIVOT, UNPIVOT and performance

Drop table in Stored Procedure not working properly?

I have a stored procedure which drops a table if it exists, then it re-creates the table & fills it with relevant data, a friend of mine has about the same code, the only real difference is in the column headers for the table.
As an illustration, here's how mine looks (not really, just a representation).
+----+-----+-----+--------+
| ID | Foo | Bar | Number |
+----+-----+-----+--------+
| 1 | x | x | 0 |
| 2 | x | x | 1 |
+----+-----+-----+--------+
And here's what his might look like
+----+--------+--------+-----+--------+
| ID | BarFoo | FooBar | Num | Suffix |
+----+--------+--------+-----+--------+
| 1 | x | x | 0 | a |
| 2 | x | x | 1 | b |
+----+--------+--------+-----+--------+
Again, these are merely representations of the situation.
As this is to be a school assignment, the teacher will be creating & executing both SP's, however when creating the SP after using another, I get this error:
Msg 207, Level 16, State 1, Procedure XYZ, Line 59
Invalid column name 'Foo'.
Msg 213, Level 16, State 1, Procedure XYZ, Line 61
Column name or number of supplied values does not match table definition.
However, at the start of both stored procedures, we have this:
CREATE PROCEDURE XYZ
AS
BEGIN
IF EXISTS (SELECT name
FROM sysobjects
WHERE name = 'TABLENAME'
AND xtype = 'u')
DROP TABLE TABLENAME;
From what I understand, this should remove the entire table? Including table/column definitions & data?
The only fix I've found so far, is to either execute the DROP TABLE separately before creating the stored procedure, which won't work for us as it really has to be within the stored procedure.
Help would be much appreciated :)
EDIT: Here's my ACTUAL code, apart from comments, this is exactly how it looks in my script (excluding other code behind it).
IF EXISTS (SELECT name
FROM sysobjects
WHERE name = 'BerekenStatistiek'
AND xtype = 'p')
DROP PROCEDURE BerekenStatistiek;
GO
CREATE PROCEDURE BerekenStatistiek
#jaar INT=0
AS
BEGIN
IF EXISTS (SELECT name
FROM sysobjects
WHERE name = 'Statistiek'
AND xtype = 'u')
DROP TABLE Statistiek;
DECLARE #year AS NVARCHAR (4);
SET #year = CONVERT (NVARCHAR (4), #jaar);
SELECT *,
CAST (Kost - Korting + Freight AS MONEY) AS Netto,
'' AS Richting
INTO Statistiek
FROM (SELECT O.Kwartaal,
CAST (SUM(O.Kost) AS MONEY) AS Kost,
CAST (SUM(O.Korting) AS MONEY) AS Korting,
CAST (SUM(O.Freight) AS MONEY) AS Freight
FROM (SELECT CASE
WHEN CONVERT (NVARCHAR (8), OrderDate, 112) BETWEEN #year + '0101' AND #year + '0331' THEN 1
WHEN CONVERT (NVARCHAR (8), OrderDate, 112) BETWEEN #year + '0401' AND #year + '0630' THEN 2
WHEN CONVERT (NVARCHAR (8), OrderDate, 112) BETWEEN #year + '0701' AND #year + '0930' THEN 3
WHEN CONVERT (NVARCHAR (8), OrderDate, 112) BETWEEN #year + '1001' AND #year + '1231' THEN 4
END AS 'Kwartaal',
ROUND(UnitPrice * Quantity, 2) AS Kost,
Round((UnitPrice * Quantity) * Discount, 2) AS Korting,
Freight
FROM Orders AS O
INNER JOIN
OrderDetails AS Od
ON O.OrderID = Od.OrderID
WHERE CONVERT (NVARCHAR (4), OrderDate, 112) = #year) AS O
GROUP BY O.Kwartaal) AS O1;
ALTER TABLE Statistiek ALTER COLUMN Kwartaal INT NOT NULL;
ALTER TABLE Statistiek ALTER COLUMN Richting NVARCHAR (8);
ALTER TABLE Statistiek
ADD PRIMARY KEY (Kwartaal);
...
And here's his code (the insertion of values in the variables are excluded just for readability (his code is a bit more bulky):
IF EXISTS (SELECT name
FROM sysobjects
WHERE name = 'BerekenStatistiek'
AND xtype = 'p')
BEGIN
DROP PROCEDURE BerekenStatistiek;
END
GO
CREATE PROCEDURE BerekenStatistiek
#jaartal INT
AS
BEGIN
DECLARE #huidigkwartaal AS INT = 1;
DECLARE #beginmaand AS INT;
DECLARE #eindmaand AS INT;
DECLARE #vorige_netto_ontvangsten AS MONEY;
IF EXISTS (SELECT *
FROM sysobjects
WHERE name = 'Statistiek'
AND xtype = 'U')
BEGIN
DROP TABLE Statistiek;
END
CREATE TABLE Statistiek
(
kwartaalnummer INT ,
beginmaand INT ,
eindmaand INT ,
orderbedrag MONEY ,
korting MONEY ,
vervoerskost MONEY ,
netto_ontvangsten MONEY ,
stijgend_dalend_gelijk NVARCHAR (10)
);
--Variables get their data here.
INSERT INTO Statistiek (kwartaalnummer, beginmaand, eindmaand, orderbedrag, korting, vervoerskost, netto_ontvangsten, stijgend_dalend_gelijk)
VALUES (#huidigkwartaal, #beginmaand, #eindmaand, #orderbedrag, #korting, #vervoerskost, #netto_ontvangsten, #stijgend_dalend_gelijk);
"however when creating the SP after using another, I get this error" (Emphasis added.)
SQL Server will insist that a stored procedure match the definitions of tables that exist as the time the stored procedure is created. If the table does not exist when the stored procedure is created, SQL Server will assume that a matching table will appear at run time.
create table t (c int)
go
create procedure p as begin
drop table t
select 1 as diff_column_name into t
select diff_colun_name from t
end
results in:
Msg 207, Level 16, State 1, Procedure p, Line 6
Invalid column name 'diff_colun_name'.
Now, drop table t, and the procedure cane be created:
drop table t
go
create procedure p as begin
drop table t
select 1 as diff_column_name into t
select diff_colun_name from t
end
Command(s) completed successfully.
If you can use a different table name, start with that. And, if the table has to exist only for a moment after the proc is executed so that it can be selected from, then create a global temporary table (i.e. table name starts with ## as in ##MyTable).
However, if it is a requirement to use the same table name as your classmate, then the teacher is probably trying to get you to learn about deferred object resolution (i.e. #Shannon's answer) and how to get around it, because outside of learning this, the scenario makes no sense since one would never do such a thing in reality.
Sub-processes (i.e. EXEC and sp_executesql) do not resolve immediately since they aren't executed when creating the stored procedure. So, simplistically, just declare a new NVARCHAR(MAX) variable to hold some Dynamic SQL and put your SELECT statement in there. Use sp_executesql to pass in the #year variable. You are creating a real table so it will survive beyond the subprocess ending and then the ALTER TABLE statement will work.
Additional notes:
You don't really need the ALTER statement to set the datatype of the [Richting] field. Just tell SQL Server what the type is in your SELECT statement:
CONVERT(NVARCHAR(8), '') AS [Richting]
You don't really want to do CONVERT(NVARCHAR(8), OrderDate, 112) to compare to a value as it invalidates the use of any indexes that might be on [OrderDate]. Instead, construct a date value from the strings and convert that to a DATETIME or DATE (i.e. CONVERT(DATETIME, #year + '0101')).
To better understand this issue, please read Sargability: Why %string% Is Slow, and at least the first link at the bottom, which is: What makes a SQL statement sargable?
You don't really want to convert the OrderDate field to NVARCHAR(4) just to compare the year, for the same reason as just mentioned in the above point. At the very least using the YEAR() function would be more direct. But if you want to make sure indexes can be used, you can't put a function on the field. But you only want the year. So isn't the year the same as BETWEEN #Year + '0101' AND #Year + '1231'? ;-)
Interestingly enough, the first example in the accepted answer in the "What makes a SQL statement sargable?" S.O. question linked in the previous bullet is exactly what I am recommending here :).
For I can understand, the wrong queries are the inserts, because the engine can't find correct table structure, check if the inserts have the same structure of your second table example. Dont forget to check the USE at the beginning of the script, maybe you are using a different db, this can happen :).
In the last bit of code, you are having
AND xtype = 'U'
If your collation is case sensitive, the drop is not taking place and thus the error.

Sql server using variable in pivot query

I have a returned string from a query that reads:
+----------------------+
| returnquerystring |
+----------------------+
| exam1,exam2,exam3 |
+----------------------+
I am using this returned string as column names in a pivot query.
select * from (select score,exam from table1) x
pivot ( max(score) for exam in (exam1,exam2,exam3)
This query works giving me
+-------------+-----------+-----------+
| exam1 | exam2 | exam3 |
+-------------+-----------+-----------+
| 10 | 20 | 30 |
+-------------+-----------+-----------+
However I have not been able to get the pivot "in" statement to use anything but the hard coded values of exam1,exam2,exam3. For example I have used SSMS and created a query that successfully puts exam1,exam2,exam3 into #var1. However #var1 will throws and error when used in place of exam1,exam2,exam3.
declare #var1 varchar(100)
select #var1 = value from table
select * from (select score,exam from table1) x
pivot ( max(score) for exam in (#var1)
Incorrect syntax near '#var1'.
To verify that I was doing it correctly I did this and it worked.
declare #var1 int
select top 1 #var1 = id from name
select * from name where id = #var1
This provided the data row for id 1 on the name table with no error.
I have noticed in my experiments that (exam1,exam2,exam3) cannot be ('exam1,exam2,exam3') with the quotes.
I am using ColdFusion CFSCRIPT and it does appear that the single quotes are getting into the query so I tried various tests with ColdFusion functions to remove them with no success.
So I tried using the SQL Server function 'replace' around the #var1 and that throws an error about syntax at replace.
This is when I tried using an example like above in SSMS and still got errors. So by removing ColdFusion from the equation it still does not work. My thought was to send the whole declare through pivot as a query to avoid ColdFusion issues but it does not work in SSMS.
I am using SQL SERVER 8 and SSMS 11.
Any ideas on how to make this work?
examColumns = exam1,exam2,exam3
public any function qryExamScores(string examColumns) {
thisQry = new Query();
thisQry.setName("returnqry");
thisQry.setDatasource(application.datasource);
thisQry.addParam(name="columnNames",value=arguments.examColumns,cfsqltype="cf_sql_varchar");
result = thisQry.execute(sql="
select * from
(select id,score,exam
from table
where value1 = 'XXXXX'
and value2 = '11111') x
pivot
(
max(score) for exam in (:columnNames)
) p
");
returnqry = result.getResult();
return returnqry;
}
You need to use Dynamic SQL to use the value of variable(#var1) inside Pivot
declare #var1 varchar(100)='',#sql nvarchar(max)
select top 1 #var1 = value from table
set #sql = 'select * from (select score,exam from table1) x
pivot ( max(score) for exam in (['+#var1+'])) piv'
exec sp_executesql #sql
If you want to have more then one value in pivot columns use this.
SELECT #var1 += '[' + Isnull(CONVERT(VARCHAR(50), value), '') + '],'
FROM table
SELECT #var1 = LEFT(#var1, Len(#var) - 1)
SET #sql = 'select * from (select score,exam from table1) x
pivot ( max(score) for exam in (' + #var1 + ')) piv'
EXEC Sp_executesql #sql
passing exam1,exam2,exam3 as a param varchar as :parametervalue
Queryparam (or bind variables) can only be used on literals. Since "exam1,exam2,exam3" are being used as column names in this specific query, you cannot apply queryparam to them. When you do that, you are telling the database those values are simple strings. That causes an error because pivot expects object names, not strings.
Remove the queryparam and the query will work as expected. However, obviously that may expose your database to sql injection (depending on the source of columnNames). The same applies to using any dynamic SQL (exec, sp_executesql, ...). So be sure to fully validate the input before implementing this approach.
...
// build pivot statement with dynamic column names
columnNames = "exam1,exam2,exam3";
sqlString = "SELECT *
FROM (
SELECT score,exam
FROM table1
) x
PIVOT
(
MAX(score) FOR exam IN ("& columnNames &")
)
AS pvt ";
result = qry.execute( sql=sqlString ).getResult();
writeDump( result );
Edit:
Also, you should probably enclose the column names in brackets to avoid syntax errors if the values contain spaces, or other invalid characters for column names.
"[exam1],[exam2],[exam3]";

Setting variable SQL Procedure

I am working on a database that contains customers, products, timesheets, etc for a store. The question I am working on involves creating a procedure that will change an "on/off" column to off (the product is available (1) by default, and this procedure turns it to 0) I have writen the procedure fine:
create proc p_fudgemart_deactivate_product
(
#product_id int
)
as
begin
update fudgemart_products
set product_is_active = 0
where product_id = #product_id
end
but the issue then comes when we are given a product NAME, and need to write a select statement to change that product to unavailable. I know that this requires the use of a variable, but I cannot figure out how to set the variable to the product id of that product. I was thinking something along the lines of:
Declare #prod_name_id int
set #prod_name_id= (select product_id from fudgemart_products
where product_name = 'Slot Screwdriver')
execute p_fudgemart_deactivate_product product_id #prod_name_id
Am I able to use a select in my variable declaration like this?
actually you're on the right track. try something like this:
declare #prod_name_id int
select #prod_name_id = product_id
from fudgemart_products
where product_name = 'Slot Screwdriver'
exec p_fudgemart_deactivate_product
#product_id = #prod_name_id
If you are using SQL Server 2008 or later, you can declare and assign in one statement:
DECLARE #prod_name_id int = ( SELECT product_id
FROM fudgemart_products
WHERE product_name = 'Slot Screwdriver'
);
EXECUTE p_fudgemart_deactivate_product #product_id = #prod_name_id;

Resources