I am trying to run an inline TVF as a raw parameterized SQL query.
When I run the following query in SSMS, it takes 2-3 seconds
select * from dbo.history('2/1/15','1/1/15','1/31/15',2,2021,default)
I was able to capture the following query through SQL profiler (parameterized, as generated by Entity framework) and run it in SSMS.
exec sp_executesql N'select * from dbo.history(#First,#DatedStart,#DatedEnd,#Number,#Year,default)',N'#First date,#DatedStart date,#DatedEnd date,#Maturity int,#Number decimal(10,5)',#First='2015-02-01',#DatedStart='2015-01-01',#DatedEnd='2015-01-31',#Year=2021,#Number=2
Running the above query in SSMS takes 1:08 which is around 30x longer than the non parameterized version.
I have tried adding option(recompile) to the end of the parameterized query, but it did absolutely nothing as far as performance. This is clearly an indexing issue to me, but I have no idea how to resolve it.
When looking at the execution plan, it appears that the parameterized version mostly gets mostly hung up on an Eager Spool (46%) and then a Clustered Index scan (30%) which are not present in the execution plan without parameters.
Perhaps there is something I am missing, can someone please point me in the right direction as to how I can get this parameterized query to work properly?
EDIT: Parameterized query execution plan, non-parameterized plan
Maybe it's a parameter sniffing problem.
Try modifying your function so that the parameters are set to local variables, and use the local vars in your SQL instead of the parameters.
So your function would have this structure
CREATE FUNCTION history(
#First Date,
#DatedStart Date,
#DatedEnd Date,
#Maturity int,
#Number decimal(10,5))
RETURNS #table TABLE (
--tabledef
)
AS
BEGIN
Declare #FirstVar Date = #First
Declare #DatedStartVar Date = #DatedStart
Declare #DatedEndVar Date = #DatedEnd
Declare #MaturityVar int = #Maturity
Declare #NumberVar decimal(10,5) = #Number
--SQL Statement which uses the local 'Var' variables and not the parameters
RETURN;
END
;
I've had similar probs in the past where this has been the culprit, and mapping to local variables stops SQL Server from coming up with a dud execution plan.
Related
currenty i am working on a report system for our data archive.
the aim is to select data for every 1st of a month, every full hour and so on.
So I have a bunch of parameters to select the data down to a single hour.
To achieve that I used CASE statements to adjust the select like this:
SELECT
MIN(cd.Timestamp) as Mintime,
--Hours
CASE
WHEN
#SelHour IS NOT NULL
THEN
DATEPART(HOUR, cd.Timestamp)
END as Hour,
... -- more CASES up to DATEPART(YEAR, cd.Timestamp)
FROM dbo.CustomerData cd
... -- filter data and other stuff
This statements works good for me so far, but I am a bit worried about the performance of the stored procedure. Because I don't know how the server will behave with this "changing" statement. The result can vary between a 20 row result up to a 250.000 rows and more. Depending on the given parameters. As far as I know the sql server saves the query plan and reuses it for future execution.
When it saves the plan for the 20 row result the performance for the 250.000 result is propably pretty poor.
Now I am wondering whats the better aproach. Using this stored procedure or create the statement inside my c# backend and pass the "adjusted" statement to the sql server?
Thanks and greetings
For 20 rows result set it will work good anywhere. But for returning 250k records to c# code seems change in design for this code since loading 250k records in memory & looping will also consume significant memory and such concurrent requests from different session/user will multiply load exponentially.
Anyway to address problem with SQL Server reusing same query plan, you can recompile query plans selectively or every time. These are options available for Recompile execution plan:
OPTION(RECOMPILE)
SELECT
MIN(cd.Timestamp) as Mintime,
--Hours
CASE
WHEN
#SelHour IS NOT NULL
THEN
DATEPART(HOUR, cd.Timestamp)
END as Hour,
... -- more CASES up to DATEPART(YEAR, cd.Timestamp)
FROM dbo.CustomerData cd
... -- filter data and other stuff
OPTION(RECOMPILE)
WITH RECOMPILE Option this will recompile execution plan every time
CREATE PROCEDURE dbo.uspStoredPrcName
#ParamName varchar(30) = 'abc'
WITH RECOMPILE
AS
...
RECOMPILE Query Hint providing WITH RECOMPILE in execute
NOTE: this will require CREATE PROCEDURE permission in the database and ALTER permission on the schema in which the procedure is being created.
EXECUTE uspStoredPrcName WITH RECOMPILE;
GO
sp_recompile System Stored Procedure
NOTE: Requires ALTER permission on the specified procedure.
EXEC sp_recompile N'dbo.uspStoredPrcName ';
GO
For more details on Recompile refer Microsoft Docs:
https://learn.microsoft.com/en-us/sql/relational-databases/stored-procedures/recompile-a-stored-procedure?view=sql-server-ver15
need a bit of help with this sql injection issue:
The following is a version of a parameterised stored procedure. Excluding how it is called from an application, is there anyway to prevent #v_string from being treated as dynamic SQL?
I think this is fairly water tight - there's no execute or concatenated sql, but still inserting a semicolon allows additional data to be returned.
I know there are multiple levels to consider this question on, but I want to know if there is some simple solution I am missing here as the majority of injection fixes involve dynamic queries.
create table dbo.Employee (EmpID int,EmpName varchar(60))
declare
#v_id int,
#v_string varchar(60)
begin
set #v_string='test'''; waitfor delay '0:0:5' --
if #v_id is null
begin
set #v_id = (select EmpID
from Abc.Employee
where EmpName=#v_string);
end
print #v_id
end
is there anyway to prevent #v_string from being treated as dynamic
SQL?
I would not expect #v_string to be treated as dynamic SQL here since the T-SQL code has no EXECUTE or EXECUTE sp_executeSQL. The value will not be executed, but treated as a WHERE clause value not vulnerable to SQL injection.
If this doesn't answer your question, post a full example that demonstrates the value being treated as dynamic SQL.
You're being confused by your own testing. The line:
set #v_string='test'''; waitfor delay '0:0:5' --
Is creating a string #v_string with the value test', and then executing waitfor delay '0:0:5'. Then your actual Employee query is being run.
So if you run your query as is, with your additional example:
set #v_string='test'''; select * from sys.databases
...what will happen is that line of code will set #v_string to be test', then immediately execute select * from sys.databases. Then the rest of your code will run, executing your actual select. So you'll see the result of select * from sys.databases, followed by the result of your Employee query, but only because you actually hard-coded the statement select * from sys.databases into your procedure without realising it :)
If you want the string #v_string to be set to test'; waitfor delay '0:0:5' then you've got the string quoting wrong. It should be:
set #v_string='test''; waitfor delay ''0:0:5'''
I have strange issue in my win-form application where I am calling a stored procedure and it takes about 6 seconds to execute.
(This stored procedure accepts several parameters including one output parameter)
From application level I used :
Dim dt1 = DateTime.Now
cmd.ExecuteNonQuery()
MessageBox.Show(DateTime.Now.Subtract(dt1).Seconds)
This is about 5-6 seconds
I have tried running the same stored procedure with same parameters on sql-server and it takes no time to execute:
declare #val decimal
exec mysp 'value1','value2','value3','value4',#val out
select #val
I am not sure what the issue is or where to start.
The issue with difference between calling SP directly and from .NET code, maybe due to parameter sniffing. SQL Server maybe caching execution plan that is not optimal for the parameters you're passing from code.
To avoid this try adding WITH RECOMPILE to your SP definition, e.g.
CREATE PROCEDURE MySP (
... parameters...
) WITH RECOMPILE
AS
BEGIN
...
It could be an issue if your stored procedure expects say nvarchar and you use a varchar. SQL server will accept the parameters but will be forced to do a conversion. Do you have more specifics?
We've a stored procedure that happens to build up some dynamic SQL and execute via a parametrised call to sp_executesql.
Under normal conditions, this works wonderfully, and has made a large benefit in execution times for the procedure (~8 seconds to ~1 second), however, under some unknown conditions, something strange happens, and performance goes completely the other way (~31 seconds), but only when executed via RPC (i.e. a call from a .Net app with the SqlCommand.CommandType of CommandType.StoredProcedure; or as a remote query from a linked server) - if executed as a SQL Batch using SQL Server Management Studio, we do not see the degradation in performance.
Altering the white-space in the generated SQL and recompiling the stored procedure, seems to resolve the issue at least in the short term, but we'd like to understand the cause, or ways to force the execution plans to be rebuilt for the generated SQL; but at the moment, I'm not sure how to proceed with either?
To illustrate, the Stored Procedure, looks a little like:
CREATE PROCEDURE [dbo].[usp_MyObject_Search]
#IsActive AS BIT = NULL,
#IsTemplate AS BIT = NULL
AS
DECLARE #WhereClause NVARCHAR(MAX) = ''
IF #IsActive IS NOT NULL
BEGIN
SET #WhereClause += ' AND (svc.IsActive = #xIsActive) '
END
IF #IsTemplate IS NOT NULL
BEGIN
SET #WhereClause += ' AND (svc.IsTemplate = #xIsTemplate) '
END
DECLARE #Sql NVARCHAR(MAX) = '
SELECT svc.[MyObjectId],
svc.[Name],
svc.[IsActive],
svc.[IsTemplate]
FROM dbo.MyObject svc WITH (NOLOCK)
WHERE 1=1 ' + #WhereClause + '
ORDER BY svc.[Name] Asc'
EXEC sp_executesql #Sql, N'#xIsActive BIT, #xIsTemplate BIT',
#xIsActive = #IsActive, #xIsTemplate = #IsTemplate
With this approach, the query plan will be cached for the permutations of NULL/not-NULL, and we're getting the benefit of cached query plans. What I don't understand is why it would use a different query plan when executed remotely vs. locally after "something happens"; I also don't understand what the "something" might be?
I realise I could move away from parametrisation, but then we'd lose the benefit of caching what are normally good execution plans.
I would suspect parameter sniffing. If you are on SQL Server 2008 you could try including OPTIMIZE FOR UNKNOWN to minimise the chance that when it generates a plan it does so for atypical parameter values.
RE: What I don't understand is why it would use a different query plan when executed remotely vs. locally after "something happens"
When you execute in SSMS it won't use the same bad plan because of different SET options (e.g. SET ARITHABORT ON) so it will compile a new plan that works well for the parameter values you are currently testing.
You can see these plans with
SELECT usecounts, cacheobjtype, objtype, text, query_plan, value as set_options
FROM sys.dm_exec_cached_plans
CROSS APPLY sys.dm_exec_sql_text(plan_handle)
CROSS APPLY sys.dm_exec_query_plan(plan_handle)
cross APPLY sys.dm_exec_plan_attributes(plan_handle) AS epa
where text like '%FROM dbo.MyObject svc WITH (NOLOCK)%'
and attribute='set_options'
Edit
The following bit is just in response to badbod99's answer
create proc #foo #mode bit, #date datetime
as
declare #Sql nvarchar(max)
if(#mode=1)
set #Sql = 'select top 0 * from sys.objects where create_date < #date /*44FC79BD-2AF5-4774-9674-04D6C3D4B228*/'
else
set #Sql = 'select top 0 * from sys.objects where modify_date < #date /*44FC79BD-2AF5-4774-9674-04D6C3D4B228*/'
EXEC sp_executesql #Sql, N'#date datetime',
#date = #date
go
declare #d datetime
set #d = getdate()
exec #foo 0,#d
exec #foo 1, #d
SELECT usecounts, cacheobjtype, objtype, text, query_plan, value as set_options
FROM sys.dm_exec_cached_plans
CROSS APPLY sys.dm_exec_sql_text(plan_handle)
CROSS APPLY sys.dm_exec_query_plan(plan_handle)
cross APPLY sys.dm_exec_plan_attributes(plan_handle) AS epa
where text like '%44FC79BD-2AF5-4774-9674-04D6C3D4B228%'
and attribute='set_options'
Returns
Recompilation
Any time the execution of the SP would be significantly different due to conditional statements the execution plan which was cached from the last request may not be optimal for this one.
It's all about when SQL compiles the execution plan for the SP. They key section regarding sp compilation on Microsoft docs is this:
... this optimization occurs automatically the first time a stored procedure is run after SQL Server is restarted. It also occurs if an underlying table that is used by the stored procedure changes. But if a new index is added from which the stored procedure might benefit, optimization does not occur until the next time that the stored procedure is run after SQL Server is restarted. In this situation, it can be useful to force the stored procedure to recompile the next time that it executes
SQL does recompile execution plans at times, from Microsoft docs
SQL Server automatically recompiles stored procedures and triggers when it is advantageous to do this.
... but it will not do this with each call (unless using WITH RECOMPILE), so if each execution could be resulting in different SQL, you may be stuck with the same old plan for at least one call.
RECOMPILE query hint
The RECOMPILE query hint, takes into account your parameter values when checking what needs to be recompiled at the statement level.
WITH RECOMPILE option
WITH RECOMPILE (see section F) will cause the execution plan to be compiled with each call, so you will never have a sub-optimal plan, but you will have the compilation overhead.
Restructure into multiple SPs
Looking at your specific case, the execution plan for the proc never changes and the 2 sql statements should have prepared execution plans.
I would suggest that restructuring the code to split the SPs rather than have this conditional SQL generation would simplify things and ensure you always have the optimal execution plan without any SQL magic sauce.
I've got a search screen on which the user can specify any combination of first name, last name, semester, or course. I'm not sure how to optimally code the SQL Server 2005 stored procedure to handle these potentially optional parameters. What's the most efficient way? Separate procedures for each combination? Taking the items in as nullable parms and building dynamic SQL?
I'd set each parameter to optional (default value being null)
and then handle it in the WHERE....
FirstName=ISNULL(#FirstName,FirstName)
AND
LastName=ISNULL(#LastName,LastName)
AND
SemesterID=ISNULL(#SemesterID,SemesterID)
That'll handle only first name, only last name, all three, etc., etc.
It's also a lot more pretty/manageable/robust than building the SQL string dynamically and executing that.
The best solution is to utilize sp_execute_sql. For example:
--BEGIN SQL
declare #sql nvarchar(4000)
set #sql =
'select * from weblogs.dbo.vwlogs
where Log_time between #BeginDate and #EndDate'
+ case when #UserName is null then '' else 'and client_user = #UserName' end
sp_execute_sql
#sql
, #params = '#UserName varchar(50)'
, #UserName = #UserName
--END SQL
As muerte mentioned, this will have a performance benefit versus exec()'ing a similar statement.
I would do it with sp_executesql because the plan will be cached just for the first pattern, or the first set of conditions.
Take a look at this TechNet article:
sp_executesql can be used instead of stored procedures to execute a Transact-SQL statement many times when the change in parameter values to the statement is the only variation. Because the Transact-SQL statement itself remains constant and only the parameter values change, the SQL Server query optimizer is likely to reuse the execution plan it generates for the first execution.
Was just posting the same concept as Kevin Fairchild, that is how we typically handle it.
You could do dynamic sql in your code to create the statement as required but if so you need to watch for sql injection.
As muerte points out, the plan will be cached for the first set of parameters. This can lead to bad performance when its run each subsequent time using alternate parameters. To resolve that use the WITH RECOMPILE option on the procedure.