I was wondering if you guys could help me get to the bottom of a weird problem I have recently had on SQL Server.
I have a stored procedure (lets call SPold) which is reasonably large with a lot of calculations (can't possibly do this in app as info for around 6000 users needs to come back in a one-er (I reduce this to 1000 based on Surname)). The stored procedure usually executes in a couple of seconds, and is called once every couple of minutes.
Now this morning, the stored procedure was suddenly taking 4-10 times as long to execute, causing a number of timeouts. I discovered that by making a copy of the procedure with a new name (SPnew) and executing, I would get the fast execution times again. This indicated to me that the execution plan was the problem with the original, SPold, so I decided to execute it with recompile. This would return the results a quicker (although not as fast as SPnew), but subsequent calls from users to SPold were once again slow. It was like the new plan wasn't being kept.
What I have done is to fix this is put Exec SPnew into SPold, and now calls to SPold are returning fast again.
Does anyone have any idea what is going on here? The only thing that updated overnight was the statistics, although I think that this should affect both SPold and SPnew.
Sounds like you are experiencing an incorrectly cached query plan due to parameter sniffing.
Can you post the stored procedure?
Batch Compilation, Recompilation, and Plan Caching Issues in SQL Server 2005
I Smell a Parameter!
In SQL Server 2005, you can use the OPTIMIZE FOR query hint for preferred values of parameters to remedy some of the problems associated with parameter sniffing:
OPTIMIZE FOR Instructs the query optimizer to use a particular value for a local
variable when the query is compiled and optimized. The value is used
only during query optimization, and not during query execution.
OPTIMIZE FOR can counteract the parameter detection behavior of the
optimizer or can be used when you create plan guides. For more
information, see Recompiling Stored Procedures and Optimizing Queries
in Deployed Applications by Using Plan Guides.
Although SQL Server 2005 does not support OPTIMIZE FOR UNKNOWN (introduced in SQL Server 2008) which
will eliminate parameter sniffing for a given parameter:
OPTION (OPTIMIZE FOR (#myParam UNKNOWN))
You can achieve the same effect in SQL Server 2005 by copying the parameter into a local variable, and then use the local variable in the query.
I've also encounterred two "strange" cases with Sql Server 2005, which might relate to your problem as well.
In the first case my procedure executed prety fast, when being run as dbo, and it was slow when being run from the application, under a different user account.
In the second case the query plan of the procedure got optimized for the parameter values with which the procedure was called for the first time, and this plan was then reused later for other parameter values as well, resulting in a slow execution.
For this second case the solution was to copy the parameter values into local variables in the procedure, and then using the variables in the queries instead of the parameters.
Related
We have a main stored procedure that returns around 1000 records, changes by the user permissions.
Lately the procedure performance became very bad - but only from the web-service - more than a minute!
but when running the same SP with the same parameters from ssms took only 3 seconds!!
When I tried to check the problem I added writes to log table, and immediately this change improved the performance again to 3 seconds from the web-service.
This is a mystery for me:
1. The difference between running from web-service and ssms
2. The change after adding the logging
Your issue is called parameter sniffing. There were 2 execution plans for this procedure, one created the first time you launched it from web server and another created when you lanuched it from SSMS. And the parameters of these two plans were different. The next time you execute this proc, one of this plans is used: when you execute from SSMS, the second plan is used, and from web service the first plan is used. The parameters passed to this proc were atypical when executed from wb service, and typical when executed from SSMS.
When you altered your procedure, those 2 plans were invalidated as the procedure has changed, then the new execution plan was built for SSMS and for web service, this times both plans were made for the same or similar paremeters.
If you could extract old plans from plan cache you'd see they were different and the parameters sniffed also were different while now the plans are the same and parameter sniffed are the same or similar.
Here you can read more on parameter sniffing: Slow in the Application, Fast in SSMS?
Understanding Performance Mysteries
Please do not use functions on TOP, on recordset, and, on WHERE/JOIN clauses.
When youre calling SP from SSMS, server optimizes it. But, when calling from frontend, it is huge problem. So, eliminate functions, if possible.
If you want to view about what im talking, please start profiler and then log RPC starting/completed, sql statament events. Call quantity is same, as recordset. So, assume, youre calling procedure 1000 times when usig FN on statement , returning recordset.
I have an SSRS report which has seen its TimeDataRetrieval (as per the ExecutionLog3 table in the ReportingServices database) increase by 60 seconds overnight and I can't figure out why.
The report has two parameters and contains a single Dataset which passes one of those report parameters to a SQL stored procedure. I can run the stored procedure standalone in SSMS and it completes in seconds, in line with the previous report performance.
I have read many threads and articles online about how parameter sniffing affects the execution plan which SQL builds for a stored procedure when it's been called from an SSRS report versus when it is run direct, but I've tried adding an internal variable to the stored procedure, assigning the incoming parameter value to that variable and using that variable in the query within the stored procedure instead of the parameter, but this didn't make any difference to the issue. I also even tried adding OPTION(RECOMPILE) to the stored procedure, but again this had no impact.
The issue began occurring right after we upgraded our Dynamics CRM 2015 system (whose database resides on the same SQL Server as this instance of SSRS - probably a bad idea I know) to Dynamics 365, so I'm wondering if that could somehow have something to do with it, but I'm at a loss as to how to troubleshoot this one, so any suggestions would be most welcome!
Do the tables that this SP runs from steadily grow in size? Sometimes you get a 'threshold' affect where suddenly the number of rows cause performance issues. I suggest you rebuild statistics on all the tables in use and add the OPTION(RECOMPILE) and retest.
Also when trying to recreate in SSMS, you must make sure you also include all the SET options. You should capture the SQL using profiler and use exactly that, including all of the four or five set options before it (i.e. SET ARITHABORT)
You might find you can then reproduce in SSMS, in which case it is definiteley a parameter sniffing issue. (although recompile usually fixes that)
I have a question: let's say I have a procedure which contains dynamic SQL inside the definition of procedure, so when I execute the procedure for the first time, it's obvious that it compiles the procedure and stores the plan for first time.
What happens during the second run? Will the same plan be used or will the procedure go for a recompile as it contains dynamic SQL in it?
Dynamic SQL is always compiled. It may result in the same execution plan as the first run (totally dependent on parameters).
I would suggest reading this article from MS. Relevant quotes:
Recompilations: Definition
Before a query, batch, stored procedure, trigger, prepared statement, or dynamic SQL statement (henceforth, "batch") begins execution on a SQL Server, the batch gets compiled into a plan. The plan is then executed for its effects or to produce results.
and
Compiled plans are stored into a part of SQL Server's memory called plan cache . Plan cache is searched for possible plan reuse opportunities. If a plan reuse for a batch happens, its compilation costs are avoided.
A similar question has already been answered in Stack Exchange for Database Administrators. Please refer: https://dba.stackexchange.com/questions/47283/when-does-sp-executesql-refresh-the-query-plan
Setup:
Using SQL Server 2008 R2.
We've got a stored procedure that has been intermittently running very long. I'd like to test a theory that parameter sniffing is causing the query engine to choose a bad plan.
Question:
How can I copy the query's execution plans from one database to another (test) database?
Note:
I'm fully aware that this may not be parameter sniffing issues. However, I'd like to go through the motions of creating a test plan and using it, if at all possible. Therefore please do not ask me to post code and/or table schema, since this is irrelevant at this time.
Plans are not portable, they bind to object IDs. You can use planguides, but they are strictly tied to the database. What you have to do is test on a restored backup of the same database. On a restored backup you can use a planguide. But for relevance the physical characteristics of the machines should be similar (CPUs, RAM, Disks).
Normally though one does not need to resort to such shenanigans as copy the plans. Looking at actual execution plans all the answers are right there.
Have you tried using OPTIMIZE FOR clause? With it you can tune your procedure easier, and without the risk that plan that you copy from another database will be inappropriate due to differences in those databases (if copying the plan is even possible).
http://www.mssqltips.com/sqlservertip/1354/optimize-parameter-driven-queries-with-sql-server-optimize-for-hint/
Why is it that if I run my query as a parameterized procedure it runs 10 times faster then if I run it directly as a parameterized query?
I'm using the exact same query in both cases, and it doesn't matter if I'm calling from Management Studio or an SqlCommand from code.
EDIT: The execution plan looks different. So, why? I'm calling it with EXACTLY the same set of parameters.
EDIT: After more testing it seems the 10x slowdown only occurs when running the parameterized query from SQL Management Studio.
One thing I've seen recently is that if you set up the query parameters wrong it can cause major problems.
For example, say you have a parameter for an indexed varchar column and set it up from .Net using the SqlCommand's AddWithValue() method. You're in for a world of hurt with this scenario. .Net uses unicode strings and will set up your parameter as an nvarchar rather than varchar. Now sql server won't be able to use your index, and you'll see a significant performance penalty.
Find out if they are using the same execution plan is to display it when running. Use "include actual execution plan" in management studio and see what is the difference.
The connection-level settings can be critical in some cases, especially ANSI NULLS, CONCAT NULL YIELDS NULL, etc. In particular, if you have calculated persisted indexed columns (including promoted "xml" columns), then it won't trust the pre-calculated, index value if the settings aren't compatible, and will recalculate for each row (i.e. table scan instead of index seek).
Parameterized queries have a lot of advantages, including often a hefty performance increases.
Caching of queries
String Concatenation problems minimized
addressing SQL injection
Data does not have to be converted to a string before processing
Parameter sniffing may be affecting the stored procedure performance.
http://omnibuzz-sql.blogspot.com/2006/11/parameter-sniffing-stored-procedures.html
Stored procedures can run faster because the execution plan is cached by sql server.
But 10 times performance is suspicious.
Does it run the same the First time after you've cleared the stored execution plans? You can use these commands to clear the cache. But they clear the whole servers cache so do it only on development servers.
DBCC FREEPROCCACHE
DBCC FLUSHPROCINDB (<dbid>)
Are you running these directly on the SQL server to eliminate any network I/O from the performance testing?
My guess is that it is running slowly the first time, then once it is cached it is running faster.