We noticed that performance wasn't stellar but didn't bother us that much because it was a background process then we were contacted by our DBA who informed us that each stored procedure call was spanning a second call to get metadata from the DB.
Obviously nowhere in our code do we make any such calls and no, we're not using any ORM framework that might be issuing them behind the scenes.
Here's our setup:
- Standalone (no container) Java application
- spring-jdbc-3.2.2 is being used for data access
- we're using Microsoft JDBC Driver 4.0 for SQL Server
- we're using this syntax: CALL SPROC_NAME(:PAR_1)
Any pointers on how remove this extra call are really appreciated. I haven't been able to turn on logging for the SQL Server driver so my next step is to try to debug the spring JDBC code.
Thanks,
MV
Like I said, every Stored Procedure invocation (using PreparedStatement) was causing 2 calls to the DB with the first call being really expensive:
exec sp_sproc_columns #procedure_name =SPROC_NAME, #ODBCVer=3 (metadata call)
EXEC SPROC_NAME #P0, #P1, #P2, #P3 (actual call)
After reading the following thread we finally had something to work with:
Why is calling a stored procedure always preceded by a call to sp_sproc_columns?
Even though we're using JDBC instead of ADO.NET the issue seemed very similar to ours therefore we decided it was worth giving it a try.
After the change was in place a trace session in the DB confirmed the first call to be no longer present in the logs.
Previously we were passing a Map as parameters. The moment we switched to Map and specified the proper data type we started seeing performance improvements.
In short, if the driver has all the data (actually metadata) about the params then it doesn't issue the metadata call.
It has been a while since this question was posted; However, I'll leave my answer given that I was having this issue at work and it took me quite a while to figure out. Hope it helps somebody.
Short Answer: Do not use named parameters in your procedure calls. Use ordered parameters instead.
Long Answer: When using named parameters, the JDBC driver issues a call to the 'sp_sproc_columns' procedure first. The driver issues this call in order to get the name of the parameters in the procedure and runs some validations in order to check if the names are correct. It later replaces the named parameters in your query and puts them in the correct order. Once the correct order is determined, the driver issues the call to the procedure using ordered parameters instead.
View the call in the source code. https://github.com/Microsoft/mssql-jdbc/blob/dev/src/main/java/com/microsoft/sqlserver/jdbc/SQLServerCallableStatement.java#L1254
Related
As mentioned in title. I wanna know that does SQL Server can partially execute a SQL like oracle do? means server wait client to process the data in last network packet and then fill next packet I noticed that when you execute a select statement in SQLServer Management studio, you can get the first rows immediately while the statement is still running. But I cannot get this in ADO component. All I can think about is use server-side cursor and use non-keyset type cursor. But ADO use sp_cursoropen and sp_cursorfetch to do this. Not using cache size like sql*plus do. It mains it is client side behavier, not like the phenomenon in SSMS. I want know why.
It appears that the corresponding feature in SQL Server is known as Array Fetch Size. (Discussed here, for example.)
Are you referring to SET ARRAYSIZE, an SQL*Plus command?
If so, an analogous setting might be SET ROWCOUNT. There may be some subtleties I'm missing here though, since I don't use Oracle on a daily basis.
If I've guessed wrong, then can you expand on what "Array Size" does, since I can't find any hits that seem relevant when searching for Oracle Array Size.
Actually, re-reading stuff I can find about set arraysize in SQL*Plus, it seems to be quite different from ROWCOUNT. But the description (which seems to control how many rows are returned in each round trip, but not limiting the overall number of rows) doesn't seem to match your description "partially execute a SQL like oracle". I can't think of anything off the top of my head that would control the batching of results returned to the client.
am used lot of time , i know the diff between sql query and sp ,
SQL query will be compiled everytime it is executed.
Stored procedures are compiled only once when they are
executed for the first time.
This is general database question
But one big doubt is ,
For example ,
one dynamic work , that is i pass the userid to SP and sp will return the username,password,full details,
So for this scenario the query should execute once again know, so what is the necessary of SP instead of SQL QUERY ,
Please clear this doubt ,
Hi thanks for all your updates,
but i dont want the advantage, comparison ,
just say ,
How sp executing , while we go with dynamic works,
For example ,
if pass userid 10 then sp also read records 10 ,
if i pass 14 then, SP again look the 14 records , see this same work NORMAL SQL QUERY
doing , but on that time execute and fetching ,so why should i go for sp ,
Regards
Stored procedures, like the name says, are stored on the database server. They are transmitted to the server and compiled when you create them, and executed when you call them.
Simple SQL queries, on the other hand, are transmitted to the server and compiled each time you use them.
So transmitting of a huge query (instead of a simple "execute procedure" command) and compiling create an overhead which can be avoided by the use of a stored procedure.
MySQL, like other RDBMS, has a query cache. But this avoid only compiling, and only if the query is exactly the same than a previously executed query, which means the cache is not used if you execute 2 times the same query, with different values in a where clause, for example.
I see no reason for a stored procedure simply to query for all user details.
Stored procedures are functional code that you execute on the database server. I can think of three reasons why you'd use them:
To create an interface for users that hides the schema details from clients.
Performance. Extensive calculations on a large data set might be done more efficiently on the database server
Sometimes it can be difficult (or impossible, depending on your skill) to express what you think you need in a declarative, set-based language like SQL. That's when some people throw up their hands and write stored procs.
Only 1. would be justifiable from your question. I would recommend sticking with SQL.
UPDATE: The new information you provided still does not justify stored procedures in my opinion. A query that returns 14 records is routine.
Does anyone know of a way to verify the correctness of the queries in all stored procedures in a database?
I'm thinking of the scenario where if you modify something in a code file, simply doing a rebuild would show you compilation errors that point you to places where you need to fix things. In a database scenario, say if you modify a table and remove a column which is used in a stored procedure you won't know anything about this problem until the first time that procedure would run.
What you describe is what unit testing is for. Stored procedures and functions often require parameters to be set, and if the stored procedure or function encapsulates dynamic SQL--there's a chance that a [corner] case is missed.
Also, all you mention is checking for basic errors--nothing about validating the data returned. For example - I can change the precision on a numeric column...
This also gets into the basic testing that should occur for the immediate issue, and regression testing to ensure there aren't unforeseen issues.
You could create all of your objects with SCHEMABINDING, which would prevent you from changing any underlying tables without dropping and recreating the views and procedures built on top of them.
Depending on your development process, this could be pretty cumbersome. I offer it as a solution though, because if you want to ensure the correctness of all procedures in the db, this would do it.
I found this example on MSDN (SQL Server 2012). I guess it can be used in some scenarios:
USE AdventureWorks2012;
GO
SELECT p.name, r.*
FROM sys.procedures AS p
CROSS APPLY sys.dm_exec_describe_first_result_set_for_object(p.object_id, 0) AS r;
Source: sys.dm_exec_describe_first_result_set_for_object
I know some ways that we can use in order to determine that whether our own Stored procedure has been executed successfully or not. (using output parameter, putting a select such as select 1 at the end of the stored procedure if it has been executed without any error, ...)
so which one is better and why?
Using RAISERROR in case of error in the procedure integrates better with most clients than using fake out parameters. They simply call the procedure and the RAISERROR translates into an exception in the client application, and exceptions are hard to avoid by the application code, they have to be caught and dealt with.
Having a print statement that clearly states whether the SP has been created or not would be more readable.
e.g.
CREATE PROCEDURE CustOrdersDetail #OrderID int
AS
...
...
...
GO
IF OBJECT_ID('dbo.CustOrdersDetail') IS NOT NULL
PRINT '<<< CREATED PROCEDURE dbo.CustOrdersDetail >>>'
ELSE
PRINT '<<< FAILED CREATING PROCEDURE dbo.CustOrdersDetail >>>'
GO
SP is very much like a method/subroutine/procedure & they all have a task to complete. The task could be as simple as computing & returning a result or could be just a simple manipulation to a record in a table. Depending on the task, you could either return a out value indicating the result of the task whether it was a success, failure or the actual results.
If you need common T-SQL solution for your entire project/database, you can use the output parameter for all procedures. But RAISEERROR is the way to handle errors in your client code, not T-SQL.
Why don't use different return values which then can be handled in code?
Introducing an extra output paramter or an extra select is unnecessary.
If the only thing you need to know is whether there is a problem, a successful execution is good enough choice. Have a look at the discussions of XACT_ABORT and TRY...CATCH here and here.
If you want to know specific error, return code is the right way to pass this information to the caller.
In the majority of production scenarios I tend to deploy a custom error reporting component within the database tier, as part of the solution. Nothing fancy, just a handful of log tables and a few of stored procedures that manage the error logging process.
All stored procedure code that is executed on a production server is then encapsulated using the TRY-CATCH-BLOCK feature available within SQL Server 2005 and above.
This means that in the unlikely event that a given stored procedures were to fail, the details of the error that occurred and the stored procedure that generated it are recorded to a log table. A simple stored procedure call is made from within the CATCH BLOCK in order to record the relevant details.
The foundations for this implementation are actually explained in books online here
Should you wish, you can easily extend this implementation further, for example by incorporating email notification to a DBA or even an SMS alert could be sent dependent on the severity of the error.
An implementation of this sort ensures that if your stored procedure did not report failure then it was of course successful.
Once you have a simple and robust framework in place, it is then straightforward to duplicate and rollout your base implementation to other production servers/application platforms.
Nothing special here, just simple error logging and reporting that works.
If on the other hand you also need to record the successful execution of stored procedures then again, a similar solution can be devised that incorporates log table/s.
I think this question is screaming out for a blog post……..
A while ago I had a query that I ran quite a lot for one of my users. It was still being evolved and tweaked but eventually it stablised and ran quite quickly, so we created a stored procedure from it.
So far, so normal.
The stored procedure, though, was dog slow. No material difference between the query and the proc, but the speed change was massive.
[Background, we're running SQL Server 2005.]
A friendly local DBA (who no longer works here) took one look at the stored procedure and said "parameter spoofing!" (Edit: although it seems that it is possibly also known as 'parameter sniffing', which might explain the paucity of Google hits when I tried to search it out.)
We abstracted some of the stored procedure to a second one, wrapped the call to this new inner proc into the pre-existing outer one, called the outer one and, hey presto, it was as quick as the original query.
So, what gives? Can someone explain parameter spoofing?
Bonus credit for
highlighting how to avoid it
suggesting how to recognise possible cause
discuss alternative strategies, e.g. stats, indices, keys, for mitigating the situation
FYI - you need to be aware of something else when you're working with SQL 2005 and stored procs with parameters.
SQL Server will compile the stored proc's execution plan with the first parameter that's used. So if you run this:
usp_QueryMyDataByState 'Rhode Island'
The execution plan will work best with a small state's data. But if someone turns around and runs:
usp_QueryMyDataByState 'Texas'
The execution plan designed for Rhode-Island-sized data may not be as efficient with Texas-sized data. This can produce surprising results when the server is restarted, because the newly generated execution plan will be targeted at whatever parameter is used first - not necessarily the best one. The plan won't be recompiled until there's a big reason to do it, like if statistics are rebuilt.
This is where query plans come in, and SQL Server 2008 offers a lot of new features that help DBAs pin a particular query plan in place long-term no matter what parameters get called first.
My concern is that when you rebuilt your stored proc, you forced the execution plan to recompile. You called it with your favorite parameter, and then of course it was fast - but the problem may not have been the stored proc. It might have been that the stored proc was recompiled at some point with an unusual set of parameters and thus, an inefficient query plan. You might not have fixed anything, and you might face the same problem the next time the server restarts or the query plan gets recompiled.
Yes, I think you mean parameter sniffing, which is a technique the SQL Server optimizer uses to try to figure out parameter values/ranges so it can choose the best execution plan for your query. In some instances SQL Server does a poor job at parameter sniffing & doesn't pick the best execution plan for the query.
I believe this blog article http://blogs.msdn.com/queryoptteam/archive/2006/03/31/565991.aspx has a good explanation.
It seems that the DBA in your example chose option #4 to move the query to another sproc to a separate procedural context.
You could have also used the with recompile on the original sproc or used the optimize for option on the parameter.
A simple way to speed that up is to reassign the input parameters to local parameters in the very beginning of the sproc, e.g.
CREATE PROCEDURE uspParameterSniffingAvoidance
#SniffedFormalParameter int
AS
BEGIN
DECLARE #SniffAvoidingLocalParameter int
SET #SniffAvoidingLocalParameter = #SniffedFormalParameter
--Work w/ #SniffAvoidingLocalParameter in sproc body
-- ...
In my experience, the best solution for parameter sniffing is 'Dynamic SQL'. Two important things to note is that 1. you should use parameters in your dynamic sql query 2. you should use sp_executesql (and not sp_execute), which saves the execution plan for each parameter values
Parameter sniffing is a technique SQL Server uses to optimize the query execution plan for a stored procedure. When you first call the stored procedure, SQL Server looks at the given parameter values of your call and decides which indices to use based on the parameter values.
So when the first call contains not very typical parameters, SQL Server might select and store a sub-optimal execution plan in regard to the following calls of the stored procedure.
You can work around this by either
using WITH RECOMPILE
copying the parameter values to local variables inside the stored procedure and using the locals in your queries.
I even heard that it's better to not use stored procedures at all but to send your queries directly to the server.
I recently came across the same problem where I have no real solution yet.
For some queries the copy to local vars helps getting back to the right execution plan, for some queries performance degrades with local vars.
I still have to do more research on how SQL Server caches and reuses (sub-optimal) execution plans.
I had similar problem. My stored procedure's execution plan took 30-40 seconds. I tried using the SP Statements in query window and it took few ms to execute the same.
Then I worked out declaring local variables within stored procedure and transferring the values of parameters to local variables. This made the SP execution very fast and now the same SP executes within few milliseconds instead of 30-40 seconds.
Very simple and sort, Query optimizer use old query plan for frequently running queries. but actually the size of data is also increasing so at that time new optimized plan is require and still query optimizer using old plan of query. This is called Parameter Sniffing.
I have also created detailed post on this. Please visit this url:
http://www.dbrnd.com/2015/05/sql-server-parameter-sniffing/
Changing your store procedure to execute as a batch should increase the speed.
Batch file select i.e.:
exec ('select * from order where order id ='''+ #ordersID')
Instead of the normal stored procedure select:
select * from order where order id = #ordersID
Just pass in the parameter as nvarchar and you should get quicker results.