I have a certain stored procedure written years ago. It uses a table variable. Inside the code it's doing:
declare #OpportunityVS3 as SF_OpportunityMerge
insert #TableVariable select * from /*...*/
This is inside a job that runs every five minutes. There used to be no issues in the past because the select used to collect around 10 to 1000 records… But now sometimes the select is trying to insert a million records.
Given this issue, I think I am forced to change the #TableVariable to a #TempTable. What do you think? Do I have any other option?
At the end of the SP, it’s passing the Variable to another SP:
exec [dbo].[SF_MergeOpportunity] #OpportunityVS3, #LastUpdateDate
I guess I will have to write it all in the same stored procedure, because temporary tables cannot be passed, right?
Below is the definition of the #TableVariable that is being used and the code I would have to change:
declare #OpportunityVS3 as SF_OpportunityMerge
insert #OpportunityVS3 select Opportunity_id, Salesforce_id, AccountId, Age__c,
-- (continues)
merge [BU2].[dbo].[Salesforce_Opportunity] as TARGET
using ( select Opportunity_id,
Salesforce_id,
AccountId,
Age__c,
Amount,
Bill99Amount__c,
BJ_Marketing__c
-- (continues)
from #OpportunityVS3 ) as SOURCE on (TARGET.Opportunity_id = SOURCE.Opportunity_id)
exec [dbo].[SF_MergeOpportunity] #OpportunityVS3, #LastUpdateDate
[![enter image description here][1]][1]
[![enter image description here][2]][2]
What you can try without changing the way you work:
Force the compiler to take cardinalities of TABLE variables into account by specifying OPTION(RECOMPILE) in queries using the TABLE variables
Supply a suitable UNIQUE INDEX or PRIMARY KEY for the TABLE variable
A downside of TABLE variables is that there are no statistics associated with them. The SQL compiler will produce better execution plans when it can take statistics into account. Therefore you can improve performance considerably when you switch to temporary tables because these do have statistics.
When you switch to temporary tables, you would no longer pass a TABLE variable to your stored procedure anymore. Your stored procedure would then be written using a temporary table that it knows exists beforehand - ie was created before the stored procedure is executed.
You would write your stored procedure as though the temporary table were an ordinary table. When you write the stored procedure using a temporary table, SQL Server management studio will underline certain parts in red so it appears as though there are errors in your stored procedure. But if the syntax is correct, creating/altering the stored procedure will work just fine.
Read this excellent essay on the differences between TABLE variables and temporary tables, to see how usage may impact performance. Especially topics No column statistics and Indexes.
Related
I have stored procedures I need to rewrite that used global temporary tables but now must use local temporary tables so that there is no cross over between users/the prod and dev db which are on the same SQL Server instances or any other new databases this database will be share the instance with.
I have to convert a lot of code that is like this
EXEC sp_executesql
N'SELECT manyDynamicallyCreatedColumns INTO ##someTempTable
I now for this to work for local tables I must instead create the table first and then INSERT INTO like
CREATE TABLE #tempTable (manyColumns);
EXEC sp_executesql
N'INSERT INTO #tempTable SELECT manyColumns FROM somewhere'
The issue I face is that this stored procedure has 8 different scenarios that lead to the global temp table. The select statement of the SQL is dynamically generated via a number of other queries. I think what would be the easiest way for me to figure out what my CREATE TABLE #tempTable should look like is if I could print/select what global temporary table looks like after it is made, for each of these 8 scenarios. Then I would just need a If/Else If statement that creates the local table appropriately before proceeding. But I don't know how or if this is possible.
For real tables I can right click -> Script to...-> Create. I don't know if there is an analogous way to do this via scripting that works for global temporary tables. Is there?
I am stress testing a system that is using a temporary table in dynamic SQL. The table is created early on in the transaction and is filled by several dynamic SQL statements in several stored procedures that are executed as part of the batch using statements of the form:
INSERT #MyTable (...)
SELECT ...
where the SELECT statement is reasonably complicated in that it may contain UNION ALL and UNPIVOT statements and refer to several UDFs. All strings are executed using sp_executesql and Parameter Sniffing is enabled.
I have noticed that under load I am seeing a lot of RESOURCE_SEMAPHORE_QUERY_COMPILE waits where the query text being recompiled is present and identical in several waits at the same time and appears throughout the stress test which lasts about 5mins. The memory consumption on the server usually sits around 60% utilization and there is no limit on how much SQL Server can consume. The limiting factor appears to be CPU, which is constantly at >95% during the test.
I have profiled the server during the test to observe the SQL:StmtRecompile event which highlights the reason for the recompile is:
5 - Temp table changed
but the temp table is the same every time and there are no DDL statements performed against the table once it has been created, apart from when it is dropped at the end of the batch.
So far, I have tried:
Enabling the "optimize for ad hoc workloads" option
OPTION(KEEPFIXED PLAN)
Changing the dynamic statement to just the SELECT and then using INSERT ... EXEC so the temp table is not in the executed string
All of these have made no difference and the waits persist.
Why would SQL think that is needs to recompile these identical queries each time they are executed and how can I get it to keep and reuse the cached plans it is creating?
Note: I cannot change the temp table to an In-Memory table because sometimes the stored procedures using this may have to query another database on the same instance.
This is using SQL Server 2016 SP1 CU7.
It appears that removing the insertion into the temp table in the dynamic SQL strings improves performance significantly. For example, changing this:
EXEC sp_executesql
N'INSERT #tempTable (...) SELECT ... FROM ...'
where the SELECT statement is non-trivial, to this:
INSERT #tempTable (...)
EXEC sp_executesql
N'SELECT ... FROM ...'
greatly reduced the number of blocks created during compilation. Unfortunately, recompilation of the queries is not avoided it's just that the queries being recompiled are now much simpler and therefore less CPU intensive.
I have also found it more performant to create an In-Memory Table Type with the same columns as the temp table, perform the complex insertions into a table variable of that type and perform a single insert from the table variable into the temp table at the end.
I want to create temp table with their unique name by a select query in a stored procedure in SQL Server.
For example: whenever I run the select query at that time different temp table name want to create.
Let be more clear, at the first time when I will run the select query at that time temptable name is temptable1, while at the second time the table name will be temptable2 and so on.
I want to know the syntax for executing the select query and creating the temptable with their unique name in a stored procedure in SQL Server.
In the context of the SQL Server Stored Procedure, the engine is handling itself the names of the temporary tables.
There is no need to worry if many users are executing the same stored procedure in same time - the temporary objects cannot be shared across them and no conflicts are going to happen.
Also, naming a temporary table in stored procedure with different name can be done using a dynamic T-SQL statement. You can for example, use a sequence to get such number and concatenate it to the table name. But, if you do so, you need to use sp_executesql to create your table and do things with it. In such way, no other stored procedure would be able to read/modify the table you have created in the current stored procedure. In other words, the temporary table cannot be shared over the routines if created using dynamic T-SQL statement. So, there is absolutely no point of doing such thing.
I saw this question quite a many times but I couldn't get the answer that would satisfy me. Basically what people and books say is "Although temporary tables are deleted when they go out of scope, you should explicitly delete them when they are no longer needed to reduce resource requirements on the server".
It is quite clear to me that when you are working in management studio and creating tables, then until you close your window or disconnect, you will use some resources for that table and it is logically that it is better to drop them.
But when you work with procedure then if you would like to cleanup tables most probably you will do that at the really end of it (I am not talking about the situation when you drop the table as soon as you really do not need that in the procedure). So the workflow is something like that :
When you drop in SP:
Start of SP execution
Doing some stuff
Drop tables
End of execution
And as far as I understand how can it possibly work when you do not drop:
Start of SP execution
Doing some stuff
End of execution
Drop tables
What's the difference here? I can only imagine that some resources are needed to identify the temporary tables. Any other thoughts?
UPDATE:
I ran simple test with 2 SP:
create procedure test as
begin
create table #temp (a int)
insert into #temp values (1);
drop table #temp;
end
and another one without drop statements. I've enabled user statistics and ran the tests:
declare #i int = 0;
while #i < 10000
begin
exec test;
SET #i= #i + 1;
end
That's what I've got (Trial 1-3 dropping table in SP, 4-6 do not dropping)
As the picture shows that all stats are the same or decreased a bit when I do not drop temporary table.
UPDATE2:
I ran this test 2nd time but now with 100k calls and also added SET NOCOUNT ON. These are the results:
As the 2nd run confirmed that if you do not drop the table in SP then you actually save some user time as this is done by some other internal process but outside of the user time.
You can read more about in in this Paul White's article: Temporary Tables in Stored Procedures
CREATE and DROP, Don’t
I will talk about this in much more detail in my next post, but the
key point is that CREATE TABLE and DROP TABLE do not create and drop
temporary tables in a stored procedure, if the temporary object can be
cached. The temporary object is renamed to an internal form when DROP
TABLE is executed, and renamed back to the same user-visible name when
CREATE TABLE is encountered on the next execution. In addition, any
statistics that were auto-created on the temporary table are also
cached. This means that statistics from a previous execution remain
when the procedure is next called.
Technically, a locally scoped temp table (one with a single hashtag before it) will automatically drop out of scope after your SPID is closed. There are some very odd cases where you get a temp table definition cached somewhere and then no real way to remove it. Usually that happens when you have a stored procedure call which is nested and contains a temp table by the same name.
It's good habit to get into dropping your tables when you're done with them but unless something unexpected happens, they should be de-scoped anyway once the proc finishes.
i have a stored proc A that contains a stored proc B.
stored proc B does an insert and returns a row of information.
is there a way to access that info in stored proc A?
You can execute the stored procedure and selected it into a temp table.
Create #table ()....
INSERT INTO #table EXEC your_procedure
The only time when it really becomes difficult (and maybe impossible, I've never seen it done) is when the stored procedure returns multiple recordsets (not multiple records) and the recordsets have different fields.
EDIT:
You can can also use a table variable (DECLARE #my_table TABLE()) to do the same thing. In your situation you'll want to try both and see which is better.
http://www.sql-server-performance.com/2007/temp-tables-vs-variables/
You could create a temp table and then insert-exec from the inner procedure call.
Share Data
Scroll down to the Insert-Exec section.
I will recommend you to create a Table variable and insert the row information into it
EDIT:
Please note that it will be useful if not under Transaction and not joining it with other tables and just acting as a intermediate for containing row information as you just said in the query.