MS SQL Server - safe concurrent use of global temp table? - sql-server

In MS SQL Server, I'm using a global temp table to store session related information passed by the client and then I use that information inside triggers.
Since the same global temp table can be used in different sessions and it may or may not exist when I want to write into it (depending on whether all the previous sessions which used it before are closed), I'm doing a check for the global temp table existence based on which I create before I write into it.
IF OBJECT_ID('tempdb..##VTT_CONTEXT_INFO_USER_TASK') IS NULL
CREATE TABLE ##VTT_CONTEXT_INFO_USER_TASK (
session_id smallint,
login_time datetime,
HstryUserName VDT_USERNAME,
HstryTaskName VDT_TASKNAME,
)
MERGE ##VTT_CONTEXT_INFO_USER_TASK As target
USING (SELECT ##SPID, #HstryUserName, #HstryTaskName) as source (session_id, HstryUserName, HstryTaskName)
ON (target.session_id = source.session_id)
WHEN MATCHED THEN
UPDATE SET HstryUserName = source.HstryUserName, HstryTaskName = source.HstryTaskName
WHEN NOT MATCHED THEN
INSERT VALUES (##SPID, #LoginTime, source.HstryUserName, source.HstryTaskName);
The problem is that between my check for the table existence and the MERGE statement, SQL Server may drop the temp table if all the sessions which were using it before happen to close in that exact instance (this actually happened in my tests).
Is there a best practice on how to avoid this kind of concurrency issues, that a table is not dropped between the check for its existence and its subsequent use?

The notion of "global temporary table" and "trigger" just do not click. Tables are permanent data stores, as are their attributes -- including triggers. Temporary tables are dropped when the server is re-started. Why would anyone design a system where a permanent block of code (trigger) depends on a temporary shared storage mechanism? It seems like a recipe for failure.
Instead of a global temporary table, use a real table. If you like, put a helpful prefix such as temp_ in front of the name. If the table is being shared by databases, then put it in a database where all code has access.
Create the table once and leave it there (deleting the rows is fine) so the trigger code can access it.

I'll start by saying that, on the long term, I will follow Gordon's advice, i.e. I will take the necessary steps to introduce a normal table in the database to store client application information which needs to be accessible in the triggers.
But since this was not really possible now because of time constrains (it takes weeks to get the necessary formal approvals for a new normal table), I came up with a solution for preventing SQL Server from dropping the global temp table between the check for its existence and the MERGE statement.
There is some information out there about when a global temp table is dropped by SQL Server; my personal tests showed that SQL Server drops a global temp table the moment the session which created it is closed and any other transactions started in other sessions which changed data in that table are finished.
My solution was to fake data changes on the global temp table even before I check for its existence. If the table exists at that moment, SQL Server will then know that it needs to keep it until the current transaction finishes, and it cannot be dropped anymore after the check for its existence. The code looks now like this (properly commented, since it is kind of a hack):
-- Faking a delete on the table ensures that SQL Server will keep the table until the end of the transaction
-- Since ##VTT_CONTEXT_INFO_USER_TASK may actually not exist, we need to fake the delete inside TRY .. CATCH
-- FUTURE 2016, Feb 03: A cleaner solution would use a real table instead of a global temp table.
BEGIN TRY
-- Because schema errors are checked during compile, they cannot be caught using TRY, this can be done by wrapping the query in sp_executesql
DECLARE #QueryText NVARCHAR(100) = 'DELETE ##VTT_CONTEXT_INFO_USER_TASK WHERE 0 = 1'
EXEC sp_executesql #QueryText
END TRY
BEGIN CATCH
-- nothing to do here (see comment above)
END CATCH
IF OBJECT_ID('tempdb..##VTT_CONTEXT_INFO_USER_TASK') IS NULL
CREATE TABLE ##VTT_CONTEXT_INFO_USER_TASK (
session_id smallint,
login_time datetime,
HstryUserName VDT_USERNAME,
HstryTaskName VDT_TASKNAME,
)
MERGE ##VTT_CONTEXT_INFO_USER_TASK As target
USING (SELECT ##SPID, #HstryUserName, #HstryTaskName) as source (session_id, HstryUserName, HstryTaskName)
ON (target.session_id = source.session_id)
WHEN MATCHED THEN
UPDATE SET HstryUserName = source.HstryUserName, HstryTaskName = source.HstryTaskName
WHEN NOT MATCHED THEN
INSERT VALUES (##SPID, #LoginTime, source.HstryUserName, source.HstryTaskName);
Although I would call it a "use it at your own risk" solution, it does prevent that the use of the global temp table in other sessions affects its use in the current one, which was the concern that made me start this thread.
Thanks all for your time! (from text formatting edits to replies)

Related

Way to Capture result set from instead of insert trigger in SQL Server 2016

I have a table that utilizes an instead of insert trigger. The trigger manipulates the data for each inserted row before inserting it. Because the inserted table is not modifiable, it was implemented using a temp (#) table. At the end of the trigger a select from the temp table is done to return the inserted data to the calling client. When I do a an insert in SSMS, I can see the data that is returned and the columns all have names and values. My Trigger looks like this:
Create TRIGGER [dbo].[RealTableInsteadOfInsert] on [dbo].[RealTable]
INSTEAD OF INSERT
AS
BEGIN
set nocount on;
declare #lastDigit int;
if exists (select * from inserted)
Begin
Select *
into #tempInserted
from inserted
.... Logic to add and manipulate column data .....
INSERT INTO RealTAble(id, col1, col2, col3,....)
Select *
from #tempInserted;
Select id as insertId, *
from #tempInserted;
END
End
My question is how can I capture the output of the instead of trigger into a table for further processing using the returned data. I can't use an output clause on the insert statement as the temp table no longer exists and the data that was calculated/modified on the insert was not done in the inserted table. Is my only option to update the trigger to use a global temp table instead of a local?
You have two main options:
Send data through service broker. This is complicated and potentially slow. On the plus side, it does let you do your further processing work decoupled from the original transaction, which is nice in certain use cases.
Write the data to a real table. This can also give you transactional decoupling, but you don't get automatic activation of the decoupled processing logic if you want that.
OK, but if you write to a real table, and a lot of processes are causing the trigger to fire, how do you find "your" data? Putting aside the fact that a global temp table has the same problem unless you want to get dynamic, one solution is to use a process keyed table, as described by Erland Sommarskog in his data sharing article.
Returning data from triggers (ie, to a client) is a bad idea. The ability to do this at all will soon be removed I know you don't need to do this in your solution, just giving you a heads up.

Drop or not drop temporary tables in stored procedures

I saw this question quite a many times but I couldn't get the answer that would satisfy me. Basically what people and books say is "Although temporary tables are deleted when they go out of scope, you should explicitly delete them when they are no longer needed to reduce resource requirements on the server".
It is quite clear to me that when you are working in management studio and creating tables, then until you close your window or disconnect, you will use some resources for that table and it is logically that it is better to drop them.
But when you work with procedure then if you would like to cleanup tables most probably you will do that at the really end of it (I am not talking about the situation when you drop the table as soon as you really do not need that in the procedure). So the workflow is something like that :
When you drop in SP:
Start of SP execution
Doing some stuff
Drop tables
End of execution
And as far as I understand how can it possibly work when you do not drop:
Start of SP execution
Doing some stuff
End of execution
Drop tables
What's the difference here? I can only imagine that some resources are needed to identify the temporary tables. Any other thoughts?
UPDATE:
I ran simple test with 2 SP:
create procedure test as
begin
create table #temp (a int)
insert into #temp values (1);
drop table #temp;
end
and another one without drop statements. I've enabled user statistics and ran the tests:
declare #i int = 0;
while #i < 10000
begin
exec test;
SET #i= #i + 1;
end
That's what I've got (Trial 1-3 dropping table in SP, 4-6 do not dropping)
As the picture shows that all stats are the same or decreased a bit when I do not drop temporary table.
UPDATE2:
I ran this test 2nd time but now with 100k calls and also added SET NOCOUNT ON. These are the results:
As the 2nd run confirmed that if you do not drop the table in SP then you actually save some user time as this is done by some other internal process but outside of the user time.
You can read more about in in this Paul White's article: Temporary Tables in Stored Procedures
CREATE and DROP, Don’t
I will talk about this in much more detail in my next post, but the
key point is that CREATE TABLE and DROP TABLE do not create and drop
temporary tables in a stored procedure, if the temporary object can be
cached. The temporary object is renamed to an internal form when DROP
TABLE is executed, and renamed back to the same user-visible name when
CREATE TABLE is encountered on the next execution. In addition, any
statistics that were auto-created on the temporary table are also
cached. This means that statistics from a previous execution remain
when the procedure is next called.
Technically, a locally scoped temp table (one with a single hashtag before it) will automatically drop out of scope after your SPID is closed. There are some very odd cases where you get a temp table definition cached somewhere and then no real way to remove it. Usually that happens when you have a stored procedure call which is nested and contains a temp table by the same name.
It's good habit to get into dropping your tables when you're done with them but unless something unexpected happens, they should be de-scoped anyway once the proc finishes.

Concurrency problems with temp tables in consequent batches?

I have sometimes a problem when running a script. I have the probelm when using an application (that I didn't write and therefore cannot debug) that launches the scripts. This app isn't returning the full error from SQL Server, but just the error description, so I don't know exactly where th error comes.
I have the error only using this tool (it is a tool that sends the queries directly to SQL Server, using a DAC component), if I run the query manuallyin management studio I don't have the error. (This error moreover occurs only on a particular database).
My query is something like:
SELECT * INTO #TEMP_TABLE
FROM ANOTHER_TABLE
GO
--some other commands here
GO
INSERT INTO SOME_OTHER_TABLE(FIELD1,FIELD2)
SELECT FIELDA, FIELDB
FROM #TEMP_TABLE
GO
DROP TABLE #TEMP_TABLE
GO
The error I get is #TEMP_TABLE is not a valid object
So somehow i suspect that the DROP statement is executed before the INSERT statement.
But AFAIK when a GO is there the next statement is not executed until the previous has been completed.
Now I suspoect that this is not true with temp tables... Or do you have another ideas?
Your problem is most likely caused by either an end of session prior to the DROP TABLE causing SQL Server to automatically drop the table or the DROP TABLE is being executed in a different session than the other code (that created and used the temporary table) causing the table not to be visible.
I am assuming that stored procedures are not involved here, because it looks like you are just executing batches, since local temporary tables are also dropped when a stored proc is exited.
There is a good description of local temporary table behavior in this article on Temporary Tables in SQL Server:
You get housekeeping with Local Temporary tables; they are
automatically dropped when they go out of scope, unless explicitly
dropped by using DROP TABLE. Their scope is more generous than a table
Variable so you don't have problems referencing them within batches or
in dynamic SQL. Local temporary tables are dropped automatically at
the end of the current session or procedure. Dropping it at the end of
the procedure that created it can cause head-scratching: a local
temporary table that is created within a stored procedure or session
is dropped when it is finished so it cannot be referenced by the
process that called the stored procedure that created the table. It
can, however, be referenced by any nested stored procedures executed
by the stored procedure that created the table. If the nested
procedure references a temporary table and two temporary tables with
the same name exist at that time, which table is the query is resolved
against?
I would start up SQL Profiler and verify if your tool uses one connection to execute all batches, or if it disconnects/reconnects. Also it could be using a connection pool.
Anyway, executing SQL batches from a file is so simple that you might develop your own tool very quickly and be better off.

Normal table or global temp table?

Me and another developer are discussing which type of table would be more appropriate for our task. It's basically going to be a cache that we're going to truncate at the end of the day. Personally, I don't see any reason to use anything other than a normal table for this, but he wants to use a global temp table.
Are there any advantages to one or the other?
Use a normal table in tempdb if this is just transient data that you can afford to lose on service restart or a user database if the data is not that transient.
tempdb is slightly more efficient in terms of logging requirements.
Global temp tables get dropped once all referencing connections are the connection that created the table is closed.
Edit: Following #cyberkiwi's edit. BOL does definitely explicitly say
Global temporary tables are visible to
any user and any connection after they
are created, and are deleted when all
users that are referencing the table
disconnect from the instance of SQL
Server.
In my test I wasn't able to get this behaviour though either.
Connection 1
CREATE TABLE ##T (i int)
INSERT INTO ##T values (1)
SET CONTEXT_INFO 0x01
Connection 2
INSERT INTO ##T VALUES(4)
WAITFOR DELAY '00:01'
INSERT INTO ##T VALUES(5)
Connection 3
SELECT OBJECT_ID('tempdb..##T')
declare #killspid varchar(10) = (select 'kill ' + cast(spid as varchar(5)) from sysprocesses where context_info=0x01)
exec (#killspid)
SELECT OBJECT_ID('tempdb..##T') /*NULL - But 2 is still
running let alone disconnected!*/
Global temp table
-ve: As soon as the connection that created the table goes out of scope, it takes
the table with it. This is damaging if you use connection pooling which can swap connections constantly and possibly reset it
-ve: You need to keep checking to see if the table already exists (after restart) and create it if not
+ve: Simple logging in tempdb reduces I/O and CPU activity
Normal table
+ve: Normal logging keeps your cache with your main db. If your "cache" is maintained but is still mission critical, this keeps it consistent together with the db
-ve: follow from above More logging
+ve: The table is always around, and for all connections
If the cache is a something like a quick lookup summary for business/critical data, even if it is reset/truncated at the end of the day, I would prefer to keep it a normal table in the db proper.

In SQL Server 2008 ,is it possible to disable auto drop of global temp table

Question 1: I am using a global temp tables in SQL Server 2008. But once my connection is closed this temp is dropped. Is there any way to disable auto drop
Question 2: If two connections are accessing same global temp table and another connection is
trying to delete that global temp table, does SQL Server handles this synchronization properly?
You can create your global temp tables in a stored procedure and mark it with the startup option.
SQL Server maintains a reference count greater than zero for all global temporary tables created within startup procedures.
some example code
CREATE PROC dbo.CreateGlobalTempTables
AS
CREATE TABLE ##my_temp_table
(
fld1 INT NOT NULL PRIMARY KEY,
fld2 INT NULL
);
GO
EXEC dbo.sp_procoption 'dbo.CreateGlobalTempTables', 'startup', 'true';
The global temporary table will be created automatically at startup and persist until someone explicitly drops it.
If you need a table to persist beyond the death of the connection that created it, you should just create a regular table instead of a temporary one. It can still be created in tempdb directly (geting you the benfits of simple logging and auto destruction on server restart) but it's name wouldn't be prefixed by ##.
DROP TABLE is a transactional statement that will block if there are any active connections using that table (with locks).
When the connection that created the ##GlobalTempTable ends, the table will be dropped, unless there is a lock against it.
You could run something like this from the other process to keep the table from being dropped:
BEGIN TRANSACTION
SELECT TOP 1 FROM ##GlobalTempTable WITH (UPDLOCK, HOLDLOCK)
...COMMIT/ROLLBACK
However, when the transaction ends, the table will be dropped. If you can't use a transaction like this, then you should use a permanent table using the Process-Keyed Table method.

Resources