Ok - so the answer is probably 'everything is within a transaction in SQLServer'. But here's my problem.
I have a stored procedure which returns data based on one input parameter. This is being called from an external service which I believe is Java-based.
I have a log-table to see how long each hit to this proc is taking. Kind of like this (simplified)...
TABLE log (ID INT PRIMARY KEY IDENTITY(1,1), Param VARCHAR(10), In DATETIME, Out DATETIME)
PROC troublesome
(#param VARCHAR(10))
BEGIN
--log the 'in' time and keep the ID
DECLARE #logid INT
INSERT log (In,Param) VALUES (GET_DATE(),#param)
SET #logid = SCOPE_IDENTITY()
SELECT <some stuff from another table based on #param>
--set the 'out' time
UPDATE log SET Out = GET_DATE() WHERE ID = #logid
END
So far so easily-criticised by SQL fascists.
Anyway - When I call this eg troublesome 'SOMEPARAM' - I get the result of the SELECT back and in the log table is a nice new row with the SOMEPARAM and the in and out timestamps.
Now - I watch this table, and even though no rows are going in - if I am to generate a row, I will see that the ID has skipped many places. I guess this is being caused by the external client code hitting it. They are getting the result of the SELECT - but I am not getting their log data.
This suggests to me they are wrapping their client call in a TRAN and rolling it back. Which is one thing that would cause this behaviour. I want to know...
If there is a way I can FORCE the write of the log even if it is contained within a transaction over which I have no control and which is subsequently rolled back (seems unlikely)
If there is a way I can determine from within this proc if it is executing within a transaction (and perhaps raise an error)
If there are other possible explanations for the ID skipping (and it's skipping alot like 1000 places in an hour. So I'm sure it's caused by the client code - as they are reporting successfully retrieving the results of the SELECT also.)
Thanks!
For the why rows are skipped part of the question, your assumption sounds plausible to me.
For how to force the write even within a transaction, I don't know.
As for how to discover whether the SP is running inside a transaction how about the following:
SELECT ##trancount
I tested this out in an sql batch and seems to be ok. Maybe it can take you a bit further.
In MS SQL Server, I'm using a global temp table to store session related information passed by the client and then I use that information inside triggers.
Since the same global temp table can be used in different sessions and it may or may not exist when I want to write into it (depending on whether all the previous sessions which used it before are closed), I'm doing a check for the global temp table existence based on which I create before I write into it.
IF OBJECT_ID('tempdb..##VTT_CONTEXT_INFO_USER_TASK') IS NULL
CREATE TABLE ##VTT_CONTEXT_INFO_USER_TASK (
session_id smallint,
login_time datetime,
HstryUserName VDT_USERNAME,
HstryTaskName VDT_TASKNAME,
)
MERGE ##VTT_CONTEXT_INFO_USER_TASK As target
USING (SELECT ##SPID, #HstryUserName, #HstryTaskName) as source (session_id, HstryUserName, HstryTaskName)
ON (target.session_id = source.session_id)
WHEN MATCHED THEN
UPDATE SET HstryUserName = source.HstryUserName, HstryTaskName = source.HstryTaskName
WHEN NOT MATCHED THEN
INSERT VALUES (##SPID, #LoginTime, source.HstryUserName, source.HstryTaskName);
The problem is that between my check for the table existence and the MERGE statement, SQL Server may drop the temp table if all the sessions which were using it before happen to close in that exact instance (this actually happened in my tests).
Is there a best practice on how to avoid this kind of concurrency issues, that a table is not dropped between the check for its existence and its subsequent use?
The notion of "global temporary table" and "trigger" just do not click. Tables are permanent data stores, as are their attributes -- including triggers. Temporary tables are dropped when the server is re-started. Why would anyone design a system where a permanent block of code (trigger) depends on a temporary shared storage mechanism? It seems like a recipe for failure.
Instead of a global temporary table, use a real table. If you like, put a helpful prefix such as temp_ in front of the name. If the table is being shared by databases, then put it in a database where all code has access.
Create the table once and leave it there (deleting the rows is fine) so the trigger code can access it.
I'll start by saying that, on the long term, I will follow Gordon's advice, i.e. I will take the necessary steps to introduce a normal table in the database to store client application information which needs to be accessible in the triggers.
But since this was not really possible now because of time constrains (it takes weeks to get the necessary formal approvals for a new normal table), I came up with a solution for preventing SQL Server from dropping the global temp table between the check for its existence and the MERGE statement.
There is some information out there about when a global temp table is dropped by SQL Server; my personal tests showed that SQL Server drops a global temp table the moment the session which created it is closed and any other transactions started in other sessions which changed data in that table are finished.
My solution was to fake data changes on the global temp table even before I check for its existence. If the table exists at that moment, SQL Server will then know that it needs to keep it until the current transaction finishes, and it cannot be dropped anymore after the check for its existence. The code looks now like this (properly commented, since it is kind of a hack):
-- Faking a delete on the table ensures that SQL Server will keep the table until the end of the transaction
-- Since ##VTT_CONTEXT_INFO_USER_TASK may actually not exist, we need to fake the delete inside TRY .. CATCH
-- FUTURE 2016, Feb 03: A cleaner solution would use a real table instead of a global temp table.
BEGIN TRY
-- Because schema errors are checked during compile, they cannot be caught using TRY, this can be done by wrapping the query in sp_executesql
DECLARE #QueryText NVARCHAR(100) = 'DELETE ##VTT_CONTEXT_INFO_USER_TASK WHERE 0 = 1'
EXEC sp_executesql #QueryText
END TRY
BEGIN CATCH
-- nothing to do here (see comment above)
END CATCH
IF OBJECT_ID('tempdb..##VTT_CONTEXT_INFO_USER_TASK') IS NULL
CREATE TABLE ##VTT_CONTEXT_INFO_USER_TASK (
session_id smallint,
login_time datetime,
HstryUserName VDT_USERNAME,
HstryTaskName VDT_TASKNAME,
)
MERGE ##VTT_CONTEXT_INFO_USER_TASK As target
USING (SELECT ##SPID, #HstryUserName, #HstryTaskName) as source (session_id, HstryUserName, HstryTaskName)
ON (target.session_id = source.session_id)
WHEN MATCHED THEN
UPDATE SET HstryUserName = source.HstryUserName, HstryTaskName = source.HstryTaskName
WHEN NOT MATCHED THEN
INSERT VALUES (##SPID, #LoginTime, source.HstryUserName, source.HstryTaskName);
Although I would call it a "use it at your own risk" solution, it does prevent that the use of the global temp table in other sessions affects its use in the current one, which was the concern that made me start this thread.
Thanks all for your time! (from text formatting edits to replies)
I have been searching around but I cannot find the correct answer, probably I search wrong because I don't know what to look for :)
Anyway, I have a TSQL with a begin and commit transaction. In the transaction I add some columns and also rename some columns.
Just after the renames and added column statement i also run some update statements to load data into the newly created columns.
Now the problem is that for some reason the update gives an error that it cannot update the given column as it does not exist (YET???).
My idea is that the statement is still working out the rename and the adding of the columns but already goes ahead with the update statements. The table is very big and has a few million records so I can imagine it takes some time to add and rename the columns
If I run first the rename and add statements and than separate the update statements, it does work. So it has to do with some wait time.
Is it possible to make sql force to execute step by step and wait until the complete statement is done before going to the next?
If you modify columns (e.g. add them), you have to finish the batch before you can continue with updating them. Insert the GO keyword between table structure changes and updates.
To illustrate that, the following code won't work:
create table sometable(col1 int)
go
alter table sometable add col2 varchar(10)
insert into sometable(col2) values ('a')
But inserting go will make the insert recognise the new column
create table sometable(col1 int)
go
alter table sometable add col2 varchar(10)
go
insert into sometable(col2) values ('a')
If you do it in the code, you may want to create separate transaction for the structure changes and data migration. You can still wrap them it in one transaction for data integrity.
It doesn't have anything to do with wait time. The queries are run in order. It's because all the queries are submitted all at once and therefore when it tries to validate your update, the column doesn't exist at that point in time. To get around it, you need to send the update in a separate batch. The following keyword needs to be added between your alter and update statement
GO
You can try using select for update,
http://docs.oracle.com/cd/B28359_01/server.111/b28286/statements_10002.htm#i2130052
This will ensure that your query will wait for lock, bit it is recommended to Specify WAIT to instruct the database to wait integer seconds so that it will not wait for indefinate time.
I have a website which is used by all branches of a store and what it does is that it records customer purchases into a table called myTransactions.myTransactions table has a column named SerialNumber. For each purchase I create a record in the transactions table and assign a serial to it. The stored procedure that does this calls a UDF function to get a new serialNumber before inserting the record. Like below :
Create Procedure mytransaction_Insert
as begin
insert into myTransactions(column1,column2,column3,...SerialNumber)
values( Value1 ,Value2,Value3,...., getTransactionNSerialNumber())
end
Create function getTransactionNSerialNumber
as
begin
RETURN isnull(SELECT TOP (1) SerialNumber FROM myTransactions READUNCOMMITTED
ORDER BY SerialNumber DESC),0) + 1
end
The website is being used by so many users in different stores at the same time and it is creating many duplicate serialNumbers(same SerialNumbers). So I added a Sql transaction with ReadCommitted level to the transaction and I still got duplicate transaction numbers. I changed it to SERIALIZABLE in order to lock the resources and I not only got duplicate transaction numbers(!!HOW!!) but I also got sporadic deadlocks between the same stored procedure calls. This is what I tried : (With omissions of try catch blocks and rollbacks)
Create Procedure mytransaction_Insert
as begin
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE
BEGIN TRASNACTION ins
insert into myTransactions(column1,column2,column3,...SerialNumber)
values( Value1 ,Value2 , Value3, ...., getTransactionNSerialNumber())
COMMIT TRANSACTION ins
SET TRANSACTION ISOLATION READCOMMITTED
end
I even copied the function that gets the serial number directly into the stored procedure instead of the UDF function call and still got duplicate serialNumbers.So,How can a stored procedure line create something Like the c# lock() {} block.
By the way, I have to implement the transaction serial number using the same pattern and I can't change the serialNumber to any other identity field or whatever.And for some reasons I need to generate the serialNumber inside the database and I can't move SerialNumber generation to application level.
Sorry but I have already tried this without READUNCOMMITTED in the function and I still get duplicate SerialNumbers.
As for the IDENTITY column, I should say that this app is going to be used by other companies that require different SerialNumbers and we can't just simply change it to identity.
You have READUNCOMMITTED in the UDF. This will cause it to ignore any exclusive locks held by other transactions.
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE is not the same as an applock, it is not a "critical section" in the database, it just controls the locking behaviour of subsequent statements in the transaction.
Take out the READUNCOMMITTED and it should start working as you expect.
Of course, this ignores the fact that you've essentially re-implemented an IDENTITY column. If your serial numbers are really incremental then you should throw all of this away and replace it with a simple IDENTITY column. You claim that you "can't", but don't provide any justification for that statement; it looks to me like you almost certainly can.
What you are missing is a unique constraint (or primary key) down your transactions table. If you had that the duplicate entry would back out when you attempt to commit it.
But I would state clearly that you should use the "Identity" (as said by #Aaronaught) column in SQL. This will start at whatever you want it to and increment forward or backward. If you need your orders to start at a given number then forward it. But if you need an identifier that is unique and also happens to be an integer value then use identity.
Consider this trigger:
ALTER TRIGGER myTrigger
ON someTable
AFTER INSERT
AS BEGIN
DELETE FROM someTable
WHERE ISNUMERIC(someField) = 1
END
I've got a table, someTable, and I'm trying to prevent people from inserting bad records. For the purpose of this question, a bad record has a field "someField" that is all numeric.
Of course, the right way to do this is NOT with a trigger, but I don't control the source code... just the SQL database. So I can't really prevent the insertion of the bad row, but I can delete it right away, which is good enough for my needs.
The trigger works, with one problem... when it fires, it never seems to delete the just-inserted bad record... it deletes any OLD bad records, but it doesn't delete the just-inserted bad record. So there's often one bad record floating around that isn't deleted until somebody else comes along and does another INSERT.
Is this a problem in my understanding of triggers? Are newly-inserted rows not yet committed while the trigger is running?
Triggers cannot modify the changed data (Inserted or Deleted) otherwise you could get infinite recursion as the changes invoked the trigger again. One option would be for the trigger to roll back the transaction.
Edit: The reason for this is that the standard for SQL is that inserted and deleted rows cannot be modified by the trigger. The underlying reason for is that the modifications could cause infinite recursion. In the general case, this evaluation could involve multiple triggers in a mutually recursive cascade. Having a system intelligently decide whether to allow such updates is computationally intractable, essentially a variation on the halting problem.
The accepted solution to this is not to permit the trigger to alter the changing data, although it can roll back the transaction.
create table Foo (
FooID int
,SomeField varchar (10)
)
go
create trigger FooInsert
on Foo after insert as
begin
delete inserted
where isnumeric (SomeField) = 1
end
go
Msg 286, Level 16, State 1, Procedure FooInsert, Line 5
The logical tables INSERTED and DELETED cannot be updated.
Something like this will roll back the transaction.
create table Foo (
FooID int
,SomeField varchar (10)
)
go
create trigger FooInsert
on Foo for insert as
if exists (
select 1
from inserted
where isnumeric (SomeField) = 1) begin
rollback transaction
end
go
insert Foo values (1, '1')
Msg 3609, Level 16, State 1, Line 1
The transaction ended in the trigger. The batch has been aborted.
You can reverse the logic. Instead of deleting an invalid row after it has been inserted, write an INSTEAD OF trigger to insert only if you verify the row is valid.
CREATE TRIGGER mytrigger ON sometable
INSTEAD OF INSERT
AS BEGIN
DECLARE #isnum TINYINT;
SELECT #isnum = ISNUMERIC(somefield) FROM inserted;
IF (#isnum = 1)
INSERT INTO sometable SELECT * FROM inserted;
ELSE
RAISERROR('somefield must be numeric', 16, 1)
WITH SETERROR;
END
If your application doesn't want to handle errors (as Joel says is the case in his app), then don't RAISERROR. Just make the trigger silently not do an insert that isn't valid.
I ran this on SQL Server Express 2005 and it works. Note that INSTEAD OF triggers do not cause recursion if you insert into the same table for which the trigger is defined.
Here's my modified version of Bill's code:
CREATE TRIGGER mytrigger ON sometable
INSTEAD OF INSERT
AS BEGIN
INSERT INTO sometable SELECT * FROM inserted WHERE ISNUMERIC(somefield) = 1 FROM inserted;
INSERT INTO sometableRejects SELECT * FROM inserted WHERE ISNUMERIC(somefield) = 0 FROM inserted;
END
This lets the insert always succeed, and any bogus records get thrown into your sometableRejects where you can handle them later. It's important to make your rejects table use nvarchar fields for everything - not ints, tinyints, etc - because if they're getting rejected, it's because the data isn't what you expected it to be.
This also solves the multiple-record insert problem, which will cause Bill's trigger to fail. If you insert ten records simultaneously (like if you do a select-insert-into) and just one of them is bogus, Bill's trigger would have flagged all of them as bad. This handles any number of good and bad records.
I used this trick on a data warehousing project where the inserting application had no idea whether the business logic was any good, and we did the business logic in triggers instead. Truly nasty for performance, but if you can't let the insert fail, it does work.
I think you can use CHECK constraint - it is exactly what it was invented for.
ALTER TABLE someTable
ADD CONSTRAINT someField_check CHECK (ISNUMERIC(someField) = 1) ;
My previous answer (also right by may be a bit overkill):
I think the right way is to use INSTEAD OF trigger to prevent the wrong data from being inserted (rather than deleting it post-factum)
UPDATE: DELETE from a trigger works on both MSSql 7 and MSSql 2008.
I'm no relational guru, nor a SQL standards wonk. However - contrary to the accepted answer - MSSQL deals just fine with both recursive and nested trigger evaluation. I don't know about other RDBMSs.
The relevant options are 'recursive triggers' and 'nested triggers'. Nested triggers are limited to 32 levels, and default to 1. Recursive triggers are off by default, and there's no talk of a limit - but frankly, I've never turned them on, so I don't know what happens with the inevitable stack overflow. I suspect MSSQL would just kill your spid (or there is a recursive limit).
Of course, that just shows that the accepted answer has the wrong reason, not that it's incorrect. However, prior to INSTEAD OF triggers, I recall writing ON INSERT triggers that would merrily UPDATE the just inserted rows. This all worked fine, and as expected.
A quick test of DELETEing the just inserted row also works:
CREATE TABLE Test ( Id int IDENTITY(1,1), Column1 varchar(10) )
GO
CREATE TRIGGER trTest ON Test
FOR INSERT
AS
SET NOCOUNT ON
DELETE FROM Test WHERE Column1 = 'ABCDEF'
GO
INSERT INTO Test (Column1) VALUES ('ABCDEF')
--SCOPE_IDENTITY() should be the same, but doesn't exist in SQL 7
PRINT ##IDENTITY --Will print 1. Run it again, and it'll print 2, 3, etc.
GO
SELECT * FROM Test --No rows
GO
You have something else going on here.
From the CREATE TRIGGER documentation:
deleted and inserted are logical (conceptual) tables. They are
structurally similar to the table on
which the trigger is defined, that is,
the table on which the user action is
attempted, and hold the old values or
new values of the rows that may be
changed by the user action. For
example, to retrieve all values in the
deleted table, use: SELECT * FROM deleted
So that at least gives you a way of seeing the new data.
I can't see anything in the docs which specifies that you won't see the inserted data when querying the normal table though...
I found this reference:
create trigger myTrigger
on SomeTable
for insert
as
if (select count(*)
from SomeTable, inserted
where IsNumeric(SomeField) = 1) <> 0
/* Cancel the insert and print a message.*/
begin
rollback transaction
print "You can't do that!"
end
/* Otherwise, allow it. */
else
print "Added successfully."
I haven't tested it, but logically it looks like it should dp what you're after...rather than deleting the inserted data, prevent the insertion completely, thus not requiring you to have to undo the insert. It should perform better and should therefore ultimately handle a higher load with more ease.
Edit: Of course, there is the potential that if the insert happened inside of an otherwise valid transaction that the wole transaction could be rolled back so you would need to take that scenario into account and determine if the insertion of an invalid data row would constitute a completely invalid transaction...
Is it possible the INSERT is valid, but that a separate UPDATE is done afterwards that is invalid but wouldn't fire the trigger?
The techniques outlined above describe your options pretty well. But what are the users seeing? I can't imagine how a basic conflict like this between you and whoever is responsible for the software can't end up in confusion and antagonism with the users.
I'd do everything I could to find some other way out of the impasse - because other people could easily see any change you make as escalating the problem.
EDIT:
I'll score my first "undelete" and admit to posting the above when this question first appeared. I of course chickened out when I saw that it was from JOEL SPOLSKY. But it looks like it landed somewhere near. Don't need votes, but I'll put it on the record.
IME, triggers are so seldom the right answer for anything other than fine-grained integrity constraints outside the realm of business rules.
MS-SQL has a setting to prevent recursive trigger firing. This is confirgured via the sp_configure stored proceedure, where you can turn recursive or nested triggers on or off.
In this case, it would be possible, if you turn off recursive triggers to link the record from the inserted table via the primary key, and make changes to the record.
In the specific case in the question, it is not really a problem, because the result is to delete the record, which won't refire this particular trigger, but in general that could be a valid approach. We implemented optimistic concurrency this way.
The code for your trigger that could be used in this way would be:
ALTER TRIGGER myTrigger
ON someTable
AFTER INSERT
AS BEGIN
DELETE FROM someTable
INNER JOIN inserted on inserted.primarykey = someTable.primarykey
WHERE ISNUMERIC(inserted.someField) = 1
END
Your "trigger" is doing something that a "trigger" is not suppose to be doing. You can simple have your Sql Server Agent run
DELETE FROM someTable
WHERE ISNUMERIC(someField) = 1
every 1 second or so. While you're at it, how about writing a nice little SP to stop the programming folk from inserting errors into your table. One good thing about SP's is that the parameters are type safe.
I stumbled across this question looking for details on the sequence of events during an insert statement & trigger. I ended up coding some brief tests to confirm how SQL 2016 (EXPRESS) behaves - and thought it would be appropriate to share as it might help others searching for similar information.
Based on my test, it is possible to select data from the "inserted" table and use that to update the inserted data itself. And, of interest to me, the inserted data is not visible to other queries until the trigger completes at which point the final result is visible (at least best as I could test). I didn't test this for recursive triggers, etc. (I would expect the nested trigger would have full visibility of the inserted data in the table, but that's just a guess).
For example - assuming we have the table "table" with an integer field "field" and primary key field "pk" and the following code in our insert trigger:
select #value=field,#pk=pk from inserted
update table set field=#value+1 where pk=#pk
waitfor delay '00:00:15'
We insert a row with the value 1 for "field", then the row will end up with the value 2. Furthermore - if I open another window in SSMS and try:
select * from table where pk = #pk
where #pk is the primary key I originally inserted, the query will be empty until the 15 seconds expire and will then show the updated value (field=2).
I was interested in what data is visible to other queries while the trigger is executing (apparently no new data). I tested with an added delete as well:
select #value=field,#pk=pk from inserted
update table set field=#value+1 where pk=#pk
delete from table where pk=#pk
waitfor delay '00:00:15'
Again, the insert took 15sec to execute. A query executing in a different session showed no new data - during or after execution of the insert + trigger (although I would expect any identity would increment even if no data appears to be inserted).