I have 3 stored procedures (simplified, please try to ignore why I'm updating the table twice and why the SP is called twice):
CREATE SP1 AS
BEGIN TRANSACTION
-- Updated twice
UPDATE Customers SET Name = 'something' Where Id = 1 OUTPUT INSERTED.*
UPDATE Customers SET Name = 'something'
COMMIT TRANSACTION;
END
CREATE SP2 AS
BEGIN TRANSACTION
UPDATE Customers SET Name = 'anothername'
COMMIT TRANSACTION;
END
CREATE SP3 AS
BEGIN TRANSACTION
-- Called twice
EXEC SP2
EXEC SP2
COMMIT TRANSACTION;
END
The problem is that I got a deadlock from sql server. It says that SP1 and SP3 are both waiting for the Customers table resource. Does it make sense? Could it be because of the inner transaction in SP2? or maybe the use of OUTPUT statement...?
The lock is a Key lock on the PK of Customers. The requested lock mode of each waiting SP is U and the owner is X (The other object i guess).
A few more details:
1. These are called from the same user multiple times on different processes.
2. The statements are called twice only for the sake of the example.
3. In my actual code, Customer is actualy called 'Pending Instructions'. The instructions table is sampled every minute by each listener (computer, actualy).
4. The first update query first gets all the pending instructions and the second one updates the status of the entire table to completed, just to make sure that none are left in pending mode.
5. SP3 is calling SP2 twice because it updates 2 proprietory instructions row, this happens once a day.
Thanks a lot!!
Why are you surprised by this? You have written the book case for a deadlock and hit it.
The first update query first gets all the pending instructions and the second one updates the status of the entire table to completed.
Yes, this will deadlock. Two concurrent calls will find different 'pending' instructions (as new 'pending' instructions can be inserted in between). Then they will proceed to attempt to update the entire table and block on each other, deadlock. Here is the timeline:
Table contains customer:1, pending
T1 (running first update of SP1) updates table and modifies customer:1
T2 inserts a new record, customer:2, pending
T3 (running first update of SP1) updates table and modifies customer:2
T1 (running second update of SP1) tries to update all table, is blocked by T3
T3 (running second update of SP1) tries to update all table, is blocked by T1. Deadlock.
I have good news though: the deadlock is the best outcome you can get. A far worse outcome is when your logic missed 'pending' customers (which will happen more often). simply stated, your SP1 will erroneously mark any new 'pending' customer inserted after the first update as 'processed', when it was actually just skipped. Here is the timeline:
Table contains customer:1, pending
T1 (running first update of SP1) updates table and modifies customer:1
T2 inserts a new record, customer:2, pending
T1 (running second update of SP1) tries updates the whole table. customer:2 was pending and is reset w/o actually had been processed (is not i SP1's result set).
Your business lost an update.
so I suggest to go back to the drawing board and design SP1 properly. I suggest SP1 should only update on the second statement what it had updated on on the first one, for instance. Posting real code, with proper DDL, would go along way toward getting a useful solution.
Related
For testing, I am trying to simulate a condition in which a query from our web application to our SQL Server backend would timeout. The web application is configured so this happens if the query runs longer than 30 seconds. I felt the easiest way to do this would be to take and hold an exclusive lock on the the table that the web application wants to query. As I understand it, an exclusive lock should prevent any additional locks (even the shared locks taken by a SELECT statement).
I used the following methodology:
CREATE A LONG-HELD LOCK
Open a first query window in SSMS and run
BEGIN TRAN;
SELECT * FROM MyTable WITH (TABLOCKX);
WAITFOR DELAY '00:02:00';
ROLLBACK;
(see https://stackoverflow.com/a/25274225/2824445 )
CONFIRM THE LOCK
I can EXEC sp_lock and see results with ObjId matching MyTable, Type of TAB, Mode of X
TRY TO GET BLOCKED BY THE LOCK
Open a second query window in SSMS and run SELECT * FROM MyTable
I would expect this to sit and wait, not returning any results until after the lock is released by the first query. Instead, the second query returns with full results immediately.
STUFF I TRIED
In the second query window, if I SET TRANSACTION ISOLATION LEVEL SERIALIZABLE, then the second query waits until the first completes as expected. However, the point is to simulate a timeout in our web application, and I do not have any easy way to alter the transaction isolation level of the web application's connections away from the default of READ COMMITTED.
In the first window, I tried modifying the table's values inside the transaction. In this case, when the second query returns immediately, the values it shows are the unmodified values.
Figured it out. We had READ_COMMITTED_SNAPSHOT turned on, which is how the second query was able to return the previous, unmodified values in part 2 of "Stuff I tried". I was able to determine this with SELECT is_read_committed_snapshot_on FROM sys.databases WHERE name = 'MyDatabase'. Once it was turned off with ALTER DATABASE MyDatabase SET READ_COMMITTED_SNAPSHOT OFF, I began to see the expected behavior in which the second query would wait for the first to complete.
We have an ETL pipeline that runs for each CSV uploaded into an storage account (Azure). It runs some transformations on the CSV and writes the outputs to another location, also as CSV, and calls a stored procedure on the database (SQL Azure) which ingests (BULK INSERT) this resulting CSV into a staging table.
This pipeline can have concurrent executions as multiple resources can be uploading files to the storage. Hence, the staging table is getting data inserted pretty often.
Then, we have an scheduled SQL job (Elastic Job) that triggers an SP that moves the data from the staging table into the final table.
At this point, we would want to truncate/empty the staging table so that we do not re-insert them in the next execution of the job.
Problem is, we cannot be sure that between the load from the staging table to the final table and the truncate command, there has not been any new data written into the staging table that could be truncated without first being inserted in to the final table.
Is there a way to lock the staging table while we're copying the data into the final table so that the SP (called from the ETL pipeline) trying to write to it will just wait until the lock is release? Is this achievable by using transactions or maybe some manual lock commands?
If not, what's the best approach to handle this?
I would propose solution with two identical staging tables. Lets name them StageLoading and StageProcessing.
Load process would have following steps:
1. At the beginning both tables are empty.
2. We load some data into StageLoading table (I assume each load is a transaction).
3. When Elastic job starts it will do:
- ALTER TABLE SWITCH to move all data from StageLoading to StageProcessing. It will make StageLoading empty and ready for next loads. It is a metadata operation, so takes miliseconds and it is fully blocking, so will be done between loads.
- load the data from StageProcessing to final tables.
- truncate table StageProcessing.
4. Now we are ready for next Elastic job.
If we try to do SWITCH when StageProcessing is not empty, ALTER will fail and it will mean that last load process failed.
I like the sp_getapplock and use this method myself in few places for its flexibility and that you have full control over the locking logic and wait times.
The only problem that I see is that in your case concurrent processes are not all equal.
You have SP1 that moves data from the staging table into the main table. Your system never tries to run several instances of this SP.
Another SP2 that inserts data into the staging table can be run several times simultaneously and it is fine to do it.
It is easy to implement the locking that would prevent any concurrent run of any combination of SP1 or SP2. In other words, it is easy if the locking logic is the same for SP1 and SP2 and they are treated equal. But, then you can't have several instances of SP2 running simultaneously.
It is not obvious how to implement the locking that would prevent concurrent run of SP1 and SP2, while allowing several instances of SP2 to run simultaneously.
There is another approach that doesn't attempt to prevent concurrent run of SPs, but embraces and expects that simultaneous runs are possible.
One way to do it is to add an IDENTITY column to the staging table. Or an automatically populated datetime if you can guarantee that it is unique and never decreases, which can be tricky. Or rowversion column.
The logic inside SP2 that inserts data into the staging table doesn't change.
The logic inside SP1 that moves data from the staging table into the main table needs to use these identity values.
At first read the current maximum value of identity from the staging table and remember it in a variable, say, #MaxID. All subsequent SELECTs, UPDATEs and DELETEs from the staging table in that SP1 should include a filter WHERE ID <= #MaxID.
This would ensure that if there happen to be a new row added to the staging table while SP1 is running, that row would not be processed and would remain in the staging table until the next run of SP1.
The drawback of this approach is that you can't use TRUNCATE, you need to use DELETE with WHERE ID <= #MaxID.
If you are OK with several instances of SP2 waiting for each other (and SP1), then you can use sp_getapplock similar to the following. I have this code in my stored procedure. You should put this logic into both SP1 and SP2.
I'm not calling sp_releaseapplock explicitly here, because the lock owner is set to Transaction and engine will release the lock automatically when transaction ends.
You don't have to put retry logic in the stored procedure, it can be within external code that runs these stored procedures. In any case, your code should be ready to retry.
CREATE PROCEDURE SP2 -- or SP1
AS
BEGIN
SET NOCOUNT ON;
SET XACT_ABORT ON;
BEGIN TRANSACTION;
BEGIN TRY
-- Maximum number of retries
DECLARE #VarCount int = 10;
WHILE (#VarCount > 0)
BEGIN
SET #VarCount = #VarCount - 1;
DECLARE #VarLockResult int;
EXEC #VarLockResult = sp_getapplock
#Resource = 'StagingTable_app_lock',
-- this resource name should be the same in SP1 and SP2
#LockMode = 'Exclusive',
#LockOwner = 'Transaction',
#LockTimeout = 60000,
-- I'd set this timeout to be about twice the time
-- you expect SP to run normally
#DbPrincipal = 'public';
IF #VarLockResult >= 0
BEGIN
-- Acquired the lock
-- for SP2
-- INSERT INTO StagingTable ...
-- for SP1
-- SELECT FROM StagingTable ...
-- TRUNCATE StagingTable ...
-- don't retry any more
BREAK;
END ELSE BEGIN
-- wait for 5 seconds and retry
WAITFOR DELAY '00:00:05';
END;
END;
COMMIT TRANSACTION;
END TRY
BEGIN CATCH
ROLLBACK TRANSACTION;
-- log error
END CATCH;
END
This code guarantees that only one procedure is working with the staging table at any given moment. There is no concurrency. All other instances will wait.
Obviously, if you try to access the staging table not through these SP1 or SP2 (which try to acquire the lock first), then such access will not be blocked.
Is there a way to lock the staging table while we're copying the data into the final table so that the SP (called from the ETL pipeline) trying to write to it will just wait until the lock is release? Is this achievable by using transactions or maybe some manual lock commands?
It looks you are searching for a mechanism that is wider than a transaction level. SQL Server/Azure SQL DB has one and it is called application lock:
sp_getapplock
Places a lock on an application resource.
Locks placed on a resource are associated with either the current transaction or the current session. Locks associated with the current transaction are released when the transaction commits or rolls back.Locks associated with the session are released when the session is logged out. When the server shuts down for any reason, all locks are released.
Locks can be explicitly released with sp_releaseapplock. When an application calls sp_getapplock multiple times for the same lock resource, sp_releaseapplock must be called the same number of times to release the lock. When a lock is opened with the Transaction lock owner, that lock is released when the transaction is committed or rolled back.
It basically means that your ETL Tool should open single session to DB, acquire the lock and release when finished. Other sessions before trying to do anything should try to acquire the lock(they cannot because it already taken), wait until when it released and continue to work.
Assuming you have a single outbound job
Add an OutboundProcessing BIT DEFAULT 0 to the table
In the job, SET OutboundProcessing = 1 WHERE OutboundProcessing = 0 (claim the rows)
For the ETL, incorporate WHERE OutboundProcessing = 1 in the query that sources the data (transfer the rows)
After the ETL, DELETE FROM TABLE WHERE OutboundProcessing = 1 (remove the rows you transferred)
If the ETL fails, SET OutboundProcessing = 0 WHERE OutboundProcessing = 1
I always prefer to "ID" each file I receive. If you can do this, you can associate the records from a given file throughout your load process. You haven't called out a need for this, but jus sayin.
However, with each file having an identity (just a int/bigint identity value should do) you can then dynamically create as many load tables as you like from a "template" load table.
When a file arrives, create a new load table named with the ID of the file.
Process your data from load to final table.
drop the load table for the file being processed.
This is somewhat similar to the other solution about using 2 tables (load and stage) but even in that solution you are still limited to having 2 files "loaded" (your still only applying one file to the final table though?)
Last, it is not clear if your "Elastic Job" is detached from the actual "load" pipeline/processing or if it is included. Being a job, I assume it is not included, if a job, you can only run a single instance at time? So its not clear why it's important to load multiple files at once if you can only move one from load to final at a time. Why the rush to get files into load?
I have a table that is pushed to me from another SQL Server. The table is dropped after it is rotated to a "Current Day" table (Current Data is rotated to prev day before this).
Currently we have jobs that are running to do the "rotating" that are set at a specific time. I had originally created a trigger but clearly a trigger won't work (as I figured out from the comments) since the DDL operation wont continue its flow until after this trigger is complete... It also looks like this is just not possible since I don't have control over the group that is pushing the data to us.
Resolution : I went to the org that pushes the data and requested they add a step that inserts a record into a TableLog table and I am doing my trigger off of that insert instead.
CREATE TRIGGER InsertTest
ON [pace].[Table_Load_Log]
after insert
AS
if exists(select Table_name from inserted where inserted.Table_name = 'POE_Task_Details_SE_TEMP')
BEGIN
--drop table dbo.newtable
exec dbo.sp_start_job N'Make Pace Tables From Temp Table Push’
END
GO
There is no way to do this with a trigger. If you need to know when a CRUD operation on a table is complete, you would need to execute a command after the CRUD operation in the same process that launches it.
I'm starting to work with SQL Server database and I'm having a hard time trying to understand Transaction Isolation Levels and how they lock data.
I'm trying to accomlish the following simple task:
Accept a pair of integers [ID, counter] in a SQL stored procedure
Determine whether ID exists in a certain table: SELCT COUNT(*) FROM MyTable WHERE Id = {idParam}
If the previous COUNT statement returns 0, insert this ID and counter:
INSERT INTO MyTable(Id, Counter) VALUES({idParam}, {counterParam})
If the COUNT statement returns 1, update the existing record: UPDATE MyTable SET Counter = Counter + {counterParam} WHERE Id = {idParam}
Now, I understand I have to wrap this whole stored procedure in a transaction, and according to this MS article the appropriate isolation level would be SERIALIZABLE (it says: No other transactions can modify data that has been read by the current transaction until the current transaction completes). Please correct me if I'm wrong here.
Suppose I called the procedure with ID=1, so the first query woluld be SELCT COUNT(*) FROM MyTable WHERE SomeId=1 (1st transaction began). Then, immediately after this query was executed, the procedure is called with ID=2 (2nd transaction began).
What I fail to understand is how much data would be locked during the execution of my stored procedure in this case:
If the 1st query of the 1st transaction returns 0 records, does this mean that 1st transaction locks nothing and other transactions are able to INSERT ID=1 before 1st transaction tries it?
Or does the 1st transaction lock the whole table making the 2nd transaction wait even though those 2 transactions can never try to read/update the same row?
Or does 1st transaction somehow forbid anyone else to read/write only records with ID=1 until it is comleted?
If your filter is on an index, that's what's going to get locked. So regardless of whether the row already exists or not, it's locked for the duration of the transaction. Take care, though - it's very easy to turn a row lock into something nastier, especially full table locks. And of course, it's easy to introduce deadlocks this way :)
However, I'd suggest a different approach. First, try to do an insert. If it works, you're done - if it doesn't, you know you can safely do an atomic update. Very fast, very cheap, very reliable :)
I have a program that connects to an Oracle database and performs operations on it. I now want to adapt that program to also support an SQL Server database.
In the Oracle version, I use "SELECT FOR UPDATE WAIT" to lock specific rows I need. I use it in situations where the update is based on the result of the SELECT and other sessions can absolutely not modify it simultaneously, so they must manually lock it first. The system is highly subject to sessions trying to access the same data at the same time.
For example:
Two users try to fetch the row in the database with the highest priority, mark it as busy, performs operations on it, and mark it as available again for later use.
In Oracle, the logic would go basically like this:
BEGIN TRANSACTION;
SELECT ITEM_ID FROM TABLE_ITEM WHERE ITEM_PRIORITY > 10 AND ITEM_CATEGORY = 'CT1'
ITEM_STATUS = 'available' AND ROWNUM = 1 FOR UPDATE WAIT 5;
UPDATE [locked item_id] SET ITEM_STATUS = 'unavailable';
COMMIT TRANSACTION;
Note that the queries are built dynamically in my code. Also note that when the previously most favorable row is marked as unavailable, the second user will automatically go for the next one and so on. Furthermore, different users working on different categories will not have to wait for each other's locks to be released. Worst comes to worst, after 5 seconds, an error would be returned and the operation would be cancelled.
So finally, the question is: how do I achieve the same results in SQL Server? I have been looking at locking hints which, in theory, seem like they should work. However, the only locks that prevents other locks are "UPDLOCK" AND "XLOCK" which both only work at a table level.
Those locking hints that do work at a row level are all shared locks, which also do not satisfy my needs (both users could lock the same row at the same time, both mark it as unavailable and perform redundant operations on the corresponding item).
Some people seem to add a "time modified" column so sessions can verify that they are the ones who modified it, but this sounds like there would be a lot of redundant and unnecessary accesses.
You're probably looking forwith (updlock, holdlock). This will make a select grab an exclusive lock, which is required for updates, instead of a shared lock. The holdlock hint tells SQL Server to keep the lock until the transaction ends.
FROM TABLE_ITEM with (updlock, holdlock)
As documentation sayed:
XLOCK
Specifies that exclusive locks are to be taken and held until the
transaction completes. If specified with ROWLOCK, PAGLOCK, or TABLOCK,
the exclusive locks apply to the appropriate level of granularity.
So solution is using WITH(XLOCK, ROWLOCK):
BEGIN TRANSACTION;
SELECT ITEM_ID
FROM TABLE_ITEM
WITH(XLOCK, ROWLOCK)
WHERE ITEM_PRIORITY > 10 AND ITEM_CATEGORY = 'CT1' AND ITEM_STATUS = 'available' AND ROWNUM = 1;
UPDATE [locked item_id] SET ITEM_STATUS = 'unavailable';
COMMIT TRANSACTION;
In SQL Server there are locking hints but they do not span their statements like the Oracle example you provided. The way to do it in SQL Server is to set an isolation level on the transaction that contains the statements that you want to execute. See this MSDN page but the general structure would look something like:
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE;
BEGIN TRANSACTION;
select * from ...
update ...
COMMIT TRANSACTION;
SERIALIZABLE is the highest isolation level. See the link for other options. From MSDN:
SERIALIZABLE Specifies the following:
Statements cannot read data that has been modified but not yet
committed by other transactions.
No other transactions can modify data that has been read by the
current transaction until the current transaction completes.
Other transactions cannot insert new rows with key values that would
fall in the range of keys read by any statements in the current
transaction until the current transaction completes.
Have you tried WITH (ROWLOCK)?
BEGIN TRAN
UPDATE your_table WITH (ROWLOCK)
SET your_field = a_value
WHERE <a predicate>
COMMIT TRAN