What is a deadlock in SQL Server and when it arises?
What are the issues with deadlock and how to resolve it?
In general, deadlock means that two or more entities are blocking some sources, and none of them is able to finish, because they are blocking sources in a cyclic way.
One example: Let's say I have table A and table B, I need to do some update in A and then B and I decide to lock both of them at the moment of usage (this is really stupid behaviour, but it serves it's purpose now). At the same moment, someone else does the same thing in opposite order - locks B first, then locks A.
Chronologically, this happens:
proc1: Lock A
proc2: Lock B
proc1: Lock B - starts waiting until proc2 releases B
proc2: Lock A - starts waiting until proc1 releases A
Neither of them will ever finish. That's a deadlock. In practice this usually results in timeout errors since it is not desired to have any query hanging forever, and the underlying system (e.g. the database) will kill queries that don't finish in time.
One real world example of a deadlock is when you lock your house keys in your car, and your car keys in your house.
What is a deadlock
A deadlock happens when two concurrent transactions cannot make progress because each one waits for the other to release a lock, as illustrated in the following diagram.
Because both transactions are in the lock acquisition phase, neither one releases a lock prior to acquiring the next one.
Recovering from a deadlock situation
If you're using a Concurrency Control algorithm that relies on locks, then there is always the risk of running in a deadlock situation. Deadlocks can occur in any concurrency environment, not just in a database system.
For instance, a multithreading program can deadlock if two or more threads are waiting on locks that were previously acquired so that no thread can make any progress. If this happens in a Java application, the JVM cannot just force a Thread to stop its execution and release its locks.
Even if the Thread class exposes a stop method, that method has been deprecated since Java 1.1 because it can cause objects to be left in an inconsistent state after a thread is stopped. Instead, Java defines an interrupt method, which acts as a hint as a thread that gets interrupted can simply ignore the interruption and continue its execution.
For this reason, a Java application cannot recover from a deadlock situation, and it is the responsibility of the application developer to order the lock acquisition requests in such a way that deadlocks can never occur.
However, a database system cannot enforce a given lock acquisition order since it's impossible to foresee what other locks a certain transaction will want to acquire further. Preserving the lock order becomes the responsibility of the data access layer, and the database can only assist in recovering from a deadlock situation.
The database engine runs a separate process that scans the current conflict graph for lock-wait cycles (which are caused by deadlocks).
When a cycle is detected, the database engine picks one transaction and aborts it, causing its locks to be released, so that the other transaction can make progress.
Unlike the JVM, a database transaction is designed as an atomic unit of work. Hence, a rollback leaves the database in a consistent state.
Deadlock priority
While the database chooses to rollback one of the two transactions being stuck, it's not always possible to predict which one will be rolled back. As a rule of thumb, the database might choose to roll back the transaction with a lower rollback cost.
Oracle
According to the Oracle documentation, the transaction that detected the deadlock is the one whose statement will be rolled back.
SQL Server
SQL Server allows you to control which transaction is more likely to be rolled back during a deadlock situation via the DEADLOCK_PRIORITY session variable.
The DEADLOCK_PRIORITY session can accept any integer between -10 and 10, or pre-defined values such as LOW (-5), NORMAL (0) or HIGH (5).
In case of a deadlock, the current transaction will roll back, unless the other transactions have a lower deadlock priority value. If both transactions have the same priority value, then SQL Server rolls back the transaction with the least rollback cost.
PostgreSQL
As explained in the documentation, PostgreSQL does not guarantee which transaction is to be rolled back.
MySQL
MySQL tries to roll back the transaction that modified the least number of records, as releasing fewer locks is less costly.
Deadlock is what happens when two people need multiple resources to execute, and where some of the resources are locked by each of the people. This leads to the fact that A can't execute without something B has and vice versa.
Lets say I have Person A and Person B. They both need to get two rows to run (Row1 and Row2).
Person A locks Row1 and tries to get Row2.
Person B locks Row2 and tries to get Row1.
Person A can't run because it needs Row2, Person B can't run because it needs Row1. Neither person will ever be able to execute because they're locking what the other needs and vice versa.
One reasonably simple way to reduce deadlock is in all your complex transactions, you should do operations in the same order. In other words, access Table1 then Table2 in the same order. This will help reduce the number of deadlocks that occur.
An impasse that may result when two (or more) transactions are each
waiting for locks to be released that are held by the other.
Deadlock is when a process or thread enters a waiting state because a requested system resources in held by another waiting process
Related
I have dead lock: sql A is waiting on sql B.
----- Information for the OTHER waiting sessions -----
A is: delete from attachment where ID=:1
----- Current SQL Statement for this session (sql_id=audwbsf163w1n) -----
B is: delete from detail where ID=:1
deadlock trace log
My big question is. Why is possible that these two sql scripts are waiting on each other but they have nothing in common in database? It do not make any sense, that they are waiting on each other.
I'd advise care with the terminology used: "Deadlock" has a very specific meaning (below) and what you are implying in your problem description is a "blocking lock".
A deadlock is the classic “deadly embrace” problem. The “deadly embrace” happens when, say Task B, attempts to lock a row which is being held by another task, say Task A, and Task A is waiting for Task B to release a lock. Oracle will abort one of the transaction to prevent a perpetual wait condition.
Root Cause (general):
• Transactions deadlocked one another while waiting for resources. [from ora docs]
General Solution:
• Generally, this is a design flaw in the application.
• The deadlock is an application issue and must be fixed within the application;
there is no DB fix for a deadlock.
A deadlock will result in Oracle terminating one of the sessions and that session will receive an error indicating the reason. This happens with 2 to 3 seconds of it being detected.
A blocking lock on the other hand will remain indefinitely until the blocking session either commits, is rolled-back or is killed (which does a rollback).
Also, check for cascading operations generated as recursive SQL by the DBMS.
I am new to SQL server and the deadlocking like issues. I read articles about it. I want to understand following things:
SQL server uses appropriate Lock Mode depending on the IsolationLevel I set while beginning the transaction. If this understanding is correct, what's the purpose of sp_getapplock ?
I am running into the SQL deadlock issue. Several instances of my ASP.NET application running on different servers access the same database. If I use the appropriate IsolationLevel on transactions, do I still need to acquire SQL lock using sp_getapplock?
(I know more appropriate solution is to identify problematic transactions and fix them. But still it's more of a theoretical question.)
Blocking and deadlocking are two different things. Please read this article for more details.
Following is copied from the article:
Developers and DBAs often think that their SQL Server instance is experiencing deadlocks when, really, it is experiencing severe blocking.
Blocking occurs when session A requests a lock on a resource (typically a row, page or table), but SQL Server cannot grant that lock because session B already holds a non-compatible lock on that resource.
This is a transient situation and can be completely resolved by the session B completing its work and releasing its locks. It is possible to have extensive blocking chains where multiple sessions are blocked waiting for a session that itself is blocked waiting for another session that is blocked and so on, repeating multiple time. However, at the head of the blocking chain will be a head ‘blocker’ that is not waiting for a lock. It may be waiting for some other resource, such as a latch, memory, or IO, but at least one session will not be waiting for a lock, and the blocking chain will clear as soon as the head blocker can continue processing.
A deadlock is different; it occurs when two or more sessions are waiting for each other, in such a way that none can complete. A deadlock can be viewed as a circular lock chain, where every single process in the blocking chain is waiting for one or more other processes in that same blocking chain.
sp_getapplock is not directly related to your problems. Applocks are custom defined locks, used by your application - for example when you need to synchronize with external data source, using some lengthy and complex procedure and you don't want multiple processes to run such procedure at once.
About deadlocks - you have to design your procedures to minimise deadlock possibility and then you need create some error handling to detect deadlocks and take appropriate action (eg retry 5 times, fail afterwards).
If you could post your deadlocking procedure code, then likely someone can help you redesign it.
We are experiencing the mentioned dead lock exception while doing CRUD on two SQL Server tables from parallel threads by calling Stored Procedures, here is the detailed scenario:
We have a desktop application where we are spinning up a code block in 100 - 150 parallel threads, the code block does insertion in TableA using SQL Bulk Copy and makes calls to three Stored Procedures, The stored procedures do insertion, updation and deletion in TableB based on some selection from TableA.
Soon as the application starts execution of the threads, SQL Server starts throwing the mentioned dead lock exception for a certain number of threads while some of the threads do run fine.
Exception Message:
Transaction (Process ID 160) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
Any help in this regard is appreciated in advance.
Thanks.
Is this SQL Server or SQL Azure/Azure SQL DB? If it's "box" SQL Server, you might consider ALTER DATABASE SET READ_COMMITTED_SNAPSHOT ON. This will enable read versioning. It's still possible to encounter deadlocks in this state, but it's as close to a silver bullet as you're likely to get.
Read versioning changes the concurrency model in some subtle ways, so be sure to read about it first and make sure it's compatible with your business logic: https://msdn.microsoft.com/en-us/library/tcbchxcb(v=vs.110).aspx
Otherwise, Srivats's other suggestion about minimizing transaction scope is not always simple to implement but is still solid. To that list, I would add: Ensure that you have well-indexed query access paths, and verify that none of the queries within your transactions require full table or index scans.
The message is clearly evident that Transaction (Process ID 160) was deadlocked on lock resources with another process.The lock could be on different levels. The locks are not getting released before another thread could lock that particular resource. Try to kill that Process Id and check the workflow if there are any lock conflicts.
I have n machines writing to DB (sql server) at the exact same time (initiating a transaction). I'm setting the Isolation level to serializable. My understanding is that whichever machine's transaction gets to the DB first, gets executed and the other transactions will be blocked while this completes.
Is that correct?
It depends - are they all performing the same activities? That is, exactly the same statements executing in the same order, with no flow control?
If not, and two connections are accessing independent objects in the DB, they can run in parallel.
If there's some overlap of resources, then some progress may be made by multiple connections until one of them wants to take a lock that another already has - at which point it will wait. There is then the possibility of deadlocks.
SERIALIZABLE:
Statements cannot read data that has been modified but not yet committed by other transactions.
No other transactions can modify data that has been read by the current transaction until the current transaction completes.
Other transactions cannot insert new rows with key values that would fall in the range of keys read by any statements in the current transaction until the current transaction completes.
whichever machine's transaction gets to the DB first, gets executed and the other transactions will be blocked while this completes
No, this i incorrect. The results should be as if each transaction was executed one after another (serially, hence the isolation level name). But the engine is free to use any implementation it likes, as long as it honors the guarantees of the serializable isolation model. And some engines actually implement it pretty much as you describe it, eg. Redis Transactions (although Redis has no 'isolation level' concept).
For SQL Server the transactions will execute in parallel until they hit a lock conflict. When a conflict occurs the transaction that has the lock granted continues undisturbed, while the one that requests the lock in a conflicting mode has to wait for the lock to free (for the granted transaction to commit). Which transaction happens to be the request and which one happens to be the granted is entirely up to what is being executed. That means that it well may be the case that the second machine gets the grant first and finishes first, while the first machine waits.
For a better understanding how the locking behavior differs under serializable isolation level, see Key-Range Locking
Yes, this will be true for write operations at any isolation level: "My understanding is that whichever machine's transaction gets to the DB first, gets executed and the other transactions will be blocked while this completes."
The isolation level helps determine what happens when you READ data while this is going on. Serializable read operations will block your write operations, which might be the behavior you want.
Lets say I open a transaction and run update queries.
BEGIN TRANSACTION
UPDATE x SET y = z WHERE w = v
The query returns successfully and the transaction stays open deliberately for a period of time before I decide to commit.
While I'm sitting on the transaction is it ever possible the MSSQL deadlock machinary would be able to preempt my open transaction that is not actually executing anything to either clear a deadlock or free resources as system memory/resource limits are reached?
I know about SET DEADLOCK_PRIORITY and have read the MSDN articles on the topic of deadlocks. Logically since I'm not actively seeking to stake claim on any additional resources I can't imagine a scenario that would trigger a sane deadlock avoidance algorithm.
Does anyone know for sure if its possible that simply holding any locks can make me a valid target? Similarly could any low resource condition trigger the killing of my SPID?
NO
For a deadlock to occur all the participants in the deadlock chain must be waiting for a resource (a lock). If your connection is idle it means it doesn't execute a request, which implies it cannot be waiting.
As for other conditions that can kill your session I can think of at least three:
administrative operations that use WITH ROLLBACK_IMMEDIATE
a mirroring failover
intentional KILL <yourspid>, perhaps as a joke by your friendly DBA
To answer your question: you can be a deadlock victim if you're not executing a query in a transaction.
It's counter-intuitive, but you can be a deadlock victim by running a SELECT statement.
It can happen if you're running a query that uses an index:
you scan indexes looking for matching rows
other process starts updating data pages
you now want to fetch data from data pages from matching rows
other process holding locks on data pages
you wait for data page locks to release
other process finished updating data pages, wants to update indexes
you are holding read locks on indexes
other process waits for index locks to release
DEADLOCK
So, strictly speaking, you can be a deadlock victim, when you're not executing a query in a transaction. The other guy wasn't executing his UPDATE statement in a transaction either.
Nobody's explicitly using a transaction, yet there's a deadlock.
Possible problems:
SQL Server only has a finite number of locks. It is possible to run out of locks.
Other resources are finite (e.g. memory, tempdb). Holding on to these resources could cause those resources to run out.
Transaction logs - the logical transaction logs cannot be freed for re-use if a transaction is open. The result could be a log that fills up. This problem could stop your process because it would halt the entire instance.
To consider:
CASCADE: DELETE may only have one table in the command, but the a CASCADE relationship may touch other tables.
Triggers: Triggers on the modified table may affect other tables.
DELETE and UPDATE commands may use the FROM clause which touch other tables. I've never seen this, but I would not rule it out.
Transactions may time out, is that what is happening.
As you have at least 1 (or more) update locks taken out and make be some read and table scan locks, you may be killed to help free up deadlocks created by other transactions. The deadlock recovery code in SQL Server is unlikely to be totally bug free and it is not normal to keep a transaction open for a long time on SQL Server. However I would not expect that to happen often.
Some system when they detach deadlock type problem, just start killing “long lived” transactions that have not done match work so as to free up locks. Just because you are not part of the deadlock loop, does not stop the system picking on you.
To understand what is going on in your case, you will have to use the Sql Server Profiler to collect all the locking and deadlock related events, as well as event about aborted connection and transactions etc. Good lack this will time some time and a good level of understanding of the profiler events you are looking at...
The detail of this sort of things are different between database vendors and versions of their database. However as it is considered bad design by most database vendors to have a transaction open for a long time, doing so tends to lead to problems and hit code paths that have not had the most testing effort.
Just because you're not in a transaction doesn't mean you're not holding locks.