I have 45 active concurrent insert transaction that each transaction try to insert (only insert without any select or update) about 250 rows to some tables.
The problem is when a transaction wants to insert the data into the tables, there are about 1000 X and IX locks (sys.dm_tran_locks) on multi index rows and index pages.
I have moved index files to an SSD but it didn't help and I still have a lot of pending transactions which each transaction takes about 200ms to 4000ms to be completed according to Adult logout on SQL profiler.
The Buffer I/O, Buffer latch, Lock, Latch, Logging Wait times are 0 or very low in activity monitor.
I have tried to increase number of transactions, but it also didn't help and number of execution in Activity monitor is still same.
My system info:
2x E5,
SSD Raid 0 for log files,
HDD Raid 10 for data,
SSD Raid 0 for indexes,
+64GB DDR3,
SQL Server 2014 SP2
There can be multiply suggestions depending on what exactly causes the problem:
Your indexes. Every time you do an insert SQL Server updates all indexes on a table. So, solution would be to decrease number of indexes on your tables.
IDENTITY column contention. Try to replace your IDENTITY columns by UNIQUEIDENTIFIER.
Extra I/O associated with page splits. Regularly rebuild clustered index with lower FILLFACTOR (Extreme scenario: <50%).
PFS contention. Create multiple files in your DB, and split indexes/tables to them.
You are on SQL2014. Try to use In-Memory features.
Related
I have two tables - Things and History. Let's say Things have a column Id as primary key and another column LastUpdated that is some kind of timestamp value. There is an index on LastUpdated column. There are other columns but I believe that should not be of much relevance. I have two business workflows from my ASP.NET MVC Web API application
Gets things that are changed since last time. Runs a query of form: SELECT [Columns] FROM Things WHERE LastUpdated > #LastUpdated. This query is fired on connection w/o transaction or transaction scope.
Updates particular thing. This happens in two database transactions - first transaction has a SELECT on Things with Id followed by UPDATE Things using Id. The second transaction inserts row in History table. Both transactions are controlled using TransactionScope with Read Committed isolation level.
I am trying to bench mark the performance with a scenario such as
5000 rows in Things table
25 queries (#1 above) per second
200 updates (#2 above) per second
Environment: My dev machine (Core i7, 8GB) with Sql Server 2016 LocalDB engine.
What I am observing is that as the number of rows increases in History table, performance degrades for both #1 and #2. #2 is understandable as it actually updates the history table. However, I am puzzled that why #1 gets degraded dramatically. For example, #2 time degrades from around 10 ms at empty database to around 65 ms for 250K rows in history table while #1 time degrades from around 10 ms to 200 ms. Note that Things table have constant number of rows (5000). Only explanation that I can think of is due to contention at Disk IO.
Am I right here? Is there any way to verify this(say, using SQL Profiler)?
I couldn't spend as much time as to my liking on this but as suspected, the apparent reason seams to be disk I/O contention. I moved the history table to another database on different machine. And now, number of rows in history table stopped affecting #1 timing, it remains more or less constant (6-7% variance) irrespective of number of rows in history table.
Open 2 windows in SQLServer 2014 Management Studio connected to a 2014 DB.
In 1st window:
SELECT * FROM categorytypes WHERE id = 12
Output:
ID DESCRIPTION ORDER
---------------------
12 Electronics 20
Then:
begin tran
UPDATE CategoryTypes SET [order] = 20 WHERE id = 12
Now go to other window (Ctrl+N):
SELECT * FROM CategoryTypes
The query will execute indefinitely until 1st window tran is committed or roll-backed. That's fine because ID=12 row is locked.
SELECT * FROM CategoryTypes WHERE ID <> 12
That works fine.
SELECT * FROM CategoryTypes WHERE Description = 'MEN'
Here it is problem, why the hell this query should execute indefinitely, we know that ID=12 has description 'Electronics'.
In a large application where huge DML process and select operation is done simultaneously on same table, this kind of locking mechanism wont allow to do these 2 things on same time on different set of records.
In Oracle this kind of use case works, as long as locked (dirty row) is not part of result set.
Guys, is there any way to avoid this, kind of Oracle locking mechanism? I don't want to use NOLOCK, or set my transaction to READ UNCOMMITTED.
Thanks.
In SQL Server, the default behavior is to use locking in the default READ_COMMITTED isolation level. With this default locking behavior, it is especially important to have useful indexes so that only needed data are touched. For example, the query with Description in the WHERE clause will not be blocked if you 1) have an index on Description and 2) that index is used in the query plan to locate the needed rows. Without an index on Description, a full table scan will result and the query will be blocked when it hits the uncommitted change.
If you want to avoid locking such that readers don't block writers and visa-versa, you can turn on the READ_COMMITTED_SNAPSHOT database option. SQL Server will then use row versioning instead of locking to ensure only committed data are returned.
Like other DBMS products that use row versioning, there is more overhead with READ_COMMITTED_SNAPSHOT than in-memory locking. SQL Server adds 14 bytes of additional storage per row plus uses tempdb more heavily for the row version store. Whether or not these overhead costs are justified depends on the concurrency benefits your workload experiences.
I have a SQL Server database where I am deleting rows from three tables A,B,C in batches with some conditions through a SQL script scheduled in a SQL job. The job runs for 2 hours as the tables have a large amount of data. While the job is running, my front end application is not accessible (giving timeout error) since the application inserts and updates data in these same tables A,B,C.
Is it possible for the front end application to run in parallel without any issues while the SQL script is running? I have checked for the locks on the table and SQL Server is acquiring page locks. Can Read Committed Snapshot or Snapshot isolation levels or converting page locks to row locks help here. Need advice.
Split the operation in two phases. In the first phase, collect the primary keys of rows to delete:
create table #TempList (ID int);
insert #TempList
select ID
from YourTable
In the second phase, use a loop to delete those rows in small batches:
while 1=1
begin
delete top (1000)
from YourTable
where ID in (select ID from #TempList)
if ##rowcount = 0
break
end
The smaller batches will allow your front end applications to continue in between them.
I suspect that SQL Server at some point escalates to table lock, and this means that the table is inaccessible, both for reading and updating.
To optimize locking and concurrency when dealing with large deletes, use batches. Start with 5000 rows at the time (to prevent lock escalation) and monitor how it behaves and whether it needs further tuning up or down. 5000 is a "magic number", but it's low enough number that lock manager doesn't consider escalating to table lock, and large enough for the performance.
Whether timeouts will happen or not depends on other factors as well, but this will surely reduce if not elliminate alltogether. If the timeout happen on read operations, you should be able to get rid of them. Another approach, of course, is to increase the command timeout value on client.
Snapshot (optimistic) isolation is an option as well, READ COMMITTED SNAPSHOT more precisely, but it won't help with updates from other sessions. Also, beware of version store (in tempdb) growth. Best if you combine it with the proposed batch approach to keep the transactions small.
Also, switch to bulk-logged recovery for the duration of delete if the database is in full recovery normally. But switch back as soon as it finishes, and make a backup.
Almost forgot -- if it's Enterprise edition of SQL Server, partition your table; then you can just switch the partition out, it's almost momentarilly and the clients will never notice it.
I have two tables T_A and T_B.
Both are empty.
Both has clustered index on them.
Recovery model is set to SIMPLE.
The insert...select.. meets the requirements of minimal logging. See
http://msdn.microsoft.com/en-us/library/ms191244.aspx
Both staging tables contains large amount of data.
I need to import data into them from staging tables.
If I perform the following T-SQL blocks individually, each takes 2 to 3 minutes to finish. The total time is about 5 to 6 minutes.
BEGIN TRAN
INSERT INTO T_A WITH(TABLOCK) FROM SRC_A WITH(NOLOCK);
COMMIT TRAN
BEGIN TRAN
INSERT INTO T_B WITH(TABLOCK) FROM SRC_B WITH(NOLOCK);
COMMIT TRAN
To make it faster I open two sessions in SMSS and execute the two blocks in parallel. To my surprise, each session takes about 10 to 12 minutes to finish. Together the total time is more than doubled. The wait_type shown is PAGEIOLATCH_SH which point to disk I/O bottleneck. What I don't understand is that even if the two sessions have to wait on each other for I/O it should not wait for that long. Can anyone help explain this?
My story has not ended here yet. Then I removed the clustered index on both table and ran the two blocks in parallel each in a different session. This time each takes about 1 minutes to finish. The total time is about 1 minutes since they are in parallel. Great! But the nightmare comes when I try to create clustered index back.
If I create the cluster index individually it takes 4 minutes each to finish. The total time is about 8 minutes. This defeated my purpose of improving performance.
Then I try to create clustered index on the two tables in parallel each on a different session. This time it is the worst: one takes 12 minutes to finish and the other takes 25 minutes to finish.
From my test result my best choice is back to square one: execute the two transactions sequentially with clustered index on the table.
Has anyone experienced similar situation and what is the best practice to make it faster?
When creating the clustered index after inserting the records SQL has to recreate this table in the background anyway so it would be faster to insert the records directly into the table with the clustered index already present.
Also disable any non clustered indexes while inserting and enable them afterwards again, creating indexes on a filled table is faster than updating them for each insert. Remember to set Max DOP option to 0 when creating indexes.
Bulk inserts are also a lot faster then insert into statement.
I use the 'SQL server import and export wizard' for copying large amounts of data and it seems to be way faster (the wizard uses bulk statements). If necessary you can try to find the statement this wizard uses and run it yourself.
I'm currently having troubles with frequent deadlocks with a specific user table in SQL Server 2008. Here are some facts about this particular table:
Has a large amount of rows (1 to 2 million)
All the indexes used on this table only have the "use row lock" ticked in their options
Edit: There is only one index on the table which is its primary Key
rows are frequently updated by multiple transactions but are unique (e.g. probably a thousand or more update statements are executed to different unique rows every hour)
the table does not use partitions.
Upon checking the table on sys.tables, I found that the lock_escalation is set to TABLE
I'm very tempted to turn the lock_escalation for this table to DISABLE but I'm not really sure what side effect this would incur. From What I understand, using DISABLE will minimize escalating locks from TABLE level which if combined with the row lock settings of the indexes should theoretically minimize the deadlocks I am encountering..
From what I have read in Determining threshold for lock escalation it seems that locking automatically escalates when a single transaction fetches 5000 rows..
What does a single transaction mean in this sense? A single session/connection getting 5000 rows thru individual update/select statements?
Or is it a single sql update/select statement that fetches 5000 or more rows?
Any insight is appreciated, btw, n00b DBA here
Thanks
LOCK Escalation triggers when a statement holds more than 5000 locks on a SINGLE object. A statement holding 3000 locks each on two different indexes of the same table will not trigger escalation.
When a lock escalation is attempted and a conflicting lock exists on the object, the attempt is aborted and retried after another 1250 locks (held, not acquired)
So if your updates are performed on individual rows and you have a supporting index on the column, then lock escalation is not your issue.
You will be able to verify this using the Locks-> lock escalation event from profiler.
I suggest you capture the deadlock trace to identify the actual cause of the deadlock.
I found this article after a quick Google of disabling table lock escalation. Although not a real answer for the OP I think it is still relevant for one off scripts and note worthy here. There's a nice little trick you can do to temporarily disable table lock escalation.
Open another connection and issue something like.
BEGIN TRAN
SELECT * FROM mytable (UPDLOCK, HOLDLOCK) WHERE 1=0
WAITFOR DELAY '1:00:00'
COMMIT TRAN
as
Lock escalation cannot occur if a different SPID is currently holding
an incompatible table lock.
from microsoft kb