I have a simple SQL,
begin tran xxx;
select top 50 * from aTable readpast for update;
delete these "top 50" records;
commit tran xxx;
And I ran from three threads (each thread starts own transaction). What I have observed is that sometimes two threads are returning the same set of records.
According to Sybase manual, "for update" should have locked the records that I have selected and readpast should have prevented me from reading locked records.
isolation level is 1.
Any pointer is appreciated.
Related
I'm working in SQL Server 2016. I have two tables, one is a queue of work items (TQueue), and the second is the work items (TWork) that are being processed.
I have a script that grabs the top 100 items from TQueue that do not have a record in TWork, and then inserts those items into TWork to be processed.
For performance reasons, I want to run multiple instances of the script simultaneously. The challenge is that Script 1 grabs 100 items, and before the transaction to insert these items into TWork is committed, Script 2 grabs the same set of items and inserts them as well.
Question
I would like to block the reading of TQueue until insert transaction into TWork has completed. Is this possible?
You may use table hints to achieve this goal.
For example:
Create Table Val (ID Int)
Insert Into Val (ID)
Values (0),(1),(2),(3),(4),(5)
First session:
Set Transaction Isolation level Read Committed
Begin Transaction
Select Top 2 * From Val With (ReadPast, XLock, RowLock)
-- Return 0,1
-- Commit has been commented for illustrative purposes.
-- Don't forget to commit the transaction later.
-- Commit
Second session:
Set Transaction Isolation level Read Committed
Begin Transaction
Select * From Val With (ReadPast, XLock, RowLock)
-- Return 2,3,4,5
Commit
I'm am working in SharePoint and am trying to understand why there is a limit of 5000 records for views on lists that can have a far greater number.
My question is not about SharePoint, but SQL. Does SQL have the limit described below, or was this something imposed on SQL by the SharePoint team?
For performance reasons, whenever SQL Server executes a single query
that returns 5000 + items, locks escalation happens within the SQL
table. As a result, the entire table will be locked. Since the entire
Share Point data is stored as a single table, a single list view query
that exceeds 5000+ items will lock the entire Share Point data table
within that content. The database and all the users will face huge
performance degradation. The entire set of users using Share Point at
the time of lock escalation will have to wait for a longer time to
retrieve the data. Hence, you can see that the list threshold is a
limitation that is imposed upon Share Point by its backend SQL Server.
This issue is generated from SQL and the reason is row lock
escalation. In order to avoid this performance degradation, Share
Point has imposed the limitation of 5000 items to be queried at any
point of time. Any queries for 5000+ items will be dealt the threshold
error message. Ref link
Thanks
EDIT____________________________________
An article on this issue:
https://www.c-sharpcorner.com/article/sharepoint-list-threshold-issue-the-traditional-problem/
Does SQL Server lock tables when queries return results greater than 5000
records?
Not generally, no.
It is documented that 5,000 is a magic number for the database engine to first attempt lock escalation (followed by further attempts at 1,250 increments) but unless running at repeatable read or serializable isolation level this will not generally be hit just by returning 5,0000 items in a SELECT. The default read committed level will release locks as soon as the data is read so never hit the threshold.
You can see the effect of isolation level on this with the following example.
CREATE TABLE T(C INT PRIMARY KEY);
INSERT INTO T
SELECT TOP 10000 ROW_NUMBER() OVER (ORDER BY ##SPID)
FROM sys.all_objects o1, sys.all_objects o2
And (uses undocumented trace flags so should only be used in dev environment)
DBCC TRACEON(3604,611);
/*5,000 key locks are held*/
SET TRANSACTION ISOLATION LEVEL REPEATABLE READ;
BEGIN TRAN
SELECT COUNT(*) FROM (SELECT TOP 5000 C FROM T) T
SELECT resource_type, request_mode, count(*) FROM sys.dm_tran_locks where request_session_id = ##spid GROUP BY resource_type, request_mode;
COMMIT
/*No key locks are held. They have been escalated to an object level lock. The messages tab shows the lock escalation (in my case after 6248 locks not 5,000)*/
SET TRANSACTION ISOLATION LEVEL REPEATABLE READ;
BEGIN TRAN
SELECT COUNT(*) FROM (SELECT TOP 10000 C FROM T) T
SELECT resource_type, request_mode, count(*) FROM sys.dm_tran_locks where request_session_id = ##spid GROUP BY resource_type, request_mode;
COMMIT
/*No key locks are held. They are released straight away at this isolation level. The messages tab shows no lock escalation messages*/
SET TRANSACTION ISOLATION LEVEL READ COMMITTED;
BEGIN TRAN
SELECT COUNT(*) FROM (SELECT TOP 10000 C FROM T) T
SELECT * FROM sys.dm_tran_locks where request_session_id = ##spid
COMMIT
DBCC TRACEOFF(3604,611);
Recently I have gone through with Hints and Locks in SQL server. While google about this topic I have read one blog where some query have been written which I am not bale to understand. Here it is
BOL states: Use update locks instead of shared locks while reading a table, and hold locks until the end of the statement or transaction. I have some trouble translating this. Does this mean that the update locks are released after the execution of the SELECT statement, unless the SELECT statement in within a transaction?
I other words, are my assumptions in the following 2 scenario's correct?
Scenario 1: no transaction
SELECT something FROM table WITH (UPDLOCK)
/* update locks released */
Scenario 2: with transaction
BEGIN TRANSACTION
SELECT something FROM table WITH (UPDLOCK)
/* some code, including an UPDATE */
COMMIT TRANSACTION
/* update locks released */
Example for scenario 2 (referred for stackoverflow blog)
BEGIN TRAN
SELECT Id FROM Table1 WITH (UPDLOCK)
WHERE AlertDate IS NULL;
UPDATE Table1 SET AlertDate = getutcdate()
WHERE AlertDate IS NULL;
COMMIT TRAN
Please help to understand the above query.
My second question is: once execution of select statement completed at same time UPDLOCK get released or not?
Your assumption in scenario 2 is correct.
To answer your second question, no. The Update locks are held on the selected row(s) until the transaction ends, or until converted to exclusive locks when the update statement modifies those row(s). Step through each statement one at a time using SSMS to verify.
BEGIN TRAN
-- execute sp_lock in second session - no locks yet
SELECT Id FROM Table1 WITH (UPDLOCK) WHERE AlertDate IS NULL;
-- execute sp_lock in second session - update locks present
UPDATE Table1 SET AlertDate = getutcdate() WHERE AlertDate IS NULL;
-- execute sp_lock in second session - update (U) locks are replace by exclusive locks (X) for all row(s) returned by SELECT and modified by the UPDATE (Lock Conversion).
-- Update locks (U) continue to be held for any row(s) returned by the SELECT but not modified by the UPDATE
-- exclusive locks (X) are also held on all rows not returned by SELECT but modified by UPDATE. Internally, lock conversion still occurs, because UPDATE statements must read and write.
COMMIT TRAN
-- sp_lock in second session - all locks gone.
As for what is going on in scenario 1, all T-SQL statements exist either in an implicit or explicit transaction. Senario 1 is implicitly:
BEGIN TRAN
SELECT something FROM table WITH (UPDLOCK)
-- execute sp_lock in second session - update locks (U) will be present
COMMIT TRAN;
-- execute sp_lock in second session - update locks are gone.
Does this mean that the update locks are released after the execution of the SELECT statement, unless the SELECT statement in within a transaction?
The locks will be released as soon as the row is read..but the lock hold will be U lock,so any parallel transaction trying to modify this will have to wait
if you wrap above select in a transaction,locks will be released only when the transaction is committed,so any parallel transaction acquiring locks incompatible with U lock will have to wait
begin tran
select * from t1 with (updlock)
for the below second scenario
BEGIN TRANSACTION
SELECT something FROM table WITH (UPDLOCK)
/* some code, including an UPDATE */
COMMIT TRANSACTION
Imagine, if your select query returned 100 rows,all will use U lock and imagine the update in same transaction affects 2 rows,the two rows will be converted to x locks .so now your query will have 98 u locks and 2 xlocks until the transaction is committed
I would like to think Updlock as repeatable read,any new rows can be added,but any parallel transaction can't delete or update existing rows
I have a database table with thousands of entries. I have multiple worker threads which pick up one row at a time, does some work (takes roughly one second each). While picking up the row, each thread updates a flag on the database row (like a timestamp) so that the other threads do not pick it up. But the problem is that I end up in a scenario where multiple threads are picking up the same row.
My general question is that what general design approach should I follow here to ensure that each thread picks up unique rows and does their task independently.
Note : Multiple threads are running in parallel to hasten the processing of the database rows. So I would like to have a as small as possible critical segment or exclusive lock.
Just to give some context, below is the stored proc which picks up the rows from the table after it has updated the flag on the row. Please note that the stored proc is not compilable as I have removed unnecessary portions from it. But generally that's the structure of it.
The problem happens when multiple threads execute the stored proc in parallel. The change made by the update statement (note that the update is done after taking up a lock) in one thread is not visible to the other thread unless the transaction is committed. And as there is a SELECT statement (which takes around 50ms) between the UPDATE and the TRANSACTION COMMIT, on 20% cases the UPDATE statement in a thread picks up a row which has already been processed.
I hope I am clear enough here.
USE ['mydatabase']
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
ALTER PROCEDURE [dbo].[GetRequest]
AS
BEGIN
-- some variable declaration here
BEGIN TRANSACTION
-- check if there are blocking rows in the request table
-- FM: Remove records that don't qualify for operation.
-- delete operation on the table to remove rows we don't want to process
delete FROM request where somecondition = 1
-- Identify the requests to process
DECLARE #TmpTableVar table(TmpRequestId int NULL);
UPDATE TOP(1) request
WITH (ROWLOCK)
SET Lock = DateAdd(mi, 5, GETDATE())
OUTPUT INSERTED.ID INTO #TmpTableVar
FROM request tur
WHERE (Lock IS NULL OR GETDATE() > Lock) -- not locked or lock expired
AND GETDATE() > NextRetry -- next in the queue
IF(##RowCount = 0)
BEGIN
ROLLBACK TRANSACTION
RETURN
END
select #RequestID = TmpRequestId from #TmpTableVar
-- Get details about the request that has been just updated
SELECT somerows
FROM request
WHERE somecondition = 1
COMMIT TRANSACTION
END
The analog of a critical section in SQL Server is sp_getapplock, which is simple to use. Alternatively you can SELECT the row to update with (UPDLOCK,READPAST,ROWLOCK) table hints. Both of these require a multi-statement transaction to control the duration of the exclusive locking.
You need start a transaction isolation level on sql for isolation your line, but this can impact on your performance.
Look the sample:
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE
GO
BEGIN TRANSACTION
GO
SELECT ID, NAME, FLAG FROM SAMPLE_TABLE WHERE FLAG=0
GO
UPDATE SAMPLE_TABLE SET FLAG=1 WHERE ID=1
GO
COMMIT TRANSACTION
Finishing, not exist a better way for use isolation level. You need analyze the positive and negative point for each level isolation and test your system performance.
More information:
https://learn.microsoft.com/en-us/sql/t-sql/statements/set-transaction-isolation-level-transact-sql
http://www.besttechtools.com/articles/article/sql-server-isolation-levels-by-example
https://en.wikipedia.org/wiki/Isolation_(database_systems)
I have upgraded from SQL Server 2005 to 2008. I remember that in 2005, ROWLOCK simply did not work and I had to use PAGELOCK or XLOCK to achieve any type of actual locking. I know a reader of this will ask "what did you do wrong?" Nothing. I conclusively proved that I could edit a "ROWLOCKED" row, but couldn't if I escalated the lock level. I haven't had a chance to see if this works in SQL 2008. My first question is has anyone come across this issue in 2008?
My second question is as follows. I want to test if a value exists and if so, perform an update on relevant columns, rather than an insert of the whole row. This means that if the row is found it needs to be locked as a maintenance procedure could delete this row mid-process, causing an error.
To illustrate the principle, will the following code work?
BEGIN TRAN
SELECT ProfileID
FROM dbo.UseSessions
WITH (ROWLOCK)
WHERE (ProfileID = #ProfileID)
OPTION (OPTIMIZE FOR (#ProfileID UNKNOWN))
if ##ROWCOUNT = 0 begin
INSERT INTO dbo.UserSessions (ProfileID, SessionID)
VALUES (#ProfileID, #SessionID)
end else begin
UPDATE dbo.UserSessions
SET SessionID = #SessionID, Created = GETDATE()
WHERE (ProfileID = #ProfileID)
end
COMMIT TRAN
An explanation...
ROWLOCK/PAGELOCK is granularity
XLOCK is mode
Granularity and isolation level and mode are orthogonal.
Granularity = what is locked = row, page, table (PAGLOCK, ROWLOCK, TABLOCK)
Isolation Level = lock duration, concurrency (HOLDLOCK, READCOMMITTED, REPEATABLEREAD, SERIALIZABLE)
Mode = sharing/exclusivity (UPDLOCK, XLOCK)
"combined" eg NOLOCK, TABLOCKX
XLOCK would have locked the row exclusively as you want. ROWLOCK/PAGELOCK wouldn't have.