Confused with MS SQL Server LOCK would help in INSERT Scenario.(Concurrency) - sql-server

Business Scenario: This is a ticketing system, and we got so many user using the application. When a ticket(stored in 1st table in below) comes in to the application, any user can hit the ownership button and take ownershipf of it.
Only one user can take ownership for one ticket. If two user tries to hit the ownership button, first one wins and second gets another incident or message that no incident exists to take ownership.
Here i am facing a concurrency issue now. I already have a lock implementation using another table(2nd table in below).
I have two tables;
Table(Columns)
Ticket(TicketID-PK, OwnerUserID-FK)
TicketOwnerShipLock(TicketID-PK, OwnerUserID-FK, LockDate)Note: Here TicketID is set as Primary Key.
Current lock implementation: whenever user one tries to own ticket puts an entry to 2nd table with TicketID, UserID and current date,then goes to update the OwnerUserID in 1st table.
Before insert the above said lock entry, Procedure checks for any other user already created any lock for the same incident.
If already there is lock, lock wont be opened for the user. Else lock entry wont be entered and the user cannot update the ticket onwership.
More Info: There are so many tickets getting opened in 1st table, whenever user tries to take ownership, we should find the next available ticket to take ownership. So need to find ticket and to do some calculation and set a status for that ticket, there one more column in 1st table StatusID. Status will be assigned as Assigned.
Problem: Somehow two user's got the ownership for same ticket at excatly same time, i have even checked the millisecond but that too same.
1. I would like to know if any MS SQL Server LOCK would help in this scenario.
2. Or do i need to block table while insert.(This 2nd rable will not have much data approx. less than 15 rows)
Lock Creation Procedure Below:
ALTER PROCEDURE [dbo].[TakeOwnerShipGetLock]
#TicketId [uniqueidentifier],
#OwnerId [uniqueidentifier]
AS
BEGIN
SET NOCOUNT ON;
BEGIN TRANSACTION TakeOwnership
BEGIN TRY
DECLARE #Lock BIT
SET #Lock = 0
DECLARE #LockDate DATETIME
SELECT #LockDate = LockDate
FROM dbo.TakeOwnershipLock
WHERE TicketId = #TicketId
IF #LockDate IS NULL
AND NOT EXISTS ( SELECT 1
FROM dbo.TakeOwnershipLock as takeOwnership WITH (UPDLOCK)
INNER JOIN dbo.Ticket as Ticket WITH (NOLOCK)
ON Ticket.TicketID = takeOwnership.TicketId
WHERE takeOwnership.TicketId = #TicketId
AND Ticket.OwnerID is NULL )
BEGIN
INSERT INTO dbo.TakeOwnershipLock
( TicketId
,OwnerId
,LockDate
)
VALUES ( #TicketId
,#OwnerId
,GETDATE()
)
IF ( ##ROWCOUNT > 0 )
SET #Lock = 1
END
SELECT #Lock
COMMIT TRANSACTION TakeOwnership
END TRY
BEGIN CATCH
-- Test whether the transaction is uncommittable.
IF XACT_STATE() = 1
BEGIN
COMMIT TRANSACTION TakeOwnership
SET #Lock = 1
SELECT #Lock
END
-- Test whether the transaction is active and valid.
IF XACT_STATE() = -1
BEGIN
ROLLBACK TRANSACTION TakeOwnership
SET #Lock = 0
SELECT #Lock
END
END CATCH
END

Related

Why is my UPDATE stored procedure executed multiple times?

I use stored procedures to manage a warehouse. PDA scanners scan the added stock and send it in bulk (when plugged back) to a SQL database (SQL Server 2016).
The SQL database is fairly remote (other country), so there's sometimes delay in some queries, but this particular one is problematic: even if the stock table is fine, I had some problems with updating the occupancy of the warehouse spots. The PDA tracks the added items in every spot as a SMALLINT, then send back this value to the stored procedure below.
PDA "send_spots" query:
SELECT spot, added_items FROM spots WHERE change=1
Stored procedure:
CREATE PROCEDURE [dbo].[update_spots]
#spot VARCHAR(10),
#added_items SMALLINT
AS
BEGIN
BEGIN TRAN
UPDATE storage_spots
SET remaining_capacity = remaining_capacity - #added_items
WHERE storage_spot=#spot
IF ##ROWCOUNT <> 1
BEGIN
ROLLBACK TRAN
RETURN - 1
END
ELSE
BEGIN
COMMIT TRAN
RETURN 0
END
END
GO
If the remaining_capacity value is 0, the PDAs can't add more items to it on next round. But with this process, I had negative values because the query allegedly ran two times (so subtracted #added_items two times).
Is there a way for that to be possible? How could I fix it? From what I understood the transaction should be cancelled (ROLLBACK) if the affected rows are != 1, but maybe that's something else.
EDIT: current solution with the help of #Zero:
CREATE PROCEDURE [dbo].[update_spots]
#spot VARCHAR(10),
#added_racks SMALLINT
AS
BEGIN
-- Recover current capacity of the spot
DECLARE #orig_capacity SMALLINT
SELECT TOP 1
#orig_capacity = remaining_capacity
FROM storage_spots
WHERE storage_spot=#spot
-- Test if double is present in logs by comparing dates (last 10 seconds)
DECLARE #is_double BIT = 0
SELECT #is_double = CASE WHEN EXISTS(SELECT *
FROM spot_logs
WHERE log_timestamp >= dateadd(second, -10, getdate()) AND storage_spot=#spot AND delta=#added_racks)
THEN 1 ELSE 0 END
BEGIN
BEGIN TRAN
UPDATE storage_spots
SET remaining_capacity= #orig_capacity - #added_racks
WHERE storage_spot=#spot
IF ##ROWCOUNT <> 1 OR #is_double <> 0
-- If double, rollback UPDATE
ROLLBACK TRAN
ELSE
-- If no double, commit UPDATE
COMMIT TRAN
-- write log
INSERT INTO spot_logs
(storage_spot, former_capacity, new_capacity, delta, log_timestamp, double_detected)
VALUES
(#spot, #orig_capacity, #orig_capacity-#added_racks, #added_racks, getdate(), #is_double)
END
END
GO
I was thinking about possible causes (and a way to trace them) and then it hit me - you have no value validation!
Here's a simple example to illustrate the problem:
Spot | capacity
---------------
x1 | 1
Update spots set capacity = capacity - 2 where spot = 'X1'
Your scanner most likely gave you higher quantity than you had capacity to take in.
I am not sure how your business logic goes, but you need to perform something in lines of
Update spots set capacity = capacity - #added_items where spot = 'X1' and capacity >= #added_items
if ##rowcount <> 1;
rollback;
EDIT: few methods to trace your problem without implementing validation:
Create a logging table (with timestamp, user id (user, that is connected to DB), session id, old value, new value, delta value (added items).
Option 1:
Log all updates that change value from positive to negative (at least until you figure out the problem).
The drawback of this option is that it will not register double calls that do not result in a negative capacity.
Option 2: (logging identical updates):
Create script that creates a global temporary table and deletes records, from that table timestamps older than... let's say 10 minutes once every minute or so (play around with numbers).
This temporary table should hold the data passed to your update statement so 'spot', 'added_items' + 'timestamp' (for tracking).
Now the crucial part: When you call your update statement check if a similar record exists in the temporary table (same spot and added_items and where current timestamp is between timestamp and [timestamp + 1 second] - again play around with numbers). If a record like that exist log that update, if not add it to temporary table.
This will register identical updates that go within 1 second of each other (or whatever time-frame you choose).
I found here an alternative SQL query that does the update the way I need, but with a temporary value using DECLARE. Would it work better in my case, or is my initial query correct?
Initial query:
UPDATE storage_spots
SET remaining_capacity = remaining_capacity - #added_items
WHERE storage_spot=#spot
Alternative query:
DECLARE #orig_capacity SMALLINT
SELECT TOP 1 #orig_capacity = remaining_capacity
FROM storage_spots
WHERE spot=#spot
UPDATE Products
SET remaining_capacity = #orig_capacity - #added_items
WHERE spot=#spot
Also, should I get rid of the ROLLBACK/COMMIT instructions?

SQL Server: using table lock hint in select for ensuring correctness?

I've got a project that is trying to apply DDD (Domain Driven Design). Currently, we've got something like this:
begin tran
try
_manager.CreateNewEmployee(newEmployeeCmd);
tran.Commit();
catch
rollback tran
Internally, the CreateNewEmployee method uses a domain service for checking if there's already an employee with the memberId. Here's some pseudo code:
void CreateNewEmployee(NewEmployeeCmd cmd)
if(_duplicateMember.AlreadyRegistered(cmd.MemberId) )
throw duplicate
// extra stuff
saveNewEmployee()
end
Now, in the end, it's as if we have the following SQL instructions executed (pesudo code again):
begin sql tran
select count(*) from table where memberId=#memberId and status=1 -- active
--some time goes by
insert into table ...
end
NOw, when I started looking at the code, I've noticed that it was using the default SQL Server locking level. In practice, that means that something like this could happen:
--thread 1
(1)select ... --assume it returns 0
--thread 2
(2)select ... ---nothing found
(3)insert recordA
--thread 1
(4)insert record --some as before
(5) commit tran
--thread 1
(6) commit tran
So, we could end up having repeated records. I've tried playing with the transaction levels, but the only way I've managed to make it work like it's intended was by changing the select that is used to check if there's already a record in the table. I've ended up using a table lock hint which instructs sql to maintain a lock until the end of the transaction. That was the only way I've managed to get a lock when the select starts (changing the other isolation levels still wouldn't do what I needed since they all allowed the select to run)
So, I've ended up using a table lock which is held from the beginning until the end of the transaction. In practice, that means that step (2) will block until thread 1 ends its job.
Is there a better option for this kind of scenarios (that don't depend on using, say, indexes)?
Thanks.
Luis
You need to get the proper locks on the initial select, which you can do with the locking hints with (updlock, serializable). Once you do that, thread 2 will wait for thread 1 to finish if thread 2 is using the same key range in its where.
You could use the Sam Saffron upsert approach.
For example:
create procedure dbo.Employee_getset_byName (#Name nvarchar(50), #MemberId int output) as
begin
set nocount, xact_abort on;
begin tran;
select #MemberId = Id
from dbo.Employee with (updlock, serializable) /* hold key range for #Name */
where Name = #Name;
if ##rowcount = 0 /* if we still do not have an Id for #Name */
begin;
/* for a sequence */
set #MemberId = next value for dbo.IdSequence; /* get next sequence value */
insert into dbo.Employee (Name, Id)
values (#Name, #MemberId);
/* for identity */
insert into dbo.Employee (Name)
values (#Name);
set #MemberId = scope_identity();
end;
commit tran;
end;
go

How to prevent multi threaded application to read this same Sql Server record twice

I am working on a system that uses multiple threads to read, process and then update database records. Threads run in parallel and try to pick records by calling Sql Server stored procedure.
They call this stored procedure looking for unprocessed records multiple times per second and sometimes pick this same record up.
I try to prevent this happening this way:
UPDATE dbo.GameData
SET Exported = #Now,
ExportExpires = #Expire,
ExportSession = #ExportSession
OUTPUT Inserted.ID INTO #ExportedIDs
WHERE ID IN ( SELECT TOP(#ArraySize) GD.ID
FROM dbo.GameData GD
WHERE GD.Exported IS NULL
ORDER BY GD.ID ASC)
The idea here is to set a record as exported first using an UPDATE with OUTPUT (remembering record id), so no other thread can pick it up again. When record is set as exported, then I can do some extra calculations and pass the data to the external system hoping that no other thread will pick this same record again in the mean time. Since the UPDATE that has in mind to secure the record first.
Unfortunately it doesn't seem to be working and the application sometimes pick same record twice anyway.
How to prevent it?
Kind regards
Mariusz
I think you should be able to do this atomically using a common table expression. (I'm not 100% certain about this, and I haven't tested, so you'll need to verify that it works for you in your situation.)
;WITH cte AS
(
SELECT TOP(#ArrayCount)
ID, Exported, ExportExpires, ExportSession
FROM dbo.GameData WITH (READPAST)
WHERE Exported IS NULL
ORDER BY ID
)
UPDATE cte
SET Exported = #Now,
ExportExpires = #Expire,
ExportSession = #ExportSession
OUTPUT INSERTED.ID INTO #ExportedIDs
I have a similar set up and I use sp_getapplock. My application runs many threads and they call a stored procedure to get the ID of the element that has to be processed. sp_getapplock guarantees that the same ID would not be chosen by two different threads.
I have a MyTable with a list of IDs that my application checks in an infinite loop using many threads. For each ID there are two datetime columns: LastCheckStarted and LastCheckCompleted. They are used to determine which ID to pick. Stored procedure picks an ID that wasn't checked for the longest period. There is also a hard-coded period of 20 minutes - the same ID can't be checked more often than every 20 minutes.
CREATE PROCEDURE [dbo].[GetNextIDToCheck]
-- Add the parameters for the stored procedure here
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
BEGIN TRANSACTION;
BEGIN TRY
DECLARE #VarID int = NULL;
DECLARE #VarLockResult int;
EXEC #VarLockResult = sp_getapplock
#Resource = 'SomeUniqueName_app_lock',
#LockMode = 'Exclusive',
#LockOwner = 'Transaction',
#LockTimeout = 60000,
#DbPrincipal = 'public';
IF #VarLockResult >= 0
BEGIN
-- Acquired the lock
-- Find ID that wasn't checked for the longest period
SELECT TOP 1
#VarID = ID
FROM
dbo.MyTable
WHERE
LastCheckStarted <= LastCheckCompleted
-- this ID is not being checked right now
AND LastCheckCompleted < DATEADD(minute, -20, GETDATE())
-- last check was done more than 20 minutes ago
ORDER BY LastCheckCompleted;
-- Start checking
UPDATE dbo.MyTable
SET LastCheckStarted = GETDATE()
WHERE ID = #VarID;
-- There is no need to explicitly verify if we found anything.
-- If #VarID is null, no rows will be updated
END;
-- Return found ID, or no rows if nothing was found,
-- or failed to acquire the lock
SELECT
#VarID AS ID
WHERE
#VarID IS NOT NULL
;
COMMIT TRANSACTION;
END TRY
BEGIN CATCH
ROLLBACK TRANSACTION;
END CATCH;
END
The second procedure is called by an application when it finishes checking the found ID.
CREATE PROCEDURE [dbo].[SetCheckComplete]
-- Add the parameters for the stored procedure here
#ParamID int
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
BEGIN TRANSACTION;
BEGIN TRY
DECLARE #VarLockResult int;
EXEC #VarLockResult = sp_getapplock
#Resource = 'SomeUniqueName_app_lock',
#LockMode = 'Exclusive',
#LockOwner = 'Transaction',
#LockTimeout = 60000,
#DbPrincipal = 'public';
IF #VarLockResult >= 0
BEGIN
-- Acquired the lock
-- Completed checking the given ID
UPDATE dbo.MyTable
SET LastCheckCompleted = GETDATE()
WHERE ID = #ParamID;
END;
COMMIT TRANSACTION;
END TRY
BEGIN CATCH
ROLLBACK TRANSACTION;
END CATCH;
END
It does not work because multiple transactions might first execute the IN clause and find the same set of rows, then update multiple times and overwrite each other.
LukeH's answer is best, accept it.
You can also fix it by adding AND Exported IS NULL to cancel double updates.
Or, make this SERIALIZABLE. This will lead to some blocking and deadlocking. This can safely be handled by timeouts and retry in case of deadlock. SERIALIZABLE is always safe for all workloads but it might block/deadlock more often.

searching through multiple tables with a stored procedures

I have two tables Vehicle and Vehicle return(it was spelled incorrectly in the code),I'm trying to create a stored procedure where I can enter the engine number and it would search through Vehicle and Vehicle return to see if it matches the engine number and the criteria that it is in either of the table but every time only thing that works is if the engine number isn't in either of the tables here is my code
create procedure outbound
(
#eng varchar(25)
)
AS
BEGIN
BEGIN TRAN
DECLARE #eng_num VARCHAR(25)
DECLARE #eng_num2 VARCHAR(25)
/* SELECT #eng_num= Engine_num from Vehicle where Engine_num=#eng and Status=1
SELECT #eng_num2= Engine_num from Vehicle_retuns where Engine_num=#eng
IF(#eng=#eng_num)
begin
UPDATE Vehicle SET Description_of_Vehicle='Vehicle has ben sent to Manufactory',Status=0 where Engine_num=#eng_num
end
ELSE IF(#eng=#eng_num2)
begin
UPDATE Vehicle_retuns SET purpose='Vehicle has ben sent to Manufactory',Status=0 where Engine_num=#eng_num2
end*/ the lines of code that is the error is occuring
ELSE
SELECT 'No such Engine number was found'
IF(##ERROR<>0)
BEGIN
SELECT 'An unexpected error has occur'
ROLLBACK TRANSACTION
RETURN -1
END
COMMIT TRANSACTION
END
Here is a rough sketch of what I think might be required here. There is no need to use an explicit transaction for a single update. Also, I would highly recommend you look at normalizing your messages instead of hard coding those long strings everywhere. Not sure you really mean to set the Description or purpose column depending on the value of status. I would rather see those be foreign keys to a lookup of values but maybe that doesn't work here. I would also recommend you not use column names like "status". Using reserved words is problematic for a number of reasons. Last but not least, you should use try/catch blocks instead of checking ##ERROR.
--EDIT--
Here is some new code that handles the fact that you really do need two updates. I missed this originally.
create procedure outbound
(
#eng varchar(25)
)
AS
BEGIN
IF EXISTS
(
SELECT Engine_num
from Vehicle
where Engine_num = #eng
UNION ALL
SELECT Engine_num
from Vehicle_retuns
where Engine_num = #eng
)
BEGIN
BEGIN TRY
BEGIN TRANSACTION
UPDATE Vehicle
SET Description_of_Vehicle = 'Vehicle has ben sent to Manufactory'
, Status = 0
where Engine_num = #eng
AND Status = 1
UPDATE Vehicle_retuns
SET purpose = 'Vehicle has ben sent to Manufactory'
, Status = 0
where Engine_num = #eng
COMMIT TRANSACTION
END TRY
BEGIN CATCH
SELECT 'An unexpected error has occurred.'
--I would prefer a message here including the error message and error number instead of just "It failed".
END CATCH
END
ELSE
SELECT 'No such Engine number was found'
END

how to handle this data concurrency problem?

I am doing a financial application in which I am expecting a data concurrency issue.
Suppose there is an account ABC which has $500 in it. User from web can transfer these funds to other accounts. This will involve 2 steps 1st checking availability of funds and 2nd transferring. I am making a transaction and doing both acts in it.
Problem is when in a time (say Time1) there are 2 or 3 seprate requests for transferring (say transaction1,transaction2, transaction3) same amount. Now committed available amount is $500. If all translations starts in same time, all will test is amount ($500) available ? which will true and next statement will transfer funds to other account.
I have read about Transaction isolation levels but I couldn't decide which isolation level I should use, actually I am confuse in its understanding. Please help me.
Thanks
The aim is to prevent another process reading the balance but minimise blocking for other users. So use the "table as a queue" type locks thus:
SET XACT_ABORT, NOCOUNT ON;
BEGIN TRY
BEGIN TRANSACTION
SELECT #balance = Balance
FROM SomeTable WITH (ROWLOCK, HOLDLOCK, UPDLOCK)
WHERE Account = 'ABC'
--some checks
UPDATE ...
COMMIT TRANSACTION
END TRY
BEGIN CATCH
...
END CATCH
The alternative is to do it in one, which is more feasible if there is one table involved.
The CROSS JOIN is a test to
SET XACT_ABORT, NOCOUNT ON;
BEGIN TRY
--BEGIN TRANSACTION
UPDATE SomeTable WITH (ROWLOCK, HOLDLOCK, UPDLOCK)
SET Balance = Balance - #request
WHERE
ST.Account = 'ABC' AND Balance > #request;
IF ##ROWCOUNT <> 1
RAISERROR ('Not enough in account', 16, 1);
--COMMIT TRANSACTION
END TRY
BEGIN CATCH
...
END CATCH
In order to avoid withdraws of amounts bigger than the price, you could do this:
update <table>
set amount = amount - #price
where amount >= #price
and account = #account
if ##rowcount = 1 print 'transaction went well' else print 'Insufficient funds'

Resources