Is the following stored procedure code robust for multi-user application. It is working fine. I was wondering if there is any better way to do it and if there is any performance issues.
The proc has three sql statements together like 1.Updating hardware status in Allocation Table 2. Calculating Next appropriate Primary Key value for new record to be inserted in DEFECT_LOG table 3.Inserting the values into DEFECT_LOG table. I am also using a variable to return 1 if the transaction was successful.
ALTER PROCEDURE spCreateDefective
(
#alloc_ID nvarchar(50),
#cur_date datetime,
#problem_desc nvarchar(MAX),
#got_defect_date datetime,
#trans_status tinyint OUTPUT --Used to check transaction status
)
AS
/* Transaction Maintainer Statements */
BEGIN TRAN transac1
SET XACT_ABORT ON
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE
/* Declaration Area */
DECLARE #temp nvarchar(10)
DECLARE #def_ID nvarchar(20)
/* Updating Inventory Status to 'Defective' and Updating Release Date to current System date */
UPDATE INV_Allocation_DB
SET INV_STATUS = 'Defective' , RELEASE_DATE=#cur_date
WHERE INV_ID=#alloc_ID
/* Calculates New Primary Key for the DEFECT_LOG Table */
-- Returns the largest number or 1 if no records are present in the table
SELECT #temp=COALESCE(CONVERT(int,SUBSTRING(MAX(DEFECT_ID),5,5))+1,1) FROM DEFECT_LOG
SET #def_ID = 'DEF_'+ RIGHT(replicate('0',5)+ convert(varchar(5),#temp),5)
/* Insert statement for inserting data into DEFECT_LOG */
INSERT INTO DEFECT_LOG (DEFECT_ID,INV_ID,PROB_DESC,GOT_DEFECT_DATE)
VALUES(#def_ID,#alloc_ID,#problem_desc,#got_defect_date)
SET #trans_status = 1
COMMIT TRAN transac1
/* Returns 1 if transaction successful */
RETURN #trans_status
Using a SERIALIZABLE transaction level is not recommended unless you absolutely have to have one. It will increase the likelihood of blocking and decrease throughput.
You seem to be using it in order to guarantee a unique DEFECT_ID? Why not use an IDENTITY column for DEFECT_ID instead?
Personally I would use an IDENTITY field as a true primary key, but then have an additional column with the alphanumeric identifier. (Possibly as a persisted computed field)
This should reduce the number of issues with regards to concurrency.
Related
I use stored procedures to manage a warehouse. PDA scanners scan the added stock and send it in bulk (when plugged back) to a SQL database (SQL Server 2016).
The SQL database is fairly remote (other country), so there's sometimes delay in some queries, but this particular one is problematic: even if the stock table is fine, I had some problems with updating the occupancy of the warehouse spots. The PDA tracks the added items in every spot as a SMALLINT, then send back this value to the stored procedure below.
PDA "send_spots" query:
SELECT spot, added_items FROM spots WHERE change=1
Stored procedure:
CREATE PROCEDURE [dbo].[update_spots]
#spot VARCHAR(10),
#added_items SMALLINT
AS
BEGIN
BEGIN TRAN
UPDATE storage_spots
SET remaining_capacity = remaining_capacity - #added_items
WHERE storage_spot=#spot
IF ##ROWCOUNT <> 1
BEGIN
ROLLBACK TRAN
RETURN - 1
END
ELSE
BEGIN
COMMIT TRAN
RETURN 0
END
END
GO
If the remaining_capacity value is 0, the PDAs can't add more items to it on next round. But with this process, I had negative values because the query allegedly ran two times (so subtracted #added_items two times).
Is there a way for that to be possible? How could I fix it? From what I understood the transaction should be cancelled (ROLLBACK) if the affected rows are != 1, but maybe that's something else.
EDIT: current solution with the help of #Zero:
CREATE PROCEDURE [dbo].[update_spots]
#spot VARCHAR(10),
#added_racks SMALLINT
AS
BEGIN
-- Recover current capacity of the spot
DECLARE #orig_capacity SMALLINT
SELECT TOP 1
#orig_capacity = remaining_capacity
FROM storage_spots
WHERE storage_spot=#spot
-- Test if double is present in logs by comparing dates (last 10 seconds)
DECLARE #is_double BIT = 0
SELECT #is_double = CASE WHEN EXISTS(SELECT *
FROM spot_logs
WHERE log_timestamp >= dateadd(second, -10, getdate()) AND storage_spot=#spot AND delta=#added_racks)
THEN 1 ELSE 0 END
BEGIN
BEGIN TRAN
UPDATE storage_spots
SET remaining_capacity= #orig_capacity - #added_racks
WHERE storage_spot=#spot
IF ##ROWCOUNT <> 1 OR #is_double <> 0
-- If double, rollback UPDATE
ROLLBACK TRAN
ELSE
-- If no double, commit UPDATE
COMMIT TRAN
-- write log
INSERT INTO spot_logs
(storage_spot, former_capacity, new_capacity, delta, log_timestamp, double_detected)
VALUES
(#spot, #orig_capacity, #orig_capacity-#added_racks, #added_racks, getdate(), #is_double)
END
END
GO
I was thinking about possible causes (and a way to trace them) and then it hit me - you have no value validation!
Here's a simple example to illustrate the problem:
Spot | capacity
---------------
x1 | 1
Update spots set capacity = capacity - 2 where spot = 'X1'
Your scanner most likely gave you higher quantity than you had capacity to take in.
I am not sure how your business logic goes, but you need to perform something in lines of
Update spots set capacity = capacity - #added_items where spot = 'X1' and capacity >= #added_items
if ##rowcount <> 1;
rollback;
EDIT: few methods to trace your problem without implementing validation:
Create a logging table (with timestamp, user id (user, that is connected to DB), session id, old value, new value, delta value (added items).
Option 1:
Log all updates that change value from positive to negative (at least until you figure out the problem).
The drawback of this option is that it will not register double calls that do not result in a negative capacity.
Option 2: (logging identical updates):
Create script that creates a global temporary table and deletes records, from that table timestamps older than... let's say 10 minutes once every minute or so (play around with numbers).
This temporary table should hold the data passed to your update statement so 'spot', 'added_items' + 'timestamp' (for tracking).
Now the crucial part: When you call your update statement check if a similar record exists in the temporary table (same spot and added_items and where current timestamp is between timestamp and [timestamp + 1 second] - again play around with numbers). If a record like that exist log that update, if not add it to temporary table.
This will register identical updates that go within 1 second of each other (or whatever time-frame you choose).
I found here an alternative SQL query that does the update the way I need, but with a temporary value using DECLARE. Would it work better in my case, or is my initial query correct?
Initial query:
UPDATE storage_spots
SET remaining_capacity = remaining_capacity - #added_items
WHERE storage_spot=#spot
Alternative query:
DECLARE #orig_capacity SMALLINT
SELECT TOP 1 #orig_capacity = remaining_capacity
FROM storage_spots
WHERE spot=#spot
UPDATE Products
SET remaining_capacity = #orig_capacity - #added_items
WHERE spot=#spot
Also, should I get rid of the ROLLBACK/COMMIT instructions?
CAVEAT: This is legacy code, we know we should be using an actual SQL sequence, the goal here is to understand why one method is providing different results from another because we can't just change to a sequence on the fly at this time.
ISSUE: We are using a single valued table as a sequence object (created pre-2012) to generate and return what needs to be a guaranteed unique, incrementing number. The stored procedure in place works but is churning high CPU and causing severe blocking under high load; The remediation looks as though it should work but does not. CPU and blocking is relieved with new code but we can't seem to be able to prevent dirty reads and this is resulting in duplication.
Here's how the table looks:
if OBJECT_ID('dbo.Sequence') is not null
drop table dbo.Sequence;
create table dbo.Sequence (number int Primary Key);
insert into dbo.Sequence values (1);
GO
This stored proc uses serializable locking and gets accurate results, but it doesn't perform well due to the blocking:
CREATE OR ALTER PROC dbo.usp_GetSequence_Serializable #AddSomeNumber INT = 1 AS
BEGIN
declare #return int; --value to be returned and declared within sproc
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE
BEGIN TRANSACTION
Select #return = number from dbo.Sequence with (updlock) --get initial value plus one
Update dbo.Sequence set number = #return + #AddSomeNumber --increment it
COMMIT TRANSACTION
Select #return + #AddSomeNumber as nextid
END
GO
Here's our faster version that avoids serializable, but it's getting duplicate values back from time to time:
CREATE OR ALTER PROC dbo.usp_GetSequence_DefaultIsolationLevel #AddSomeNumber INT = 1 AS
BEGIN
declare #return int; --value to be returned and declared within sproc
update dbo.Sequence set #return = number = number + #AddSomeNumber --we tried interchanging various locking hints with no change in results (likely due to an exclusive being taken already)
Select #return as nextid
END
GO
What can we do to get the faster performance of the second (non-serializable) proc, while avoiding duplicate generated IDs?
In largish databases times will arise when you are unable to avoid deadlocks in your code. The database will not always look ahead on all things to avoid contention. In these rare and special cases you can make use of the high performance locking used by SQL Server itself --> SP_GETAPPLOCK(). You can structure your code in such a way to give exclusive access to a critical section in your stored procedure and serialize access to that. SP_GETAPPLOCK is definitely not a bad thing when all traditional lock hints fail.
DECLARE #MyTimeoutMiliseconds INT =5000--Wait only five seconds max then timeout
BEGIN TRAN
EXEC #LockRequestResult=SP_GETAPPLOCK 'MyCriticalWork','Exclusive','Transaction',#MyTimeoutMiliseconds
IF(#LockRequestResult>=0)BEGIN
/*
DO YOUR CRITICAL READS AND WRITES HERE
You may prefer try finally to release
*/
COMMIT TRAN -- <--Releases the lock!
END ELSE
ROLLBACK TRAN --<--Releases the lock!
I have three tables and three stored procedures respectively to insert/update records in these tables. The first table has a primary key column, RecNo, which is auto-generated.
Around 1000 users are entering records in these tables simultaneously from different geographic locations. I am noticing that sometimes inserts in the second or third table get missed even when inserts were successfully done and no warning was generated.
I want to know how the auto-generated primary key column handles concurrent issues. Do I need to set isolation level to SERIALIZABLE on top of each stored procedure?
I am using SQL Server 2008 R2 Express with default isolation level, i.e., READ COMMITTED.
One of my stored procedure looks like:
ALTER PROCEDURE [dbo].[pheSch_CreateOrUpdateTubewellDetails]
-- Add the parameters for the stored procedure here
#TwTaskFlag nvarchar(6),
#TwParameterID bigint,
#SerialNumber bigint,
#TotalNum int,
#TwType nvarchar(50),
#Depth nvarchar(60),
#Diameter nvarchar(60),
#WaterCapacity nvarchar(60),
#PS nvarchar(15),
#PSNum int,
#PSType nvarchar(60),
#Remarks nvarchar(80)
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
-- Insert statements for procedure here
BEGIN
UPDATE tw_details
SET
TotalNum = #TotalNum,
TwType = #TwType,
Depth = #Depth,
Diameter = #Diameter,
WaterCapacity = #WaterCapacity,
PS = #PS,
PSNum = #PSNum,
PSType = #PSType,
Remarks = #Remarks
WHERE twpid = #TwParameterID;
END
END
You need not change the isolation level,
the Identity column is well suited for concurrent inserts.
IF
you have no any Triggers attached to the table - then show all the details
BUT
you noticed the INSERTS - i do not see any of them here
I have a system which requires I have IDs on my data before it goes to the database. I was using GUIDs, but found them to be too big to justify the convenience.
I'm now experimenting with implementing a sequence generator which basically reserves a range of unique ID values for a given context. The code is as follows;
ALTER PROCEDURE [dbo].[Sequence.ReserveSequence]
#Name varchar(100),
#Count int,
#FirstValue bigint OUTPUT
AS
BEGIN
SET NOCOUNT ON;
-- Ensure the parameters are valid
IF (#Name IS NULL OR #Count IS NULL OR #Count < 0)
RETURN -1;
-- Reserve the sequence
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE;
BEGIN TRANSACTION
-- Get the sequence ID, and the last reserved value of the sequence
DECLARE #SequenceID int;
DECLARE #LastValue bigint;
SELECT TOP 1 #SequenceID = [ID], #LastValue = [LastValue]
FROM [dbo].[Sequences]
WHERE [Name] = #Name;
-- Ensure the sequence exists
IF (#SequenceID IS NULL)
BEGIN
-- Create the new sequence
INSERT INTO [dbo].[Sequences] ([Name], [LastValue])
VALUES (#Name, #Count);
-- The first reserved value of a sequence is 1
SET #FirstValue = 1;
END
ELSE
BEGIN
-- Update the sequence
UPDATE [dbo].[Sequences]
SET [LastValue] = #LastValue + #Count
WHERE [ID] = #SequenceID;
-- The sequence start value will be the last previously reserved value + 1
SET #FirstValue = #LastValue + 1;
END
COMMIT TRANSACTION
END
The 'Sequences' table is just an ID, Name (unique), and the last allocated value of the sequence. Using this procedure I can request N values in a named sequence and use these as my identifiers.
This works great so far - it's extremely quick since I don't have to constantly ask for individual values, I can just use up a range of values and then request more.
The problem is that at extremely high frequency, calling the procedure concurrently can sometimes result in a deadlock. I have only found this to occur when stress testing, but I'm worried it'll crop up in production. Are there any notable flaws in this procedure, and can anyone recommend any way to improve on it? It would be nice to do with without transactions for example, but I do need this to be 'thread safe'.
MS themselves offer a solution and even they say it locks/deadlocks.
If you want to add some lock hints then you'd reduce concurrency for your high loads
Options:
You could develop against the "Denali" CTP which is the next release
Use IDENTITY and the OUTPUT clause like everyone else
Adopt/modify the solutions above
On DBA.SE there is "Emulate a TSQL sequence via a stored procedure": see dportas' answer which I think extends the MS solution.
I'd recommend sticking with the GUIDs, if as you say, this is mostly about composing data ready for a bulk insert (it's simpler than what I present below).
As an alternative, could you work with a restricted count? Say, 100 ID values at a time? In that case, you could have a table with an IDENTITY column, insert into that table, return the generated ID (say, 39), and then your code could assign all values between 3900 and 3999 (e.g. multiply up by your assumed granularity) without consulting the database server again.
Of course, this could be extended to allocating multiple IDs in a single call - provided that your okay with some IDs potentially going unused. E.g. you need 638 IDs - so you ask the database to assign you 7 new ID values (which imply that you've allocated 700 values), use the 638 you want, and the remaining 62 never get assigned.
Can you get some kind of deadlock trace? For example, enable trace flag 1222 as shown here. Duplicate the deadlock. Then look in the SQL Server log for the deadlock trace.
Also, you might inspect what locks are taken out in your code by inserting a call to exec sp_lock or select * from sys.dm_tran_locks immediately before the COMMIT TRANSACTION.
Most likely you are observing a conversion deadlock. To avoid them, you want to make sure that your table is clustered and has a PK, but this advice is specific to 2005 and 2008 R2, and they can change the implementation, rendering this advice useless. Google up "Some heap tables may be more prone to deadlocks than identical tables with clustered indexes".
Anyway, if you observe an error during stress testing, it is likely that sooner or later it will occur in production as well.
You may want to use sp_getapplock to serialize your requests. Google up "Application Locks (or Mutexes) in SQL Server 2005". Also I described a few useful ideas here: "Developing Modifications that Survive Concurrency".
I thought I'd share my solution. I doesn't deadlock, nor does it produce duplicate values. An important difference between this and my original procedure is that it doesn't create the queue if it doesn't already exist;
ALTER PROCEDURE [dbo].[ReserveSequence]
(
#Name nvarchar(100),
#Count int,
#FirstValue bigint OUTPUT
)
AS
BEGIN
SET NOCOUNT ON;
IF (#Count <= 0)
BEGIN
SET #FirstValue = NULL;
RETURN -1;
END
DECLARE #Result TABLE ([LastValue] bigint)
-- Update the sequence last value, and get the previous one
UPDATE [Sequences]
SET [LastValue] = [LastValue] + #Count
OUTPUT INSERTED.LastValue INTO #Result
WHERE [Name] = #Name;
-- Select the first value
SELECT TOP 1 #FirstValue = [LastValue] + 1 FROM #Result;
END
What I am looking to avoid concurrency in my stored procedure
here is my script, I am trying SET TRANSACTION ISOLATION LEVEL SERIALIZABLE;
ALTER proc [dbo].[SP_GenerateNextReportID]
#type nvarchar(255), #identity int output
AS BEGIN
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE;
declare #id int;
set #id = IsNull((select LastUsedIdentity from ReportProcessGenerator where Type = #type), 0)
if(#id =0)
insert ReportProcessGenerator values(#type, #id +1)
else
update ReportProcessGenerator set LastUsedIdentity = #id +1 where Type = #type
set #identity = #id +1
END
Not sure is this a right way or not?
If you have a UNIQUE index or a PRIMARY KEY on ReportProcessGenerator.Type, then your stored procedures will not be able to modify the record for the same type concurrently.
Note that you should use SELECT FOR UPDATE or OUTPUT clause to avoid deadlocks, as #Martin points out. With SERIALIZABLE, concurrent SELECT queries won't lift the shared locks which the UPDATE queries will not later be able to upgrade.
However, why would you want to maintain separate per-type identities? Usually, one identity which is unique across types is as good as multiple ones unique within types, the former being much easier to maintain.