I am facing trouble when the following code is called two times almost at the same time.
DECLARE #membershipIdReturn as uniqueidentifier=null
SELECT #membershipIdReturn = MembershipId
FROM [Loyalty].[Membership]
WITH (NOLOCK)
WHERE ContactId = #customerIdFront
AND
IsDeleted = 0
IF (#membershipIdReturn IS NULL)
//InsertStatementHere
The calls are so close (about 3 thousandth of a second), that the second call also enter inside the if statement. Then an unicity failure is lift because this is not supposed to happen.
Is the bug because of the (NOLOCK)? I need it for transaction issues.
Is there any workaround for correcting this behavior ?
Thanks Al
Two options
1.Use unique constraint then put your insert statement in Try Catch block
ALTER TABLE [Loyalty].[Membership]
ADD CONSTRAINT uc_ContactId_IsDeleted UNIQUE(ContactId, IsDeleted)
2.Use Merge with serializable hint. Therefore, there will be no gap between select and insert.
MERGE [Loyalty].[Membership] WITH (SERIALIZABLE) as T
USING [Loyalty].[Membership] as S
ON ContactId = #customerIdFront
AND IsDeleted = 0
WHEN NOT MATCHED THEN
INSERT (MemberName, MemberTel) values ('','');
Related
I would like to know which is more efficient and why.
if not exists (select 1 from table where ID = 101 and TT = 5)
begin
update table
set TT = 5
where ID = 101;
end;
or
update table
set TT = 5
where ID = 101 and TT <> 5;
Assume there is a clustered index on ID (nothing more table used default table creation setting)
WHERE, IF EXISTS and IN all have different performance benefits. I would suggest checking out these two articles.
https://www.sqlshack.com/t-sql-commands-performance-comparison-not-vs-not-exists-vs-left-join-vs-except/
https://sqlchitchat.com/sqldev/tsql/semi-joins-in-sql-server/
SQL Server will generally optimize a non-updating UPDATE to not actually issue any updates. Therefore, with a simple table, you are not going to see much difference.
If you have triggers, they will be fired if the UPDATE statement executes, irrelevant of how many rows are updated.
If the UPDATE statement executes over rows, even if they are modified to the same value, they will appear in the trigger.
If rows are filtered out with a WHERE clause, for example and TT <> 5, then the trigger will fire with 0 rows
rowversion and GENERATED AS columns will be updated regardless.
Clustered key columns will cause a delete and insert of the whole row.
If ALLOW_SNAPSHOT_ISOLATION or READ_COMMITTED_SNAPSHOT are on, even if not being used, then due to the way row-versioning works, an actual update will always be made.
If the IF EXISTS is complex, it still may not be worth doing, but in simple cases it usually is.
Question: What things could cause the following problem?
We have a stored proc that updates or inserts a price record into a table. Very straight forward. However, at intermittent times we get a primary key violation trying to insert a row that already exists. As you see, we are checking for the existence of a record and then updating or inserting as appropriate.
This happens very rarely but the last time it happened we had 18 occurrences in a 3 minute window. We updated a lot more rows than 18 so it is not every time. This SP is called quite often. We check and there was no index maintenance going on at the time. The application that calls this SP just loops through a queue to update/insert these prices and there is only one instance of the application running.
This is running on an 2016 Availability Group with 3 servers.
ALTER PROCEDURE [dbo].[mw_UpdatePrice] #CustTypeID INT
,#ID INT
,#Price MONEY
,#OldPrice MONEY
,#ExpirationDate DATETIME
,#PriceStatusID INT
,#PriceStatusDesc VARCHAR(80)
,#FreeFreightShipviaServiceLevelID INT = NULL
,#FreeFreightShipViaServiceLevelDescription VARCHAR(150) = NULL
,#FreeFreightShipViaServiceLevelRank INT = NULL
AS
BEGIN
SET NOCOUNT ON;
IF EXISTS (
SELECT 1
FROM dbo.Price
WHERE ID = #ID
AND CustTypeID = #CustTypeID
)
BEGIN
UPDATE dbo.Price
SET Price = #Price
,OldPrice = #OldPrice
,ExpirationDate = #ExpirationDate
,PriceStatusID = #PriceStatusID
,PriceStatusDescription = #PriceStatusDesc
,ServiceLevelName = #FreeFreightShipViaServiceLevelDescription
,ServiceLevelId = #FreeFreightShipviaServiceLevelID
,ServiceLevelRank = #FreeFreightShipViaServiceLevelRank
WHERE ID = #ID
AND CustTypeID = #CustTypeID
END
ELSE
BEGIN
INSERT dbo.Price (
ID
,CustTypeID
,Price
,OldPrice
,ExpirationDate
,PriceStatusID
,PriceStatusDescription
,ServiceLevelName
,ServiceLevelID
,ServiceLevelRank
)
VALUES (
#ID
,#CustTypeID
,#Price
,#OldPrice
,#ExpirationDate
,#PriceStatusID
,#PriceStatusDesc
,#FreeFreightShipViaServiceLevelDescription
,#FreeFreightShipviaServiceLevelID
,#FreeFreightShipViaServiceLevelRank
)
END
END
As comments have mentioned, this stored procedure is not safe to call concurrently - it has race conditions for both insert and update scenarios. This can be fixed only by ensuring that each call to the stored procedure is contained in a separate transaction (add begin/end tran inside the SP, or initiate transaction from the application code), and applying HOLDLOCK to the SELECT and INSERT statements:
SELECT 1
FROM dbo.Price WITH (UPDLOCK, HOLDLOCK)
WHERE ID = #ID
AND CustTypeID = #CustTypeID
...
INSERT dbo.Price WITH (HOLDLOCK)
...
Even if this gets refactored to a MERGE statement, HOLDLOCK will still be necessary to prevent the issue. Example here.
HOLDLOCK locks the entire table and will reduce the concurrent throughput of this SP.
Aside from the concurrency problem, another possible cause of PRIMARY KEY violation could occur if ANSI_NULLS is on, potentially allowing for an application logic error where the SP is being called with NULL values for #CustTypeID or #ID, causing the application to behave in a way that wasn't intended. For example, calling with #CustTypeID = NULL and #ID = 1 would always result in an INSERT, which might not be the intended behavior.
Edit:
Added UPDLOCK as per #DavidBrowne
We are using the technique outlined here to generate random record IDs without collisions. In short, we create a randomly-ordered table of every possible ID, and mark each record as 'Taken' as it is used.
I use the following Stored Procedure to obtain an ID:
ALTER PROCEDURE spc_GetId #retVal BIGINT OUTPUT
AS
DECLARE #curUpdate TABLE (Id BIGINT);
SET NOCOUNT ON;
UPDATE IdMasterList SET Taken=1
OUTPUT DELETED.Id INTO #curUpdate
WHERE ID=(SELECT TOP 1 ID FROM IdMasterList WITH (INDEX(IX_Taken)) WHERE Taken IS NULL ORDER BY SeqNo);
SELECT TOP 1 #retVal=Id FROM #curUpdate;
RETURN;
The retrieval of the ID must be an atomic operation, as simultaneous inserts are possible.
For large inserts (10+ million), the process is quite slow, as I must pass through the table to be inserted via a cursor.
The IdMasterList has a schema:
SeqNo (BIGINT, NOT NULL) (PK) -- sequence of ordered numbers
Id (BIGINT) -- sequence of random numbers
Taken (BIT, NULL) -- 1 if taken, NULL if not
The IX_Taken index is:
CREATE NONCLUSTERED INDEX (IX_Taken) ON IdMasterList (Taken ASC)
I generally populate a table with Ids in this manner:
DECLARE #recNo BIGINT;
DECLARE #newId BIGINT;
DECLARE newAdds CURSOR FOR SELECT recNo FROM Adds
OPEN newAdds;
FETCH NEXT FROM newAdds INTO #recNo;
WHILE ##FETCH_STATUS=0 BEGIN
EXEC spc_GetId #newId OUTPUT;
UPDATE Adds SET id=#newId WHERE recNo=#recNo;
FETCH NEXT FROM newAdds INTO #id;
END;
CLOSE newAdds;
DEALLOCATE newAdds;
Questions:
Is there any way I can improve the SP to extract Ids faster?
Would a conditional index improve peformance (I've yet to test, as
IdMasterList is very big)?
Is there a better way to populate a table with these Ids?
As with most things in SQL Server, if you are using cursors, you are doing it wrong.
Since you are using SQL Server 2012, you can use a SEQUENCE to keep track of what random value you already used and effectively replace the Taken column.
CREATE SEQUENCE SeqNoSequence
AS bigint
START WITH 1 -- Start with the first SeqNo that is not taken yet
CACHE 1000; -- Increase the cache size if you regularly need large blocks
Usage:
CREATE TABLE #tmp
(
recNo bigint,
SeqNo bigint
)
INSERT INTO #tmp (recNo, SeqNo)
SELECT recNo,
NEXT VALUE FOR SeqNoSequence
FROM Adds
UPDATE Adds
SET id = m.id
FROM Adds a
INNER JOIN #tmp tmp ON a.recNo = tmp.recNo
INNER JOIN IdMasterList m ON tmp.SeqNo = m.SeqNo
SEQUENCE is atomic. Subsequent calls to NEXT VALUE FOR SeqNoSequence are guaranteed to return unique values, even for parallel processes. Note that there can be gaps in SeqNo, but it's a very small trade off for the huge speed increase.
Put a PK inden of BigInt on each table
insert into user (name)
values ().....
update user set = user.ID = id.ID
from id
left join usr
on usr.PK = id.PK
where user.ID = null;
one
insert into user (name) value ("justsaynotocursor");
set #PK = select select SCOPE_IDENTITY();
update user set ID = (select ID from id where PK = #PK);
Few ideas that came to my mind:
Try if removing the top, inner select etc. helps to improve the performance of the ID fetching (look at statistics io & query plan):
UPDATE top(1) IdMasterList
SET #retVal = Id, Taken=1
WHERE Taken IS NULL
Change the index to be a filtered index, since I assume you don't need to fetch numbers that are taken. If I remember correctly, you can't do this for NULL values, so you would need to change the Taken to be 0/1.
What actually is your problem? Fetching single IDs or 10+ million IDs? Is the problem CPU / I/O etc. caused by the cursor & ID fetching logic, or are the parallel processes being blocked by other processes?
Use sequence object to get the SeqNo. and then fetch the Id from idMasterList using the value returned by it. This could work if you don't have gaps in IdMasterList sequences.
Using READPAST hint could help in blocking, for CPU / I/O issues, you should try to optimize the SQL.
If the cause is purely the table being a hotspot, and no other easy solutions seem to help, split it into several tables and use some kind of simple logic (even ##spid, rand() or something similar) to decide from which table the ID should be fetched. You would need more checking if all tables have free numbers, but it shouldn't be that bad.
Create different procedures (or even tables) to handle fetching of single ID, hundreds of IDs and millions of IDs.
I am trying to add a check constraint which verity if after an update the new value (which was inserted) is greater than old values which is already stored in table.
For example i have a "price" column which already stores value 100, if the update comes with 101 the is ok, if 99 comes then my constraint should reject the update process. Can this behavior be achieved using check constraints or should i try to use triggers or functions ?
Please advice me regarding this...
Thanks,
Mircea
Check constraints can't access the previous value of the column. You would need to use a trigger for this.
An example of such a trigger would be
CREATE TRIGGER DisallowPriceDecrease
ON Products
AFTER UPDATE
AS
IF NOT UPDATE(price)
RETURN
IF EXISTS(SELECT * FROM inserted i
JOIN deleted d
ON i.primarykey = d.primarykey
AND i.price< d.price)
BEGIN
ROLLBACK TRANSACTION
RAISERROR('Prices may not be decreased', 16, 1)
END
Triggers start as a quick fix, and end with a maintenance nightmare. The two big problems with triggers are:
It's hard to see when a trigger is called. You can easily write an update statement without being aware that a trigger will run.
When triggers start triggering other triggers, it becomes hard to tell what will happen.
As an alternative, wrap access to the table in a stored procedure. For example:
create table TestTable (productId int, price numeric(6,2))
insert into TestTable (productId, price) values (1,5.0)
go
create procedure dbo.IncreasePrice(
#productId int,
#newPrice numeric(6,2))
with execute as owner
as
begin
update dbo.TestTable
set price = #newPrice
where productId = #productId
and price <= #newPrice
return ##ROWCOUNT
end
go
Now if you try to decrease the price, the procedure will fail and return 0:
exec IncreasePrice 1, 4.0
select * from TestTable --> 1, 5.00
exec IncreasePrice 1, 6.0
select * from TestTable --> 1, 6.00
Stored procedures are pretty easy to read. Compared to triggers, they'll cause you a lot less headaches. You can enforce the use of stored procedures by not giving any user the right to UPDATE tables. That's a good practice anyway.
I'm looking for the best way to go about adding a constraint to a table that is effectively a unique index on the relationship between the record and the rest of the records in that table.
Imagine the following table describing the patrols of various guards (from the previous watchman scenario)
PK PatrolID Integer
FK GuardID Integer
Starts DateTime
Ends DateTime
We start with a constraint specifying that the start and end times must be logical:
Ends >= Starts
However I want to add another logical constraint: A specific guard (GuardID) cannot be in two places at the same time, meaning that for any record the period specified by Start/Ends should not overlap with the period defined for any other patrol by the same guard.
I can think of two ways of trying to approach this:
Create an INSTEAD OF INSERT trigger. This trigger would then use cursors to go through the INSERTED table, checking each record. If any record conflicted with an existing record, an error would be raised. The two problems I have with this approach are: I dislike using cursors in a modern version of SQL Server, and I'm not sure how to go about implimenting the same logic for UPDATEs. There may also be the complexity of records within INSERTED conflicting with each other.
The second, seemingly better, approach would be to create a CONSTRAINT that calls a user defined function, passing the PatrolID, GuardID, Starts and Ends. The function would then do a WHERE EXISTS query checking for any records that overlap the GuardID/Starts/Ends parameters that are not the original PatrolID record. However I'm not sure of what potential side effects this approach might have.
Is the second approach better? Does anyone see any pitfalls, such as when inserting/updating multiple rows at once (here I'm concerned because rows within that group could conflict, meaning the order they are "inserted" makes a difference). Is there a better way of doing this (such as some fancy INDEX trick?)
Use an after trigger to check that the overlap constraint has not been violated:
create trigger Patrol_NoOverlap_AIU on Patrol for insert, update as
begin
if exists (select *
from inserted i
inner join Patrol p
on i.GuardId = p.GuardId
and i.PatrolId <> p.PatrolId
where (i.Starts between p.starts and p.Ends)
or (i.Ends between p.Starts and p.Ends))
rollback transaction
end
NOTE: Rolling back a transaction within a trigger will terminate the batch. Unlike a normal contraint violation, you will not be able to catch the error.
You may want a different where clause depending on how you define the time range and overlap. For instance if you want to be able to say Guard #1 is at X from 6:00 to 7:00 then Y 7:00 to 8:00 the above would not allow. You would want instead:
create trigger Patrol_NoOverlap_AIU on Patrol for insert, update as
begin
if exists (select *
from inserted i
inner join Patrol p
on i.GuardId = p.GuardId
and i.PatrolId <> p.PatrolId
where (p.Starts <= i.Starts and i.Starts < p.Ends)
or (p.Starts <= i.Ends and i.Ends < p.Ends))
rollback transaction
end
Where Starts is the time the guarding starts and Ends is the infinitesimal moment after guarding ends.
The simplest way would be to use a stored procedure for the inserts. The stored procedure can do the insert in a single statement:
insert into YourTable
(GuardID, Starts, Ends)
select #GuardID, #Starts, #Ends
where not exists (
select *
from YourTable
where GuardID = #GuardID
and Starts <= #Ends
and Ends >= #Start
)
if ##rowcount <> 1
return -1 -- Failure
In my experience triggers and constraints with UDF's tend to become very complex. They have side effects that can require a lot of debugging to figure out.
Stored procedures just work, and they have the added advantage that you can deny INSERT permissions to clients, giving you fine-grained control over what enters your database.
CREATE TRIGGER [dbo].[emaill] ON [dbo].[email]
FOR INSERT
AS
BEGIN
declare #email CHAR(50);
SELECT #email=i.email from inserted i;
IF #email NOT LIKE '%_#%_.__%'
BEGIN
print 'Triggered Fired';
Print 'Invalid Emaill....';
ROLLBACK TRANSACTION
END
END
Can be done with constraints too:
http://www2.sqlblog.com/blogs/alexander_kuznetsov/archive/2009/03/08/storing-intervals-of-time-with-no-overlaps.aspx