Inserting a series of records from my listview to my SQL Server table using a stored procedure. However I get a deadlock like the image below
This is my procedure:
declare #debtorCollectorHistoryId bigint
set #debtorCollectorHistoryId = (Select top 1 id
from FVOfficer with(nolock)
where debtorid = #debtorid
order by id desc)
if #currentFvCollectorid is null
set #currentFvCollectorid = (SELECT FvCollectorID
from Debtor with(nolock)
where id = #debtorid)
if #assignon is null
set #assignon = GETDATE()
print #debtorCollectorHistoryId
print #currentFvCollectorid
print #newFvCollectorid
if #currentFvCollectorid <> #newFvCollectorid
BEGIN
if #currentFvCollectorid <> 0
Begin
update FVOfficer
set terminateon = GetDate()
where id = #debtorCollectorHistoryId
End
Begin
--create new Collector History for new collector
--NOTE: CurrentCollectorID is a previous collectorID for new Collector History
insert into FVOfficer (debtorid, prevCollectorID, collectorid, assignon, terminateon, loginID)
values (#debtorid,#currentFvCollectorid, #newFvCollectorid, GetDate(), null, #loginID)
print' create new Collector History for new FV collector'
exec dbo.[ValidatePrevFVCollectorByDebtorID] #debtorid
end
begin
print #debtorid
update Debtor set FVCollectorid = #newFvCollectorid where id = #debtorid
end
From my snapshot , i am assuming the deadlock happens on the table 'FVOfficer' , but my currentFvCollectorid is always 0 and therefore no update statement is performed . However, i pass my #debtorid to the procedure [ValidatePrevCollectorByDebtorID] which perform some validation with a cursor on a different table not FVOfficer table. Could it be that the parameter #debtorid is locked by the second procedure and while its required in the insert ? if so please how do i resolve this.
When i check my table FVOfficer , i noticed it has no primary key or any index, I index the id column which is an identity(1,1) column . I created a clustered index on the id and this does not solve my problem. Please how do i proceed with this ?
Update
Below is my [ValidatePrevFVCollectorByDebtorID] please does it have anything to do with the issue ?
DECLARE #ID AS BIGINT
--LOOP ALL THE DEBTOR COLLECTOR HISTORY ID
----------------------------------------------------------------------------
DECLARE LOOPDEBTOR_CURSOR CURSOR
FOR SELECT ID FROM DEBTOR WITH(NOLOCK) WHERE ID = #REFDEBTORID
--WHERE COLLECTORID > 0
OPEN LOOPDEBTOR_CURSOR
FETCH NEXT FROM LOOPDEBTOR_CURSOR INTO #ID
WHILE ##FETCH_STATUS = 0
BEGIN
DECLARE #COLLECTORYHISTORYID AS BIGINT
DECLARE #ASSIGNON AS DATETIME
DECLARE #DEBTORID AS BIGINT
DECLARE #COLLECTORID AS BIGINT
DECLARE #PREVCOLLECTORID AS BIGINT
--GET THE 1ST HISTORY ID BASE ON DEBTORID
---------------------------------------------------------------------------
DECLARE #FIRSTID AS BIGINT
SET #FIRSTID = (SELECT TOP 1 ID FROM FVOFFICER WITH(NOLOCK)
WHERE DEBTORID = #ID ORDER BY ID )
-- LIST ALL THE DEBTOR COLLECTOR HISTORY BASE ON DEBTORID = #ID
---------------------------------------------------------------------------
DECLARE LOOPHISTORY_CURSOR CURSOR
FOR SELECT ID,DEBTORID,COLLECTORID,ASSIGNON FROM FVOFFICER
WHERE DEBTORID = #ID ORDER BY ID ASC
OPEN LOOPHISTORY_CURSOR
FETCH NEXT FROM LOOPHISTORY_CURSOR INTO
#COLLECTORYHISTORYID,#DEBTORID,#COLLECTORID,#ASSIGNON
WHILE ##FETCH_STATUS = 0
BEGIN
--FIX PREV COLLECTOR ID VALUE
IF #FIRSTID != #COLLECTORYHISTORYID
UPDATE FVOFFICER SET PREVCOLLECTORID = #PREVCOLLECTORID
WHERE ID = #COLLECTORYHISTORYID
SET #PREVCOLLECTORID = #COLLECTORID
FETCH NEXT FROM LOOPHISTORY_CURSOR INTO
#COLLECTORYHISTORYID,#DEBTORID,#COLLECTORID,#ASSIGNON
END
CLOSE LOOPHISTORY_CURSOR
DEALLOCATE LOOPHISTORY_CURSOR
FETCH NEXT FROM LOOPDEBTOR_CURSOR INTO #ID
END
CLOSE LOOPDEBTOR_CURSOR
DEALLOCATE LOOPDEBTOR_CURSOR
Sorry i posted the wrong proc as my 'ValidatePrevFVCollectorByDebtorID'. After careful deliberation . I understand that the lock happened in the procedure ValidatePrevFVCollectorByDebtorID at
DECLARE LOOPHISTORY_CURSOR CURSOR
FOR SELECT ID,DEBTORID,COLLECTORID,ASSIGNON FROM FVOFFICER
WHERE DEBTORID = #ID ORDER BY ID ASC
OPEN LOOPHISTORY_CURSOR
The debtorid was locked above. Since i am executing a bulk insert in a loop, i new insert instance was called , however the debtorid was locked on the previous task/cursor. First
1 I added the keyword with(nolock) at the select statement like below
SELECT ID,DEBTORID,COLLECTORID,ASSIGNON FROM FVOFFICER with(nolock)
WHERE DEBTORID = #ID ORDER BY ID ASC
Then i added a non-clustered index on the column debtorid on the table FVOfficer to enhance the speed of my select statements. After i run, the deadlock didn't happened .
Related
Azure SQL Server - I have an inherited stored procedure which runs asynchronously, triggered by an Azure Service Fabric service which runs infinitely:
PROCEDURE Sources.ForIndexing
(#SourceId BIGINT)
AS
BEGIN
SET NOCOUNT ON
DECLARE #BatchId uniqueidentifier
SELECT #BatchId = CaptureBatch
FROM [Data].[Raw]
WHERE CaptureId = (SELECT MAX(CaptureId)
FROM [Data].[Raw]
WHERE SourceId = #SourceId)
UPDATE [Data].[Raw]
SET [Status] = 501
WHERE SourceId = #SourceId
AND CaptureBatch = #BatchId
END
GO
In this Data.Raw table CaptureId is the Primary Key and is auto-incrementing. The records in this table are grouped by SourceId and CaptureBatch. One SourceId can have several CaptureBatch's. The first part of this procedure finds the latest CaptureBatch group by looking at the MAX CaptureId of a given SourceId. The UPDATE statement then sets the Status column of those records to 501.
What I need to do is add a condition to the stored procedure where, after the SELECT statement runs, says if the Status column of any given record over which this procedure iterates has a value of 1, do not execute the UPDATE statement on that record.
I thought it might be as simple as modifying the SELECT part to say:
WHERE CaptureId = (SELECT MAX(CaptureId)
FROM [Data].[Raw]
WHERE SourceId = #SourceId
AND Status <> 1)
But I believe that's only going to select a Status that's not 1 for that one record which contains the MAX CaptureId, correct? I may be overthinking this, but it seems I need some kind of IF statement added to this.
SELECT TOP (1)
#BatchId = r.CaptureBatch
FROM [Data].[Raw] r
WHERE r.SourceId = #SourceId
ORDER BY r.CaptureId DESC
UPDATE r SET
[Status] = 501
FROM [Data].[Raw] r
WHERE r.SourceId = #SourceId
AND r.CaptureBatch = #BatchId
AND r.Status <> 1
IF (SELECT count(CaptureId)
FROM [Data].[Raw]
WHERE SourceId = #SourceId and Status = 1) > 0 BEGIN
UPDATE [Data].[Raw]
SET [Status] = 501
WHERE SourceId = #SourceId
AND CaptureBatch = #BatchId
END
I have a code below that should insert records into the table but unfortunately this code foes not work in case multiple records are inserted or updated or deleted. How should I rewrite the code for procedure to loop through all the inserted / deleted records? And I do need to use that stored procedure with Input parameters (not just simple insert into ... select ... from ...)
IF EXISTS (SELECT * FROM MyDB.sys.triggers WHERE object_id = OBJECT_ID(N'[dbo].[MyTable_DEL_UPD_INS]'))
DROP TRIGGER [dbo].[MyTable_DEL_UPD_INS]
GO
CREATE TRIGGER [dbo].[MyTable_DEL_UPD_INS]
ON [MyDB].[dbo].[MyTable]
AFTER DELETE, UPDATE, INSERT
NOT FOR REPLICATION
AS
BEGIN
DECLARE #PKId INT,
#Code VARCHAR(5),
#AuditType VARCHAR(10)
SET #Code = 'TEST'
IF EXISTS (SELECT * FROM deleted d)
AND NOT EXISTS (SELECT * FROM inserted i)
BEGIN
SELECT TOP 1
#PKId = d.[MyTable_PK],
#AuditType = 'DELETE'
FROM
deleted d WITH (NOLOCK)
IF #PKId IS NOT NULL
AND #Code IS NOT NULL
EXEC MyDB.[dbo].[SP_Audit] #PKId, #Code, #AuditType
END
IF EXISTS (SELECT * FROM deleted d)
AND EXISTS (SELECT * FROM inserted i)
BEGIN
SELECT TOP 1
#PKId = d.[MyTable_PK],
#AuditType = 'UPDATE'
FROM
deleted d WITH (NOLOCK)
IF #PKId IS NOT NULL
AND #Code IS NOT NULL
EXEC MyDB.[dbo].[SP_Audit] #PKId, #Code, #AuditType
END
IF NOT EXISTS (SELECT * FROM deleted d)
AND EXISTS (SELECT * FROM inserted i)
BEGIN
SELECT TOP 1
#PKId = d.[MyTable_PK],
#AuditType = 'INSERT'
FROM
deleted d WITH (NOLOCK)
IF #PKId IS NOT NULL
AND #Code IS NOT NULL
EXEC MyDB.[dbo].[SP_Audit] #PKId, #Code, #AuditType
END
END
GO
ALTER TABLE [MyDB].[dbo].[MyTable] ENABLE TRIGGER [MyTable_DEL_UPD_INS]
You should avoid using loops in triggers.
Triggers should be as quick to run as possible, since SQL Server will not return control to whatever statement that fired the trigger until the trigger is completed.
So instead of a loop, you should modify your SP_Audit procedure to work with multiple records instead of a single one.
usually, this is easily be done using a table valued parameter.
If you could post the SP_Audit as well, we could give you a complete solution.
Since you didn't post it, you can use these guidelines as a start:
First, you create a user defined table type:
CREATE TYPE dbo.Ids AS TABLE
(
Id int NOT NULL PRIMARY KEY
)
GO
Then, you create the procedure to use it:
CREATE PROCEDURE [dbo].[STP_Audit_MultipleRecords]
(
#IDs dbo.Ids readonly,
#Code CHAR(4),
#AuditType CHAR(6)
)
AS
-- Implementation here
GO
Last, your write your trigger like this:
CREATE TRIGGER [dbo].[MyTable_DEL_UPD_INS]
ON [MyDB].[dbo].[MyTable]
AFTER DELETE, UPDATE, INSERT
NOT FOR REPLICATION
AS
BEGIN
DECLARE #HasDeleted bit = 0,
#HasInserted bit = 0,
#AuditType CHAR(6),
#Code CHAR(4)
SET #Code = 'TEST'
DECLARE #IDs as dbo.Ids
IF EXISTS (SELECT * FROM deleted d)
SET #HasDeleted = 1
IF EXISTS (SELECT * FROM inserted i)
SET #HasInserted = 1
IF #HasDeleted = 1
BEGIN
IF #HasInserted = 1
BEGIN
SET #AuditType = 'UPDATE'
END
ELSE
BEGIN
SET #AuditType = 'DELETE'
END
END
ELSE
IF #HasInserted = 1
BEGIN
SET #AuditType = 'INSERT'
END
INSERT INTO #IDs (Id)
SELECT [MyTable_PK]
FROM inserted
UNION
SELECT [MyTable_PK]
FROM deleted
EXEC [dbo].[STP_Audit_MultipleRecords] #IDs, #Code, #AuditType
END
GO
Notes:
The #HasDeleted and #HasInserted variables are to allow you to only execute the EXISTS query once for every procedure.
Getting the primary key values from the deleted and inserted table is done using a single union query. Since union eliminates duplicate values, you can write this query just once. If you want to, you can write a different query for each audit type, but then you will have to repeat the same query 3 times (with different tables)
I've changed the data types of your #code and #AuditType variables to char, since they have a fixed length.
I may be wording this question very poorly but I am not 100% sure what I need to question.
I am trying to iterate over rows in a table and call a stored procedure using the data from the rows.
This is the code I already have, the problem with this is a timing issue (1000 rows takes around 1 minute);
--Set up a temp table with all non email alerts
SELECT TOP(1000)
RowNum = ROW_NUMBER() OVER(ORDER BY AlertID),
a.*, i.ImgData
INTO
#temp
FROM
dbo.ALERTS a
JOIN
dbo.IMAGES i ON i.VehicleID = a.VehicleID
WHERE
a.EmailImageSent = 0 OR a.EmailSent = 0
DECLARE #MaxRownum INT
SET #MaxRownum = (SELECT MAX(RowNum) FROM #temp)
DECLARE #Iter INT
SET #Iter = (SELECT MIN(RowNum) FROM #temp)
DECLARE #ImgData VARBINARY(MAX)
WHILE #Iter <= #MaxRownum
BEGIN
SELECT #VehicleID = VehicleID, #ImgData = ImgData
FROM #temp
WHERE RowNum = #Iter
IF #ImgData IS NOT NULL
BEGIN
EXEC dbo.someProcedure #VehicleID, #ImgData
--SELECT 'Image data found for', #VehicleID, #ImgData
END
SET #Iter = #Iter + 1
END
DROP TABLE #temp
Is there anyway I can run the stored procedure (dbo.someProcedure) while using a set based statement as the input?
Sorry if this has been asked before, I've had a look and couldn't find an answer or if this question isn't informative enough.
Thanks in advance
AFAIK sp_send_dbmail will need to be called once for each email, so either you have a loop here or you have a loop inside dbo.someProcedure.
Still I think that you could make some improvements. Use a FAST_FORWARD cursor rather than creating iteration variables and returning to the table each time to find the next row (thus creating 1000 table scans). Don't store redundant data in your #temp table, only what you need. This makes the table quicker to read.
Try this:
--Set up a temp table with all non email alerts
Create Table #temp (VehicleID int Primary Key Clustered, ImgData varbinary(max));
INSERT INTO #temp (VehicleID, ImgData)
SELECT TOP(1000)
a.VehicleID, i.ImgData
FROM
dbo.ALERTS a
JOIN
dbo.IMAGES i ON i.VehicleID = a.VehicleID
WHERE
a.EmailImageSent = 0 OR a.EmailSent = 0;
DECLARE #VehicleID int;
DECLARE #ImgData VARBINARY(MAX);
DECLARE Alert_Cursor Cursor Fast_Forward For (
Select VehicleID, ImgData From #temp);
OPEN Alert_Cursor;
FETCH NEXT FROM Alert_Cursor INTO #VehicleID, #ImgData;
WHILE ##FETCH_STATUS = 0
BEGIN
IF #ImgData IS NOT NULL
EXEC dbo.someProcedure #VehicleID, #ImgData;
FETCH NEXT FROM Alert_Cursor INTO #VehicleID, #ImgData;
END
CLOSE Alert_Cursor;
DEALLOCATE Alert_Cursor;
DROP TABLE #temp;
I m having a trouble with my trigger, it works for single row update, but for multiple updates it gives error that sub query is returning more then one value. how to handle this.
GO
ALTER TRIGGER [dbo].[OnpaymentUpdate]
ON [dbo].[paymentData]
AFTER UPDATE
AS
BEGIN
SET NOCOUNT ON;
DECLARE #customerID NCHAR(50), #lastpaymentDate DATETIME, #stat nchar(50), #month int;
SET #customerID= (SELECT customerID FROM inserted)
SET #stat= (SELECT stat FROM inserted) --table inserted contains inserted rows (or new updated rows)
set #lastpaymentDate = (SELECT MAX(paymentDate) FROM paymentReceipt where customerID=#customerID)
SET #month= (SELECT DATEDIFF(MONTH, #lastpaymentDate,GETDATE()))
DECLARE #balance BIGINT
SET #balance =
(
SELECT (totalprice-(paidAmount+concession))
FROM paymentData
WHERE customerID = #customerID
)
Declare #paid int
SET #paid =
(
SELECT paidAmount
FROM paymentData
WHERE customerID = #customerID
)
UPDATE PaymentData
SET balanceAmount = #balance ,
lastpaymentDate=#lastpaymentDate
WHERE customerID = #customerID
if (#month >=2 and #stat!='Cancel' and #stat!='Refund' And #stat!='Refunded' and #stat!='Transfered' and #stat!='Transfer')
Begin
IF (#month <2 and #stat='Defaulter')
SET #stat='Regular'
IF (#balance<=0 and #paid >0)
SET #stat='Payment Completed'
else
SET #stat='Defaulter'
End
else
Begin
if #stat='Refund'
Set #stat='Refunded'
if #stat='Cancled'
Set #stat='Cancel'
if #stat='Transfer'
Set #stat='Transfered'
End
UPDATE PaymentData
SET stat =#stat
WHERE customerID = #customerID
END
I wouldn't have a trigger at all. I'd rebuild your table and then create a view that mimics your current table definition. Of course, I don't know your current tables, so I can only write my best guess for now. And as I said, I don't understand your status logic at the bottom where #month can apparently be, simultaneously, both >=2 and <2, so I've left that portion incomplete:
create table dbo._PaymentData (
CustomerID nchar(50) not null,
_Status nchar(50) not null,
TotalPrice bigint not null,
PaidAmount bigint not null,
Concession bigint not null,
Balance as TotalPrice - (PaidAmount + Concession)
)
go
create view dbo.PaymentData
with schemabinding
as
with RecentReceipts as (
select CustomerID,MAX(PaymentDate) as LastPayment from dbo.PaymentReceipt group by CustomerID
), MonthsDelinquent as (
select CustomerID,LastPayment,DATEDIFF(month,LastPayment,CURRENT_TIMESTAMP) as Months from RecentReceipts
)
select
pd.CustomerID,
TotalPrice,
PaidAmount,
Concession,
Balance,
LastPayment,
CASE
WHEN _Status in ('Cancel','Refund','Refunded','Transfered','Transfer')
THEN _Status
WHEN md.Months > 2 and Balance<= 0 and PaidAmount > 0
THEN 'Payment Complete'
--More conditions here to work out the actual status
END as Status
from
dbo._PaymentData pd
left join
MonthsDelinquent md
on
pd.CustomerID = md.CustomerID
go
And now, we don't need the trigger - the table and/or the view are always up correct (although a trigger may be required on the view to allow Status/_Status to be updated - it's not currently clear whether that's necessary or whether _Status actually needs to exist at all)
Could their be more than one CustomerID added in the process? This line is trouble:
SET #customerID= (SELECT customerID FROM inserted)
SET #stat= (SELECT stat FROM inserted) --table inserted contains inserted
If customerID and stat are guaranteed to be consistent for all rows, you can fix it with a MAX, like:
SET #customerID= (SELECT MAX(customerID) FROM inserted)
SET #stat= (SELECT MAX(stat) FROM inserted) --table inserted contains inserted
But, if those items aren't guaranteed to be consistent for all rows inserted, you will run into trouble. If that's the case, you'll need a cursor to cursor through the values.
Also Change to this:
SET #balance =
(
SELECT SUM( (totalprice-(paidAmount+concession)) )
FROM paymentData
WHERE customerID = #customerID
)
Declare #paid int
SET #paid =
(
SELECT SUM(paidAmount)
FROM paymentData
WHERE customerID = #customerID
)
I am trying to see if its possible to perform Update within a cursor loop and this updated data gets reflected during the second iteration in the loop.
DECLARE cur CURSOR
FOR SELECT [Product], [Customer], [Date], [Event] FROM MyTable
WHERE [Event] IS NULL
OPEN cur
FETCH NEXT INTO #Product, #Customer, #Date, #Event
WHILE ##FETCH_STATUS = 0
BEGIN
SELECT * FROM MyTable WHERE [Event] = 'No Event' AND [Date] < #DATE
-- Now I update my Event value to 'No Event' for records whose date is less than #Date
UPDATE MyTable SET [Event] = 'No Event' WHERE [Product] = #Product AND [Customer] = #Customer AND [Date] < #DATE
FETCH NEXT INTO #Product, #Customer, #Date, #Event
END
CLOSE cur
DEALLOCATE cur
Assume when the sql executes the Event column is NULL for all records
In the above sql, I am doing a select inside the cursor loop to query MyTable where Event value is 'No Event' but the query returns no value even though I am doing an update in the next line.
So, I am thinking if it is even possible to update a table and the updated data get reflected in the next iteration of the cursor loop.
Thanks for any help,
Javid
Firstly You shouldn't need a cursor here. Something like the following would have the same semantics (from a starting position where all Event are NULL) and be more efficient.
WITH T
AS (SELECT [Event],
RANK() OVER (PARTITION BY [Product], [Customer]
ORDER BY [Date] DESC) AS Rnk
FROM MyTable)
UPDATE T
SET [Event] = 'No Event'
WHERE Rnk > 1
Secondly regarding the question in the title to commit inside a cursor loop is the same as anywhere else. You just need a COMMIT statement. However if you aren't running this inside a larger transaction the UPDATE statement will be auto committed anyway.
Thirdly Your real question doesn't seem to be about commit anyway. It is about the cursor reflecting updates to the data on subsequent iterations. For the case in the question you would need a DYNAMIC cursor
Defines a cursor that reflects all data changes made to the rows in
its result set as you scroll around the cursor. The data values,
order, and membership of the rows can change on each fetch.
Not all queries support dynamic cursors. The code in the question would but without an ORDER BY it is undeterministic what order the rows would be processed in and thus whether you would see visible results. I have added an ORDER BY and an index to support this to allow a dynamic cursor to be used.
If you try the following you will see the cursor only fetches one row as the dates are processed in descending order and when the first row is processed the table is updated such that no more rows qualify for the next fetch. If you comment out the UPDATE inside the cursor loop all three rows are fetched.
CREATE TABLE MyTable
(
[Product] INT,
[Customer] INT,
[Date] DATETIME,
[Event] VARCHAR(10) NULL,
PRIMARY KEY ([Date], [Product], [Customer])
)
INSERT INTO MyTable
VALUES (1,1,'20081201',NULL),
(1,1,'20081202',NULL),
(1,1,'20081203',NULL)
DECLARE #Product INT,
#Customer INT,
#Date DATETIME,
#Event VARCHAR(10)
DECLARE cur CURSOR DYNAMIC TYPE_WARNING FOR
SELECT [Product],
[Customer],
[Date],
[Event]
FROM MyTable
WHERE [Event] IS NULL
ORDER BY [Date] DESC
OPEN cur
FETCH NEXT FROM cur INTO #Product, #Customer, #Date, #Event
WHILE ##FETCH_STATUS = 0
BEGIN
SELECT #Product,
#Customer,
#Date,
#Event
-- Now I update my Event value to 'No Event' for records whose date is less than #Date
UPDATE MyTable
SET [Event] = 'No Event'
WHERE [Product] = #Product
AND [Customer] = #Customer
AND [Date] < #Date
FETCH NEXT FROM cur INTO #Product, #Customer, #Date, #Event
END
CLOSE cur
DEALLOCATE cur
DROP TABLE MyTable
Even if this worked, this would not guarantee the correct result since you miss an ORDER BY clause in your query.
Depending on this, all records, no records or any random subset of records could be updated.
Could you please explain in plain English what your stored procedure should do?
Use Below template
DECLARE #CCount int = 100
DECLARE #Count int = 0
DECLARE #id AS BigInt
DECLARE cur Cursor fast_forward for
SELECT t1.Id
FROM Table1 t1 WITH (NOLOCK) WHERE <Some where clause>
OPEN cur
Begin Tran
While (1=1)
Begin
Fetch next from cur into #id
If ##Fetch_Status <> 0
break
-- do some DML actions
Delete From Table1 WITH (ROWLOCK) where Id = #id
Set #count = #count + ##Rowcount
if (#count % #CCount = 0)
Begin
if (#count % 100 = 0)
Print 'Table1: DML action ' + Cast(#count as Varchar(15)) + ' rows'
-- for every 100 rows commit tran will trigger , and starts a new one.
While ##Trancount > 0 Commit Tran
Begin Tran
End
End
While ##Trancount > 0 Commit Tran
Close cur
Deallocate cur