I have a procedure which runs every midnight to delete expired rows from the database. During the job my application gets deadlock error when it tries to perform insert or update on the table. What should be the best way to avoid deadlock?
I have millions of record to delete every midnight, which I am deleting in a batch of 1000 rows at a time.
#batch=1000
WHILE #rowCount > 0
BEGIN TRY
DELETE TOP (#batch) FROM application
WHERE job_id IN (SELECT ID FROM JOB
WHERE process_time < #expiryDate)
OR prev_job_id IN (SELECT ID FROM [dbo].JOB
WHERE process_time < #expiryDate)
SELECT #rowCount = ##ROWCOUNT
END TRY
BEGIN CATCH
//some code here...
Related
I have table dbo.Logs in SQL Server in which I am storing application logs, More than 100k rows are getting generated daily on this table. So I have written a job to delete the data periodically from the table.
I am simply using SQL DELETE query to delete data from the Logs table. When there is too many rows in the table (approx. 2,000,000) then this DELETE statement query is getting timeout from SQL SERVER management studio.
Does anyone have idea to fix the issue, I tried increasing the query timeout but the same is not working.
Delete them in chunks that don't cause a timeout.
For example:
SET NOCOUNT ON;
DECLARE #r INT = 1;
WHILE #r > 0
BEGIN
BEGIN TRANSACTION;
DELETE TOP (100000)
FROM dbo.Logs
WHERE CreatedAt < DATEADD(month,-3, DATEADD(day,1, EOMONTH(GETDATE())));
SET #r = ##ROWCOUNT;
COMMIT TRANSACTION;
END
I currently have a lot of triggers running on most of my tables. I'm using Insert, Update and Delete triggers on all of them. They log into a separate tabel. However the processing time to use the software has increased because of this. It is barely/not noticable for smaller changes, however for big changes it can go from 10-15min to 1hr.
I would like to change my triggers to stop insterting new log records after say 250 log records in 1 minute (bulk action), delete the newly created logs and create 1 record mentiong bulk and the query used. Problem is I can't seem to get the trigger to stop when activated.
I have already created the conditions needed for this:
CREATE TRIGGER AUDIT_LOGGING_INSERT_ACTUALISERINGSCOEFFICIENT ON ACTUALISERINGSCOEFFICIENT FOR INSERT AS
BEGIN
SET NOCOUNT ON
DECLARE #Group_ID INT = (SELECT COALESCE(MAX(Audit_Trail_Group_ID), 0) FROM NST.dbo.Audit_Trail) + 1
DECLARE #BulkCount INT = (SELECT COUNT(*) FROM NST.dbo.Audit_Trail WHERE Audit_Trail_User = CONCAT('7090-LOCAL-', UPPER(SUSER_SNAME())) AND GETDATE() >= DATEADD(MINUTE, -1, GETDATE()))
IF #BulkCount < 250
BEGIN
INSERT ...
END
ELSE
BEGIN
DECLARE #BulkRecordCount INT = (SELECT COUNT(*) FROM NST.dbo.Audit_Trail WHERE Audit_Trail_User = CONCAT('7090-LOCAL-', UPPER(SUSER_SNAME())) AND GETDATE() >= DATEADD(MINUTE, -60, GETDATE()) AND Audit_Trail_Action LIKE '%BULK%')
IF #BulkRecordCount = 0
BEGIN
INSERT ...
END
END
END
However when I execute a query that changes 10000 plus records the trigger still inserts all 10000. When I execute it again right after it inserts 10000 BULK records. Probably because it executes the first time it triggers (goes through the function) 10000 times?
Also as you can see, this would work only if 1 bulk operation is used in the last 60 min.
Any ideas for handling bulk changes are welcome.
Didn't get it to work by logging the first 250 records.
Instead I did the following:
Created a new table with 'Action' and 'User' columns
I add a record everytime a bulk action starts and delete it when it ends
Changed the trigger so that if a record is found for the user in the new table that it only writes 1 bulk record in the log table
Problems associated:
Problem with this is that I also have had to manually go through the
biggest bulk functions and implement the add and delete.
An extra point of failure if the add record gets added but an exception occurs that doesnt delete the record again. -> Implemented a Try Catch where needed.
I have customer machine what inserts data into my SQL Server database at midnight. In SQL Server there is trigger to delete old records upon insert.
The problem is, the customer does bunch of single insert commands (instead of using bulk insert), actually hundreds of them. I cannot control that. What I did is to record last trigger time and if datediff is less than one hour, don't do anything. I read using WAITFOR in triggers is a bad idea as it locks the table until trigger execution is done. Is there any other way? Cheers.
BEGIN
SET NOCOUNT ON;
IF DATEDIFF(HOUR, (SELECT TOP 1 TimeStamp
FROM HouseKeepingStats
ORDER BY TimeStamp DESC), GETDATE()) > 1
BEGIN
EXEC HouseKeeping;
END
END
How to achieve following transaction lock?
In big simplification - I have a table of "Tasks" with statuses (Created, Started, Completed). I want to create stored procedure GetNext to get top 1 task that wasn't yet started (has Created status).
In this procedure I want to mark the task as Started. Obviously I want to avoid the situation when two processes call this procedure and get the same task.
The procedure will not be called frequently so performance is not an issue, keeping data uncorrupted is an issue.
So I want to do something like this:
UPDATE tblTasks
SET Status = 'Started'
WHERE TaskId = (SELECT TOP 1 TaskId
FROM tblTasks
WHERE Status = 'Created')
I also want to receive the task that I just updated so rather than what is above I need something like:
DECLARE #TaskId AS INT = (SELECT TOP 1 TaskId FROM tblTasks WHERE Status = 'Created')
UPDATE tblTasks
SET Status = 'Started'
WHERE TaskId = #TaskId
[... - Do something with #TaskId - not relevant]
OR
DECLARE #TaskIds AS TABLE(Id INT)
UPDATE tblTasks
SET Status = 'Started'
OUTPUT INSERTED.Id INTO #TaskIdS
WHERE TaskId = #TaskId
[... - Do something with #TaskIds - not relevant]
So assuming that I need select + update to achieve what I need - how can I assure that no other process will execute even first operation (select) until existing process is done?
As far as I understand even Serializable isolation level of transaction is not enough here because other process can read data, then wait until I finish (because its update is being held by lock) and update the data that I just updated.
I feel that table hints XLOCK or HOLDLOCK might help but I'm no expert and MS doc scared me with :
Caution
Because the SQL Server query optimizer typically selects the best execution plan for a query, we recommend that hints be used only as a last resort by experienced developers and database administrators.
(from https://learn.microsoft.com/en-us/sql/t-sql/queries/hints-transact-sql-table)
So how do I make sure that two processes will not update one item and also how do I make sure that if one process is running the other will wait and do its job after the first finishes instead of failing?
Typically SQL Server locks automatically for each step until you either have a GOor reaches the end of the script.
From what I understand, what you want/need is a way to SELECT/UPDATE "in one go". You should be able to achieve that with a combination of TRANSACTION, TRY ... CATCH and CTE.
DECLARE #TaskIds AS TABLE (TaskId INT);
BEGIN TRANSACTION;
BEGIN TRY
WITH myTasks (TaskId) AS (
SELECT TOP 1 t.TaskId
FROM tblTasks AS t
WHERE t.Status = 'Created'
)
UPDATE t
SET t.Status = 'Started'
OUTPUT INSERTED.TaskId INTO #TaskIds
FROM tblTasks AS t
INNER JOIN myTasks AS mt
ON mt.TaskId = t.TaskId;
END TRY
BEGIN CATCH
IF ##TRANCOUNT > 0
ROLLBACK TRANSACTION;
THROW;
END CATCH;
IF ##TRANCOUNT > 0
BEGIN
COMMIT TRANSACTION;
SELECT TaskId FROM #TaskIds;
[.. do other stuff ..]
END
GO
Business Scenario: This is a ticketing system, and we got so many user using the application. When a ticket(stored in 1st table in below) comes in to the application, any user can hit the ownership button and take ownershipf of it.
Only one user can take ownership for one ticket. If two user tries to hit the ownership button, first one wins and second gets another incident or message that no incident exists to take ownership.
Here i am facing a concurrency issue now. I already have a lock implementation using another table(2nd table in below).
I have two tables;
Table(Columns)
Ticket(TicketID-PK, OwnerUserID-FK)
TicketOwnerShipLock(TicketID-PK, OwnerUserID-FK, LockDate)Note: Here TicketID is set as Primary Key.
Current lock implementation: whenever user one tries to own ticket puts an entry to 2nd table with TicketID, UserID and current date,then goes to update the OwnerUserID in 1st table.
Before insert the above said lock entry, Procedure checks for any other user already created any lock for the same incident.
If already there is lock, lock wont be opened for the user. Else lock entry wont be entered and the user cannot update the ticket onwership.
More Info: There are so many tickets getting opened in 1st table, whenever user tries to take ownership, we should find the next available ticket to take ownership. So need to find ticket and to do some calculation and set a status for that ticket, there one more column in 1st table StatusID. Status will be assigned as Assigned.
Problem: Somehow two user's got the ownership for same ticket at excatly same time, i have even checked the millisecond but that too same.
1. I would like to know if any MS SQL Server LOCK would help in this scenario.
2. Or do i need to block table while insert.(This 2nd rable will not have much data approx. less than 15 rows)
Lock Creation Procedure Below:
ALTER PROCEDURE [dbo].[TakeOwnerShipGetLock]
#TicketId [uniqueidentifier],
#OwnerId [uniqueidentifier]
AS
BEGIN
SET NOCOUNT ON;
BEGIN TRANSACTION TakeOwnership
BEGIN TRY
DECLARE #Lock BIT
SET #Lock = 0
DECLARE #LockDate DATETIME
SELECT #LockDate = LockDate
FROM dbo.TakeOwnershipLock
WHERE TicketId = #TicketId
IF #LockDate IS NULL
AND NOT EXISTS ( SELECT 1
FROM dbo.TakeOwnershipLock as takeOwnership WITH (UPDLOCK)
INNER JOIN dbo.Ticket as Ticket WITH (NOLOCK)
ON Ticket.TicketID = takeOwnership.TicketId
WHERE takeOwnership.TicketId = #TicketId
AND Ticket.OwnerID is NULL )
BEGIN
INSERT INTO dbo.TakeOwnershipLock
( TicketId
,OwnerId
,LockDate
)
VALUES ( #TicketId
,#OwnerId
,GETDATE()
)
IF ( ##ROWCOUNT > 0 )
SET #Lock = 1
END
SELECT #Lock
COMMIT TRANSACTION TakeOwnership
END TRY
BEGIN CATCH
-- Test whether the transaction is uncommittable.
IF XACT_STATE() = 1
BEGIN
COMMIT TRANSACTION TakeOwnership
SET #Lock = 1
SELECT #Lock
END
-- Test whether the transaction is active and valid.
IF XACT_STATE() = -1
BEGIN
ROLLBACK TRANSACTION TakeOwnership
SET #Lock = 0
SELECT #Lock
END
END CATCH
END