Afternoon. I have several SQL Agent jobs running on an MS 2K8 BI server, some of them on a daily basis, others hourly, and one every two minutes (a heartbeat monitor for another process). There is also an app which imports data every few minutes, around the clock. Occasionally some combination of updates and reports collide and one or another hangs for a half hour or more, instead of the usual 60 seconds.
While I need to get to the root of these race conditions, in the meantime I'd like to set certain jobs to automatically die after, say, five minutes. I can do this in SSIS or a Windows scheduled task, but I don't see any way to do so in SQL Agent. Is this possible, or do I need to wrap the task in an SSIS package to get this kind of control?
FYI, here's the SQL Agent job I ended up using:
DECLARE #Cancelled BIT
EXEC dbo.CancelJob #JobName = 'ETL - Daily', #Cancelled = #Cancelled OUT
IF #Cancelled = 1
BEGIN
DECLARE #Success INT
EXEC #Success = msdb..sp_send_dbmail
#profile_name = 'Reporting',
#recipients = 'reporting#mycompany.com',
#subject = 'Cancelled Daily ETL'
IF #Success <> 0 RAISERROR('An error occurred while attempting to send an e-mail.', 16, #Success)
END
...and here's the code behind CancelJob:
CREATE PROCEDURE dbo.CancelJob(#JobName VARCHAR(100), #OwnerName VARCHAR(100) = NULL, #Cancelled BIT OUT)
AS BEGIN
IF #OwnerName IS NULL SET #OwnerName = SUSER_NAME()
SET #Cancelled = 0
CREATE TABLE #JobInfo
(
Job_ID UNIQUEIDENTIFIER,
Last_Run_Date INT,
Last_Run_Time INT,
Next_Run_Date INT,
Next_Run_Time INT,
Next_Run_Schedule_ID INT,
Requested_To_Run INT,
Request_Source INT,
Request_Source_ID VARCHAR(100),
Running INT, -- This is the only field we want (sigh)
Current_Step INT,
Current_Retry_Attempt INT,
State INT
)
INSERT INTO #JobInfo
EXEC xp_sqlagent_enum_jobs 1, #OwnerName
DECLARE #Running INT = (SELECT Running FROM #JobInfo AS JI INNER JOIN msdb..sysjobs_view AS J ON JI.Job_ID = J.job_id WHERE J.name = #JobName)
IF #Running = 1
BEGIN
BEGIN TRY
EXEC msdb..sp_stop_job #job_name = #JobName
SET #Cancelled = 1
END TRY
BEGIN CATCH
-- If an error occurs, it is *probably* because the job finished before we could cancel it, which is fine
END CATCH
END
END
GO
xp_sqlagent_enum_jobs was the trick to avoid the uncatchable error.
I have never had to do this frequently, so there may be better long-term solutions, but I have created a second job to stop the first on the rare occasions that I had to perform this task. I just used the sp_stopjob procedure to do this.
Related
Reading Microsoft Docs this is the relevant system procedure:
sys.sp_cdc_cleanup_change_table
I tried using it like this:
DECLARE #max_lsn binary(10);
SET #max_lsn = sys.fn_cdc_get_max_lsn();
Exec sys.sp_cdc_cleanup_change_table
#capture_instance = N'dbo_mytable',
#low_water_mark = #max_lsn;
The query is executed successfully but checking the table again with the query:
DECLARE #from_lsn binary(10), #to_lsn binary(10);
SET #from_lsn = sys.fn_cdc_get_min_lsn('dbo_mytable');
SET #to_lsn = sys.fn_cdc_get_max_lsn();
SELECT * FROM cdc.fn_cdc_get_all_changes_dbo_mytable
(#from_lsn, #to_lsn, N'all');
Still returns a non empty table. I'm not familiar with SQL much. What am I missing?
I built a little test for this, and yes, I saw the same thing. It took me a couple of minutes to figure out what was going on.
The "gotcha" is this little entry in the docs:
If other entries in cdc.lsn_time_mapping share the same commit time as the entry identified by the new low watermark, the smallest LSN associated with that group of entries is chosen as the low watermark.
In other words, if the result of sys.fn_cdc_get_max_lsn() maps to a cdc.lsn_time_mapping.tran_begin_time that also has other start_lsn values associated with it, then the cleanup proc won't actually use the value of sys.fn_cdc_get_max_lsn() as the new low water mark.
In other other words, if the max lsn currently in the change table you want to clean up has the same tran_begin_time as other LSN's, and it is not the lowest of those LSNs, you cannot get a "complete" cleanup of the change table.
The easiest way to get a complete cleanup in those cases is probably to make a minor change to the target table to advance the max lsn and force a new entry, and "hope" that the new entry isn't also associated with any other LSNs with the same tran begin time.
To make that more explicit, here's my little test. Running it over and over has a result that in some cases cleanup is predicted to fail (and fails) and in other cases it is predicted to succeed (and succeeds).
/*
one time setup:
create table t(i int primary key, c char);
create table u(i int primary key, d char);
go
exec sp_cdc_enable_db;
go
exec sys.sp_cdc_enable_table #source_schema = 'dbo',
#source_name = 't',
#supports_net_changes = 1,
#role_name = null;
exec sys.sp_cdc_enable_table #source_schema = 'dbo',
#source_name = 'u',
#supports_net_changes = 1,
#role_name = null;
*/
set nocount on;
delete from t;
delete from u;
go
insert t select 1, 'a';
insert u select 1, 'b';
waitfor delay '00:00:01';
go
declare #fail int;
select #fail = count(*)
from cdc.lsn_time_mapping
where tran_begin_time = (
select tran_begin_time
from cdc.lsn_time_mapping
where start_lsn = sys.fn_cdc_get_max_lsn()
);
print iif(#fail > 1, 'this wont do the cleanup you expect', 'this will do the cleanup');
DECLARE #max_lsn binary(10) = sys.fn_cdc_get_max_lsn();
Exec sys.sp_cdc_cleanup_change_table
#capture_instance = N'dbo_t',
#low_water_mark = #max_lsn;
go
if exists (select * from cdc.dbo_t_ct) print 'did not cleanup';
else print 'did the cleanup';
I have a queue that fills up with multiple types of messages and I want to have two stored procedures running on that same queue, one for each message type.
I don't want one that checks for multiple types.
The issue is, I can't see how to define two in the queue definition:
CREATE QUEUE MyQueue WITH ACTIVATION
(
STATUS = ON,
-- How have two?
PROCEDURE_NAME = [my_listener_proc],
MAX_QUEUE_READERS = 4,
EXECUTE AS SELF
)
GO
A comment above has the right of it - you only get one activation procedure. But you can put logic in that proc to do other things based on the message type. Here's a slightly redacted version of one that I have in production:
ALTER PROCEDURE [repl].[myActivation]
AS
BEGIN
DECLARE
#message_type nvarchar(256),
#message XML,
#rc INT,
#queuing_order BIGINT;
DECLARE #messages TABLE (
[message_type] sysname,
[message] XML
);
SET NOCOUNT ON;
WHILE(1=1)
BEGIN
WAITFOR(
RECEIVE TOP (1000)
[message_type_name],
CAST([message_body] AS XML)
FROM [repl].[myQueue]
INTO #messages
), TIMEOUT 5000;
IF (##rowCount = 0)
BREAK;
ELSE
BEGIN
DECLARE [messages] CURSOR FAST_FORWARD LOCAL
FOR
SELECT [message], [message_type]
FROM #messages;
OPEN [messages];
WHILE(1=1)
BEGIN
FETCH NEXT FROM [messages]
INTO #message, #message_type;
IF (##fetch_Status <> 0)
BREAK;
BEGIN TRY
IF (#message_type = 'A')
BEGIN
EXEC #rc = [repl].[ProcessAMessage]
#message = #message;
END
ELSE IF (#message_type = 'B')
BEGIN
EXEC #rc = [repl].[ProcessBMessage]
#message = #message;
END
IF (#rc <> 0)
BEGIN
INSERT INTO [repl].[DeadLetters]
( [Payload], [MessageType] )
VALUES ( #message, #message_type );
END
END TRY
BEGIN CATCH
INSERT INTO [repl].[DeadLetters]
( [Payload], [MessageType] )
VALUES ( #message, #message_type );
END CATCH
END
CLOSE [messages];
DEALLOCATE [messages];
DELETE #messages;
END
END
END
GO
There's a fair bit going on there (aside from routing to different procs based on the message type) including:
Receiving multiple messages at a time so as not to lock the conversation too often
Taking a couple of steps to make sure that the activation procedure doesn't roll back/error out. Poison messages can wreck your day.
I don't use transactions in mine because there's another process that ensures that the effects of the message are applied. So, it doesn't matter much if any given message makes it (as the fact that it doesn't will be detected relatively quickly).
Can someone to explain me what the next procedure does?
CREATE PROCEDURE [add_100*Clients-runView2-del_50*Reductions] AS
DECLARE #procName NVARCHAR(100) = OBJECT_NAME(##PROCID), #currentName NVARCHAR(50)
DECLARE #index int
INSERT INTO TestRuns (Description, StartAt, EndAt)
VALUES ('Add Clients - View 2 - Delete Reductions', GETDATE(), null)
DECLARE #currentID int
SET #currentID = (SELECT SCOPE_IDENTITY())
SET #index = CHARINDEX('-', #procName)
WHILE #index > 0
BEGIN
SET #currentName = SUBSTRING(#procName, 1, #index-1)
SET #procName = SUBSTRING(#procName, #index+1, (LEN(#procName) - #index))
SET #index = CHARINDEX('-', #procName)
EXEC #currentName
END
SET #currentName = #procName
EXEC #currentName
UPDATE TestRuns
SET EndAt = GETDATE()
WHERE TestRunID = #currentID
GO
I can't understand what does getDate and how it influences the tables.
The purpose of the procedure lies within
EXEC #currentName.
I believe this is some sort of performance test where you see how much time procedures takes to run
I guess you are passing some sort of procedure names separated by - and parse each procedure and run them.
While running them , you are recording your start of the time using GetDate and after everything is run, end of the run using GetDate.(As GetDate gives you the current time, the difference will tell you how long did it take to run all the procedures.
You record that information in an audit table called TestRuns.
I'm running into performance issues with Sql Server 2008 R2, which I've narrowed down to the query optimizer (I think!). I'm looking for a definitive "why does this happen, or is it a bug?".
For sake of discussion I'll use this example, but the same problem has been seen across multiple sprocs with the same scenario.
We have a table containing payment methods; key fields are PaymentMethodId and UserId.
PaymentMethodId is an int, and the PK; UserId is a nvarchar(255) with a non-clustered index.
The query is similar to the following:
Sproc params worth mentioning:
#id int = null
#userId nvarchar(255) = null
There is an if statement at the beginning of the sproc to disallow both parameters being null.
select * from PaymentMethods (nolock) pm
where (#userId is null or #userId = pm.UserId)
and (#id is null or #id = pm.PaymentMethodId)
In the case where #userId is null, I would expect the optimizer to detect the first where clause as always true; if #userId is NOT null, I would expect it to use the index on UserId.
I have the same expectations for #id.
What we're seeing, is that regardless of input values the database is electing to do a full table scan.
While this is concerning on its own, it gets more interesting.
When updating the query where clause to equivalent of below, it is using the indecies correctly.
select * from PaymentMethods (nolock) pm
where ((#userId is null and pm.UserId is null) OR #userId = pm.UserId)
and (#id is null or #id = pm.PaymentMethodId)
What is going on? Why is "#userId is null" being considered for every record, (or is it?) Or is the real issue sitting in front of they keyboard?!
There can be a number of reasons why your sp is slow. For instance, stored procedures create a plan depending on the values for the parameters when you first run that sp. This means that you get the same plan even when the new values may return a completely different result set, one that could benefit from another plan. You could try using dynamic SQL or run the sp with OPTION(RECOMPILE) so the optimizer can create another execution plan. This is one example:
CREATE STORED PROCEDURE dbo.Test #userid INT, #id INT
AS
DECLARE #sql NVARCHAR(4000)
SET #sql = N'SELECT *
FROM PaymentMethods pm
WHERE 1 = 1'
SET #sql = #sql +
CASE
WHEN #userid IS NOT NULL THEN N' AND pm.UserId = #userid '
ELSE N''
END +
CASE
WHEN #id IS NOT NULL THEN N' AND pm.PaymentMethodId = #id '
ELSE N''
END
EXEC sp_executesql #sql, N'#userid INT, #id INT', #userid, #id;
I have the following SP for SQL Server. Strangely the SP has weired behaviour when executing the query
Select #max_backup_session_time = Max(MachineStat.BackupSessionTime) from MachineStat where MachineStat.MachineID = #machine_id;
It takes 1 second if the MachineStat table has rows pertaining to #machine_id but if there are no rows for a #machine_id then it takes more than half a minute to execute. Can someone please help me understand this.
SET NOCOUNT ON;
DECLARE #MachineStatsMId TABLE (
MachineId INT NULL,
BackupSessiontime BIGINT NULL,
MachineGroupName NVARCHAR(128) NULL )
DECLARE #machine_id AS INT;
DECLARE #Machine_group_id AS INT;
DECLARE #machine_group_name AS NVARCHAR(128);
DECLARE #max_backup_session_time AS BIGINT;
SET #machine_id = 0;
SET #Machine_group_id = 0;
SET #machine_group_name = '';
DECLARE MachinesCursor CURSOR FOR
SELECT m.MachineId,
m.MachineGroupId,
mg.MachineGroupName
FROM Machines m,
MachineGroups mg
WHERE m.MachineGroupId = mg.MachineGroupId;
OPEN MachinesCursor;
FETCH NEXT FROM MachinesCursor INTO #machine_id, #machine_group_id, #machine_group_name;
WHILE ##FETCH_STATUS = 0
BEGIN
SELECT #max_backup_session_time = Max(MachineStat.BackupSessionTime)
FROM MachineStat
WHERE MachineStat.MachineID = #machine_id;
INSERT INTO #MachineStatsMId
VALUES (#machine_id,
#max_backup_session_time,
#machine_group_name);
FETCH NEXT FROM MachinesCursor INTO #machine_id, #machine_group_id, #machine_group_name;
END;
SELECT *
FROM #MachineStatsMId;
CLOSE MachinesCursor;
DEALLOCATE MachinesCursor;
GO
Here is an alternate version that avoids a cursor and table variable entirely, uses proper (modern) joins and schema prefixes, and should run a lot quicker than what you have. If it still runs slow in certain scenarios, please post the actual execution plan for that scenario as well as an actual execution plan for the fast scenario.
ALTER PROCEDURE dbo.procname
AS
BEGIN
SET NOCOUNT ON;
SELECT
m.MachineId,
BackupSessionTime = MAX(ms.BackupSessionTime),
mg.MachineGroupName
FROM dbo.Machines AS m
INNER JOIN dbo.MachineGroups AS mg
ON m.MachineGroupId = mg.MachineGroupId
INNER JOIN dbo.MachineStat AS ms -- you may want LEFT OUTER JOIN here, not sure
ON m.MachineId = ms.MachineID
GROUP BY m.MachineID, mg.MachineGroupName;
END
GO