I am reading an expert from a google preview book.
shorturl.at/htF38
I want to basically simulate a poision message in a Service Broker Queue.
The author presents this code fragment to simulate 5 rollbacks .
My question is how does this pieces of code simulate 5 rollbacks beats me.
Per me On line 31 if it does not see a message in the queue the rollback happens once
and the break statement breaks the loop no ?
DECLARE #ch UNIQUEIDENTIFIER
DECLARE #messagetypename NVARCHAR(256)
DECLARE #messagebody XML
WHILE (1=1)
BEGIN
BEGIN TRANSACTION
WAITFOR (
RECEIVE TOP (1)
#ch = conversation_handle,
#messagetypename = message_type_name,
#messagebody = CAST(message_body AS XML)
FROM TargetQueue
),
TIMEOUT 60000
IF (##ROWCOUNT = 0)
BEGIN
ROLLBACK TRANSACTION
BREAK **--Line No 31**
END
-- Rolling back the current transaction
PRINT 'Rollback the current transaction - simulating a poison message...'
ROLLBACK TRANSACTION
END
GO
Now, when you place a message into the TargetQueue and execute the service program
shown in Listing ,you'll see that Service Broker deactivates the queue automatically, and
you’ll get an error message
Related
I wish to comply with Remus Rusanu's dialog recycling technique in my SSB implementation. I wrote some activation procedure for initiator queue, in order to hook EndDialog message back from target and clean the Dialog table from the closed conversation handle.
Nevertheless, though EndDialog ack properly reaches initiator side, no activation is triggered, so my message handler cannot operate and clean the place.
CREATE PROCEDURE fdwh.ProcessResponse
AS
BEGIN
DECLARE #dlgId UNIQUEIDENTIFIER;
DECLARE #msgTypeName SYSNAME;
DECLARE #msgBody VARBINARY(MAX);
DECLARE #payloadHistoryId INT;
BEGIN TRY
BEGIN TRANSACTION
WAITFOR(
RECEIVE TOP(1)
#dlgId = [conversation_handle],
#msgTypeName = message_type_name,
#msgBody = message_body
FROM [fdwh].[SenderQueue]), TIMEOUT 10;
-- Message is regular end of conversation, terminate it
IF (#msgTypeName = N'http://schemas.microsoft.com/SQL/ServiceBroker/EndDialog')
BEGIN
END CONVERSATION #dlgId;
DELETE FROM DWH_BOARD.dbo.Dialog
WHERE (DbId = DB_ID()) AND
(DialogId = #dlgId);
END
-- Message is error, extracts and logs number and description
IF (#msgTypeName = N'http://schemas.microsoft.com/SQL/ServiceBroker/Error')
BEGIN
[...]
I expect queue activation to be triggered and http://schemas.microsoft.com/SQL/ServiceBroker/EndDialog message to be processed as well, but it's not. Isn't EndDialog ACK a regular message?
Please find below a Profiler trace screenshot that is self explaining: .
Example is pure local (single instance/two DBs).
Thanks,
Update
A few more metrics for failing queue:
`SELECT que.[name], que.is_activation_enabled, que.is_receive_enabled, que.is_poison_message_handling_enabled, que.activation_procedure, que.max_readers, [execute_as] = (SELECT pri.[name] FROM sys.database_principals pri WHERE pri.principal_id = que.execute_as_principal_id) FROM sys.service_queues que WHERE que.[name] = 'SenderQueue';
GO
SELECT conversation_handle, to_service_name, message_type_name, is_conversation_error, is_end_of_dialog, enqueue_time, transmission_status FROM sys.transmission_queue;
GO
SELECT [name], is_broker_enabled, log_reuse_wait_desc FROM sys.databases WHERE database_id = 8;
GO
EXEC sp_spaceused 'fdwh.SenderQueue';
GO
SELECT * FROM sys.dm_broker_activated_tasks WHERE database_id=8;
GO
SELECT [state], last_activated_time, tasks_waiting FROM sys.dm_broker_queue_monitors WHERE database_id = 8;
GO
`
Activation occurs only when new messages arrive.
"When STATUS = ON, the queue starts the stored procedure specified
with PROCEDURE_NAME when the number of procedures currently running is
less than MAX_QUEUE_READERS and when messages arrive on the queue
faster than the stored procedures receive messages."
https://learn.microsoft.com/en-us/sql/t-sql/statements/alter-queue-transact-sql?view=sql-server-2017
The activation procedure is expected to continue consuming messages until the queue is empty, and remains empty for the duration of its WATFOR ... RECEIVE.
Your activation procedure is missing the loop. It's RECEIVING a single message and exiting. So every time a new message arrives a single old message is consumed. This may appear to work for a while, but if you ever have get a backlog of messages, you'll never catch up.
I am writing a stored procedure that would perform the following operations:
BEGIN a transaction
perform some tasks
use Service Broker to kick-off a background process
wait for a response message (with job status) from Service Broker
COMMIT or ROLLBACK transaction, depending on the response message
The issue is that Service Broker communication is not working inside a TRANSACTION:
the message queue has activation enabled, but the associated stored procedure is not executed (PRINT statements in the stored procedure are not written to ERRORLOG file)
RECEIVE command times out
Here's an extract of my code:
-- Comment out the following line to make everything work
begin tran t1
DECLARE #Update_Msg XML([sb].[Service_Broker_xxx_Schemas]) = '
<Request xmlns="xxx">
<Table xmlns="xxx">
<Fields>
xxx
</Fields>
</Table>
<Requested_By>xxx</Requested_By>
</Request>'
DECLARE #conversation_handle UNIQUEIDENTIFIER
,#message_body varbinary(max)
,#message_type_name nvarchar(256)
,#timestamp datetime2
BEGIN DIALOG CONVERSATION #conversation_handle
FROM SERVICE [xxx_Initiating_Service]
TO SERVICE 'xxx_Target_Service'
ON CONTRACT xxx_Contract
WITH ENCRYPTION = OFF;
SEND ON CONVERSATION #conversation_handle
MESSAGE TYPE [xxx_Command](#Update_Msg);
select * from sys.transmission_queue with(nolock)
--PRINT #conversation_handle
WAITFOR (
-- just handle one message at a time
RECEIVE TOP(1) #conversation_handle = conversation_handle -- the identifier of the dialog this message was received on
,#message_type_name = message_type_name
,#message_body=message_body -- the message contents
,#timestamp = GETDATE()
FROM [sb].[xxx_Initiator_Queue]
WHERE conversation_handle = #conversation_handle
), TIMEOUT 1000 -- if the queue is empty for one second, give UPDATE and go away
IF ##ROWCOUNT > 0
BEGIN
SELECT ##ROWCOUNT, #message_type_name, CONVERT(XML, #message_body)
END CONVERSATION #conversation_handle;
END
ELSE
BEGIN
PRINT 'Did not receive any response from Service Broker.'
END
-- Comment out the following line to make everything work
commit tran t1
What is the correct way to implement Service Broker messaging inside a transaction?
Sending messages via Service Broker is transactional. That is, if you do begin tran; send;, the message isn't actually sent until you commit.
One of the basics of building a distributed application that uses the asynchronous communication can be expressed as Do not wait actively for any event! This way, the natural solution based on SQL Service Broker is to use the activation of a stored procedure by the message that arrived to the queue.
The Lesson 2: Creating an Internal Activation Procedure from the official Microsoft tutorial shows how to bind the stored procedure to the message queue. It also suggests the way how the sp should be implemented.
(I am new to SQL. But should not be there one more BEGIN after the CREATE PROCEDURE... AS, and one more END before the GO?)
Do I understand it corectly? See my questions below the code...
CREATE PROCEDURE TargetActivProc
AS
DECLARE #RecvReqDlgHandle UNIQUEIDENTIFIER;
DECLARE #RecvReqMsg NVARCHAR(100);
DECLARE #RecvReqMsgName sysname;
WHILE (1=1)
BEGIN
BEGIN TRANSACTION;
WAITFOR
( RECEIVE TOP(1)
#RecvReqDlgHandle = conversation_handle,
#RecvReqMsg = message_body,
#RecvReqMsgName = message_type_name
FROM TargetQueueIntAct
), TIMEOUT 5000;
IF (##ROWCOUNT = 0)
BEGIN
ROLLBACK TRANSACTION;
BREAK;
END
IF #RecvReqMsgName =
N'//AWDB/InternalAct/RequestMessage'
BEGIN
DECLARE #ReplyMsg NVARCHAR(100);
SELECT #ReplyMsg =
N'<ReplyMsg>Message for Initiator service.</ReplyMsg>';
SEND ON CONVERSATION #RecvReqDlgHandle
MESSAGE TYPE
[//AWDB/InternalAct/ReplyMessage]
(#ReplyMsg);
END
ELSE IF #RecvReqMsgName =
N'http://schemas.microsoft.com/SQL/ServiceBroker/EndDialog'
BEGIN
END CONVERSATION #RecvReqDlgHandle;
END
ELSE IF #RecvReqMsgName =
N'http://schemas.microsoft.com/SQL/ServiceBroker/Error'
BEGIN
END CONVERSATION #RecvReqDlgHandle;
END
COMMIT TRANSACTION;
END
GO
When the message arrives, the procedure is called, and it enters to the "infinite" loop. Actually, the loop is not infinite because of the BREAK after the ROLLBACK when no data arrived (after the TIMEOUT).
If the data arrived, the BREAK is skipped. If the expected message arrived, the reply is sent back. If the ...EndDialog or ...Error message is received, the END CONVERSATION is executed. Can also some other kind of message be observed here?
As some message arrived (and was processed) the transaction is commited.
But why the loop now? Is the intention to process the other messages that got stuck in the queue because of the broken communication line in the past? Or because of more messages came at once and could not be processed so quickly?
What happens when another message is queued up, and the stored procedure is still running. Is another working process assigned for its processing? Can another stored procedure be launched in parallel? If yes, then why the loop?
Thanks for your help, Petr
Internal activation is not like a trigger. Specifically, the activated procedure does not get launched for each message that arrived. Instead the procedure is launched when there is something to process and is supposed to dequeue messages contiguously (in a loop) while the SSB infrastructure is monitoring the progress and, if necessary, launches a second procedure to help, up to the max specified. See Understanding Queue Monitors.
Having a loop in the activated procedure is not not strictly required, things should work fine even w/o the loop. The loop should perform better in a very busy environment. See also this old MSDN discussion.
I'd like to clear my queue in SQL Server Management Studio, but I don't want to delete the whole queue just the content in the queue (the messages).
Just combining the two previous answers (by Ben and Jānis) for clarity. This worked for me:
declare #c uniqueidentifier
while(1=1)
begin
select top 1 #c = conversation_handle from dbo.queuename
if (##ROWCOUNT = 0)
break
end conversation #c with cleanup
end
Something like this should work:
while(1=1)
begin
waitfor (
receive top(1)
conversation_group_id
from dbo.yourQueue
), timeout 1000;
if (##rowcount = 0)
break;
end
I would use end conversation (that will also remove all related messages from all queues) using statement:
End Conversation #c With CleanUp
if you just receive message, then you leave conversation open.
End Conversation With CleanUp is for specific situations only.
If you are using SQL Server (starting with 2008) you can use RECEIVE
WHILE (0=0)
BEGIN
RECEIVE * FROM dbo.YourQueue;
IF (##ROWCOUNT = 0) BREAK;
END
I had the same issue as the original poster, but I was needing to clear queues with
millions of messages (failed message queues, especially in non production environments that had not been checked for years).
The solution above would have worked, but was processing at less than 10 messages per minute. By doing it in batches I was getting 30000 messages per minute.
The only noteworthy item in the code is where validation = 'N'. This limits the conversation handles to the real messages. There is a duplicate conversation handle for the response/error which gets removed by the end conversation. Without this clause the script would still work, but generate a lot of errors in the output.
declare #conversationBatch table (convH uniqueidentifier)
declare #conversationHandle uniqueidentifier
declare convCursor cursor for
select convH from #conversationBatch
insert into #conversationBatch
select top 1000 conversation_handle
from dbo.queuename WITH (NOLOCK)
where validation = 'N'
while ##rowcount > 0
begin
open convCursor
fetch next from convCursor into #conversationHandle
while ##FETCH_STATUS = 0
begin
end conversation #conversationHandle with cleanup
fetch next from convCursor into #conversationHandle
end
close convCursor
delete from #conversationBatch
insert into #conversationBatch
select top 1000 conversation_handle
from dbo.queuename WITH (NOLOCK)
where validation = 'N'
end
deallocate convCursor
while(1=1)
begin
waitfor (
receive top(1)
conversation_group_id
from kartokumaqueue2), timeout 1000;
if(##ROWCOUNT = 0) break;
end
We recently identified a problem with one of our databases where as a result of a 'fire & forget' setup (i.e: conversations being closed immediately after sending), our sys.conversation_endpoints table was filling up with DI/DISCONNECTED_INBOUND messages. This eventually spilled over into the tempDB, causing it to grow enormously and eat up precious disk space. We eventually resolved this issue by commenting out the line
END CONVERSATION #handle WITH CLEANUP
in our sending SP and closing the conversations in our receiving SP using the same code,
END CONVERSATION #handle WITH CLEANUP
However, we now have a new issue. Since moving servers (and migrating from SQL Server 2005 to SQL Server 2008) we've recently discovered that sys.conversation_endpoints is now filling up with CO/CONVERSING messages, indicating that the conversations are not being closed. The receiving SP is closing them, or at least is running the command to do so, so I don't understand where these messages are coming from.
I've tried going back to ending the conversation at the point of send, but it has no effect. Is it wrong to end conversations on the receiving end using WITH CLEANUP? Or is there some other problem?
This post on techtarget seems to suggest its a bug, and that running a job to cleanup the leftovers is the only solution...
UPDATE:
Pawel pointed out below that I should be avoiding the Fire & Forget Pattern, and I've added an activated SP to the initiator queue to end any conversations. However, sys.conversation_endpoints is STILL filling up, this time with CD/CLOSED messages. Here's the structure of my queues
Send_SP:
DECLARE #h UNIQUEIDENTIFIER
BEGIN DIALOG CONVERSATION #h
FROM SERVICE 'InitiatorQueue' TO SERVICE 'TargetQueue'
ON CONTRACT 'MyContract' WITH ENCRYPTION = OFF;
SEND ON CONVERSATION #h MESSAGE TYPE 'MyMessage' (#msg)
Receive_SP (Activated SP on TargetQueue)
DECLARE #type SYSNAME, #h UNIQUEIDENTIFIER, #msg XML;
DECLARE #target TABLE (
[message_type_name] SYSNAME,
[message_body] VARBINARY(MAX),
[conversation_handle] UNIQUEIDENTIFIER
)
WHILE(1=1)
BEGIN TRANSACTION
WAITFOR(RECEIVE TOP (1000)
[message_type_name],[message_body],[conversation_handle]
FROM TargetQueue INTO #target), TIMEOUT 2000
IF(##rowcount!=0)
BEGIN
WHILE((SELECT count(*) FROM #target) > 0)
BEGIN
SELECT TOP (1) #type = [message_type_name],
#msg = [message_body],
#h = [conversation_handle] FROM #target;
// Handle Message Here
END CONVERSATION #h;
DELETE TOP (1) FROM #target;
END
END
COMMIT TRANSACTION;
End_SP (Activated SP on InitiatorQueue)
DECLARE #type SYSNAME, #h UNIQUEIDENTIFIER, #msg XML;
DECLARE #init TABLE (
[message_type_name] SYSNAME,
[message_body] VARBINARY(MAX),
[conversation_handle] UNIQUEIDENTIFIER
)
WHILE(1=1)
BEGIN TRANSACTION
WAITFOR(RECEIVE TOP (1000)
[message_type_name],[message_body],[conversation_handle]
FROM InitiatorQueue INTO #init), TIMEOUT 2000
IF(##rowcount!=0)
BEGIN
WHILE((SELECT count(*) FROM #init) > 0)
BEGIN
SELECT TOP (1) #type = [message_type_name],
#msg = [message_body],
#h = [conversation_handle] FROM #init;
END CONVERSATION #h;
DELETE TOP (1) FROM #init;
END
END
COMMIT TRANSACTION;
Using the fire-and-forget pattern will inevitably lead to this and other types of issues. Additionally it will make any hypothetical errors go unnoticed. Is there any reason why you can't change the message exchange pattern so that the target issues END CONVERSATION (without cleanup!) once it receives a message and then the initiator only calls END CONVERSATION (again, without cleanup) upon receiving end conversation message from the target?