I am writing a stored procedure that would perform the following operations:
BEGIN a transaction
perform some tasks
use Service Broker to kick-off a background process
wait for a response message (with job status) from Service Broker
COMMIT or ROLLBACK transaction, depending on the response message
The issue is that Service Broker communication is not working inside a TRANSACTION:
the message queue has activation enabled, but the associated stored procedure is not executed (PRINT statements in the stored procedure are not written to ERRORLOG file)
RECEIVE command times out
Here's an extract of my code:
-- Comment out the following line to make everything work
begin tran t1
DECLARE #Update_Msg XML([sb].[Service_Broker_xxx_Schemas]) = '
<Request xmlns="xxx">
<Table xmlns="xxx">
<Fields>
xxx
</Fields>
</Table>
<Requested_By>xxx</Requested_By>
</Request>'
DECLARE #conversation_handle UNIQUEIDENTIFIER
,#message_body varbinary(max)
,#message_type_name nvarchar(256)
,#timestamp datetime2
BEGIN DIALOG CONVERSATION #conversation_handle
FROM SERVICE [xxx_Initiating_Service]
TO SERVICE 'xxx_Target_Service'
ON CONTRACT xxx_Contract
WITH ENCRYPTION = OFF;
SEND ON CONVERSATION #conversation_handle
MESSAGE TYPE [xxx_Command](#Update_Msg);
select * from sys.transmission_queue with(nolock)
--PRINT #conversation_handle
WAITFOR (
-- just handle one message at a time
RECEIVE TOP(1) #conversation_handle = conversation_handle -- the identifier of the dialog this message was received on
,#message_type_name = message_type_name
,#message_body=message_body -- the message contents
,#timestamp = GETDATE()
FROM [sb].[xxx_Initiator_Queue]
WHERE conversation_handle = #conversation_handle
), TIMEOUT 1000 -- if the queue is empty for one second, give UPDATE and go away
IF ##ROWCOUNT > 0
BEGIN
SELECT ##ROWCOUNT, #message_type_name, CONVERT(XML, #message_body)
END CONVERSATION #conversation_handle;
END
ELSE
BEGIN
PRINT 'Did not receive any response from Service Broker.'
END
-- Comment out the following line to make everything work
commit tran t1
What is the correct way to implement Service Broker messaging inside a transaction?
Sending messages via Service Broker is transactional. That is, if you do begin tran; send;, the message isn't actually sent until you commit.
Related
I am reading an expert from a google preview book.
shorturl.at/htF38
I want to basically simulate a poision message in a Service Broker Queue.
The author presents this code fragment to simulate 5 rollbacks .
My question is how does this pieces of code simulate 5 rollbacks beats me.
Per me On line 31 if it does not see a message in the queue the rollback happens once
and the break statement breaks the loop no ?
DECLARE #ch UNIQUEIDENTIFIER
DECLARE #messagetypename NVARCHAR(256)
DECLARE #messagebody XML
WHILE (1=1)
BEGIN
BEGIN TRANSACTION
WAITFOR (
RECEIVE TOP (1)
#ch = conversation_handle,
#messagetypename = message_type_name,
#messagebody = CAST(message_body AS XML)
FROM TargetQueue
),
TIMEOUT 60000
IF (##ROWCOUNT = 0)
BEGIN
ROLLBACK TRANSACTION
BREAK **--Line No 31**
END
-- Rolling back the current transaction
PRINT 'Rollback the current transaction - simulating a poison message...'
ROLLBACK TRANSACTION
END
GO
Now, when you place a message into the TargetQueue and execute the service program
shown in Listing ,you'll see that Service Broker deactivates the queue automatically, and
you’ll get an error message
I wish to comply with Remus Rusanu's dialog recycling technique in my SSB implementation. I wrote some activation procedure for initiator queue, in order to hook EndDialog message back from target and clean the Dialog table from the closed conversation handle.
Nevertheless, though EndDialog ack properly reaches initiator side, no activation is triggered, so my message handler cannot operate and clean the place.
CREATE PROCEDURE fdwh.ProcessResponse
AS
BEGIN
DECLARE #dlgId UNIQUEIDENTIFIER;
DECLARE #msgTypeName SYSNAME;
DECLARE #msgBody VARBINARY(MAX);
DECLARE #payloadHistoryId INT;
BEGIN TRY
BEGIN TRANSACTION
WAITFOR(
RECEIVE TOP(1)
#dlgId = [conversation_handle],
#msgTypeName = message_type_name,
#msgBody = message_body
FROM [fdwh].[SenderQueue]), TIMEOUT 10;
-- Message is regular end of conversation, terminate it
IF (#msgTypeName = N'http://schemas.microsoft.com/SQL/ServiceBroker/EndDialog')
BEGIN
END CONVERSATION #dlgId;
DELETE FROM DWH_BOARD.dbo.Dialog
WHERE (DbId = DB_ID()) AND
(DialogId = #dlgId);
END
-- Message is error, extracts and logs number and description
IF (#msgTypeName = N'http://schemas.microsoft.com/SQL/ServiceBroker/Error')
BEGIN
[...]
I expect queue activation to be triggered and http://schemas.microsoft.com/SQL/ServiceBroker/EndDialog message to be processed as well, but it's not. Isn't EndDialog ACK a regular message?
Please find below a Profiler trace screenshot that is self explaining: .
Example is pure local (single instance/two DBs).
Thanks,
Update
A few more metrics for failing queue:
`SELECT que.[name], que.is_activation_enabled, que.is_receive_enabled, que.is_poison_message_handling_enabled, que.activation_procedure, que.max_readers, [execute_as] = (SELECT pri.[name] FROM sys.database_principals pri WHERE pri.principal_id = que.execute_as_principal_id) FROM sys.service_queues que WHERE que.[name] = 'SenderQueue';
GO
SELECT conversation_handle, to_service_name, message_type_name, is_conversation_error, is_end_of_dialog, enqueue_time, transmission_status FROM sys.transmission_queue;
GO
SELECT [name], is_broker_enabled, log_reuse_wait_desc FROM sys.databases WHERE database_id = 8;
GO
EXEC sp_spaceused 'fdwh.SenderQueue';
GO
SELECT * FROM sys.dm_broker_activated_tasks WHERE database_id=8;
GO
SELECT [state], last_activated_time, tasks_waiting FROM sys.dm_broker_queue_monitors WHERE database_id = 8;
GO
`
Activation occurs only when new messages arrive.
"When STATUS = ON, the queue starts the stored procedure specified
with PROCEDURE_NAME when the number of procedures currently running is
less than MAX_QUEUE_READERS and when messages arrive on the queue
faster than the stored procedures receive messages."
https://learn.microsoft.com/en-us/sql/t-sql/statements/alter-queue-transact-sql?view=sql-server-2017
The activation procedure is expected to continue consuming messages until the queue is empty, and remains empty for the duration of its WATFOR ... RECEIVE.
Your activation procedure is missing the loop. It's RECEIVING a single message and exiting. So every time a new message arrives a single old message is consumed. This may appear to work for a while, but if you ever have get a backlog of messages, you'll never catch up.
I am experiencing poison messages and I am not sure why.
My broker setup looks like this:
CREATE MESSAGE TYPE
[//DB/Schema/RequestMessage]
VALIDATION = WELL_FORMED_XML;
CREATE MESSAGE TYPE
[//DB/Schema/ReplyMessage]
VALIDATION = WELL_FORMED_XML;
CREATE CONTRACT [//DB/Schema/Contract](
[//DB/Schema/RequestMessage] SENT BY INITIATOR,
[//DB/Schema/ReplyMessage] SENT BY TARGET
)
CREATE QUEUE Schema.TargetQueue
CREATE SERVICE [//DB/Schema/TargetService]
ON QUEUE Schema.TargetQueue (
[//DB/Schema/Method3Contract]
)
CREATE QUEUE Schema.InitiatorQueue
CREATE SERVICE [//DB/Schema/InitiatorService]
ON QUEUE Schema.InitiatorQueue
Then I have my internal activation procedure:
CREATE PROCEDURE Schema.Import
AS
DECLARE #RequestHandle UNIQUEIDENTIFIER;
DECLARE #RequestMessage VARCHAR(8);
DECLARE #RequestMessageName sysname;
WHILE (1=1)
BEGIN
BEGIN TRANSACTION;
WAITFOR (
RECEIVE TOP(1)
#RequestHandle = conversation_handle,
#RequestMessage = message_body,
#RequestMessageName = message_type_name
FROM
Schema.TargetQueue
), TIMEOUT 5000;
IF (##ROWCOUNT = 0)
BEGIN
COMMIT TRANSACTION;
BREAK;
END
EXEC Schema.ImportStep1 #ID = #RequestMessage;
--EXEC Schema.ImportStep2 #ID = #RequestMessage;
END CONVERSATION #RequestHandle;
COMMIT TRANSACTION;
END
My activation is enabled by:
ALTER QUEUE Schema.TargetQueue
WITH
STATUS = ON,
ACTIVATION
( STATUS = ON,
PROCEDURE_NAME = Schema.Import,
MAX_QUEUE_READERS = 10,
EXECUTE AS SELF
)
I initiate this process with this stored procedure
CREATE PROCEDURE Schema.ImportStart
AS
BEGIN
DECLARE #ID VARCHAR(8);
DECLARE Cursor CURSOR FOR
SELECT ID FROM OtherDatabase.OtherSchema.ImportTable
EXCEPT
SELECT ID FROM Table
OPEN Cursor;
FETCH NEXT FROM Cursor INTO #ID;
WHILE ##FETCH_STATUS = 0
BEGIN
DECLARE #InitiateHandle UNIQUEIDENTIFIER;
DECLARE #RequestMessage VARCHAR(8);
BEGIN TRANSACTION;
BEGIN DIALOG
#InitiateHandle
FROM SERVICE
[//DB/Schema/InitiatorService]
TO SERVICE
N'//DB/Schema/TargetService'
ON CONTRACT
[//DB/Schema/Contract]
WITH
ENCRYPTION = OFF;
SELECT #RequestMessage = #ID;
SEND ON CONVERSATION
#InitiateHandle
MESSAGE TYPE
[//DB/Schema/RequestMessage]
(#RequestMessage);
COMMIT TRANSACTION;
FETCH NEXT FROM Cursor INTO #ID;
END
CLOSE Cursor;
DEALLOCATE Cursor;
END
So how this should work is:
I execute ImportStart
A message for each ID gets generated
Internal activation makes Import steps execute
Instead, I get poison messaging and the queue becomes disabled.
If however,
I I set Schema.TargetQue Activation to OFF
EXEC schema.ImportStart
EXEC schema.Import manually
It works fine.
Any insights anyone?
Well:
Your message types are defined as well_formed_xml, yet you send varchar(8) as a message body. Does it really work?
You use [//DB/Schema/Method3Contract] for the target queue, but do not define it. A misspelling, most likely.
You specify EXECUTE AS SELF in the queue activation. BOL says some mystical thing about this case:
SELF
Specifies that the stored procedure executes as the current user. (The database principal executing this ALTER QUEUE statement.)
I'm not really sure I understand the quoted statement, because it apparently contradicts with your experience. If it would be your user account, everything should have been fine, because you seem to have all permissions necessary to do the job.
So, just in case - who is the owner of the Schema schema? What permissions does this principal possess? And, if it's not you, who executes the alter queue statement (and why)?
Without access to logs, it's significantly more difficult to diagnose the problem, but I would start with creating a new user account with permissions identical to yours, setting it the owner of the Schema schema and then slowly working it down, revoking unnecessary permissions until it breaks. Assuming, of course, it will work at all.
Summary: I do observe the "The Service Broker protocol transport is disabled or not configured." in the transmission_status of the queue. It seems that some more detail should be set. Or what am I missing?
The question is directly related to the answer to my earlier question.
I did follow the advice with using special trick GRANT CONNECT ON ENDPOINT::[<brokerendpointname>] TO [public] to simplify the plug-in kind of configuration in the trusted local network:
create message type...
create contract ...
create queue ...
create service [tcp://MACHINE3:4022/Satellite]
on ...
([...]);
grant send on service::[tcp://MACHINE3:4022/Satellite] to [public];
create route transport with address = 'TRANSPORT';
For the case the detail may play the role, here is the extract of the real code:
USE [MySatelliteDB];
CREATE MESSAGE TYPE [//x/y/RequestMessage]
VALIDATION = WELL_FORMED_XML;
CREATE MESSAGE TYPE [//x/y/ReplyMessage]
VALIDATION = WELL_FORMED_XML;
CREATE CONTRACT [//x/y/SimpleContract]
([//x/y/RequestMessage] SENT BY INITIATOR,
[//x/y/ReplyMessage] SENT BY TARGET);
CREATE QUEUE GenericQueue;
CREATE SERVICE [tcp://192.168.4.120:4022/GenericService]
ON QUEUE GenericQueue ([//x/y/SimpleContract]);
GRANT SEND ON SERVICE::[tcp://192.168.4.120:4022/GenericService] TO [public];
CREATE ROUTE transport WITH ADDRESS = 'TRANSPORT';
The central machine uses the same except of the database name, and the IP address (ends with 158 in the case). Then I tried to send the explicit dummy message...
DECLARE #InitDlgHandle UNIQUEIDENTIFIER;
DECLARE #RequestMsg NVARCHAR(100);
BEGIN TRANSACTION;
BEGIN DIALOG #InitDlgHandle
FROM SERVICE [tcp://192.168.4.120:4022/GenericService]
TO SERVICE N'tcp://192.168.4.158:4022/GenericService'
ON CONTRACT [//x/y/SimpleContract]
WITH
ENCRYPTION = OFF;
SELECT #RequestMsg = N'<RequestMsg>Message for Target service.</RequestMsg>';
SEND ON CONVERSATION #InitDlgHandle
MESSAGE TYPE [//x/y/RequestMessage]
(#RequestMsg);
SELECT #RequestMsg AS SentRequestMsg;
COMMIT TRANSACTION;
GO
It displayed
SentRequestMsg
<RequestMsg>Message for Target service.</RequestMsg>
... apparently as the result of the last select. This way I guess the COMMIT TRANSACTION succeeded. (No error message observed.)
Then I opened the SSMS window for the other SQL server and tried to receive...
DECLARE #RecvReqDlgHandle UNIQUEIDENTIFIER;
DECLARE #RecvReqMsg NVARCHAR(100);
DECLARE #RecvReqMsgName sysname;
BEGIN TRANSACTION;
WAITFOR
( RECEIVE TOP(1)
#RecvReqDlgHandle = conversation_handle,
#RecvReqMsg = message_body,
#RecvReqMsgName = message_type_name
FROM GenericQueue
), TIMEOUT 1000;
SELECT #RecvReqMsg AS ReceivedRequestMsg;
IF #RecvReqMsgName =
N'//x/y/RequestMessage'
BEGIN
DECLARE #ReplyMsg NVARCHAR(100);
SELECT #ReplyMsg =
N'<ReplyMsg>Message for Initiator service.</ReplyMsg>';
SEND ON CONVERSATION #RecvReqDlgHandle
MESSAGE TYPE
[//x/y/ReplyMessage] (#ReplyMsg);
END CONVERSATION #RecvReqDlgHandle;
END
SELECT #ReplyMsg AS SentReplyMsg;
COMMIT TRANSACTION;
GO
It displayed two results with unexpected values
ReceivedRequestMsg
NULL
---------------------------------------------
SentReplyMsg
NULL
No error messages observed.
When running the script...
USE [MySatelliteDB];
SELECT * FROM GenericQueue WITH (NOLOCK);
SELECT * FROM sys.transmission_queue;
I can see that something does not work correctly (was one line, wrapped manually)
conversation_handle to_service_name
1A227CA7-6F24-E211-B1EC-004063F5CE90 tcp://192.168.4.158:4022/GenericService
------------------------------------------------------------------------------...
to_broker_instance from_service_name
<empty> tcp://192.168.4.120:4022/GenericService
------------------------------------------------------------------------------...
service_contract_name enqueue_time
//x/y/SimpleContract 2012-11-01 22:01:15.440
------------------------------------------------------------------------------...
message_sequence_number message_type_name
0 /x/y/RequestMessage
------------------------------------------------------------------------------...
is_conversation_error is_end_of_dialog message_body
0 0 0x3C0052006...snip...3E00
------------------------------------------------------------------------------...
transmission_status priority
The Service Broker protocol transport is disabled or not configured. 5
What should I focus on to make it working?
The Service Broker protocol transport is disabled or not configured
means the endpoint is not started. Try ALTER ENDPOINT [<brokerendpointname>] STATE=STARTED;. If the statement succeeds but the problem persists, check the ERRORLOG, it should display an error message shortly after you ran this statement stating why it could not start the endpoint. Most likely there is a listening port conflict.
I am would like to display progress feedback for user while stored procedure is running (multiple setps). All sql code is in transaction so I am using a service broker to get a real time updates. Could you suggest what is wrong with code below as I am not getting any messages in my queue?
Is there a better way of doing it?
--queue
create queue myQueue
--service
create service myLogs
on queue myQueue ([http://schemas.microsoft.com/SQL/Notifications/PostEventNotification]);
--log handler
create event notification myLogHandler
on server
for userconfigurable_0
to service 'myLogs', 'current database' ;
--message
EXEC master..sp_trace_generateevent #event_class = 82, #userinfo = N'test msg'
---transaction test
begin transaction
EXEC master..sp_trace_generateevent #event_class = 82, #userinfo = N'tran test msg'
rollback transaction
--receive message
declare #message_body xml;
receive top (1) #message_body = message_body
from myQueue
select #message_body
--display queue
select * from myQueue
An arguably simpler way would be to:
--------------------------------
--Worker process
--------------------------------
BEGIN TRANSACTION
--While in your loop or at each stage of the proc you can do this
INSERT INTO MyLoggingTable VALUES('My Message')
COMMIT
--------------------------------
--Reporting process
--------------------------------
SELECT * FROM MyLoggingTable WITH (NOLOCK)
The NOLOCK lets you read the progress of the proc even though its logging to it inside the transaction.
NOTE that this allows you to read the data "LIVE" but if the worker process dose a rollback then the data will be removed.
You can get around that by also logging to a #MyTable, this will survive the rollback and so you can rollback and then copy the content of #MyTable into MyLoggingTable.
That works fine for me though I needed to wait a few moments and run the receive from queue bit again before I saw anything.
Perhaps you need to ENABLE_BROKER on the database.
ALTER DATABASE YourDB SET ENABLE_BROKER WITH ROLLBACK IMMEDIATE
(Warning: Will kill existing connections to the DB)