We use MQ as a primary route to transfer messages. It is integral to how our system works. There are times the message broker fails, and with it all the associated queues. Is there a way, in camel, to kickstart a failover, and revert back to master when it's up?
In general, messaging systems do not want to interrupt client -> server communication for any reason. Once a failover connection is re-established, the preference is to stay connected to that server. IBM MQ supports client failover as well, but I do not know of a way to do a rebalance after a failover has occurred.
Media Driver's Integrated Console supports rebalancing ActiveMQ/JBoss A-MQ clients after a failover or before a maintenance window: https://mediadriver.com/software/ (see: Client rebalancing video at the bottom)
Disclaimer: I am a co-founder of Media Driver and head up the product development of the Integrated Console.
Related
I have a client that uses MS SQL with availability groups. I develop a java based software and connect to the server in the following fasion: jdbc:sqlserver://[serverName[:portNumber]]
Everytime the DBA does a update on the servers, we loose the connection to the server (We get a Connection Closed). According to the DBA this is normal behavior in SQL-Servers and our software should just do a retry.
Is it really normal that the sql server closes all connections in a failover situation? Shouldn't it just redirect all connections to the new instance?
Unfortunately I am no SQL-expert and the DBA is anything but helpful, he just claims that our software should simple reconnect after receiving a closed connection. Am I missing something or is this really the desired experience in sql server?
Is it really normal that the sql server closes all connections in a failover situation? Shouldn't it just redirect all connections to the new instance?
Yes. On the assumption that the Availability Group (hereafter abbreviated "AG") is built on top of Windows Failover Clusters†, the AG listener is brought offline, transferred to new owner of the AG, and brought back online. Moreover, the databases in the AG on both the primary and all secondary replicas are transitioned from online → recovery pending → back online in their new (perhaps same as old) capacity as read/write or read-only.
Either of these phenomena alone would cause your application to lose communication with a database that it had previously established a connection with. But also, two of the fallacies of distributed computing are:
The network is reliable.
Topology doesn't change.
Regardless of the reason for those assumptions being violated, if your application isn't resilient to them, there's going to be a problem.
† - while it's technically possible to build an AG without Windows Clustering (i.e. a basic AG or if the AG is on Linux and is using Pacemaker as the coordinator), I think that the majority of implementations do use Windows Clustering.
I have a master/slave mysql (mariadb) setup for replication. Is it possible to let an existing database connection fallback to master when it's available again after a failover to slave has happened? I'm using Glassfish connection pools and have tried with different properties with no effect, eg. secondsBeforeRetryMaster. My URL is: jdbc:mysql:sequential://master,slave/db
I'm following
mysql connector
It works when new connections are created, but for existing ones, fallback does not happen at all. Since I'm reusing existing connections a lot, I'm kind of lost. The secondary server is read-only, so all writes to database fail after a failover.
I have a primary and secondary Tibco EMS queues and need to send messages to the queues. The secondary will be in standby mode until primary goes down.
From camel code, I need to handle the failover scenario - if primary ems goes down, the application should send messages to secondary instance.
I have been searching sample for this scenario and finding something for ApacheMQ using
brokerURL=failover:(endpoint1,endpoint)
Can someone help how to achieve this for EMS provider?
Should it be something like this for EMS?
connectionFactory.setServerUrl(endpoint1,endpoint2)
The connection string for an EMS in HA is of the form:
tcp://host1:7222,tcp://host2:7222
Either one will be active at one point in time... the client will figure it out automatically (in the JMS(2).jar provided by TIBCO.)
Here is a nice tutorial (not by me).
I'm in need of an Enterprise Service Bus/Message Queueing solution for publisher/subscriber functionality. I know MANY exist... MSMQ, MS Series, RabbitMQ, NServiceBus, etc etc etc...
My one requirement is that in a shared hosting solution, the only dependency that I can guarantee will exist is SQL 2005 and later... this leads me directly to SQL Service Broker.
If it sounds like I'm trying to shoehorn ESB functionality into SSB... I suppose I am...
My question is: does anyone know of a .NET API or framework that sits on top of SQL Service Broker and already provides much of the plumbing?
If I were to use pure ADO.net, I could add items to the queues by calling a stored procedure, but then:
Do to the nature of conversations, would I make one conversation per message?
If so, do I lose sequential message processing?
How do I receive messages (I know the receive syntax in t-SQL), do I call a stored procedure repeatedly in a message loop to try to
get a message off the queue?
Or would I WAITFORever? Keeping the connection open and executing the stored procedure forever?
SQL Service Broker doesn't support monologue conversations, but I read they can be implemented...
It's these kind of questions that make me wish there existed a .net solution that already managed all of this.
There was an effort to package a WCF Transport Channel for SQL Server Service Broker but, afaik, is abandonware.
But NServiceBus supports Service Broker as a transport, see Using NServiceBus and ServiceBroker.net and there are github projects like A simple wrapper API for SQL Service Broker and an ITransport plugin for NServiceBus. While not exactly mainstream, some support and community effort does exists.
As an ESB I think you will have problems due to lack of true pub-sub and broadcast. SQL Server 2012 has the ability to SEND a message to multiple targets, see How to Multicast messages with SQL Server Service Broker, but you will still have to implement the pub-sub infrastructure (publishing topics, subscribers etc) from scratch. MySpace did that and was a major effort, see Scale out SQL Server by using Reliable Messaging. My observation reffers to the low level direct use of SSB, I have never used NServiceBus so I cannot tell how well does it abstracts/expose unicast/broadcast/multicast/pub-sub over SSB.
As for your specific questions, I recommend reading Writing Service Broker Procedures and Reusing Conversations.
Service Broker
What can be the Benefits of Programming with Service Broker?
In which condition one should use Service Broker?
Do you have Sample link for Code/Video that easily/precisely explains it's usage?
Queuing and asynchronous messaging are needed for many database
applications today. Service Broker provides a new, queue-based durable
messaging framework to address these needs. Using the Transact-SQL API
provided by Service Broker, you can easily develop services to handle
application requirements for queuing or asynchronous communications.
Some of the benefits of programming with Service Broker are:
Flexible development: The programs used in a single distributed
application can be written in multiple languages. Each program
provides the functionality of each distributed application component.
Improved security: You can express security requirements via
certificates, so application components do not need to share the same
security context. Service Broker uses SQL Server security features to
help you secure your applications.
Transactional processing: Message processing occurs within SQL Server
transactions to ensure data integrity. Service Broker supports remote
transactional messaging over a standard connection to the database.
Guaranteed ordering: Service Broker provides strong guarantees
regarding the delivery and processing of a related set of messages
exactly once and in order, so no additional coding is required to
provide this functionality.
Reliable delivery: All of the data needed for a conversation--a set of
related communications between two or more services--is persisted in
SQL Server. Service Broker supports clustering and database mirroring.
A conversation may be maintained through system restarts, server
failover, network outages, and so on without failing or losing data.
Improved scalability: Service Broker routing delivers messages based
on the name of the service, rather than on the network address of the
computer where the service runs. This allows you to install an
application on multiple computers without changing application code.
Ability to leverage existing knowledge: Service Broker uses
Transact-SQL to create objects. Applications that use Service Broker
are most often implemented in Transact-SQL or Microsoft .NET
Framework-compatible languages. You do not have to learn a new
language to create Service Broker applications.
For further detail check this link
http://dotnet.dzone.com/articles/c-sqldependency-monitoring
Service brokers are useful when we want do asynchronous processing in the database. which will improve the performance.
When we need inter database communication.
you can find sample code for creating and sending the messages using service broker here.