We have two wildlfy 10.1.0 server with the activeMq (Artemis) running. We want the two artemis to be in sync with messages. And if one is down, other one should start receiving and sending messages. There is lack of documentation in google as well as artemis is new.
Also, keeping both as active for loadbalancing., is it possible?
I am assuming that you are using the internal instance of Artemis in wildfly. In the Wildfly Quickstarts, there is a good demo on Messaging Clustering. This will also work for 10.1. It shows how to configure using Standalone-HA or Domain
https://github.com/wildfly/quickstart/tree/11.x/messaging-clustering.
Related
I have built Salesforce platform event client using CometD java and It's similar to example EMP-Connector provided by forcedotcom.
I installed this client on OpenShift cloud and my app is running with 2 pods. Problem I am facing is that since there are two pods and each are running the same docker image thus both are getting the same of event. That means duplication event.
Per my understanding, Salesforce platform event should behave like Kafka subscriber.
I am unable to find a solution about how to avoid getting the same copy of events. Any suggestions here would be the great help.
Note: As of now I am able to create client side solution which drop duplicate copy of event. which is not an optimal solution.
I have to run my app atleast with 2 pods. That's limit I have on my cloud.
This is expected / by design. In CometD, when a message is published on a broadcast channel all subscribers listening on this channel will receive a copy of this message. The broadcast channel behaves like a messaging topic where one sender wants to send the same info to multiple recipients. There are other types of channels in CometD with different semantics. The broadcast channel and one-to-many message semantics is what you get with platform events available via CometD in Salesforce.
In your case it sounds like you have multiple subscribers, thus what you're seeing is expected. You can deduplicate the message stream on the client side as you have done or you can change your architecture so that you have a single subscription.
I'm using ActiveMQ Artemis version 2.6.2 and using Apache Camel to route messages.
When I connect Camel with jms-component in AMQ, for some reason in ActiveMQ Artemis the new queue is created with jms.queue. as a prefix.
I know if I add the follow code on acceptor in broker.xml the problem is solved. But sadly I don't have access to do that.
anycastPrefix=jms.queue.;multicastPrefix=jms.topic.
Is there a way to solve this in Java Code? I tried these steps but no success.
from("amq:QUEUE.TEST").setProperty("anycastPrefix", simple("jms.queue."))
from("amq:jms:queue:QUEUE.TEST")
The reason that the queue is being created with the jms.queue. prefix is almost certainly because Camel is using an Artemis 1.x client instead of a 2.x client. The 1.x client is hard-coded to use the jms.queue. and jms.topic. prefixes.
As you note, the simplest way to solve this issue is by configuring the prefixes on the appropriate acceptor in broker.xml. I don't know of any way to solve this issue in Java code. I think your best bet would be just to upgrade the Artemis client implementation which Camel is using.
The MuleSoft HTTP ListenerConnector transforms an HTTP request into a Mule Message. It is my understanding that, as a consequence, the message is stored in a queue - therefore it can be guaranteed that the message will be delivered (once the request reached the integration platform).
In contrast, apache camel does not come with a messaging system. Therefore, I assume, the message is not stored anywhere and is lost when the destination of the message is temporarily not available.
Is that right? When using camel together with some messaging system, can the mule behavior be implemented easily?
Yes, that is true Camel does not come with a messaging or runtime system. You need to do decide on those yourself. I have used Camel with Karaf and messaging system such as ActiveMQ and RabbitMQ. You can easily ensure delivery use both of those together with the correct configuration. Both support persistent messaging, persistent queues and redelivery. Off course the main to do decide is, what happens if the host running your eventbus dies. In that case you need to consider clustering. The same applies for Mule.
My idea is to use camel to decouple modules. In order to support scalability and failover, I am wondering if the following architecture is adviced?
I have two applications with Camel embedded AppCamel1 and AppCamel2. Then I have standalone camel nodes Camel1 and Camel2.
AppCamel1 would have a route with fail-over/load balancing to Camel1 and Camel2. This way, if Camel1 crashes for example, Camel2 is used for failover.
Camel1 and 2 would do a REST call with the http component for example. Also there would be a request-reply from AppCamel1 up to camel1 or 2.
Is it a valid scenario?
What should I use to interconnect the different Camel instances (AppCamel1 to Camel1 or 2)? (I would like to know if it's possible to avoid another component like a jms server in the middle)
Thank you!
Edited following Boday's answer
the REST calls are from Camel1/2. I'd like to interconnect AppCamel1/2 to Camel1/2 and see if I can avoid anything in between. I guess mina is a possibility or even http but in that case a AppCamel1 and AppCamel2 need to know Camel1/2 which is not so good.
This is also being discussed at the Camel mailing list, where there is also some pointers and suggestions
http://camel.465427.n5.nabble.com/scalable-bus-with-multiple-Camel-instances-tp5606593p5606593.html
If you are trying to load balance HTTP requests to your AppCamel1/2, then you'd need a proxy server in between (apache mod_proxy, perlbal, etc). To load balance from AppCamel1/2 to Camel1/2, you can use Camel's load balancer or even JMS request/reply...
From AppCamel1/2 to Camel1/2, it sounds like you are using REST as the interface. If you need more complex communication between the instances, then I'd use JMS (via camel-activemq) for messaging and Hazelcast (via camel-hazelcast) for distributed caching/locking, etc.
If you use jms to communicate then you do not need a special load balancer. Just use one queue and let both Camel1/2 listen to the queue. Then they will automatically failover and load balance.
I would definetly go for a jms middleware. Activemq is the natural choice (camel is even considered a sub project of activemq). It is trivial to embedd amq along with your canel instances and cluster them. Activemq will then be able to handle both load balancing and failover for you.
I am in the process of learning ActiveMQ and Camel, with the goal to create a little prototype system that works something like this:
(source: paulstovell.com)
(big)
When an order is placed in the Orders system, a message is sent out to any subscribers (a pub/sub system), and they can play their part in processing the order. The Orders, Shipping and Invoicing applications have their own ActiveMQ installations, so that if any of the three systems are offline, the others can continue to function. Something takes care of moving messages between the ActiveMQ installs.
Getting Apache Camel to move messages from one queue to another via routes is quite easy, if they are on the same ActiveMQ instance. So this works for managing the subscription queues.
The next challenge is pushing messages from one ActiveMQ instance to another, and it's the bit where I am not sure what to look at next.
Can Camel route between different ActiveMQ installations? (I can't figure out what the JMI endpoint URI would be if they are on different machines).
I understand ActiveMQ has store and forward capabilities. Is this what I would use to move messages between Orders and Shipping/Invoicing?
Or is this what Apache ServiceMix is meant to solve?
This is a pretty straightforward asynchronous, event-driven application that is well-suited for ActiveMQ and Camel.
Actually you do not move messages explicitly from one ActiveMQ instance to another. The way it works is using what's known as a network of brokers. In your case, you'd have three brokers: ActiveMQ-purple, ActiveMQ-green and ActiveMQ-blue. ActiveMQ-purple creates a uni-directional broker network with ActiveMQ-green and ActiveMQ-blue. This allows ActiveMQ-purple to store-and-forward messages to ActiveMQ-green and ActiveMQ-blue based on consumer demand.
The Orders app accepts orders on the orders queue on ActiveMQ-purple. The Orders app uses Camel to consume and process a message to determine if it is an invoicing message or a shipping message. Camel routes the messages to either the invoicing queue or the shipping queue on ActiveMQ-purple.
Consumer demand comes from the Invoicing app and the Shipping app. The Invoicing uses Camel to consume messages from the invoicing queue on ActiveMQ-green. The Shipping app uses Camel to consume messages from the shipping queue on ActiveMQ-blue. Because of the broker network and because of the consumer demand on the ActiveMQ-green.invoicing queue and the ActiveMQ-blue.shipping queue, messages will be forwarded from ActiveMQ-purple to the appropriate broker and queue. There is no need to explicitly route messages to specific broker.
I hope this answers your questions. Let me know if you have anymore.
Bruce
Hmmmm, I've only dabbled at best, and not for a fair while, but I'll try and offer something.
ActiveMQ can route between different installations and just uses standard URIs to my knowledge so I'm not sure what the problem is here. I would think that using TCP you'd be fine. Using ServiceMix (you mention it later) you'd just specify a connectionFactory & then provide the URI in that. This link shows some examples http://servicemix.apache.org/servicemix-jms-new-endpoints.html.
Camel has support for Durable Subscriber if that's what you were after (http://camel.apache.org/durable-subscriber.html)? This pattern will ensure that if the subscriber is offline when the message is ready, it will be held until the subscriber is back online. This is also supported by ServiceMix (see link given above and look for 'subscriptionDurable'.