The apache camel API doc for the Producer interface says "Producer interface provides a channel on which clients can create and invoke message exchanges on an Endpoint". (https://camel.apache.org/maven/current/camel-core/apidocs/org/apache/camel/Producer.html)
If we take the Producer implementation of a camel weather component , it has a process method in weather information is requested. But I do not see where a "channel" as per EIP is provided (http://www.enterpriseintegrationpatterns.com/patterns/messaging/MessageChannel.html) . Does "channel" in this sentence does not refer to to channel in EIP?
The EIP patterns and the EIP book are covering some patterns that are on a conceptual/abstract level. And the EIP book takes offset in a messaging world, where these patterns are doing messaging exchanges with messaging, and then the channel represents message queues/topics etc.
Related
I am new to Messaging and Integration. As I am trying to understand how Apache Camel makes EIP easy, Message Router caught my attention to a specific scenario where it consumes a message from one queue and transfers to some other queue based on a predicate.
Here are my questions:
Why use Message Router to perform this task?
Why not the message Producer directly send the message to the right Queue/Destination?
Why not the Consumer can rely on Message Selectors to do the same?
Can someone share your thoughts on real life use cases?
Well, your questions are more about SoC than messaging.
The Message Router is an EIP that can perform this task well and with Camel it is also very easy to implement, but as you write, there are other possibilities too.
If the message producer (client) sends messages directly to final destinations, it must know about all possible destinations. It must also be changed whenever a destination is added/removed or changed. This is tight coupling and in most situations not desirable.
If the message receiver(s) consume(s) with message selectors, the client can send all messages to the same queue. This is totally OK, you can implement individual consumers with different selectors. The selector feature of the Broker basically enables you to "divide a queue into multiple queues".
So if you want to implement and run an intermediate integration component between the client and the receivers, you can use the Message Router in this component.
If you want to omit this integration component, you can use message selectors to implement the receivers individual and de-coupled from each other.
For an IoT project I am working on I am researching a next, enhanced, version of our “Socket Handler” which is over 5 years of age and has evolved in a big beast doing apart from handling socket connections with IoT devices also in thread processing that has become a real pain to manage.
For my total rewrite I am looking into Apache Camel as a routing and transformation toolkit and understand how this can help us split processing steps into micro-services, loosely coupled through message queues.
One thing that I have trouble understanding however is how I can implement the following logic “the Apache Camel way”:
An IoT device sends an initial message which contains its id, some extra headers and a message payload.
Apart from extracting the message payload and routing it to a channel, I also need to use the device Id to check a message queue, named after the device id, for any commands that have to go to the device over the same socket connection that received the initial message.
Although it seems that Netty4, which is included in Camel, can deal with synchronous duplex comms, I cannot see how the above logic can be implemented in the Camel Netty4 component. Camel Routing seems to be one way only.
Is there a correct way to do this or should I forget about using camel for this and just use Netty4 bare?
It is possible to reproduce a full duplex communication by using two Camel routes :
thanks by using reuseChannel property
As this full duplex communication will be realised thanks to two different Camel routes, sync property must be set to false.
Here the first route:
from("netty4:tcp://{{tcpAddress}}:{{tcpPort}}?decoders=#length-decoder,#string-decoder&encoders=#length-encoder,#bytearray-encoder&sync=false&reuseChannel=true") .bean("myMessageService", "receiveFromTCP").to("jms:queue:<name>")
This first route will create a TCP/IP consumer thanks to a server socket (it is also possible to use a client socket thanks to property clientMode)
As we want to reuse the just created connection, it is important to initialize decoders but also encoders that will be used later thanks to a bean (see further). This bean will be responsible of sending data by using the Channel created in this first route (Netty Channel contains a pipeline for decoding / encoding messages before receiving from / sending to TCP/IP.
Now, we want to send back some data to the parter connected to the consumer (from) endpoint of the first route. Since we are not able to do it thanks to classical producer endpoint (to), we use a bean object :
from("jsm:queue:<name>").bean("myMessageService", "sendToTCP");
Here the bean code :
public class MessageService {
private Channel openedChannel;
public void sendToTCP(final Exchange exchange) {
// opened channel will use encoders before writing on the socket already
// created in the first route
openedChannel.writeAndFlush(exchange.getIn().getBody());
}
public void receiveFromTCP(final Exchange exchange) {
// record the channel created in the first route.
this.openedChannel = exchange.getProperty(NettyConstants.NETTY_CHANNEL, Channel.class);
}
}
Of course,
the same bean instance is used in the two routes, you need to use a registry to do so :
SimpleRegistry simpleRegistry = new SimpleRegistry();
simpleRegistry.put("myMessageService", new MessageService());
Since the bean is used in two different asynchronous routes, you will have to deal with some unexected situations, by for example protecting the access of openedChannel member, or dealing with unexpected deconnexion.
This post helped me a lot to find this solution :
how-to-send-a-response-back-over-a-established-tcp-connection-in-async-mode-usin
reuseChannel property documentation
After end of camel route, exchange's body and headers will back to requester as response.
I am trying to get time taken for a message just to be delivered to the IBM MQ queues in Apache Camel. Below is the route configuration,
from("direct:deliver-route-2").process("mySampleProcessor").split().method("messageSplitterBean", "splitMessage").shareUnitOfWork().stopOnException().toD("mqDeliverJms:${headers.Deliver}?preserveMessageQos=true");
I have a splitter in place to split the list of messages and dynamically deliver (toD) it using the messages header "Deliver".
I used the MessageHistory suggestion given in one of the questions, but then I don't see the elapsed time for the (toD) delivery of the messages to IBM MQ.
String MessageHist = org.apache.camel.util.MessageHelper.dumpMessageHistoryStacktrace(exchange,
new DefaultExchangeFormatter(), false);
Here is the message history,
You can use the 'EventNotifier' to capture the ExchangeSent events where you have the time, eg there is little example in the Camel docs at: http://camel.apache.org/eventnotifier-to-log-details-about-all-sent-exchanges.html
Is it possible to implement asynchronous request-reply pattern using two JMS message broker instances? Such that service A sends message to queue A, and gets the response from queue B (different broker instance)?
Does JMS API (or Apache Camel) provide some complete mechanism to achieve this transparently with correlation identifiers? What is the necessary configuration?
Bonus question:
To shuffle deck even more, I would like to cluster the queues. Could this be achieved transparently as per specification? Basically I have multiple Spring boot applications (services) with ActiveMQ broker embedded in the Spring context. Each broker acts as an one-way channel for the service, and each service excepts a response for a specific message to other service in it's own broker. Now, I would like run multiple instances of each services and retain the correlation between messages.
Since your request sender (A) and response receiver (B) are two different services - it becomes your responsiblity to store context of original request somewhere.
Probably the easiest is to use a dedicated queue for this purpose (let's call it echo).
(A) could store original request message with some specific correlation Id, for example taken from message Id of request sent. The same correlation Id is supposed to be set for response message, which will be taken by (B). So once response is recieved by (B), it is able to get the original request from echo queue, using selector with the same correlation Id.
I am in the process of learning ActiveMQ and Camel, with the goal to create a little prototype system that works something like this:
(source: paulstovell.com)
(big)
When an order is placed in the Orders system, a message is sent out to any subscribers (a pub/sub system), and they can play their part in processing the order. The Orders, Shipping and Invoicing applications have their own ActiveMQ installations, so that if any of the three systems are offline, the others can continue to function. Something takes care of moving messages between the ActiveMQ installs.
Getting Apache Camel to move messages from one queue to another via routes is quite easy, if they are on the same ActiveMQ instance. So this works for managing the subscription queues.
The next challenge is pushing messages from one ActiveMQ instance to another, and it's the bit where I am not sure what to look at next.
Can Camel route between different ActiveMQ installations? (I can't figure out what the JMI endpoint URI would be if they are on different machines).
I understand ActiveMQ has store and forward capabilities. Is this what I would use to move messages between Orders and Shipping/Invoicing?
Or is this what Apache ServiceMix is meant to solve?
This is a pretty straightforward asynchronous, event-driven application that is well-suited for ActiveMQ and Camel.
Actually you do not move messages explicitly from one ActiveMQ instance to another. The way it works is using what's known as a network of brokers. In your case, you'd have three brokers: ActiveMQ-purple, ActiveMQ-green and ActiveMQ-blue. ActiveMQ-purple creates a uni-directional broker network with ActiveMQ-green and ActiveMQ-blue. This allows ActiveMQ-purple to store-and-forward messages to ActiveMQ-green and ActiveMQ-blue based on consumer demand.
The Orders app accepts orders on the orders queue on ActiveMQ-purple. The Orders app uses Camel to consume and process a message to determine if it is an invoicing message or a shipping message. Camel routes the messages to either the invoicing queue or the shipping queue on ActiveMQ-purple.
Consumer demand comes from the Invoicing app and the Shipping app. The Invoicing uses Camel to consume messages from the invoicing queue on ActiveMQ-green. The Shipping app uses Camel to consume messages from the shipping queue on ActiveMQ-blue. Because of the broker network and because of the consumer demand on the ActiveMQ-green.invoicing queue and the ActiveMQ-blue.shipping queue, messages will be forwarded from ActiveMQ-purple to the appropriate broker and queue. There is no need to explicitly route messages to specific broker.
I hope this answers your questions. Let me know if you have anymore.
Bruce
Hmmmm, I've only dabbled at best, and not for a fair while, but I'll try and offer something.
ActiveMQ can route between different installations and just uses standard URIs to my knowledge so I'm not sure what the problem is here. I would think that using TCP you'd be fine. Using ServiceMix (you mention it later) you'd just specify a connectionFactory & then provide the URI in that. This link shows some examples http://servicemix.apache.org/servicemix-jms-new-endpoints.html.
Camel has support for Durable Subscriber if that's what you were after (http://camel.apache.org/durable-subscriber.html)? This pattern will ensure that if the subscriber is offline when the message is ready, it will be held until the subscriber is back online. This is also supported by ServiceMix (see link given above and look for 'subscriptionDurable'.