I'd like to split exchange message body (it's list of MyCustomClass object), process them (one by one) and them aggregate the all exchanges together. Split is ok, process one by one also ok, but I can't figure out how to aggregate them.
from("mysource")
.unmarshal(new ListJacksonDataFormat(MyClass.class))
.split().body()
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
// process MyClass item
exchange.getIn().setBody(processedItem);
}
})
.to("destinationForProcessedItem")
.aggregate(new GroupedExchangeAggregationStrategy()) <== Seems like problem is here
.process(new Processor() {
// handle result of aggregation
})
I don't need complicated aggregation, just collect list of splitted Exchanges and handle them in the final processor.
Use the built-in aggregator in the splitter, see the composed message processor EIP pattern: https://camel.apache.org/components/latest/eips/composed-message-processor.html#_sample.
write like this
.aggregate(new AggregationStrategy() {
#Override
public Exchange aggregate(Exchange exchange, Exchange exchange1) {
//logic for aggregation using exchnage and exchange1
}
})
Related
I have the following route:
from("file:/home/tmp/test?move=.done")
.routeId("file")
.split(body().tokenize("\n"),new GroupedBodyAggregationStrategy())
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
exchange.getMessage().setHeader("Test", "test");
System.out.println(exchange.getIn().getBody());
}
})
and the file that is being consumed is:
1234,56676,2345
234,213,412
124,423,5236
I would expect only one exchange to reach the processor after split and that its body contains all the above lines, however it behaves like I do not use an aggregation strategy at all. 3 exchanges reach the processor each with the body that corresponds to a single line. Also, it does not matter if I use GroupedBodyAggregationStrategy or a custom one, it is always the same. What am I doing wrong?
Give a look here enter link description here
What you look is :
from("file:/home/tmp/test?move=.done")
.routeId("file")
.split(body().tokenize("\n"),new GroupedBodyAggregationStrategy())
// instructions on each message from the split (.ie each row here)
.end()
// instructions on aggregated result
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
exchange.getMessage().setHeader("Test", "test");
System.out.println(exchange.getIn().getBody());
}
})
I am using Camel JMS component for request-reply for communication with MQ. For some of my requests I can receive n messages in reply. How can I aggregate these reply messages?
I thought of using aggregator pattern with aggregation strategy, but can't use it as I am not sure on number of messages which can come in reply.
Can community help me understand what's the right way to do it? I did some google search but couldn't find something useful. Below is my sample route code
from("direct:"+routeName).routeId(routeName)
.setHeader("JMSCorrelationID", constant(UUID.randomUUID().toString()))
.circuitBreaker()
.resilience4jConfiguration()
.minimumNumberOfCalls(3)
.end()
.to(mqComponentBeanName+"://CAMELDEMO?exchangePattern=InOut&requestTimeout=10000&replyTo=CAMELDEMOREPLY")
.log("${body}")
.unmarshal(customerDetailsOutBound)
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
System.out.println(exchange.getIn().getBody().toString());
}
})
.onFallback().process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
System.out.println("Store this message to backup");
}
})
.end();
Looking forward to get some good insights from community. Thank you.
Message flow
your first route sends a message to CAMELDEMO queue and start waiting for a single aggreagted message on a new queue CAMELDEMO_AGGREGATED_REPLY
component that received the message on CAMELDEMO, start sending responses to CAMELDEMOREPLY queue and also indicates how many responses will be sent
Second route below starts listening on CAMELDEMOREPLY, aggregates the message and send the aggregated message to CAMELDEMO_AGGREGATED_REPLY.
Your first route that was waiting for the reply on CAMELDEMO_AGGREGATED_REPLY gets the aggregated reply, receives single message and sends it back
Original route updated to await for reply on CAMELDEMO_AGGREGATED_REPLY
...
.to(mqComponentBeanName+"://CAMELDEMO?exchangePattern=InOut&requestTimeout=10000&
replyTo=CAMELDEMO_AGGREGATED_REPLY")
.log("${body}")
.unmarshal(customerDetailsOutBound)
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
System.out.println(exchange.getIn().getBody().toString());
}
})
....
Second route to aggregate the messages
from(mqComponentBeanName+"://CAMELDEMOREPLY?
exchangePattern=In&requestTimeout=10000)
.aggregate(header("JMSCorrelationID"), new MyAggregationStrategy())
.to(mqComponentBeanName+"://CAMELDEMO_AGGREGATED_REPLY?
exchangePattern=Out&requestTimeout=10000)
public final class MyCompletionStrategy implements AggregationStrategy {
#Override
public Exchange aggregate(Exchange oldExch, Exchange newExchange)
{
...
//Here you check your flag regarding the number of responses
// you were supposed to receive, and if it is met
// complete the aggregation by setting it to true
oldExch.setProperty(Exchange.AGGREGATION_COMPLETE_CURRENT_GROUP, true);
...
return oldExchange;
}
}
I was able to solve this with single route. Solution may not be that neat, but works and fulfils the purpose. I have used loopDoWhile and in the processor inside loopDoWhile I am fetching message from queue using plain java code.
from("direct:"+routeName).routeId(routeName)
.setHeader("JMSCorrelationID", constant(UUID.randomUUID().toString()))
.circuitBreaker()
.resilience4jConfiguration()
.minimumNumberOfCalls(3)
.end()
.to(mqComponentBeanName+"://CAMELDEMO?exchangePattern=InOut&requestTimeout=10000&replyTo=CAMELDEMOREPLY")
.log("${body}")
.unmarshal(customerDetailsOutBound)
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
System.out.println(exchange.getIn().getBody().toString());
int msgCount = getMsgCountfromFirstReposnse;
if (msgCount > 1) {
exchange.getIn().setHeader("COUNTER", 0);
exchange.getIn().setHeader("MSG_COUNT", msgCount-1);
exchange.setProperty("connectionFactory", connectionFactory);
}
}
})
.loopDoWhile(simple("${headers.COUNTER} != ${headers.MSG_COUNT}"))
.process(simpleJMSConsumerProcess)
.end().endCircuitBreaker()
.onFallback().process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
System.out.println("Store this message to backup");
}
})
Code inside processor:
ConnectionFactory connectionFactory = (ConnectionFactory) exchange.getProperty("connectionFactory");
Connection connection = connectionFactory.createConnection();
Session session = connection.createSession(false,
Session.AUTO_ACKNOWLEDGE);
try {
Queue queue = session.createQueue("CAMELDEMOREPLY?consumer.priority=10");
MessageConsumer consumer = session.createConsumer(queue, "JMSCorrelationID = '"+exchange.getIn().getHeader("JMSCorrelationID").toString()+"'");
connection.start();
TextMessage textMsg = (TextMessage) consumer.receive();
System.out.println(textMsg);
System.out.println("Received: " + textMsg.getText());
exchange.getIn().setHeader("COUNTER", ((Integer)exchange.getIn().getHeader("COUNTER"))+1);
if (connection != null) {
connection.close();
}
} finally {
if (session != null) {
session.close();
}
if (connection != null) {
connection.close();
}
}
Well, traditional request-reply has by design just 1 reply message. The thread waiting for the response stops listening as soon as the first reply arrives.
With JMS correlation IDs (no dedicated thread per request) it would theoretically be possible to receive multiple replies for the same request, but I don't know if this really works/is allowed in JMS.
Update based on comments
You write in comments that you are able to receive multiple JMS replies for one request and that you even get the number of answers to expect.
If this all works, you can use the Aggregator EIP in your Camel route to collect all responses before sending a reply to the caller.
The Aggregator is highly configurable. You can decide how to combine the responses and you can also define multiple completion criteria (timeout, number of messages etc).
lets say, I have
from("direct:start").filter("...")
How can I filter out messages coming from an endpoint like smtp? Is there a way to check the type of the previous endpoint?
Edit: As per advice from vikingsteve, I have implemented the code this way:
from("direct:source")
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
exchange.setProperty("source", "smtps");
}
})
.to("direct:start");
from("direct:start").filter(
new Predicate() {
#Override
public boolean matches(Exchange exchange) {
if(exchange.getProperty("source") == "smtps")
return true;
else
return false;
}}));
You would normally set an exchange property if you need to identify some information about the original message (such as where it came from, or some other metadata not contained in the body).
from("smtp:...").setProperty("source", "smtp").to("...")
edit: here's a simpler version of your solution, I havent tested it, but it might work like this:
from("direct:source")
.setProperty("source", "smtps")
.to("direct:start");
from("direct:start")
.filter(exchangeProperty("source").isEqualTo("smtp"))
I'm using camel-hystrix-eip in my project, I want a processor before my fallback, but it's not working
from("direct:sample").id("id:direct:sample").process(requestProcessor)
.hystrix()
.to(endPoint)
.onFallbackViaNetwork()
.to(fallback_endPoint)
I want to alter my fallback_endpoint using a processor, but seems like after onFallbackViaNetwork() we have to immediately provide to().
please suggest if there any way to do so.
I tried something like below, but it's not working.
from("direct:sample").id("id:direct:sample").process(requestProcessor)
.hystrix()
.to(endPoint)
.onFallbackViaNetwork()
.process(fallbackProcessor)
.to(fallback_endPoint)
Actually, I'm using requestProcessor to override the actual endpoint, and in case of a fallback, fallback_endPoint is also getting overridden, is there any way to avoid this.
You can have a processor after onFallbackViaNetwork(). You can also use the toD EIP to send the message to a dynamic endpoint.
Based on your code, you could set a Header MyEndpoint which contains your new endpoint string, and then reference it using .toD("${header.MyEndpoint}"). Repeat this pattern whenever you need to set a dynamic endpoint.
For example:
from("direct:sample")
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
// do something
exchange.getIn().setHeader("EndpointHystrix", "mock:hystrix");
}
})
.hystrix()
.toD("${header.EndpointHystrix}")
.onFallbackViaNetwork()
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
// do something more
exchange.getIn().setHeader("EndpointFallback", "mock:fallback");
}
})
.toD("${header.EndpointFallback}")
.end()
.to("...");
I've tested this in Camel 2.20.0 and 2.21.0.
We defined a route in Camel with split and aggregate functionality, but can't propagate the exception back to split after the aggregator. Which cause's the split to run even if we encounter an exception
below is the code which is not working
from("direct:MyRoute")
.routeId("MyRouteID")
.split().tokenize("\n", 1)
.streaming().stopOnException()
.choice()
.when(simple("${property.CamelSplitIndex} > 0"))
.unmarshal(domainDataFormat)
.choice()
.when(simple("${property.CamelSplitComplete}"))
.process(
new Processor()
{
#Override
public void process(Exchange exchange) throws Exception
{
exchange.getIn().getHeaders().put(Exchange.AGGREGATION_COMPLETE_ALL_GROUPS_INCLUSIVE, true);
}
}
)
.end()
.aggregate(myAggregationStrategy).constant(true) //if i comment this line split will be stop on exception
.threads().executorService(executorService)
.process(myProcessor).end()
.end();
Processor(myProcessor) from the above code is below:
counter.incrementAndGet(); //atomic counter
if(counter.get()==3)
{
exchange.setException(new RuntimeException());
throw new RuntimeCamelException();
}
But, the moment I remove the aggregate from the route, Split is able to stop the route on Exception.
Use the composite message processor EIP which is split + aggregate (fork/join) in the same unit of work.
See docs at: https://camel.apache.org/components/next/eips/composed-message-processor.html
And see the splitter only section where you can specify an aggregation strategy to the splitter to have it work together.