I am using rabbitmq component in camel. I have a following route:
public void configure() throws Exception {
from("rabbitmq://localhost:5672/test_op?queue=out_queue&routingKey=test_out&username=guest&password=guest" +
"&autoAck=false&durable=true&exchangeType=direct&autoDelete=false&exchangePattern=InOut")
.aggregate(constant(true), new ArrayListAggregationStrategy())
.completionSize(2000).completionTimeout(60000).eagerCheckCompletion()
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
Message m = exchange.getIn();
org.apache.camel.TypeConverter tc = exchange.getContext().getTypeConverter();
String strValue = tc.convertTo(String.class, m.getBody());
System.out.println("[[out_queue]]: " + strValue);
}
});
}
Problem is that the use of aggregate is acknowledging the message to rabbitmq even before process() is called. I want to acknowledge message only when process() execution is successful, and not when aggregate is invoked. How can I achieve this?
FYI: Without aggregate this route works as expected. That means it acknowledges message only when process() is executed successfully.
Related
Issue:
I have multiple route sending messages to an ActiveMQ queue which later gets processed by a processor to save status information
The same message sending from ProducerTemplate to ActiveMQ queue somehow breakes the same code by not triggering logs on console, and saving status information to a randomly generated file name.
Desired Behavior:
Both sending method gets the messages processed the same way
Code Explanation:
On below codes the Save processor is the one producing the weird behavior where logs dont show up on console and writes to some random file, file name is basically the clientID from ActiveMQ
StartRoute calling the activemq:save works correctly
Code:
public class Save implements Processor {
public void process(Exchange exchange) throws Exception {
try {
Map<String, Object> map = new HashMap<String, Object>() {};
map.put(".....", ".....");
map.put(".....", ".....");
map.put(".....", ".....");
map.put(".....", ".....");
ProducerTemplate template = exchange.getContext().createProducerTemplate();
String response = template.requestBodyAndHeaders("activemq:save", "Batch Job Started", map, String.class);
FluentProducerTemplate FluentTemplate = exchange.getContext().createFluentProducerTemplate();
String result = FluentTemplate
.withHeader(".....", ".....")
.withHeader(".....", ".....")
.withHeader(".....", ".....")
.withHeader(".....", ".....")
.withHeader(".....", ".....")
.to("activemq:save")
.request(String.class);
} catch (Exception e) {
.....
}
}
}
public class StartRoute extends RouteBuilder {
restConfiguration()
.component("servlet")
.enableCORS(true);
rest("/start").id("start")
.post("/{filename}")
.route()
.....
.process(new Save())
.wireTap(startBackgroundProcessRoute)
.to("activemq:save")
.endRest();
}
public class SaveRoute extends RouteBuilder {
from("activemq:save")
.log("started saving")
.process(new FormatText())
.to("file:///status");
}
This question came from my original problem described here:
Camel Multicast Route call order
Solution can be found also there:
Camel Multicast Route call order
However to satisfy this question:
It seems Camel have few bugs regarding producer templates and maybe ActiveMQ, this is my initial conclusion.
The Only way i was able to use ProducerTemplate without issue is to use the send function with Exchanges, send() sends the messages correctly to the ActiveMQ same way as the to() however for whatever reason the content was still not written to the file.
After I dropped the ActiveMQ between the routes, everything started to work consistently. So possible miss configuration on ActiveMQ component or possible another camel framework bug.
If anyone knows the exact answer i would be happy to hear / see the reason for this behavior.
Code example:
public class SaveProcessor implements Processor {
public void process(Exchange exchange) throws Exception {
ProducerTemplate template = exchange.getContext().createProducerTemplate();
template.send(Utilities.trackBatchJobStatus, exchange);
/** NOTE
* DO NOT USE any other functions which are not working with EXCHANGES.
* Functions that uses body, header values and such are bugged
*/
}
}
I have the following route:
from("file:/home/tmp/test?move=.done")
.routeId("file")
.split(body().tokenize("\n"),new GroupedBodyAggregationStrategy())
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
exchange.getMessage().setHeader("Test", "test");
System.out.println(exchange.getIn().getBody());
}
})
and the file that is being consumed is:
1234,56676,2345
234,213,412
124,423,5236
I would expect only one exchange to reach the processor after split and that its body contains all the above lines, however it behaves like I do not use an aggregation strategy at all. 3 exchanges reach the processor each with the body that corresponds to a single line. Also, it does not matter if I use GroupedBodyAggregationStrategy or a custom one, it is always the same. What am I doing wrong?
Give a look here enter link description here
What you look is :
from("file:/home/tmp/test?move=.done")
.routeId("file")
.split(body().tokenize("\n"),new GroupedBodyAggregationStrategy())
// instructions on each message from the split (.ie each row here)
.end()
// instructions on aggregated result
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
exchange.getMessage().setHeader("Test", "test");
System.out.println(exchange.getIn().getBody());
}
})
I am using Camel JMS component for request-reply for communication with MQ. For some of my requests I can receive n messages in reply. How can I aggregate these reply messages?
I thought of using aggregator pattern with aggregation strategy, but can't use it as I am not sure on number of messages which can come in reply.
Can community help me understand what's the right way to do it? I did some google search but couldn't find something useful. Below is my sample route code
from("direct:"+routeName).routeId(routeName)
.setHeader("JMSCorrelationID", constant(UUID.randomUUID().toString()))
.circuitBreaker()
.resilience4jConfiguration()
.minimumNumberOfCalls(3)
.end()
.to(mqComponentBeanName+"://CAMELDEMO?exchangePattern=InOut&requestTimeout=10000&replyTo=CAMELDEMOREPLY")
.log("${body}")
.unmarshal(customerDetailsOutBound)
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
System.out.println(exchange.getIn().getBody().toString());
}
})
.onFallback().process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
System.out.println("Store this message to backup");
}
})
.end();
Looking forward to get some good insights from community. Thank you.
Message flow
your first route sends a message to CAMELDEMO queue and start waiting for a single aggreagted message on a new queue CAMELDEMO_AGGREGATED_REPLY
component that received the message on CAMELDEMO, start sending responses to CAMELDEMOREPLY queue and also indicates how many responses will be sent
Second route below starts listening on CAMELDEMOREPLY, aggregates the message and send the aggregated message to CAMELDEMO_AGGREGATED_REPLY.
Your first route that was waiting for the reply on CAMELDEMO_AGGREGATED_REPLY gets the aggregated reply, receives single message and sends it back
Original route updated to await for reply on CAMELDEMO_AGGREGATED_REPLY
...
.to(mqComponentBeanName+"://CAMELDEMO?exchangePattern=InOut&requestTimeout=10000&
replyTo=CAMELDEMO_AGGREGATED_REPLY")
.log("${body}")
.unmarshal(customerDetailsOutBound)
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
System.out.println(exchange.getIn().getBody().toString());
}
})
....
Second route to aggregate the messages
from(mqComponentBeanName+"://CAMELDEMOREPLY?
exchangePattern=In&requestTimeout=10000)
.aggregate(header("JMSCorrelationID"), new MyAggregationStrategy())
.to(mqComponentBeanName+"://CAMELDEMO_AGGREGATED_REPLY?
exchangePattern=Out&requestTimeout=10000)
public final class MyCompletionStrategy implements AggregationStrategy {
#Override
public Exchange aggregate(Exchange oldExch, Exchange newExchange)
{
...
//Here you check your flag regarding the number of responses
// you were supposed to receive, and if it is met
// complete the aggregation by setting it to true
oldExch.setProperty(Exchange.AGGREGATION_COMPLETE_CURRENT_GROUP, true);
...
return oldExchange;
}
}
I was able to solve this with single route. Solution may not be that neat, but works and fulfils the purpose. I have used loopDoWhile and in the processor inside loopDoWhile I am fetching message from queue using plain java code.
from("direct:"+routeName).routeId(routeName)
.setHeader("JMSCorrelationID", constant(UUID.randomUUID().toString()))
.circuitBreaker()
.resilience4jConfiguration()
.minimumNumberOfCalls(3)
.end()
.to(mqComponentBeanName+"://CAMELDEMO?exchangePattern=InOut&requestTimeout=10000&replyTo=CAMELDEMOREPLY")
.log("${body}")
.unmarshal(customerDetailsOutBound)
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
System.out.println(exchange.getIn().getBody().toString());
int msgCount = getMsgCountfromFirstReposnse;
if (msgCount > 1) {
exchange.getIn().setHeader("COUNTER", 0);
exchange.getIn().setHeader("MSG_COUNT", msgCount-1);
exchange.setProperty("connectionFactory", connectionFactory);
}
}
})
.loopDoWhile(simple("${headers.COUNTER} != ${headers.MSG_COUNT}"))
.process(simpleJMSConsumerProcess)
.end().endCircuitBreaker()
.onFallback().process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
System.out.println("Store this message to backup");
}
})
Code inside processor:
ConnectionFactory connectionFactory = (ConnectionFactory) exchange.getProperty("connectionFactory");
Connection connection = connectionFactory.createConnection();
Session session = connection.createSession(false,
Session.AUTO_ACKNOWLEDGE);
try {
Queue queue = session.createQueue("CAMELDEMOREPLY?consumer.priority=10");
MessageConsumer consumer = session.createConsumer(queue, "JMSCorrelationID = '"+exchange.getIn().getHeader("JMSCorrelationID").toString()+"'");
connection.start();
TextMessage textMsg = (TextMessage) consumer.receive();
System.out.println(textMsg);
System.out.println("Received: " + textMsg.getText());
exchange.getIn().setHeader("COUNTER", ((Integer)exchange.getIn().getHeader("COUNTER"))+1);
if (connection != null) {
connection.close();
}
} finally {
if (session != null) {
session.close();
}
if (connection != null) {
connection.close();
}
}
Well, traditional request-reply has by design just 1 reply message. The thread waiting for the response stops listening as soon as the first reply arrives.
With JMS correlation IDs (no dedicated thread per request) it would theoretically be possible to receive multiple replies for the same request, but I don't know if this really works/is allowed in JMS.
Update based on comments
You write in comments that you are able to receive multiple JMS replies for one request and that you even get the number of answers to expect.
If this all works, you can use the Aggregator EIP in your Camel route to collect all responses before sending a reply to the caller.
The Aggregator is highly configurable. You can decide how to combine the responses and you can also define multiple completion criteria (timeout, number of messages etc).
lets say, I have
from("direct:start").filter("...")
How can I filter out messages coming from an endpoint like smtp? Is there a way to check the type of the previous endpoint?
Edit: As per advice from vikingsteve, I have implemented the code this way:
from("direct:source")
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
exchange.setProperty("source", "smtps");
}
})
.to("direct:start");
from("direct:start").filter(
new Predicate() {
#Override
public boolean matches(Exchange exchange) {
if(exchange.getProperty("source") == "smtps")
return true;
else
return false;
}}));
You would normally set an exchange property if you need to identify some information about the original message (such as where it came from, or some other metadata not contained in the body).
from("smtp:...").setProperty("source", "smtp").to("...")
edit: here's a simpler version of your solution, I havent tested it, but it might work like this:
from("direct:source")
.setProperty("source", "smtps")
.to("direct:start");
from("direct:start")
.filter(exchangeProperty("source").isEqualTo("smtp"))
I am getting a strange situation at the code below which simply routes request to Google and returns response.
It works well but when I activate the line commented out as "//Activating this line causes empty response on browser" to print out returned response from http endpoint (Google), response is disappear, nothing is displayed on browser. I thought it might be related with input stream of http response which can be consumed only once and I activated Stream Caching on context but nothing changed.
Apache Camel version is 2.11.0
Any suggestions are greatly appreciated, thanks in advance.
public class GoogleCaller {
public static void main(String[] args) throws Exception {
CamelContext context = new DefaultCamelContext();
context.addRoutes(new RouteBuilder() {
public void configure() {
from("jetty:http://0.0.0.0:8081/myapp/")
.to("jetty://http://www.google.com?bridgeEndpoint=true&throwExceptionOnFailure=false")
.process(new Processor() {
public void process(Exchange exchange) throws Exception {
System.out.println("Response received from Google, is streamCaching = " + exchange.getContext().isStreamCaching());
System.out.println("----------------------------------------------IN MESSAGE--------------------------------------------------------------");
System.out.println(exchange.getIn().getBody(String.class));
System.out.println("----------------------------------------------OUT MESSAGE--------------------------------------------------------------");
//System.out.println(exchange.getOut().getBody(String.class)); //Activating this line causes empty response on browser
}
});
}
});
context.setTracing(true);
context.setStreamCaching(true);
context.start();
}
}
As you use a custom processor to process the message, you should keep it in mind the in message of the exchange has response message from the google, if you are using exchange.getOut(), camel will create a new empty out message for you and treat it as response message.
Because you don't set the out message body in the processor, it makes sense that you get the empty response in the browser.