I would like to run a route once and stop the context when the route is completed. Currently I do the usual Thread.sleep(3000) in the main Java class to leave some time for the route to finish but it's obviously not accurate, my route may take 1 second or 20 seconds I can not know in advance.
The Java class:
public static void main(String[] args) throws Exception {
try (ClassPathXmlApplicationContext context = new ClassPathXmlApplicationContext("camel-context.xml")) {
CamelContext camelContext = SpringCamelContext.springCamelContext(context);
// context.start(); // apparently not necessary
camelContext.startRoute("route1");
try { Thread.sleep(3000); } catch (InterruptedException e) { e.printStackTrace(); }
// context.stop(); // apparently not necessary
}
}
The Spring xml:
<route id="route1" autoStartup="false">
<from uri="timer://runOnce?repeatCount=1&delay=3000" />
...
</route>
After reading http://camel.465427.n5.nabble.com/Need-control-back-in-the-Main-routine-so-that-we-can-terminate-JVM-td4483312.html#a4484845 especially the 4th post, from Claus Ibsen, I was thinking of using camelContext.getRouteStatus() in a loop with a Thread.sleep() but wherever I try to get the route status in the code (even after the Thread.sleep(3000)), the status is always "started". I don't know any other way to detect when the route is done.
What is the recommended way to stop the Camel context when a/all route(s) is/are completed, using Spring?
The route will never stop because routes do not have complete state. They can just be started, stopped or paused. A route will always be running if it's in the started state unless you do something to change that.
To accomplish what you are looking for, you can do a couple of things:
You can use the controlbus component and stop the route in the last step of your route. That way you can check (for example the way you mentioned checking for camelContext.getRouteStatus()) when you should stop the context as well.
You can write a small Processor that whenever it receives an Exchange it will stop the camelContext. Once ready, you will add it to the last step of your route.
Camel supports onCompletion callbacks, which can be equivalent to the option above. See the camel page.
Probably, the first option is the easiest for your use case, however I would go for the second option. It seems cleaner to me.
A more elegant way would be to use a synchronisation mechanism provided by Java like CountDownLatch. Main thread will wait for the latch to be opened by the Route thread. Something like :
CountDownLatch latch = new CountDownLatch(1);
camelContext.addRoutes(createRoute(latch));
and somewhere in the createRoute Method add a processor at the end of the route to open the latch. This worked perfectly for me.
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
latch.countDown();
}
});
Related
I am migration to Camel 3.20.1 from 2.20.1 and am facing some difficulties with my current workflow.
I have three RouteBuilders with routes which are called in hierarchical order like this.
RouteBuilderA
#Override
public void configure() throws Exception {
errorHandler(deadLetterChannel("direct:dead")
.maximumRedeliveries(2)
.redeliveryDelay(1000));
from("direct:A")
.to("direct:B");
}
RouteBuilderB
#Override
public void configure() throws Exception {
from("direct:B")
.to("direct:C");
}
RouteBuilderC
#Override
public void configure() throws Exception {
errorHandler(noErrorHandler())
from("direct:C")
.throwException(new RuntimeException("Something went wrong"))
.to("direct:end");
}
Current flow:
As you can see in the code, route in RouteBuilder3 will trigger exception which will be passed to RouteBuilder2. From there it will not be handled, as no error handler is defined for that route, and it will fail with stacktrace...
Desired flow:
What I need is to pass that exception to RouteBuilder1 and handle it there where (my global deadLetterChannel) error handler is defined.
One could say, just add errorHandler(noErrorHandler()) into RouteBuilderB, right. That is clear and it works, BUT I don't want that because...
Because, in that case redelivery will kick in and route message through routes in RouteBuilderB and RouteBuilderC. But, as I got exception in RouteBuilderC, I only want to route message through route where it failed, that is RouteBuilderC.
Background:
In old Camel version I somehow managed to make it work by using includeRoutes(injectedRouteBuilder), which is not available anymore. RouteBuilderC bean is injected into RouteBuilderB, which is injected in RouteBuilderA.
Question:
Does anybody knows how to solve that? I cannot find a way to make this work. I tried a lot of things, including getCamelContext().addRoutes(injectedRouteBuilder);.
*p.s. I am aware of recommendation to use inheritance for RouteBuilders, but in my case this is not an option. I need to pass it to RouteBuilder1 as only that one knows to which error queue, message needs to be send. *
I would like to route messages from more routes to the same route but it does not work in the manner as I assumed. I set up the following (I am just puting down the essence):
from("direct:a") [...]
.to("direct:c");
from("direct:b") [...]
.to("direct:c");
from(direct:c) <my aggregator functionality comes here>
.to("direct:someOtherRoute");
However, this works only when exactly one route either "a" or "b" goes to "c" but not both. How should I achieve to route both "a" and "b" to "c"? Thanks.
EDIT1:
I tried the solution of Alexey but using "seda" or "vm" did not solve the problem. Actually, regardless of calling route "c" with seda or vm, the aggregator is invoked only once either from route "a" or from route "b".
However, if I create another route "c2" with the same content and route e.g. "b" to "c2", then it works. Nevertheless, it is not really nice way to solve it.
Do you have any further ideas? I am using the routes within the same CamelContext, so within the same JVM.
I have also found an interesting remark on the link http://camel.apache.org/seda.html
It states as Alexey and Sunar also told that seda and vm are asynchronous and direct synchronous but you can also implement asynchronous functionality with direct as follows:
from("direct:stageName").thread(5).process(...)
"[...] Instead, you might wish to configure a Direct endpoint with a thread pool, which can process messages both synchronously and asynchronously. [...]
I also tested it but in my case it did not yield any fruits.
EDIT2:
I add here how I am using the aggregator, i.e. route "c" in this example:
from("vm:AGGREGATOR").routeId("AGGREGATOR")
.aggregate( constant("AGG"), new RecordAggregator())
.completionTimeout(AGGREGATOR_TIMEOUT)
.process(new Processor() {
public void process(Exchange exchange) throws Exception {
LOGGER.info("### Process AGGREGATOR");
[...]
}
})
.marshal().csv()//.tracing()
.to("file:extract?fileName=${in.header.AGG}.csv")
.end();
In the log the String "### Process Aggregator" appears only once. I am just wondering whether it cannot depend on the .completionTimeout(AGGREGATOR_TIMEOUT) I am using. In my undestanding, a file should be created for each different AGG value in the header within this time. Is this understanding correct?
I think the using of asynchronous components, such as seda, vm, activemq might solve your problem.
Such behavior direct component because direct is synchronous component, this is also likely related to using the aggregator in the third route.
Example:
from("direct:a") [...]
.to("seda:c");
from("direct:b") [...]
.to("seda:c");
from(seda:c) <your aggregator functionality comes here>
.to("direct:someOtherRoute");
EDIT1:
Now, when I see an aggregator, I think that's the problem, in the completion criteria.
In your case, you have to use expression for correlationExpression:
from("vm:AGGREGATOR").routeId("AGGREGATOR")
.aggregate().simple("${header.AGG}",String.class) // ${property.AGG}
.aggregationStrategy(new RecordAggregator())
.completionInterval(AGGREGATOR_TIMEOUT) //.completionTimeout(AGGREGATOR_TIMEOUT)
.forceCompletionOnStop()
.process(new Processor() {
public void process(Exchange exchange) throws Exception {
LOGGER.info("### Process AGGREGATOR");
[...]
}
})
.marshal().csv()//.tracing()
.to("file:extract?fileName=${in.header.AGG}.csv&fileExist=Override")
.end();
and, maybe completionTimeout is too low...
Try the below , this is only a sample
from("timer:foo?repeatCount=1&delay=1000").routeId("firstroute")
.setBody(simple("sundar")).to("direct:a");
from("timer:foo1?repeatCount=1&delay=1000").routeId("secondRoute")
.setBody(simple("sundar1")).to("direct:a");
from("direct:a")
.aggregate(new AggregationStrategy() {
#Override
public Exchange aggregate(Exchange arg0, Exchange arg1) {
Exchange argReturn = null;
if (arg0 == null) {
argReturn= arg1;
}
if (arg1 == null) {
argReturn= arg0;
}
if (arg1 != null && arg0 != null) {
try {
String arg1Str = arg1.getIn()
.getMandatoryBody().toString();
String arg2Str = arg0.getIn()
.getMandatoryBody().toString();
arg1.getIn().setBody(arg1Str + arg2Str);
} catch (Exception e) {
e.printStackTrace();
}
argReturn= arg1;
}
return argReturn;
}
}).constant(true).completionSize(2)
.to("direct:b").end();
from("direct:b").to("log:sundarLog?showAll=true&multiline=true");
You can use seda or other asynchronous routes as pointed out by Yakunin.Using an aggregator here the major contention point would be the completionSize , where i have used 2 , since two routes are sending in the messages.
I am using Camel for my messaging application. In my use case I have a producer (which is RabbitMQ here), and the Consumer is a bean.
from("rabbitmq://127.0.0.1:5672/exDemo?queue=testQueue&username=guest&password=guest&autoAck=false&durable=true&exchangeType=direct&autoDelete=false")
.throttle(100).timePeriodMillis(10000)
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
MyCustomConsumer.consume(exchange.getIn().getBody())
}
});
Apparently, when autoAck is false, acknowledgement is sent when the process() execution is finished (please correct me if I am wrong here)
Now I don't want to acknowledge when the process() execution is finished, I want to do it at a later stage. I have a BlockingQueue in my MyCustomConsumer where consume() is putting messages, and MyCustomConsumer has different mechanism to process them. I want to acknowledge message only when MyCustomConsumer finishes processing messages from BlockingQueue. How can I achieve this?
You can consider to use the camel AsyncProcessor API to call the callback done once you processing the message from BlockingQueue.
I bumped into the same issue.
The Camel RabbitMQConsumer.RabbitConsumer implementation does
consumer.getProcessor().process(exchange);
long deliveryTag = envelope.getDeliveryTag();
if (!consumer.endpoint.isAutoAck()) {
log.trace("Acknowledging receipt [delivery_tag={}]", deliveryTag);
channel.basicAck(deliveryTag, false);
}
So it's just expecting a synchronous processor.
If you bind this to a seda route for instance, the process method returns immediately and you're pretty much back to the autoAck situation.
My understanding is that we need to make our own RabbitMQ component to do something like
consumer.getAsyncProcessor().process(exchange, new AsynCallback() {
public void done(doneSync) {
if (!consumer.endpoint.isAutoAck()) {
long deliveryTag = envelope.getDeliveryTag();
log.trace("Acknowledging receipt [delivery_tag={}]", deliveryTag);
channel.basicAck(deliveryTag, false);
}
}
});
Even then, the semantics of the "doneSync" parameter is not clear to me. I think it's merely a marker to identify whether we're dealing with a real async processor or a synchronous processor that was automatically wrapped into an async one.
Maybe someone can validate or invalidate this solution?
Is there a lighter/faster/stronger alternative?
Or could this be suggested as the default implementation for the RabbitMQConsumer?
Hope this doesn't sound ridiculous, but how can I discard a message in Camel on purpose?
Until now, I sent them to the Log-Component, but meanwhile I don't even want to log the withdrawal.
Is there a /dev/null Endpoint in Camel?
You can use the message filter eip to filter out unwanted messages.
http://camel.apache.org/message-filter
There is no dev/null, component.
Also there is a < stop /> you can use in the route, and when a message hit that, it will stop continue routing.
And the closest we got on a dev/null, is to route to a log, where you set logLeve=OFF as option.
With credit to my colleague (code name: cayha)...
You can use the Stub Component as a camel endpoint that is equivalent to /dev/null.
e.g.
activemq:route?abc=xyz
becomes
stub:activemq:route?abc=xyz
Although I am not aware of the inner workings of this component (and if there are dangers for memory leaks, etc), it works for me and I can see no drawbacks in doing it this way.
one can put uri/mock-uri to the config using property component
<camelContext ...>
<propertyPlaceholder id="properties" location="ref:myProperties"/>
</camelContext>
// properties
cool.end=mock:result
# cool.end=result
// route
from("direct:start").to("properties:{{cool.end}}");
I'm a little late to the party but you can set a flag on the exchange and use that flag to skip only that message (by calling stop) if it doesn't meet your conditions.
#Override
public void configure() throws Exception {
from()
.process(new Processor() {
#SuppressWarnings("unchecked")
#Override
public void process(Exchange exchange) throws Exception {
exchange.setProperty("skip", false);
byte[] messageBytes = exchange.getIn().getBody(byte[].class);
if (<shouldNotSkip>) {
} else { //skip
exchange.setProperty("skip", true);
}
}
}).choice()
.when(exchangeProperty("skip").isEqualTo(true))
.stop()
.otherwise()
.to();
}
I am using activemq route and needs to send reply in normal cases, so exchange pattern is InOut. When I configure a filter in the route I find that even it does not pass message to next step, the callback is executed(sending reply), just same as the behavior when calling stop(). And it will send the same message back to reply queue, which is not desirable.
What I do is to change the exchange pattern to InOnly conditionally and stop if I want to filter out the message, so reply is not sent. MAIN_ENDPOINT is a direct:main endpoint I defined to include normal business logic.
from("activemq:queue:myqueue" + "?replyToSameDestinationAllowed=true")
.log(LoggingLevel.INFO, "Correlation id is: ${header.JMSCorrelationID}; will ignore if not null")
.choice()
.when(simple("${header.JMSCorrelationID} == null"))
.to(MAIN_ENDPOINT)
.endChoice()
.otherwise()
.setExchangePattern(ExchangePattern.InOnly)
.stop()
.endChoice()
.end();
Note that this message is also consumed and not in the queue anymore. If you want to preserve the message in the queue(not consuming it), you may just stop() or just filter() so the callback(sending reply which is the original message) works, putting the message back to the queue.
Using only filter() would be much simpler:
from("activemq:queue:myqueue" + "?replyToSameDestinationAllowed=true")
.log(LoggingLevel.INFO, "Correlation id is: ${header.JMSCorrelationID}; will ignore if not null")
.filter(simple("${header.JMSCorrelationID} == null"))
.to(MAIN_ENDPOINT);
I'm working on a camel prototype which uses two start points in the same camel context.
The first route consumes messages which are used to "configure" the application. Messages are loaded in a configuration repository through a configService bean:
// read configuration files
from("file:data/config?noop=true&include=.*.xml")
.startupOrder(1)
.to("bean:configService?method=loadConfiguration")
.log("Configuration loaded");
The second route implements a recipient list eip pattern, delivering a different kind of input messages to a number of recipients, which are read dinamically from the same configuration repository:
// process some source files (using configuration)
from("file:data/source?noop=true")
.startupOrder(2)
.unmarshal()
.to("setupProcessor") // set "recipients" header
.recipientList(header("recipients"))
// ...
The question that arises now is how to synchronize them, so the second route "waits" if the first is processing new data.
I'm new to Apache Camel and pretty lost on how to approach such a problem, any suggestion would be appreciated.
Use aggregate in combination with the possibility to start and stop routes dynamically:
from("file:data/config?noop=true&include=.*.xml")
.id("route-config")
.aggregate(constant(true), new MyAggregationStrategy()).completionSize(2).completionTimeout(2000)
.process(new Processor() {
#Override
public void process(final Exchange exchange) throws Exception {
exchange.getContext().startRoute("route-source");
}
});
from("file:data/source?noop=true&idempotent=false")
.id("route-source") // the id is needed so that the route is found by the start and stop processors
.autoStartup(false) // this route is only started at runtime
.aggregate(constant(true), new MyAggregationStrategy()).completionSize(2).completionTimeout(2000)
.setHeader("recipients", constant("direct:end")) // this would be done in a separate processor
.recipientList(header("recipients"))
.to("seda:shutdown"); // shutdown asynchronously or the route would be waiting for pending exchanges
from("seda:shutdown")
.process(new Processor() {
#Override
public void process(final Exchange exchange) throws Exception {
exchange.getContext().stopRoute("route-source");
}
});
from("direct:end")
.log("End");
That way, route-source is only started when route-config is completed. route-config and consequently route-source are restarted if new files are found in the config directory.
You can also place an "on completion" http://camel.apache.org/oncompletion.html in the first route that activates the second one.
Apache camel File will create a lock for the file that being processed. Any other File process on this file will not pool on if there is a lock (except if you put consumer.exclusiveReadLock=false)
source :
http://camel.apache.org/file.html => URI Options => consumer.exclusiveReadLock