Apache Camel - Delay when exception occurs - apache-camel

I have written a camel route that polls a file folder, picks up request, checks for memory consumption on server (in a java file). If its below threshold it drops the request on a JMS queue otherwise it throws an exception and picks it again for processing.
What i need to do is that when exception is thrown i need to delay processing for a configurable amount of time say 15 mins. This will give some time for server to stabilize instead of keeping it polling unnecessarily.
I am using errorHandler mechanism of camel however it doesnt seem to work. Camle keeps on picking up the request without any delay. Please help with this issue.
Below is the bundle context snapshot:
<camel:onException>
<camel:exception>java.lang.Exception</camel:exception>
<camel:redeliveryPolicy backOffMultiplier="500" />
<camel:log message="Default error handler was called"></camel:log>
</camel:onException>
<camel:route>
<!-- Reading from REST url -->
<camel:from uri="<my url>" />
<!-- If else loop -->
<camel:choice>
<camel:when>
<camel:xpath>Some path</camel:xpath>
<!-- Parsing OrderNumber and OrderVersion-->
<camel:log message="Recieved request ${headers.OrderNumber}-${headers.OrderVersion}.xml"/>
<camel:setHeader headerName="OrderNumber">
<xpath>Some path</xpath>
</camel:setHeader>
<camel:setHeader headerName="OrderVersion">
<camel:xpath>Some path</camel:xpath>
</camel:setHeader>
<!-- Request being put in file folder -->
<to
uri="file:data/inbox?fileName=${header.OrderNumber}-${header.OrderVersion}.xml"
pattern="InOut" />
</camel:when>
<!-- For all other requests put on queue -->
<camel:otherwise>
<camel:log message="Request ${headers.OrderNumber}-${headers.OrderVersion}.xml directly sent to queue"/>
<to uri="my queue"
pattern="InOut" />
</camel:otherwise>
</camel:choice>
</camel:route>
<camel:route errorHandlerRef="errorHandler">
<!-- Route to put message from folder to JMS queue if memory consumption is below limit-->
<camel:from uri="file:data/inbox"/>
<camel:process ref="checkMemoryConsumption"/>
<camel:convertBodyTo type="String" />
<camel:log message="Sucessfully processing service order ${headers.OrderNumber}-${headers.OrderVersion}.xml"/>
<to uri="my queue"
pattern="InOut" />
</camel:route>

Did you try redeliveryDelay of redeliveryPolicyProfile?
Below policy profile will retry 3 times with the delay of 1000mS between each try.
<redeliveryPolicyProfile id="myRedeliveryProfile"
maximumRedeliveries="3"
redeliveryDelay="1000" allowRedeliveryWhileStopping="false"
retryAttemptedLogLevel="INFO" />
Read more on here

By making below modification in my bundle context (bundle context displayed above), re-delivery policy kicked in and seems to do the trick.

Related

Skip Camel Processor if Previous Processor Failed

I have a camel route that goes through a series of processors which each call some service. If processor 1 fails gracefully, I no longer want to call processors 2-5, but I do want to call consolidateResponse. The same for all the other processors. Is there a best practices way of achieving this without throwing an exception on failures?
<camel:routeContext id="myRouteRouteContext">
<camel:route id="my-route-route">
<camel:from uri="{{camel.uri.myRoute}}" />
<camel:process ref="{{bean.processor.processor1}}" />
<camel:process ref="{{bean.processor.processor21}}" />
<camel:process ref="{{bean.processor.processor3}}" />
<camel:process ref="{{bean.processor.processor4}}" />
<camel:process ref="{{bean.processor.processor5}}" />
<!-- Stringfy response object into a JSon text response -->
<camel:process ref="{{bean.processor.consolidateResponse}}" />
<!-- All catch exception handler -->
<camel:onException>
<camel:exception>java.lang.Exception</camel:exception>
<camel:handled>
<camel:constant>true</camel:constant>
</camel:handled>
<camel:to uri="{{camel.uri.error}}" />
</camel:onException>
</camel:route>
</camel:routeContext>
I think try catch is best solution also it is common programming solution also camel also has stop
https://camel.apache.org/manual/try-catch-finally.html
from("direct:start")
.choice()
.when(body().contains("Hello")).to("mock:hello")
.when(body().contains("Bye")).to("mock:bye").stop()
.otherwise().to("mock:other")
.end()
.to("mock:result");

How to read multiple files using camel pollEnrich?

I need to fetch multiple files from a directory using camel pollEnrich. When I searched for this implementation, I found out that camel pollEnrich will pick up only one file at a time. But it was given that we can fetch multiple files by repeating pollEnrich by loop. How can this be correctly implemented?
Sample code for current implementation below:
<camel:route id="fileRoute">
<camel:from uri="jetty:{{file.api.endpoint}}" />
<camel:convertBodyTo type="java.lang.String"/>
<camel:unmarshal ref="json" />
<camel:choice>
<camel:when>
<camel:simple>Condition check for validating the request</camel:simple>
<camel:pollEnrich timeout="5000">
<camel:simple>file:{{file.input.dir}}?preMove={{file.inprogress.dir}}&move={{file.processed.dir}}&filter=#fileFilter</camel:simple>
</camel:pollEnrich>
<!-- Logic -->
</camel:when>
<camel:otherwise>
<!-- Invalid request -->
</camel:otherwise>
</camel:choice>
</camel:route>
I also tried loop doWhile aproach, but it didn't work out for me. My requirement is to get all the files from a directory using camel pollEnrich.
Sample code for loop doWhile approach:
<camel:route id="fileRoute">
<camel:from uri="jetty:{{file.api.endpoint}}" />
<camel:convertBodyTo type="java.lang.String"/>
<camel:unmarshal ref="json" />
<camel:loop doWhile="true">
<camel:simple>Condition check for validating the request</camel:simple>
<camel:pollEnrich timeout="5000">
<camel:simple>file:{{file.input.dir}}?preMove={{file.inprogress.dir}}&move={{file.processed.dir}}&filter=#fileFilter</camel:simple>
</camel:pollEnrich>
<!-- Logic -->
</camel:loop>
</camel:route>

camel: use header value in multiple routes

Is it possible to set a header in one route and then use it again later in another route in the same context?
For example, I have a route that sets a header as so:
<setHeader headerName="clientId">
<xpath>/Alarm/clientid/text()</xpath>
</setHeader>
The route uses Dead Letter Channel to send the message to an error route in the same context when delivery fails and I would like to use the clientId header I defined above in the message the error route sends, but currently calling the header returns nothing (an empty string I assume).
It should work , I am using camel 2.15.1 ,check the code below , if you run it you should see the header
<bean id="mybean" class="java.lang.Exception" />
<camel:camelContext xmlns="http://camel.apache.org/schema/spring"
trace="false">
<camel:errorHandler id="deadLetterErrorHandler"
type="DeadLetterChannel" deadLetterUri="direct:b">
</camel:errorHandler>
<camel:route>
<camel:from uri="timer:foo?repeatCount=1" />
<camel:setHeader headerName="myheader">
<camel:simple>Sundar</camel:simple>
</camel:setHeader>
<camel:to uri="direct:a" />
</camel:route>
<camel:route errorHandlerRef="deadLetterErrorHandler">
<camel:from uri="direct:a" />
<camel:throwException ref="mybean"></camel:throwException>
</camel:route>
<camel:route>
<camel:from uri="direct:b" />
<camel:log message="${in.header.myheader}"/>
</camel:route>
</camel:camelContext>
It should print a log like below
24 Feb 2016 19:09:47,707 route3 INFO Sundar

maximumPoolSize in Camel-Netty

I have been playing around with camel-netty and camel-netty-http components, and trying to figure out what the setting maximumPoolSize does.
However, from what i observe based on this is that, the OrderPool always processes 16 concurrent requests. I am trying to change the maximumPoolSize to a value of 5 like the route as below,
<bean id="nettyBean" class="com.redhat.NettyTestImpl">
<property name="message" value="Netty maximumPoolSize test"/>
</bean>
<camelContext trace="false" xmlns="http://camel.apache.org/schema/blueprint" autoStartup="true">
<route>
<from uri="netty-http:http://localhost:8080/hello?maximumPoolSize=5&sync=true"/>
<log message="Forwarding to Netty component ....."/>
<setBody>
<method ref="nettyBean" method="sayHi"/>
</setBody>
<delay>
<constant>3000</constant>
</delay>
<log message="The body contains : ${body}"/>
</route>
</camelContext>
But it seems that i cannot get the maximumPoolSize to set to a value. So, what am i doing wrong ? How can i get the maximumPoolSize set ?
I check this by load testing with 20 concurrent requests and all are processed.
I just checked the code of camel-netty and camel-netty-http, maximumPoolSize is used for the OrderPool, you can use jconsoler to check the thread pool size.
As the OrderPool is used to handle the processor of camel, netty still has the work thread to handle the connection, which means it can still server 20 concurrent request, this time. But the TPS could be just about 5.

Camel Split and Aggregate failing because messages going to multiple concurrent consumers

I have a simple camel route that takes a list of items, splits them sending each element to a mq node for processing then joins them back together via an aggregator.
Very close to the Composed Message Processor: http://camel.apache.org/composed-message-processor.html
But we noticed that after the split, camel will create multiple concurrent consumers? or exchanges? Since the message is being sent to multiple consumers they never complete.
List: 1,2,3,4
Split: amq::process_each_item
Aggregate:
[Camel (camel-3) thread #41 - Aggregating 1 - Waiting on 3 more items
[Camel (camel-1) thread #16 - Aggregating 2 - Waiting on 3 more items
[Camel (camel-3) thread #49 - Aggregating 3 - Waiting on 2 more items
[Camel (camel-1) thread #15 - Aggregating 4 - Waiting on 2 more items
So, camel spawned 2 aggregators, and each is waiting on 4 items, but they only ever get two each.
Camel Route:
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route> <!-- This route splits the reg request into it's items. Adding needed info to the message header. -->
<from uri="activemq:registration.splitByItemQueue" /> <!-- pick up the reg req -->
<setHeader headerName="regReqId"> <!-- Need to store the Reg Req in the header -->
<simple>${body.registrationRequest.id}</simple>
</setHeader>
<split parallelProcessing="false" strategyRef="groupedExchangeAggregator"> <!-- Split the RegRequestInfo into it's individual requestItems (add, drop, etc) -->
<method ref="requestSplitter" method="split" /> <!-- does the actual splitting -->
<setHeader headerName="JMSXGroupID"> <!-- This is CRITICAL. It is how we ensure valid seat check counts without db locking -->
<simple>FOID=${body.formatOfferingId}</simple> <!-- grouping on the foid -->
</setHeader>
<to uri="activemq:registration.lprActionQueue"/> <!-- send to queue's for processing-->
</split>
</route>
<route> <!-- performs the registration + seat check -->
<from uri="activemq:registration.lprActionQueue" />
<bean ref="actionProcessor" method="process"/> <!-- go to the java code that makes all the decisions -->
<to uri="activemq:registration.regReqItemJoinQueue"/> <!-- send to join queue's for final processing-->
</route>
<route> <!-- This route joins items from the reg req item split. Once all items have completed, update state-->
<from uri="activemq:registration.regReqItemJoinQueue" /> <!-- Every Reg Req Item will come here-->
<aggregate strategyRef="groupedExchangeAggregator" ignoreInvalidCorrelationKeys="false" completionFromBatchConsumer="true"> <!-- take all the Reg Req Items an join them to their req -->
<correlationExpression>
<header>regReqId</header> <!-- correlate on the regReqId we stored in the header -->
</correlationExpression>
<bean ref="actionProcessor" method="updateRegistrationRequestStatus"/> <!-- update status -->
</aggregate>
</route>
</camelContext>
<bean id="groupedExchangeAggregator" class="org.apache.camel.processor.aggregate.GroupedExchangeAggregationStrategy" />
On my local machine the above works fine, but when we deploy to our test server half of the messages go to one camel aggregator, half to the other. causing none to ever finish. Notice in the config below that we've set concurrent consumers to 1 for camel.
Here's the camel / activemq config
<amq:broker useJmx="false" persistent="false">
<amq:plugins>
<amq:statisticsBrokerPlugin />
</amq:plugins>
<amq:transportConnectors>
<amq:transportConnector uri="tcp://localhost:0" />
</amq:transportConnectors>
</amq:broker>
<!-- Basic AMQ connection factory -->
<amq:connectionFactory id="amqConnectionFactory" brokerURL="vm://localhost" />
<!-- Wraps the AMQ connection factory in Spring's caching (ie: pooled) factory
From the AMQ "Spring Support"-page: "You can use the PooledConnectionFactory for efficient pooling... or you
can use the Spring JMS CachingConnectionFactory to achieve the same effect."
See "Consuming JMS from inside Spring" at http://activemq.apache.org/spring-support.html
Also see http://codedependents.com/2010/07/14/connectionfactories-and-caching-with-spring-and-activemq/
Note: there are pros/cons to using Spring's caching factory vs Apache's PooledConnectionFactory; but, until
we have more explicit reasons to favor one over the other, Spring's is less tightly-coupled to a specific
AMQP-implementation.
See http://stackoverflow.com/a/19594974
-->
<bean id="connectionFactory" class="org.springframework.jms.connection.CachingConnectionFactory">
<constructor-arg ref="amqConnectionFactory"/>
<property name="sessionCacheSize" value="1"/>
</bean>
<bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate">
<constructor-arg ref="connectionFactory" />
</bean>
<bean id="jmsConfig"
class="org.apache.camel.component.jms.JmsConfiguration">
<property name="connectionFactory" ref="connectionFactory"/>
<property name="concurrentConsumers" value="1"/>
</bean>
<bean id="activemq"
class="org.apache.activemq.camel.component.ActiveMQComponent">
<property name="configuration" ref="jmsConfig"/>
</bean>
Turns out we had another spring context / servlet importing our config. We believe this was the issue.

Resources