I have 4 routes in my application. I have used below code to close route in same order they are started:
getContext().getShutdownStrategy().setShutdownRoutesInReverseOrder(false);
From the logs I can see that routes are stopping in same order. but the problem is I can see in logs multiple "DirectConsumerNotAvailableException" because first route is sending exchanges to second route but second route is not available by that time. Why is it happening?
Related
I have application which allows dynamic creation and starting of routes. For example user can create route from IMAP to file and start it.
Problem: Multiple routes reading from same mail account through IMAP.
Caused by: org.apache.camel.FailedToStartRouteException: Failed to start route c1152_route because of Multiple consumers for the same endpoint is not allowed: imap://localhost:3143?delay=1000&password=xxxxxx&searchTerm.fromSentDate=now-24h&searchTerm.unseen=false&username=user#user
I've tried two approaches:
1.Creating additional single route from IMAP to multiple recipient routes. This one works but I'm not sure about this solution. It requires additional checks on starting/stopping of route because few routes might be dependent on this one.
from(imap())
.recipientList(imapMsgToDirectChannelRouter())
2.Creating IMAP endpoints with unique uri. For example unique searchTerm.fromSentDate for each routes. This one also works.
Is there any better solution to this problem?
I would definitely go for option#1 (recipientList).
I do not understand why would have to stop & start the 'master' route (the "imap" one).
Depending on the recipients you add or remove, you would start and stop the corresponding child routes, but not touch the master one.
I'm thinking on something like this:
from("imap:...")
.id("master-route")
.recipientList( simple("bean:computeRecipients") )
.parallelProcessing();
Where the on-the-fly calculation of the exact list of recipients is delegated to some bean.
This way, you do not have to touch the "master-route", instead just make the bean aware of the recipients
I'm looking for a best practise how to monitor the functionality of camel routes.
I know there are monitoring tools like hawtio and camelwatch, but that's not exactly what I'm looking for.
I want to know if a route is "working" as aspected, for example you have a route which listens on a queue(from("jms...")). Maybe there are messages in the queue, but the listener is not able to dequeue them because of some db issues or something else(depends on the jms provider). With the monitoring tools mentioned above you just see inflight/failed/completed messages but you don't see if the listener is able to get the messages -> so the route is not "working".
I know there is also apache BAM, maybe I have to do some more research, but somehow it looks like BAM creates new routes and you can't monitor existing routes. I also don't want to implement/define such business cases for each route, I look for a more generic way. It's also mentioned on the camel 3.0 idea board that BAM wasn't touched for 5 years, so I think people don't use it that often(which means for me it doesn't fit their needs exactly).
I had similar requirement some time ago and at the end I developed a small Camel application for monitoring.
It run on timer, query different Camel applications installed in remote servers through JMX/Jolokia and if LastExchangeCompletedTimestamp of the route I am interested in is older than some time interval, send a mail to administrators.
Maybe this approach is too simple for your scenario, but could be an option.
(Edit: more details added)
Principal points:
Main routes queries DB for entities to control and spawns controlling routes
Controlling routes fires on quartz and http post the following url
.to("http://server:port/app/jolokia/?"+
"maxDepth=7&maxCollectionSize=500&ignoreErrors=true&canonicalNaming=false")
sending the following jsonRequest body
LinkedHashMap<String,Object> request=new LinkedHashMap<String,Object>();
request.put("type","read");
request.put("mbean","org.apache.camel:"+entity.getRouteId());
jsonRequest=mapper.writeValueAsString(request);
As response you get another JSON, parse it and get LastExchangeCompletedTimestamp value
We are exposing a SOAP/HTTP based camel-cxf web service where on receiving a request from 'Client-A', the route execution starts which involves calling one or more external web services lets say 'Server1', 'Server2' and 'Server3' in a sequentialmanner. In this case, we need to understand what happens to the execution of the route, when the original TCP connection with 'Client-A' is closed unexpectedly.
Will the route get executed successfully and an error is logged when it tries to send the final response?
Or will the route execution be stopped immediately as soon as the TCP connection is closed?
You can capture any errors during the route execution by camel's error handling mechanism and then define the policy on how you want to handle the exception, for example you can setup rules that state how many times to try redelivery, and the delay in between attempts, and so forth.
For more information you can refer to this link:
http://camel.apache.org/redeliverypolicy.html
We're having some came routes defined in a single CamelContext which contains Web services,activemq.. in the Route.
Initially we've deployed the Routes as WAR in single Jboss node.
To scale out(usually we're doing for web services) , I've deployed the same CamelContext in multiple Jboss nodes.
But the performance is actually decreased.
FYI: All the CamelContexts points to the Same activemq brokers.
Here are my questions:
How to load balance/ Fail over camel context in different machines?
If CamelContexts are deployed in multiple nodes, Will aggregation work correctly?
Kindly give your thoughts!
Without seeing your system in detail, there is no way of knowing why it has slowed down so I'll pass over that. For your other two questions:
Failover
You don't say what sort of failover/load balancing behaviour you want. The not-very-helpful Camel documentation is here: http://camel.apache.org/clustering-and-loadbalancing.html.
One mechanism that works easily with Camel and ActiveMQ is to deploy to multiple servers and run active-active, sharing the same ActiveMQ queues. Each route attempts to read from the same queue to get a message to process. Only one route will get the message and therefore only one route processes it. Other routes are free to read subsequent messages, giving you simple load balancing. If one route crashes, the other routes will continue to process the messages, there will just be reduced capacity on your system.
If you need to provide fault tolerance for your web services then you need to look outside Camel and use something like Elastic Load Balancing. http://aws.amazon.com/elasticloadbalancing/
Aggregation
Each Camel context will run independently of the other contexts so one context will aggregate messages independently of what other contexts are up to. For example, suppose you have an aggregator that stores messages from ActiveMQ queue until receives a special end-of-batch message. If you have the aggregator running in two different routes, the messages will be split between the two routes and only one route will receive the end-of-batch message. So one aggregator will sit there with half the messages and do nothing. The other aggregator will have the other messages and will process the end-of-batch message but won't know about the messages the other route picked up.
I am using Apache camel to implement dispatcher EIP. There are thousands of messages in a queue which needs to be delivered at different URLs. Each message has its own delivery URL and delivery protocol (ftp,email,http etc).
The way it is been implemented:
Boot a single camel context, the context is disabled for JMX and the
loadStatisticsEnabled is set to false on the ManagementStrategy. As
mentioned in a jira issue, addressed in 2.11.0 version, for disabling
the background management thread creation.
For each message a route is being constructed , the message is being
pushed to the route for delivery.
After the message is processed route is shutdown and removed from
context.
Did a small perf test by having 200 threads of dispatcher component, each sharing the same context.
Observed that the time to start a route increases upto a maximum of 60 seconds while the time to process is in milliseconds.
Issue CAMEL-5675 mentions that this has been fixed but still observing significant time being taken in starting up routes.
https://issues.apache.org/jira/browse/CAMEL-5675
The route that is being creating for http is
from("direct:"+dispatchItem.getID())
.toF("%s?httpClient.soTimeout=%s&disableStreamCache=true", dispatchItem.getEndPointURL(),timeOutInMillis);
Each dispatchItem has a unique ID.
This is being active discussed elsewhere, where the user posted this question first: http://camel.465427.n5.nabble.com/Slow-startup-of-routes-tp5732356.html