We are building a system where an external system sends us "Refresh" events. These events tell us to refresh the state of an item identified by the id provided in the RefreshEvent.
We handle these events through a Camel route.
The refresh may go wrong, though. If this is case the event is moved to a "Retry" SEDA queue with appropiate retry/backup setup. However, we may experience multiple "Refresh" event for the same item, while we are unable to fetch the new state.
Ideally we'd like to only have unique itemIds in the Retry queue. The inflight itemIds should be unique.
Is there a way to do this in Camel.
I have looked at the IdempotentConsumer, but that has a scope of all received exchanges, I'm only interested in the current inflight exchanges.
If you are not able to use the IndempotentConsumer pattern, you will have to implement the logic yourself and possible change the Seda endpoint to something more sophisticated.
You could use a message queue supporting JMS selectors or property filtering.
ActiveMQ, to name one, has some capabilities that could be used in implementing the solution you need.
Related
I am new to Messaging and Integration. As I am trying to understand how Apache Camel makes EIP easy, Message Router caught my attention to a specific scenario where it consumes a message from one queue and transfers to some other queue based on a predicate.
Here are my questions:
Why use Message Router to perform this task?
Why not the message Producer directly send the message to the right Queue/Destination?
Why not the Consumer can rely on Message Selectors to do the same?
Can someone share your thoughts on real life use cases?
Well, your questions are more about SoC than messaging.
The Message Router is an EIP that can perform this task well and with Camel it is also very easy to implement, but as you write, there are other possibilities too.
If the message producer (client) sends messages directly to final destinations, it must know about all possible destinations. It must also be changed whenever a destination is added/removed or changed. This is tight coupling and in most situations not desirable.
If the message receiver(s) consume(s) with message selectors, the client can send all messages to the same queue. This is totally OK, you can implement individual consumers with different selectors. The selector feature of the Broker basically enables you to "divide a queue into multiple queues".
So if you want to implement and run an intermediate integration component between the client and the receivers, you can use the Message Router in this component.
If you want to omit this integration component, you can use message selectors to implement the receivers individual and de-coupled from each other.
I am using Apache camel to trigger various actions based on timer events. For timer events to trigger, I am using camel-quartz. And a sample trigger looks like this:
from("quartz://job_timers/customer_activation?cron=0+0+0+*+*+?+*&trigger.timeZone=America/Chicago")
.routeId("TrigCustActivation")
.log(LoggingLevel.INFO, "Triggered Customer Activation Job")
.to("direct://set/headers/activation")
.to("direct://customer/activation");
Over the period of time, the number of such triggers have increased a lot. And they will be increasing even more. So, instead of keeping a hard-copy of when which job will trigger, I was planning to create a web-route, so that when I perform a GET request, it would fetch all the active routes, and print their RouteIds (I will update all the RouteIds to have time as well. Say, the routeId for above flow will become "TrigCustActivation_00_00_CST_Daily").
I was able to print all the route at application startup, but failed to fetch those dynamically through a GET request.
Can we access other RouteIds from a running route? Is my approach feasible?
Yes there is API on CamelContext where you can fetch all the current routes, and you can then get their IDs etc. See the javadoc of this class.
http://static.javadoc.io/org.apache.camel/camel-core/2.21.0/org/apache/camel/CamelContext.html
We have a stand alone application where we should be able to stand up JMS listeners using database configuration dynamically. If there is any change in the concurrentConsumers count in the database, then we need to update the listeners accordingly. We are using camel 2.16.2 and Tibco 6.3.0. Precisely, the below are our requirements.
Ability to add and remove JMS listeners dynamically.
To achieve this, we are creating our OwnRouteBuilder (which extends camel RouteBuilder) and adding it using camelContext.addRoutes(RouteBuilder) dynamically which is working as expected.
Ability to increase or decrease JMS listeners dynamically.
To achieve this, initially we tried to stop the route, update concurrentConsumers count, and then start route. It is working fine only when we increase the concurrentConsumers count. But its not working if we decrease( its not changing the concurrentConsumers count).We can still see the same receivers count from tibco console.
As an alternate solution , we tried with camelContext.stopRoute(routeId) and then camelContext.removeRoute(routeId) so that we can add a fresh route with updated concurrentConsumers count later. In this case also, though it is stopping the routes from consuming messages, it is not changing the receivers count from tibco console.
I tried with camelContext.removeEndpoints(endpointUri), camelContext.shutdownRoute(routeId), camelContext.removeRouteDefinition(routedefinition) methods.
But all these methods only stopping the routes from consuming messages, but not clearing those receivers count from tibco console.
The receivers count from tibco console is becoming zero only when we terminate the application.
Is there any way that we can make it from camel so that those receivers count from tibco console becomes zero.
You can change the concurrentConsumer counts dynamically at runtime using JMX to set the values on the JMX attribute on the endpoint. Then the route dont have to be stopped/removed/started etc.
I'm looking for a best practise how to monitor the functionality of camel routes.
I know there are monitoring tools like hawtio and camelwatch, but that's not exactly what I'm looking for.
I want to know if a route is "working" as aspected, for example you have a route which listens on a queue(from("jms...")). Maybe there are messages in the queue, but the listener is not able to dequeue them because of some db issues or something else(depends on the jms provider). With the monitoring tools mentioned above you just see inflight/failed/completed messages but you don't see if the listener is able to get the messages -> so the route is not "working".
I know there is also apache BAM, maybe I have to do some more research, but somehow it looks like BAM creates new routes and you can't monitor existing routes. I also don't want to implement/define such business cases for each route, I look for a more generic way. It's also mentioned on the camel 3.0 idea board that BAM wasn't touched for 5 years, so I think people don't use it that often(which means for me it doesn't fit their needs exactly).
I had similar requirement some time ago and at the end I developed a small Camel application for monitoring.
It run on timer, query different Camel applications installed in remote servers through JMX/Jolokia and if LastExchangeCompletedTimestamp of the route I am interested in is older than some time interval, send a mail to administrators.
Maybe this approach is too simple for your scenario, but could be an option.
(Edit: more details added)
Principal points:
Main routes queries DB for entities to control and spawns controlling routes
Controlling routes fires on quartz and http post the following url
.to("http://server:port/app/jolokia/?"+
"maxDepth=7&maxCollectionSize=500&ignoreErrors=true&canonicalNaming=false")
sending the following jsonRequest body
LinkedHashMap<String,Object> request=new LinkedHashMap<String,Object>();
request.put("type","read");
request.put("mbean","org.apache.camel:"+entity.getRouteId());
jsonRequest=mapper.writeValueAsString(request);
As response you get another JSON, parse it and get LastExchangeCompletedTimestamp value
I have a Camel based application which receives a request and gives the reply from cache but in between this process it updates the database which i want it to run in a different thread , can anyone tell me how can i achieve this, i tried with WireTap and SEDA but it does not work that way...any help appreciated.
<camel:wireTap uri="seda:tap" processorRef="updateHitCountProcessor"/>
In updateHitCountProcessor I have written code to update table
it is updating the database in same thread (i.e main route thread)
You need to do
<camel:wireTap uri="ref:updateHitCountProcessor"/>
The processorRef attribute is creating and sending a new message, and not for tapping the existing message. So you should not use that.
The uri is used for sending the message which happens in a separate thread. So when you send it to the ref endpoint it will do that in another thread, and call your processor.
You can find details on the wire tap page at: http://camel.apache.org/wire-tap
From the documentation of the camel-seda component (here):
By default, the SEDA endpoint uses a single consumer thread, but you
can configure it to use concurrent consumer threads.
You can add a thread pool to a SEDA endpoint like this:
<from uri="seda:stageName?concurrentConsumers=5" />