I'm new to activeMQ / Camel, so please bear with me.
In a camel route, I use a splitter to spawn multiple sub routes. Each one of this routes will use some external APIs to do some job, and poll until job is done. I'm that far.
Now I need to trigger a last action to collect the result of all of these routes. How would I do that the Camel / AMQ way ?
I was thinking of posting a message in each sub-route to an AMQ queue, but I haven't found a way yet to wait for the N messages to be received in that queue before consuming it in my final Camel route.
Thank you.
If you need to collect all the results from the splitter sub-routes and execute some action on those results after all the sub-routes are completed then you can use an aggregation strategy(https://camel.apache.org/manual/latest/aggregate-eip.html) on the splitter.
By using an aggregation strategy, the Exchange after the split will contain the results of all the sub-routes.
Example using the Java DSL:
.split(expression, new GroupedExchangeAggregationStrategy())
...
.end()
The GroupedExchangeAggregationStrategy aggregates all split Exchanges into a list. You can use another predefined aggregation strategy if you need, or you can create a custom one by extending org.apache.camel.processor.aggregate.AggregationStrategy.
Related
I want to listen on ActiveMQ topic based on the hostname of system and some other logic. I planned to use pollEnrich for it so I evaluate my logic and provide topic name in pollEnrich but as per document:
pollEnrich or enrich does not access any data from the current Exchange which means when polling it cannot use any of the existing headers you may have set on the Exchange. For example you cannot set a filename in the Exchange.FILE_NAME header and use pollEnrich to consume only that file. For that you must set the filename in the endpoint URI.
How i can figure out this
from("timer://ipc?repeatCount=1")
.. some logic..
.setHeader("topic_no",simple("{{env:HOSTNAME}}"))
.pollEnrich("mqtt:foo?host=tcp://0.0.0.0:1883&subscribeTopicNames=${header.topic_no}/status&clientId=ipc")
.to("log:my?showAll=true&multiline=true");
Please don't suggest to use hostname directly in URI. As I highlighted I have to compute other logic too.
What other option or way I can use?
Will pollEnrich kept listening on topic or it will listen once and end the route?
Update1:
I figured out we can use simple expression with for dynamic URI, But one issue with pollEnrich it only pick one message how i can make sure it kept on listening as consumer? I want that before pollEnrich part get execute once and TopicListener kept listening till application is up.
Will pollEnrich kept listening on topic or it will listen once and end the route?
Same as the fact you have figure out, Camel pollEnrich component will listen on topic and consume at most one message per call.
What other option or way I can use?
Repeat pollEnrich by loop
Create new route at run-time by routeBuilder
Option 1 is naive, but simple in concept. pollEnrich will do once and loop will repeat it. However, this method need to handle more scenario than you might expected.
Option 2 is a better approach. You create a route at run-time and the consumer endpoint URI is pass by variable. That said, you can create the consumer route dynamically after your computation logic.
Example for routeBuilder
I am using Apache camel to trigger various actions based on timer events. For timer events to trigger, I am using camel-quartz. And a sample trigger looks like this:
from("quartz://job_timers/customer_activation?cron=0+0+0+*+*+?+*&trigger.timeZone=America/Chicago")
.routeId("TrigCustActivation")
.log(LoggingLevel.INFO, "Triggered Customer Activation Job")
.to("direct://set/headers/activation")
.to("direct://customer/activation");
Over the period of time, the number of such triggers have increased a lot. And they will be increasing even more. So, instead of keeping a hard-copy of when which job will trigger, I was planning to create a web-route, so that when I perform a GET request, it would fetch all the active routes, and print their RouteIds (I will update all the RouteIds to have time as well. Say, the routeId for above flow will become "TrigCustActivation_00_00_CST_Daily").
I was able to print all the route at application startup, but failed to fetch those dynamically through a GET request.
Can we access other RouteIds from a running route? Is my approach feasible?
Yes there is API on CamelContext where you can fetch all the current routes, and you can then get their IDs etc. See the javadoc of this class.
http://static.javadoc.io/org.apache.camel/camel-core/2.21.0/org/apache/camel/CamelContext.html
We're having some came routes defined in a single CamelContext which contains Web services,activemq.. in the Route.
Initially we've deployed the Routes as WAR in single Jboss node.
To scale out(usually we're doing for web services) , I've deployed the same CamelContext in multiple Jboss nodes.
But the performance is actually decreased.
FYI: All the CamelContexts points to the Same activemq brokers.
Here are my questions:
How to load balance/ Fail over camel context in different machines?
If CamelContexts are deployed in multiple nodes, Will aggregation work correctly?
Kindly give your thoughts!
Without seeing your system in detail, there is no way of knowing why it has slowed down so I'll pass over that. For your other two questions:
Failover
You don't say what sort of failover/load balancing behaviour you want. The not-very-helpful Camel documentation is here: http://camel.apache.org/clustering-and-loadbalancing.html.
One mechanism that works easily with Camel and ActiveMQ is to deploy to multiple servers and run active-active, sharing the same ActiveMQ queues. Each route attempts to read from the same queue to get a message to process. Only one route will get the message and therefore only one route processes it. Other routes are free to read subsequent messages, giving you simple load balancing. If one route crashes, the other routes will continue to process the messages, there will just be reduced capacity on your system.
If you need to provide fault tolerance for your web services then you need to look outside Camel and use something like Elastic Load Balancing. http://aws.amazon.com/elasticloadbalancing/
Aggregation
Each Camel context will run independently of the other contexts so one context will aggregate messages independently of what other contexts are up to. For example, suppose you have an aggregator that stores messages from ActiveMQ queue until receives a special end-of-batch message. If you have the aggregator running in two different routes, the messages will be split between the two routes and only one route will receive the end-of-batch message. So one aggregator will sit there with half the messages and do nothing. The other aggregator will have the other messages and will process the end-of-batch message but won't know about the messages the other route picked up.
I have a Camel based application which receives a request and gives the reply from cache but in between this process it updates the database which i want it to run in a different thread , can anyone tell me how can i achieve this, i tried with WireTap and SEDA but it does not work that way...any help appreciated.
<camel:wireTap uri="seda:tap" processorRef="updateHitCountProcessor"/>
In updateHitCountProcessor I have written code to update table
it is updating the database in same thread (i.e main route thread)
You need to do
<camel:wireTap uri="ref:updateHitCountProcessor"/>
The processorRef attribute is creating and sending a new message, and not for tapping the existing message. So you should not use that.
The uri is used for sending the message which happens in a separate thread. So when you send it to the ref endpoint it will do that in another thread, and call your processor.
You can find details on the wire tap page at: http://camel.apache.org/wire-tap
From the documentation of the camel-seda component (here):
By default, the SEDA endpoint uses a single consumer thread, but you
can configure it to use concurrent consumer threads.
You can add a thread pool to a SEDA endpoint like this:
<from uri="seda:stageName?concurrentConsumers=5" />
What's the best strategy to send SMS via SMPP with Camel ? Should I use the ProducerTemplate ?
I'm new to camel so I'm not confident if my strategy is the best.
In my application upon reception of an SMS, I have to send back an other SMS with some computed content.
I created a
route smsIn that looks like this
from "uri=smpp ..."
unmarshal ref="bindyDataFormat"
to "uri=bean:myBean
and a route smsOut with
from "uri=direct:smsOut"
to "uri=smpp ..."
The smsIn route, receives the sms, transforms its conent (csv data) in a pojo and send that pojo to myBean.
In myBean I do some processing and then call a ProducerTemplate which send my computed message to the endpoint "direct:smsOut".
The reason I use the producerTemplate is that I have to set some info from my pojo in the header (CamelSmppDestAddr) and the body of the Exchange.
I have tested with the logica SMSC simulator, this seems to work fine, but would like to have your opinion about this solution ?
What about reliability , transaction ?
Should I store my message before trying to send it to the SMSC ?
Should I store it in a database, post it to a queue ?
I'm not sure why you have a producer template, you could just build up the route instead (given that you return something from your bean or takes an Exchange as paramter).
<from uri="smpp: ..."/>
<bean ref="bean:myBean"/>
<to uri="jms:queue:myQueue"/>
then not use direct, but use a JMS queue that is transactional and persistent. Say your smpp call fails, the message would have been gone. Using a queue like this and make sure its transactional, you can make sure not to lose data in this stage of the route.
<from uri="jms:queue:myQueue"/>
<transactional/>
<to uri="smpp.."/>
I suggest using Apache ActiveMQ as JMS middleware. Actually, if you download ActiveMQ, you get camel bundled, so you could actually run your Camel routes from ActiveMQ.
You might want to tweak how retries and error handling occurs dependent on what you want to happen (retry every second forever?, retry five times, then put to error queue? etc).
Read this page: Transaction Error handling in Camel
For deeper info and more tweaks, you might also want to read this:
Transactional Client