I have just started working with an application that uses Apache Camel 2.15.3. I am new to Camel and I am trying to understand how the message gets sent and what it looks like between components in a route.
Routes in the application is set up using the spring extension xml. Here is an examlpe of how one of the routes might look like.
<route id="register">
<from uri="{{in.queue}}"/>
<enrich uri="direct:getSequenceNumber" strategyRef="sequenceNumb"/>
<to uri="bean:extractor"/>
<to uri="bean:mover"/>
<to uri="bean:normalizer"/>
<to uri="bean:logger"/>
<to uri="bean:packager"/>
<split parallelProcessing="true">
<simple>${body}</simple>
<to uri="{{out.queue}}"/>
</split>
</route>
First a short terminology question: what are the intermediate beans in this exemples called? Endpoints? Components? Or something else? Right now I'm calling them components.
My main confusion right now is to understand what gets input and what gets passed on from one component to the next. In this case all of the components are javabeans with one public method. The method sometime have void return type and sometimes returns some object. For example String, List, a camel Message-object, a project custom object that does not implement any kind of camel interface. Sometimes the return class matches the argument of the following bean and sometimes it doesn't.
I wan't to understand what are the limitations on what is returned and input to the components and be able to read and predict what the message will look like once inside a bean.
I have read through some of the documentation on the Camel website and my googling has not turned up anything helpful.
Does someone have an explanation or hints on what to search for or a link to some place that explains what is going on?
I suggest to read chapter 1 (free chapter) of the Camel in Action book - it covers all the important Camel concepts of its architecture and how a message looks like in Camel
https://www.manning.com/books/camel-in-action-second-edition
Related
Trying to google "route" in relation to Camel is like trying to google "the". Browsing the docs and can't find it either, only an interface called Route.
Inherited some code that looks like
rest("/someRoute")
.description("Some description")
.consumes("text/plain")
.produces("text/plain")
.post()
.route()
.to("direct:toSomewhere");
What does route() do? I have tried with and without route() and it doesn't seem to do anything.
Using .route allows you to define new route(s) within your rest-definition. It can be handy if your route is short or if you just want to process/transform/validate the message in someway before sending it to your actual consumer endpoint.
For example
rest("/someRoute")
.id("someRoute")
.description("Some description")
.post()
.consumes("text/plain")
.produces("text/plain")
.route()
.routeId("someRoutePost")
.process(new SomeMessageProcessor())
.to("direct:toSomewhere")
.end()
.endRest()
.get()
.route()
.routeId("someRouteGet")
.setHeader(Exchange.HTTP_RESPONSE_CODE, constant(405))
.setBody(constant("GET not allowed on this route"))
.end()
.endRest()
But if you just want to call direct consumer endpoint and do this stuff there instead you can do that.
it is up to ones preference really.
thanks, I see if I wanted to say call .log() I would have to put .route() first
Yes. Camel uses method-chaining with its Java-DSL where something like this is often required. When defining Rest most methods return RestDefinition but if you look closely .route method returns RouteDefition instead.
To get back to RestDefition from route one can use .endRest() as the .end() in the example doesn't really do anything other than make it easier to see where to RouteDefition block ends.
Update: Note that this example is for Camel 3.14.0. In Newer versions of Camel route() and endRest() methods have been Removed from RestDefition class. Example for Camel 3.18.1 can be found here.
I have two Camel routes, configured in XML and pasted below: -
Route 1:
<camel:route id="statementsArchivingPollRoute">
<camel:from uri="timer://tempQueue?fixedRate=true&period=30s&delay=30s"/>
<camel:transacted ref="PROPAGATION_REQUIRED">
<camel:process ref="statementsArchivingRequestZipProcessor"/>
<camel:choice>
<camel:when>
<camel:simple>${body.size} >= 1</camel:simple>
<camel:split>
<camel:simple>${body}</camel:simple>
<camel:marshal ref="archiveFileInterfaceMetadataMapper"/>
<camel:to pattern="InOnly"
uri="activemq:{{ccs.activemq.queue.prefix}}.sr.archive.bulk.ingestion.req?jmsMessageType=Text"/>
</camel:split>
<camel:log loggingLevel="INFO" message="Archiving content was processed"/>
</camel:when>
<camel:otherwise>
<camel:log loggingLevel="INFO" message="No archiving content to process"/>
</camel:otherwise>
</camel:choice>
</camel:transacted>
</camel:route>
Route 2:
<camel:route id="statementsArchivingBulkIngestionRequestRoute">
<camel:from uri="activemq:{{ccs.activemq.queue.prefix}}.sr.archive.bulk.ingestion.req"/>
<camel:throttle timePeriodMillis="4000">
<camel:constant>1</camel:constant>
<camel:setExchangePattern pattern="InOnly"/>
<camel:unmarshal ref="archiveFileInterfaceMetadataMapper"/>
<camel:bean ref="archiveFileEntryTransformer" method="transform"/>
<camel:setHeader headerName="CamelHttpMethod">
<camel:constant>POST</camel:constant>
</camel:setHeader>
<camel:toD uri="{{ccs.bulk.ingestion.service.ingest.archive.file}}"/>
</camel:throttle>
</camel:route>
The processor in the first route returns a list of request objects. The list is then split and each request is marshalled and placed on a queue.
The second route listens to this queue. When it de-queues a message, it unmarshals it, performs a transform and then uses it to send a post request to another service. I am throttling this route so that it only processes one message per second so as not to overwhelm the downstream service.
This all works fine when the list only contains a few requests and hence only a few messages enter the queue, but when there are many items in the list, route 2 times out and the log entry below appears:
Atomikos:12] c.a.icatch.imp.ActiveStateHandler : Timeout/setRollbackOnly of ACTIVE coordinator !
The timeout leads to the process repeating itself and the downstream service ends up getting called multiple times per message instead of just once.
I can't understand why the number of times route 2 is invoked should cause it to timeout. I thought that a single instance of the route would be launched for every message de-queued from the activemq. If a single message takes a long time to complete, then I would understand, but clearly the timeout is based on the cumulative times of all messages being de-queued.
I am fairly new to Camel and I have clearly misunderstood something from an architectural point of view. I would be extremely grateful for any guidance into how to stop these timeouts occurring. Thank you for reading.
Jon, you may want to try following to investigate
disable/comment the 2nd route. Purpose of using activemq would be to make process async which means 1st route should not have any impact due to 2nd route. If this works without route 2 then problem is elsewhere.
If you find 1st route is working fine without 2nd route then next would be to try setting fewer number of threads in 2nd route. May be put 1 or 2 threads and see if that helps. I am thinking it is contention on activemq rather than these route configurations.
Check the payload size that you are pushing to activemq. If you are publishing very large message to activemq that may also have an impact as large number of items and each item is large causing contention in activemq and transaction is taking longer than the timeout setting.
If you are pushing large data set to activemq, you may want to revisit the design. Save payload in some persistence area (db/file/cache) and sent notification events containing only reference to the payload and some metadata. 2nd route can then take the reference from event and retrieve the payload from where it was saved by route 1.
I use Camel with Fuse, and I have trouble with setting JMSReplyTo. Here is an excerpt from my route:
<setHeader headerName="JMSReplyTo" id="_setHeader2">
<constant>QTEST</constant>
</setHeader>
<setHeader headerName="CamelJmsDestinationName" id="_setHeader1">
<constant>queue://QM_TEST/SYSTEM.DEFAULT.LOCAL.QUEUE?targetClient=1</constant>
</setHeader>
<to id="_to1" uri="websphere:queue:SYSTEM.DEFAULT.LOCAL.QUEUE?replyTo=QTEST"/>
I took as an example the code given here:
Implementing native websphere MQ with CoD over Camel JMS component
At first, I thought it is because I removed unwanted jms header with targetClient=1 directive set in CamelJmsDestinationName header, but even without it, it won't set anything to ReplyToQ attribute of MQMD. I tried the suggestion given here too How to send message to different Queue hosted in different queue manager and hostname in IBM MQ cluster, but this also doesn't work for me, that is like this:
queue://QM_TEST/QTEST?mdReadEnabled=true&messageBody=0&mdWriteEnabled=true&XMSC_WMQ_REPLYTO_STYLE=1&targetClient=1
The question is why does it not work?
I have figured out how to set ReplyToQ attribute, but this is only a part of the problem that I'm facing now. As explained here, in JMS Producer chapter:
http://camel.apache.org/jms.html
all that is needed is this:
<setHeader headerName="CamelJmsDestinationName" id="_setHeader1">
<constant>queue://QMib_TEST/OUTPUTQ?targetClient=1</constant>
</setHeader>
<to id="_to1" uri="websphere:queue:SYSTEM.DEFAULT.LOCAL.QUEUE?replyTo=REPLYQ" pattern="InOut"/>
What this does is it puts request message to OUTPUTQ, and then listens on REPLYQ, but with matching some autogenerated Correlation ID. Good thing is that ReplyToQ is now set to REPLYQ in a request message, due to pattern="InOut" setting, bad thing is that the in our case replying application responds with setting Correlation ID to Message ID of received request, all from MQMD, and this Camel pattern by default doesn't generate message id in MQMD of request equal to (JMS?) correlation ID that it expects, so that response remains in queue, not consumed, even though it was put in proper queue. In fact, it even repeats putting requests after a wait interval for get elapses, producing further reponse messages unconsumed in REPLYQ. So, that is another problem I have to solve, how to deal with MessageID and CorrelationID properly, but the one from the subject, I have solved.
I'd like to dynamically invoke bean method depending on some header value:
This seems doesn't work: <to uri="bean:myBean?method=${in.header.methodName}&multiParameterArray=true"/>
Any ideas?
You can just set the method name in the header called "CamelBeanMethodName" and route your message to "bean" without specifying any method.
See this FAQ
http://camel.apache.org/how-to-use-a-dynamic-uri-in-to.html
From Camel 2.16 its easier as you can use <toD .. which is the dynamic-to. In older releases you can use the recipient list EIP. Its all explained in that FAQ.
I have an camel route consuming on a JMS (activemq) queue targeted to be called in a request/reply manner. Inside this route I split the message and invoke another activemq queue (also in a request/reply manner).
Heres a minimal route showing the situation
<route>
<from uri="activemq:A" />
<split>
<xpath>/root/subpart</xpath>
<inOut uri="activemq:B" />
</split>
</route>
The problem is that Camel does not set a new JMSCorrelationId (since there is already one from the incoming message). If nothing is done, you get responses with unknown correlationId's and the exchanges never end.
I didn't go into details but my guess is that the same temporaryQueue is used for the hole splitter but that it (logically) expects different correlation id's for each of the messages. All using the same, it recieves the first and does not know what to do with the others.
What would be the best solution to handle the situation ?
The one I've found working is to save in another header the incoming JMSCorrelationId (not sure I need to though), and removing it. This is not really as clean as I would want it to be, but I couldn't think of something else. Any ideas ?
Essentially, your case is described in this Jira issue It seems there will be an addition in 2.11 where you can ask Camel to create a new corr-id.
So, in the meantime, why don't you continue what you had working - to remove the JMSCorrelationId header <removeHeader headerName="JMSCorrelationId" /> before you send it to "activemq:B"? I guess that is the best solution for now.
You could, of course, play with the "useMessageIDAsCorrelationID" option as well on the second endpoint.