I use Camel with Fuse, and I have trouble with setting JMSReplyTo. Here is an excerpt from my route:
<setHeader headerName="JMSReplyTo" id="_setHeader2">
<constant>QTEST</constant>
</setHeader>
<setHeader headerName="CamelJmsDestinationName" id="_setHeader1">
<constant>queue://QM_TEST/SYSTEM.DEFAULT.LOCAL.QUEUE?targetClient=1</constant>
</setHeader>
<to id="_to1" uri="websphere:queue:SYSTEM.DEFAULT.LOCAL.QUEUE?replyTo=QTEST"/>
I took as an example the code given here:
Implementing native websphere MQ with CoD over Camel JMS component
At first, I thought it is because I removed unwanted jms header with targetClient=1 directive set in CamelJmsDestinationName header, but even without it, it won't set anything to ReplyToQ attribute of MQMD. I tried the suggestion given here too How to send message to different Queue hosted in different queue manager and hostname in IBM MQ cluster, but this also doesn't work for me, that is like this:
queue://QM_TEST/QTEST?mdReadEnabled=true&messageBody=0&mdWriteEnabled=true&XMSC_WMQ_REPLYTO_STYLE=1&targetClient=1
The question is why does it not work?
I have figured out how to set ReplyToQ attribute, but this is only a part of the problem that I'm facing now. As explained here, in JMS Producer chapter:
http://camel.apache.org/jms.html
all that is needed is this:
<setHeader headerName="CamelJmsDestinationName" id="_setHeader1">
<constant>queue://QMib_TEST/OUTPUTQ?targetClient=1</constant>
</setHeader>
<to id="_to1" uri="websphere:queue:SYSTEM.DEFAULT.LOCAL.QUEUE?replyTo=REPLYQ" pattern="InOut"/>
What this does is it puts request message to OUTPUTQ, and then listens on REPLYQ, but with matching some autogenerated Correlation ID. Good thing is that ReplyToQ is now set to REPLYQ in a request message, due to pattern="InOut" setting, bad thing is that the in our case replying application responds with setting Correlation ID to Message ID of received request, all from MQMD, and this Camel pattern by default doesn't generate message id in MQMD of request equal to (JMS?) correlation ID that it expects, so that response remains in queue, not consumed, even though it was put in proper queue. In fact, it even repeats putting requests after a wait interval for get elapses, producing further reponse messages unconsumed in REPLYQ. So, that is another problem I have to solve, how to deal with MessageID and CorrelationID properly, but the one from the subject, I have solved.
Related
I have two Camel routes, configured in XML and pasted below: -
Route 1:
<camel:route id="statementsArchivingPollRoute">
<camel:from uri="timer://tempQueue?fixedRate=true&period=30s&delay=30s"/>
<camel:transacted ref="PROPAGATION_REQUIRED">
<camel:process ref="statementsArchivingRequestZipProcessor"/>
<camel:choice>
<camel:when>
<camel:simple>${body.size} >= 1</camel:simple>
<camel:split>
<camel:simple>${body}</camel:simple>
<camel:marshal ref="archiveFileInterfaceMetadataMapper"/>
<camel:to pattern="InOnly"
uri="activemq:{{ccs.activemq.queue.prefix}}.sr.archive.bulk.ingestion.req?jmsMessageType=Text"/>
</camel:split>
<camel:log loggingLevel="INFO" message="Archiving content was processed"/>
</camel:when>
<camel:otherwise>
<camel:log loggingLevel="INFO" message="No archiving content to process"/>
</camel:otherwise>
</camel:choice>
</camel:transacted>
</camel:route>
Route 2:
<camel:route id="statementsArchivingBulkIngestionRequestRoute">
<camel:from uri="activemq:{{ccs.activemq.queue.prefix}}.sr.archive.bulk.ingestion.req"/>
<camel:throttle timePeriodMillis="4000">
<camel:constant>1</camel:constant>
<camel:setExchangePattern pattern="InOnly"/>
<camel:unmarshal ref="archiveFileInterfaceMetadataMapper"/>
<camel:bean ref="archiveFileEntryTransformer" method="transform"/>
<camel:setHeader headerName="CamelHttpMethod">
<camel:constant>POST</camel:constant>
</camel:setHeader>
<camel:toD uri="{{ccs.bulk.ingestion.service.ingest.archive.file}}"/>
</camel:throttle>
</camel:route>
The processor in the first route returns a list of request objects. The list is then split and each request is marshalled and placed on a queue.
The second route listens to this queue. When it de-queues a message, it unmarshals it, performs a transform and then uses it to send a post request to another service. I am throttling this route so that it only processes one message per second so as not to overwhelm the downstream service.
This all works fine when the list only contains a few requests and hence only a few messages enter the queue, but when there are many items in the list, route 2 times out and the log entry below appears:
Atomikos:12] c.a.icatch.imp.ActiveStateHandler : Timeout/setRollbackOnly of ACTIVE coordinator !
The timeout leads to the process repeating itself and the downstream service ends up getting called multiple times per message instead of just once.
I can't understand why the number of times route 2 is invoked should cause it to timeout. I thought that a single instance of the route would be launched for every message de-queued from the activemq. If a single message takes a long time to complete, then I would understand, but clearly the timeout is based on the cumulative times of all messages being de-queued.
I am fairly new to Camel and I have clearly misunderstood something from an architectural point of view. I would be extremely grateful for any guidance into how to stop these timeouts occurring. Thank you for reading.
Jon, you may want to try following to investigate
disable/comment the 2nd route. Purpose of using activemq would be to make process async which means 1st route should not have any impact due to 2nd route. If this works without route 2 then problem is elsewhere.
If you find 1st route is working fine without 2nd route then next would be to try setting fewer number of threads in 2nd route. May be put 1 or 2 threads and see if that helps. I am thinking it is contention on activemq rather than these route configurations.
Check the payload size that you are pushing to activemq. If you are publishing very large message to activemq that may also have an impact as large number of items and each item is large causing contention in activemq and transaction is taking longer than the timeout setting.
If you are pushing large data set to activemq, you may want to revisit the design. Save payload in some persistence area (db/file/cache) and sent notification events containing only reference to the payload and some metadata. 2nd route can then take the reference from event and retrieve the payload from where it was saved by route 1.
I have just started working with an application that uses Apache Camel 2.15.3. I am new to Camel and I am trying to understand how the message gets sent and what it looks like between components in a route.
Routes in the application is set up using the spring extension xml. Here is an examlpe of how one of the routes might look like.
<route id="register">
<from uri="{{in.queue}}"/>
<enrich uri="direct:getSequenceNumber" strategyRef="sequenceNumb"/>
<to uri="bean:extractor"/>
<to uri="bean:mover"/>
<to uri="bean:normalizer"/>
<to uri="bean:logger"/>
<to uri="bean:packager"/>
<split parallelProcessing="true">
<simple>${body}</simple>
<to uri="{{out.queue}}"/>
</split>
</route>
First a short terminology question: what are the intermediate beans in this exemples called? Endpoints? Components? Or something else? Right now I'm calling them components.
My main confusion right now is to understand what gets input and what gets passed on from one component to the next. In this case all of the components are javabeans with one public method. The method sometime have void return type and sometimes returns some object. For example String, List, a camel Message-object, a project custom object that does not implement any kind of camel interface. Sometimes the return class matches the argument of the following bean and sometimes it doesn't.
I wan't to understand what are the limitations on what is returned and input to the components and be able to read and predict what the message will look like once inside a bean.
I have read through some of the documentation on the Camel website and my googling has not turned up anything helpful.
Does someone have an explanation or hints on what to search for or a link to some place that explains what is going on?
I suggest to read chapter 1 (free chapter) of the Camel in Action book - it covers all the important Camel concepts of its architecture and how a message looks like in Camel
https://www.manning.com/books/camel-in-action-second-edition
I have a Camel route consuming messages from a Hazelcast SEDA queue. In case of any error when processing a message, I want to reduce the consuming throughput in order to avoid moving lot of messages to a dead letter queue. I have been investigating hacking something by using http://camel.apache.org/throttler.html
with dynamic values and Circuit Breaker: http://camel.apache.org/load-balancer.html but without sucess.
Is there any other approach to accomplish this ?
Is it another Camel Route of yours that sends messages to this Seda consumer? In that case you can use Throttler on that endpoint. Something like this:
<route>
<from uri="....." />
<throttle maximumRequestsPerPeriod="3" timePeriodMillis="10000">
<to uri="hazelcast:seda:foo" />
</throttle>
</route>
<route>
<from uri="hazelcast:seda:foo" />
<to uri="....." />
</route>
To deliver this feature, I ended building my own (and first) Camel component:
https://github.com/rodolfodpk/camel-hz-queue
I have an camel route consuming on a JMS (activemq) queue targeted to be called in a request/reply manner. Inside this route I split the message and invoke another activemq queue (also in a request/reply manner).
Heres a minimal route showing the situation
<route>
<from uri="activemq:A" />
<split>
<xpath>/root/subpart</xpath>
<inOut uri="activemq:B" />
</split>
</route>
The problem is that Camel does not set a new JMSCorrelationId (since there is already one from the incoming message). If nothing is done, you get responses with unknown correlationId's and the exchanges never end.
I didn't go into details but my guess is that the same temporaryQueue is used for the hole splitter but that it (logically) expects different correlation id's for each of the messages. All using the same, it recieves the first and does not know what to do with the others.
What would be the best solution to handle the situation ?
The one I've found working is to save in another header the incoming JMSCorrelationId (not sure I need to though), and removing it. This is not really as clean as I would want it to be, but I couldn't think of something else. Any ideas ?
Essentially, your case is described in this Jira issue It seems there will be an addition in 2.11 where you can ask Camel to create a new corr-id.
So, in the meantime, why don't you continue what you had working - to remove the JMSCorrelationId header <removeHeader headerName="JMSCorrelationId" /> before you send it to "activemq:B"? I guess that is the best solution for now.
You could, of course, play with the "useMessageIDAsCorrelationID" option as well on the second endpoint.
I'm trying to implement a mina service where the response to the final message should be based on the previous messages. Each message (header (1), data (n), end (1)) should receive a response, but the response to the "end" message should be based on the "header", and any "data" messages received as well as the "end" message. Currently, I'm routing the messages to an aggregator which completes when it finds a "header" and "end" message for a particular correlation id. Unfortunately, the response is being sent before (or at the same time?) the message is sent to the aggregator, so I don't have access to the aggregated message (which contains all the data I need to build the correct response) when building the response.
Is there a way to do this without manually storing and accessing the accumulated data (that is, without re-implementing camel's aggregator)?
Edit:
Route is something like:
<camelContext>
<route>
<from uri="mina:..."/>
<process ref="messageProcessor"/>
<aggregate>
<process ref="completeMessageProcessor"/>
</aggregate>
</route>
</camelContext>
I left out some tags and attributes (correlationExpression, completionPredicate, strategyRef, etc.) for clarity.
The messages were being aggregated properly, and they were being processed properly when "completed" (that is, when aggregated). But the response sent back through the mina endpoint to the client was the one generated by the messageProcessor, never the one generated by the completeMessageProcessor.
For example (and yes, it's a rather contrived example, but bear with me), let's say the protocol involves the client sending a header message which includes the total number of data messages it expects to send. Then it sends a number of data messages, which might be different in number to what it expected to send. Finally, it sends a footer, or end, message. The server should then respond back with the difference between the expected number of messages and the actual number of messages. With the route as written, that is impossible, since the number of messages is not known by the messageProcessor, which only processes individual messages. The completeMessageProcessor, having the aggregated message (consisting of header, all the data, and the end) does know this number, but the response generated at that point is not propagated back to the mina endpoint.
Changing the parsing of the messages to generate a message only when entire composed message is received is not an option, since the server must respond to the individual messages.
off the top, my guess is that the messageProcessor is setting up the OUT message, but the completeMessageProcessor is setting up the IN message. The mina consumer response is expecting/using the OUT message instead.
you can add some logging to verify this. if this is the case, then you might change your messageProcessor to use the IN body instead (or use exchanges headers) and add a transform after your completeMessageProcessor to set the OUT body based on the IN body
<transform>
<simple>${in.body}</simple>
</transform>
see this for more information: http://camel.apache.org/using-getin-or-getout-methods-on-exchange.html
UPDATE: after some discussion, the real issue is that the aggregator currently only handles "InOnly" exchanges