After going through Camel In Action book, I encountered following doubts.
I have below 2 routes
A.
from("file:/home/src") //(A.1)
.transacted("required") //(A.2)
.bean("dbReader", "readFromDB()") //(A.3) only read from DB
.bean("dbReader", "readFromDB()") //(A.4) only read from DB
.to("jms:queue:DEST_QUEUE") //(A.5)
Questions:
A.a. Is transacted in (A.2) really required here ?
A.b. If answer to #a is yes, then what should be the associated transaction manager of the "required" policy ? Should it be JmsTransactionManager or JpaTransactionManager ?
A.c. As DEST_QUEUE is at the producer end, so does JMS component in (A.5) need to be transacted ?
B.
from("jms:queue:SRC_QUEUE") //(B.1) transactional jms endpoint
.transacted("required") //(B.2)
.bean("someBean", "someMethod()") //(B.3) simple arithmetic computation
.to("jms1:queue:DEST_QUEUE") //(B.4)
SRC_QUEUE and DEST_QUEUE are queues of different jms broker
Questions:
B.a. The JMS component in (B.1) is marked as transacted, so in this case does route need to be transacted as mentioned in (B.2) ?
B.b. As DEST_QUEUE is at the producer end, so does JMS component in (B.4) need to be transacted ?
Very good questions to talk about Camel transaction handling.
General remark: when talking about Camel transactions it means to consume transacted from a transaction capable system like a database or JMS broker. The transacted statement in a route must immediately follow the from statement because it is always related to the consumption.
A.a. Is transacted in (A.2) really required here ?
No, it is not. Since the filesystem is not transaction capable, it can't be of any help in this route.
A.b. If answer to #a is yes, then ... ?
There is no "filesystem transaction manager"
A.c. As DEST_QUEUE is at the producer end, so does JMS component in (A.5) need to be transacted ?
Not sure, but I don't think so. The producer tries to hand over a message to the broker. Transactions are used to enable a rollback, but if the broker has not received the data, what could a rollback do?
B.a. The JMS component in (B.1) is marked as transacted, so in this case does route need to be transacted as mentioned in (B.2) ?
It depends because SRC and DEST are on different brokers.
If you want an end-to-end-transaction between the brokers, you need to use an XA-transaction manager and then you have to mark the route as transacted.
If you are OK with consumer transaction, you can configure the JMS component for it and omit the Spring Tx manager and the Camel transacted statement.
To clarify the last point: if you consume with local broker transaction, Camel does not commit the message until the route is successfully processed. So if any error occurs, a rollback would happen and the message would be redelivered.
In most cases this is totally OK, however, what still could happen with two different brokers is that the route is successfully processed, the message is delivered to DEST broker but Camel is no more able to commit against SRC broker. Then a redelivery occurs, the route is processed one more time and the message is delivered multiple times to DEST broker.
In my opinion the complexity of XA transactions is harder to handle than the very rare edge cases with local broker transactions. But this is a very subjective opinion and perhaps also depends on the context or data you are working with.
And important to note: if SRC and DEST broker are the same, local broker transactions are 100% sufficient! Absolutely no need for Spring Tx manager and Camel transacted.
B.b. As DEST_QUEUE is at the producer end, so does JMS component in (B.4) need to be transacted ?
Same as answer to B.a.
Good afternoon,
I'd like to take a minute to reply to your questions. I'll address the 'B' side questions.
WRT:
B.a. The JMS component in (B.1) is marked as transacted, so in this case does route need to be transacted as mentioned in (B.2) ?
Yes. Both the source and destination components need to be marked as transacted. Marking the components as transacted will start local JMS transactions on the source and destination session. Note that these are two separate local JMS transactions that are managed by two separate JmsTransactionManagers.
Marking the route as 'transacted' will start a JTA transaction context. Note that the PlatformTransactionManager must be a JtaTransactionManager. When the 'to' component is called, the local JMS transaction for the message send will be synchronized with the local transaction for the message get. (JTA synchronized transactions). This means that the send will get a callback when the remote broker acknowledges the commit for the send. At that point, the message receive will be committed. This is 'dups OK' transactional behavior (not XA). You have a window where the message has been sent, but the receive has not been ack'ed.
Actually getting this working is tricky. Here is a sample:
<!-- ******************** Camel route definition ********************* -->
<camelContext allowUseOriginalMessage="false"
id="camelContext-Bridge-Local" streamCache="true" trace="true" xmlns="http://camel.apache.org/schema/blueprint">
<route id="amq-to-amq">
<from id="from" uri="amqLoc:queue:IN"/>
<transacted id="trans"/>
<to id="to" uri="amqRem:queue:OUT"/>
</route>
</camelContext>
<!-- ********************* Local AMQ configuration ************************** -->
<bean class="org.apache.activemq.camel.component.ActiveMQComponent" id="amqLoc">
<property name="configuration">
<bean class="org.apache.camel.component.jms.JmsConfiguration">
<property name="connectionFactory" ref="AmqCFLocalPool"/>
<property name="receiveTimeout" value="100000"/>
<property name="maxConcurrentConsumers" value="3"/>
<property name="cacheLevelName" value="CACHE_NONE"/>
<property name="transacted" value="true"/>
</bean>
</property>
</bean>
<bean class="org.apache.activemq.jms.pool.PooledConnectionFactory" id="AmqCFLocalPool">
<property name="maxConnections" value="1"/>
<property name="idleTimeout" value="0"/>
<property name="connectionFactory" ref="AmqCFLocal"/>
</bean>
<bean class="org.apache.activemq.ActiveMQConnectionFactory" id="AmqCFLocal">
<property name="brokerURL" value="tcp://10.0.0.170:61616?jms.prefetchPolicy.all=0"/>
<property name="userName" value="admin"/>
<property name="password" value="admin"/>
</bean>
<!-- ********************* Remote AMQ configuration ************************** -->
<bean class="org.apache.activemq.camel.component.ActiveMQComponent" id="amqRem">
<property name="configuration">
<bean class="org.apache.camel.component.jms.JmsConfiguration">
<property name="connectionFactory" ref="AmqCFRemotePool"/>
<property name="transacted" value="true"/>
</bean>
</property>
</bean>
<bean class="org.apache.activemq.jms.pool.PooledConnectionFactory"
destroy-method="stop" id="AmqCFRemotePool" init-method="start">
<property name="maxConnections" value="1"/>
<property name="idleTimeout" value="0"/>
<property name="connectionFactory" ref="AmqCFRemote"/>
</bean>
<bean class="org.apache.activemq.ActiveMQConnectionFactory" id="AmqCFRemote">
<property name="brokerURL" value="tcp://10.0.0.171:61616"/>
<property name="userName" value="admin"/>
<property name="password" value="admin"/>
</bean>
Enable DEBUG logging for the org.springframework.jms.connection.JmsTransactionManager, and DEBUG/TRACE level logging for the JTA transaction manager that you are using.
I want old messages to be automatically removed from topic so I'm using the option timeToLive which is described in the Camel ActiveMQ component documentation. However, it only works with queue and not with topic. Any idea why?
For testing purposes I made this simple route:
<route>
<from uri="timer://foo?repeatCount=1"/>
<to uri="activemq:topic:test?timeToLive=10000"/>
</route>
Route sends message to my test topic with TTL=10seconds. I would expect that after 10 seconds message would disappear.
When I use queue instead everything works as expected ( <to uri="activemq:queue:test?timeToLive=10000"/> )
There is a route which is consuming messages from a jms queue and after doing some processing sending a request to unreliable web service(Which can be down at times).
So in case of service is down then i need to stop consuming from queue for some time. I tried to use ThrottlingExceptionRoutePolicy . It will stop route as per configuration but issue is for the current message which gets the error as message is getting moved to dead letter queue. I've gone through the code of ThrottlingExceptionRoutePolicy as per code this will be called after all error handling is done for route.
So i need to modify default error handling of camel to avoid putting message to DLQ for some specific cases. I tried to configure various error handlers provided by camel but in all cases camel is putting messages to DLQ.
Is there some configuration or i need to write custom error handler to achieve the same.
<camel:route id="someroute" routePolicyRef="throttlingExceptionRoutePolicy"
errorHandlerRef="myTransactionErrorHandlerErrorHandler">
<camel:from uri="activemq:inputQueue" />
<camel:transacted />
<camel:bean ref="afterQueueProcessor" />
<camel:setHeader headerName="CamelHttpMethod">
<camel:constant>POST</camel:constant>
</camel:setHeader>
<camel:setHeader headerName="Content-Type">
<camel:constant>application/xyz</camel:constant>
</camel:setHeader>
<camel:to
uri="http://localhost:8080/some-ws/newOrder?orderId=dd&productName=bb&quantity=1" />
</camel:route>
This questions is related to async communication
My Camel:
<from uri="quartz2://processTimers?cron=5+*+*+*+*+*" />
<to uri="mybatis:selectProducts?statementType=SelectList&onConsume=consumeProduct"/>
<bean ref="productService" method="process" />
<to uri="mq:queue:my.queue"/>
When using a Quartz from, the selectProducts returns the expected results but the onConsume for some doesnt execute at the end, I suspect this is because its a "to" and not a "from" method.
Is there anyway to have a cron scheduled mybatis select with an onConsume?
Updated:
<from uri="mybatis:selectProducts?statementType=SelectList&onConsume=markProductAsExtracted&maxMessagesPerPoll={{MAX_RECORDS_PER_PROCESS}}&scheduler=quartz2&scheduler.cron=5+*+*+*+*+?"/>
<bean ref="productService" method="process" />
<to uri="mq:queue:my.queue"/>
Yes see the scheduled polling consumer: http://camel.apache.org/polling-consumer.html
You can specify on the mybatis endpoint that the scheduler is cron and then setup the cron value as well. See that doc for more details.
Also I wrote a little blog once: http://www.davsclaus.com/2013/08/apache-camel-212-even-easier-cron.html its about the file component but its the same for mybatis.
I'm using "Fuse Tooling Routes Editor" (aka "Fuse Integration Editor", aka "JBoss Fuse Tooling Apache Camel Editor"), described for example here. The version is "Nightly build version 8.0.0.v20150805-1820-H573-MASTER".
I'd like to create a Camel route having a multicast which aggregates the responses it receives from the components it sends a message to, and then forwards the aggregate to a single final receiver. This is certainly doable in Camel, as one of the parameters of a multicast, strategyRef, has the description:
Refers to an AggregationStrategy to be used to assemble the replies from the multicasts, into a single outgoing message from the Multicast. By default Camel will use the last reply as the outgoing message.
However, when I write the following set of routes in my camel-context.xml (which compiles and runs fine):
<route>
<from uri="direct:a"/>
<setExchangePattern pattern="InOut"/>
<multicast strategyRef="x">
<to uri="direct:b"/>
<to uri="direct:c"/>
</multicast>
<log message="This flow works!"/>
<to uri="mock:p"/>
</route>
<route>
<from uri="direct:b"/>
<to uri="mock:q"/>
</route>
<route>
<from uri="direct:c"/>
<to uri="mock:r"/>
</route>
and then click to go to the visual editor, it produces the following (incorrect) picture:
When I switch back to the XML editing, my routes are automatically changed to:
<route>
<from uri="direct:a"/>
<setExchangePattern pattern="InOut"/>
<log message="This flow works!"/>
<to uri="mock:p"/>
</route>
<route>
<from uri="direct:b"/>
<to uri="mock:q"/>
</route>
<route>
<from uri="direct:c"/>
<to uri="mock:r"/>
</route>
And on switching to the editor once again, I get the picture:
This seems like a bug in the editor, but perhaps I'm doing something wrong here?
Many thanks!
Edit:
We found that indeed this used to be an issue logged with the Fuse Camel editor, which however should have been fixed in version 7.1 of the tooling.
that seems to be a bug in the editor. Thank you for reporting it.
I reopened the original issue in Jira. You can track the progress there.
Lars