ActiveMQ embedded bridge to Camel JMS bridge - apache-camel

I have an old application which handle JMS messages with ActiveMQ 5.8.0 and some JNDI remote topic connected to this ActiveMQ.
I have a connector like that :
<bean class="org.apache.activemq.network.jms.JmsConnector">
<property name="outboundTopicConnectionFactory" ref="jmsConnectionFactoryTo" />
<property name="outboundClientId" value="${remote.clientId}" />
<property name="jndiOutboundTemplate" ref="jndiTemplateTo" />
<property name="preferJndiDestinationLookup" value="true" />
<property name="inboundTopicBridges">
<list>
<bean class="org.apache.activemq.network.jms.InboundTopicBridge">
<property name="inboundTopicName" value="${remote.topic.to}"/>
<property name="localTopicName" value="${local.topic.to}"/>
<property name="consumerName" value="${remote.consumer.name}"/>
<property name="selector" value="${remote.selector}"/>
</bean>
</list>
</property>
</bean>
It works great, but now, for some technical reasons (strict JMS 1.1), I need to use "ConnectionFactory" instead of "TopicConnectionFactory".
With the actual configuration, I'm stuck because ActiveMQ seems to use "TopicConnectionFactory" instead of "ConnectionFactory", and my new class "MyConnectionFactoryImpl" implements "ConnectionFactory" now :
nested exception is org.springframework.beans.ConversionNotSupportedException:
Failed to convert property value of type 'com.webmethods.jms.impl.MyConnectionFactoryImpl'
to required type 'javax.jms.TopicConnectionFactory'
for property 'outboundTopicConnectionFactory';
nested exception is java.lang.IllegalStateException:
Cannot convert value of type [com.webmethods.jms.impl.MyConnectionFactoryImpl]
to required type [javax.jms.TopicConnectionFactory] for property 'outboundTopicConnectionFactory':
no matching editors or conversion strategy found
In "org.apache.activemq.network.jms.JmsConnector" class, it use everywhere "TopicConnectionFactory", which is not recommended anymore in JMS 1.1.
EDIT :
According to #Justin Bertram, I need to use Camel instead of ActiveMQ embedded bridge. But I can't find any example of XML configuration which I can use to replace my actual two beans JMSConnector. Which is the simple way to do this keeping my XML config files ?

As the documentation for the JMS to JMS Bridge (i.e. org.apache.activemq.network.jms.JmsConnector) states:
ActiveMQ provides bridging functionality to other JMS providers that implement the JMS 1.0.2 and above specification.
In other words, the whole goal of the JMS to JMS Bridge is to use the JMS 1.0.2 interface(s). Changing it so that it only used JMS 1.1 would defeat the purpose.
The documentation also states that you should use Camel instead of the JMS to JMS Bridge:
Warning, try Camel first!
Note that we recommend you look at using Apache Camel for bridging ActiveMQ to or from any message broker (or indeed any other technology, protocol or middleware) as its much easier to:
keep things flexible; its very easy to map different queue/topic to one or more queues or topics on the other provider
perform content based routing, filtering and other Enterprise Integration Patterns
allows you to work with any technology, protocol or middleware, not just JMS providers
Therefore I recommend you use Camel instead of org.apache.activemq.network.jms.JmsConnector.

I would think that having your code return a TopicConnectionFactory would be the simplest solution. Even the JMS 2.0 specification provides the TopicConnectionFactory. No matter what version of ActiveMQ you are using, you certainly have the option of using the TopicConnectionFactory in your code and providing that to your bridge.
Note that the Camel route:
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="mqseries:Foo.Bar"/>
<to uri="activemq:Cheese"/>
</route>
</camelContext>
has no error handling. For example, if the 'to' endpoint is down, this route will read from the 'from' endpoint and just throw the messages on the floor. Furthermore, if the 'to' component is not configured to use a caching/pooling connection factory, then a new JMS connection will be created for each message sent. This has poor performance and can result in many sockets in the TIME_WAIT state. Bottom line - beware trivial Camel routes.

Related

Jms component transacted and camel route transacted

After going through Camel In Action book, I encountered following doubts.
I have below 2 routes
A.
from("file:/home/src") //(A.1)
.transacted("required") //(A.2)
.bean("dbReader", "readFromDB()") //(A.3) only read from DB
.bean("dbReader", "readFromDB()") //(A.4) only read from DB
.to("jms:queue:DEST_QUEUE") //(A.5)
Questions:
A.a. Is transacted in (A.2) really required here ?
A.b. If answer to #a is yes, then what should be the associated transaction manager of the "required" policy ? Should it be JmsTransactionManager or JpaTransactionManager ?
A.c. As DEST_QUEUE is at the producer end, so does JMS component in (A.5) need to be transacted ?
B.
from("jms:queue:SRC_QUEUE") //(B.1) transactional jms endpoint
.transacted("required") //(B.2)
.bean("someBean", "someMethod()") //(B.3) simple arithmetic computation
.to("jms1:queue:DEST_QUEUE") //(B.4)
SRC_QUEUE and DEST_QUEUE are queues of different jms broker
Questions:
B.a. The JMS component in (B.1) is marked as transacted, so in this case does route need to be transacted as mentioned in (B.2) ?
B.b. As DEST_QUEUE is at the producer end, so does JMS component in (B.4) need to be transacted ?
Very good questions to talk about Camel transaction handling.
General remark: when talking about Camel transactions it means to consume transacted from a transaction capable system like a database or JMS broker. The transacted statement in a route must immediately follow the from statement because it is always related to the consumption.
A.a. Is transacted in (A.2) really required here ?
No, it is not. Since the filesystem is not transaction capable, it can't be of any help in this route.
A.b. If answer to #a is yes, then ... ?
There is no "filesystem transaction manager"
A.c. As DEST_QUEUE is at the producer end, so does JMS component in (A.5) need to be transacted ?
Not sure, but I don't think so. The producer tries to hand over a message to the broker. Transactions are used to enable a rollback, but if the broker has not received the data, what could a rollback do?
B.a. The JMS component in (B.1) is marked as transacted, so in this case does route need to be transacted as mentioned in (B.2) ?
It depends because SRC and DEST are on different brokers.
If you want an end-to-end-transaction between the brokers, you need to use an XA-transaction manager and then you have to mark the route as transacted.
If you are OK with consumer transaction, you can configure the JMS component for it and omit the Spring Tx manager and the Camel transacted statement.
To clarify the last point: if you consume with local broker transaction, Camel does not commit the message until the route is successfully processed. So if any error occurs, a rollback would happen and the message would be redelivered.
In most cases this is totally OK, however, what still could happen with two different brokers is that the route is successfully processed, the message is delivered to DEST broker but Camel is no more able to commit against SRC broker. Then a redelivery occurs, the route is processed one more time and the message is delivered multiple times to DEST broker.
In my opinion the complexity of XA transactions is harder to handle than the very rare edge cases with local broker transactions. But this is a very subjective opinion and perhaps also depends on the context or data you are working with.
And important to note: if SRC and DEST broker are the same, local broker transactions are 100% sufficient! Absolutely no need for Spring Tx manager and Camel transacted.
B.b. As DEST_QUEUE is at the producer end, so does JMS component in (B.4) need to be transacted ?
Same as answer to B.a.
Good afternoon,
I'd like to take a minute to reply to your questions. I'll address the 'B' side questions.
WRT:
B.a. The JMS component in (B.1) is marked as transacted, so in this case does route need to be transacted as mentioned in (B.2) ?
Yes. Both the source and destination components need to be marked as transacted. Marking the components as transacted will start local JMS transactions on the source and destination session. Note that these are two separate local JMS transactions that are managed by two separate JmsTransactionManagers.
Marking the route as 'transacted' will start a JTA transaction context. Note that the PlatformTransactionManager must be a JtaTransactionManager. When the 'to' component is called, the local JMS transaction for the message send will be synchronized with the local transaction for the message get. (JTA synchronized transactions). This means that the send will get a callback when the remote broker acknowledges the commit for the send. At that point, the message receive will be committed. This is 'dups OK' transactional behavior (not XA). You have a window where the message has been sent, but the receive has not been ack'ed.
Actually getting this working is tricky. Here is a sample:
<!-- ******************** Camel route definition ********************* -->
<camelContext allowUseOriginalMessage="false"
id="camelContext-Bridge-Local" streamCache="true" trace="true" xmlns="http://camel.apache.org/schema/blueprint">
<route id="amq-to-amq">
<from id="from" uri="amqLoc:queue:IN"/>
<transacted id="trans"/>
<to id="to" uri="amqRem:queue:OUT"/>
</route>
</camelContext>
<!-- ********************* Local AMQ configuration ************************** -->
<bean class="org.apache.activemq.camel.component.ActiveMQComponent" id="amqLoc">
<property name="configuration">
<bean class="org.apache.camel.component.jms.JmsConfiguration">
<property name="connectionFactory" ref="AmqCFLocalPool"/>
<property name="receiveTimeout" value="100000"/>
<property name="maxConcurrentConsumers" value="3"/>
<property name="cacheLevelName" value="CACHE_NONE"/>
<property name="transacted" value="true"/>
</bean>
</property>
</bean>
<bean class="org.apache.activemq.jms.pool.PooledConnectionFactory" id="AmqCFLocalPool">
<property name="maxConnections" value="1"/>
<property name="idleTimeout" value="0"/>
<property name="connectionFactory" ref="AmqCFLocal"/>
</bean>
<bean class="org.apache.activemq.ActiveMQConnectionFactory" id="AmqCFLocal">
<property name="brokerURL" value="tcp://10.0.0.170:61616?jms.prefetchPolicy.all=0"/>
<property name="userName" value="admin"/>
<property name="password" value="admin"/>
</bean>
<!-- ********************* Remote AMQ configuration ************************** -->
<bean class="org.apache.activemq.camel.component.ActiveMQComponent" id="amqRem">
<property name="configuration">
<bean class="org.apache.camel.component.jms.JmsConfiguration">
<property name="connectionFactory" ref="AmqCFRemotePool"/>
<property name="transacted" value="true"/>
</bean>
</property>
</bean>
<bean class="org.apache.activemq.jms.pool.PooledConnectionFactory"
destroy-method="stop" id="AmqCFRemotePool" init-method="start">
<property name="maxConnections" value="1"/>
<property name="idleTimeout" value="0"/>
<property name="connectionFactory" ref="AmqCFRemote"/>
</bean>
<bean class="org.apache.activemq.ActiveMQConnectionFactory" id="AmqCFRemote">
<property name="brokerURL" value="tcp://10.0.0.171:61616"/>
<property name="userName" value="admin"/>
<property name="password" value="admin"/>
</bean>
Enable DEBUG logging for the org.springframework.jms.connection.JmsTransactionManager, and DEBUG/TRACE level logging for the JTA transaction manager that you are using.

how to configure camel route for using transactions with database

I need to handle transactions with OSGI applications under Karaf(I am using ServiceMix 7.0). All logic is under camel route.
First, I receive Web Requests and process it in route, make transformations and then I need to start transaction : call storedProcedures in Oracle, call another web service and if call to web service is successfull I need to commit stored procedure invocation, otherwise rollback. May be using container managed transactions as in EJB or JTA.
Camel doesn't have out of box components for calling stored procedure. So I am using org.springframework.jdbc.object.StoredProcedure. And now I don't know how to make SP invokation with transactional context.
Here is what you can do:
from("direct:mainRoute")
.transacted()
.to("direct:invokeService")
.to("sql-stored:SUBNUMBERS(INTEGER ${headers.num1},INTEGER ${headers.num2},OUT INTEGER resultofsub)
You will also need to add TransactionManager to the context.
<!-- spring transaction manager -->
<!-- this is the transaction manager Camel will use for transacted routes -->
<bean id="txManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="dataSource"/>
</bean>
For more details on Camel SQL stored proc Component refer to:
http://camel.apache.org/sql-stored-procedure.html

Protect CXF Service in Fuse with Basic Authentication on LDAP Users

I have a SOAP/REST service implemented in CXF inside Red Hat JBoss Fuse (in a Fabric).
I need to protect it with Basic Authentication, and credentials must be checked on a LDAP server.
Can this be done without a custom interceptor?
Can I maybe use the container JAAS security (configured with LDAP) to protect the service the same way I can protect the console?
Yes the container JAAS security realm can be used to protect a web service.
An example is here.
The example page doesn't explain the implementation, but a quick look at the blueprint.xml file reveals the following configuration:
<jaxrs:server id="customerService" address="/securecrm">
<jaxrs:serviceBeans>
<ref component-id="customerSvc"/>
</jaxrs:serviceBeans>
<jaxrs:providers>
<ref component-id="authenticationFilter"/>
</jaxrs:providers>
</jaxrs:server>
<bean id="authenticationFilter" class="org.apache.cxf.jaxrs.security.JAASAuthenticationFilter">
<!-- Name of the JAAS Context -->
<property name="contextName" value="karaf"/>
</bean>
So it's just a matter of configuring a JAAS authentication filter.
"karaf" is the default JAAS realm for the container: users are defined in etc/users.properties
To define more realms, info is here.
To have users on LDAP, see here.
The answer above is correct, but please note that for more recent versions of Fuse (past 6.1), the "rank" in the LDAP configuration must be greater than 100 in order to override the default karaf realm.
Also, with current patches applied, in Fuse 6.2.X, connection pooling for the LDAP connections can be enabled:
<!-- LDAP connection pooling -->
<!-- http://docs.oracle.com/javase/jndi/tutorial/ldap/connect/pool.html -->
<!-- http://docs.oracle.com/javase/jndi/tutorial/ldap/connect/config.html -->
context.com.sun.jndi.ldap.connect.pool=true
</jaas:module>
</jaas:config>
This is very important for high volume web-services. A connection pool is maintained to the LDAP server. This both avoids connection creation overhead and having closing sockets lingering in TIME-WAIT state.

Simple way to monitor Camel's localhost broker

as stated, is there a simple way to monitor Camel's vm incoming messages to the embedded broker ?
<bean id="jms" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="vm://localhost?broker.persistent=false" />
</bean>
Thanks!
Your question isn't quite clear on what you would like to monitor.
-You could log what you send to the broker and read from it.
-You could connect to jmx to view a bunch of processing information.
-If you are on the FUSE platform there is a management console that exposes the jmx endpoints to a web url.
-You could also setup JBoss operations network to poll all of your jmx info and do trending on the data.
If I didn't cover the use case you had in mind please update your question with some details and ping me in a comment so I can get back to you.

Camel + Java DSL Fluent builder with real ActiveMQ Broker

I'm trying to implement a WireTap with Java DSL Fluent Builders, which gives the following example code snippet.
from("direct:start")
.to("log:foo")
.wireTap("direct:tap")
.to("mock:result");
This works if I run a mock example (e.g. camel-example-jms-file). However if I take the sample code and try to substitute a real Broker instance and Queue to replace the mock objects it fails with error below.
from("tcp://localhost:61616")
.to("ativemq:atsUpdateQueue")
.wireTap("activemq:fdmCaptureQueue");
Then it fails
org.apache.camel.FailedToCreateRouteException: Failed to create route route2: Route(route2)[[From[tcp://localhost:61616?queue=atsUpdateQue... because of Failed to resolve endpoint: tcp://localhost:61616?queue=atsUpdateQueue due to: No component found with scheme: tcp
I've googled extensively and all the example I've found use the virtual mock queues none seem to illustrate working with a real broker and but I cannot find any documentation on the URI specification for camel.
The important part of the error message describes the problem No component found with scheme: tcp, This is becasuse there is no "tcp" component for camel, however you can use the netty component if you want to interact with a tcp endpoint:
from("netty:tcp://localhost:61616")
more info here - http://camel.apache.org/netty.html
"tcp://localhost:61616" looks like the activemq broker address.
You need to setup the broker address to activemq component in Java DSL
camelContext.addComponent("activemq", activeMQComponent("tcp://localhost:61616"));
or in spring configuration file
<bean id="activemq"
class="org.apache.activemq.camel.component.ActiveMQComponent">
<property name="brokerURL" value="tcp://somehost:61616"/>
</bean>
You can find more information about camel-activemq here
Thank you for the suggestions, while useful in increasing my understanding neither actually resolved my problem. My code was wrong and for the benefit of others I should have been using the following names.
final String sourceQueue = "activemq:queue:atsUpdateQueue";
final String destinationQueue = "activemq:queue:atsEndPoint";
final String wiretapQueue = "activemq:queue:fdmCaptureQueue";
from(sourceQueue).wireTap(wiretapQueue).copy().to(destinationQueue);

Resources