I currently am using ServiceMix to route messages using QPID libraries.
My working config is below :
<?xml version="1.0" encoding="UTF-8"?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://www.osgi.org/xmlns/blueprint/v1.0.0
http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd">
<bean id="amqp" class="org.apache.camel.component.amqp.AMQPComponent">
<property name="connectionFactory">
<bean class="org.apache.qpid.jms.JmsConnectionFactory">
<property name="remoteURI" value="failover:amqps://remote-broker:9551?transport.trustStoreLocation=/vault/QA_UM_A.jks />
</bean>
</property>
</bean>
<bean id="kubeActivemq" class="org.apache.activemq.camel.component.ActiveMQComponent">
<property name="brokerURL" value="tcp://10.100.10.13:61616" />
</bean>
<route>
<from uri="amqp:queue:///queue-1" />
<to uri="kubeActivemq:local-queue?jmsMessageType=Text" />
</route>
</camelContext>
</blueprint>
The above config works.
Now I need to implement durable subscription so that if servicemix goes down, it is able to recover all messages when it comes back up (the ones sent when servicemix is down.)
Based on the documentation , I have implemented a route :
<route>
<from uri="amqp:queue:///queue-1?clientId=1&durableSubscriptionName=bar1" />
<to uri="kubeActivemq:queue:///local-queue" />
</route>
But this implementation gives the ERROR :
2018-05-28 13:28:58,421 | ERROR | mix-7.0.0/deploy | BlueprintCamelContext | 40 - org.apache.camel.camel-blueprint - 2.16.4 | Error occurred during starting Camel: CamelContext(camel-1) due A durable subscription requires a topic (pub-sub domain)
java.lang.IllegalArgumentException: A durable subscription requires a topic (pub-sub domain)
at org.springframework.jms.listener.AbstractMessageListenerContainer.validateConfiguration(AbstractMessageListenerContainer.java:435)
at org.springframework.jms.listener.AbstractJmsListeningContainer.afterPropertiesSet(AbstractJmsListeningContainer.java:157)
at org.apache.camel.component.jms.JmsConsumer.prepareAndStartListenerContainer(JmsConsumer.java:163)
at org.apache.camel.component.jms.JmsConsumer.doStart(JmsConsumer.java:155)
at org.apache.camel.support.ServiceSupport.start(ServiceSupport.java:61)
at org.apache.camel.impl.DefaultCamelContext.startService(DefaultCamelContext.java:3234)
at org.apache.camel.impl.DefaultCamelContext.doStartOrResumeRouteConsumers(DefaultCamelContext.java:3528)
at org.apache.camel.impl.DefaultCamelContext.doStartRouteConsumers(DefaultCamelContext.java:3464)
at org.apache.camel.impl.DefaultCamelContext.safelyStartRouteServices(DefaultCamelContext.java:3394)
at org.apache.camel.impl.DefaultCamelContext.doStartOrResumeRoutes(DefaultCamelContext.java:3162)
at org.apache.camel.impl.DefaultCamelContext.doStartCamel(DefaultCamelContext.java:3018)
.
.
.
Is there something Ive missed here ? Arent the topic names supposed to be same as queue names ?
Your route is instructing Camel to connect to a queue, and you need to change it to use a 'topic' for Durable Subscriptions. (Queue subscriptions are durable by default)
amqp:queue:///queue-1...
And Durable Subscriptions require a topic
amqp:topic:///topic-1...
Related
I am setting up a ServieMix instance using apache-camel as the routing engine, with my routes defined in a blueprint.xml. I am trying to configure ActiveMQ for my blueprint to be completely isolated from anything else (use its own, private, broker).
Here is my camel blueprint XML
<?xml version="1.0" encoding="UTF-8"?>
<blueprint
xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
xmlns:cm="http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.0.0"
>
<bean id="record_ip" class="my.service.RecordIP"/>
<bean id="jmsConnectionFactory"
class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="vm://myBroker?create=true&waitForStart=10000" />
<property name="userName" value="shadow"/>
<property name="password" value="broker"/>
</bean>
<bean id="pooledConnectionFactory"
class="org.apache.activemq.pool.PooledConnectionFactory" init-method="start" destroy-method="stop">
<property name="maxConnections" value="8" />
<property name="connectionFactory" ref="jmsConnectionFactory" />
</bean>
<bean id="jmsConfig"
class="org.apache.camel.component.jms.JmsConfiguration">
<property name="connectionFactory" ref="pooledConnectionFactory"/>
<property name="concurrentConsumers" value="10"/>
</bean>
<bean id="activemq"
class="org.apache.activemq.camel.component.ActiveMQComponent">
<property name="configuration" ref="jmsConfig"/>
</bean>
<camelContext xmlns="http://camel.apache.org/schema/blueprint">
<route id="tracing_route">
<from uri="jetty:http://localhost:9696/trace"/>
<inOnly uri="activemq:queue:ip_capture"/>
</route>
<route id="ip_capture">
<from uri="activemq:queue:ip_capture?concurrentConsumers=1&maxConcurrentConsumers=64&maxMessagesPerTask=100"/>
<bean ref="record_ip"/>
<log message="Finished!" loggingLevel="WARN" />
</route>
</camelContext>
</blueprint>
I don't think it's using the setup at all because I get the following error
Could not refresh JMS Connection for destination 'ip_capture' - retrying in 5000 ms. Cause: Error while attempting to add new Connection to the pool; nested exception is javax.jms.JMSException: Could not create Transport. Reason: java.io.IOException: Broker named 'amq-broker' does not exist.
And amq-broker is the default broker.
I'm pouring through everything I can find but something important is missing
http://camel.apache.org/activemq.html
http://activemq.apache.org/networks-of-brokers.html
http://activemq.apache.org/how-do-i-embed-a-broker-inside-a-connection.html
http://activemq.apache.org/xml-configuration.html
I can't use xmlns:amq="http://activemq.apache.org/schema/core" (ServiceMix can't resolve it)
I am using
ServiceMix 7.0.0
apache-camel/camel-blueprint 2.16.4
activemq-client/camel/blueprint 5.14.3
So long story sort, How do I properly configure ActiveMQ for my blueprint to be completely isolated from anything else?
I ended up not really needing to do this, because for all intents and purposes, the broker is independent, and setting up a new one doesn't change anything.
I already googled for a while looking for step by step tutorials explaining setup and integration of ActiveMQ with Camel, but had little success to find a basic tutorial.
I already have a running and configured ActiveMQ server but I just can't get the Camel Component up and running. It always creates a separate broker and I just can't make Camel connect to the existing broker instance.
Any hints where I can find basic tutorials on how to integrate Camel and develop a better understanding on how those two work together?
Please do not refere to Camel-Website, as this along with the docs for ActiveMQ where my primary source to fight through the stuff, but it helped only a little on setup, configuration and through understanding of both packages.
Thanks
Following is configuration of Active MQ with camel.
<bean id="jmsConnectionFactory"
class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="tcp://localhost:61616"/>
</bean>
<bean id="pooledConnectionFactory"
class="org.apache.activemq.pool.PooledConnectionFactory" init-method="start" destroy-method="stop">
<property name="maxConnections" value="8"/>
<property name="connectionFactory" ref="jmsConnectionFactory"/>
</bean>
<bean id="jmsConfig"
class="org.apache.camel.component.jms.JmsConfiguration">
<property name="connectionFactory" ref="pooledConnectionFactory"/>
<property name="concurrentConsumers" value="10"/>
</bean>
<bean id="activemq"
class="org.apache.activemq.camel.component.ActiveMQComponent">
<property name="configuration" ref="jmsConfig"/>
</bean>
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route id="*****">
<from uri="+++++++++" />
<choice>
<to uri="activemq:queue:**********"/>
</choice>
</route>
</camelContext>
You can consider some of the Camel books which has extensive information about all sorts of stuff with Camel, also how to use it with JMS and AMQ.
http://camel.apache.org/books
And there is some examples from AMQ you can look at as well: https://github.com/apache/camel/tree/master/examples
And you can try to find some 3rd party blogs or articles that cover AMQ with Camel: http://camel.apache.org/articles
And recently there was this article about AMQ with Camel:
http://www.puretechy.com/blog/apache-camel-activemq-example
I am using activemq with camel for consuming messages from a queue and send them to a http server. I am using following camel configuration :-
<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="activemq:queue:Consumer.A.VirtualTopic.Orders"/>
<to uri="http://localhost:8080/" />
<!-- <to uri="file:///Users/vinod/activemq.txt"/> -->
<!-- <to uri="activemq:queue:sssss"/> -->
</route>
</camelContext>
<bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent" >
<property name="connectionFactory">
<bean class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="vm://localhost?create=false"/>
</bean>
</property>
</bean>
In first block creates a route which consumer messages from activemq:queue:Consumer.A.VirtualTopic.Orders queue and send them to server at http://localhost:8080/. The other two commented destinations are working fine, but when I start activemq with above configuration for sending messages over http, the server stops without throwing any error message. Activemq log for this is https://gist.github.com/kumar003vinod/1e5944cb246edb74c47fef7a0b433387
Please provide some insight.
Make sure to include camel-http and camel-http-common JARs in the ActiveMQ lib/camel directory. You may also need to include the transitive dependencies from camel-http in that directory so ActiveMQ has all the needed JARs in its classpath.
That would be commons-httpclient and commons-codec JARs but I think they are already included in lib/optional.
So I'm trying to use apache servicemix/camel to aggregate some messages coming through a JMS queue. The logic I have is dead-simple I just want it to use the last message received and only send it through 3 seconds after the last one.
I have servicemix setup as a message broker and I'm able to use it in such a capacity however it doesn't seem to trigger the route. I've never done this before so odds are I'm horribly off-base but here is what I have so far (put this in deploy/fedora-messaging/camel-context.xml)
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:camel="http://camel.apache.org/schema/spring"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.5.xsd
http://camel.apache.org/schema/spring
http://camel.apache.org/schema/spring/camel-spring.xsd">
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="activemq:topic:fedora.apim.update.merge"/>
<aggregate completionTimeout="3000">
<correlationExpression>
<simple>header.pid</simple>
</correlationExpression>
<to uri="log:events"/>
<to uri="activemq:topic:fedora.apim.update"/>
</aggregate>
</route>
</camelContext>
<bean id="activemq"
class="org.apache.activemq.camel.component.ActiveMQComponent">
<property name="brokerURL" value="tcp://localhost:61616"/>
</bean>
</beans>
The osgi application appears to start and run without error, and messages are received and queued in the topic:fedora.apim.update.merge however they just sit in the queue and never get consumed or pushed out to the destination queue.
I tested your route as a standalone application (outside of a ServiceMix container) with following broker configuration and everything worked as expected:
<broker xmlns="http://activemq.apache.org/schema/core" useJmx="true"
persistent="false">
<transportConnectors>
<transportConnector uri="tcp://localhost:61616" />
</transportConnectors>
</broker>
So, I guess, we can assume that your Camel route definition is not the problem but the ActiveMQ configuration and/or ServiceMix set-up.
EDIT:
Note, that messages to JMS topic destinations are lost if no active subscriber is connected to the destination. If you need some kind of persistence use durable subscribers or JMS queue destinations.
I am trying to determine the best way to combine message streams from two hornetq broker instances into a single stream for processing, using Apache Camel and Spring. This is essentially the opposite of the Camel reciepient list pattern; but instead of one to many I need many to one. One idea is to implement this functionality with the direct component:
<?xml version="1.0" encoding="UTF-8"/>
<beans xmlns="..."
xmlns="...">
<!-- JMS Connection 1 -->
<bean id="jndiTemplate1" class="org.springframework.jndi.JndiTemplate">
<property name="environment">
<props>
...Connection 1 Specific Information...
</props>
</property>
</bean>
<bean id="jmsTopicConnectionFactory1"
class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiTemplate">
<ref bean="jndiTemplate1"/>
</property>
<property name="jndiName">
<value>java:jms/RemoteConnectionFactory</value>
</property>
</bean>
<bean id="jms1" class="org.apache.camel.component.jms.JmsComponent">
<property name="connectionFactory" ref="jmsTopicConnectionFactory1"/>
</bean>
<!-- JMS Connection 2 -->
<bean id="jndiTemplate2" class="org.springframework.jndi.JndiTemplate">
<property name="environment">
<props>
...Connection 2 Specific Information...
</props>
</property>
</bean>
<bean id="jmsTopicConnectionFactory2"
class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiTemplate">
<ref bean="jndiTemplate2"/>
</property>
<property name="jndiName">
<value>java:jms/RemoteConnectionFactory</value>
</property>
</bean>
<bean id="jms2" class="org.apache.camel.component.jms.JmsComponent">
<property name="connectionFactory" ref="jmsTopicConnectionFactory2"/>
</bean>
<!-- Camel route many to 1 using direct component -->
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route id="hornetQ_broker_1">
<from uri="jms1:topic:testTopic1">
<to uri="direct:process_message">
</route>
<route id="hornetQ_broker_2">
<from uri="jms2:topic:testTopic2">
<to uri="direct:process_message">
</route>
<route id="message_processor">
<from uri="direct:process_message">
<log message="message_processor received message">
</route>
</camelContext>
</beans>
Question: Is the above approach recommended when a many->1 integration pattern is required? If multiple Apache Camel solutions exist, what are the key performance impacts of each approach?
Runtime environment:
HornetQ brokers are JBoss EAP6.
Camel context deployed to FuseSource 4.4.1
Each entity exists on a seperate server/jvm.
Notes:
The hornetQ broker instances cannot be clustered.
The hornetQ broker instances do not contain duplicate data.
I think that your approach is valid for your scenario. However, maybe direct is not the component you need to use for this if you are running in different JVMs.
There are different components for internal queues: Direct, Direct-VM, SEDA, VM, Disruptor... but I believe all of them are if you are running in the JVM (and some of the if you just running in the same CamelContext). For more info: How do the direct, event, seda and vm endpoints compare
If you are going to have different CamelContexts across different JVM will need to use a different component.