I am trying to create a durable subscriber using polling consumer.
The URI is correct as same uri is working when used in camel route and durable subscriber is correctly created.
For some reason PollingConsumer is not able to create durable subscriber and instead creates normal subscriber.
Is it not possible to create durable subscribers using polling consumer?
public class OutQWaitingProcesser implements Processor {
#Override
public void process(Exchange exchange) throws Exception {
Endpoint endpoint = exchange.getContext().getEndpoint("jms:topic:response?clientId=5&durableSubscriptionName=abcd");
PollingConsumer consumer = endpoint.createPollingConsumer();
consumer.start();
Exchange exchange2 = consumer.receive();
String body = exchange2.getIn().getBody(String.class);
exchange.getIn().setBody(body);
consumer.stop();
}
}
Camel JmsPollingConsumer is based on Spring JMSTemplate which doesn't support to set the durableSubscription option.
Related
What is the proper way to use endpoint DSL and then reference the endpoint with ProducerTemplate? When creating a route and using endpoint DSL, it seems that Camel is creating a different uri for the endpoint. My EndpointRouteBuilder class:
#Component
public class MyRoutes extends EndpointRouteBuilder {
#Override
public void configure() throws Exception {
from(seda("STATUS_ENDPOINT"))
.routeId("stateChangeRoute")
.to(activemq("topic:statusTopic"))
}
}
and then injecting the endpoint to ProducerTemplate
#Component
public class StateChangePublisher {
#EndpointInject(value="seda:STATUS_ENDPOINT")
private ProducerTemplate producer;
public void publish(String str) {
try {
producer.sendBody(str);
} catch(CamelExecutionException e) {
e.printStackTrace();
}
}
}
When camel starts, I see two entries in the log:
o.a.camel.component.seda.SedaEndpoint : Endpoint seda:STATUS_ENDPOINT is using shared queue: seda:STATUS_ENDPOINT with size: 1000
o.a.camel.component.seda.SedaEndpoint : Endpoint seda://STATUS_ENDPOINT is using shared queue: seda://STATUS_ENDPOINT with size: 1000
The queue eventually fills up and nothing gets delivered to the "to" endpoint.
If I define the route without using the endpoint DSL method "seda()"
from("seda:STATUS_ENDPOINT")
then it works.
Is this a bug or am I doing something wrong?
I'm using camel 3.2.0 and
This was a bug in the endpoint dsl. Try upgrading to camel 3.3.0. I think it was fixed in the new release.
https://issues.apache.org/jira/browse/CAMEL-14859
I am trying to find the fastest way using Camel to transfer messages from one ActiveMQ Artemis queue to another. I thought that Camel's SJMS2 component would be faster than Camel's traditional JMS component, but routing with the JMS component is 2.5 times faster (20,000 vs 8,000 msg/s). I use Camel version 2.20.2 and Artemis version 2.11.0.
Route with JMS
import org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.impl.JndiRegistry;
import org.apache.camel.test.junit4.CamelTestSupport;
import org.junit.Test;
import org.messaginghub.pooled.jms.JmsPoolConnectionFactory;
import javax.jms.ConnectionFactory;
import java.util.concurrent.TimeUnit;
public class JMSTransferTest extends CamelTestSupport {
#Test
public void testArtemis() throws Exception {
TimeUnit.SECONDS.sleep(100);
}
#Override
protected RouteBuilder createRouteBuilder() throws Exception {
return new RouteBuilder() {
public void configure() {
from("jms://TEST.IN?connectionFactory=#artemisCF&concurrentConsumers=200")
.to("jms://TEST.OUT?connectionFactory=#artemisCF");
}
};
}
#Override
protected JndiRegistry createRegistry() throws Exception {
JndiRegistry registry = super.createRegistry();
final ConnectionFactory connFactory = new ActiveMQConnectionFactory("tcp://localhost:61622");
final ConnectionFactory connFactoryDeadLeatter = new ActiveMQConnectionFactory("tcp://localhost:61622");
JmsPoolConnectionFactory pooledConnectionFactory = new JmsPoolConnectionFactory();
pooledConnectionFactory.setConnectionFactory(connFactory);
pooledConnectionFactory.setMaxConnections(20);
pooledConnectionFactory.setMaxSessionsPerConnection(100);
registry.bind("artemisCF", pooledConnectionFactory);
registry.bind("deadLetterCF", connFactoryDeadLeatter);
return registry;
}
}
Route with SJMS2, other settings as in the code above
#Override
protected RouteBuilder createRouteBuilder() throws Exception {
return new RouteBuilder() {
public void configure() {
from("sjms2://TEST.IN?connectionFactory=#artemisCF&consumerCount=200&asyncStartListener=true")
.to("sjms2://TEST.OUT?connectionFactory=#artemisCF");
}
};
}
How can I use the SJMS2 component to get the same speeds as JMS component?
Claus Ibsen replied on the mailing list as follows
200 consumers is too much. That instead makes it slower as you have
200 consumers racing for messages. Instead try to find a lower balance
that is closer to cpu cores etc.
Also often a JMS client has a prefetch buffer (or some concept like
this) which means a consumer may pre-download 1000 messages and then
the other 199 consumers cant process these messages. So you need to
tweak this option too.
Also if you have too many consumers and remote network connections
then you get too chatty over IO etc. So its all about tuning depending
on use-cases.
spring-jms has a thread pool built in that can automatic grow/shrink
depending on load, and this can explain why its out of the box without
tuning can appear to be faster.
Writing such logic is a bit more complex and this hasnt been added to
sjms. I created a ticket about this
https://issues.apache.org/jira/browse/CAMEL-14637
You can get in touch with commercial Camel supports as there are
companies and consultants that has great experience with JMS brokers
and Camel and to get them tuned to very high performance. The settings
for JVM and OS and hardware can all make a big difference.
And I also found a good article on this topic https://dzone.com/articles/performance-tuning-ideas-for-apache-camel
I'm want to use the n-tier architecture for an application so the client tier, web tier, business tier and data tier is separate. I'm wondering how a message driven bean which has a message save it to the database without changing the architecture. (That is using a normal session bean i retrieved data entered through a JSP page to a servlet and from the servlet called the bean class which had operations to the database, it is not possible to do this with the message driven beans since it already has a overridden method onMessage)
So far i can retrieve the values from the servlet directly using the message bean but i want to change this to a 4-tier architecture where database operations are not in the servlet.
my servlet code
#Resource(mappedName = "jms/dest")
private Queue dest;
#Resource(mappedName = "jms/queue")
private ConnectionFactory queue;
protected void doGet(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
response.setContentType("text/html;charset=UTF-8");
String str = request.getParameter("message");
try {
sendJMSMessageToDest(str);
} catch (JMSException ex) {
}
private Message createJMSMessageForjmsDest(Session session, Object messageData) throws JMSException{
TextMessage tm = session.createTextMessage();
tm.setText(messageData.toString());
return tm;
}
private void sendJMSMessageToDest(Object messageData) throws JMSException{
Connection connection = null;
Session session = null;
try {
connection = queue.createConnection();
session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
MessageProducer messageProducer = session.createProducer(dest);
messageProducer.send(createJMSMessageForjmsDest(session,messageData));
} catch (JMSException ex) {
}
}
You must think in two possible workflows:
Synchronous interaction.
Asynchronous interaction.
Below I draw a possible architecture wich covers boot workflows. The components are:
DAO: Data Access Object layer. This is responsible of persist and query database without business logic. Implemented with Stateless Session Beans
BL: Business Logic layer. This is responsible of process Business Logic, don't know where the data will be persisted or queried just invokes DAO layer. Also is independent of View layer (Web, Web Service, Rest, etc.).
Serlvet: In this case represents the view layer or web interaction with the user calls directly to BL for process de data retrieved.
MDB: This layer is for asynchronous events it dequeues messages from a Queue (or Topic) then call to BL layer for process the data retrieved.
This architecture enables code reutilization and separation of responsibilities.
Theres is the Diagram with two workflows.
Syncrhonous workflow:
Servlet call BL.
BL call DAO.
DAO inteacts with Database
Asyncrhonous workflow:
i. Servlet enqueues message A,B,C
ii. MDB dequeue A
iii. MDB call BL.
iv. BL call DAO.
v. DAO interacts with Database
From a programming point of view, I have a very simple business case. However, I can't figure out how to implement it using Apache Camel... Well, I have 2 JMS queues: one to receive commands, another - to store large number of message which should be delivered to external system in a batches of 1000 or less.
Here is the concept message exchange algorithm:
upon receiving a command message in 1st JMS queue I prepare XML
message
Send the XML message to external SOAP Web Service to obtain a usertoken
Using the usertoken, prepare another XML message and send it to a REST service to obtain jobToken
loop:
4.1. aggregate messages from 2nd JMS queue in batches of 1000, stop aggregation at timeout
4.2. for every batch, convert it to CSV file
4.3. send csv via HTTP Post to a REST service
4.4. retain batchtoken assigned to each batch
using the jobtoken prepare XML message and send to REST service to commit the batches
using batchtoken check execution status of each batch via XML message to REST service
While looking at Camel I could create a sample project where I can model out the exchange 1-3, 5:
from("file:src/data?noop=true")
.setHeader("sfUsername", constant("a#fd.com"))
.setHeader("sfPwd", constant("12345"))
.to("velocity:com/eip/vm/bulkPreLogin.vm?contentCache=false")
.setHeader(Exchange.CONTENT_TYPE, constant("text/xml; charset=UTF-8"))
.setHeader("SOAPAction", constant("login"))
.setHeader("CamelHttpMethod", constant("POST"))
.to("http4://bulklogin") // send login
.to("xslt:com/eip/xslt/bulkLogin.xsl") //xslt transformation to retrieve userToken
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
String body = (String) exchange.getIn().getBody();
String[] bodyParts = body.split(",");
exchange.getProperties().put("userToken", bodyParts[0]);
.....
}
})
.to("velocity:com/eip/vm/jobInsertTeamOppStart.vm")
.setHeader(Exchange.CONTENT_TYPE, constant("application/xml; charset=UTF-8"))
.setHeader("X-Session", property("userToken"))
.setHeader("CamelHttpMethod", constant("POST"))
.to("http4://scheduleJob") //schedule job
.to("xslt:com//eip/xslt/jobInfoTransform.xsl")
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
String body = (String) exchange.getIn().getBody();
exchange.getProperties().put("jobToken",body.trim());
}
})
//add batches in a loop ???
.to("velocity:com/eip/vm/jobInsertTeamOppEnd.vm")
.setHeader(Exchange.HTTP_URI, simple("https://na15.com/services/async/job/${property.jobToken}"))
.setHeader(Exchange.CONTENT_TYPE, constant("application/xml; charset=UTF-8"))
.setHeader("X-ID-Session", property("userToken"))
.setHeader("CamelHttpMethod", constant("POST"))
.to("http4://closeJob") //schedule job
//check batch?
.bean(new SomeBean());
So, my question is:
How can I read messages from my 2nd JMS queue?
This doesn't strike me as a very good use-case for a single camel route. I think you should implement the main functionality in a POJO and use Camels Bean Integration for consuming and producing messages. This will result in much more easy to maintain code, and also for easier Exception handling.
See https://camel.apache.org/pojo-consuming.html
I need a way to consume messages from multiple activemq jms queues.
As per activemq documentation, it supports wildcard consumers
I am using camel as a messaging bus.Is it possible to look at below named queues
aaa.processQueue
bbb.processQueue
ccc.processQueue
By configuring camel route to look at activemq:*.processQueue endpoint?
Also let me know, if there is more cleaner alternative for this.
Yes. It should be doable as Camel is using the OpenWire/JMS client.
Your options are:
from("activemq:*.processQueue")
from("activemq:aaa.processQueue,bbb.processQueue,ccc.processQueue")
Multiple routes with a sub route for logic:
from("activemq:aaa.processQueue").to("direct:doProcess");
from("activemq:bbb.processQueue").to("direct:doProcess");
from("activemq:ccc.processQueue").to("direct:doProcess");
from("direct:doProcess").whatever..
This way, you can easily turn on/off routes as well as assigning more consumers to one, given you need to have more priority on aaa.processQueue messages than the rest.
They have an example on their github of a routebuilder using wildcards:
protected RouteBuilder createRouteBuilder() throws Exception {
return new RouteBuilder() {
public void configure() throws Exception {
// use wildcard to consume from all sports
from("activemq:queue:sport.>")
.to("log:received?showHeaders=true")
.choice()
// the JMSDestination contains from which queue the message was consumed from
.when(header("JMSDestination").isEqualTo("queue://sport.pl.chelsea"))
.to("mock:chelsea")
// we can use a reg exp to match any message from 1st division
.when(header("JMSDestination").regex("queue://sport.1st.*"))
.to("mock:1st")
.otherwise()
.to("mock:other")
.end();
}
};
}
Ref: https://github.com/apache/camel/blob/master/components/camel-jms/src/test/java/org/apache/camel/component/jms/activemq/ActiveMQConsumeWildcardQueuesTest.java