I need a way to consume messages from multiple activemq jms queues.
As per activemq documentation, it supports wildcard consumers
I am using camel as a messaging bus.Is it possible to look at below named queues
aaa.processQueue
bbb.processQueue
ccc.processQueue
By configuring camel route to look at activemq:*.processQueue endpoint?
Also let me know, if there is more cleaner alternative for this.
Yes. It should be doable as Camel is using the OpenWire/JMS client.
Your options are:
from("activemq:*.processQueue")
from("activemq:aaa.processQueue,bbb.processQueue,ccc.processQueue")
Multiple routes with a sub route for logic:
from("activemq:aaa.processQueue").to("direct:doProcess");
from("activemq:bbb.processQueue").to("direct:doProcess");
from("activemq:ccc.processQueue").to("direct:doProcess");
from("direct:doProcess").whatever..
This way, you can easily turn on/off routes as well as assigning more consumers to one, given you need to have more priority on aaa.processQueue messages than the rest.
They have an example on their github of a routebuilder using wildcards:
protected RouteBuilder createRouteBuilder() throws Exception {
return new RouteBuilder() {
public void configure() throws Exception {
// use wildcard to consume from all sports
from("activemq:queue:sport.>")
.to("log:received?showHeaders=true")
.choice()
// the JMSDestination contains from which queue the message was consumed from
.when(header("JMSDestination").isEqualTo("queue://sport.pl.chelsea"))
.to("mock:chelsea")
// we can use a reg exp to match any message from 1st division
.when(header("JMSDestination").regex("queue://sport.1st.*"))
.to("mock:1st")
.otherwise()
.to("mock:other")
.end();
}
};
}
Ref: https://github.com/apache/camel/blob/master/components/camel-jms/src/test/java/org/apache/camel/component/jms/activemq/ActiveMQConsumeWildcardQueuesTest.java
Related
Requisite disclaimer about being new to Camel--and, frankly, new to developing generally. I'd like to have a string generated as the output of some function be the source of my camel route which then gets written to some file. It's the first part that seems challenging: I have a string, how do I turn it into a message? I can't write it into a file nor can I use JMS. I feel like it should be easy and obvious, but I'm having a hard time finding a simple guide to help.
Some pseudo-code using the Java DSL:
def DesiredString() {return "MyString";}
// A camel route to be implemented elsewhere; I want something like:
class MyRoute() extends RouteBuilder {
source(DesiredString())
.to("file://C:/out/?fileName=MyFileFromString.txt");
}
I vaguely understand using the bean component, but I'm not sure that solves the problem: I can execute my method that generates the string, but how do I turn that into a message? The "vague" is doing a lot of work there: I could be missing something there.
Thanks!
Not sure if I understand your problem. There is a bit of confusion about what the String should be become: the route source or the message body.
However, I guess that you want to write the String returned by your method into a File through a Camel route.
If this is correct, I have to clarify first the route source. A Camel Route normally starts with
from(component:address)
So if you want to receive requests from remote via HTTP it could be
from("http4:localhost:8080")
This creates an HTTP server that listens on port 8080 for messages.
In your case I don't know if the method that returns the String is in the same application as the Camel route. If it is, you can use the Direct component for "method-like" calls in the same process.
from(direct:input)
.to("file:...");
input is a name you can freely choose. You can then route messages to this route from another Camel route or with a ProducerTemplate
ProducerTemplate template = camelContext.createProducerTemplate();
template.sendBody("direct:input", "This is my string");
The sendBody method takes the endpoint where to send the message and the message body. But there are much more variants of sendBody with different signatures depending on what you want to send it (headers etc).
If you want to dive into Camel get a copy of Camel in Action 2nd edition. It contains everything you need to know about Camel.
Example:Sending String(as a body content)to store in file using camel Java DSL:
CamelContext context = new DefaultCamelContext();
context.addRoutes(new RouteBuilder() {
public void configure() {
from("timer:StringSentToFile?period=2000")
.setBody(simple(DesiredString()))
.to("file:file://C:/out/?fileName=MyFileFromString.txt&noop=true")
.log("completed route");
}
});
ProducerTemplate template = context.createProducerTemplate();
context.start();
I would like to write a Camel Route that gets in a URI (can be http, ftp, file, ...) and then fetches the data and stores it locally in a file.
This URI-String could be, for example:
"ftp://localhost/example.txt"
"file://tmp/example.txt"
"jms:queue:dataInputQueue"
...
Based on this string, the correct Camel Component should be used to access the data. Something like a case/switch in Java:
(1) Receive URI (from uri="vm:incomingUri")
(2) Chose "right" Camel Component
switch(URI)
case HTTP: use Camel HTTP component
case FTP: use Camel FTP component
case JMS: use Camel JMS component
...
(3) Read data from that URI, using the "right" Camel component
(4) Store file locally (to uri="file://...)
Example:
From "vm:incomingUri" I read a String "ftp://localhost/example.txt". That what finally needs to happen now should be equivalent to this:
<route>
<from uri="ftp://localhost/example.txt"/>
<to uri="file://tmpDir/example.txt"/>
</route>
How would this look like in Camel?
I believe one difficulty will be that, for the components you mention (HTTP, FTP, file, JMS), you may want to use either a producer or a consumer:
FTP, File: definitely a consumer to read a file.
HTTP (or HTTP4): definitely a producer, to send a request to the server (the server's reply will by the new message body)
JMS: depends on wether you want to read from a queue (consumer), or send a message to a queue with a ReplyTo header, then wait for the answer (producer).
Producers :
If you are using Camel 2.16+, you can use the new "dynamic to" syntax. It's basically the same as a regular "to", except that the endpoint uri can be evaluated dynamically using a simple expression (or, optionnaly, another type of expression). Alternatively, you can use the enrich flavor of the content-enricher pattern, wich also supports dynamic uris starting with Camel 2.16.
If you are using an older version of Camel, or if you need to dynamically route to several endpoints (not just one), you can use the recipient list pattern.
Here's an exemple. We will transform the message body by calling an endpoint; the uri for that endpoint will be found in a header named TargetUri and will be evaluated dynamically for each message.
// An instance of this class is registered as 'testbean' in the registry. Instead of
// sending to this bean, I could send to a FTP or HTTP endpoint, or whatever.
public class TestBean {
public String toUpperCase(final String str) {
return str.toUpperCase();
}
}
// This route sends a message to our example route for testing purpose. Of course, we
// could send any message as long as the 'TargetUri' header contains a valid endpoint uri
from("file:inbox?move=done&moveFailed=failed")
.setHeader("TargetUri").constant("bean:testbean?method=toUpperCase")
.setBody().constant("foo")
.to("direct:test");
// 1. The toD example :
from("direct:test")
.toD("${header.TargetUri}")
.to("log:myRoute");
// 2. The recipient list example :
from("direct:test")
.recipientList(header("TargetUri"))
.to("log:myRoute");
// 3. The enrich example :
from("direct:test")
.enrich().simple("${header.TargetUri}") // add an AggregationStrategy if necessary
.to("log:myRoute");
Consumers :
With Camel 2.16+, you can use the pollEnrich flavor of the content-enricher pattern.
For older versions of Camel, you can use a ConsumerTemplate in a processor.
// 4. The pollEnrich example (assuming the TargetUri header contains, e.g., a file
// or ftp uri) :
from("direct:test")
.pollEnrich().simple("${header.TargetUri}") // add an AggregationStrategy if necessary
.to("log:myRoute");
// 5. The ConsumerTemplate example (same assumption as above)
from("direct:test")
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
String uri = exchange.getIn().getHeader("TargetUri", String.class);
ConsumerTemplate consumer = exchange.getContext().createConsumerTemplate();
final Object data = consumer.receiveBody(uri);
exchange.getIn().setBody(data);
}
})
.to("log:myRoute");
Producer or consumer?
Sadly, I can't think of any really elegant solution to handle both - I think you will have to route to two branches based on the uri and known components... Here's the sort of thing I might do (with Camel 2.16+), it's not very pretty:
// This example only handles http and ftp endpoints properly
from("direct:test")
.choice()
.when(header("TargetUri").startsWith("http"))
.enrich().simple("${header.TargetUri}")
.endChoice()
.when(header("TargetUri").startsWith("ftp"))
.pollEnrich().simple("${header.TargetUri}")
.endChoice()
.end()
.to("log:myRoute");
It is possible by using
<to uri="{{some.endpoint}}"/>
But you would require to add it in property .
<cm:property name="some.endpoint" value="SomeEndPoint"/>
And you can add any endpoint you want http, ftp, file, log, jms, vm etc.
Value of SomeEndPoint.
Log Component: log:mock
JMS Component: activemq:someQueueName
File Component: file://someFileShare
VMComponent: vm:toSomeRoute
Hope this doesn't sound ridiculous, but how can I discard a message in Camel on purpose?
Until now, I sent them to the Log-Component, but meanwhile I don't even want to log the withdrawal.
Is there a /dev/null Endpoint in Camel?
You can use the message filter eip to filter out unwanted messages.
http://camel.apache.org/message-filter
There is no dev/null, component.
Also there is a < stop /> you can use in the route, and when a message hit that, it will stop continue routing.
And the closest we got on a dev/null, is to route to a log, where you set logLeve=OFF as option.
With credit to my colleague (code name: cayha)...
You can use the Stub Component as a camel endpoint that is equivalent to /dev/null.
e.g.
activemq:route?abc=xyz
becomes
stub:activemq:route?abc=xyz
Although I am not aware of the inner workings of this component (and if there are dangers for memory leaks, etc), it works for me and I can see no drawbacks in doing it this way.
one can put uri/mock-uri to the config using property component
<camelContext ...>
<propertyPlaceholder id="properties" location="ref:myProperties"/>
</camelContext>
// properties
cool.end=mock:result
# cool.end=result
// route
from("direct:start").to("properties:{{cool.end}}");
I'm a little late to the party but you can set a flag on the exchange and use that flag to skip only that message (by calling stop) if it doesn't meet your conditions.
#Override
public void configure() throws Exception {
from()
.process(new Processor() {
#SuppressWarnings("unchecked")
#Override
public void process(Exchange exchange) throws Exception {
exchange.setProperty("skip", false);
byte[] messageBytes = exchange.getIn().getBody(byte[].class);
if (<shouldNotSkip>) {
} else { //skip
exchange.setProperty("skip", true);
}
}
}).choice()
.when(exchangeProperty("skip").isEqualTo(true))
.stop()
.otherwise()
.to();
}
I am using activemq route and needs to send reply in normal cases, so exchange pattern is InOut. When I configure a filter in the route I find that even it does not pass message to next step, the callback is executed(sending reply), just same as the behavior when calling stop(). And it will send the same message back to reply queue, which is not desirable.
What I do is to change the exchange pattern to InOnly conditionally and stop if I want to filter out the message, so reply is not sent. MAIN_ENDPOINT is a direct:main endpoint I defined to include normal business logic.
from("activemq:queue:myqueue" + "?replyToSameDestinationAllowed=true")
.log(LoggingLevel.INFO, "Correlation id is: ${header.JMSCorrelationID}; will ignore if not null")
.choice()
.when(simple("${header.JMSCorrelationID} == null"))
.to(MAIN_ENDPOINT)
.endChoice()
.otherwise()
.setExchangePattern(ExchangePattern.InOnly)
.stop()
.endChoice()
.end();
Note that this message is also consumed and not in the queue anymore. If you want to preserve the message in the queue(not consuming it), you may just stop() or just filter() so the callback(sending reply which is the original message) works, putting the message back to the queue.
Using only filter() would be much simpler:
from("activemq:queue:myqueue" + "?replyToSameDestinationAllowed=true")
.log(LoggingLevel.INFO, "Correlation id is: ${header.JMSCorrelationID}; will ignore if not null")
.filter(simple("${header.JMSCorrelationID} == null"))
.to(MAIN_ENDPOINT);
I'm working on a camel prototype which uses two start points in the same camel context.
The first route consumes messages which are used to "configure" the application. Messages are loaded in a configuration repository through a configService bean:
// read configuration files
from("file:data/config?noop=true&include=.*.xml")
.startupOrder(1)
.to("bean:configService?method=loadConfiguration")
.log("Configuration loaded");
The second route implements a recipient list eip pattern, delivering a different kind of input messages to a number of recipients, which are read dinamically from the same configuration repository:
// process some source files (using configuration)
from("file:data/source?noop=true")
.startupOrder(2)
.unmarshal()
.to("setupProcessor") // set "recipients" header
.recipientList(header("recipients"))
// ...
The question that arises now is how to synchronize them, so the second route "waits" if the first is processing new data.
I'm new to Apache Camel and pretty lost on how to approach such a problem, any suggestion would be appreciated.
Use aggregate in combination with the possibility to start and stop routes dynamically:
from("file:data/config?noop=true&include=.*.xml")
.id("route-config")
.aggregate(constant(true), new MyAggregationStrategy()).completionSize(2).completionTimeout(2000)
.process(new Processor() {
#Override
public void process(final Exchange exchange) throws Exception {
exchange.getContext().startRoute("route-source");
}
});
from("file:data/source?noop=true&idempotent=false")
.id("route-source") // the id is needed so that the route is found by the start and stop processors
.autoStartup(false) // this route is only started at runtime
.aggregate(constant(true), new MyAggregationStrategy()).completionSize(2).completionTimeout(2000)
.setHeader("recipients", constant("direct:end")) // this would be done in a separate processor
.recipientList(header("recipients"))
.to("seda:shutdown"); // shutdown asynchronously or the route would be waiting for pending exchanges
from("seda:shutdown")
.process(new Processor() {
#Override
public void process(final Exchange exchange) throws Exception {
exchange.getContext().stopRoute("route-source");
}
});
from("direct:end")
.log("End");
That way, route-source is only started when route-config is completed. route-config and consequently route-source are restarted if new files are found in the config directory.
You can also place an "on completion" http://camel.apache.org/oncompletion.html in the first route that activates the second one.
Apache camel File will create a lock for the file that being processed. Any other File process on this file will not pool on if there is a lock (except if you put consumer.exclusiveReadLock=false)
source :
http://camel.apache.org/file.html => URI Options => consumer.exclusiveReadLock
Is there something like "browse" option (see section 2.4.3.3. browse for details) for amqp endpoint?
There is a "browse" component. to("browse:dummy"), but that will only make it possible to browse through messages that has passed this route. Handy in some cases, but not really as the JMS browse option.
What you can do is to use a BrowsableEndpoint to do a JMS browse (should work with AMQP as well, as it's based on a JMS client, haven't tried though).
You can't really receive a message and not delete it though, so you would need something else to trigger your browsing. Such as a timer or a trigger queue.
from("amqp:queue:trigger")
.process(new Processor(){
#Override
public void process(Exchange arg0) throws Exception {
BrowsableEndpoint browse = arg0.getContext().getEndpoint("amqp:queue:archive", BrowsableEndpoint.class);
List<Exchange> exchanges = browse.getExchanges();
System.out.println("Browsing queue: "+ browse.getEndpointUri() + " size: " + exchanges.size());
for (Exchange exchange : exchanges) {
String payload = exchange.getIn().getBody(String.class);
String msgId = exchange.getIn().getHeader("JMSMessageID", String.class);
System.out.println(msgId + "=" +payload);
}
}
});
You could do a pre-route to achive this trigger.
from("amqp:queue:processQueue")
.to("amqp:queue:archive")
.transform().constant("trigger msg")
.to("amqp:queue:trigger");
I figured out it eventually, I just thought that the option should be there on camel component options level, i.e. after connection string in form of e.g. "?mode=browse", but the option is actually to be set on amqp connection level - "my-queue; {mode: browse}". You can set camel component options adding "?option=value".
P.S. Setting the option on amqp connection level works for even-driven (default camel) routing, it does not work however for polling-driven one, see https://issues.apache.org/jira/browse/CAMEL-6784 for details.
Yes. In AMQP 0-10 there is an acquire mode option on message.subscribe which if set tonot-acquired results in a browsing subscription.In AMQP 1.0 the same is achieved by specifying a distribution mode of 'copy' for the source when establishing a subscriber link.
However, assuming you are accessing AMQP via a JMS client then as Petter says above, you should be able to use the browsing facility as defined by JMS I would imagine (I'm not familiar enough with camel to know how that is done).