I'm trying to use the RecipientList pattern in Camel but I think I may be missing the point. The following code only displays one entry to the screen:
#Override
protected RouteBuilder createRouteBuilder() {
return new RouteBuilder() {
public void configure() {
from("direct:start").recipientList(bean(MyBean.class, "buildEndpoint"))
.streaming()
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
System.out.println(exchange.getExchangeId());
}
});
}
};
}
public static class MyBean {
public static String[] buildEndpoint() {
return new String[] { "exec:ls?args=-la", "exec:find?args=."};
}
}
I also tried just returning a comma-delimited string from the buildEndpoint() method and using tokenize(",") in the expression of the recipientList() component definition but I still got the same result. What am I missing?
That is expected, the recipient list sends a copy of the same message to X recipients. The processor you do afterwards is doing after the recipient lists is done, and therefore is only executed once.
Related
In our application we are using Apache Camel with camel-cdi component in JBoss EAP 7.1 environment. After upgrade of Apache Camel to actual version the application started to behave incorrectly in parallel execution.
I have found, that bean component invokes always the same instance. From my understanding, bean with #Dependent scope should be always fresh instance for every CDI lookup.
I have tried endpoint parameter cache=false, which should be default, but the behavior stays the same. Also tried to specify #Dependent, which should be default too.
Attaching MCVE, which fails on Apache Camel 2.20.0 and newer. Works well with 2.19.5 and older. Full reproducible project on Github.
#ApplicationScoped
#Startup
#ContextName("cdi-context")
public class MainRouteBuilder extends RouteBuilder {
public void configure() throws Exception {
from("timer:test")
.to("bean:someDependentBean?cache=false");
}
}
#Named
//#Dependent //Dependent is default
public class SomeDependentBean implements Processor {
private int numOfInvocations = 0;
private static Logger log = LoggerFactory.getLogger(SomeDependentBean.class);
public void process(Exchange exchange) throws Exception {
log.info("This is: "+toString());
numOfInvocations++;
if (numOfInvocations!=1){
throw new IllegalStateException(numOfInvocations+"!=1");
} else {
log.info("OK");
}
}
}
Is there anything I can do in our application to change this behavior and use actual version of Apache Camel?
EDIT:
Removing tags camel-cdi and jboss-weld. I have created unit test, to simulate this situation without dependencies to camel-cdi and Weld. This test contains assertion to test JndiRegistry#lookup, which returns correct instance. According this test I believe, the issue is in bean component itself. Fails with version >=2.20.0 and passes with <=2.19.5
public class CamelDependentTest extends CamelTestSupport {
private Context context;
private JndiRegistry registry;
#Override
protected RoutesBuilder createRouteBuilder() throws Exception {
return new RouteBuilder() {
#Override
public void configure() throws Exception {
from("direct:in")
.to("bean:something?cache=false");
}
};
}
#Override
protected JndiRegistry createRegistry() throws Exception {
JndiRegistry registry = super.createRegistry();
registry.bind("something", new SomeDependentBean());
this.context = registry.getContext();
this.registry = registry;
return registry;
}
#Test
public void testFreshBeanInContext() throws Exception{
SomeDependentBean originalInstance = registry.lookup("something", SomeDependentBean.class);
template.sendBody("direct:in",null);
context.unbind("something");
context.bind("something", new SomeDependentBean()); //Bind new instance to Context
Assert.assertNotSame(registry.lookup("something"), originalInstance); //Passes, the issue is not in JndiRegistry.
template.sendBody("direct:in",null); //fails, uses cached instance of SameDependentBean
}
}
According CAMEL-12610 is Processor supposed to be singleton scope. This behavior was introduced in version 2.20.0. Do not implement Processor interface, instead annotate invokable method as #Handler.
Replace
#Named
public class SomeDependentBean implements Processor {
public void process(Exchange exchange) throws Exception {
}
}
with
#Named
public class SomeDependentBean {
#Handler
public void process(Exchange exchange) throws Exception {
}
}
If you cannot afford that as me, because it is breaking behavior for our app extensions, I have implemented simple component. This component have no caching and allows to invoke Processor directly from registry.
CdiEndpoint class
public class CdiEndpoint extends ProcessorEndpoint {
private String beanName;
protected CdiEndpoint(String endpointUri, Component component) {
super(endpointUri, component);
}
public void setBeanName(String beanName) {
this.beanName = beanName;
}
#Override
protected void onExchange(Exchange exchange) throws Exception {
Object target = getCamelContext().getRegistry().lookupByName(beanName);
Processor processor = getCamelContext().getTypeConverter().tryConvertTo(Processor.class, target);
if (processor != null){
processor.process(exchange);
} else {
throw new RuntimeException("CDI bean "+beanName+" not found");
}
}
}
CdiComponent class
public class CdiComponent extends DefaultComponent {
#Override
protected Endpoint createEndpoint(String uri, String remaining, Map<String, Object> parameters) throws Exception {
CdiEndpoint endpoint = new CdiEndpoint(uri, this);
endpoint.setBeanName(remaining);
return endpoint;
}
}
Usage
public void configure() throws Exception {
getContext().addComponent("cdi", new CdiComponent());
from("direct:in")
.to("cdi:something");
}
I am trying to test a Camel route (polling messages from an SQS queue) containing
.bean("messageParserProcessor")
where messageParserProcessor is a Processor.
The test:
public class SomeTest extends CamelTestSupport {
private final String queueName = ...;
private final String producerTemplateUri = "aws-sqs://" + queueName + ...;
private static final String MESSAGE_PARSER_PROCESSOR_MOCK_ENDPOINT = "mock:messageParserProcessor";
#EndpointInject(uri = MESSAGE_PARSER_PROCESSOR_MOCK_ENDPOINT)
protected MockEndpoint messageParserProcessor;
#Override
public boolean isUseAdviceWith() {
return true;
}
#Before
public void setUpContext() throws Exception {
context.getRouteDefinitions().get(0).adviceWith(context, new AdviceWithRouteBuilder() {
#Override
public void configure() throws Exception {
interceptSendToEndpoint("bean:messageParserProcessor")
.skipSendToOriginalEndpoint()
.process(MESSAGE_PARSER_PROCESSOR_MOCK_ENDPOINT);
}
});
}
#Test
public void testParser() throws Exception {
context.start();
String expectedBody = "test";
messageParserProcessor.expectedBodiesReceived(expectedBody);
ProducerTemplate template = context.createProducerTemplate();
template.sendBody(producerTemplateUri, expectedBody);
messageParserProcessor.assertIsSatisfied();
context.stop();
}
}
When I run the test I get this error:
org.apache.camel.FailedToCreateRouteException:
Failed to create route route1 at:
>>> InterceptSendToEndpoint[bean:messageParserProcessor -> [process[ref:mock:messageParserProcessor]]] <<< in route: Route(route1)[[From[aws-sqs://xxx...
because of No bean could be found in the registry for: mock:messageParserProcessor of type: org.apache.camel.Processor
Same error if I replace interceptSendToEndpoint(...) with mockEndpointsAndSkip("bean:messageParserProcessor")
The test can be executed (but obviously doesn't pass) when I don't use a mock:
interceptSendToEndpoint("bean:messageParserProcessor")
.skipSendToOriginalEndpoint()
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {}
});
So the problem is the mock that is not found, what is wrong in the way I create it?
So I found a workaround to retrieve mocks from the registry:
interceptSendToEndpoint("bean:messageParserProcessor")
.skipSendToOriginalEndpoint()
.bean(getMockEndpoint(MESSAGE_PARSER_PROCESSOR_MOCK_ENDPOINT));
// Instead of
// .process(MESSAGE_PARSER_PROCESSOR_MOCK_ENDPOINT);
But I still don't understand why using .process("mock:someBean") doesn't work...
I am trying Camel-Kafka integration.
I have two queues :
queue1 and queue2.
There are three routes :
Route1 puts a list of two messages in queue1 (It should do it only once).
Route2 reads the list from queue1, splits it, and puts the individual messages in queue2
Route3 reads the messages from queue2 and just prints it.
The code is as follows :
import java.util.ArrayList;
import java.util.List;
import org.apache.camel.CamelContext;
import org.apache.camel.Exchange;
import org.apache.camel.Processor;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.impl.DefaultCamelContext;
public class CamelListTest {
public static void main(String[] args) throws Exception {
CamelContext context = new DefaultCamelContext();
context.addRoutes(new CamelListRoute());
context.start();
Thread.sleep(30000);
context.stop();
}
}
class CamelListRoute extends RouteBuilder {
#Override
public void configure() throws Exception {
//Route1, expected to run once
from("timer://timerName?repeatCount=1").process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
List<String> inOrderList = new ArrayList<String>();
inOrderList.add("1");
inOrderList.add("2");
exchange.getIn().setBody(inOrderList, ArrayList.class);
}
})
.to("kafka:<ip>:9092?topic=queue1");
//Route2
from("kafka:<ip>:9092?topic=queue1&groupId=testing&autoOffsetReset=latest&consumersCount=1")
.split()
.body().process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
System.out.println("2nd Route : " + (exchange.getIn().getBody().toString()));
}
})
.to("kafka:<ip>:9092?topic=queue2");
//Route3
from("kafka:<ip>:9092?topic=queue2&groupId=testing&autoOffsetReset=latest&consumersCount=1")
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
System.out.println("3rd Route : " + (exchange.getIn().getBody().toString()));
}
});
}
}
It is not working as expected, and there are few issues observed :
The first route, which is expected to run only once (repeatCount=1), runs continuously, putting the same message in queue1 again and again.
The second route reads the messages from queue1, splits it, but does not put it in queue2
Since second route does not put anything in queue2, this route does not get any messages.
Can anyone help me figure out what is wrong here?
I see couple of things:
I hope you are giving Kafka Url like this: "kafka://localhost:9092?topic=queue1"
note: kafka://
Providing zookeeper urls for consumers eg: kafka://?topic=queue1&zookeeperConnect=&consumerStreams=1&groupId=testing&autoOffsetReset=largest
Note in previous point autoOffsetReset value will be largest or smallest instead of latest.
I think you shoud exchange the message.
in processor do something like:
exchng.getOut().setHeader("type", "queue");
exchng.getOut().setBody(exchng.getIn().getBody() );
then could add a choice in the second route, does not require the third route.
I believe the first issue of running continuously and putting the same message in queue1 again and again is happening because you are using the same consumer groupId, groupId=testing for both your kafka consumers in routes 2 and 3.
Amend the kafka consumers to consume from different groupIds like so and this will ensure that the message is not consumed again and again.
//Route2
from("kafka:<ip>:9092?topic=queue1&groupId=testing-queue1&autoOffsetReset=latest&consumersCount=1")
and
//Route3
from("kafka:<ip>:9092?topic=queue2&groupId=testing-queue2&autoOffsetReset=latest&consumersCount=1")
The other issues of producing to queue2 and consuming from it to print, I think might be due to version incompatibilities. I have used camel-kafka version 2.20.1 (that uses kafka-clients 0.11.0.1 under the hood) and 2.21.0 (that uses kafka-clients 1.0.0 under the hood) and changed the routes to reflect the changes like so and this seems to consume-produce-consume fine.
class CamelListRoute extends RouteBuilder {
#Override
public void configure() throws Exception {
//Route1, expected to run once
from("timer://timerName?repeatCount=1").process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
List<String> inOrderList = new ArrayList<String>();
inOrderList.add("1");
inOrderList.add("2");
exchange.getIn().setBody(inOrderList, ArrayList.class);
}
})
.to("kafka:queue1?brokers=<ip>:9092");
//Route2
from("kafka:queue1?brokers=<ip>:9092&groupId=testing-queue1&autoOffsetReset=latest&consumersCount=1")
.split()
.body().process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
System.out.println("2nd Route : " + (exchange.getIn().getBody().toString()));
}
})
.to("kafka:queue2?brokers=<ip>:9092");
//Route3
from("kafka:queue2?brokers=<ip>:9092&groupId=testing-queue2&autoOffsetReset=latest&consumersCount=1")
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
System.out.println("3rd Route : " + (exchange.getIn().getBody().toString()));
}
});
}
}
All modifications of onRedelivery's processor is reset in next redelivery. Is there any way to make the modifications becomes permanent?
Properties are kept at each redelivery. You can use them to store information that you want to use after.
Code :
public class OnRedeliveryTest extends CamelTestSupport {
public static final String PROP_TEST = "PROP_TEST";
#Produce(uri = "direct:start")
ProducerTemplate producerTemplate;
#Override
public RouteBuilder createRouteBuilder() {
return new RouteBuilder() {
#Override
public void configure() throws Exception {
onException(Exception.class)
.onRedelivery(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
final String current = (String) exchange.getProperty(PROP_TEST);
exchange.setProperty(PROP_TEST, "property" + current);
System.out.println((String) exchange.getProperty(PROP_TEST));
}
})
.maximumRedeliveries(3).redeliveryDelay(0)
.handled(true)
.end();
from("direct:start")
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
}
})
.throwException(new Exception("BOOM"))
.to("mock:end");
}
};
}
#Test
public void smokeTest() throws Exception {
producerTemplate.sendBody("1");
}
}
In output, you will have :
propertynull
propertypropertynull
propertypropertypropertynull
During the processing of an Exchange received from JMS I'm creating dynamically a route that fetches a file from FTP to the file system and when the batch is done I need to remove that same route. The following code fragment shows how I do this:
public void execute() {
try {
context.addRoutes(createFetchIndexRoute(routeId()));
} catch (Exception e) {
throw Throwables.propagate(e);
}
}
private RouteBuilder createFetchIndexRoute(final String routeId) {
return new RouteBuilder() {
#Override
public void configure() throws Exception {
from("ftp://" + getRemoteQuarterDirectory() +
"?fileName=" + location.getFileName() +
"&binary=true" +
"&localWorkDirectory=" + localWorkDirectory)
.to("file://" + getLocalQuarterDirectory())
.process(new Processor() {
RouteTerminator terminator;
#Override
public void process(Exchange exchange) throws Exception {
if (camelBatchComplete(exchange)) {
terminator = new RouteTerminator(routeId,
exchange.getContext());
terminator.start();
}
}
})
.routeId(routeId);
}
};
}
I'm Using a thread to stop a route from a route, which is an approach recommended in the Camel Documentation - How can I stop a route from a route
public class RouteTerminator extends Thread {
private String routeId;
private CamelContext camelContext;
public RouteTerminator(String routeId, CamelContext camelContext) {
this.routeId = routeId;
this.camelContext = camelContext;
}
#Override
public void run() {
try {
camelContext.stopRoute(routeId);
camelContext.removeRoute(routeId);
} catch (Exception e) {
throw Throwables.propagate(e);
}
}
}
In result the route does stop. But what I see in the jconsole is that the thread that corresponds to the route isn't removed. Thus in time these abandoned threads just keep accumulating.
Is there a way to properly stop/remove a route dynamically/programmatically and also to release the route's thread, so that they don't accumulate through time?
This is fixed in the next Camel release 2.9.2 and 2.10. Fixed by this ticket:
https://issues.apache.org/jira/browse/CAMEL-5072