We are trying to implement circuit breaker pattern in our product.
I tried to implement it and observed like if i give threshold value as '0' it is working fine.but any value >0 it simply waits for 2 sec and allowing request to communicate external rest services.
I tried below simple example where timer component is triggering messages for every 1 sec and in my processor i wrote logic like it will throw MyCustomException for 3 times, then circuit breaker will come to open state amd it should wait wait for 10sec as i read in camel document.but it is simply blocking request for 2 sec and then allowing request to process.
import org.apache.camel.CamelContext;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.impl.DefaultCamelContext;
public class CamelHttp {
public static void main(String[] args) throws Exception {
CamelContext context = new DefaultCamelContext();
context.addRoutes(new RouteBuilder() {
#Override
public void configure() throws Exception {
from("timer://simpleTimer?period=1000")
.setBody(simple("Hello from timer at ${header.firedTime}"))
.loadBalance()
.circuitBreaker(2, 10000, MyCustomException.class)
.process(new MyProcessor())
.to("mock:result");
}
});
context.start();
Thread.sleep(20000);
context.stop();
}
}
I am just stuck at this point.
Any help or suggestion is greatly appreciated.
Thanks in Advance.
The issue got resolved when I update my camel-core version from 2.15.0 to 2.16.0.
Now it is working as expected.
Related
I am trying to create a camel component which consumes an API from an external service.
My Route is as follows
from("myComponent:entity?from=&to=")
.to("seda:one")
from("seda:one")
.aggregate(constant(true), new GroupedBodyAggregationStrategy())
.completionSize(5)
.completionTimeout(5000)
.process( new Processor1() )
to("seda:two")
.
.
.
from("seda:five")
.to("myComponent2:entity")
I implemented my component consumer as follows
public class MyComponentConsumer extends DefaultConsumer {
public MyComponentConsumer(MyComponentEndpoint endpoint, Processor processor) {
super(endpoint, processor);
}
#Override
protected void doStart() throws Exception {
super.doStart();
flag = true;
while ( flag ) {
//external API call
Resource resource = getNextResource();
if ( resource.next() == null ) {
flag = false;
}
Exchange ex = endpoint.createExchange(ExchangePattern.InOnly);
ex.getIn().setBody(resource.toString());
getAsyncProcessor().process(
ex
doneSync -> {
LOG.info("Message processed");
}
);
}
}
#Override
protected void doStop() throws Exception {
super.doStop();
System.out.println("stop ---- ");
}
}
Everything worked fine and the data was propogating through the route. My only problem was that data did not propogate to the next part until the whole of this process was completed. And the next parts were running asynchronously.
I looked at the example of StreamConsumer and tried to implement it to my code using a runnable and an executorService. But if I do that consumer stops as soon as it starts.
I changed the code to
public class MyComponentConsumer extends DefaultConsumer implements Runnable
and added
private ExecutorService executor;
getEndpoint().getCamelContext().getExecutorServiceManager().newSingleThreadExecutor(this, "myComponent");
executor.execute(this);
and moved my logic inside the run() method. But, the consumer thread ends as soon as it starts. and the async processor does not transfer the data properly.
Is there any other way to implement the functionality I need or am I mistaken somewhere here. Any help would be appreciated.
What version of camel are you using?
There was an issue with managing the state of consumer in camel 2.x which was fixed in camel 3.x CAMEL-12765 which can lead to the issue you are describing here.
If you are on camel 2.x try using newScheduledThreadPool instead of newSingleThreadExecutor.
Also executor.schedule(this, 5L, TimeUnit.SECONDS) instead of executor.execute(this).
Delayed start of executor might help avoid the problem you are facing.
I want to test below camel route. All the example which i find online has route starting with file, where as in my case i have a spring bean method which is getting called every few minutes and finally message is transformed and moved to jms as well as audit directory.
I am clue less on write test for this route.
All i have currently in my test case is
Mockito.when(tradeService.searchTransaction()).thenReturn(dataWithSingleTransaction);
from("quartz2://tsTimer?cron=0/20+*+8-18+?+*+MON,TUE,WED,THU,FRI+*")
.bean(TradeService.class)
.marshal()
.jacksonxml(true)
.to("jms:queue:out-test")
.to("file:data/test/audit")
.end();
Testing with Apache Camel and Spring-Boot is really easy.
Just do the following (the example below is an abstract example just to give you a hint how you can do it):
Write a Testclass
Use the Spring-Boot Annotations to configure the test class.
#SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.NONE)
#RunWith(SpringRunner.class)
public class MyRouteTest {
#EndpointInject(uri = "{{sourceEndpoint}}")
private ProducerTemplate sourceEndpoint;
....
public void test() {
// send your body to the endpoint. See other provided methods too.
sourceEndpoint.sendBody([your input]);
}
}
In the src/test/application.properties:
Configure your Camel-Endpoints like the source and the target:
sourceEndpoint=direct:myTestSource
Hints:
It's good not to hardwire your start-Endpoint in the route directly when using spring-boot but to use the application.properties. That way it is easier to mock your endpoints for unit tests because you can change to the direct-Component without changing your source code.
This means instead of:
from("quartz2://tsTimer?cron=0/20+*+8-18+?+*+MON,TUE,WED,THU,FRI+*")
you should write:
from("{{sourceEndpoint}}")
and configure the sourceEndpoint in your application.properties:
sourceEndpoint=quartz2://tsTimer?cron=0/20+*+8-18+?+*+MON,TUE,WED,THU,FRI+*
That way you are also able to use your route for different situations.
Documentation
A good documentation about how to test with spring-boot can be found here: https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-testing.html
For Apache Camel: http://camel.apache.org/testing.html
#the hand of NOD Thanks for your hints, i was going into completely wrong direction. After reading your answer i was able to write the basic test and from this i think i can take it forward.
Appreciate your time, however i see that based on my route it should drop an XML file to audit directory which is not happening.
Look like intermediate steps are also getting mocked, without I specifying anything.
InterceptSendToMockEndpointStrategy - Adviced endpoint [xslt://trans.xslt] with mock endpoint [mock:xslt:trans.xslt]
INFO o.a.c.i.InterceptSendToMockEndpointStrategy - Adviced endpoint [file://test/data/audit/?fileName=%24%7Bheader.outFileName%7D] with mock endpoint [mock:file:test/data/audit/]
INFO o.a.camel.spring.SpringCamelContext - StreamCaching is not in use. If using streams then its recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html
TradePublisherRoute.java
#Override
public void configure() throws Exception {
logger.info("TradePublisherRoute.configure() : trade-publisher started configuring camel route.");
from("{{trade-publisher.sourceEndpoint}}")
.doTry()
.bean(tradeService)
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
String dateStr = Constant.dateFormatForFileName.format(new Date());
logger.info("this is getting executed : " + dateStr);
exchange.setProperty(Constant.KEY_INCOMING_XML_FILE_NAME, "REQ-" + dateStr + Constant.AUDIT_FILE_EXTENSION);
exchange.setProperty(Constant.KEY_OUTGOING_XML_FILE_NAME, "RESP-" + dateStr + Constant.AUDIT_FILE_EXTENSION);
}
})
.marshal()
.jacksonxml(true)
.wireTap("{{trade-publisher.requestAuditDir}}" + "${header.inFileName}")
.to("{{trade-publisher.xsltFile}}")
.to("{{trade-publisher.outboundQueue}}")
.to("{{trade-publisher.responseAuditDir}}" + "${header.outFileName}")
.bean(txnService, "markSuccess")
.endDoTry()
.doCatch(Exception.class)
.bean(txnService, "markFailure")
.log(LoggingLevel.ERROR, "EXCEPTION: ${exception.stacktrace}")
.end();
TradePublisherRouteTest.java
#ActiveProfiles("test")
#RunWith(CamelSpringBootRunner.class)
#SpringBootTest(classes = TradePublisherApplication.class)
#MockEndpoints
public class TradePublisherRouteTest {
#EndpointInject(uri = "{{trade-publisher.outboundQueue}}")
private MockEndpoint mockQueue;
#EndpointInject(uri = "{{trade-publisher.sourceEndpoint}}")
private ProducerTemplate producerTemplate;
#MockBean
TradeService tradeService;
private List<Transaction> transactions = new ArrayList<>();
#BeforeClass
public static void beforeClass() {
}
#Before
public void before() throws Exception {
Transaction txn = new Transaction("TEST001", "C001", "100", "JPM", new BigDecimal(100.50), new Date(), new Date(), 1000, "P");
transactions.add(txn);
}
#Test
public void testRouteConfiguration() throws Exception {
Mockito.when(tradeService.searchTransaction()).thenReturn(new Data(transactions));
producerTemplate.sendBody(transactions);
mockQueue.expectedMessageCount(1);
mockQueue.assertIsSatisfied(2000);
}
Please correct me if i am doing something wrong!
To retrieve some open data from a remote web server to process, I'm trying out Apache Camel.
The problem is that it seems that the data is never received. I have tried the jetty, ahc and cxf components but can't get it to work. For example like this:
import org.apache.camel.CamelContext;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.impl.DefaultCamelContext;
public class CamelHttpDemo {
public static void main(final String... args) {
final CamelContext context = new DefaultCamelContext();
try {
context.addRoutes(new RouteBuilder() {
#Override
public void configure() throws Exception {
this.from("direct:start")
.to("ahc:http://camel.apache.org/")
.process(exchange -> {
System.out.println(exchange);
});
}
});
context.start();
Thread.sleep(10000);
context.stop();
} catch (final Exception e) {
e.printStackTrace();
}
}
}
No output is written so the line System.out.println(exchange); is never executed and I assume the data is not retrieved.
I'm using the most recent version of Apache Camel, 2.17.1.
You need some message producer in your route to emit Exchange that would trigger the http component. Your route starts with direct:start which cannot emit new Exchanges, it just sits and waits for someone to initiate the process.
The easiest way to make your route work is to replace direct:start with some producer. For instance, replacing it with this timer .from("timer://foo?fixedRate=true&period=10000") will trigger your http-request once every 10 seconds.
If you want to initiate the request manually, you need to create a ProducerTemplate and use it to send a message to direct:start. That would be:
ProducerTemplate template = context.createProducerTemplate();
template.sendMessage("direct:start", "Message body");
I am writing my first camel application. it is a standalone application with a main method. As starting point i used the maven camel java archetype. It provides a simple main method that calls main.run().
Now i re-factored it a little bit and pulled the main.run out in a new class (and method) that will be my main-control of all camel stuff.
Now i want to create the "opposite" method of run(). At the moment i want to implement tests for single routs that start (run()) the context then wait (at the moment i am unsure how to wait 'til a route is finished) and the stop the context.
But now i discovered many method that could start and stop stuff all in Main class. The Jvadoc didn't help - that some methods are inherited doesn't make it easier ;-). So someone please tell me the exact meaning (or use case) for:
Main.run()
Main.start()
Main.stop()
Main.suspend()
Main.resume()
Thanks in advance.
See this page about the lifecycle of the various Camel services
http://camel.apache.org/lifecycle
And for waiting until a route is finished, then you can check the inflight registry if there is any current in-flight exchanges to know if a route is finished.
http://camel.apache.org/maven/current/camel-core/apidocs/org/apache/camel/spi/InflightRepository.html
We must separate the methods into 2 groups.
The first is the one described in the life cycle http://camel.apache.org/lifecycle
The second is composed of run and shutdown.
run runs indefinitely and can be stopped when invoking shutdown, the latter must be invoked in a different thread and sent before the run invocation.
Example:
import org.apache.camel.main.Main;
public class ShutdownTest {
public static void main(String[] args) throws Exception {
Main camel = new Main();
camel.addRouteBuilder( new MyRouteBuilder() );
// In this case the thread will wait a certain time and then invoke shutdown.
MyRunnable r = new MyRunnable(5000, camel);
r.excecute();
camel.run();
}
}
Simple Runnable class
public class MyRunnable implements Runnable {
long waitingFor = -1;
Main camel;
public MyRunnable(long waitingFor, Main camel){
this.waitingFor = waitingFor;
this.camel = camel;
}
public void excecute(){
Thread thread = new Thread(this);
thread.start();
}
#Override
public void run() {
try {
synchronized (this) {
this.wait( waitingFor );
}
} catch (InterruptedException e) {
}
try {
System.out.println("camel.shutdown()");
camel.shutdown();
} catch (Exception e) {
e.printStackTrace();
}
}
}
I would like to set up a Camel CXF endpont, and have the SOAP response asynchronous to much of my Camel Route. The route will have a lot of processing steps, and I do not want to have the response generated at the very end.
An example endpoint:
<cxf:cxfEndpoint id="someEndpoint"
address="/Service"
serviceClass="com.SomeImpl" />
An example route:
public class MyRouteBuilder extends RouteBuilder {
#Override
public void configure() throws Exception {
from("cxf:bean:someEndpoint")
to("bean:processingStep1")
to("bean:replyToSOAP") // I would like to respond to cxf:cxfEndpoint here!
to("bean:processingStep2")
to("bean:processingStep3")
to("bean:processingStep4");
// The response to cxf:cxfEndpoint actually happens here.
}
}
I have tried many options in MyRouteBuilder to "fork" the process (i.e. bean:replyToSOAP):
.multicast().parallelProcessing()
In-memory Asynchronous messaging ("seda" and "vm")
I have NOT tried JMS. It could be overkill for what I want to do.
I can get the route steps to process in parallel, but all steps must be completed before the response is generated.
In addition to the answer Claus gives below, I'd like to add that the placement of wireTap is important. Using:
.wireTap("bean:replyToSOAP")
will not get the desired behavior. What will is:
public class MyRouteBuilder extends RouteBuilder {
#Override
public void configure() throws Exception {
from("cxf:bean:someEndpoint")
to("bean:processingStep1")
.wireTap("direct:furtherProcessing")
to("bean:replyToSOAP") // respond to cxf:cxfEndpoint here
from("direct:furtherProcessing") // steps happen independantly of beann:replyToSOAP
to("bean:processingStep2")
to("bean:processingStep3")
to("bean:processingStep4");
}
}
There is the WireTap EIP which can spin of a copy of the message to be processed independently from the current route: http://camel.apache.org/wire-tap