Apache Camel Main-Class and its methods start, stop, run, suspend and resume - apache-camel

I am writing my first camel application. it is a standalone application with a main method. As starting point i used the maven camel java archetype. It provides a simple main method that calls main.run().
Now i re-factored it a little bit and pulled the main.run out in a new class (and method) that will be my main-control of all camel stuff.
Now i want to create the "opposite" method of run(). At the moment i want to implement tests for single routs that start (run()) the context then wait (at the moment i am unsure how to wait 'til a route is finished) and the stop the context.
But now i discovered many method that could start and stop stuff all in Main class. The Jvadoc didn't help - that some methods are inherited doesn't make it easier ;-). So someone please tell me the exact meaning (or use case) for:
Main.run()
Main.start()
Main.stop()
Main.suspend()
Main.resume()
Thanks in advance.

See this page about the lifecycle of the various Camel services
http://camel.apache.org/lifecycle
And for waiting until a route is finished, then you can check the inflight registry if there is any current in-flight exchanges to know if a route is finished.
http://camel.apache.org/maven/current/camel-core/apidocs/org/apache/camel/spi/InflightRepository.html

We must separate the methods into 2 groups.
The first is the one described in the life cycle http://camel.apache.org/lifecycle
The second is composed of run and shutdown.
run runs indefinitely and can be stopped when invoking shutdown, the latter must be invoked in a different thread and sent before the run invocation.
Example:
import org.apache.camel.main.Main;
public class ShutdownTest {
public static void main(String[] args) throws Exception {
Main camel = new Main();
camel.addRouteBuilder( new MyRouteBuilder() );
// In this case the thread will wait a certain time and then invoke shutdown.
MyRunnable r = new MyRunnable(5000, camel);
r.excecute();
camel.run();
}
}
Simple Runnable class
public class MyRunnable implements Runnable {
long waitingFor = -1;
Main camel;
public MyRunnable(long waitingFor, Main camel){
this.waitingFor = waitingFor;
this.camel = camel;
}
public void excecute(){
Thread thread = new Thread(this);
thread.start();
}
#Override
public void run() {
try {
synchronized (this) {
this.wait( waitingFor );
}
} catch (InterruptedException e) {
}
try {
System.out.println("camel.shutdown()");
camel.shutdown();
} catch (Exception e) {
e.printStackTrace();
}
}
}

Related

Asynchronous Camel Component - doStop() called immediately

I am trying to create a camel component which consumes an API from an external service.
My Route is as follows
from("myComponent:entity?from=&to=")
.to("seda:one")
from("seda:one")
.aggregate(constant(true), new GroupedBodyAggregationStrategy())
.completionSize(5)
.completionTimeout(5000)
.process( new Processor1() )
to("seda:two")
.
.
.
from("seda:five")
.to("myComponent2:entity")
I implemented my component consumer as follows
public class MyComponentConsumer extends DefaultConsumer {
public MyComponentConsumer(MyComponentEndpoint endpoint, Processor processor) {
super(endpoint, processor);
}
#Override
protected void doStart() throws Exception {
super.doStart();
flag = true;
while ( flag ) {
//external API call
Resource resource = getNextResource();
if ( resource.next() == null ) {
flag = false;
}
Exchange ex = endpoint.createExchange(ExchangePattern.InOnly);
ex.getIn().setBody(resource.toString());
getAsyncProcessor().process(
ex
doneSync -> {
LOG.info("Message processed");
}
);
}
}
#Override
protected void doStop() throws Exception {
super.doStop();
System.out.println("stop ---- ");
}
}
Everything worked fine and the data was propogating through the route. My only problem was that data did not propogate to the next part until the whole of this process was completed. And the next parts were running asynchronously.
I looked at the example of StreamConsumer and tried to implement it to my code using a runnable and an executorService. But if I do that consumer stops as soon as it starts.
I changed the code to
public class MyComponentConsumer extends DefaultConsumer implements Runnable
and added
private ExecutorService executor;
getEndpoint().getCamelContext().getExecutorServiceManager().newSingleThreadExecutor(this, "myComponent");
executor.execute(this);
and moved my logic inside the run() method. But, the consumer thread ends as soon as it starts. and the async processor does not transfer the data properly.
Is there any other way to implement the functionality I need or am I mistaken somewhere here. Any help would be appreciated.
What version of camel are you using?
There was an issue with managing the state of consumer in camel 2.x which was fixed in camel 3.x CAMEL-12765 which can lead to the issue you are describing here.
If you are on camel 2.x try using newScheduledThreadPool instead of newSingleThreadExecutor.
Also executor.schedule(this, 5L, TimeUnit.SECONDS) instead of executor.execute(this).
Delayed start of executor might help avoid the problem you are facing.

Flink RichParallelSourceFunction - close() vs cancel()

I am implementing a RichParallelSourceFunction which reads files over SFTP. RichParallelSourceFunction inherits cancel() from SourceFunction and close() from RichFunction(). As far as I understand it, both cancel() and close() are invoked before the source is teared down. So in both of them I have to add logic for stopping the endless loop which reads files.
When I set the parallelism of the source to 1 and I run the Flink job from the IDE, Flink runtime invokes stop() right after it invokes start() and the whole job is stopped. I didn't expect this.
When I set the parallelism of the source to 1 and I run the Flink job in a cluster, the job runs as usual.
If I leave the parallelism of the source to the default (in my case 4), the job runs as usual.
Using Flink 1.7.
public class SftpSource<TYPE_OF_RECORD>
extends RichParallelSourceFunction<TYPE_OF_RECORD>
{
private final SftpConnection mConnection;
private boolean mSourceIsRunning;
#Override
public void open(Configuration parameters) throws Exception
{
mConnection.open();
}
#Override
public void close()
{
mSourceIsRunning = false;
}
#Override
public void run(SourceContext<TYPE_OF_RECORD> aContext)
{
while (mSourceIsRunning)
{
synchronized ( aContext.getCheckpointLock() )
{
// use mConnection
// aContext.collect() ...
}
try
{
Thread.sleep(1000);
}
catch (InterruptedException ie)
{
mLogger.warn("Thread error: {}", ie.getMessage() );
}
}
mConnection.close();
}
#Override
public void cancel()
{
mSourceIsRunning = false;
}
}
So I have workarounds and the question is more about the theory. Why is close() invoked if parallelism is 1 and the job is run from the IDE (i.e. from the command line)?
Also, do close() and cancel() do the same in a RichParallelSourceFunction?
Why is close() invoked if parallelism is 1 and the job is run from the
IDE.
close is called after the last call to the main working methods (e.g. map or join). This method can be used for clean up work.
It will be called independent of the number defined in parallelism.
Also, do close() and cancel() do the same in a RichParallelSourceFunction?
They aren't the same thing, take a look at how it's described.
Cancels the source. Most sources will have a while loop inside the run(SourceContext) method. The implementation needs to ensure that the source will break out of that loop after this method is called.
https://ci.apache.org/projects/flink/flink-docs-master/api/java/org/apache/flink/streaming/api/functions/source/SourceFunction.html#cancel--
The following link may help you to understand the task lifecycle:
https://ci.apache.org/projects/flink/flink-docs-stable/internals/task_lifecycle.html#operator-lifecycle-in-a-nutshell
I think javadocs are more than self-explanatory:
Gracefully Stopping Functions
Functions may additionally implement the {#link org.apache.flink.api.common.functions.StoppableFunction} interface. "Stopping" a function, in contrast to "canceling" means a graceful exit that leaves the state and the emitted elements in a consistent state.
-- SourceFunction.cancel
Cancels the source. Most sources will have a while loop inside the run(SourceContext) method. The implementation needs to ensure that the source will break out of that loop after this method is called.
A typical pattern is to have an "volatile boolean isRunning" flag that is set to false in this method. That flag is checked in the loop condition.
When a source is canceled, the executing thread will also be interrupted (via Thread.interrupt()). The interruption happens strictly after this method has been called, so any interruption handler can rely on the fact that this method has completed. It is good practice to make any flags altered by this method "volatile", in order to guarantee the visibility of the effects of this method to any interruption handler.
-- SourceContext.close
This method is called by the system to shut down the context.
Note, you can cancel SourceFunction, but stop SourceContext
I found a bug in my code. Here is the fix
public void open(Configuration parameters) throws Exception
{
mConnection.open();
mSourceIsRunning = true;
}
Now close() is not invoked until I decide to stop the workflow in which case first is invoked cancel() and then close(). I am still wondering how did parallelism affect the behaviour.

Camel Loadbalancer CircuitBreaker is not working as expected

We are trying to implement circuit breaker pattern in our product.
I tried to implement it and observed like if i give threshold value as '0' it is working fine.but any value >0 it simply waits for 2 sec and allowing request to communicate external rest services.
I tried below simple example where timer component is triggering messages for every 1 sec and in my processor i wrote logic like it will throw MyCustomException for 3 times, then circuit breaker will come to open state amd it should wait wait for 10sec as i read in camel document.but it is simply blocking request for 2 sec and then allowing request to process.
import org.apache.camel.CamelContext;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.impl.DefaultCamelContext;
public class CamelHttp {
public static void main(String[] args) throws Exception {
CamelContext context = new DefaultCamelContext();
context.addRoutes(new RouteBuilder() {
#Override
public void configure() throws Exception {
from("timer://simpleTimer?period=1000")
.setBody(simple("Hello from timer at ${header.firedTime}"))
.loadBalance()
.circuitBreaker(2, 10000, MyCustomException.class)
.process(new MyProcessor())
.to("mock:result");
}
});
context.start();
Thread.sleep(20000);
context.stop();
}
}
I am just stuck at this point.
Any help or suggestion is greatly appreciated.
Thanks in Advance.
The issue got resolved when I update my camel-core version from 2.15.0 to 2.16.0.
Now it is working as expected.

Akka/Camel UntypedConsumerActor not consuming from file-based queue

I'm trying to put together my first Akka/Camel application from "scratch" (read, "noob") using the following lib versions:
akka-camel: 2.2.0-RC1
According to all of the documentation I can find (Akka docs, user groups, etc.) all I have to do to consume from a file-based queue is set up my system this way:
Main class:
actorSystem = ActorSystem.create("my-system");
Props props = new Props(Supervisor.class);
ActorRef supervisor = actorSystem.actorOf(props, "supervisor");
Camel camel = CamelExtension.get(actorSystem);
CamelContext camelContext = camel.context();
camelContext.start();
Supervisor class:
import akka.actor.ActorRef;
import akka.actor.Props;
import akka.camel.javaapi.UntypedConsumerActor;
import org.apache.camel.Message;
/**
* Manages creation and supervision of UploadBatchWorkers.
*/
public class Supervisor extends UntypedConsumerActor {
#Override
public String getEndpointUri() {
return "file:///Users/myhome/queue";
}
#Override
public void preStart() {
String test = "test";
}
#Override
public void onReceive(Object message) {
if (message instanceof CamelMessage) {
// do something
}
}
My problem is that even though I know the supervisor object is being created and breaks during debugging on the preStart() method's "test" line (not to mention that if I explicitly "tell" it something it processes fine), it does not consume from the defined endpoint, even though I have another application producing messages to the same endpoint.
Any idea what I'm doing wrong?
Ok, the problem was my own fault and is clearly visible in the example code if you look at the Consumer trait from which the UntypedConsumerActor inherits.
This method:
#Override
public void preStart() {
String test = "test";
}
overrides its parent's preStart() method, right? Well, that parent method is actually the one that registers the consumer with the on-the-fly created endpoint, so while you can override it, you must call super() or it will not work.
Hope this is useful to someone down the road!
Try changing your instanceof inside of onReceive to this:
if (message instanceof CamelMessage){
//do processing here
}
Where CamelMessage is from package akka.camel. That's what the examples in the akka camel docs are doing.

EJB 3.1 and NIO2: Monitoring the file system

I guess most of us agree, that NIO2 is a fine thing to make use of. Presumed you want to monitor some part of the file system for incoming xml - files it is an easy task now. But what if I want to integrate the things into an existing Java EE application so I don't have to start another service (app-server AND the one which monitors the file system)?
So I have the heavy weight app-server with all the EJB 3.1 stuff and some kind of service monitoring the file system and take appropriate action once a file shows up. Interestingly the appropriate action is to create a Message and send it by JMS and it might be nice to integrate both into the app server.
I tried #Startup but deployment freezes (and I know that I shouldn't make use of I/O in there, was just a try). Anyhow ... any suggestions?
You could create a singleton that loads at startup and delegates the monitoring to an Asynchronous bean
#Singleton
#Startup
public class Initialiser {
#EJB
private FileSystemMonitor fileSystemMonitor;
#PostConstruct
public void init() {
String fileSystemPath = ....;
fileSystemMonitor.poll(fileSystemPath);
}
}
Then the Asynchronous bean looks something like this
#Stateless
public class FileSystemMonitor {
#Asynchronous
public void poll(String fileSystemPath) {
WatchService watcher = ....;
for (;;) {
WatchKey key = null;
try {
key = watcher.take();
for (WatchEvent<?> event: key.pollEvents()) {
WatchEvent.Kind<?> kind = event.kind();
if (kind == StandardWatchEventKinds.OVERFLOW) {
continue; // If events are lost or discarded
}
WatchEvent<Path> watchEvent = (WatchEvent<Path>)event;
//Process files....
}
} catch (InterruptedException e) {
e.printStackTrace();
return;
} finally {
if (key != null) {
boolean valid = key.reset();
if (!valid) break; // If the key is no longer valid, the directory is inaccessible so exit the loop.
}
}
}
}
}
Might help if you specified what server you're using, but have you considered implementing a JMX based service ? It's a bit more "neutral" than EJB, is more appropriate for a background service and has fewer restrictions.

Resources