Camel route shutdown during Day light Savings time - apache-camel

Below are my camel routes to send periodic messages and forward messages in queue to endpoint.
Event Route:
from("activemq:Queue.External?cacheLevelName=CACHE_CONSUMER&transacted=true")
.routeId("EventRoute")
.autoStartup(false)
.filter(messageFilter)
.process(eventTransformer)
.setHeader(Exchange.HTTP_METHOD, constant(HttpMethods.POST))
.setHeader(Exchange.ACCEPT_CONTENT_TYPE, constant("application/xml"))
.setProperty("eventEndpoint", constant(eventEndpoint))
.to(eventUri)
.process(eventResponse);
Periodic message route:
from("timer:monitor?fixedRate=true&period=" + (periodicMessageInterval() * 1000))
.routeId("periodicMessageRoute")
.autoStartup(false)
.process(periodicMessageTransformer)
.setHeader(Exchange.HTTP_METHOD, constant(HttpMethods.POST))
.setHeader(Exchange.ACCEPT_CONTENT_TYPE, constant("application/xml"))
.doTry()
.to(periodicMessageUri)
.process(periodicMessageResponse)
.doCatch(Exception.class)
.log(LoggingLevel.DEBUG, "Error response received: ${body}");
Route initialization for periodic message detection
builder.from("activemq:topic:Topic.Heartbeat?concurrentConsumers=1&maxConcurrentConsumers=1")
.routeId("periodic")
.log(LoggingLevel.DEBUG, "Periodic Message Received: ${id}")
.process(this);
builder.from("timer:monitor?fixedRate=true&period=" + monitoringIntervalInMilliseconds)
.routeId("timer")
.log(LoggingLevel.DEBUG, "Checking periodic message reception")
.process(exchange -> exchange.getIn().setBody(check()))
.choice().when(builder.body(Boolean.class)).to("direct:stopActiveMqRoutes")
.otherwise().to("direct:startActiveMqRoutes").end();
ProcessorDefinition routeDefinitionStop = builder.from("direct:stopActiveMqRoutes");
for (final String routeId : routeIds) {
routeDefinitionStop = routeDefinitionStop
.to("controlbus:route?routeId=" + routeId + "&action=status")
.choice().when(builder.body().isNotEqualTo("Stopped"))
.log(LoggingLevel.INFO, "Stopping route execution: " + routeId)
.to("controlbus:route?routeId=" + routeId + "&action=stop&async=true")
.end();
}
routeDefinitionStop.end();
ProcessorDefinition routeDefinitionStart = builder.from("direct:startActiveMqRoutes");
for (final String routeId : routeIds) {
routeDefinitionStart = routeDefinitionStart
.to("controlbus:route?routeId=" + routeId + "&action=status")
.choice().when(builder.body().isNotEqualTo("Started"))
.log(LoggingLevel.INFO, "Starting route execution: " + routeId)
.to("controlbus:route?routeId=" + routeId + "&action=start&async=true")
.end();
}
routeDefinitionStart.end();
Both the routes stopped during the day light savings time for 1 hour and the routes started automatically after 1 hour. Is it because of the Timer component in JDK that caused this issue?
Error log:
2017-11-05 01:01:13,921 INFO [route2] Stopping route execution: EventRoute
2017-11-05 01:01:13,921 INFO [route2] Stopping route execution: periodicMessageRoute
2017-11-05 01:01:26,671 INFO [org.apache.camel.impl.DefaultShutdownStrategy] Route: EventRoute shutdown complete, was consuming from: activemq://Queue.External?cacheLevelName=CACHE_CONSUMER&transacted=true
2017-11-05 01:01:26,672 INFO [org.apache.camel.impl.DefaultShutdownStrategy] Graceful shutdown of 1 routes completed in 22 seconds
2017-11-05 01:01:26,673 INFO [org.apache.camel.spring.SpringCamelContext] Route: EventRoute is stopped, was consuming from: activemq://Queue.External?cacheLevelName=CACHE_CONSUMER&transacted=true
2017-11-05 01:01:26,675 INFO [org.apache.camel.impl.DefaultShutdownStrategy] Starting to graceful shutdown 1 routes (timeout 300 seconds)
2017-11-05 01:01:26,676 INFO [org.apache.camel.impl.DefaultShutdownStrategy] Route: periodicMessageRoute shutdown complete, was consuming from: timer://monitor?fixedRate=true&period=30000
2017-11-05 01:01:26,680 INFO [org.apache.camel.impl.DefaultShutdownStrategy] Graceful shutdown of 1 routes completed in 0 seconds
2017-11-05 01:01:26,681 INFO [org.apache.camel.spring.SpringCamelContext] Route: periodicMessageRoute is stopped, was consuming from: timer://monitor?fixedRate=true&period=30000
2017-11-05 01:01:26,681 INFO [org.apache.camel.impl.DefaultShutdownStrategy] Starting to graceful shutdown 1 routes (timeout 300 seconds)
2017-11-05 01:01:26,681 INFO [org.apache.camel.impl.DefaultShutdownStrategy] Route: EventRoute shutdown complete, was consuming from: activemq://Queue.External?cacheLevelName=CACHE_CONSUMER&transacted=true
2017-11-05 01:01:26,681 INFO [org.apache.camel.impl.DefaultShutdownStrategy] Graceful shutdown of 1 routes completed in 0 seconds
2017-11-05 01:00:03,921 INFO [route3] Starting route execution: EventRoute
2017-11-05 01:00:03,924 INFO [route3] Starting route execution: periodicMessageRoute
2017-11-05 01:00:03,943 INFO [org.apache.camel.spring.SpringCamelContext] Route: EventRoute started and consuming from: activemq://Queue.External?cacheLevelName=CACHE_CONSUMER&transacted=true

Related

Embedded ActiveMQ connection taking a long time to shutdown

I'm using Camel with embedded ActiveMQ for some functional tests. But after the tests have run, ActiveMQ is having trouble shutting down, and I see logs like this:
2022-01-31 19:12:34,873 [MQ ShutdownHook] INFO BrokerService - Apache ActiveMQ 5.15.9 (localhost, ID:8dc9dbba2775-37769-1643655318556-0:2) is shutting down
...
2022-01-31 19:12:34,875 [MQ ShutdownHook] INFO TransportConnector - Connector tcp://localhost:0 stopped
2022-01-31 19:12:35,052 [m://localhost#4] INFO PooledConnectionFactory - Expiring connection ActiveMQConnection {id=ID:8dc9dbba2775-37769-1643655318556-4:3,clientId=ID:8dc9dbba2775-37769-1643655318556-3:2,started=false} on IOException: peer (vm://localhost#5) stopped.
...
2022-01-31 19:12:35,053 [MQ ShutdownHook] INFO BrokerService - Apache ActiveMQ 5.15.9 (localhost, ID:8dc9dbba2775-37769-1643655318556-0:19) is shutdown
2022-01-31 19:12:40,052 [MQ ShutdownHook] INFO TransportConnection - The connection to 'vm://localhost#12' is taking a long time to shutdown.
2022-01-31 19:12:45,052 [MQ ShutdownHook] INFO TransportConnection - The connection to 'vm://localhost#12' is taking a long time to shutdown.
2022-01-31 19:12:50,053 [MQ ShutdownHook] INFO TransportConnection - The connection to 'vm://localhost#12' is taking a long time to shutdown.
A connection shutdown is failing and keeps repeating that last message.
The config is done programmatically for the tests :
private static void addActiveMqComponent()
{
if ( FunctionalTestFramework.framework().context().getComponent("activemq2") == null)
{
JmsConfiguration jmsConfig = new JmsConfiguration();
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory("vm://localhost:61616?broker.persistent=false");
connectionFactory.setTrustAllPackages(true);
RedeliveryPolicy redeliveryPolicy = new RedeliveryPolicy();
redeliveryPolicy.setInitialRedeliveryDelay(1000);
redeliveryPolicy.setRedeliveryDelay(1000);
redeliveryPolicy.setBackOffMultiplier(3.5);
redeliveryPolicy.setUseExponentialBackOff(true);
redeliveryPolicy.setMaximumRedeliveries(-1);
connectionFactory.setRedeliveryPolicy(redeliveryPolicy);
jmsConfig.setConnectionFactory(connectionFactory );
transactionManager = new JmsTransactionManager();
transactionManager.setConnectionFactory(connectionFactory);
jmsConfig.setTransactionManager(transactionManager);
ActiveMQComponent activeMqComponent = new ActiveMQComponent();
activeMqComponent.setConfiguration(jmsConfig);
activeMqComponent.setTransacted(true);
activeMqComponent.setCacheLevelName("CACHE_CONSUMER");
FunctionalTestFramework.framework().context().addComponent("activemq2", activeMqComponent);
}
}
I've seen some other threads suggesting to set useShutdownHook="false" but it only applies when they're explicitly triggering shutdown (broker.stop()) which I'm not doing. And I'm not finding much else on this specific problem.
Any ideas on what the issue could be?

How to start one route to another with interval time in Apache Camel

I wanna call route2 from route1 and it should execute interval time, I created below code, is it correct code, write multiple from methods, can anyone please give me suggestions?
//Route1
from("timer:repeatcount=1").
.to("direct:route2 ");
//Route2
from("direct:route2").
from("timer://simpleTimer?period=1000")
.setBody(simple("Hello from timer at ${header.firedTime}"))
.to("stream:out");
Below you'll find a minimal complete Camel 3 (version 3.4.3) example how to trigger a route (TimerRoute) with Timer component and how to call another route (SaveFileRoute). The example is implemented with Camel Main module.
The routes use From EIP to consume messages from endpoints generated by Timer and Direct components and To EIP to produce (or "to send") messages to the consumers.
First the Timer component is used to automatically generate route invocations. This is the starting point of the message flow:
from("timer:exampleTimer?fixedRate=true&period=3s")
In the end of the first route the message is forwarded to the second route:
.to("direct:savefile")
Next step takes place in the second route where the message is consumed:
from("direct:savefile")
Because the second route doesn't forward the message anywhere the message flow ends there.
Note that in the string "direct:savefile":
direct is the type of component generating the endpoint
saveFile is a name that uniquely identifies the endpoint
The complete example:
// https://camel.apache.org/components/latest/others/main.html
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.main.Main;
public class App {
public static void main(String[] args) throws Exception {
Main main = new Main();
main.configure().addRoutesBuilder(
new RouteBuilder() {
public void configure() {
from("direct:savefile")
.routeId("SaveFileRoute")
.log("ROUTE START: body = ${body}")
// just logging here
// .to("file://outbox")
.log("ROUTE END:")
;
}
}
);
main.configure().addRoutesBuilder(
new RouteBuilder() {
public void configure() {
// triggers on every 3 seconds
// https://camel.apache.org/components/latest/timer-component.html
from("timer:exampleTimer?fixedRate=true&period=3s")
.routeId("TimerRoute")
.log("ROUTE START:")
// https://camel.apache.org/components/latest/eips/setBody-eip.html
.setBody(constant("HELLO FROM TIMER!"))
.to("direct:savefile")
.log("ROUTE END:")
;
}
}
);
main.run(args);
}
}
Example run:
$ mvn compile exec:java
[INFO] Scanning for projects...
[INFO]
[INFO] -----------------------< net.jani-hur:004-timer >-----------------------
[INFO] Building 004-timer 1.0
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) # 003-timer ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) # 003-timer ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- exec-maven-plugin:3.0.0:java (default-cli) # 004-timer ---
[ App.main()] BaseMainSupport INFO Using properties from: classpath:application.properties;optional=true
[ App.main()] DefaultRoutesCollector INFO No additional Camel XML routes discovered from: classpath:camel/*.xml
[ App.main()] DefaultRoutesCollector INFO No additional Camel XML rests discovered from: classpath:camel-rest/*.xml
[ App.main()] AbstractCamelContext INFO Apache Camel 3.4.3 (camel-1) is starting
[ App.main()] AbstractCamelContext INFO StreamCaching is not in use. If using streams then its recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html
[ App.main()] InternalRouteStartupManager INFO Route: SaveFileRoute started and consuming from: direct://savefile
[ App.main()] InternalRouteStartupManager INFO Route: TimerRoute started and consuming from: timer://exampleTimer
[ App.main()] AbstractCamelContext INFO Total 2 routes, of which 2 are started
[ App.main()] AbstractCamelContext INFO Apache Camel 3.4.3 (camel-1) started in 0.031 seconds
[read #0 - timer://exampleTimer] TimerRoute INFO ROUTE START:
[read #0 - timer://exampleTimer] SaveFileRoute INFO ROUTE START: body = HELLO FROM TIMER!
[read #0 - timer://exampleTimer] SaveFileRoute INFO ROUTE END:
[read #0 - timer://exampleTimer] TimerRoute INFO ROUTE END:
[read #0 - timer://exampleTimer] TimerRoute INFO ROUTE START:
[read #0 - timer://exampleTimer] SaveFileRoute INFO ROUTE START: body = HELLO FROM TIMER!
[read #0 - timer://exampleTimer] SaveFileRoute INFO ROUTE END:
[read #0 - timer://exampleTimer] TimerRoute INFO ROUTE END:
[read #0 - timer://exampleTimer] TimerRoute INFO ROUTE START:
[read #0 - timer://exampleTimer] SaveFileRoute INFO ROUTE START: body = HELLO FROM TIMER!
[read #0 - timer://exampleTimer] SaveFileRoute INFO ROUTE END:
[read #0 - timer://exampleTimer] TimerRoute INFO ROUTE END:
[read #0 - timer://exampleTimer] TimerRoute INFO ROUTE START:
[read #0 - timer://exampleTimer] SaveFileRoute INFO ROUTE START: body = HELLO FROM TIMER!
[read #0 - timer://exampleTimer] SaveFileRoute INFO ROUTE END:
[read #0 - timer://exampleTimer] TimerRoute INFO ROUTE END:
[read #0 - timer://exampleTimer] TimerRoute INFO ROUTE START:
[read #0 - timer://exampleTimer] SaveFileRoute INFO ROUTE START: body = HELLO FROM TIMER!
[read #0 - timer://exampleTimer] SaveFileRoute INFO ROUTE END:
[read #0 - timer://exampleTimer] TimerRoute INFO ROUTE END:
[read #0 - timer://exampleTimer] TimerRoute INFO ROUTE START:
[read #0 - timer://exampleTimer] SaveFileRoute INFO ROUTE START: body = HELLO FROM TIMER!
[read #0 - timer://exampleTimer] SaveFileRoute INFO ROUTE END:
[read #0 - timer://exampleTimer] TimerRoute INFO ROUTE END:
^C$

Apache Camel - Parallel Routes Inflight Exchanges

I have a Camel context with many routes that starts every 15m with Timer Component.
These routes set some properties in exchange (Target host, Query and Current Date that I use a Processor to get date, -12 hours and transform to GMT).
After set these properties, using Direct, another route is called to execute the HTTP Get. When the Request finished, another Route is called to Post the return on Artemis ActiveMQ.
The project is deployed on Wildfly 13.
The problem is:
Sometimes the routes simply freeze. Don't start after 15 minutes.
When I try to stop/start the route, I got the follow log:
[0m[0m08:27:45,230 INFO [org.apache.camel.impl.DefaultShutdownStrategy] (Camel (camel-example) thread #70 - ShutdownTask) There are 1 inflight exchanges: InflightExchange: [exchangeId=ID-exchange-ID, fromRouteId=Route1, routeId=GetDataAutoBySinceTime, nodeId=toD7, elapsed=0, duration=216958569]
[0m[0m08:27:46,231 INFO [org.apache.camel.impl.DefaultShutdownStrategy] (Camel (camel-example) thread #70 - ShutdownTask) Waiting as there are still 1 inflight and pending exchanges to complete, timeout in 299 seconds. Inflights per route: [Route1 = 1]
[0m[0m08:27:46,231 INFO [org.apache.camel.impl.DefaultShutdownStrategy] (Camel (camel-example) thread #70 - ShutdownTask) There are 1 inflight exchanges: InflightExchange: [exchangeId=ID-exchange-ID, fromRouteId=Route1, routeId=GetDataAutoBySinceTime, nodeId=toD7, elapsed=0, duration=216959570]
[0m[0m08:27:47,231 INFO [org.apache.camel.impl.DefaultShutdownStrategy] (Camel (camel-example) thread #70 - ShutdownTask) Waiting as there are still 1 inflight and pending exchanges to complete, timeout in 298 seconds. Inflights per route: [Route1 = 1]
I don't know if some processes are stuck making it impossible another processes to start.
I thought to remove the generic routes (PostMessageInActiveMQ and
GetDataAutomaticallyBySinceTime and to implements the same code in another routes (Route1, Route2 and Route3) but I don't think this is the best approach.
Routes:
Route1 (Route2 and Route3 are almost the same, just change properties values)
from("timer:Route1Timer?period=15m")
.routeId("Route1")
.autoStartup(false)
.setProperty("targetAddress", simple("hostname.route1"))
.process(new GetCurrentDate())
.setProperty("query",
simple("DataQuery%26URI=Route1%26format=xml%26Mode=since-time%26p1=${header.currentDate}"))
.to("direct:GetDataAutoBySinceTime");
GetDataAutomaticallyBySinceTime
from("direct:GetDataAutoBySinceTime")
.routeId("GetDataAutoBySinceTime")
.autoStartup(true)
.removeHeaders("*")
.setHeader(Exchange.HTTP_METHOD, constant("GET"))
.toD("http4:${header.targetAddress}/command=${header.query}%26httpClient.socketTimeout=3000")
.convertBodyTo(String.class, "utf-8")
.to("direct:PostMessageInActiveMQ");
PostMessageInActiveMQ
CamelArtemisComponent components = new CamelArtemisComponent();
getContext().addComponent("artemis", components.getArtemisComponent());
from("direct:PostMessageInActiveMQ")
.routeId("PostMessageInActiveMQ")
.autoStartup(true)
.convertBodyTo(String.class, "utf-8")
.inOnly("artemis:ARTEMIS.QUEUE");
Entire code: https://github.com/vitorvr/camel-example
EDIT:
Camel Version: 2.22.0

How to consume messages from a Topic ActiveMQ Artemis

I'm trying to work with topics on ActiveMQ Artemis.
I have created a Multicast Address and a Multicast Queue inside this Address.
Created 2 routes with Apache Camel to connect in this Topic, but when I post message only one Route consume the message and when I post another message, the secont Route that consume this message message.
Below the code and the output.
public class CamelRoutes {
public static void main(String[] args) throws Exception {
ActiveMQJMSConnectionFactory connection = new ActiveMQJMSConnectionFactory("tcp://localhost:61616", "admin", "admin");
CamelContext camel = new DefaultCamelContext();
camel.addComponent("amq", JmsComponent.jmsComponent(connection));
camel.addRoutes(new RouteBuilder(){
#Override
public void configure() throws Exception {
from("amq:TEST.TOPIC")
.routeId("Route1")
.log("ROUTE1: ${body}");
}
});
camel.addRoutes(new RouteBuilder(){
#Override
public void configure() throws Exception {
from("amq:TEST.TOPIC")
.routeId("Route2")
.log("ROUTE2: ${body}");
}
});
camel.start();
Thread.sleep(20000000);
}
}
2019-02-11 16:35:42 [Camel (camel-1) thread #1 - JmsConsumer[TEST.TOPIC]] INFO Route1:159 - ROUTE1: {"message":1}
2019-02-11 16:35:45 [Camel (camel-1) thread #2 - JmsConsumer[TEST.TOPIC]] INFO Route2:159 - ROUTE2: {"message":2}
2019-02-11 16:35:48 [Camel (camel-1) thread #1 - JmsConsumer[TEST.TOPIC]] INFO Route1:159 - ROUTE1: {"message":3}
2019-02-11 16:35:51 [Camel (camel-1) thread #2 - JmsConsumer[TEST.TOPIC]] INFO Route2:159 - ROUTE2: {"message":4}
2019-02-11 16:35:54 [Camel (camel-1) thread #1 - JmsConsumer[TEST.TOPIC]] INFO Route1:159 - ROUTE1: {"message":5}
You are consuming from the queue, not from the topic.
You need to correct your consumer's URI scheme.
Change your consumer to:
from("amq:topic:TEST.TOPIC");
This is how you can create queue consumer :
from("amq:queue:YOUR.QUEUE.NAME);
// or as queue is default value
from("amq:YOUR.QUEUE.NAME);
This is how you can create topic consumer :
from("amq:topic.YOUR.TOPIC.NAME);

Unable to process large files in camel

I am trying to do simple transformation on a Csv file.But my programm is getting stuck and not giving any output and on console its printing something like below.
22:38:02.001 [main] INFO o.a.camel.impl.DefaultCamelContext - Apache Camel 2.15.2 (CamelContext: camel-1) is shutting down
22:38:02.135 [main] INFO o.a.c.impl.DefaultShutdownStrategy - Starting to graceful shutdown 1 routes (timeout 300 seconds)
22:38:02.167 [main] DEBUG o.a.c.i.DefaultExecutorServiceManager - Created new ThreadPool for source: org.apache.camel.impl.DefaultShutdownStrategy#65ead16a with name: ShutdownTask. -> org.apache.camel.util.concurrent.RejectableThreadPoolExecutor#52c0a65f[Running, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 0][ShutdownTask]
22:38:02.173 [Camel (camel-1) thread #1 - ShutdownTask] DEBUG o.a.c.impl.DefaultShutdownStrategy - There are 1 routes to shutdown
22:38:02.177 [Camel (camel-1) thread #1 - ShutdownTask] DEBUG o.a.c.impl.DefaultShutdownStrategy - Route: route1 suspended and shutdown deferred, was consuming from: Endpoint[file:///home/cloudera/Desktop/camelinput/?delay=15m&noop=true]
22:38:02.177 [Camel (camel-1) thread #1 - ShutdownTask] INFO o.a.c.impl.DefaultShutdownStrategy - Waiting as there are still 2 inflight and pending exchanges to complete, timeout in 300 seconds.
22:38:02.179 [Camel (camel-1) thread #1 - ShutdownTask] DEBUG o.a.c.impl.DefaultShutdownStrategy - There are 1 inflight exchanges:
InflightExchange: [exchangeId=ID-quickstart-cloudera-40574-1441345060577-0-2, fromRouteId=route1, routeId=route1, nodeId=unmarshal1, elapsed=10787, duration=10791]
22:38:05.436 [Camel (camel-1) thread #1 - ShutdownTask] INFO o.a.c.impl.DefaultShutdownStrategy - Waiting as there are still 2 inflight and pending exchanges to complete, timeout in 299 seconds.
22:38:05.437 [Camel (camel-1) thread #1 - ShutdownTask] DEBUG o.a.c.impl.DefaultShutdownStrategy - There are 1 inflight exchanges:
InflightExchange: [exchangeId=ID-quickstart-cloudera-40574-1441345060577-0-2, fromRouteId=route1, routeId=route1, nodeId=unmarshal1, elapsed=14045, duration=14049]
22:38:08.201 [Camel (camel-1) thread #1 - ShutdownTask] INFO o.a.c.impl.DefaultShutdownStrategy - Waiting as there are still 2 inflight and pending exchanges to complete, timeout in 298 seconds.
22:38:08.202 [Camel (camel-1) thread #1 - ShutdownTask] DEBUG o.a.c.impl.DefaultShutdownStrategy - There are 1 inflight exchanges:
InflightExchange: [exchangeId=ID-quickstart-cloudera-40574-1441345060577-0-2, fromRouteId=route1, routeId=route1, nodeId=unmarshal1, elapsed=16810, duration=16814]
Actually the same program worked for small file but when I try to do with large file I am getting this issue.I think it may problem with Threads .Please Help me out to figure out the issue.
Following is my Program
Main Class
TestRouter myRoute = new TestRouter();
HDFSTransfer hdfsTransfer = new HDFSTransfer();
String copy = hdfsTransfer.copyToLocal(
"hdfs://localhost:8020",
"/user/cloudera/input/CamelTestIn.csv",
"/home/cloudera/Desktop/camelinput/");
boolean flag = false;
if ("SUCCESS".equals(copy)) {
myContext.addRoutes(myRoute);
// Launching the context
myContext.start();
// Pausing to let the route do its work
Thread.sleep(10000);
myContext.stop();
flag = true;
}
if (flag) {
hdfsTransfer.moveFile(
"hdfs://localhost:8020",
"file:/home/cloudera/Desktop/camelout/out.csv",
"/user/cloudera/output/");
}
RouterBuilder Class
{
CsvDataFormat csv = new CsvDataFormat();
from("file:/home/cloudera/Desktop/camelinput/?noop=true&delay=15m")
.unmarshal(csv)
.convertBodyTo(List.class)
.process(new Processor() {
#Override
public void process(Exchange msg) throws Exception {
List<List<String>> data = (List<List<String>>) msg.getIn().getBody();
for (List<String> line : data) {
// Checks if column two contains text STANDARD
// and alters its value to DELUXE.
// System.out.println("line "+line);
/*
if("Aug-04".equalsIgnoreCase(line.get(6))){
line.set(6, "04-August");}
*/
}
}
}).marshal(csv)
.to("file:/home/cloudera/Desktop/camelout/?fileName=out.csv")
.log("done.").end();
}
If you have a bigger file then you need to sleep for longer than 10 seconds to let it have time to process the file.
Also mind that your current route reads the file into memory when means you can run out of memory if the file is very big.
See the lazyLoad option on: http://camel.apache.org/csv.html
Also if all your route is doing is to change some text in a big file, then there is better and faster ways doing that than maybe a Camel route.

Resources