I have a Camel route which, when a bit simplified, it boils down to the following one:
<bean id="myProcessor" class="com.acme.MyProcessor" />
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route>
<from
uri="file:/home/inbox?fileName=file.txt&noop=true" />
<split>
<tokenize token="#" />
<process ref="myProcessor" />
</split>
<to
uri="file:/home/outbox" />
</route>
</camelContext>
To my surprise I have found that even if the Processor is being invoked, it is not able to change the single tokens. For example:
public class MyProcessor implements Processor {
public void process(Exchange exchange) throws Exception {
String myString = exchange.getIn().getBody(String.class);
exchange.getIn().setBody(myString.toUpperCase());
}
}
In the end, the file produced with the tokens is not altered by the Processor. Why ?
Try to modify your route like this:
<route>
<from
uri="file:/home/inbox?fileName=file.txt&noop=true" />
<split>
<tokenize token="#" />
<process ref="myProcessor" />
<to uri="file:/home/outbox?fileExist=Append" />
</split>
</route>
IMHO, you split your file, processed it with processor and don't have anything to do with the results.
Related
I have a project with Camel and my route has a recursive call to itself in order to implement logic "call stored procedure while it returns a data set":
<route id="trxnReader">
<from uri="direct:query"/>
<to uri="sql-stored:classpath:sql/getTrxnsProcedure.sql?dataSource=myDataSource"
id="storedprocGetTrxns"/>
<choice>
<when>
<simple>${body} != null</simple>
<split>
<simple>${body.transactions}</simple>
<filter>
<method ref="trnxFilter" method="filter"/>
<to uri="direct:processTrxn"/>
</filter>
</split>
<to uri="direct:query"/>
</when>
<otherwise>
<log id="failUploadInfo" message="Transactions don't exist" loggingLevel="INFO"/>
</otherwise>
</choice>
</route>
The problem with this code is that if my stored procedure constantly returns something for a long time not allowing to exit the recursion, I get java.lang.StackOverflowError. I need something like loop. What is the best way to implement such logic with Camel? I'm using Camel 2.15.3.
I found a workaround with custom bean and loop exit condition (isLoopDone property)
public class LoopBean {
#Handler
public void loop(#ExchangeProperty("loopEndpoint") String endpoint, Exchange exchange) {
ProducerTemplate producerTemplate = exchange.getContext().createProducerTemplate();
boolean isLoopDone = false;
Exchange currentExchange = exchange;
do {
currentExchange = producerTemplate.send(endpoint, currentExchange);
Object isLoopDoneProperty = currentExchange.getProperty("isLoopDone");
if (isLoopDoneProperty != null) {
isLoopDone = (boolean) isLoopDoneProperty;
}
}
while (!isLoopDone);
}
}
<route id="trxnReader">
<from uri="direct:query"/>
<setProperty propertyName="loopEndpoint">
<simple>direct:callStoredProcWhileItHasTransactions</simple>
</setProperty>
<bean ref="loopBean"/>
</route>
<route id="storedProcCallingLoop">
<from uri="direct:callStoredProcWhileItHasTransactions"/>
<to uri="sql-stored:classpath:sql/getTrxnsProcedure.sql?dataSource=myDataSource"
id="storedprocGetTrxns"/>
<choice>
<when>
<simple>${body} != null</simple>
<split>
<simple>${body.transactions}</simple>
<filter>
<method ref="trnxFilter" method="filter"/>
<to uri="direct:processTrxn"/>
</filter>
</split>
</when>
<otherwise>
<log id="failUploadInfo" message="Transactions don't exist" loggingLevel="INFO"/>
<setProperty propertyName="isLoopDone">
<simple resultType="java.lang.Boolean">true</simple>
</setProperty>
</otherwise>
</choice>
</route>
Hi I have a route which is
<route id="invokeGetMortgageAccountDetails">
<from uri="direct:invokeGetMortgageAccountDetails" />
<removeHeaders pattern="operationNamespace" />
<setHeader headerName="operationName">
<constant>getMortgageDetailsRequest</constant>
</setHeader>
<to uri="cxf:bean:getBastionAcctDetailsClient" />
<removeHeaders pattern="*" />
</route>
Now i want to change the 'to uri' if length of parameter account is equal to 8.
I am new to Apache camel and there is not very helpful information on internet.
I am using camel version 2.15 and i tried passing an extra property called length of account number in exchange and tried to match with value in route but it did not work.
Processor:
public void processMortgage(final Exchange exchange) throws
ServiceException { MessageContentsList messageContentsList =
(MessageContentsList) exchange.getIn().getBody(); List
paramsList = new ArrayList(); String systemID =
messageContentsList.get(0).toString().trim(); String brandID =
messageContentsList.get(1).toString().trim(); String account =
messageContentsList.get(2).toString().trim(); String len =
Integer.toString(account.length()); paramsList.add(Constants.HUB);
paramsList.add(brandID.toUpperCase()); paramsList.add(account);
exchange.setProperty(Constants.SystemID, systemID);
exchange.setProperty(len, len);
exchange.setProperty(Constants.ErrorCode, null);
exchange.setProperty("mortgageAccountNumber",
Integer.parseInt(account)); }
exchange.getIn().setBody(paramsList); }
Route Config:
<route id="invokeGetMortgageAccountDetails">
<from uri="direct:invokeGetMortgageAccountDetails" /> <removeHeaders pattern="operationNamespace" />
<setHeader headerName="operationName">
<constant>getMortgageDetailsRequest</constant> </setHeader> <choice>
<when>
<simple>${body.len} == '8'</simple>
<to uri="cxf:bean:getPhoebusClient" />
</when>
<otherwise>
<to uri="cxf:bean:getBastionAcctDetailsClient" />
</otherwise>
</choice>
<removeHeaders pattern="*" />
</route>
If you are using Apache Camel version > 2.16 then you can use the
Dynamic To Endpoint
You will probably need to use Spring Expression Language to build your dynamic uri
My xml is given below
<camelContext trace="false" xmlns="http://camel.apache.org/schema/spring">
<propertyPlaceholder id="placeholder" location="classpath:application.properties" />
<!--Route:1 for POLLUX Data Processing -->
<route id="processPolluxData-Route" startupOrder="1">
<from uri="{{POLLUX_INPUT_PATH}}?noop=true"/>
<unmarshal ref="csvBindyDataformatForPolluxData"/>
<camel:bean ref="polluxDataController" method="processPolluxData"/>
<camel:log message="Line:${body}" loggingLevel="INFO"/>
<to uri="sqlComponent:{{sql.insertPolluxData}}?batch=true" />
</route>
<!-- Route:2 for RSI Data Processing -->
<route id="processRsiData-Route" startupOrder="2">
<from uri="{{RSI_INPUT_PATH}}?noop=true"/>
<unmarshal ref="csvBindyDataformatForRsiData"/>
<camel:bean ref="rsiDataController" method="processRsiData"/>
<camel:log message="Line:${body}" loggingLevel="INFO"/>
<to uri="sqlComponent:{{sql.insertRsiData}}?batch=true" />
</route>
<!-- Route for Global Data Processing -->
<route id="processGlobalData-Route" >
<from uri="sqlComponent:{{sql.selectOrder}}?consumer.useIterator=false" />
<camel:bean ref="globalDataController" method="processGlobalData" />
<marshal>
<csv delimiter=","/>
</marshal>
<log message="${body}" />
<setHeader headerName="camelFilename">
<constant>result.csv</constant>
</setHeader>
<to uri="{{GLOBAL_OUTPUT_PATH}}?fileExist=Append" />
</route>
My sql statement is
sql.selectOrder=select STID,CLLTR,SOURCE from GSI_DEVL.POLLUX_DATA
bean class for processing result set is
public class GlobalDataController {
List<Map<String, Object>> globalStationProccessedList = new ArrayList<Map<String, Object>>();
List<Map<String, Object>> globalStationMap = new ArrayList<Map<String, Object>>();
#SuppressWarnings("unchecked")
public List<Map<String, Object>> processGlobalData(Exchange exchange) throws Exception {
// System.out.println("Processing " + exchange.getIn().getBody());
globalStationMap = (List<Map<String, Object>>) exchange.getIn().getBody();
globalStationProccessedList.addAll(globalStationMap);
return globalStationProccessedList;
}
}
Problem now is Route 1 data is transffered to csv file with exact number of rows in database.But no data in the route 2 is append to the csv file
I am using camel 2.16
If the problem only in large number of files (not in the file format), then here is the solution:
<route id="processOrder-route">
<from uri="sqlComponent:{{sql.selectOrder}}"/>
<camel:bean ref="controllerformarshalling" method="processGlobalData" />
<marshal >
<csv delimiter="," useMaps="true" > </csv>
</marshal>
<log message="${body}"/>
<setHeader headerName="CamelFileName">
<constant>result.csv</constant>
</setHeader>
<to uri="file://D://cameltest//output&fileExist=Append" />
</route>
For next pool you can set another file name, based on current time, maybe.
Have you tried setting this parameter on the sql component?
consumer.useIterator boolean true
Camel 2.11: SQL consumer only: If true each row returned when
polling will be processed individually. If false the entire
java.util.List of data is set as the IN body.
Try setting this to false. That way you should get the entire sql resultset into a single list and then write the entire list to a single file.
I want to send a massage in the form of csv file to webservice endpoint, split message to process each csv row separately, aggregate checked exceptions, and send a response with exceptions summary:
My route is:
<route>
<from uri="cxf:bean:MyEndpoint" />
<split strategyRef="myAggregateStrategy" >
<tokenize token="\n" />
<unmarshal>
<csv delimiter=";" />
</unmarshal>
<process ref="MyProcessor" />
<to uri="bean:myWebservice?method=process" />
</split>
</route>
How can I do that? Response must be send to webservice
How about using <doTry> and <doCatch> within your logic? You could have whatever logic you want inside the catch, e.g. a bean to handle/aggregate/summarize the exceptions.
Something roughly like this:
<route>
<from uri="cxf:bean:MyEndpoint" />
<split strategyRef="myAggregateStrategy" >
<doTry>
<tokenize token="\n" />
<unmarshal>
<csv delimiter=";" />
</unmarshal>
<process ref="MyProcessor" />
<to uri="bean:myWebservice?method=process" />
<doCatch>
<exception>java.lang.Exception</exception>
<handled>
<constant>true</constant>
</handled>
<bean ref="yourExceptionHandlingBean" method="aggregateException"/>
</doCatch>
</doTry>
</split>
</route>
I finally found solution to my problem. I used aggregator and in case of exception, a collect it on the list in old exchange body and remove exception from new exchange:
public class ExceptionAggregationStrategy implements AggregationStrategy {
#Override
public Exchange aggregate(Exchange oldExchange, Exchange newExchange) {
Object body = newExchange.getIn().getBody(String.class);
Exception exception = newExchange.getException();
if (exception != null) {
newExchange.setException(null); // remove the exception
body = exception;
}
if (oldExchange == null) {
List<Object> list = new ArrayList<>();
list.add(body);
newExchange.getIn().setBody(list);
return newExchange;
}
#SuppressWarnings("unchecked")
List<Object> list = oldExchange.getIn().getBody(List.class);
list.add(body);
return oldExchange;
}
}
The list is of type java.lang.Object because I collect original message too (in case of there is no excepton).
I have the following routes:
<camelContext xmlns="http://camel.apache.org/schema/spring">
<threadPoolProfile id="defaultProfile"
defaultProfile="true" poolSize="100" maxPoolSize="200" />
<route>
<from uri="amq:example.MyQueue" />
<setHeader headerName="myRoutingSlipHeader">
<constant>amq:one#amq:two#amq:three#amq:four</constant>
</setHeader>
<log message="Makan" />
<setExchangePattern pattern="InOut" />
<routingSlip uriDelimiter="#">
<header>myRoutingSlipHeader</header>
</routingSlip>
<setExchangePattern pattern="InOnly" />
<log message="End: ${body}" />
</route>
<route>
<from uri="amq:one" />
<to uri="bean:helloBean?method=stepOne" />
</route>
<route>
<from uri="amq:two" />
<to uri="bean:helloBean?method=stepTwo" />
</route>
<route>
<from uri="amq:three" />
<to uri="bean:helloBean?method=stepThree" />
</route>
<route>
<from uri="amq:four" />
<to uri="bean:helloBean?method=stepFour" />
</route>
</camelContext>
<bean id="amq" class="org.apache.activemq.camel.component.ActiveMQComponent"
p:brokerURL="tcp://localhost:61616" p:transacted="true"
p:cacheLevelName="CACHE_CONSUMER" p:concurrentConsumers="20"
p:maxConcurrentConsumers="500" p:idleConsumerLimit="10"
/>
Given that example.MyQueue is preloaded with 1000 messages, and each hello bean's step* methods takes 250ms, when I do camel:run, the performance is still bad. It prints "End: ..." each 1 secs in sequence not parallel. What would be the problem here?
In the following much simple case, I see a strange behavior. When there is no JMS producer putting messages into the queue, the printings happen in sequence. But when there is, the printings happen in parallel. What's the explanation?
<threadPoolProfile id="defaultProfile"
defaultProfile="true" poolSize="100" maxPoolSize="200" />
<route>
<from uri="amq:example.MyQueue" />
<delay>
<constant>1000</constant>
</delay>
<log message="End: ${body}" />
</route>
<bean id="amq" class="org.apache.activemq.camel.component.ActiveMQComponent"
p:brokerURL="tcp://localhost:61616" p:transacted="true"
p:cacheLevelName="CACHE_CONSUMER" p:concurrentConsumers="20"
p:maxConcurrentConsumers="500" p:idleConsumerLimit="10"
/>
Try replacing
<from uri="amq:example.MyQueue" />
with
<from uri="amq:example.MyQueue?concurrentConsumers=200&maxConcurrentConsumers=500" />
The routing slip runs in sequence, and yo do request/reply over JMS (eg the MEP is InOut) so processing one message would take
call amq:one = 250 millis (request/reply)
call amq:two = 250 millis (request/reply)
call amq:three = 250 millis (request/reply)
call amq:four = 250 millis (request/reply)
A total of 1 sec per message.
The AMQ route in < from > could process messages in parallel though. But each message would still take 1 second to process.
I guess routingSlip pattern is synchronized. you need something asynchronized component to handle this. Please check this:http://camel.apache.org/async.html
Just one question, why you need set the ExchangePattern?