I am using quartz and poll enrich to download multiple files at once , one of the configuration used is localWorkDirectory=/tmp/sftp_tmp/ . I see in logs that says
org.apache.camel.component.file.remote.SftpConsumer.processExchange - About to process file: RemoteFile[filename] . But it randomly sometimes places the file in /tmp/sftp_tmp/ instead of the at final destination c:/cameldata at times. I am using camel version 2.24.1
<route id="sftp-read">
<from uri="quartz2://sftp?stateful=true&trigger.repeatInterval=3s"/>
<pollEnrich><spel>file:c:/cameldata?include=SFTPRECFILE&noop=true&idempotent=false&readLock=markerFile</spel></pollEnrich>
<to uri="seda:startSftpExecutor?waitForTaskToComplete=Always&timeout=-1"/>
</route>
<route id="sftp-executor">
<from uri="seda:startSftpExecutor" />
<pollEnrich><spel>{{sftpUrl}}&sendEmptyMessageWhenIdle=true&binary=false&include=BR-.*&maximumReconnectAttempts=3&noop=true&maxMessagesPerPoll=1&localWorkDirectory=/tmp/sftp_tmp/&stepwise=false</spel></pollEnrich>
<choice>
<when>
<simple>${in.body} != null</simple>
<to uri="file:c:/cameldata?fileName=${file:onlyname}&tempPrefix=.tmp"/>
</when>
<otherwise>
<setBody><constant></constant></setBody>
<log loggingLevel="INFO" message="file(s) downloaded successfully" />
<to uri="file:c:/cameldata?fileName=LOADFILE" />
</otherwise>
</choice>
</route>
Related
I am doing batch insertion in mysql using mybatis.I keep adding messages to an ArrayList and fire the query when ArrayList size has reached the batch size.
<route id="billingRoute">
<from uri="activemq:queue:{{billing.queue}}" />
<log message="Message received from billing queue is ${body}"></log>
<unmarshal ref="gsonBilling"></unmarshal>
<bean beanType="com.bng.upload.processors.GetCdr" method="process(com.bng.upload.beans.BillingEvent,${exchange})" />
log message="Multicasting data ${body} to file system and database" />
<multicast>
<pipeline>
<transform>
<method ref="insertionBean" method="billingBatchInsertion"></method>
</transform>
<choice>
<when>
<simple> ${body.size()} == ${properties:batch.size}</simple>
<to uri="mybatis:batchInsertBilling?statementType=InsertList"></to>
<log message="Inserted in billing table : ${in.header.CamelMyBatisResult}"></log>
</when>
</choice>
</pipeline>
<pipeline>
<choice>
<when>
<simple>${properties:billing.write.file} == true</simple>
<setHeader headerName="path">
<simple>${properties:billing.cdr.folder}</simple>
</setHeader>
<log message="Going to write to file : ${body}"></log>
<bean beanType="com.bng.upload.processors.ToFile"
method="process(${exchange},com.bng.upload.beans.BillingCdr)" />
<to uri="file://?fileExist=Append"></to>
</when>
</choice>
</pipeline>
</multicast>
</route>
</routeContext>
We are not using transactions as there are multiple components included in route -Files,ActiveMq and Database. The issue is that when the tomcat is restarted,if the ArrayList size has not reached the batch size,those messages get lost.Is there any solution to it ?
I have a requirement where i want to use mutlicast in Apache Camel for than single time in a single route. i.e Multicast within a multicast.
<routeContext id="myRoute" xmlns="http://camel.apache.org/schema/spring">
<route id="myRouteId">
<from uri="activemq:queue:{{XXXX.queue}}" />
....
<multicast parallelProcessing="true">
<pipeline>
##everything working fine here
</pipeline>
<pipeline>
<multicast>
<pipeline>
<log message="Inserting in database now"></log>
<transform>
<method ref="insertBean" method="myBatchInsertion"></method>
</transform>
<choice>
<when>
<simple>${in.header.myCount} == ${properties:batch.size} </simple>
<to uri="sql:{{sql.core.insertMyQuery}}?batch=true"></to>
<log message="Inserted rows ${body}"></log>
</when>
</choice>
</pipeline>
</multicast>
</pipeline>
</multicast>
</route>
</routeContext>
Is it possible to do that?
When i am trying to do that, my program is not getting executed successfully.
Is the unsuccessful execution is a result of mulitple multicast?
Can anybody help?
I got the reference from following link:
http://camel.apache.org/multicast.html
Why do you use pipeline? It "is" pipeline by default.
Also all the log and transform and choice statements can be put outside of the multicast. And since you are generating your endpoints dynamically, put the values in a header and use recipientlist for dynamic endpoints. Multicast is for hard-coded endpoints. Here is an example:
<route>
<from uri="direct:a" />
<!-- use comma as a delimiter for String based values -->
<recipientList delimiter=",">
<header>myHeader</header>
</recipientList>
</route>
http://camel.apache.org/recipient-list.html
Is there any tool available that can convert java DSL to XML route or vice versa.
I want to convert the following XML route into Java DSL
<route id="test">
<from uri="file://{{VAR_DATA_PATH}}/test/xml"/>
<multicast>
<choice>
<when>
<xpath>/bookinfo</xpath>
<doTry>
<to uri="downloadBook"/>
<marshal ref="xstream-utf8"/>
<to uri="another URI"/>
<doCatch>
<exception>java.lang.Exception</exception>
<handled>
<constant>false</constant>
</handled>
<to uri="3rd URI"/>
</doCatch>
</doTry>
</when>
<when>
<xpath>somePath</xpath>
<to uri="4th URI" />
<to uri="5th URI"/>
</when>
</choice>
<to uri="6th URI" />
</multicast>
</route>
From Java DSL to XML, it's fairly easy. You can use Hawtio or karaf's "route-info" command. Even though the routes are in Java DSL, when you view them it would be XML.
I'm not aware of the other way around (from XML to Java) but it's not difficult at all to do it yourself.
I am new to Camel and I am trying to do the following.
I want to process the same message twice. I have to transform both messages and process one message first and then second only if the first one is sucessfully executed (I have a condition).
I tried to use a multicast first. Then I transform the messages in each route. The first one (the operation_DC) only sends a successfull message to the second one (the operation_AC) only if it is sucessfull. The second operation contains an aggregation that waits until it times out for both messages. I only want to process the messaged from the multicast and discard the other.
<route id="t_operation_ME">
<from uri="direct:operation_ME" id="t.direct.aggregatedService"></from>
<setHeader headerName="id">
<simple>exchangeId</simple>
</setHeader>
<multicast parallelProcessing="true" strategyRef="defaultAggregationStrategy" stopOnException="true" onPrepareRef="cleanHeader" parallelAggregate="false" completionPredicate="">
<to uri="direct:operation_DC"></to>
<to uri="direct:operation_AC"></to>
</multicast>
</route>
<route id="direct:operation_DC">
<from uri="direct:operation_DC" />
<log message="ENTER DC"></log>
<to uri="xslt:{{depasse:core.transformation.xml.path}}client/t/Operation_toDC_request.xsl" id="t.dc.transform.productos" />
<to uri="activemq:QCIn" id="t.dc.qcin.queue.send"></to>
<log message="EXIT DC ${body}"></log>
<choice>
<when>
<xpath>//Data/Status[. = 'OK']</xpath>
<log message="SEND TO AC"></log>
<to uri="direct:operation_AC"></to>
</when>
</choice>
</route>
<route id="direct:operation_AC">
<from uri="direct:operation_AC" />
<log message="ENTER AC"></log>
<aggregate completionTimeout="20000" completionSize="2" discardOnCompletionTimeout="true" strategyRef="tAggregationStrategy">
<correlationExpression>
<simple>header.id</simple>
</correlationExpression>
<log message="ENTER AGG AC ${body}"></log>
<to uri="xslt:{{depasse:core.transformation.xml.path}}client/t/Operation_toAC_request.xsl" id="t.ac.transform.productos" />
<to uri="activemq:QCIn" id="t.ac.qcin.queue.send"></to>
<log message="EXIT AC ${body}"></log>
</aggregate>
<log message="END AC\n${body}"></log>
</route>
The thing is that when I log "EXIT AC" and "END AC" the message is different. This means that while in the server I am watching the process correctly, the AC client receives an incorrect message.
Multicast send copy of messages always to every endpoint between tags. If you want to call route operation_AC only after operation_DC run successfully, you shouldn't use multicast, but you should save your body at the beginning of operation_DC, for example using property and Camel Simple:
<from uri="direct:operation_DC" />
<setProperty propertyName="body">
<simple>${body}</simple>
</setProperty>
and before calling operation_AC set body to bring message body from the beginning:
<setBody>
<simple>${property.body}</simple>
</setBody>
<to uri="direct:operation_AC"/>
Sorry for my english, I'll make my best effort for write this.
I have a xml file like this:
<routes>
<route id="id1">
<from uri="timer://foo?fixedRate=true&period=60s"/>
<onCompletion mode="BeforeConsumer">
<log message="finish"/>
</onCompletion>
<setBody>
<simple>Probando 1</simple>
</setBody>
<log message="working1"/>
<to uri="direct:test1"/>
</route>
<route id="id2">
<from uri="direct:test1"/>
<setBody>
<simple>Test2</simple>
</setBody>
<log message="working2"/>
<to uri="direct:test2"/>
</route>
<route id="id3">
<from uri="direct:test2"/>
<setBody>
<simple>Test3</simple>
</setBody>
<log message="working3"/>
</route>
</routes>
if remove mode="BeforeConsumer" from the first route i get one "finish" in the log, but if i use the xml with mode="BeforeConsumer" i get the message 3 times (one per each route). Is that the behavior expected?
I need that the onCompletion code runs one time but i need too the mode="BeforeConsumer"