I want to copy contents from one file in input folder to a new file in output folder using Apache camel xml. When this didn't work I tried to simply set the body and copy it to a new file. The log message is displayed correctly and there are no errors but no output file is created.
<route id="route1">
<from uri="timer://tenSecondsTimer?repeatCount=1"/>
<setBody>
<simple>Today is thursday</simple>
</setBody>
<log message="This is the message: ${body}"/>
<to uri="file:/output"/>
</route>
Also tried this syntax -
<to uri="file:///c:/Users/<route_to_output_folder>/output"/>
Code I tried for file to file transfer -
<route id="route1">
<from uri="file:/input/?fileName=file1.txt"/>
<log message="This is the message: ${body}"/>
<to uri="file:/output"/>
</route>
Related
I am using quartz and poll enrich to download multiple files at once , one of the configuration used is localWorkDirectory=/tmp/sftp_tmp/ . I see in logs that says
org.apache.camel.component.file.remote.SftpConsumer.processExchange - About to process file: RemoteFile[filename] . But it randomly sometimes places the file in /tmp/sftp_tmp/ instead of the at final destination c:/cameldata at times. I am using camel version 2.24.1
<route id="sftp-read">
<from uri="quartz2://sftp?stateful=true&trigger.repeatInterval=3s"/>
<pollEnrich><spel>file:c:/cameldata?include=SFTPRECFILE&noop=true&idempotent=false&readLock=markerFile</spel></pollEnrich>
<to uri="seda:startSftpExecutor?waitForTaskToComplete=Always&timeout=-1"/>
</route>
<route id="sftp-executor">
<from uri="seda:startSftpExecutor" />
<pollEnrich><spel>{{sftpUrl}}&sendEmptyMessageWhenIdle=true&binary=false&include=BR-.*&maximumReconnectAttempts=3&noop=true&maxMessagesPerPoll=1&localWorkDirectory=/tmp/sftp_tmp/&stepwise=false</spel></pollEnrich>
<choice>
<when>
<simple>${in.body} != null</simple>
<to uri="file:c:/cameldata?fileName=${file:onlyname}&tempPrefix=.tmp"/>
</when>
<otherwise>
<setBody><constant></constant></setBody>
<log loggingLevel="INFO" message="file(s) downloaded successfully" />
<to uri="file:c:/cameldata?fileName=LOADFILE" />
</otherwise>
</choice>
</route>
I am processing file in cluster environment. The cluster works fine. It is being processed on Only one server.
But on the second server It identifies as duplicates but still execute the form route delete=true
ERROR:
org.apache.camel.component.file.GenericFileOperationFailedException: Cannot delete file:
I am setting header CamelRouteStop to true but the exchange still try's to delete a file, instead of stop executing the route.
All I need is to end the route if it is duplicate.
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="smb:url?delete=true"/>
<idempotentConsumer messageIdRepositoryRef="myRepo">
<header>messageId</header>
<setHeader headerName="fileExist">
<simple>true</simple>
</setHeader>
</idempotentConsumer>
<when>
<simple>${header.fileExist} == null</simple>
<log message="File ${header.CamelFileName} processing/processed by other Nodes - DUPLICATE" loggingLevel="INFO" />
<setHeader headerName="CamelRouteStop">
<simple <simple resultType="java.lang.Boolean">true</simple>>true</simple>
</setHeader>
</when>
</route>
</camelContext>
For CamelRouteStop you need to use setProperty, not setHeader.
I am using Apache Camel blueprints, my route is triggered from a URL at port 8081 on localhost. This route generates a file but the problem is the file is a binary file without the HTML showing.
I then navigate my browser to http://localhost:8081/foo which triggers the URL.
What is it that I don't understand that is causing it to output a binary file rather than a text file? I guess that I must transform the body somehow?
<route id="url1">
<from uri="netty4-http:http://0.0.0.0:8081/foo" />
<to uri="http://www.google.com/?bridgeEndpoint=true" />
<to uri="file:/test"/>
</route>
Update: I think the problem is that the content coming back from www.google.com is gzip. When I look at the bridgeEndpoint parameter in the documentation it mentions something about gzip... now I am not sure, as I tried it on another website and it still doesn't work.
This line in my log might be relevant.
writing body: DefaultFullHttpResponse(decodeResult: success,
version: HTTP/1.1, content: UnpooledUnsafeDirectByteBuf(ridx: 0, widx: 0, cap: 0))
Update: I discover if I do:
<from uri="timer:secondfoo?period=20s" />
Replacing the from, then it works. Hmm... Something flowing from the netty4-http causes problem.
Update: I found something which works! Obsession pays off.
<route id="url1">
<from uri="netty-http:http://0.0.0.0:8081/foo" />
<removeHeaders pattern="*" />
<setBody>
<simple></simple>
</setBody>
<setHeader headerName="CamelHttpMethod">
<constant>GET</constant>
</setHeader>
<to uri="http://www.google.com/?bridgeEndpoint=true" />
<to uri="file:/test"/>
</route>
You can convert the message to a String type then its text based
<route id="url1">
<from uri="netty4-http:http://0.0.0.0:8081/foo" />
<to uri="http://www.google.com/?bridgeEndpoint=true" />
<convertBodyTo type="String"/>
<to uri="file:/test"/>
</route>
I wrote the following route and expected that the bean 'teaserService' should be called only one time, at the end of processing of all files, but ... it's called after processing of each file:
<route id="teaserInterface">
<from
uri="file://{{teaser.dropInDir}}?readLock=changed&delete=true&delay=60000" />
<choice>
<when>
<simple>${file:ext} == 'properties'</simple>
<to uri="file://{{teaser.config.directory}}" />
</when>
<when>
<simple>${file:ext} == 'jpg' || ${file:ext} == 'JPG'</simple>
<to uri="sftp://{{apache.ftp.user}}#{{apache.ftp.host}}/{{apache.teaser.ftp.targetDir}}?password={{apache.ftp.password}}&binary=true&separator=UNIX" />
</when>
<otherwise>
<transform>
<simple>Dear user,\n\n the Teaser interface only accept *.jpg and *.properties files, but we found the file ${file.name}.\n\n Have a nice day,\nYour lovely Teaser interface</simple>
</transform>
<to
uri="smtp://smtp.blabla.com?contentType=text/plain&from=blabla#blabla.com&to=chica#chicas.com&subject=A problem occured while setting up new teaser!" />
</otherwise>
</choice>
<bean ref="teaserService" method="updateTeaser" />
</route>
How to achieve such a behavior?
Thanks
The Camel file compoment is a batch consumer and adds properties to the exchange regarding the batch it is processing. You can test for the property CamelBatchComplete and if that is set to true, then call your bean.
If you want to proceed only after all files have been read, you must sample them somehow. This can be achieved using the aggregator pattern:
<route>
<from uri="file://src/data/aggregate-and-process?readLock=changed&delete=true&delay=60000" />
<aggregate strategyRef="aggregationStrategy" completionFromBatchConsumer="true">
<correlationExpression>
<constant>true</constant>
</correlationExpression>
<to uri="direct:sub" />
</aggregate>
</route>
<route>
<from uri="direct:sub" />
<!-- processing aggregated body -->
</route>
Please note that I set completionFromBatchConsumer="true". From the Camel documentation:
This option is if the exchanges are coming from a Batch Consumer. Then when enabled the Aggregator2 will use the batch size determined by the Batch Consumer in the message header CamelBatchSize. [...] This can be used to aggregate all files consumed from a File endpoint in that given poll.
We are using the below configuration to send a file from single source to multiple remote destinations.
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:camel="http://camel.apache.org/schema/spring"
xmlns:util="http://www.springframework.org/schema/util"
xsi:schemaLocation="
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd
http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd">
<routeContext id="gcgRatesOutbound" xmlns="http://camel.apache.org/schema/spring">
<route id="gcgRatesFileOut">
<from uri="file:{{nas.root}}/{{gcg.out.prices.dir}}?delay={{poll.delay}}&initialDelay={{initial.delay}}&readLock=rename&scheduledExecutorService=#scheduledExecutorService" />
<multicast stopOnException="true">
<to uri="scp://{{gcg.ste.Client1_User_Name}}#{{gcg.ste.Host_Name}}:{{gcg.ste.Port_Number}}/{{gcg.ste.Destination_Client1}}?knownHostsFile={{ssh.knownHosts}}&privateKeyFile={{ssh.privateKey}}" />
<to uri="scp://{{gcg.ste.Client2_User_Name}}#{{gcg.ste.Host_Name}}:{{gcg.ste.Port_Number}}/{{gcg.ste.Destination_Client2}}?knownHostsFile={{ssh.knownHosts}}&privateKeyFile={{ssh.privateKey}}" />
<to uri="scp://{{gcg.ste.Client3_User_Name}}#{{gcg.ste.Host_Name}}:{{gcg.ste.Port_Number}}/{{gcg.ste.Destination_Client3}}?knownHostsFile={{ssh.knownHosts}}&privateKeyFile={{ssh.privateKey}}" />
</multicast>
</route>
</routeContext>
</beans>
Using the above confirguration the file reaches the remote destinations, which confirmed that the connection to all the remote destinations were successfull.
We need that the file should be moved to the archive folder after the file has been successfully transfered to all the remote destinations.
And should move to error folder incase of any error.
However, when I add the archival code ( element) as child element to the multicast element in the above configuration and use the and the tag to move the file to error folder incase of an error. The file does not reach the remote destinations.
<doTry>
<multicast stopOnException="true" parallelProcessing="true">
<to uri="scp://{{gcg.ste.Client1_User_Name}}#{{gcg.ste.Host_Name}}:{{gcg.ste.Port_Number}}/{{gcg.ste.Destination_Client1}}?knownHostsFile={{ssh.knownHosts}}&privateKeyFile={{ssh.privateKey}}" />
<to uri="scp://{{gcg.ste.Client2_User_Name}}#{{gcg.ste.Host_Name}}:{{gcg.ste.Port_Number}}/{{gcg.ste.Destination_Client2}}?knownHostsFile={{ssh.knownHosts}}&privateKeyFile={{ssh.privateKey}}" />
<to uri="scp://{{gcg.ste.Client3_User_Name}}#{{gcg.ste.Host_Name}}:{{gcg.ste.Port_Number}}/{{gcg.ste.Destination_Client3}}?knownHostsFile={{ssh.knownHosts}}&privateKeyFile={{ssh.privateKey}}" />
<to uri="file://{{nas.root}}/{{gcg.out.prices.dir}}?fileName={{archive.dir}}" />
</multicast>
<doCatch>
<exception>java.lang.Exception</exception>
<handled>
<constant>true</constant>
</handled>
<to uri="file://{{nas.root}}/{{gcg.out.prices.dir}}?fileName={{error.dir}}" />
</doCatch>
</doTry>
The file does not reach the remote destinations nor does it produce any log and the file is moved to archive folder.
I tried placing the remote destinations, the archival code and the move to error code in seperate routes and have its reference in a single multicast as below
<to uri="direct:Client1FileOut" />
<to uri="direct:Client2FileOut" />
<to uri="direct:Client3FileOut" />
<to uri="direct:MoveToArchive" />
<route id="gcgFileOut1">
<from uri="direct:Client1FileOut" />
<doTry>
<to uri="scp://{{gcg.ste.Client1_User_Name}}#{{gcg.ste.Host_Name}}:{{gcg.ste.Port_Number}}/{{gcg.ste.Destination_Client1}}?knownHostsFile={{ssh.knownHosts}}&privateKeyFile={{ssh.privateKey}}" />
<doCatch>
<exception>java.lang.Exception</exception>
<handled>
<constant>true</constant>
</handled>
<to uri="direct:gcgError" />
</doCatch>
</doTry>
</route>
However, The file does not reach the remote destinations nor does it produce any log and the file is moved to archive folder.
I am new to camel.
I tried using the shareUnitOfWork attribute as below
<multicast shareUnitOfWork="true">
However, the file moved to archive folder along with Test_2_19082013_3.txt.camelLock file
Not sure why this file is also moved to archive when the file dropped was Test_2_19082013_3.txt
Also there is folder named "ARCHIV~1" created in the drop location.
Something might went wrong when moving the file to archive. You have stopOnException="true" parallelProcessing="true" and the local file is probably done first as it should be the fastest.
You probably want to print the error somewhere. Now, you catch the exception and mark the error as handled. Still you expect logs. Use the log component and output some log statements manually. Not only in case or error but also in case of success. You can log in the debug level so that you can manually enable log outprint in case you need it - like now.
Another option for you to figure out what's going on is to enable trace which will make you less "blind".
That said about debugging on your own -
You are archiving and storeing error messages in the input directory. The lock file you see is when Camel is reading. This is likely what's casuing your application to malfunction.
{{nas.root}}/{{gcg.out.prices.dir}}
The fileName=... is the filename, not another sub directory.
So: file://{{nas.root}}/{{gcg.out.prices.dir}}/{{archive.dir}} should do it (likewise for the error path).