I am trying to append the data to a file which is in SFTP server using Apache Camel SFTP component but data is not appending to the file in SFTP server rather it is creating a new file. I am using this file append in split component. Below is the small snippet used to transfer the data. Any suggestions please?
<to uri="sftp://{{rue.sftp.username}}#{{rue.sftp.host}}:{{rue.sftp.port}}/{{rdx.FileLocation}}?password={{rue.sftp.password}}&fileName=${exchangeProperty.failedFileName}&fileExist=Append" />
Related
I have created a camel (2.20.1) route using groovy DSL. I need to use multicast for 2 endpoints viz. ftps and file. If the order of routes is ftps and file then file is property written on ftp server but on file system empty file is written (with size 0 bytes). If i reverse the order i.e. file and then ftps then file is written on file system properly and empty file is written on ftp server.
It is working fine on Apache Mina FTP server, but with client ftp server it is working as mentioned above.
I have tried both multicast options:
.to("ftps:....").to("file:...")
as well as
.to("ftps:...").to("file:..."))
Also tried parallelProcessing(), but still the same result.
camelContext.addRoutes(new RouteBuilder() {
def void configure() {
from("file:///home/xyz/?fileName=file.txt&charset=utf-8&noop=true")
.multicast()
.to("ftps://localhost:21/files?username=anonymous&password=anonymous&binary=true&fileName=file.txt&passiveMode=true&fileExist=Fail")
.to("file://${directory}?fileName=\${file:name}-\${date:now:yyyyMMddHHmmssSSS}")
}
})
I am expecting that multicast should write the same content to both endpoints without data loss.
See Why is my message empty?. File consumer returns InputStream, which is consumed by first endpoint and thus is empty for the second endpoint. You need to enable Stream Caching, or convert body to some reusable object (e.g. String) before multicasting.
Enable stream caching:
from("file:///home/xyz/?fileName=file.txt&charset=utf-8&noop=true")
.streamCaching()
.multicast()
...
Convert body:
from("file:///home/xyz/?fileName=file.txt&charset=utf-8&noop=true")
.convertBodyTo(String.class)
.multicast()
...
This is my Camel Route:
<route>
<from uri="file:///c:/"/>
<to uri="file:///D:/"/>
</route>
In case of any failure in this route I want to persistently store files in Kaha DB so that files won't be lost. But am not aware of blueprint.xml configuration of Kaha DB persistence for storing files. And my Activemq.xml file is as follows
<broker brokerName="kahaDB_Persistence" persistent="true" useShutdownHook="false">
<persistenceAdapter>
<kahaDB directory="${data}/kahadb/"
journalMaxFileLength="100mb"
concurrentStoreAndDispatchQueues="false"
concurrentStoreAndDispatchTopics="false"/>
</persistenceAdapter>
</broker>
Please advise me how to connect to this KahaDB from blueprint.xml by considering above mentioned route.
Camel's file component has a built in archive feature that saves files that have been processed. It copies them into a folder called ".camel", but it can be changed with a configuration option.
I would not recommend using KahaDB, as it doesn't fit the "right tool for the job" mantra.
Camel File component docs
My use case is to poll a local directory for a list of new files periodically, and then upload them to a FTP server in 1 connection. The Camel route is defined in Spring XML as follows:
<route>
<from uri="file:data/inbox?noop=true&delay=1000&maxMessagesPerPoll=3" />
<to uri="ftp:uid:xxxxx#host:21/data?disconnect=false"/>
</route>
The route is functioning well, except that the FTP connection will retain connected until the FTP server timeout my connection. I hope to reuse the same connection to upload a batch of files and then close the connection immediately when the last file in the batch completed the upload. How can achieve this in Camel?
This is not possible currently. You will need to write some code to do the disconnect yourself.
You are welcome to log a JIRA to enhance this in camel-ftp: https://issues.apache.org/activemq/browse/CAMEL. For example a new option to disconnectOnBatchComplete.
There might be a way but it is not pretty.
You could try to wrap your route based on a cronSchedulePolicy. So say you kick start the route once every hour and poll the directory and send the files. Then you simply add a stop(). Not sure if the stop is exactly the same in the xml dsl. Alternatively, you could also write that onExchangeComplete(new Processor(StopProcessor(routeId)) and inside that processor you via exchange.getContext.stopRoute(routeid) stop the route. Again this depends on your requirements allow you to do this.
<route>
<from uri="file:data/inbox?noop=true&delay=1000&maxMessagesPerPoll=3" />
<to uri="ftp:uid:xxxxx#host:21/data?disconnect=false"/>
<stop/>
</route>
I am using camel version 2.16.0.
I am trying to send some files through ftp camel component and move them to another location after finishing the transmission.
I am using the "ftp://127.0.0.1/folder1/folder2?username=dev_user&passiveMode=true&password=dev_password&maximumReconnectAttempts=500&reconnectDelay=300000&move=folder3" route.
My files are sent properly, but not moved from Folder2 to Folder3 as I would expect after finishing the transmission.
Any thoughts?
Thanks!
Marcelo
Move -option exists only for the consumer (from), not for the producer (to). Camel's FTP-component uses the File2-component, please study the API http://camel.apache.org/file2.html
If however you are using consumer, then it could be e.g. permission issue and you should study the logs.
When I tried to put file to remote server through camel FTP component, I want to customize the PUT behavior to meet my requirement as I did for get file from remote server.
However, I cannot found which method is actually performing put file to remote through FTP component.
Camel uses commons-net library as the FTP client. You can look at that project how you can use it yourself to write you own code to upload a file to a FTP server and do any commands before/after and whatnot.
https://commons.apache.org/proper/commons-net/