I have a camel route consuming from an FTP server and storing any files it consumes to a directory with move=.dealtWith. However, the number of files in this .dealtWith directory can quickly become unmanageable for users to view, so I would like to move the file to a .dealtWith/{the_date} directory. Is there a way to specify this functionality in camel without bringing the route down?
Use Camel Simple Expression Language
ftp:url?move=.dealtWith/${date:now:yyyy-MM-dd}/${header.CamelFileName}
Related
I have two separate routes running in an application and I want to control the total amount of work in flight across the entire path.
Route 1: Gzipped file on SFTP --> unzip --> local directory
Route 2: local directory --> process stuff --> Kafka
If route 2 has problems or falls behind in its work, I don't want route 1 to fill up the local directory. How can I limit the total number of files sitting in local dir waiting to be processed?
(if it was a single route I may be able to throttle() easier, but are there other options to look at the overall picture of multiple routes?)
You can implement a custom RoutePolicy where you check the number of files in that directory and if its bigger than X then suspend the route, and resume it again if its lower than X.
See more details at the Camel docs: http://camel.apache.org/routepolicy.html
You can look at the existing ThrottlingInflightRoutePolicy how its implemented for inspiration.
I am using camel version 2.16.0.
I am trying to send some files through ftp camel component and move them to another location after finishing the transmission.
I am using the "ftp://127.0.0.1/folder1/folder2?username=dev_user&passiveMode=true&password=dev_password&maximumReconnectAttempts=500&reconnectDelay=300000&move=folder3" route.
My files are sent properly, but not moved from Folder2 to Folder3 as I would expect after finishing the transmission.
Any thoughts?
Thanks!
Marcelo
Move -option exists only for the consumer (from), not for the producer (to). Camel's FTP-component uses the File2-component, please study the API http://camel.apache.org/file2.html
If however you are using consumer, then it could be e.g. permission issue and you should study the logs.
I have to implement an client application that need to sync IMAP messages. But currently IMAP server that store messages are not ready yet and i need to finish my task. So i am using to Gmail for testing my implementation of retrieving IMAP messages.
I simulate what the spec said for the folder structures in Gmail. Creating folder and subfolders.
/Default
/Default/User1/Session1
/Default/User1/Session1/File1
/Default/User1/Session2
/Default/User2/Session1
/Default/User3
Then i use java mail to list out the folders under default.
store.getFolder("Default").list("*");
I loop the folder to print out the folder count and folder name. The returned folder includes all the subfolders of each User like above.
I would like to know is this the way that IMAP server return for the folder list query? all the subfolders under it?
If you use list(*"), yes. If you only want one level use list("%"). This is well explained in the javadocs.
I am currently working on a Camel based test harness application, that processes groups of files from multiple folders and compares with the files present in the local repository.
Is there any way to change the folder location from the end point in Camel route dynamically? I want to use a single route for polling files from the multiple folders.
According dynamic change endpoint camel, use following procedure:
stop the route
remove the route
change the endpoint
add the route
start the route
Can i download with camel a specific file list from a sftp server and then shutdown the service?
I know this should be a common question but i can't figure out how to do it without waiting the context stopping.
In some way, camel can ensure data integrity?
I guess you can do something like this using a direct route, pollEnrich and a template
from("direct:grabOneFile")
.pollEnrich("sftp://somewhere/blah/blah?fileName=foobar");
then from some java code somewhere, just grab a camel template and invoke the "direct:grabOneFile route.
String ret = template.requestBody("direct:grabOneFile","",String.class);
In this case, you don't have to worry about when to shut down the camel context with the chance of having multiple files etc.
Camel ftp component can only poll directories.
You can use a combination of maxMessagesPerPoll and fileName, like
from("ftp://.../xyz?maxMessagesPerPoll=x&fileName=y");
Also take a look at this.
This link has examples regarding shutdown.
Did you check this out at the bottom of the Camel FTP page.