As per https://camel.apache.org/file2.html, file component cannot be configured with dynamic input/output folder locations?
Is there any workaround/alternative for the same?
Thanks.
Regards
Senthil Kumar Sekar
For the file consumer
You can use a customer filter as dynamic selection of which directories/files to pickup. But the starting directory is configured once (hardcoded).
For that to change you would need to stop the route, and change the directory, and start the route again - if you want to attempt that direction.
For the file producer
The file name is fully dynamic you can just set a name as the file header you want. See documentation for details.
You can use the header CamelFileName also to dynamically set a directory. Slashes will be turned into directories.
Example: CamelFileName = "directory/dyn_subdirectory_n/myfilename.txt" will be placed into the directory "dyn_subdirectory_n" located in "directory". The filename will be "myfilename.txt".
Related
I have an URL like
url = "file:D:/inputFolder?move=D:/outputFolder". we are making this url dynamically.
I want to rename the file while moving, So I made it something like this
url = "file:D:/inputFolder?move=D:/outputFolder&fileName=abc.txt". But I think move and fileName do not work together, it is not renaming.
Is there any alternative to do it? Please remember I want with "move" only.
I cannot use .setHeader(..) also.
Thanks,
Hy,
as far as I understand you, your trying to move the file in one single uri.
That is not really how camel works.
The idea of camel is to have a "consumer" and a "producer", where the consumer loads data (e.g. your file) and the producer puts the data somewhere (e.g. save the file into a folder)
That being said, here is what worked for me with a java route:
from("file:/home/chris/temp/camel/in")
.to("file:/home/chris/temp/camel/out/?fileName=test.txt");
The from part configures the folder where camel looks for new files. A few notes on that:
The file component checks the folder each 0.5 sec for new files. This can be changed with the delay parameter
The option noop configures, if the file is being moved or copied. By default it is set to false, which means it is moved
In the to part you configure, where the file is supposed to be moved. Here you can use the fileName parameter to rename the file.
Be careful with this though, because setting an option in the uri directly does make it "static".
What I mean by that is, that the only way of changing the parameter is by completely reconfiguring the route or by restarting it, where neither is something you would want to do normally.
Note 1:
Moving all files that are put into one folder into the same file always overrides the previous file by default.
You could, for example, use the fileExists parameter to always just append the content of the file: fileExists=Append (See camel file docu for details)
Note 2:
There is an option in the file component to not "move" the file, but copy, rename and delete it, which sometimes is necessary, when you want to move it onto a different drive and a simple copy does not work.
Also see the docu for the camel file component for details on that.
Note 3:
You can have multiple to() statements in the same route to have the file moved to multiple locations. For example:
from("file:/home/chris/temp/camel/in")
.to("file:/home/chris/temp/camel/out/?fileName=test.txt")
.to("smtp:....");
Hope I could help you and answer you question.
Greets
Chris
Two possible ways to achieve your goal.
Use both "consumer" and "producer"
Using this way, you are free to control where and how your destination can be set and has great freedom to control filename with the use of a processor/bean.
from("file:D:/inputFolder")
.to("file:D:/outputFolder?fileName=abc.txt")
Use "consumer" only
Using this way, you are treating your work as source data control. This can be use when your file is going to move within same drive. The drawback is the filename rename pattern is limited (refer to camel file language)
from("file:D:/inputFolder?move=${file:parent}/../outputFolder/abc.txt")
source_dir Have files like: ABC_02022018_162301.CSV, ABC_02022018_231801.CSV, controlFile
<route id="Test">
<from uri="file:source_dir?include=ABC_.*\.CSV&doneFileName=controlFile&delete=true&readLock=changed&readLockTimeout=20000&readLockCheckInterval=5000&eadLockMinLength=0"/>
<log message="${file:name}"/>
**destination directory **
</route>
I am looking here is, route has to check controlFile for every main file. If any main file doesn't have control file, main file shouldn't move from source folder.
In my above code, camel only once checking for control file existence in source folder and moving all the files to destination folder. Can anyone please help on this?
According to the Camel docs (section 'Using 'done' Files'), in order to have one doneFile per main file you need to specify dynamic doneFile names:
it is more common to have one done file per target file. This means
there is a 1:1 correlation. To do this you must use dynamic
placeholders in the doneFileName option. Currently Camel supports the
following two dynamic tokens: file:name and file:name.noext which must
be enclosed in ${}
if you don't, then Camel will consume all files and then delete the doneFile unless noop=true
Short answer: if you define doneFileName dynamically like this "doneFileName=${file:name.noext}.trig" then each file you want to transfer must have a copy with .trig extension.
If you define doneFileName statically like "doneFileName=files.trig" then every file will be moved when files.trig appears.
So you have to think of a unique doneFileName for each file instead of "controlFile" and set it dynamically. Easiest way is to use the actual fileName and add some extension.
Examples:
Imagine this files: file1.txt, file2.txt
and this route from:
from("ftp://admin#localhost/from?password=admin"
+ "&fileName=${file:name}"
+ "&doneFileName=${file:name.noext}.done")
In this case file1.txt will be moved only if file1.done is also present in the same folder. file2.txt will be moved only if file2.done is also present in the same folder.
Now imagine this route:
from("ftp://admin#localhost/from?password=admin"
+ "&fileName=${file:name}"
+ "&doneFileName=controlFile");
In this case both files file1.txt and file2.txt will be moved when files.done file is present in the same folder.
I'm trying to build a file based integration where files are dumped in one of the subdirectories of a main directory for processing. I need to get the name of the sub-directory to know which client the file is for. So if I have:
/uploads/foo/bar.txt
I need to process that file and know that it's for client "foo". I'm not sure how to get that part and set it as a header for the processor that processes the bar.txt file. I've got it picking up files and processing, now I need to add in this piece.
Anyone have ideas for me?
You can get most of this information in the header of the exchange. In your situation as you are consuming the file the following items are avlable:
CamelFileName: Name of the consumed file as a relative file path with
offset from the starting directory configured on the endpoint.
CamelFileNameOnly: Only the file name (the name with no leading
paths).
CamelFileAbsolute: A boolean option specifying whether the consumed
file denotes an absolute path or not. Should normally be false for
relative paths. Absolute paths should normally not be used but we
added to the move option to allow moving files to absolute paths. But
can be used elsewhere as well.
CamelFileAbsolutePath: The absolute path to the file. For relative
files this path holds the relative path instead.
CamelFilePath: The file path. For relative files this is the starting directory + the relative filename. For absolute files this is the absolute path.
CamelFileRelativePath: The relative path.
CamelFileParent: The parent path.
CamelFileLength: A long value containing the file size.
CamelFileLastModified: A Date value containing the last modified
timestamp of the file.
You can query these headers for the information you are looking for using the following example as guidelines:
<log message ="${header.CamelFileAbsolutePath}"/>
See the file component documents at the Camel website for more details.
What is the file name extension ".done"?
How to handle the files with this extension? For example filename.log.gz.done is file name, can we just remove the done extension and use it?
What is the use of this file extension?
".done" is just a marker that signifies that the file is ready for consumption.
So yes, get rid of the done extension and use it.
More details can be found here: http://www.davsclaus.com/2010/12/camel-26-using-done-files-with-fileftp.html
The ".done" file extension may be appended onto any type of file, such as a .TXT or .LOG file, and may be found on an FTP server where multiple have access to the file. The DONE file helps prevents a user from accessing a file that is not meant to be accessed. The ".done" extension should be removed in order to open the actual file.
This takes you to the source from which I'm quoting my answer from. You will see a clear explanation of .done file extension and it's also a great reference for any other file extensions that you might need information about
I need to read file in my program so while providing path I want to give relative paths because all files to be opened will be in some folder within current folder.
I tried this:
FILE *f=fopen("./abc/p.txt","r")
abc is folder withing current folder, but fopen returns NULL. How to do this thing?
This comes from either one of those:
. or ./abc/ is not readable or traversable
./abc/p.txt is not readable
./abc/p.txt does not exist
./abc/p.txt is a broken link
Look at errno to know what's the real problem.
this will run:
FILE *f=fopen("...\\abc\\p.txt","r");