Apache Camel - How doneFileName Works - apache-camel

source_dir Have files like: ABC_02022018_162301.CSV, ABC_02022018_231801.CSV, controlFile
<route id="Test">
<from uri="file:source_dir?include=ABC_.*\.CSV&doneFileName=controlFile&delete=true&readLock=changed&readLockTimeout=20000&readLockCheckInterval=5000&eadLockMinLength=0"/>
<log message="${file:name}"/>
**destination directory **
</route>
I am looking here is, route has to check controlFile for every main file. If any main file doesn't have control file, main file shouldn't move from source folder.
In my above code, camel only once checking for control file existence in source folder and moving all the files to destination folder. Can anyone please help on this?

According to the Camel docs (section 'Using 'done' Files'), in order to have one doneFile per main file you need to specify dynamic doneFile names:
it is more common to have one done file per target file. This means
there is a 1:1 correlation. To do this you must use dynamic
placeholders in the doneFileName option. Currently Camel supports the
following two dynamic tokens: file:name and file:name.noext which must
be enclosed in ${}
if you don't, then Camel will consume all files and then delete the doneFile unless noop=true

Short answer: if you define doneFileName dynamically like this "doneFileName=${file:name.noext}.trig" then each file you want to transfer must have a copy with .trig extension.
If you define doneFileName statically like "doneFileName=files.trig" then every file will be moved when files.trig appears.
So you have to think of a unique doneFileName for each file instead of "controlFile" and set it dynamically. Easiest way is to use the actual fileName and add some extension.
Examples:
Imagine this files: file1.txt, file2.txt
and this route from:
from("ftp://admin#localhost/from?password=admin"
+ "&fileName=${file:name}"
+ "&doneFileName=${file:name.noext}.done")
In this case file1.txt will be moved only if file1.done is also present in the same folder. file2.txt will be moved only if file2.done is also present in the same folder.
Now imagine this route:
from("ftp://admin#localhost/from?password=admin"
+ "&fileName=${file:name}"
+ "&doneFileName=controlFile");
In this case both files file1.txt and file2.txt will be moved when files.done file is present in the same folder.

Related

Apache Camel doneFileName with changing name

I'm currently creating some route and for one of them I have a problem.
Usually I have a data file and then a done file which have the same name prefixed by "ACK" and this works perfectly with camel and the doneFileName option.
But for one of my route I have to work with a different situation, I still receive two files but they have the same typology, it's like: MyFILE-{{timestamp}}. The data file contains the data, and the done file contains just "done".
So I need something to check the content of the file, and if it's juste "done" then process the other file.
Is there a way to handle this with camel?
The most pragmatic solution I see is to write an "adapter script" (bash or whatever you have at your disposal) that peeks into every file with a timestamp in its name.
If the file content is "done":
Lookup the other "MyFILE-{{timestamp}}" (the data file) and rename it to "MyFILE"
Rename the done file to "MyFILE.done"
Camel can then import the data file using the standard done-file-option. Because both files are renamed to something without a timestamp, the peek-script ignores them after renaming.

How to rename a file while using "move" in URL in apache camel

I have an URL like
url = "file:D:/inputFolder?move=D:/outputFolder". we are making this url dynamically.
I want to rename the file while moving, So I made it something like this
url = "file:D:/inputFolder?move=D:/outputFolder&fileName=abc.txt". But I think move and fileName do not work together, it is not renaming.
Is there any alternative to do it? Please remember I want with "move" only.
I cannot use .setHeader(..) also.
Thanks,
Hy,
as far as I understand you, your trying to move the file in one single uri.
That is not really how camel works.
The idea of camel is to have a "consumer" and a "producer", where the consumer loads data (e.g. your file) and the producer puts the data somewhere (e.g. save the file into a folder)
That being said, here is what worked for me with a java route:
from("file:/home/chris/temp/camel/in")
.to("file:/home/chris/temp/camel/out/?fileName=test.txt");
The from part configures the folder where camel looks for new files. A few notes on that:
The file component checks the folder each 0.5 sec for new files. This can be changed with the delay parameter
The option noop configures, if the file is being moved or copied. By default it is set to false, which means it is moved
In the to part you configure, where the file is supposed to be moved. Here you can use the fileName parameter to rename the file.
Be careful with this though, because setting an option in the uri directly does make it "static".
What I mean by that is, that the only way of changing the parameter is by completely reconfiguring the route or by restarting it, where neither is something you would want to do normally.
Note 1:
Moving all files that are put into one folder into the same file always overrides the previous file by default.
You could, for example, use the fileExists parameter to always just append the content of the file: fileExists=Append (See camel file docu for details)
Note 2:
There is an option in the file component to not "move" the file, but copy, rename and delete it, which sometimes is necessary, when you want to move it onto a different drive and a simple copy does not work.
Also see the docu for the camel file component for details on that.
Note 3:
You can have multiple to() statements in the same route to have the file moved to multiple locations. For example:
from("file:/home/chris/temp/camel/in")
.to("file:/home/chris/temp/camel/out/?fileName=test.txt")
.to("smtp:....");
Hope I could help you and answer you question.
Greets
Chris
Two possible ways to achieve your goal.
Use both "consumer" and "producer"
Using this way, you are free to control where and how your destination can be set and has great freedom to control filename with the use of a processor/bean.
from("file:D:/inputFolder")
.to("file:D:/outputFolder?fileName=abc.txt")
Use "consumer" only
Using this way, you are treating your work as source data control. This can be use when your file is going to move within same drive. The drawback is the filename rename pattern is limited (refer to camel file language)
from("file:D:/inputFolder?move=${file:parent}/../outputFolder/abc.txt")

How to get the relative path segment for Apache Camel File2

I'm trying to build a file based integration where files are dumped in one of the subdirectories of a main directory for processing. I need to get the name of the sub-directory to know which client the file is for. So if I have:
/uploads/foo/bar.txt
I need to process that file and know that it's for client "foo". I'm not sure how to get that part and set it as a header for the processor that processes the bar.txt file. I've got it picking up files and processing, now I need to add in this piece.
Anyone have ideas for me?
You can get most of this information in the header of the exchange. In your situation as you are consuming the file the following items are avlable:
CamelFileName: Name of the consumed file as a relative file path with
offset from the starting directory configured on the endpoint.
CamelFileNameOnly: Only the file name (the name with no leading
paths).
CamelFileAbsolute: A boolean option specifying whether the consumed
file denotes an absolute path or not. Should normally be false for
relative paths. Absolute paths should normally not be used but we
added to the move option to allow moving files to absolute paths. But
can be used elsewhere as well.
CamelFileAbsolutePath: The absolute path to the file. For relative
files this path holds the relative path instead.
CamelFilePath: The file path. For relative files this is the starting directory + the relative filename. For absolute files this is the absolute path.
CamelFileRelativePath: The relative path.
CamelFileParent: The parent path.
CamelFileLength: A long value containing the file size.
CamelFileLastModified: A Date value containing the last modified
timestamp of the file.
You can query these headers for the information you are looking for using the following example as guidelines:
<log message ="${header.CamelFileAbsolutePath}"/>
See the file component documents at the Camel website for more details.

Camel - Why the file consumer move option working differently with pollEnrich?

When I use a file consumer
<from uri="file:in?move=$simple{file:name}-transfered&include=^demo_keys\.ks$&sortBy=file:name" />
the file(s) are renamed to xxx-transfered (as I expected and stated in the doc) after processing.
But when I use the same with pollEnrich (for just one file)
<pollEnrich>
<simple>file://in?fileName=demo_keys.ks2&move=${camelId}-uploaded&sendEmptyMessageWhenIdle=true&maxMessagesPerPoll=1&delay=8000</simple>
</pollEnrich>
the file is not renamed after processing, instead moved into a newly created sub-directory with the original name.
How can I rename the pollEnrich processed file, achieve the same behaviour as a normal file consume?
I've tested it with v2.17.2 and v2.18.0
Thanks!
I think this may be a bug in Camel; try using a file language expression (e.g. ${file:name}) instead of ${camelId} just to be sure but the official documentation is pretty much clear in this case - it should interpret the value as a file name instead of directory.
I guess you should report a bug in Camel's JIRA.

dynamic input/output folder location in Camel's file component

As per https://camel.apache.org/file2.html, file component cannot be configured with dynamic input/output folder locations?
Is there any workaround/alternative for the same?
Thanks.
Regards
Senthil Kumar Sekar
For the file consumer
You can use a customer filter as dynamic selection of which directories/files to pickup. But the starting directory is configured once (hardcoded).
For that to change you would need to stop the route, and change the directory, and start the route again - if you want to attempt that direction.
For the file producer
The file name is fully dynamic you can just set a name as the file header you want. See documentation for details.
You can use the header CamelFileName also to dynamically set a directory. Slashes will be turned into directories.
Example: CamelFileName = "directory/dyn_subdirectory_n/myfilename.txt" will be placed into the directory "dyn_subdirectory_n" located in "directory". The filename will be "myfilename.txt".

Resources