I'm using apache camel in a routing context to integrate some CSV files. If I want to skip the first line of the csv I can easily do it in the unmarshal:
<unmarshal>
<csv delimiter=";" skipFirstLine="true"/>
</unmarshal>
My question is, whats the best way to ignore/skip the last line of the file ?
Thank you in advance,
Camel CSV component (ver. 2.18.1) does not support skipping the footer out of the box. In case you can keep the whole file in memory I would advice to use a Camel processor to remove the last line before unmarshalling.
There is also a similar question answered by Claus Ibsen for the Bindy Component: Camel CSVRecord Camel Bindy
The last line can not be skipped. Please read the file from the end and remove the last line using a File Reader
Related
I have one requirement, where I need to move multiple files present in one folder to another. This should be done based on filename which should be dynamic.
As Far I have tried out pollenrich and file (antInclude) but in both case I got struck.
<route id="readFile" autoStartup="true">
<from uri="timer://timer1?repeatCount=1"/>
<setHeader headerName="xxx">
<simple>1234</simple>
</setHeader>
<pollEnrich timeout="15000">
<simple>file://{{baseDirectory}}?move={{destinationDirectory}}&antInclude=*_${header.xxx}.txt</simple>
</pollEnrich>
</route>
Note: Header value will be dynamic, using javascript will pull that data and set it in header.
Any help on this would most welcome.
Thank you for all your response,
I achieved it by using pollenrich but tweaking something in pollenrich.
Got CamelBatchsize using pollenrich.
Running into a loop using size.
Using the above code used to move the file. I achieved it.
Thanks.
How can I get a trigger once my route completes processing a directory and its sub directories in camel ? Is there any way I can specify this in the route ?
Yes if its from("file:..") then its a batch consumer, and you can find the information on the last file which has a message header with CamelBatchComplete with the value true.
See more details at the Camel website: http://camel.apache.org/batch-consumer.html
I have an Apache Camel route which purpose is to fetch xml-documents and files linked in the document.
<route id="route-ftp">
<from uri="ftp://foo#server:21/data?password=xxx&include=.*.xml"/>
<to uri="myBean"/>
</route>
When it reaches myBean I want to parse the file and use the same ftp settings to fetch the files listed in the xml-file.
Or perhaps fetch them all at the same time using xpath.
<root>
<article>
<headline>Headline</headline>
<image src="images/cat.jpg"/>
</article>
</root>
The filenames is taken from the above xml image tag and src attribute.
I then want to deliver the original file together with my images in a package.
I can't find the right approach to my problem.
I would probably do it this way.
Depending on how your file is structured, I would consume it as you have done. I would then split the body per row (assuming each row contains a filename). The split is done using the splitter EIP. Now each body contains the file name.
I would then use Camel's dynamicTo to enrich the route with the file using the ftp parameters. You can save the ftp parameters are properties so that they are always the same.
In this way, you fetch the list, iterate over the list and fetch the file for every row in the list.
I want to download xml files from an S3 bucket, then unmarshal them to insert the data in a database:
...
from("aws-s3://myBucket?accessKey=xxx&secretKey=yyy")
.to("file:D:/output?fileName=${in.header.CamelAwsS3Key}")
.unmarshal(new JaxbDataFormat("com.xxx"))
...
I am able to download the files when I don't try to unmarshal, but I get this error when I try to unmarshal:
org.apache.camel.TypeConversionException:
Error during type conversion from type:
com.amazonaws.services.s3.model.S3ObjectInputStream to the required type:
javax.xml.stream.XMLStreamReader with value com.amazonaws.services.s3.model.S3ObjectInputStream#67b8328b due javax.xml.stream.XMLStreamException:
java.io.IOException: Attempted read on closed stream.
Since I am new to Camel there are maybe things I didn't understand yet...
When I pipe endpoints, doesn't the current endpoint get the message the way the previous endpoint "modified" it? In my case it looks like the S3 stream is being marshalled, instead of the xml file newly created locally from the download, hence the error.
My understanding is if I do .from().to().to(), the second .to() doesn't know what is coming from .from() so if my first .to() creates an xml file, the second .to() handles the message as an xml file. Am I wrong?
Maybe I need to create 2 routes? I was able to do the other way around with only 1 route though, from database to file to S3.
Do I need to write my own converter in that case?
Thanks!
When you have the 2nd to Camel needs to read the stream again, but its closed. Its becomes sometimes streams are only readable once. And it was read in the first to, where you write it to a file.
So you can either enable stream caching to allow Camel to be able to re-read the stream again. See more in this FAQ and the links
http://camel.apache.org/why-is-my-message-body-empty.html
http://camel.apache.org/stream-caching.html
You can also break it into 2 routes
from amq
to file
from file
unmarshal
i have a processor which puts an unmarshaled ical file in a custom object. After this a list of my object is saved into exchange.
After this i need to put a second ical into exchange. Problem is that i cant use "from" two times in a route. How can i solve this?
Code for processing one ical is the following:
from("file://src/test/resources?fileName=Calendar.ics&delete=false&noop=true")
.unmarshal("ical")
.process(icsToBeanProcessor)
.end();
Perhaps you can improve the design a bit. Instead of specifying filenames use a directory instead. Process all files present in the directory and set them in header properties. Something like
from("file://src/test/resources?delete=false&noop=true")
.unmarshal("ical")
.process(icsToBeanProcessor) // in your icsToBeanProcessor set a header property as content.fileName as the output of your unmarshaller bean
.end();
Also you can be more specific by using a regex for specifying filenames. For your case pick only those files which ends with "ics". See http://camel.apache.org/file.html for more details.