how can write a liste of line into a file in Apache Camel ?
first, i get a resultSet with a proc
<to id="getdata" uri="sql-stored:{{sql.comptes}}?dataSource=dataSource" />
passing this resultSet to create a list of String
<to uri="bean:Converter" />
passing this List of String to the file componnet to write in a file
<to id="fileWriter" uri="file:{{data.out}}?fileName=Comptes-${date:now:yyyyMMdd'T'HHmmss'T'SSS}.txt" />
no file is created.
The file endpoint expects exchange body to be something that can be serialized to the byte array. If you need a list of strings (like one line per string) in the file you need to convert exchange body from list of strings to just string for example (by concatinating strings with newline char).
Related
I have a problem reading different fileName from Camel file component.
from("file:/in?fileName={{property.name}}")
.to(file:/out)
I used fileName={{property.name}} from application.yml, but I need to use it from String.
Is there any way to use it like:
String name = "blabla.xml";
from("file:/in?fileName=${name}")
.to(file:/out)
Camel don't support it. String concatenation can solve your problem:
from("file:/in?fileName="+name)
or you can set a property and then read it:
String name="name";
from("direct:start")
.setProperty("name",constant(name))
.to("file:/in?fileName=${exchangeProperty.name}");
As the title suggests, I want to move a file into a different folder after I am done writing DB records to to it.
I have already looked into several questions related to this: Apache camel file with doneFileName
But my problem is a little different since I am using split, stream and parallelProcessing for getting the DB records and writing to a file. I am not able to know when and how to create the done file along with the parallelProcessing. Here is the code snippet:
My route to fetch records and write it to a file:
from(<ROUTE_FETCH_RECORDS_AND_WRITE>)
.setHeader(Exchange.FILE_PATH, constant("<path to temp folder>"))
.setHeader(Exchange.FILE_NAME, constant("<filename>.txt"))
.setBody(constant("<sql to fetch records>&outputType=StreamList))
.to("jdbc:<endpoint>)
.split(body(), <aggregation>).streaming().parallelProcessing()
.<some processors>
.aggregate(header(Exchange.FILE_NAME), (o, n) -> {
<file aggregation>
return o;
}).completionInterval(<some time interval>)
.toD("file://<to the temp file>")
.end()
.end()
.to("file:"+<path to temp folder>+"?doneFileName=${file:header."+Exchange.FILE_NAME+"}.done"); //this line is just for trying out done filename
In my aggregation strategy for the splitter I have code that basically counts records processed and prepares the response that would be sent back to the caller.
And in my other aggregate outside I have code for aggregating the db rows and post that writing into the file.
And here is the file listener for moving the file:
from("file://<path to temp folder>?delete=true&include=<filename>.*.TXT&doneFileName=done")
.to(file://<final filename with path>?fileExist=Append);
Doing something like this is giving me this error:
Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - Cannot store file: <folder-path>/filename.TXT] org.apache.camel.component.file.GenericFileOperationFailedException: Cannot store file: <folder-path>/filename.TXT
at org.apache.camel.component.file.FileOperations.storeFile(FileOperations.java:292)[209:org.apache.camel.camel-core:2.16.2]
at org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:277)[209:org.apache.camel.camel-core:2.16.2]
at org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:165)[209:org.apache.camel.camel-core:2.16.2]
at org.apache.camel.component.file.GenericFileProducer.process(GenericFileProducer.java:79)[209:org.apache.camel.camel-core:2.16.2]
at org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)[209:org.apache.camel.camel-core:2.16.2]
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:141)[209:org.apache.camel.camel-core:2.16.2]
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)[209:org.apache.camel.camel-core:2.16.2]
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:460)[209:org.apache.camel.camel-core:2.16.2]
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:190)[209:org.apache.camel.camel-core:2.16.2]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:121)[209:org.apache.camel.camel-core:2.16.2]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:83)[209:org.apache.camel.camel-core:2.16.2]
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:190)[209:org.apache.camel.camel-core:2.16.2]
at org.apache.camel.component.seda.SedaConsumer.sendToConsumers(SedaConsumer.java:298)[209:org.apache.camel.camel-core:2.16.2]
at org.apache.camel.component.seda.SedaConsumer.doRun(SedaConsumer.java:207)[209:org.apache.camel.camel-core:2.16.2]
at org.apache.camel.component.seda.SedaConsumer.run(SedaConsumer.java:154)[209:org.apache.camel.camel-core:2.16.2]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)[:1.8.0_144]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)[:1.8.0_144]
at java.lang.Thread.run(Thread.java:748)[:1.8.0_144]
Caused by: org.apache.camel.InvalidPayloadException: No body available of type: java.io.InputStream but has value: Total number of records discovered: 5
What am I doing wrong? Any inputs will help.
PS: Newly introduced to Apache Camel
I would guess that the error comes from .toD("file://<to the temp file>") trying to write a file, but finds the wrong type of body (String Total number of records discovered: 5 instead of InputStream.
I don't understand why you have one file-destinations inside the splitter and one outside of it.
As #claus-ibsen suggested try to remove this extra .aggregate(...) in your route. To split and re-aggregate it is sufficient to reference the aggregation strategy in the splitter. Claus also pointed to an example in the Camel docs
from(<ROUTE_FETCH_RECORDS_AND_WRITE>)
.setHeader(Exchange.FILE_PATH, constant("<path to temp folder>"))
.setHeader(Exchange.FILE_NAME, constant("<filename>.txt"))
.setBody(constant("<sql to fetch records>&outputType=StreamList))
.to("jdbc:<endpoint>)
.split(body(), <aggregationStrategy>)
.streaming().parallelProcessing()
// the processors below get individual parts
.<some processors>
.end()
// The end statement above ends split-and-aggregate. From here
// you get the re-aggregated result of the splitter.
// So you can simply write it to a file and also write the done-file
.to(...);
However, if you need to control the aggregation sizes, you have to combine splitter and aggregator. That would look somehow like this
from(<ROUTE_FETCH_RECORDS_AND_WRITE>)
.setHeader(Exchange.FILE_PATH, constant("<path to temp folder>"))
.setHeader(Exchange.FILE_NAME, constant("<filename>.txt"))
.setBody(constant("<sql to fetch records>&outputType=StreamList))
.to("jdbc:<endpoint>)
// No aggregationStrategy here so it is a standard splitter
.split(body())
.streaming().parallelProcessing()
// the processors below get individual parts
.<some processors>
.end()
// The end statement above ends split. From here
// you still got individual records from the splitter.
.to(seda:aggregate);
// new route to do the controlled aggregation
from("seda:aggregate")
// constant(true) is the correlation predicate => collect all messages in 1 aggregation
.aggregate(constant(true), new YourAggregationStrategy())
.completionSize(500)
// not sure if this 'end' is needed
.end()
// write files with 500 aggregated records here
.to("...");
My input message:
<file>
<node1>
...
</node1>
.....
<node10>
.....
</node10>
</file>
I want to:
Process the whole file using stylesheet and output to Dest A
For a few elements in the file (say, node1, node3 and node7) I want to extract them and output the content of each individually to Dest B
I know how to process the file using stylesheet but I'm at a loss how to do the other, let alone combine them together.
I'm looking for something like:
from(direct:start).magic_split(
to("xslt:mysheet").to("destA"),
setBody(xpath("//node1").to("destB"),
setBody(xpath("//node3").to("destB"),
setBody(xpath("//node7").to("destB"),
).transform(constant(responseOK);
if you can split the XML, then each node is put in it's own exchange, if you can identify the node after it is split, then you can use a Content Based Router to route the exchange to the appropriate destination. This might require a custom splitter bean, or you might be able to do it from xpath if the nodes are named nodeX where X is a number.
Use the Wire Tap pattern. This pattern allows you to route messages to a separate location while they are being forwarded to the ultimate destination:
from("cxf:bean:submitOrder")
.wireTap("direct:tap")
.beanRef("customBean2");
from("direct:tap")
.to("xslt:my.xsl")
.beanRef("customBean1);
For what I needed, custom beans did the job well. The only thing I had to figure out was how to restore the message to the original content. I guess there is a more elegant way of doing it but works fine:
from("cxf:bean:submitOrder")
.setProperty("originalData", simple("${in.body}")) //save original input msg
.to("xslt:my.xsl").beanRef("customBean1)
.setBody(simple("${property.originalData}")) //restore original message
.beanRef("customBean2");
Camel 2.13.0
I am attempting to consume a json string containing multiple records and produce an output file for each record.
public void configure() {
from("file:data/input")
// my bean splits the input string and returns a json string via Gson
.bean(new com.camel.Tokenizer(), "getSentences")
.split(new JsonPathExpression("$[*]"))
.convertBodyTo(String.class)
.to("file:data/output");
}
}
Sample input:
"This is string #1. This is string #2."
If I run the code above as-is I get a single output file containing "This is string #2.". If I remove the .split() call I get a single output file containing json as expected:
[
{
"token": "This is string #1."
},
{
"token": "This is string #2."
}
]
How can I achieve two output files representing both lines of data?
It occurred to me that perhaps the split was working correctly and the second output file was overwriting the first. According to the documentation, the default behavior if CamelFileName is not set is to create a unique generated ID but I do not experience this. In my case the output file name always matches the input file name.
How can I get unique file name within each folder?
Thanks!
Finally stumbled upon the proper search terms and came across the following helpful post: Camel: Splitting a collection and writing to files
The trick is to use a bit of logic in the .to() to achieve unique output file names:
.to("file:data/sentence_q?fileName=${header.CamelSplitIndex}.txt");
The JsonPathExpression works like a charm and no need for a Processor() or unmarshal(), as I'd tried previously.
I want to use camel in my project to check a file's last modified date every xx minutes using camel's scheduling/timer service.
I read the document for file component it seems there is a polling function, however there is also a timer component for camel.
Anyone has some code example if i want to do with the requirement?
I would use the file consumer end point.
Something like this:
file:c:/foldername?delay=5000
This will scan the folder every 5 seconds for files and for each file send a message on the route.
You would probably need to store the previous times somewhere such as a text file or database and then compare the modified variable passed in the message to the modified one stored in the database or file.
A rough example of this would look like follows:
<route id="CheckFileRoute">
<from uri="file:d:/RMSInbox?delay=5000&readLock=changed/>
<log message="${ file:modified }/>
<bean ref="CompareDates"/>
</route>
The file consumer will place a lot of information regarding the file in the header such as modified date. Go read this link for more details on the variables in the header http://camel.apache.org/file2.html
The compare dates bean would be java class that acts like a processor which would have a structure like this:
public class CompareDates {
#Handler
public void CheckDates
(
#Body Object msgbody
, #Headers Map hdr
)
{
Date newDate = (Date)hdr.get("CamelFileLastModified");
Date oldDate = readfromfileorDatabase
if(newDate>oldDate)
{
//the date has changed look busy
}
}
Hope this gets you going.