Given a simple route like this
route.from("direct:foo")
.split()
.tokenize("\n")
.streaming()
.to("stream:file?fileName=target/streaming${header.count}.txt&closeOnDone=true");
which I then trigger with this
#Test
public void splitAndStreamToFile() {
StringBuilder builder = new StringBuilder();
for(int i = 0; i < 500; i++) {
builder.append(i);
builder.append("\n");
}
for(int i = 0; i < 10; i++) {
template.sendBodyAndHeader(builder.toString(), "count", i);
}
}
I get one big file that contains 10 times 500 lines, where I would have hoped to have 10 files that contain 500 lines each.
In other words, it seems that the fileName in the stream:file endpoint is not dynamic. I am wondering if this is at all possible? My google-fu turned up nothing so far.
EDIT:
With Claus' answer, I got it to work like this:
route.from("direct:foo")
.split()
.tokenize("\n")
.streaming()
.recipientList(route.simple("stream:file?fileName=target/streaming${header.count}.txt&closeOnDone=true"));
Its a dynamic to which there is an EIP pattern for:
http://camel.apache.org/how-to-use-a-dynamic-uri-in-to.html
But it could be a good idea to support the file/simple language on the fileName option as the regular file component does. Fell free to log a JIRA ticket about this improvement.
Sourcecode of the StreamProducer looks like it does not support any of the expression languages of Camel yet:
private OutputStream resolveStreamFromFile() throws IOException {
String fileName = endpoint.getFileName();
ObjectHelper.notEmpty(fileName, "fileName");
LOG.debug("About to write to file: {}", fileName);
File f = new File(fileName);
// will create a new file if missing or append to existing
f.getParentFile().mkdirs();
f.createNewFile();
return new FileOutputStream(f, true);
}
See sourecode.
If you need dynamic filenames, you should take a look at the file component, which supports the file language and the CamelFileName header.
In short,
toD uri=stream:file...
will do it.
The "toD" basically translates the "simple" or "file language" before it hits the stream component code...so that works for "fileName=..."
Related
I am new to Apache Camel.
I need to split a file line by line and to do some operation on each lines.
At the end I need a footer line with information from previous lines (number of lines and sum of the values of a column)
My understanding is that I should be using an aggregation strategy, so I tried something like that:
.split(body().tokenize("\r\n|\n"), sumAggregationStrategy)
.process("fileProcessor")
In my aggregation strategy I just set two headers with the incremented values:
newExchange.getIn().setHeader("sum", sum);
newExchange.getIn().setHeader("numberOfLines", numberOfLines);
And in the processor I try to access those headers:
int sum = inMessage.getIn().getHeader("sum", Integer.class);
int numberOfLines = inMessage.getIn().getHeader("numberOfLines", Integer.class);
There are two problems.
First of all the aggregation strategy seem to be called after the first iteration of the processor.
Second, my headers don't exist in the processors, so I can't access the information I need when I am at the last line of the file. The headers do exist in the oldExchange of the aggregators though.
I think I can still do it, but I would have to create a new processor just for the purpose of making the last line of the file.
Is there something I'm missing with the aggregation strategies ? Is there a better way to do this ?
An aggregator will be called for every iteration of the split. This is how they are supposed to work.
The reason you don't see the headers within the processor is, headers live and die with the message and not visible outside. You need to set the 'sum' and 'numberOfLines' as exchange properties instead. Because every iteration within a split results in an exchange, you need get the property from old exchange and set them again in the new exchange to pass them to subsequent components in the route.
This is how you could do,
AggregationStrategy:
public class SumAggregationStrategy implements AggregationStrategy {
public Exchange aggregate(Exchange oldExchange, Exchange newExchange) {
long sum = 0;
long numberOfLines = 0;
if(oldExchange != null) {
sum = (Long) oldExchange.getProperty("sum");
numberOfLines = oldExchange.getProperty("numberOfLines ");
}
sum = sum + ((Line)newExchange.getIn().getBody()).getColumnValue();
numberOfLines ++;
newExchange.setProperty("sum", sum);
newExchange.setProperty("numberOfLines",numberOfLines);
oldExchange.setProperty("CamelSplitComplete", newExchange.getProperty("CamelSplitComplete")); //This is for the completion predicate
return newExchange;
}
}
Route:
.split(body().tokenize("\r\n|\n"),sumAggregationStrategy)
.completionPredicate(simple("${exchangeProperty.CamelSplitComplete} == true"))
.process("fileProcessor").to("file:your_file_name?fileExist=Append");
Processor:
public class FileProcessor implements Processor {
public void process(Exchange exchange) throws Exception {
long sum = exchange.getProperty("sum");
long numberOfLines = exchange.getProperty("numberOfLines");
String footer = "Your Footer String";
exchange.getIn().setBody(footer);
}
}
Using custom aggregator like Srini suggested is a good idea. It might also support streaming large files better.
However if you want to keep things simple and avoid split and aggregation you could just use .tokenize("\r\n|\n") and convertBodyTo(List.class) to convert the string to a list of strings.
from("direct:addFooter")
.routeId("addFooter")
.setBody().tokenize("\r\n|\n")
.convertBodyTo(List.class)
.process(exchange -> {
List<String> rows = exchange.getMessage().getBody(List.class);
int sum = 0;
for (int i = 0; i < rows.size(); i++) {
sum += Integer.parseInt(rows.get(i));
}
int numberOfLines = rows.size();
exchange.getMessage().setHeader("numberOfLines", numberOfLines);
exchange.getMessage().setHeader("sum", sum);
})
// Write data to file using file or stream component
// you could also use Velocity, FreeMarker or Mustache templates to format the
// result before writing it to file.
;
I have tried doing this by encrypting individual files but I have a lot of data (~20GB) and hence it would take a lot of time. In my test it took 2.28 minutes to encrypt a single file of size 80MB.
Is there a quicker way to be able to password protect that would apply to any any file (text/binary/multimedia)?
If you are just trying to hide the file from others, you can try to encrypt the file path instead of encrypting the whole huge file.
For the path you mentioned: text/binary/multimedia, you can try to encrypt it by a method as:
private static String getEncryptedPath(String filePath) {
String[] tokens = filePath.split("/");
List<String> tList = new ArrayList<>();
for (int i = 0; i < tokens.length; i++) {
tList.add(Hashing.md5().newHasher() // com.google.common.hash.Hashing;
.putString(tokens[i] + filePath, StandardCharsets.UTF_8).hash().toString()
.substring(2 * i, 2 * i + 5)); // to make it impossible to encrypt, add your custom secret here;
}
return String.join("/", tList);
}
and then it becomes an encrypted path as:
72b12/9cbb3/4a5f3
Once you know the real path text/binary/multimedia, any time you want to access the file, you can just use this method to get the real file path 72b12/9cbb3/4a5f3.
In my app I tried to pass the file path from one activity to another activity using intent.In my receiving activity I got the file path as "null".But when I print the file in first activity it prints the path.From my second activity I attach that file to mail using Gmailsender.This was the code I tried,
private void startRecord()
{
File file = new File(Environment.getExternalStorageDirectory(), "test.pcm");
try
{
file.createNewFile();
OutputStream outputStream = new FileOutputStream(file);
BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(outputStream);
DataOutputStream dataOutputStream = new DataOutputStream(bufferedOutputStream);
int minBufferSize = AudioRecord.getMinBufferSize(8000,
AudioFormat.CHANNEL_IN_MONO,
AudioFormat.ENCODING_PCM_16BIT);
short[] audioData = new short[minBufferSize];
AudioRecord audioRecord = new AudioRecord(MediaRecorder.AudioSource.MIC,
8000,
AudioFormat.CHANNEL_IN_MONO,
AudioFormat.ENCODING_PCM_16BIT,
minBufferSize);
audioRecord.startRecording();
while(recording)
{
int numberOfShort = audioRecord.read(audioData, 0, minBufferSize);
for(int i = 0; i < numberOfShort; i++)
{
dataOutputStream.writeShort(audioData[i]);
}
}
audioRecord.stop();
audioRecord.release();
dataOutputStream.close();
}
catch (IOException e)
{
e.printStackTrace();
}
String audiofile;
audiofile=file.getAbsolutePath();
System.out.println("File Path::::"+audiofile);
}
Intent is,
Intent sigout=new Intent(getApplicationContext(),WeeklyendActivity.class);
sigout.putExtra("mnt/sdcard-test.pcm",audiofile);
startActivity(sigout);
In my receiving activity,
String patty=getIntent().getStringExtra("mnt/sdcard-text.pcm");
System.out.println("paathhhy frfom ::"+patty);
It prints null.Can anyone help me how to get the file path.And more thing I am not sure whether the audio would save in that file correctly?
Please anyone help me!!!Thanks in advance!
Based on your information that audioFile is a variable of type File, when you do this:
sigout.putExtra("mnt/sdcard-test.pcm",audiofile);
you are putting a File object in the extras Bundle. Then, when you try to get the extra from the Bundle you do this:
String patty=getIntent().getStringExtra("mnt/sdcard-text.pcm");
However, the object in this extra is of type File, not type String. This is why you are getting null.
If you only want to pass the name of the file, then put the extra like this:
sigout.putExtra("mnt/sdcard-test.pcm",audiofile.getAbsolutePath());
When I try to get attatchment from POP 3 mail, I am getting them as winmail.dat, not the original attached file name. How can I get the original file name?
for (int i = 0; i < multipart.getCount(); i++)
{
BodyPart bodyPart = multipart.getBodyPart(i);
if(!Part.ATTACHMENT.equalsIgnoreCase(bodyPart.getDisposition()))
{
//do something
}
else
{
bodyPart.getFileName(); // here only get the winmail.dat
}
}
This is part of the Exchange Settings, and sadly you going to need to extract the original contents from the WinMail.dat using JTNEF.
"The Java TNEF package is an open source code implementation of a TNEF message handler, which can be used as a command-line utility or integrated into Java-based mail applications to extract the original message content."
This is found on the JavaMail's third party tools.
As alternative and what looks simpler is POI-HMEF
Sample extraction:
public void extract(String winmailFilename, String directoryName) throws Exception {
HMEFContentsExtractor ext = new HMEFContentsExtractor(new File(winmailFilename));
File dir = new File(directoryName);
File rtf = new File(dir, "message.rtf");
if(! dir.exists()) {
throw new FileNotFoundException("Output directory " + dir.getName() + " not found");
}
System.out.println("Extracting...");
ext.extractMessageBody(rtf);
ext.extractAttachments(dir);
System.out.println("Extraction completed");
}
There is also a sample for printing the contents here.
I have an array of strings which I am trying to store in Isolated storage, However I need to store each string in the array in a new file of its own.
Any approach is welcomed.
Thanks.
I do something similar in an app with code roughly along these lines. Though I am serializing objects in an array to json. Same rough idea though.
using (IsolatedStorageFile file = IsolatedStorageFile.GetUserStoreForApplication()) {
for (int i = 0; i < array.Length; i++) {
string fileName = "file" + i.ToString() + ".dat";
using (var stream = file.CreateFile(filename)) {
using (var writer = new StreamWriter(stream)) {
writer.Write(array[i]);
}
}
}
}
Note this is just typed straight in, I may have a mistake in there :)
Your question is a little vauge, but here I go.
What is stopping you from just serializing each string to a file with the index as the name? For example, store stringarray[0] in a file 0.xml.
Just check whether the file exists before trying to read it.