I m using spring boot with camel . I have the following route configured :
from("file://C:/LOGS/HTTPBESample?delay=1000&recursive=true&noop=true").process(new Processor() {
public void process(Exchange msg) {
File file = msg.getIn().getBody(File.class);
LOG.info("Processing file: " + file.getName()+" file size "+file.length());
rocessing file: " + s);
}
});
However it runs only once , delay should work like a poller which isnt happening ?
It is still polling, but it is not finding any new files. If you drop a new file in the directory, it will process that one. If you want it to reprocess that same file each poll, you can set the idempotent=false.
Related
I am trying to setup a simple camel route which reads from a sqlite table and prints the record (later it would be written to a file).
The flow I have setup is below
bindToRegistry("sqlConsumer", new SqliteConsumer());
bindToRegistry("sqliteDatasource", dataSource());
from("sql:select * from recordsheet_record_1 where col_1 = 'A5'?dataSource=#sqliteDatasource")
.to("bean:sqlConsumer?method=consume")
.end();
And the SqliteConsmer as below
public class SqliteConsumer {
public void consume(Map<String, Object> data, Exchange exchange) {
System.out.println("Map: '" + data + "'");
//TODO: append to file
}
}
When I execute the route, it should only execute once (prints once), but, it keeps on printing... Am I doing anything wrong here?
I am new to camel framework so any help or guide would be much appreciated.
Thanks.
It is a polling consumer so it polls the source according to the configuration, you can find more info here: https://camel.apache.org/components/latest/eips/polling-consumer.html
I want to run a processor upon file appearance in a directory. My file url is like this:
file:{{file.root}}in?include=.*\.csv&charset=windows-1251&move=../out/done
A procedure that associates an url with a processor is like this:
MessageProcessor getOrCreateConsumer(CamelContext context, String uri) {
Endpoint endpoint = context.getEndpoint(uri);
endpoint.setCamelContext(context); // added this out of desperation, doesn't help
processor = new MessageProcessor();
try {
Consumer consumer = endpoint.createConsumer(processor);
endpoint.start(); // do we need this at all? works the same without it
consumer.start();
} catch (Exception e) {
throw new RuntimeException(e);
}
return processor;
}
}
MessageProcessor is a processor that does some things to an exchange.
Everything seems to work except the file doesn't get moved to the ../out/done directory. While debugging I can't get when the endpoint is configured to provide the file message exchange with this post operation.
I think I am missing some magic call that is normally invoked by a RouteBuilder and that will fully configure the file endpoint. Can you please help me out?
[JDK 8 / 11, Camel 2.23.1, Spring Boot 2.1.3]
Hi,
I'm currently learning Camel and I can't get past a strange problem.
I want to download a URL to a file.
The URL works, e.g. http://localhost/date shows "11:59:00".
The target file is created, but it's empty. Why?
#Component
public class DownloadRoute extends RouteBuilder {
#Override
public void configure() throws Exception {
String timer1 = "timer://foo?fixedRate=true&period=10000&delay=5000";
String url = "http://localhost:8888/date";
String file = "file:target/date";
from(timer1).to(url).log("${id}->${body}").to(file); // ID-Laptop-1551464093962-0-2->11:59:00, that's ok
}
}
The logger outputs the correct content, e.g.
2019-03-01 19:15:00.421 INFO 2984 --- [3 - timer://foo] route1
: ID-Laptop-1551464093962-0-2->11:59:00
but the generated file (e.g. target/date/ID-Laptop-1551464093962-0-2) is empty.
Any ideas?
Solution (thanks to Tache):
from(timer1).to(url).to(file); // without logging
OR
from(timer1).to(url).convertBodyTo(String.class).log("${id}->${body}").to(file); // with logging
The output of your http endpoint (http://localhost:8888/date) is probably a stream that is read using a InputStream. Such stream can be read only once.
Possible solutions:
remove your log statement
enable stream caching in your Camel context or route
convert message body to a string (.convertBodyTo(String.class) ) before logging it and sending it to a file
I am using jdeveloper version 11.1.1.5.0. In my use case I have created Mail Client Send Mail program where I used ADF InputFile component to attach File on mail.
But problem is that InputFile Component only return path of file(only get file name). And in my mail program DataSource class use full path to access file name.
UploadedFile uploadfile=(UploadedFile) actionEvent.getNewValue();
String fname= uploadfile.getFilename();//this line only get file name.
So how can I get full path using adf InputFile component or any other way to fulfill my requirement.
You could save the uploaded file in a path at the server. Only take care about naming that file, because of concurrency of users you should follow a policy about it, for example, adding te time in milliseconds to the name of the file. Like this...
private String writeToFile(UploadedFile file) {
ServletContext servletCtx =
(ServletContext)FacesContext.getCurrentInstance().getExternalContext().getContext();
String fileDirPath = servletCtx.getRealPath("/files/tmp");
String fileName = getTimeInMilis()+file.getFilename();
try {
InputStream is = file.getInputStream();
OutputStream os =
new FileOutputStream(fileDirPath + "/"+fileName);
int readData;
while ((readData = is.read()) != -1) {
os.write(readData);
}
is.close();
os.close();
} catch (IOException ex) {
ex.printStackTrace();
}
return fileName;
}
This method also returns the new name of the uploaded file. You can replace getTimeInMilis() with any naming policy you like.
It would be a security issue if a web app is able to see anything other than the data stream for an uploaded file. The directory structure of the client would not be exposed to the webapp. As such, unless you plan to upload the file from the same host as the server, you will not have access to the file path on the client.
Note: Using answer instead of comment due to reputation threshold
Problem Background
I am currently working on a camel based ETL application that processes groups of files as they appear in a dated directory. The files need to be processed together as a group determined by the beginning of the file name. The files can only be processed once the done file (".flag") has been written to the directory. I know the camel file component has a done file option, but that only allows you to retrieve files with the same name as the done file. The application needs to run continuously and start polling the next day's directory when the date rolls.
Example Directory Structure:
/process-directory
/03-09-2011
/03-10-2011
/GROUPNAME_ID1_staticfilename.xml
/GROUPNAME_staticfilename2.xml
/GROUPNAME.flag
/GROUPNAME2_ID1_staticfilename.xml
/GROUPNAME2_staticfilename2.xml
/GROUPNAME2_staticfilename3.xml
/GROUPNAME2.flag
Attempts Thus Far
I have the following route (names obfuscated) that kicks off the processing:
#Override
public void configure() throws Exception
{
getContext().addEndpoint("processShare", createProcessShareEndpoint());
from("processShare")
.process(new InputFileRouter())
.choice()
.when()
.simple("${header.processorName} == '" + InputFileType.TYPE1 + "'")
.to("seda://type1?size=1")
.when()
.simple("${header.processorName} == '" + InputFileType.TYPE2 + "'")
.to("seda://type2?size=1")
.when()
.simple("${header.processorName} == '" + InputFileType.TYPE3 + "'")
.to("seda://type3?size=1")
.when()
.simple("${header.processorName} == '" + InputFileType.TYPE4 + "'")
.to("seda://type4?size=1")
.when()
.simple("${header.processorName} == '" + InputFileType.TYPE5 + "'")
.to("seda://type5?size=1")
.when()
.simple("${header.processorName} == '" + InputFileType.TYPE6 + "'")
.to("seda://type6?size=1")
.when()
.simple("${header.processorName} == '" + InputFileType.TYPE7 + "'")
.to("seda://type7?size=1")
.otherwise()
.log(LoggingLevel.FATAL, "Unknown file type encountered during processing! --> ${body}");
}
My problems are around how to configure the file endpoint. I'm currently trying to programatically configure the endpoint without a lot of luck. My experience in camel thus far has been predominently using the Spring DSL and not the Java DSL.
I went down the route of trying to instantiate a FileEndpoint object, but whenever the route builds I get an error saying that the file property is null. I believe this is because I should be creating a FileComponent and not an endpoint. I'm not creating the endpoint without using a uri because I am not able to specify the dynamic date in the directory name using the uri.
private FileEndpoint createProcessShareEndpoint() throws ConfigurationException
{
FileEndpoint endpoint = new FileEndpoint();
//Custom directory "ready to process" implementation.
endpoint.setProcessStrategy(getContext().getRegistry().lookup(
"inputFileProcessStrategy", MyFileInputProcessStrategy.class));
try
{
//Controls the number of files returned per directory poll.
endpoint.setMaxMessagesPerPoll(Integer.parseInt(
PropertiesUtil.getProperty(
AdapterConstants.OUTDIR_MAXFILES, "1")));
}
catch (NumberFormatException e)
{
throw new ConfigurationException(String.format(
"Property %s is required to be an integer.",
AdapterConstants.OUTDIR_MAXFILES), e);
}
Map<String, Object> consumerPropertiesMap = new HashMap<String, Object>();
//Controls the delay between directory polls.
consumerPropertiesMap.put("delay", PropertiesUtil.getProperty(
AdapterConstants.OUTDIR_POLLING_MILLIS));
//Controls which files are included in directory polls.
//Regex that matches file extensions (eg. {SOME_FILE}.flag)
consumerPropertiesMap.put("include", "^.*(." + PropertiesUtil.getProperty(
AdapterConstants.OUTDIR_FLAGFILE_EXTENSION, "flag") + ")");
endpoint.setConsumerProperties(consumerPropertiesMap);
GenericFileConfiguration configuration = new GenericFileConfiguration();
//Controls the directory to be polled by the endpoint.
if(CommandLineOptions.getInstance().getInputDirectory() != null)
{
configuration.setDirectory(CommandLineOptions.getInstance().getInputDirectory());
}
else
{
SimpleDateFormat dateFormat = new SimpleDateFormat(PropertiesUtil.getProperty(AdapterConstants.OUTDIR_DATE_FORMAT, "MM-dd-yyyy"));
configuration.setDirectory(
PropertiesUtil.getProperty(AdapterConstants.OUTDIR_ROOT) + "\\" +
dateFormat.format(new Date()));
}
endpoint.setConfiguration(configuration);
return endpoint;
Questions
Is implementing a GenericFileProcessingStrategy the correct thing to do in this situation? If so, is there an example of this somewhere? I have looked through the camel file unit tests and didn't see anything that jumped out at me.
What am I doing wrong with configuring the endpoint? I feel like the answer to cleaning up this mess is tied in with question 3.
Can you configure the file endpoint to roll dated folders when polling and the date changes?
As always thanks for the help.
You can refer to a custom ProcessStrategy from the endpoint uri using the processStrategy option, eg file:xxxx?processStrategy=#myProcess. Notice how we prefix the value with # to indicate it should lookup it from the registry. So in Spring XML you just add a
<bean id="myProcess" ...> tag
In Java its probably easier to grab the endpoint from the CamelContext API:
FileEndpoint file = context.getEndpoint("file:xxx?aaa=123&bbb=456", FileEndpoint.class);
This allows you to pre configure the endpoint. And of course afterwards you can use the API on FileEndpoint to set other configurations.
In java, this is how to use GenericFileProcessingStrategy :
#Component
public class CustomGenericFileProcessingStrategy<T> extends GenericFileProcessStrategySupport<T> {
public CustomFileReadyToCopyProcessStrategy() {
}
public boolean begin(GenericFileOperations<T> operations, GenericFileEndpoint<T> endpoint, Exchange exchange, GenericFile<T> file) throws Exception {
super.begin(operations, endpoint, exchange, file);
...
}
public void commit(GenericFileOperations<T> operations, GenericFileEndpoint<T> endpoint, Exchange exchange, GenericFile<T> file) throws Exception {
super.commit(operations, endpoint, exchange, file);
...
}
public void rollback(GenericFileOperations<T> operations, GenericFileEndpoint<T> endpoint, Exchange exchange, GenericFile<T> file) throws Exception {
super.rollback(operations, endpoint, exchange, file);
...
}
}
And then create you route Builer class:
public class myRoutes() extends RouteBuilder {
private final static CustomGenericFileProcessingStrategy customGenericFileProcessingStrategy;
public myRoutes(CustomGenericFileProcessingStrategy
customGenericFileProcessingStrategy) {
this.customGenericFileProcessingStrategy = customGenericFileProcessingStrategy ; }
#Override public void configure() throws Exception {
FileEndpoint fileEndPoint= camelContext.getEndpoint("file://mySourceDirectory");
fileEndPoint.setProcessStrategy(myCustomGenericFileProcessingStrategy );
from(fileEndPoint).setBody(...)process(...).toD(...);
...
}