I m using apache Camel to treat files.
However, i m asking if i can split a file depending on the number of byte.
For exemple , if i have a file with a size like 1 GO, is that possible to read it by block of 10 MO. (with a parameter that defines the size of a block)
Is there a component Camel EIP able to do this ?
I have tried with stream-caching but nothing happened.
<camelContext xmlns="http://camel.apache.org/schema/spring" trace="true">
<properties>
<property key="CamelCachedOutputStreamBufferSize" value="1024"/>
</properties>
<route id="READ-FILE-STREAM-CACHE" streamCache="true">
<from uri="file:src/data?noop=true"/>
<bean ref="readBean" method="readFileStream"/>
<to uri="file:src/out"/>
</route>
</camelContext>
Thanks
Related
I'm using a ActiveMQ Broker with built-in Camel Routes. I want to read a file after an Event received.
<pseudo>
from Event A
read File XY
to Event B with Body from File XY
</pseuod>
I simple tried moving files from a temporary directory based on an event but only event B is written. In the Log file are no Exceptions or Error messages.
<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring">
<!-- You can use Spring XML syntax to define the routes here using the <route> element -->
<route>
<description>Example Camel Route</description>
<from uri="activemq:example.A"/>
<from uri="file://tmp/a?delete=true"/>
<to uri="file://tmp/b?overruleFile=copy-of-${file:name}"/>
<to uri="activemq:example.B"/>
</route>
</camelContext>
Update with working solution for single file:
<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring">
<!-- You can use Spring XML syntax to define the routes here using the <route> element -->
<route>
<description>Example Camel Route</description>
<from uri="activemq:example.A"/>
<pollEnrich>
<constant>file:///tmp/a?fileName=file1</constant>
</pollEnrich>
<log message="file content ${body}"/>
<to uri="activemq:example.B"/>
</route>
</camelContext>
You need to use Content Enrichers for this. This is exactly what you are looking for.
<route>
<from uri="activemq:example.A"/>
<pollEnrich>
<constant>file://tmp/a?delete=true</constant>
</pollEnrich>
<to uri="activemq:example.B"/>
</route>
Please be aware that for camel version 2.15 or older
pollEnrich does not access any data from the current Exchange which
means when polling it cannot use any of the existing headers you may
have set on the Exchange. For example you cannot set a filename in the
Exchange.FILE_NAME header and use pollEnrich to consume only that
file. For that you must set the filename in the endpoint URI.
Below is my camel context, but reading vlaue from property file doesnt work, it is not listening to Queue which is mentioned in queueName. Is using poll
enrich A good Idea, because it uses direct component?
<bean id="bridgePropertyPlaceholder"
class="org.apache.camel.spring.spi.BridgePropertyPlaceholderConfigurer">
<property name="location" value="classpath:/config/queue.properties"/>
</bean>
<Route>
<from uri="activemq:queue:{{queueName}}/>
.......
</Route>
I'm trying to pick a file from a directory, split a file and add each splitted lines to activemq. I'm facing a problem with exception handling during this process. Lets say a file in the directory is a binary file (executable), then splitter thows org.apache.camel.RuntimeCamelException and java.nio.charset.MalformedInputException exceptions. If this occurs, then I need to catch these exceptions, nothing should add in activemq and that just particular thread should exit after logging exception. Referred online and wrote the following code but don't know how to stop adding to activemq and quit specific thread.
<route id="msg_producer">
<from uri="input.file.from" />
<doTry>
<split parallelProcessing="true" executorServiceRef="msgProducer"
streaming="true">
<tokenize token="\n"></tokenize>
<to uri="input.activemq.to" />
</split>
<doCatch>
<exception>org.apache.camel.RuntimeCamelException</exception>
<exception>java.nio.charset.MalformedInputException</exception>
<handled> <constant>true</constant></handled>
<setBody>
<simple>${exception.stacktrace}</simple>
</setBody>
<setHeader headerName="CamelFileName">
<simple>${file:onlyname.noext}_error.log</simple>
</setHeader>
</doCatch>
</doTry>
</route>
as #claus Ibsen said to filter the files you can use a filefilter property so that you pick only files based on extension and some standard pattern something like this
<bean id="FileFilter" class="org.apache.camel.component.file.AntPathMatcherGenericFileFilter">
<!-- ? matches one character
* matches zero or more characters
** matches zero or more directories in a path -->
<property name="includes" value="#{databaseProperties.getProperties().getProperty('file.name.pattern')}"/>
<!-- if you wan to exclude specific files say bad in name or .exe files. Use comma to separate multiple excludes -->
<!-- <property name="excludes" value="**/*bad*,**/*.exe"/> -->
</bean>
and your file.name.pattern can be something like this **/contract.csv
I have been playing around with camel-netty and camel-netty-http components, and trying to figure out what the setting maximumPoolSize does.
However, from what i observe based on this is that, the OrderPool always processes 16 concurrent requests. I am trying to change the maximumPoolSize to a value of 5 like the route as below,
<bean id="nettyBean" class="com.redhat.NettyTestImpl">
<property name="message" value="Netty maximumPoolSize test"/>
</bean>
<camelContext trace="false" xmlns="http://camel.apache.org/schema/blueprint" autoStartup="true">
<route>
<from uri="netty-http:http://localhost:8080/hello?maximumPoolSize=5&sync=true"/>
<log message="Forwarding to Netty component ....."/>
<setBody>
<method ref="nettyBean" method="sayHi"/>
</setBody>
<delay>
<constant>3000</constant>
</delay>
<log message="The body contains : ${body}"/>
</route>
</camelContext>
But it seems that i cannot get the maximumPoolSize to set to a value. So, what am i doing wrong ? How can i get the maximumPoolSize set ?
I check this by load testing with 20 concurrent requests and all are processed.
I just checked the code of camel-netty and camel-netty-http, maximumPoolSize is used for the OrderPool, you can use jconsoler to check the thread pool size.
As the OrderPool is used to handle the processor of camel, netty still has the work thread to handle the connection, which means it can still server 20 concurrent request, this time. But the TPS could be just about 5.
I am trying to read time delay from property file .
have defined in my property file :
time_inMilis=15000
I have configured my camel context xml to be :
<bean id="property" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations">
<value>file:/D:/Develop/resources/my.properties
</value>
</property>
</bean>
<camel:camelContext id="camel" xmlns="http://camel.apache.org/schema/spring">
<propertyPlaceholder id="properties" location="file:/D:/Develop/resources/my.properties"/>
<camel:route id="delayQueue">
<camel:from uri="seda:queue1" />
<delay asyncDelayed="true">
<constant>${time_inMilis}</constant>
</delay>
<camel:to uri="seda:queue2" />
</camel:route>
</camel:camelContext>
camel do not throw any error but it seems that it ignores ${time_inMilis} and set 0 for my delay time.
What is the right way to read the delay constant from my property file ?
First, it would be enough just to use camel:propertyPlaceholder instead of declaring bean property.
Second mistake is that you are using Constant instead of Simple expression when trying to read your time_inMilis property value.
Third, when trying to get value of you property, you should specifically tell Camel that your are looking at properties.
If your context defines propertiesPlaceholder like this:
<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring">
<propertyPlaceholder id="props" location="classpath:/org/smp/eip/sample.properties"/>
<package>org.apache.camel.example.spring</package>
</camelContext>
them with java DSL you'll be able to read the textProeprty value like this
from("file:src/data?noop=true")
.transform().simple("Text read from properties: ${properties:textProperty}")
.bean(new SomeBean());
Using Spring DSL from your original post, the correct way of reading property would be:
<camel:route id="delayQueue">
<camel:from uri="seda:queue1" />
<delay asyncDelayed="true">
<simple>${properties:time_inMilis}</simple>
</delay>
<camel:to uri="seda:queue2" />
</camel:route>