I'm trying to make a route that will process big xml files using camel-stax. A file content processing works fine, but at the end it fails with a following error:
Caused by: java.io.IOException: Renaming file from: C:\workdir\file.xml to: C:\workdir\.camel\file.xml failed due cannot delete from file: C:\workdir\file.xml after copy succeeded
It seems that camel doesn't close a file input stream, so after processing it cannot move a file to a target location. Of course, I can set noop=true, bit I wanted to remove processed files.
My route looks like following:
<route id="myRoute">
<from uri="file:{{working_dir}}?include=file.xml" />
<split streaming="true">
<ref>staxRecord</ref>
<to uri="log:test"/>
</split>
</route>
Initially it was a little bit more complex and I simplified it as possible. Now it looks just like a last sample from here http://camel.apache.org/stax.
Additional note: I execute the route on Windows. Camel version: 2.12.2.
So it looks like a bug in the camel-stax component.
I've found an alternative way of how to deal with big xml files. I've rewritten my route as following:
<route id="myRoute">
<from uri="file:{{working_dir}}?include=file.xml&delete=true" />
<split streaming="true">
<tokenize token="entry" xml="true"/>
<unmarshal ref="myJaxb"/>
<!-- ... -->
</split>
</route>
Related
I have created multiple routes(say department, Employee) which takes input from file system folders(say department, Empployee) and process those files.
Now, I want to make them dependent. So, if I upload both emp.csv and dept.csv in those folders then it will process department file first and once complete it will start processing file for employee.
is there any way in camel to achieve this.
I looked at Route startupOrdering and AutoStartup feature, but it will work only for the first time when starting routes. However, I need same behavior for entire route life.
Thanks.
<route id="b" xmlns="http://camel.apache.org/schema/spring">
<from uri="file:/home/dev/code/Integration/RunCamleExample/src/main/resources/csv/Department?repeatCount=1&noop=true&delay=10000"/>
<log message="Department data is : ${body}"/>
</route>
<route id="employee" xmlns="http://camel.apache.org/schema/spring">
<from uri="file:/home/dev/code/Integration/RunCamleExample/src/main/resources/csv/Employee?noop=true&delay=10000"/>
<log message="Employee data is : ${body}"/>
</route>
I suggest to use other logic to handle the task. Two simple ways to go:
Use pollEnrich
Use pollEnrich to collect extra resource (e.g. a file with known name in file system) once at the middle of a route
Flow: Collect department files (From Endpoint) --(for each department file from file system) -> collect single employee file (trigger pollEnrich once with known name) ----> do anything else (if any)
Use ControlBus
Use ControlBus component to control the status of routes (only one of the route in 'start' status)
Flow: Start route A --(when route A complete its goal)-> Suspend route A ---> Start route B --(when route B complete its goal)-> Suspend route B ---> Start route A [loop back to head]
Dependent route execution first can be achieved in Camel using "RouteContext".
Example: If route 'A' is executed before route 'B' then route 'A' should be defined as 'RouteContext' and route be is defined inside "camelContext" like below:
<routeContext id="A" xmlns="http://camel.apache.org/schema/spring">
<route id="A">
<from uri="file:/home/dev/code/Integration/RunCamleExample/src/main/resources/csv/Department?repeatCount=1&noop=true&delay=10000"/>
<log message="Department data is : ${body}"/>
</route>
</routeContext>
Then regular "camelContext" should be defined with reference to this routeContext first.
<camelContext id="test" xmlns="http://camel.apache.org/schema/spring">
<routeContextRef ref="A"/>
<route id="B">
<from uri="file:/home/dev/code/Integration/RunCamleExample/src/main/resources/csv/Employee?noop=true&delay=10000"/>
<log message="Employee data is : ${body}"/>
</route>
</camelContext>
I'm using a ActiveMQ Broker with built-in Camel Routes. I want to read a file after an Event received.
<pseudo>
from Event A
read File XY
to Event B with Body from File XY
</pseuod>
I simple tried moving files from a temporary directory based on an event but only event B is written. In the Log file are no Exceptions or Error messages.
<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring">
<!-- You can use Spring XML syntax to define the routes here using the <route> element -->
<route>
<description>Example Camel Route</description>
<from uri="activemq:example.A"/>
<from uri="file://tmp/a?delete=true"/>
<to uri="file://tmp/b?overruleFile=copy-of-${file:name}"/>
<to uri="activemq:example.B"/>
</route>
</camelContext>
Update with working solution for single file:
<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring">
<!-- You can use Spring XML syntax to define the routes here using the <route> element -->
<route>
<description>Example Camel Route</description>
<from uri="activemq:example.A"/>
<pollEnrich>
<constant>file:///tmp/a?fileName=file1</constant>
</pollEnrich>
<log message="file content ${body}"/>
<to uri="activemq:example.B"/>
</route>
</camelContext>
You need to use Content Enrichers for this. This is exactly what you are looking for.
<route>
<from uri="activemq:example.A"/>
<pollEnrich>
<constant>file://tmp/a?delete=true</constant>
</pollEnrich>
<to uri="activemq:example.B"/>
</route>
Please be aware that for camel version 2.15 or older
pollEnrich does not access any data from the current Exchange which
means when polling it cannot use any of the existing headers you may
have set on the Exchange. For example you cannot set a filename in the
Exchange.FILE_NAME header and use pollEnrich to consume only that
file. For that you must set the filename in the endpoint URI.
Im having trouble trying to attach different paths to a FROM Uri in the Xml configuration file, in java it can be done like this:
String[] uris = new String[]{"file:source/path1","file:source/path2"};
from (uris).to("file:dest/path")
The resulting route will move the files from the source paths to the destination path, how can i achieve this using Spring XML? i have been trying different aproachs and can't find to have any of them working
<route id="bar">
<from uri= "file:source/path1,file:source/path2" />
<to uri="file:dest/path"/>
</route>
Fabian
You can have multiple from :
<route id="bar">
<from uri="file:source/path1"/>
<from uri="file:source/path2"/>
<to uri="file:dest/path"/>
</route>
This create one RouteDefinition, but 2 Route instances
I am using Apache Camel blueprints, my route is triggered from a URL at port 8081 on localhost. This route generates a file but the problem is the file is a binary file without the HTML showing.
I then navigate my browser to http://localhost:8081/foo which triggers the URL.
What is it that I don't understand that is causing it to output a binary file rather than a text file? I guess that I must transform the body somehow?
<route id="url1">
<from uri="netty4-http:http://0.0.0.0:8081/foo" />
<to uri="http://www.google.com/?bridgeEndpoint=true" />
<to uri="file:/test"/>
</route>
Update: I think the problem is that the content coming back from www.google.com is gzip. When I look at the bridgeEndpoint parameter in the documentation it mentions something about gzip... now I am not sure, as I tried it on another website and it still doesn't work.
This line in my log might be relevant.
writing body: DefaultFullHttpResponse(decodeResult: success,
version: HTTP/1.1, content: UnpooledUnsafeDirectByteBuf(ridx: 0, widx: 0, cap: 0))
Update: I discover if I do:
<from uri="timer:secondfoo?period=20s" />
Replacing the from, then it works. Hmm... Something flowing from the netty4-http causes problem.
Update: I found something which works! Obsession pays off.
<route id="url1">
<from uri="netty-http:http://0.0.0.0:8081/foo" />
<removeHeaders pattern="*" />
<setBody>
<simple></simple>
</setBody>
<setHeader headerName="CamelHttpMethod">
<constant>GET</constant>
</setHeader>
<to uri="http://www.google.com/?bridgeEndpoint=true" />
<to uri="file:/test"/>
</route>
You can convert the message to a String type then its text based
<route id="url1">
<from uri="netty4-http:http://0.0.0.0:8081/foo" />
<to uri="http://www.google.com/?bridgeEndpoint=true" />
<convertBodyTo type="String"/>
<to uri="file:/test"/>
</route>
I have a camel route:
from("file:///u01/www/images/nonprofits-test?move=.done&preMove=.processing&filter=#nonpFileFilter&minDepth=2&recursive=true")
Later on in the route I need to access the origin file name. How do I get that information? All of the headers contain information in like ${file:name}, but not the actual file name.
Thanks in advance!
The base problem is that simple language is not being evaluated correctly in while running Camel with grails. This is being discussed further on the Camel user list.
there is a header called "CamelFileName" that stores this
see camel-file2 headers section for more details...
If your simple language is not working it would be because you are not using <simple> tag try something like below.
<route id="movedFailedFileForRetry">
<from uri="file:///opt/failed?delete=true" />
<log loggingLevel="INFO" message="Moving failed file ${header.CamelFileName} for retry" />
<choice>
<when>
<simple>${headers.CamelFileName} == 'file1.txt'</simple>
<to uri="file:///opt/input1" />
</when>
<otherwise>
<to uri="file:///opt/input2" />
</otherwise>
</choice>
</route>
Hope it helps!!
${headers.CamelFileName} will provide you with the CamelFileName that is read for processing. We have many other header properties that you can find from the Camel Documentation.