I am new to camel and we are building an EDI engine and our requirement is to read the files from multiple folders then second step is to parse the message type and the receiver id and based on that the messages need to be routed to different folders.
The source, message type, receiver id and destination cannot be hardcoded in camel instead it should be read from the database and the routes need to be built dynamically.
Please let me know what should be the strategy that we need to follow.
Thanks,
Jayadeep
As I understand from your comments, you can read from multiple folders by adding routes dynamically but are facing issue when trying to decide on where to send the messages as the destination , headers etc is being read from database.
Here's how I would do it.
Get the file --> Enrich it with database call and get the receiever id etc --> Use Xpath and get the receiver id etc and set them in propertyor header --> Use XSLT and remove the values that you enriched for database call so now you have the original message ---> Now use router and look at properties/headers to decide the <camel:to> path
Related
I need to create a file filter (to pick files from a folder) based on the content received from another file.
I set up a route like this:
File1 Url -> pollEnrich(File2 Url with filter, aggregationStrategy) -> log
But the issue is that in pollEnrich, the value obtained from File1 is not available. Hence I am not able to create a filter based on which I have to pick the files from Folder2.
I tried both the filer option in the URL as well as the programmatic filter (by extending GenericFileFilter class). Any suggestions are very much appreciated.
Recall the fact from content-enricher,
pollEnrich only accept one message as response
The pollEnrich will collect single file when call with file component. Thus, you should use file component with fileName option inside pollEnrich to collect single file and use while loop to call pollEnrich multiple time.
I have a camel route that splits and aggregates according to some ids. When an id is retrieved, a call is made to another endpoint to retrieve the project information according to this id. After retrieving the project information i had to enrich it by calling multiple enrich() methods on it. In the first enrich method i have to do some xpath processing wherein ill be able to retrieve a primaryOrgId value that i will set as a property in the exchange, dont worry about the xpath processing, i had that sorted out but my problem is when I set the property (primaryOrgId) inside the 1st enrich. The property value doesn't get persisted when the route goes to the 2nd enrich part. When I log the primaryOrgId value, the original value of "testValue" (this was set in the direct:createSomeIds route) is the one getting displayed instead of "changeTheValueHere" which was set in the 1st enrich part.
I am using Camel 2.15 based on Fuse 6.2.1.
I went to the camel site and read this part from http://camel.apache.org/content-enricher.html . I'm not sure I understood how to implement... "For that you must set the filename in the
endpoint URI" .. this text was talking about the header, i'm thinking its also applicable to the properties in the exchange.
pollEnrich or enrich does not access any data from the current
Exchange which means when polling it cannot use any of the existing
headers you may have set on the Exchange. For example you cannot set a
filename in the Exchange.FILE_NAME header and use pollEnrich to
consume only that file. For that you must set the filename in the
endpoint URI.
Here is my code:
from("direct:createSomeIds")
.routeId("createSomeIds")
.process(new IdCreatorProcessor()
.setProperty("primaryOrgId").constant("testValue")
.split(xpath("/TempProjects/TempProject/Code/text()").namespaces(ns) , new IdStrategy())
.to("direct:splitRouteById")
.end();
from("direct:splitRouteById")
.routeId("splitRouteById")
.to("direct:getProjectByID")
.to("xquery:template/AllProjectToSingleProject.xq") //xquery template
.convertBodyTo(Project.class)
.enrich("direct:getAdditionalInfo", new ProjectStrategy(ProjectStrategy.AggregatorType.AdditionalInfo))
.enrich("direct:getSecondaryInfo", new ProjectStrategy(ProjectStrategy.AggregatorType.SecondaryInfo))
.end();
from("direct:getAdditionalInfo")
//some xpath stuff here
.setProperty("primaryOrgId").constant("changeTheValueHere")
.end();
from("direct:getSecondaryInfo")
.log("Value of primaryOrgId = " + "${exchangeProperty.primaryOrgId}")
.end();
If you can provide some code example, that would be helpful.
If you read a bit further down you will see that it's recommended that you instead use RecipientList with an AggregationStrategy.
.recipientList("direct:getAdditionalInfo", "direct:getSecondaryInfo")
.aggregationStrategy(new ProjectStrategy())
The setting of filename in your endpoint URI would only be applicable if you were to access some file on an FTP or some other file area.
Edit:
I now see that you need the property from the first enrichment in your second enrichment. However, if you're not modifying the message body in the first enrich then I don't actually see the need for it at all.
If you are in fact modifying the body then you can still use the RecipientList but instead you use two separate ones calling only one endpoint in each.
I want to download xml files from an S3 bucket, then unmarshal them to insert the data in a database:
...
from("aws-s3://myBucket?accessKey=xxx&secretKey=yyy")
.to("file:D:/output?fileName=${in.header.CamelAwsS3Key}")
.unmarshal(new JaxbDataFormat("com.xxx"))
...
I am able to download the files when I don't try to unmarshal, but I get this error when I try to unmarshal:
org.apache.camel.TypeConversionException:
Error during type conversion from type:
com.amazonaws.services.s3.model.S3ObjectInputStream to the required type:
javax.xml.stream.XMLStreamReader with value com.amazonaws.services.s3.model.S3ObjectInputStream#67b8328b due javax.xml.stream.XMLStreamException:
java.io.IOException: Attempted read on closed stream.
Since I am new to Camel there are maybe things I didn't understand yet...
When I pipe endpoints, doesn't the current endpoint get the message the way the previous endpoint "modified" it? In my case it looks like the S3 stream is being marshalled, instead of the xml file newly created locally from the download, hence the error.
My understanding is if I do .from().to().to(), the second .to() doesn't know what is coming from .from() so if my first .to() creates an xml file, the second .to() handles the message as an xml file. Am I wrong?
Maybe I need to create 2 routes? I was able to do the other way around with only 1 route though, from database to file to S3.
Do I need to write my own converter in that case?
Thanks!
When you have the 2nd to Camel needs to read the stream again, but its closed. Its becomes sometimes streams are only readable once. And it was read in the first to, where you write it to a file.
So you can either enable stream caching to allow Camel to be able to re-read the stream again. See more in this FAQ and the links
http://camel.apache.org/why-is-my-message-body-empty.html
http://camel.apache.org/stream-caching.html
You can also break it into 2 routes
from amq
to file
from file
unmarshal
i have a processor which puts an unmarshaled ical file in a custom object. After this a list of my object is saved into exchange.
After this i need to put a second ical into exchange. Problem is that i cant use "from" two times in a route. How can i solve this?
Code for processing one ical is the following:
from("file://src/test/resources?fileName=Calendar.ics&delete=false&noop=true")
.unmarshal("ical")
.process(icsToBeanProcessor)
.end();
Perhaps you can improve the design a bit. Instead of specifying filenames use a directory instead. Process all files present in the directory and set them in header properties. Something like
from("file://src/test/resources?delete=false&noop=true")
.unmarshal("ical")
.process(icsToBeanProcessor) // in your icsToBeanProcessor set a header property as content.fileName as the output of your unmarshaller bean
.end();
Also you can be more specific by using a regex for specifying filenames. For your case pick only those files which ends with "ics". See http://camel.apache.org/file.html for more details.
I have a scenario where multiple files will be dropped at a location. I am interested in reading two files out of it as a pair (e.g., 913.xml and 913.pdf). I want to pass both files to a Java component which will merge them.
I know how to merge them but I am not sure how to pass two files at once to my Java component
I can think of two options:
Configure the file inbound endpoint to select say the XML files and in your component use the MuleClient to request the other file,
Configure the file inbound endpoint to select both XML and PDF files and add a collection-aggregator configured with an expression-message-info-mapping that uses the originalFilename property in a correlation ID expression that extracts the file name sans extension from it. Don't forget to set the MULE_CORRELATION_GROUP_SIZE property to 2 before the aggregator.