I have some problems with a following route:
// from("cxf:....")...
from("direct:start").process(startRequestProcessor) // STEP 1
.choice()
.when(body().isNull())
.to("direct:finish")
.otherwise()
.split(body()) // STEP 2
.bean(TypeMapper.class) // STEP 3
.log("Goes to DynamicRouter:: routeByTypeHeader with header: ${headers.type}")
.recipientList().method(Endpoint1DynamicRouter.class, "routeByTypeHeader") // STEP 4
.ignoreInvalidEndpoints();
from("direct:endpoint2") // STEP 6
.log("Goes to DynamicRouter::routeByCollectionHeader with header: ${headers.collection}")
.recipientList().method(Endpoint2DynamicRouter.class, "routeByCollectionHeader")
.ignoreInvalidEndpoints();
from("direct:endpoint1.1") // STEP 5
.process(new DateRangeProcessor())
.to("direct:collections");
from("direct:endpoint1.2") // STEP 5
.process(new SingleProcessor())
.to("direct:collections");
from("direct:endpoint2.2") // STEP 7
.aggregate(header("collection" /** endpoint2.2 */), CollectionAggregationStrategy)
.completionSize(exchangeProperty("endpoint22"))
.process(new QueryBuilderProcessor())
.bean(MyService, "getDbCriteria")
.setHeader("collection", constant("endpoint2.1"))
.to("direct:endpoint2.1").end();
from("direct:endpoint2.1") // STEP 8
.aggregate(header("collection" /** endpoint2.1 */), CollectionAggregationStrategy)
.completionSize(exchangeProperty("CamelSplitSize"))
.to("direct:finish").end();
from("direct:finish")
.process(new QueryBuilderProcessor())
.bean(MyRepository, "findAll")
.log("ResponseData: ${body}").
marshal().json(JsonLibrary.Gson).end();
The route
Receives json string an converts it to list (HashSet) of JSONObjects.
split the received list to json objects.
Set corresponding headers according to object content
Routes the messages according to headers to endpoint1.1 or endpoint1.2
Convert messages to mongodb Criteria and send to endpoint2
Endpoint2 routes messages according to another header to endpoint2.1 or endpoint2.2.
Endpoint2.2 aggregates all received messages, processes it to get mongodb Criteria and sends it to endpoint2.1 (completionSize is calculated at step 2 and saved in property "endpoint22").
Enpoint2.1 aggregates ALL messages (CamelSplitSize) converts aggregated messages to Query object and sends it to Repository to retrieve the data.
I can see valid response object in debugger but anyway I get an error:
No message body writer has been found for class java.util.HashSet, ContentType: application/json
The problem is not in response object as it works with other routes and it does not contain HashSets.
My guess is that route sends to the output the HashSet created tat STEP 1...
My questions are:
what is wrong in the route output?
both recipientList() try to forward
messages to invalid endpoint ( I have to use .ignoreInvalidEndpoints() to avoid exception):
org.apache.camel.NoSuchEndpointException: No endpoint could be found for:
org.springframework.data.mongodb.core.query.Criteria#20f55e70, please
check your classpath contains the needed Camel component jar.
Any help would be much appreciated!
Thanks.
I find it very strange, but .aggregate() function does not reply exchange. It uses you aggregation strategy but always reply incoming exchange. This is not clear when reading documentation, but you have to use aggregation strategy along with split() to be able to return exchange.
Related
I'd like to ask if there is some way to skip rest of the split route and jump directly to aggregator part, when in the exception handler I mark the split route to continue.
I have a route like this:
receive a message
fetch config for 3 endpoints
merge config and message as a tuple for each endpoint, and create a list of it
.split(), and in the split route I convert message according to config for each endpoint(s1), fetch oauth token(s2), send to final endpoint with token(s3), collect response for each endpoint(s4), aggregate(split aggregator; splitting ends here, let's call it sa)
return as a whole one result
stop
You can see, in the split route there are 4 steps(s1-s4); if any of these step fails I want to jump to aggregation(sa). For example, it does not make sense to continue the split route, if s1 or s2 fails.
I define an onException() clause to handle the exception and mark it to continue(continued(true)),because anyway I want to reach aggregator. Also, if I mark continue(false), not only split route, but the whole route(meaning the main route even before splitting) will be rolled back. I want to decide rollback after getting all the causes/exceptions in each split branch.
I have a workaround for a simple case, which is, in exception handler for errors in s2, I add a property in the exchange oauth_failed to be true, and add a condition check choice().when() after s2; if this prop is null, then go to s3 (continue sending). Solely for this purpose I must isolated s3 as a separate route(direct:s3).
.bean(S2Bean.class)
.choice()
.when(simple("${exchangeProperty.oauth_failed} == null")) // null = continue the flow
.to("direct:s3")
.endChoice()
// otherwise, it will skip s3 and s4, and jump to aggregator directly
.end()
But, what can I do if s1 throws exception? Do I need to isolate s2 as a direct endpoint too? Then each step in the pipeline should be a separate endpoint. I don't like that.
Find a solution: use doTry and doCatch in split route and don't .stop().
from("direct:split")
.doTry()
.bean(S1Bean.class)
.bean(S2Bean.class)
.bean(S3Bean.class)
.bean(S4Bean.class)
.endDoTry()
.doCatch(javax.ws.rs.ProcessingException.class) // oauth timeout
.log(LoggingLevel.ERROR, "Time out, never retry, just aggregate")
.bean(MyGenericExceptionHandler.class)
.doCatch(Exception.class)
.log(LoggingLevel.ERROR, "Other exceptions, mark as failed, aggregate")
.bean(MyGenericExceptionHandler.class)
.end();
And in the MyGenericExceptionHandler, exchange.getIn().setBody(xxx) to set body to the expected type which my aggregator needs. The exception is in exchange.getProperty(Exchange.EXCEPTION_CAUGHT, Exception.class), response code is null. (I create a dto to contain both status code and/or exception, so that either success or failure, I aggregate with same class)
Don't call stop().
I need to configure some camel routes based on some configuration files.
All configured routes will need to split a message into one or two sub messages then do some JMS integration work on the first one and then aggregate together the JMS reply with the optional second message. In a simplified picture it will look like below:
message -- > split --> message 1 --> JMS request/reply --> aggregate --> more processing
\--> message 2 /
The aggregation will be done on completion size which I am able to know upfront if it is going to be 1 or 2 depending of the route meta data. When the second message is present no other processing is needed before being merged back with the JMS reply.
Si in short I need a split followed by a routing followed by an aggregation which is quite a common pattern. The only particularity is is that in case the second split message is present I don't need to do anything on it before aggregating it back.
In java DSL it will looks something like this:
from("direct:abc")
// The splitter below will set the JmsIntegration flag
.split().method(MySplitter.class, "split")
.choice()
.when(header("JmsIntegration"))
.inOut("jms:someQueue"))
.otherwise()
// what should I have on here?
.to(???)
.end()
.aggregate(...)to(...);
So my questions would be:
What should I put on the otherwise branch?
What I need in fact is an if: if the split message needs JMS go to JMS and then move to aggregator if it is not just go straight to the aggregator. I am considering creating a dummy processor which will actually do nothing but this seems to me a naive approach.
Am I on a wrong path. If so what would be the alternative
Initially I was thinking about a message enricher but I would not like to sent the original message to the JMS
I also considered putting my aggregation strategy inside my splitter but again I could not put it all together.
Based off your post it looks like you are trying to have the return of your enrichment merge with the original message, but you want to send a custom message to the jms endpoint. I would recommend storing your original message in either a bean or a cache or something of the sort, leveraging all of your conversions with camel and then have your aggregation strategy leverage your storage to return your desired format.
from("direct:abc")
.split().method(MySplitter.class, "split")
.choice()
.when(header("JmsIntegration"))
.beanRef("MyStorageBean", "storeOriginal")
.convertBodyTo(MyJmsFormat.class)
//This aggregation strategy could have a reference
//to your storage bean and retrieve the instance
.enrich("jms:someQueue", myCustomAggreationStrategyInstance)
.otherwise()
.end()
.aggregate(...)
.to("direct:continueProcessing");
Option #2: Based off of your comment saying you needed the "original message that the direct:abc endpoint received this can be simplified a lot. In this example we can use camel's existing Original message store to retrieve the message that was passed into direct:abc. If Your message after the split has a JmsIntegration header we will convert the body to the desired format for the jms call, leverage the enrich statement to make the jms call and a custom aggregator that gives you access to the message used to call the jms endpoint, the message that came back, and the original message direct:abc has. If your flow does not have a JmsIntegration header the message will go to the Otherwise statement in your route which does no additional processing before ending the choice statement and then the spit messages are aggregated back together with whatever custom strategy you need.
from("direct:abc")
.split().method(MySplitter.class, "split")
.choice()
.when(header("JmsIntegration"))
.convertBodyTo(MyJmsFormat.class)
//See aggregationStrategy sample below
.enrich("jms:someQueue", myAggStrat)
.otherwise()
//Non JmsIntegration header messages come here,
//but receive no work and are passed on.
.end()
.aggregate(...)
.to("direct:continueProcessing");
//Your Custom Aggregator
public Exchange aggregate(Exchange oldExchange, Exchange newExchange) {
//This logic will retrieve the original message passed into direct:abc
Message originalMessage =(Message)exchange.getUnitOfWork().getOriginalInMessage();
//TODO logic for manipulating your exchanges and returning the desired result
}
You said you considered using Enricher, but you don't want to send raw message. You can resolve this neatly by using a pre-JMS route:
from("direct:abc")
.enrich("direct:sendToJms", new MyAggregation());
.to("direct:continue");
from("direct:sendToJms")
// do marshalling or conversion here as necessary
.convertBodyTo(MyJmsRequest.class)
.to("jms:someQueue");
public class MyAggregation implements AggregationStrategy {
public Exchange aggregate(Exchange original, Exchange resource) {
MyBody originalBody = original.getIn().getBody(MyBody.class);
MyJmsResponse resourceResponse = resource.getIn().getBody(MyJmsResponse.class);
Object mergeResult = ... // combine original body and resource response
original.getIn().setBody(mergeResult);
return original;
}
}
Splitter automatically aggregates split exchanges back together. However, default (since 2.3) aggregation strategy is to return the original exchange. You can easily override the default strategy with your own by specifying it directly on the Splitter. Furthermore, if you don't have an alternative flow for your Choice, then it's much easier to use Filter. Example:
from("direct:abc")
.split().method(MySplitter.class, "split").aggregationStrategy(new MyStrategy())
.filter(header("JmsIntegration"))
.inOut("jms:someQueue"))
.end()
.end()
.to(...);
You still need to implement MyStrategy to combine the two messages.
I'm trying to build a Split / Aggregate pattern in a Camel Route that consumes a REST endpoint. It takes a request object that contains a list of request details. I want to parallel process the request details and then return an aggregated result back to the caller. I want this to be a Synchronous call.
Here's the code in my Route.
from("{{generate.route.endpoint}}")
.routeId(EXAMPLE_ROUTE_ID)
.split().method(RequestDetailsSplitter.class).stopOnException().parallelProcessing()
.to("direct:processRequestDetails")
.aggregate(header(TRANSACTION_ID_PARAM), responseAggregator)
.completionSize(simple("${property.CamelSplitSize}"))
.completionTimeout(5000L).parallelProcessing()
.log(LoggingLevel.INFO, "After Aggregation ---> ${body}")
.end()
.removeHeaders("*")
.setHeader(Exchange.HTTP_RESPONSE_CODE,
simple(String.valueOf(HttpStatus.SC_CREATED)));
I would expect that the result of the call is the output of Aggregate call, my response object. But what I actually get is the request object returned from the REST call??
When I put more logging statements in I can see that the Split call is firing multiple threads, which is great. And I can see the log statement above 'After Aggregation --->' with the response I want, this has it's own thread. But what happens to this output? How can I get that back into my default Exchange so it can returned to the REST call.
From Apache Camel: Splitter:
What the Splitter returns
Camel 2.3 and newer:
The Splitter will by default return the original input message.
For all versions
You can override this by suppling your own strategy as an AggregationStrategy.
If you want to aggregate the result of a split, you just need something like to:
from("{{generate.route.endpoint}}")
.routeId(EXAMPLE_ROUTE_ID)
.split().method(RequestDetailsSplitter.class, responseAggregator)
.stopOnException().parallelProcessing()
.to("direct:processRequestDetails")
.end()
.log(LoggingLevel.INFO, "After Split aggregate ---> ${body}")
.removeHeaders("*")
.setHeader(Exchange.HTTP_RESPONSE_CODE, simple(String.valueOf(HttpStatus.SC_CREATED)));
I would like to achieve the following kind of orchestration with CAMEL:
Client sends a HTTP POST request to CAMEL
CAMEL sends HTTP POST request to external endpoint (server)
External server replies with a 200 OK
CAMEL sends HTTP GET request to external endpoint (server)
External server replies
After step 5, I want to check the reply: if the reply is a 200 OK and state = INPROGRESS (this state can be retrieved from the received XML body), I want to re-transmit the HTTP GET to the external endpoint until the state is different from INPROGRESS.
I was thinking to use the retryWhile statement, but I am not sure how to build the routine within the route.
Eg, for checking whether the reply is a 200 OK and state = INPROGRESS, I can easily introduce a Predicate. So the retryWhile already becomes like:
.retryWhile(Is200OKandINPROGRESS)
but where should I place it in the route so that the HTTP GET will be re-transmitted ?
Eg: (only taking step 4 and 5 into account)
from("...")
// here format the message to be sent out
.to("external_server")
// what code should I write here ??
// something like:
// .onException(alwaysDo.class)
// .retryWhile(Is200OKandINPROGRESS)
// .delay(2000)
// .end ()
// or maybe it should not be here ??
I am also a bit confused how the "alwaysDo.class" should look like ??
Or ... should I use something completely different to solve this orchestration ?
(I just want to re-transmit as long as I get a 200 OK with INPROGRESS state ...)
Thanks in advance for your help.
On CAMEL Nabble, someone replied my question. Check out:
http://camel.465427.n5.nabble.com/Camel-Apache-can-I-use-a-retryWhile-to-re-send-a-request-td5498382.html
By using a loop statement, I could re-transmit the HTTP GET request until I received a state different from INPROGRESS. The check on the state needs to be put inside the loop statement using a choice statement. So something like:
.loop(60)
.choice()
.when(not(Is200OKandINPROGRESS)).stop() // if state is not INPROGRESS, then stop the loop
.end() // choice
.log("Received an INPROGRESS reply on QueryTransaction ... retrying in 5 seconds")
.delay(5000)
.to(httpendpoint")
.end() //loop
I never experimented what you are trying to do but it seems does not seem right.
In the code you are showing, the retry will only occur when an alwaysDo Exception is thrown.
The alwaysDo.class you are refering to should be the name of the Java Exception class you are expecting to handle. See http://camel.apache.org/exception-clause.html for more details.
The idea should be to make the call and inspect the response content then do a CBR based on the state attribute. Either call the GET again or terminate/continue the route.
You probably should write a message to the Apache Camel mailing list (or via Nabble) . Commiters are watching it and are very reactive.
I would like to obtain the following kind of routing:
HTTP POST message with XML body enters CAMEL
I store some of the parameters of the XML body
The message is routed to an external endpoint
The external endpoint (external server) replies
-> at this moment, I would like to check whether the reply from the external endpoint is a HTTP 200 OK containing a XML parameter equal to SUCCESS.
-> if so, then I would like to use some of the stored parameters to construct a new HTTP message (method = PUT this time) and send it out to an external endpoint
Problem that I am currently having, is the following:
.choice()
.when(simple("${in.headers.CamelHttpResponseCode} == 200"))
// now I want do a few things, eg: check also the XML body via xpath
// and change the message to be sent out (change Method to PUT, ...)
.to("http://myserver.com")
.otherwise()
// if no 200 OK, I want the route to be stopped ... not sure how ?
.end()
Question: any idea how to add those extra statements in case the HTTP response code was 200 OK ? It looks like the when does not allow me to add extra statements ...
(I got an error in my Eclipse IDE).
Thanks in advance.
Note: could it be that I have to route the message in case the 200 OK matches to a 'new endpoint' and then create a new from route with this new endpoint ?
Eg:
.choice()
.when(simple("${in.headers.CamelHttpResponseCode} == 200"))
.to("mynewendpoint")
.otherwise()
// if no 200 OK, I want the route to be stopped ... not sure how ?
.end();
from("mynewendpoint").
.setHeader(etc etc)
.to("http://myserver.com")
In this latter case, how exactly should I define this 'newendpoint' ?
In the programming language DSLs such as Java, you can build predicates together. I posted a blog entry some years ago about this at: http://davsclaus.blogspot.com/2009/02/apache-camel-and-using-compound.html
For example having two predicates
Predicate p1 = header("hl7.msh.messageType").isEqualTo("ORM"):
Predicate p2 = header("hl7.msh.triggerEvent").isEqualTo("001");
You can chain them together, using and or or.
Predicate isOrm = PredicateBuilder.and(p1, p2);
And then you can use isOrm in the route
from("hl7listener")
.unmarshal(hl7format)
.choice()
.when(isOrm).beanRef("hl7handler", "handleORM")
.otherwise().beanRef("hl7handler", "badMessage")
.end()
.marshal(hl7format);
yep, you can have multiple statements between the .when() and .otherwise() and you can always call .endChoice() to explicitly end each conditional block...
to your other question, you can use camel-direct to chain together multiple routes, etc...