Camel Reslet Component with async processing - apache-camel

I have a requirement which is as follows:
Accept HTTP POST requests containing XML to a certain URL.
Perform pre-requisite actions such as saving the request XML to a file.
Validate the incoming XML matches the corresponding schema.
If the schema validation fails, synchronously respond with a HTTP 400 response code.
If the schema validation passes, synchronously respond with a HTTP 200 response code.
Pass the XML message on for further processing.
When this further processing completes, asynchronously respond to the caller with a HTTP 200 response code.
This is currently how I have the route configured:
onException(IOException.class)
.log(LoggingLevel.INFO, "Schema validation error on incoming message: ${id}")
.handled(true)
.maximumRedeliveries(0)
.process(schemaValidationErrorProcessor);
from("restlet:http://localhost:" + portNum + "/api/XX/XXX?restletMethod=POST")
.log(LoggingLevel.INFO, "Received message")
.convertBodyTo(String.class)
.multicast()
.parallelProcessing()
.to(SAVE_REQUEST_TO_FILE_QUEUE, PROCESS_PROVISIONING_REQUEST_QUEUE);
from(SAVE_REQUEST_TO_FILE_QUEUE)
.log(LoggingLevel.INFO, "Storing message: ${id}")
.to("file://" + requestLogFolder);
from(PROCESS_PROVISIONING_REQUEST_QUEUE)
.log(LoggingLevel.INFO, "Processing provisioning request: ${id}")
.process(requestGate)
.choice()
.when(header(SYSTEM_STATUS_HEADER).isEqualTo(true))
.unmarshal(xmlParser)
.inOnly("bean:requestHandler?method=handle")
.when(header(SYSTEM_STATUS_HEADER).isEqualTo(false))
.log(LoggingLevel.INFO, "Intentially dropping message")
.endChoice();
The schema validation part is achieved via the .unmarshal(xmlParser) line (I have a JaxbDataFormat object configured elsewhere with the schema set in that). When schema validation fails, an IOException is thrown and this is handled by my schemaValidationErrorProcessor which adds the HTTP 400 to the response.
That is all working fine.
The problem I am having is passing the XML message on for further processing. Basically, I need this to be done asynchronously because when the schema validation passes I need to synchronously respond with a 200 response. The processing that I need to do is in the .inOnly("bean:requestHandler?method=handle") line.
I naively thought that setting the routing to my bean to inOnly would set this to be asynchronous and that main route would not wait for a response. However, this is not the case as when the requestHandler.handle method throws an exception, this is thrown back to the caller of the REST endpoint. I don't want this to happen as I want all of this processing to be done in 'the background' as the consumer will have already received a 200 response.
So, my question is, how would I go about achieving such behaviour? I have thought about using queues etc but ideally would like to avoid such components if possible.

Use Camel Websocket component for asynchronously respond to the caller.
From the Camel documentation:
from("activemq:topic:newsTopic")
.routeId("fromJMStoWebSocket")
.to("websocket://localhost:8443/newsTopic?sendToAll=true&staticResources=classpath:webapp");

Related

Camel - onException & redelivery

I am trying to implement an error handler & re-oauth mechanism for a Route which performs a GraphQL API call. However, I seem to be misunderstanding the usage of the onException handler and can't seem to get it to work.
Here's the scenario I am trying to cover:
Execute a Route which performs a GraphQL API call (including the setting of variables etc.). The API call itself will always respond with a HTTP 200.
Once the response is received, check the response payload for any error records
If errors exist with a code 401, throw an AuthorizationException
The onException handler catches the exception and is supposed to:
trigger the oauth route to refresh the token (updating the exchangeProperty)
re-run the Route (re-executing the GraphQL call)
After reaching the configured maximumRedeliveries in the onException handler, fail the execution.
Here's the code I have so far:
onException(AuthorizationException.class)
.handled(true)
.redeliveryDelay(1000)
.maximumRedeliveries(3)
.useOriginalMessage()
.to(ROUTE_OAUTH_URI) // updates the token (?)
.to(ROUTE_QUERY_URI) // re-executes the route (?)
;
from(ROUTE_QUERY_URI)
.process(exchange -> {
...set variables
})
.toD("graphql://...")
.setProperty("graphqlResponseErrors").jsonpath("$.errors.length()", true)
.choice()
.when(exchangeProperty("graphqlResponseErrors").isNotEqualTo(0))
.setProperty("graphqlResponseCode").jsonpath("$.errors[0].code", true)
.choice()
.when(exchangeProperty("graphqlResponseCode").isEqualTo(401))
.throwException(new AuthorizationException(
"Authorization exception"))
.end()
.end();
I have tried a few different combinations, including onRedeliver instead of the oauth route, remove the useOriginalMessage and re-trigger the route again but I couldn't get it to work.
Any help or hints would be much appreciated. Thanks!
Update:
Adding the below code to the onException, does retrieve and set a new token, however now the route seems go into an infinite loop:
.onRedelivery(exchange -> {
ProducerTemplate producerTemplate = exchange.getContext().createProducerTemplate();
Exchange oauthExchange = producerTemplate.send(ROUTE_OAUTH_URI, exchange);
exchange.setProperty("accessToken", oauthExchange.getProperty("accessToken"));
});

Camel errorHandler / deadLetterChannel REST response

I have a Camel rest endpoint (Jetty) which validates and processes incoming requests. Besides specific Exception handlers (onException) it uses a DLQ error handler (errorHandler(deadLetterChannel...)) which is setup to retry 3 times - if unsuccessful the message is moved to the DLQ.
My question is, how do I still return a user friendly error message back to the client if an unexpected Exception occurs rather than the full Exception body? Is there some config I'm missing on the errorHandler?
I've tried to find some examples on the camel unit tests (DeadLetterChannelHandledExampleTest) and camel in action 2 (Chapter 11) but none seemed to have specific examples for this scenario.
Code is:
.from(ROUTE_URI)
.errorHandler(deadLetterChannel("{{activemq.webhook.dlq.queue}}")
.onPrepareFailure(new FailureProcessor())
.maximumRedeliveries(3)
.redeliveryDelay(1000))
.bean(ParcelProcessor.class, "process");
Thank you for your help!
Use a 2nd route as the DLQ, eg direct:dead and then send the message first to the real DLQ, and then do the message transformation afterwards to return a friendly response.
errorHandler(deadLetterChannel("direct:dead")
from("direct:dead")
.to("{{activemq.webhook.dlq.queue}}")
.transform(constant("Sorry something was wrong"));

Apache Camel: Unable to get the Exception Body

Whenever there is normal flow in my Camel Routes I am able to get the body in the next component. But whenever there is an exception(Http 401 or 500) I am unable to get the exception body. I just get a java exception in my server logs.
I have also tried onException().. Using that the flow goes into it on error, but still I do not get the error response body that was sent by the web service(which I get when using POSTMAN directly), I only get the request in the body that I had sent to the web service.
Also adding the route:
from("direct:contractUpdateAds")
.to("log:inside_direct:contractUpdateAds_route_CompleteLog?level=INFO&showAll=true&multiline=true")
.streamCaching()
.setHeader(Exchange.HTTP_METHOD, constant("POST"))
.setHeader(Exchange.CONTENT_TYPE, constant("application/json"))
.log("before calling ADS for ContractUpdate:\nBody:${body}")
.to("{{AdsContractUpdateEndpoint}}")
.log("after calling ADS for ContractUpdate:\nBody:${body}")
.convertBodyTo(String.class)
.end();
Option 1: handle failure status codes yourself
The throwExceptionOnFailure=false endpoint option (available at least for camel-http and camel-http4 endpoints) is probably what you want. With this option, camel-http will no longer consider an HTTP Status >= 300 as an error, and will let you decide what to do - including processing the response body however you see fit.
Something along those lines should work :
from("...")
.to("http://{{hostName}}?throwExceptionOnFailure=false")
.choice()
.when(header(Exchange.HTTP_RESPONSE_CODE).isLessThan(300))
// HTTP status < 300
.to("...")
.otherwise()
// HTTP status >= 300 : would throw an exception if we had "throwExceptionOnFailure=true"
.log("Error response: ${body}")
.to("...");
This is an interesting approach if you want to have special handling for certains status codes for example. Note that the logic can be reused in several routes by using direct endpoints, just like any other piece of Camel route logic.
Option 2 : Access the HttpOperationFailedException in the onException
If you want to keep the default error handling, but you want to access the response body in the exception handling code for some reason, you just need to access the responseBody property on the HttpOperationFailedException.
Here's an example:
onException(HttpOperationFailedException.class)
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
// e won't be null because we only catch HttpOperationFailedException;
// otherwise, we'd need to check for null.
final HttpOperationFailedException e =
exchange.getProperty(Exchange.EXCEPTION_CAUGHT, HttpOperationFailedException.class);
// Do something with the responseBody
final String responseBody = e.getResponseBody();
}
});

Handle exception from load balancer only once (when failover is exhausted)

My code consumes jms queue and through lb redirects to external http client.
I need to log original message for every failed delivery to local directory.
Problem is that onException is caught by each failover.
Is there any way how to achieve this?
Pseudo code:
onException(Exception.class).useOriginalMessage()
.setHeader(...)
.to("file...")
.setHeader(...)
.to("file...")
from("activemq...")
.process(...)
.loadBalance().failover(...)
.to("lb-route1")
.to("lb-route2")
.end()
.process()
.to("file...")
from("lb-route1")
.recipientList("dynamic url")
.end()
from("lb-route2")
.recipientList("dynamic url")
.end()
I have not tested this logic with multiple to statements, but when I have my list of endpoints in an array this logic functions just fine. I will attempt to deliver to the first endpoint, if I cannot I will attempt to deliver this to the second endpoint. If an exception occurs at the second endpoint it will propagate back to the route's error handler. If you need it to round robin instead, change the last false on the failover statement to a true.
String[] endpointList = {"direct:end1", "direct:end2"};
from("direct:start")
.loadbalance().failover(endpointList.length-1, false, false)
.to(endpointList);

Camel - Split() and doCatch(....) does not work

I'm trying to build a route that tries to validate an xml and if everything is correct then has to split this file otherwise an exception is thrown and it has to do something else. So I did the following:
from("file:"+fileOutboxTransformed+"?preMove=inprogress&move="+backupFolderTransformed+"/"+labelMessageType+"_${date:now:yyyyMMddHHmmssSSS}-${file:name.noext}.${file:ext}")
.log(LoggingLevel.INFO, "Got transformed file and sending it to jms queue: "+queue)
.doTry()
.to("validator:classpath:"+validator)
.split(xPathMessageTypeSplit)
.to("jms:"+queue+"?jmsMessageType=Text")
.doCatch(ValidationException.class)
.log(LoggingLevel.INFO, "Validation Exception for message ${body}")
.to("xslt:classpath:"+transformationsErrorAfter)
.split(xPathNotificationSplit)
.to("file:"+fileOutboxInvalid+"?fileName=${file:name.noext}-${date:now:yyyyMMddHHmmssSSS}.err2")
.end();
But it does not compile (if I do not use the split then it compiles and works) and the error is:
The method doCatch(Class<ValidationException>) is undefined for the type ExpressionNode
So I tried the following
from("file:"+fileOutboxTransformed+"?preMove=inprogress&move="+backupFolderTransformed+"/"+labelMessageType+"_${date:now:yyyyMMddHHmmssSSS}-${file:name.noext}.${file:ext}")
.log(LoggingLevel.INFO, "Got transformed file and sending it to jms queue: "+queue)
.doTry()
.to("direct:validate")
.doCatch(ValidationException.class)
.log(LoggingLevel.INFO, "Validation Exception for message ${body}")
.to("xslt:classpath:"+transformationsErrorAfter)
.split(xPathNotificationSplit)
.to("file:"+fileOutboxInvalid+"?fileName=${file:name.noext}-${date:now:yyyyMMddHHmmssSSS}.err2")
.end();
from("direct:validate")
.to("validator:classpath:"+validator)
.to("direct:split_message");
from("direct:split_message")
.split(xPathMessageTypeSplit)
.to("jms:"+queue+"?jmsMessageType=Text");
This time I get the error of duplicate endpoint
org.apache.camel.FailedToStartRouteException: Failed to start route route312 because of Multiple consumers for the same endpoint is not allowed: Endpoint[direct://validate]
do you have any idea on how to solve this problem?
In order to get back to the doTry() block from a split() block (or choice or other nested type), you need to use the endDoTry(). Contrary to its name, this method will end the nested split block and return back to the doTry() DSL.
I'd use the first route you posted with these changes:
from("file:"+fileOutboxTransformed+"?preMove=inprogress&move="+backupFolderTransformed+"/"+labelMessageType+"_${date:now:yyyyMMddHHmmssSSS}-${file:name.noext}.${file:ext}")
.log(LoggingLevel.INFO, "Got transformed file and sending it to jms queue: "+queue)
.doTry()
.to("validator:classpath:"+validator)
.split(xPathMessageTypeSplit)
.to("jms:"+queue+"?jmsMessageType=Text")
.endDoTry()
.doCatch(ValidationException.class)
.log(LoggingLevel.INFO, "Validation Exception for message ${body}")
.to("xslt:classpath:"+transformationsErrorAfter)
.split(xPathNotificationSplit)
.to("file:"+fileOutboxInvalid+"?fileName=${file:name.noext}-${date:now:yyyyMMddHHmmssSSS}.err2")
.endDoTry()
.end();
Your second try seems fine. The error you're getting is caused by two routes beginning with from("direct:validate")
Don't you have an other route in your application consuming from the same endpoint?
Edit : Try to name it differently, maybe validate exists already (in your app or inside camel)

Resources