I am getting a strange situation at the code below which simply routes request to Google and returns response.
It works well but when I activate the line commented out as "//Activating this line causes empty response on browser" to print out returned response from http endpoint (Google), response is disappear, nothing is displayed on browser. I thought it might be related with input stream of http response which can be consumed only once and I activated Stream Caching on context but nothing changed.
Apache Camel version is 2.11.0
Any suggestions are greatly appreciated, thanks in advance.
public class GoogleCaller {
public static void main(String[] args) throws Exception {
CamelContext context = new DefaultCamelContext();
context.addRoutes(new RouteBuilder() {
public void configure() {
from("jetty:http://0.0.0.0:8081/myapp/")
.to("jetty://http://www.google.com?bridgeEndpoint=true&throwExceptionOnFailure=false")
.process(new Processor() {
public void process(Exchange exchange) throws Exception {
System.out.println("Response received from Google, is streamCaching = " + exchange.getContext().isStreamCaching());
System.out.println("----------------------------------------------IN MESSAGE--------------------------------------------------------------");
System.out.println(exchange.getIn().getBody(String.class));
System.out.println("----------------------------------------------OUT MESSAGE--------------------------------------------------------------");
//System.out.println(exchange.getOut().getBody(String.class)); //Activating this line causes empty response on browser
}
});
}
});
context.setTracing(true);
context.setStreamCaching(true);
context.start();
}
}
As you use a custom processor to process the message, you should keep it in mind the in message of the exchange has response message from the google, if you are using exchange.getOut(), camel will create a new empty out message for you and treat it as response message.
Because you don't set the out message body in the processor, it makes sense that you get the empty response in the browser.
Related
Issue:
I have multiple route sending messages to an ActiveMQ queue which later gets processed by a processor to save status information
The same message sending from ProducerTemplate to ActiveMQ queue somehow breakes the same code by not triggering logs on console, and saving status information to a randomly generated file name.
Desired Behavior:
Both sending method gets the messages processed the same way
Code Explanation:
On below codes the Save processor is the one producing the weird behavior where logs dont show up on console and writes to some random file, file name is basically the clientID from ActiveMQ
StartRoute calling the activemq:save works correctly
Code:
public class Save implements Processor {
public void process(Exchange exchange) throws Exception {
try {
Map<String, Object> map = new HashMap<String, Object>() {};
map.put(".....", ".....");
map.put(".....", ".....");
map.put(".....", ".....");
map.put(".....", ".....");
ProducerTemplate template = exchange.getContext().createProducerTemplate();
String response = template.requestBodyAndHeaders("activemq:save", "Batch Job Started", map, String.class);
FluentProducerTemplate FluentTemplate = exchange.getContext().createFluentProducerTemplate();
String result = FluentTemplate
.withHeader(".....", ".....")
.withHeader(".....", ".....")
.withHeader(".....", ".....")
.withHeader(".....", ".....")
.withHeader(".....", ".....")
.to("activemq:save")
.request(String.class);
} catch (Exception e) {
.....
}
}
}
public class StartRoute extends RouteBuilder {
restConfiguration()
.component("servlet")
.enableCORS(true);
rest("/start").id("start")
.post("/{filename}")
.route()
.....
.process(new Save())
.wireTap(startBackgroundProcessRoute)
.to("activemq:save")
.endRest();
}
public class SaveRoute extends RouteBuilder {
from("activemq:save")
.log("started saving")
.process(new FormatText())
.to("file:///status");
}
This question came from my original problem described here:
Camel Multicast Route call order
Solution can be found also there:
Camel Multicast Route call order
However to satisfy this question:
It seems Camel have few bugs regarding producer templates and maybe ActiveMQ, this is my initial conclusion.
The Only way i was able to use ProducerTemplate without issue is to use the send function with Exchanges, send() sends the messages correctly to the ActiveMQ same way as the to() however for whatever reason the content was still not written to the file.
After I dropped the ActiveMQ between the routes, everything started to work consistently. So possible miss configuration on ActiveMQ component or possible another camel framework bug.
If anyone knows the exact answer i would be happy to hear / see the reason for this behavior.
Code example:
public class SaveProcessor implements Processor {
public void process(Exchange exchange) throws Exception {
ProducerTemplate template = exchange.getContext().createProducerTemplate();
template.send(Utilities.trackBatchJobStatus, exchange);
/** NOTE
* DO NOT USE any other functions which are not working with EXCHANGES.
* Functions that uses body, header values and such are bugged
*/
}
}
I am using Camel JMS component for request-reply for communication with MQ. For some of my requests I can receive n messages in reply. How can I aggregate these reply messages?
I thought of using aggregator pattern with aggregation strategy, but can't use it as I am not sure on number of messages which can come in reply.
Can community help me understand what's the right way to do it? I did some google search but couldn't find something useful. Below is my sample route code
from("direct:"+routeName).routeId(routeName)
.setHeader("JMSCorrelationID", constant(UUID.randomUUID().toString()))
.circuitBreaker()
.resilience4jConfiguration()
.minimumNumberOfCalls(3)
.end()
.to(mqComponentBeanName+"://CAMELDEMO?exchangePattern=InOut&requestTimeout=10000&replyTo=CAMELDEMOREPLY")
.log("${body}")
.unmarshal(customerDetailsOutBound)
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
System.out.println(exchange.getIn().getBody().toString());
}
})
.onFallback().process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
System.out.println("Store this message to backup");
}
})
.end();
Looking forward to get some good insights from community. Thank you.
Message flow
your first route sends a message to CAMELDEMO queue and start waiting for a single aggreagted message on a new queue CAMELDEMO_AGGREGATED_REPLY
component that received the message on CAMELDEMO, start sending responses to CAMELDEMOREPLY queue and also indicates how many responses will be sent
Second route below starts listening on CAMELDEMOREPLY, aggregates the message and send the aggregated message to CAMELDEMO_AGGREGATED_REPLY.
Your first route that was waiting for the reply on CAMELDEMO_AGGREGATED_REPLY gets the aggregated reply, receives single message and sends it back
Original route updated to await for reply on CAMELDEMO_AGGREGATED_REPLY
...
.to(mqComponentBeanName+"://CAMELDEMO?exchangePattern=InOut&requestTimeout=10000&
replyTo=CAMELDEMO_AGGREGATED_REPLY")
.log("${body}")
.unmarshal(customerDetailsOutBound)
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
System.out.println(exchange.getIn().getBody().toString());
}
})
....
Second route to aggregate the messages
from(mqComponentBeanName+"://CAMELDEMOREPLY?
exchangePattern=In&requestTimeout=10000)
.aggregate(header("JMSCorrelationID"), new MyAggregationStrategy())
.to(mqComponentBeanName+"://CAMELDEMO_AGGREGATED_REPLY?
exchangePattern=Out&requestTimeout=10000)
public final class MyCompletionStrategy implements AggregationStrategy {
#Override
public Exchange aggregate(Exchange oldExch, Exchange newExchange)
{
...
//Here you check your flag regarding the number of responses
// you were supposed to receive, and if it is met
// complete the aggregation by setting it to true
oldExch.setProperty(Exchange.AGGREGATION_COMPLETE_CURRENT_GROUP, true);
...
return oldExchange;
}
}
I was able to solve this with single route. Solution may not be that neat, but works and fulfils the purpose. I have used loopDoWhile and in the processor inside loopDoWhile I am fetching message from queue using plain java code.
from("direct:"+routeName).routeId(routeName)
.setHeader("JMSCorrelationID", constant(UUID.randomUUID().toString()))
.circuitBreaker()
.resilience4jConfiguration()
.minimumNumberOfCalls(3)
.end()
.to(mqComponentBeanName+"://CAMELDEMO?exchangePattern=InOut&requestTimeout=10000&replyTo=CAMELDEMOREPLY")
.log("${body}")
.unmarshal(customerDetailsOutBound)
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
System.out.println(exchange.getIn().getBody().toString());
int msgCount = getMsgCountfromFirstReposnse;
if (msgCount > 1) {
exchange.getIn().setHeader("COUNTER", 0);
exchange.getIn().setHeader("MSG_COUNT", msgCount-1);
exchange.setProperty("connectionFactory", connectionFactory);
}
}
})
.loopDoWhile(simple("${headers.COUNTER} != ${headers.MSG_COUNT}"))
.process(simpleJMSConsumerProcess)
.end().endCircuitBreaker()
.onFallback().process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
System.out.println("Store this message to backup");
}
})
Code inside processor:
ConnectionFactory connectionFactory = (ConnectionFactory) exchange.getProperty("connectionFactory");
Connection connection = connectionFactory.createConnection();
Session session = connection.createSession(false,
Session.AUTO_ACKNOWLEDGE);
try {
Queue queue = session.createQueue("CAMELDEMOREPLY?consumer.priority=10");
MessageConsumer consumer = session.createConsumer(queue, "JMSCorrelationID = '"+exchange.getIn().getHeader("JMSCorrelationID").toString()+"'");
connection.start();
TextMessage textMsg = (TextMessage) consumer.receive();
System.out.println(textMsg);
System.out.println("Received: " + textMsg.getText());
exchange.getIn().setHeader("COUNTER", ((Integer)exchange.getIn().getHeader("COUNTER"))+1);
if (connection != null) {
connection.close();
}
} finally {
if (session != null) {
session.close();
}
if (connection != null) {
connection.close();
}
}
Well, traditional request-reply has by design just 1 reply message. The thread waiting for the response stops listening as soon as the first reply arrives.
With JMS correlation IDs (no dedicated thread per request) it would theoretically be possible to receive multiple replies for the same request, but I don't know if this really works/is allowed in JMS.
Update based on comments
You write in comments that you are able to receive multiple JMS replies for one request and that you even get the number of answers to expect.
If this all works, you can use the Aggregator EIP in your Camel route to collect all responses before sending a reply to the caller.
The Aggregator is highly configurable. You can decide how to combine the responses and you can also define multiple completion criteria (timeout, number of messages etc).
To retrieve some open data from a remote web server to process, I'm trying out Apache Camel.
The problem is that it seems that the data is never received. I have tried the jetty, ahc and cxf components but can't get it to work. For example like this:
import org.apache.camel.CamelContext;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.impl.DefaultCamelContext;
public class CamelHttpDemo {
public static void main(final String... args) {
final CamelContext context = new DefaultCamelContext();
try {
context.addRoutes(new RouteBuilder() {
#Override
public void configure() throws Exception {
this.from("direct:start")
.to("ahc:http://camel.apache.org/")
.process(exchange -> {
System.out.println(exchange);
});
}
});
context.start();
Thread.sleep(10000);
context.stop();
} catch (final Exception e) {
e.printStackTrace();
}
}
}
No output is written so the line System.out.println(exchange); is never executed and I assume the data is not retrieved.
I'm using the most recent version of Apache Camel, 2.17.1.
You need some message producer in your route to emit Exchange that would trigger the http component. Your route starts with direct:start which cannot emit new Exchanges, it just sits and waits for someone to initiate the process.
The easiest way to make your route work is to replace direct:start with some producer. For instance, replacing it with this timer .from("timer://foo?fixedRate=true&period=10000") will trigger your http-request once every 10 seconds.
If you want to initiate the request manually, you need to create a ProducerTemplate and use it to send a message to direct:start. That would be:
ProducerTemplate template = context.createProducerTemplate();
template.sendMessage("direct:start", "Message body");
I am using spring 4 + hibernate 4 + spring security (RESTFull Webservice API) and angular at front. (Very new to this all).
My failure handler is as follows:
#Component
public class AuthFailure extends SimpleUrlAuthenticationFailureHandler {
#Override
public void onAuthenticationFailure(HttpServletRequest request, HttpServletResponse response, AuthenticationException exception) throws IOException, ServletException {
response.setStatus(HttpServletResponse.SC_UNAUTHORIZED);
}
}
If my request fails I get 401 response which is as expected. Now I want to add AccountLocked and CredentialsExpiredException functionality. I am setting "false" appropriately when I return "User". Again I do get response with status as 401.
return new User(username, password, enabled, accountNonExpired, credentialsNonExpired, accountNonLocked, AuthorityUtils.NO_AUTHORITIES);
As I always get 401 and I do not get any user object inside response(Or may be don't know how to get it), at front end I am unable to find whether it's due to bad credentials or account locked or credentials expired as I want to redirect to another page. I also tried to catch exceptions and tried to forward different statuses but does not seems to hit this code. I always get 401.
#ExceptionHandler(LockedException.class)
#ResponseStatus(value = HttpStatus.BAD_REQUEST)
public ModelAndView handleLockedException(Exception e) {
logger.error("Exception occurred => " + e.getMessage());
return new ErrorResponse(HttpStatus.BAD_REQUEST.value(), e.getMessage(), "Spring security exception").asModelAndView();
}
Please help me - how shall I handle at angular end so that I can redirect to appropriate page?
I think solution was very simple, just did not pay attention to AuthFailure method parameters. Modified my AuthFailuar as follows:
#Component
public class AuthFailure extends SimpleUrlAuthenticationFailureHandler {
#Override
public void onAuthenticationFailure(HttpServletRequest request, HttpServletResponse response, AuthenticationException exception) throws IOException, ServletException {
if (exception instanceof LockedException) {
response.setStatus(HttpServletResponse.SC_PRECONDITION_FAILED);
}else if {
....
} else {
response.setStatus(HttpServletResponse.SC_UNAUTHORIZED);
}
}
}
Now as I have given different status codes I can easily differentiate at the front. Hope for somebody this would be helpful.
I have a Route like the following.
SimpleScheduledRoutePolicy policy = new SimpleScheduledRoutePolicy();
policy.setRouteStartDate(new Date());
policy.setRouteStartRepeatInterval(1000);
from("file:data/in")
.routePolicy(policy)
.to("direct:validateInput")
.to("direct:changeInput")
.to("file:data/out");
So the Route takes an file from the inputfolder every second. After some validation and changing it writes it out to an out folder.
Now I would like to be able to close the actual route at every point. So If some error happens at the direct route validateInput the following two parts should not be excecuted.
I could do this with some doTry() and doCatch() but this will look ugly and hard to read.
Question: is it somehow possible to stop on loop of the main route without stopping the route complete? Like this the actual file won't be printed to the outfolder but the file comming in 5 seconds can be processed in normal way.
Creating a new Process and stopping the main Route in a seperate Thread doesn't work.
The file is still written into the date/out folder
The route is stopped complet and won't take any files anymore.
just have your validateInput step throw an Exception and use an onException() clause to handle the exception, that will short-circuit the flow for just that message and allow for processing future files dropped into the 'data/in' directory normally
I believe you best option is using doCatch and doFinally. I think anything different would be much more difficult to read and very ugly.
http://camel.apache.org/validation.html
This is the most logical approach to solving this problem.
It works good, but 1 Problem is still left. My actual Example looks like the following:
SimpleScheduledRoutePolicy policy = new SimpleScheduledRoutePolicy();
policy.setRouteStartDate(new Date());
policy.setRouteStartRepeatInterval(1000);
//start main route
from("file:test-data/in?delete=true")
.routePolicy(policy)
.onException(Exception.class)
.log(LoggingLevel.ERROR, "${exception}")
.handled(true).useOriginalMessage()
.to("file://test-data/error?autoCreate=true")
.end()
.log(LoggingLevel.DEBUG, "Processing file ${file:name}")
.to("direct:validateInput")
.to("direct:validateContent")
.to("direct:validateOutput")
.to("file:test-data/out");
from("direct:validateInput")
.doTry()
.to("validator:message.xsd")
.doCatch(ValidationException.class)
.process(getErrorProcessor("Could not validate import against the xsd. Message: ${exception}; File: ${body}"));
//...
}
}
private Processor getErrorProcessor(final String message) {
return new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
throw new Exception(message);
}
};
}
If I put an incorrect file into the in folder I get the following error message:
ERROR route1:145 - java.lang.Exception: Could not validate import against the xsd. Message: ${exception}; File: ${body}
As you can see, camel replaced the ${...} attribute one time. But after this I won't replace the new ${...} elements. Any Idear how I can tell camel replace any new {...} parts or how I can replace them before by my self?
found an answer which is good. For all folks who find this question here an "complete" solution.
public void start() {
try {
CamelContext camelContext = new DefaultCamelContext();
camelContext.addRoutes(new RouteBuilder() {
#Override
public void configure() throws Exception {
errorHandler(loggingErrorHandler().level(LoggingLevel.ERROR));
SimpleScheduledRoutePolicy policy = new SimpleScheduledRoutePolicy();
policy.setRouteStartDate(new Date());
policy.setRouteStartRepeatInterval(1000);
//Exception Handling in case the xsd validation fails
onException(SomeSpecialException.class)
.log(LoggingLevel.ERROR, "${exception}")
.handled(true).useOriginalMessage()
.to("file:test-data/error?autoCreate=true")
.end();
//start main route
from("file:test-data/in?delete=true")
.routePolicy(policy)
.log(LoggingLevel.DEBUG, "Processing file ${file:name}")
.to("direct:validateInput")
.to("direct:validateContent")
.to("direct:validateOutput")
.to("file:test-data/out");
//start of separate Routes
from("direct:validateInput")
.doTry()
.to("validator:message.xsd")
.doCatch(ValidationException.class)
.process(getErrorProcess());
//...
}
});
camelContext.start();
} catch (Exception e) {
LOGGER.error("Error while processing camel route: " + e.getMessage());
}
}
/**
* throws an SomeSpecialException in case of calling
*
* #return a Processor which is supposed to be called in case an {#link org.apache.camel.ValidationException} happens
*/
private Processor getErrorProcess() {
return new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
StringBuilder message = new StringBuilder();
String fileContent = exchange.getIn().getBody(String.class);
String originalErrorMessage;
try {
SchemaValidationException schemaValidationException = (SchemaValidationException) exchange.getProperties().get("CamelExceptionCaught");
originalErrorMessage = schemaValidationException.getMessage();
} catch (Exception e) {
originalErrorMessage = "Could not retrieve original error message.";
}
message.append("Could not validate import against the xsd. ")
.append("Original message: ").append(originalErrorMessage).append("; ")
.append("File:").append(fileContent);
throw new SomeSpecialException(message.toString());
}
};
}
Thanks for everyone that helped me.