Best way to unmashal an object to a POJO in Apache Camel - apache-camel

I am trying to retrieve an object out of the process method in Camel response.
However once I got an empty response in the following code:
from("timer://simpleTimer?repeatCount=1").routeId("myroute")
.setHeader("client_id", constant("abc"))
.setHeader("client_secret",constant("def"))
.setHeader(Exchange.HTTP_METHOD, constant("POST"))
.setHeader(Exchange.CONTENT_TYPE, constant("application/json"))
.setHeader(Exchange.HTTP_URI, constant(url))
.marshal().json(JsonLibrary.Gson)
.log("trying to send message")
.to(url)
.convertBodyTo(String.class)
.process(new Processor(){
#Override
public void process(Exchange exchange) throws Exception {
final Message message = exchange.getIn();
int responseCode = message.getHeader(Exchange.HTTP_RESPONSE_CODE, Integer.class);
final String responseBody = message.getBody(String.class);
System.out.println("in final block of process:" +
responseCode + ",Body class name=" + responseBody.getClass()+
"body="+responseBody);
}
);
Here body is not being printed.This is strange as the body is being printed
I need a JSON representation back and also want to store it in a object so that we can return it while returning thi object from this method.
Is there something missing?What should be added to meet the requirement?

Related

Apache Flink side-output not outputing exected results when order of processors swapped in the original stream

I have a small Flink app:
public class App {
public static final OutputTag<String> numberOutputTag = new OutputTag<String>("side-output") {
};
public static void main(String[] args) throws Exception {
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
DataStreamSource<String> text = env.fromElements(
"abc,123"
);
// Router will split input on commas and redirect number strings to the side output
SingleOutputStreamOperator<String> ingestStream = text
.process(new RouterProcessor())
.process(new UppercaseProcessor())
;
DataStream<String> numberStream = ingestStream.getSideOutput(numberOutputTag)
// Prepends a "$" to the values.
.map(new MoneyMapper());
numberStream.print();
ingestStream.print();
env.execute();
}
}
class RouterProcessor extends ProcessFunction<String, String> {
#Override
public void processElement(String value, Context ctx, Collector<String> out) throws Exception {
String[] tokens = value.split(",");
for (String token : tokens) {
if (token.matches("[0-9]+")) {
ctx.output(App.numberOutputTag, token);
} else {
out.collect(token);
}
}
}
}
class MoneyMapper implements MapFunction<String, String> {
#Override
public String map(String t) throws Exception {
return "$" + t;
}
}
class UppercaseProcessor extends ProcessFunction<String, String> {
#Override
public void processElement(String value, Context ctx, Collector<String> out) throws Exception {
out.collect(value.toUpperCase());
}
}
I'd expect it to output something similar to:
18> ABC
18> $123
However, it only outputs:
10> ABC
If I swap the order of the processors to:
.process(new UppercaseProcessor())
.process(new RouterProcessor())
everything works as expected.
I've read the documentation but I don't see anything that would explain why this is as it is. I'm curious if I'm missing something or doing something wrong.
I've included a GitHub jist here for easier viewing with all the supporting files: https://gist.github.com/baelec/95f41d875dda0a2806a0fb9b9313b90e
Here is a repo if you'd prefer to download the sample project: https://github.com/baelec/flink_sample_broken_0
EDIT: I see that StackOverflow asks us to avoid comments like "Thanks!" but I don't have enough rep to visibly upvote the responses so thanks David and Jaya for your help. I had made some incorrect assumptions regarding side outputs. I appreciate the clarification.
The problem is that you are taking the side output from the UppercaseProcessor, which doesn't use a side output.
It's easier to see what's wrong if you look at the job graph, which looks like this:
If you rearrange the code to be like this:
SingleOutputStreamOperator<String> ingestStream = text
.process(new RouterProcessor());
DataStream<String> numberStream = ingestStream.getSideOutput(numberOutputTag)
.map(new MoneyMapper());
numberStream.print();
ingestStream
.process(new UppercaseProcessor())
.print();
then it works as you expected, and the job graph has become this:
numberOutputTag side output emit logic happens inside RouterProcessor. So you need to extract the side output from the SingleOutputStreamOperator returned by the RouterProcessor process function. But in your code, your side output logic extraction happens after the UppercaseProcessor function.
Change something like below,
SingleOutputStreamOperator<String> tempStream = text.process(new RouterProcessor());
SingleOutputStreamOperator<String> ingestStream = tempStream.process(new UppercaseProcessor());
DataStream<String> numberStream = tempStream.getSideOutput(numberOutputTag).map(new MoneyMapper());
numberStream.print();
ingestStream.print();
Note: Check the usage of tempStream variable in the above example.

How to change properties of AMQ endpoint?

When I create ActiveMQ queue through the console, it already has pre-defined property values.
For example:
Max message size 3301921
Min message size 1091
Similarly, when I create a queue in the client code:
...
ConnectionFactory connFactory =
new ActiveMQConnectionFactory(username, password, activeMqBrokerUri);
ActiveMQSession session =
(ActiveMQSession) connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
MessageProducer msgProducer = session.createProducer(session.createQueue(queueName));
...
I have a simple Camel route that gets the serialized files from the queue and sends it to some external resource:
public class MyRouteBuilder extends RouteBuilder {
#Override
public void configure() throws Exception {
from("activemq:alfresco-queue")
.process(new Processor() {
public void process(Exchange exchange) throws Exception {
MultipartEntityBuilder multipartEntityBuilder = MultipartEntityBuilder.create();
multipartEntityBuilder.setMode(HttpMultipartMode.BROWSER_COMPATIBLE);
multipartEntityBuilder.addPart("file", new ByteArrayBody(exchange.getIn().getBody(byte[].class),
exchange.getIn().getHeader("fileName", String.class)));
exchange.getIn().setBody(multipartEntityBuilder.build().getContent());
}
})
.setHeader(Exchange.HTTP_METHOD, constant(org.apache.camel.component.http4.HttpMethods.POST))
.to("http4://vm-alfce5-31.....com:8080/alfresco/s/someco/queuefileuploader?guest=true")
.process(new Processor() {
public void process(Exchange exchange) throws Exception {
System.out.println("The response code is: " +
exchange.getIn().getHeader(Exchange.HTTP_RESPONSE_CODE));
}
});
}
}
My OSGi bundle runs in JBoss FUSE ESB.
When I send a file with size that exceeds the pre-defined maximum size, I get the following error:
Failed delivery for (MessageId: ID:63-DP-TAV-51262-1531978204588-1:1:1:1:1
on ExchangeId: ID-63-DP-TAV-64708-1531973651576-0-3).
Exhausted after delivery attempt:
1 caught: org.apache.http.ContentTooLongException:
Content length is too long: 3301060
The stacktrace is shown below:
org.apache.http.ContentTooLongException: Content length is too long: 3301060
at org.apache.http.entity.mime.MultipartFormEntity.getContent(MultipartFormEntity.java:103)[commons-codec:commons-codec:1.9 org.apache.httpcomponents:fluent-hc:4.5.2 org.apache.httpcomponents:httpclient-cache:4.5.2 org.apache.httpcomponents:httpclient-osgi:4.5.2 org.apache.httpcomponents:httpclient:4.5.2 org.apache.httpcomponents:httpmime:4.5.2]
at org.fusesource.example.MyRouteBuilder$2.process(MyRouteBuilder.java:37)[org.fusesource.example:camel-basic:1.0-SNAPSHOT]
at org.apache.camel.processor.DelegateSyncProcessor.process(DelegateSyncProcessor.java:63)[org.apache.camel:camel-core:2.17.0.redhat-630187 com.googlecode.concurrentlinkedhashmap:concurrentlinkedhashmap-lru:1.4.2]
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)[org.apache.camel:camel-core:2.17.0.redhat-630187 com.googlecode.concurrentlinkedhashmap:concurrentlinkedhashmap-lru:1.4.2]
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:468)[org.apache.camel:camel-core:2.17.0.redhat-630187 com.googlecode.concurrentlinkedhashmap:concurrentlinkedhashmap-lru:1.4.2]
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:196)[org.apache.camel:camel-core:2.17.0.redhat-630187 com.googlecode.concurrentlinkedhashmap:concurrentlinkedhashmap-lru:1.4.2]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:121)[org.apache.camel:camel-core:2.17.0.redhat-630187 com.googlecode.concurrentlinkedhashmap:concurrentlinkedhashmap-lru:1.4.2]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:83)[org.apache.camel:camel-core:2.17.0.redhat-630187 com.googlecode.concurrentlinkedhashmap:concurrentlinkedhashmap-lru:1.4.2]
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:196)[org.apache.camel:camel-core:2.17.0.redhat-630187 com.googlecode.concurrentlinkedhashmap:concurrentlinkedhashmap-lru:1.4.2]
at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:109)[org.apache.camel:camel-core:2.17.0.redhat-630187 com.googlecode.concurrentlinkedhashmap:concurrentlinkedhashmap-lru:1.4.2]
at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:91)[org.apache.camel:camel-core:2.17.0.redhat-630187 com.googlecode.concurrentlinkedhashmap:concurrentlinkedhashmap-lru:1.4.2]
at org.apache.camel.component.jms.EndpointMessageListener.onMessage(EndpointMessageListener.java:112)[org.apache.camel:camel-jms:2.17.0.redhat-630187]
at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:555)[org.apache.servicemix.bundles:org.apache.servicemix.bundles.spring-jms:3.2.16.RELEASE_2]
at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:515)[org.apache.servicemix.bundles:org.apache.servicemix.bundles.spring-jms:3.2.16.RELEASE_2]
at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:485)[org.apache.servicemix.bundles:org.apache.servicemix.bundles.spring-jms:3.2.16.RELEASE_2]
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:325)[org.apache.servicemix.bundles:org.apache.servicemix.bundles.spring-jms:3.2.16.RELEASE_2]
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:263)[org.apache.servicemix.bundles:org.apache.servicemix.bundles.spring-jms:3.2.16.RELEASE_2]
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1103)[org.apache.servicemix.bundles:org.apache.servicemix.bundles.spring-jms:3.2.16.RELEASE_2]
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1095)[org.apache.servicemix.bundles:org.apache.servicemix.bundles.spring-jms:3.2.16.RELEASE_2]
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:992)[org.apache.servicemix.bundles:org.apache.servicemix.bundles.spring-jms:3.2.16.RELEASE_2]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)[:1.8.0_121]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)[:1.8.0_121]
at java.lang.Thread.run(Thread.java:745)[:1.8.0_121]
How to change properties of the ActiveMQ queue?
That error is from the http endpoint, not the ActiveMQ queue.
I don't recommend changing the max message size on the broker for one message type.
Instead, I recommend:
Update the max size on the http service
Add a check for max size of payload off the queue and handle accordingly

Changing apache camel message type to InOut

From what I understand, an InOut message is one where a response can be received from the destination.
However, I have not been able to find any example of how to convert a message to InOut type, and how to access the response from the destination
For example, given a route like:
from("direct:start").to("smtps://smtp.gmail.com:465?username=user#gmail.com&password=usrpw&to=address#gmail.com")
How to convert the message routed to smtps component into InOut type?
Can I expect a response from the smtp component, e.g. indicating that the message was sent successfully?
how to access this response?
By default, each ".to(uri)" are in InOut. The body in the next step will be replaced by the response of the InOut destination. For example, in HTTP component, if you have the following route :
from(direct:start)
.to(http://...)
.log(INFO, "${body}")
The response to the http call will be logged.
If you don't find good informations in the document, I highly recommend you to check the code of the related producer to know what's returned or can be used.
https://github.com/apache/camel
For example, for SMTP, I haven't found the doc saying what's happening to the body, but the code is pretty much clear :
public void process(final Exchange exchange) {
ClassLoader tccl = Thread.currentThread().getContextClassLoader();
try {
ClassLoader applicationClassLoader = getEndpoint().getCamelContext().getApplicationContextClassLoader();
if (applicationClassLoader != null) {
Thread.currentThread().setContextClassLoader(applicationClassLoader);
}
MimeMessage mimeMessage;
final Object body = exchange.getIn().getBody();
if (body instanceof MimeMessage) {
// Body is directly a MimeMessage
mimeMessage = (MimeMessage) body;
} else {
// Create a message with exchange data
mimeMessage = new MimeMessage(sender.getSession());
getEndpoint().getBinding().populateMailMessage(getEndpoint(), mimeMessage, exchange);
}
if (LOG.isDebugEnabled()) {
LOG.debug("Sending MimeMessage: {}", MailUtils.dumpMessage(mimeMessage));
}
sender.send(mimeMessage);
// set the message ID for further processing
exchange.getIn().setHeader(MailConstants.MAIL_MESSAGE_ID, mimeMessage.getMessageID());
} catch (MessagingException e) {
exchange.setException(e);
} catch (IOException e) {
exchange.setException(e);
} finally {
Thread.currentThread().setContextClassLoader(tccl);
}
}
Your exchange will have an header with the MAIL_ID ("CamelMailMessageId"), and in case of any messaging exception, the exception will be propagated. The body seems to be left untouched, even if it's InOut.

How to close a Beanstalkd message before the end of a camel route

I have a beanstalkd queue and have enabled job dependencies via a sub queue that posts when the dependent job is complete, in this example I have distributed parallel processing, so that I can trigger an action once all jobs are complete.
The issue is that I am looping through the response queue picking up messages, but once complete it does not attempt to delete any of the messages until the end, causing it to attempt to close the same message multiple times. For this I am trying to enrich the original message with the responses from the parallel processor, is there any way to make message end after the direct is complete, or via another method of message aggregation?
#Bean
RouteBuilder exampleRoute() {
return new RouteBuilder() {
#Override
public void configure() throws Exception {
/**
* Step 1: Read a message from the listener tube, this will take the form:
* {
* parent_job_id : The parent job id
* split_count: Number of responses to expect
* }
**/
from("beanstalk://localhost/dev_job_listener?onFailure=release&jobDelay=20&jobTimeToRun=10")
.unmarshal().json(JsonLibrary.Jackson, Map.class)
.setProperty("message", simple("${body}"))
// Get required tube jobId to allow deletion of job at end of this route
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
final Deque<Long> stack = new ArrayDeque<>();
stack.push(Long.valueOf(exchange.getIn().getHeader("beanstalk.jobId").toString()));
exchange.setProperty("stack", stack);
Map<String, Object> message = (HashMap) exchange.getProperties().get("message");
exchange.setProperty("parent_job_id", Integer.valueOf(message.get("parent_job_id").toString()));
exchange.setProperty("message_count", Integer.valueOf(message.get("split_count").toString()));
}
})
/**
* Step 2: Listed to the response queue job_{parent_job_id} for the number of completion messages
* from the split_count.
*/
.to("direct:verifySubJobs").end()
/**
* Step 3: Post to the complete queue that all jobs have completed
*/
.to("beanstalk://localhost/next_step?jobTimeToRun=10")
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
exchange.getIn().setHeader("beanstalk.jobId", exchange.getProperty("stack", Deque.class).pop());
}
})
.log("Ending Job : " + simple("${header.beanstalk.jobId}"));
from("direct:verifySubJobs")
.log("-> direct:verifySubJobs")
.setProperty("url", simple("beanstalk://localhost/dev_job_" + simple("${property.parent_job_id}").getText() + "?onFailure=release&jobTimeToRun=10"))
.pollEnrich().simple("${property.url}")
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
Deque<Long> stack = exchange.getProperty("stack", Deque.class);
stack.push(Long.valueOf(exchange.getIn().getHeader("beanstalk.jobId").toString()));
Integer counter = exchange.getProperty("message_count", Integer.class);
exchange.setProperty("message_count", counter);
}
}).end()
.choice().when(simple("${property.message_count} > 0"))
.to("direct:verifySubJobs")
.end()
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
exchange.getIn().setHeader("beanstalk.jobId", exchange.getProperty("stack", Deque.class).pop());
}
}).end()
.log("<- direct:verifySubJobs");
}
};
}

RecipientList with RequestReply Using Apache Camel

I think that I have a gap in my understanding of the RecipientList . My understanding is that RecipientList EIP can be used to represent dynamic destinations. I am attempting to use is with the RequestReply EIP but I am getting some strange results.
The code below is a unit test for RequestReply and aggregation of replies back to the sender. Message arrives at incomingMessages1-update, gets routed to outgoingMessages-[123]-update queues. The results come back on outgoingMessages-[123]-reply queues. The results are aggregated and sent back on incomingMessages1-reply queue.
See below a unit test that works:
public class AggregateStrategyTestOnMultipleReplyQueues extends CamelTestSupport {
#Test
public void testRequestReplyWithRecipientListAndCustomGather()
throws Exception {
int numberOfMessages = 5;
getMockEndpoint("mock:end").setExpectedMessageCount(numberOfMessages);
context.addRoutes(new RouteBuilder() {
public void configure() throws Exception {
from("jms:incomingMessages1-update")
.multicast(new GatherResponses())
.to("jms:outgoingMessages1-update?exchangePattern=InOut&replyTo=queue:outgoingMessages1-reply&preserveMessageQos=true") //1
.to("jms:outgoingMessages2-update?exchangePattern=InOut&replyTo=queue:outgoingMessages2-reply&preserveMessageQos=true") //2
.to("jms:outgoingMessages3-update?exchangePattern=InOut&replyTo=queue:outgoingMessages3-reply&preserveMessageQos=true") //3
.to("mock:end");
//this is what the adapters will be doing
from("jms:outgoingMessages1-update").setBody(constant("Hello World")).to(
"mock:end");
from("jms:outgoingMessages2-update").setBody(constant("Welcome World")).to(
"mock:end");
from("jms:outgoingMessages3-update").setBody(constant("Hi World")).to(
"mock:end");
}
});
String messageSent = "Message sent from template";
Object response = template
.requestBodyAndHeader(
"jms:incomingMessages1-update?exchangePattern=InOut&preserveMessageQos=true",
messageSent, "JMSReplyTo", "incomingMessages1-reply");
assertEquals("Hello World" + " "+ "Welcome World"+ " "+ "Hi World"+ " " + messageSent ,
response);
}
private class GatherResponses implements AggregationStrategy {
public Exchange aggregate(Exchange oldExchange, Exchange newExchange) {
if (oldExchange == null) {
return newExchange;
}
String oldBody = oldExchange.getIn().getBody(String.class);
String newBody = newExchange.getIn().getBody(String.class);
String body = oldBody + " " + newBody;
oldExchange.getIn().setBody(body);
return oldExchange;
}
}
}
I attempted to change the code above (//1, //2 and //3 to a recipient list like below) and it didn't work:
from("jms:incomingMessages1-update")
.recipientList(header("myRecipientList")).aggregationStrategy(new GatherResponses()).parallelProcessing().end()
.to("mock:end");
I loaded the URIs like this:
List<String> recipientList = new ArrayList<String>();
recipientList.add("jms:outgoingMessages1-update?exchangePattern=InOut&replyTo=queue:outgoingMessages1-reply&preserveMessageQos=true");
recipientList.add("jms:outgoingMessages2-update?exchangePattern=InOut&replyTo=queue:outgoingMessages1-reply&preserveMessageQos=true");
recipientList.add("jms:outgoingMessages3-update?exchangePattern=InOut&replyTo=queue:outgoingMessages1-reply&preserveMessageQos=true");
Map<String, Object> headers = new HashMap<String, Object>();
headers.put("JMSReplyTo", "incomingMessages1-reply");
headers.put("myRecipientList", recipientList);
I am getting the original message back and I am not seeing the reply queues created. Can you please point me to what I am missing?
You cannot send a List/Map etc as JMS headers. The JMS spec does not allow that.
See section Message format when sending at
http://camel.apache.org/jms
And also the JMS spec / api / javadoc etc.
You can instead store the values in a String separated by comma. The Camel recipient list will automatic use comma as delimiter, so that should then work out of the box.

Resources