I was trying integrate Apache camel with Kafka and wrote a sample program to read a file and write to Kafka Topic. But I am getting below error while doing so. I could be able to do it the reverse way read from Kafka topic and write to a file.
Stacktrace
org.apache.kafka.common.errors.SerializationException: Can't convert value of class org.apache.camel.component.file.GenericFile to class org.apache.kafka.common.serialization.StringSerializer specified in value.serializer
[#0 - file://C:%5Cshare%5Cinput] KafkaProducer WARN No message key or partition key set
[#0 - file://C:%5Cshare%5Cinput] GenericFileOnCompletion WARN Rollback file strategy: org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy#7127845b for file: GenericFile[C:\share\input\file.txt]
[#0 - file://C:%5Cshare%5Cinput] DefaultErrorHandler ERROR Failed delivery for (MessageId: ID-L8-CWBL462-49953-1480494317350-0-21 on ExchangeId: ID-L8-CWBL462-49953-1480494317350-0-22). Exhausted after delivery attempt: 1 caught: org.apache.kafka.common.errors.SerializationException: Can't convert value of class org.apache.camel.component.file.GenericFile to class org.apache.kafka.common.serialization.StringSerializer specified in value.serializer
Code
#ContextName("myCdiCamelContext")
public class MyRoutes extends RouteBuilder {
#Inject
#Uri("file:C:\\share\\input?fileName=file.txt&noop=true")
private Endpoint inputEndpoint;
#Inject
#Uri("kafka:localhost:9092?topic=test&groupId=testing&autoOffsetReset=earliest&consumersCount=1")
private Endpoint resultEndpoint;
#Override
public void configure() throws Exception {
from(inputEndpoint)
.to(resultEndpoint);
}
}
After adding a new processor it worked for me
public void configure() throws Exception {
from(inputEndpoint).process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
exchange.getIn().setBody(exchange.getIn().getBody(),String.class);
exchange.getIn().setHeader(KafkaConstants.PARTITION_KEY, 0);
exchange.getIn().setHeader(KafkaConstants.KEY, "1");
}
})
.to(resultEndpoint);
}
How can we stop looping on camel splitter on specific exception?
The "stopOnException()" is stopping the looping for every exception, but instead I want to stop looping only on some specific exceptions. And if the exception is "HttpOperationFailedException", I want to stop looping based on response code.
For example if response code is "500" stop execution and if response code is 404 continue execution.
Is it possible?
Original Question
from("timer:categoryRouter?delay=0")
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
exchange.getIn().setBody("A,F,B,D,C");
}
})
// tell Splitter to use the aggregation strategy which handles and ignores exceptions
.split(body(), new MyIgnoreFailureAggregationStrategy())
.stopOnException()
// log each splitted message
.log("Split line ${body}")
// and have them translated into a quote
.bean(WordTranslateBean.class)
// and send it to a mock
.to("mock:split")
.end()
// log the outgoing aggregated message
.log("Aggregated ${body}")
// and send it to a mock as well
.to("mock:result");
Bean which throws exception:
public class WordTranslateBean {
private Map<String, String> words = new HashMap<String, String>();
public WordTranslateBean() {
words.put("A", "Camel rocks");
words.put("B", "Hi mom");
words.put("C", "Yes it works");
}
public String translate(String key) throws HttpOperationFailedException {
if (!words.containsKey(key)) {
HttpOperationFailedException httpOperationFailedException = null;
if(key.equals("F")) {
httpOperationFailedException = new HttpOperationFailedException("uri",500,"Internal Server Error","location",null,"Key not a known word " + key);
}
else {
httpOperationFailedException = new HttpOperationFailedException("uri",404,"Resource Not Found","location",null,"Operation not supported on word " + key);
}
throw httpOperationFailedException;
}
return words.get(key);
}
}
Working Solution:
from("timer:categoryRouter?delay=0")
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
exchange.getIn().setBody("A,F,B,D,C");
}
})
// tell Splitter to use the aggregation strategy which handles and ignores exceptions
.split(body(), new MyIgnoreFailureAggregationStrategy())
.stopOnException()
// log each splitted message
.log("Split line ${body}")
// and have them translated into a quote
.doTry()
.bean(WordTranslateBean.class)
// and send it to a mock
.to("mock:split")
.doCatch(HttpOperationFailedException.class)
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
HttpOperationFailedException e = (HttpOperationFailedException) exchange.getProperty(Exchange.EXCEPTION_CAUGHT);
if(e.getStatusCode()!=404){
throw e;
}
}
})
.end()
.end()
// log the outgoing aggregated message
.log("Aggregated ${body}")
// and send it to a mock as well
.to("mock:result");
Why don't you throw a custom exception based on a response code ? That's one option . Basically you can catch the original http exception , check the response code , throw your custom exception. Can you post your route ? It's easy to implement this way, just want to see how you have organised your routes .
Basically we still need to use "stopOnException" to stop the splitter when exception occurred. But to control on which exception the splitter should break, you can use "doTry..doCatch" block and in the respective catch block throw back the exception again.
from("timer:categoryRouter?delay=0")
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
exchange.getIn().setBody("A,F,B,D,C");
}
})
// tell Splitter to use the aggregation strategy which handles and ignores exceptions
.split(body(), new MyIgnoreFailureAggregationStrategy())
// log each splitted message
.log("Split line ${body}")
// and have them translated into a quote
.doTry()
.bean(WordTranslateBean.class)
// and send it to a mock
.to("mock:split")
.doCatch(HttpOperationFailedException.class)
.log("Ignore Exception")
.doCatch(IOException.class)
.throwException(new IOException())
.doCatch(UnsupportedOperationException.class)
.log("Ignore Exception")
.end()
.end()
// log the outgoing aggregated message
.log("Aggregated ${body}")
// and send it to a mock as well
.to("mock:result");
If the exception is related to http and want to inspect the response code to act accordingly then you can my question which has the working solution.
You can catch the exceptions and decide what to do with them. Inside of your splitter:
<doTry>
<!-- Your Splitter logic here -->
<doCatch>
<exception>java.lang.IllegalStateException</exception>
<log message="This exception happened here, but not a problem.."/>
</doCatch>
<doCatch>
<exception>java.io.IOException</exception>
<log message="Big problem here. STOPPING.."/>
<stop/>
</doCatch>
<doFinally>
<to uri="mock:finally"/>
</doFinally>
</doTry>
I'm using deadLetterChannel to take care of exceptions and send them to the error queue.
errorHandler(deadLetterChannel(QUEUE_ERROR).maximumRedeliveries(3).redeliveryDelay(2000));
Is it possible to enrich the message with additional message headers? Or do i have to use onException for it?
You can use onRedelivery and with a processor to add headers before redelivering
errorHandler(deadLetterChannel(QUEUE_ERROR).maximumRedeliveries(3).redeliveryDelay(2000).onRedelivery(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
//add headers here
}
}));
Example route:
onException(Exception.class)
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
System.out.println("it works");
}
})
.handled(true);
from("jetty://http://0.0.0.0:8888/test")
.idempotentConsumer(header("myid"), MemoryIdempotentRepository.memoryIdempotentRepository(1000000))
.skipDuplicate(false)
.filter(property(Exchange.DUPLICATE_MESSAGE).isEqualTo(true))
.throwException(new DuplicateRequestException())
.end();
Sending a request to the listener URL without myid parameter throws org.apache.camel.processor.idempotent.NoMessageIdException: No message ID could be found using expression: header(myid) on message exchange: Exchange[Message: [Body is instance of org.apache.camel.StreamCache]]
without ever passing from onException.
Yes this is in fact a bug in Apache Camel. I have logged a ticket to get this fixed in the next releases.
https://issues.apache.org/jira/browse/CAMEL-7990
My log lines throw nullpointer exception when I run unit tests. I get no errors when I run it on local server or upload to appengine. Have I forgotten to include a test library somewhere?
java.lang.NullPointerException
at javax.servlet.GenericServlet.getServletContext(GenericServlet.java:160)
at javax.servlet.GenericServlet.log(GenericServlet.java:254)
at se.stuff.servlet.MyServlet.doGet(MyServlet.java:14)
at se.stuff.MyServletTest.test(MyServletTest.java:14)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
...
My servlet:
public class MyServlet extends HttpServlet {
#Override
public void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException {
// Do stuff...
log("log stuff");
}
}
My test:
public class MyServletTest {
#Test
public void test() throws IOException {
MyServlet s = new MyServlet();
s.doGet(null, null);
}
}
The nulls in the test call s.doGet(null, null) cause the NullPointerException. The servlet is probably fine, but the test does not make sense. I suggest scrapping that test and first adding some content into doGet, then use QUnit to test your servlet from the outside.