How can delete an entry (a filename that has been processed) from MemoryIdempotent in apache camel
Related
I have the following Apache Camel FTP file download Route:
from(downloadUri)
.routeId(routeId)
.aggregate(new CustomListAggregationStrategy())
.constant(true)
.completionFromBatchConsumer()
.to("direct:" + routeDestinationId);
I add this Route to a context and then request data with a ConsumerTemplate:
List<ResultType> result = consumerTemplate.receiveBody(CAMEL_DIRECT_OBJECT_PREFIX
+ routeId, List.class);
When a connection error occurs (e.g. unknown host, host not reachable), I want to shutdown the Route and throw an exception after the "receiveBody" line where I try to read the downloaded files.
How can I do this?
I tried an onException-handler for the Route where I added a process block to it and called exchange.getContext().stop(); in that processing block. But the application just keeps running.
Stopping a route during routing an existing message is a bit tricky. The reason for that is Camel will Graceful Shutdown the route you are stopping. And if you do that while a message is being routed the Graceful Shutdown will try to wait until that message has been processed.
You can find more info here : https://camel.apache.org/manual/faq/how-can-i-stop-a-route-from-a-route.html
The Documentation of Camel transport for CXF with blueprint
https://camel.apache.org/components/latest/cxf-transport.html
says, the configuration looks like:
client:
<camel:conduit id="*.camel-conduit" camelContextId="camel1" />
server:
<camel:destination id="*.camel-destination" camelContextId="camel1" />
But Blueprint complaints: '*.camel-destination' is not a valid value for 'NCname'. Same for '*.camel-conduit' .
If I leave out the id attribute, the CXF client or CXF server starts up.
But when called, it doesn't find the Camel context.
CXF client:
org.apache.camel.component.cxf.transport.CamelConduit says
IllegalAgumentException "CamelContext must be specified on: conduit:"
CXF server:
org.apache.camel.component.cxf.transport.CamelDestination says
IllegalAgumentException "CamelContext must be specified on:"
Running on Fuse 6.3.
Does anybody know how I must configure CXF transport for Camel in Blueprint?
Try using the "name" attribute instead of the "id" one.
It seems the documentation page(https://camel.apache.org/components/latest/cxf-transport.html) indicates that both the attributes can be used by I think that "name" is the correct one.
I am deploying a jar in karaf .
My jar consists of a camel route
Copyin only the route part :
from("file:/app/billing/billingip/HOBSRating/data/mediation/voice/input?include=USAGE_VOICE.*.txt")
.doTry()
.log("#########The Camel Header before loading into kafka topic ######## :${headers}")
.log("#########The Camel Body before loading into kafka topic ######## :${body}")
.to("kafka:172.20.211.201:9092?topic=VoiceStream&zookeeperHost=172.20.211.201&zookeeperPort=9092&serializerClass=kafka.serializer.StringEncoder")
/* .to("kafka:${kafkaserver}?topic=DataStream&zookeeperHost=${zookeeperHost}&zookeeperPort=${zookeeperport}&serializerClass=kafka.serializer.StringEncoder")*/
.to("file:/app/billing/billingip/HOBSRating/data/mediation/voice/success")
.doCatch(Exception.class)
.log("########The exception message is ####### :${exception.message}")
.log("########The stack trace of the exception is ####### :${exception.stacktrace}")
.to("file:/app/billing/billingip/HOBSRating/data/mediation/voice/error")
.log("############### End of Voice Cdr to Kafka Topic Route ################")
Currently ,I am hard coding the kafka server credentials but i want to make this property file driven.
Read about Camel's property placeholder where you can externalize configuration and refer to them from your Camel routes: http://camel.apache.org/using-propertyplaceholder.html
I have done the below steps for Solr Integration to tomcat on windows machine.Can you please clarify what am I doing wrong here.
1) Download Solr and unzipped Solr 5.2.1 to the below directory C:\downloads\solr-5.2.1\solr-5.2.1.
2)Download Tomcat 7 zipped version and unzipped it to below location C:\downloads\apache-tomcat-7.0.62\apache-tomcat-7.0.62
3)Copy Jar files from C:\downloads\solr-5.2.1\solr-5.2.1\dist\solrj-lib directory to C:\downloads\apache-tomcat-7.0.62\apache-tomcat-7.0.62\lib directory.
4) Create a solr.xml in the C:\downloads\apache-tomcat-7.0.62\apache-tomcat-7.0.62\conf\Catalina\localhost folder.
<?xml version='1.0' encoding='UTF-8'?>
<context docBase="C:/downloads/apache-tomcat-7.0.62/apache-tomcat-7.0.62/webapps/solr.war" debug="0" crossContext="true" >
<environment name="solr" type="java.lang.String" value="/apache-tomcat-7.0.62/webapps/" override="true"></environment>
</context>
5)Copy solr.war file from C:\downloads\solr-5.2.1\solr-5.2.1\server\webapps to
C:\downloads\apache-tomcat-7.0.62\apache-tomcat-7.0.62\webapps folder.
6)Start the tomcat using startup.bat command in bin folder
7)Edit web.xml to
<env-entry>
<env-entry-name>solr/home</env-entry-name>
<env-entry-value>C:/downloads/solr-5.2.1/solr-5.2.1</env-entry-value>
<env-entry-type>java.lang.String</env-entry-type>
</env-entry>
8)Restart the tomcat and hit the url http://localhost:8080/solr I get 404 Not found Error.The error in the console is
SEVERE: Error deploying configuration descriptor C:\downloads\apache-tomcat-7.0.
62\apache-tomcat-7.0.62\conf\Catalina\localhost\solr.xml
java.lang.NullPointerException
at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.ja
va:645)
The Solr wiki states that running 5.x versions on Tomcat is no longer supported:
Internally, Solr is still implemented via Servlet APIs and is powered by Jetty -- but this is simply an implementation detail. Deployment as a "webapp" to other Servlet Containers (or other instances of Jetty) is not supported, and may not work in future 5.x versions of Solr when additional changes are likely to be made to Solr internally to leverage custom networking stack features.
I have a Apache Camel route publishing an AVRO message onto a Apache Kafka topic. I only got this to work when setting the producer property 'serializerClass=kafka.serializer.StringEncoder'. Otherwise I get
java.lang.ClassCastException: java.lang.String cannot be cast to [B
at kafka.serializer.DefaultEncoder.toBytes(Encoder.scala:34) at
kafka.producer.async.DefaultEventHandler$$anonfun$serialize$1.apply(DefaultEventHandler.scala:130)
at
kafka.producer.async.DefaultEventHandler$$anonfun$serialize$1.apply(DefaultEventHandler.scala:125)
at
scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
at
scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:34)
at
kafka.producer.async.DefaultEventHandler.serialize(DefaultEventHandler.scala:125)
at
kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:52)
at kafka.producer.Producer.send(Producer.scala:77) at
kafka.javaapi.producer.Producer.send(Producer.scala:33) at
org.apache.camel.component.kafka.KafkaProducer.process(KafkaProducer.java:84)
On the other end I have a second Apache Camel route supposed to consume from the above topic which failes with
java.io.IOException: Invalid long encoding at
org.apache.avro.io.BinaryDecoder.innerLongDecode(BinaryDecoder.java:217)
at org.apache.avro.io.BinaryDecoder.readLong(BinaryDecoder.java:176)
at
org.apache.avro.io.ResolvingDecoder.readLong(ResolvingDecoder.java:162)
at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:160)
at
org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:193)
at
org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:183)
at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:151)
at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:142)
at
org.apache.camel.dataformat.avro.AvroDataFormat.unmarshal(AvroDataFormat.java:133)
at
org.apache.camel.processor.UnmarshalProcessor.process(UnmarshalProcessor.java:67)
Here is the Apache Camel consumer code I use:
<route id="cassandra.publisher">
<from
uri="{{kafka.base.uri}}&topic=sensordata&groupId=Cassandra_ConsumerGroup&consumerId=CassandraConsumer_Instance_1&clientId=adapter2" />
<unmarshal>
<custom ref="avroSensorData" />
</unmarshal>
In order to solve this problem you have to provide the keyDeserializer and valueDeserializer for camel kafka consumer as follows:
&keyDeserializer=org.apache.kafka.common.serialization.StringDeserializer &valueDeserializer=org.apache.kafka.common.serialization.ByteArrayDeserializer
http://camel.465427.n5.nabble.com/Camel-Kafka-Component-td5749525.html#a5769561
describes that Apache Camel version 2.16.0/2.15.3 will support various datatype and not only String messages.
As promised this has been fixed with Apache Camel 2.15.3 and was fixed with CAMEL-8790 (https://issues.apache.org/jira/browse/CAMEL-8790).