I have a Apache Camel route publishing an AVRO message onto a Apache Kafka topic. I only got this to work when setting the producer property 'serializerClass=kafka.serializer.StringEncoder'. Otherwise I get
java.lang.ClassCastException: java.lang.String cannot be cast to [B
at kafka.serializer.DefaultEncoder.toBytes(Encoder.scala:34) at
kafka.producer.async.DefaultEventHandler$$anonfun$serialize$1.apply(DefaultEventHandler.scala:130)
at
kafka.producer.async.DefaultEventHandler$$anonfun$serialize$1.apply(DefaultEventHandler.scala:125)
at
scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
at
scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:34)
at
kafka.producer.async.DefaultEventHandler.serialize(DefaultEventHandler.scala:125)
at
kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:52)
at kafka.producer.Producer.send(Producer.scala:77) at
kafka.javaapi.producer.Producer.send(Producer.scala:33) at
org.apache.camel.component.kafka.KafkaProducer.process(KafkaProducer.java:84)
On the other end I have a second Apache Camel route supposed to consume from the above topic which failes with
java.io.IOException: Invalid long encoding at
org.apache.avro.io.BinaryDecoder.innerLongDecode(BinaryDecoder.java:217)
at org.apache.avro.io.BinaryDecoder.readLong(BinaryDecoder.java:176)
at
org.apache.avro.io.ResolvingDecoder.readLong(ResolvingDecoder.java:162)
at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:160)
at
org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:193)
at
org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:183)
at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:151)
at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:142)
at
org.apache.camel.dataformat.avro.AvroDataFormat.unmarshal(AvroDataFormat.java:133)
at
org.apache.camel.processor.UnmarshalProcessor.process(UnmarshalProcessor.java:67)
Here is the Apache Camel consumer code I use:
<route id="cassandra.publisher">
<from
uri="{{kafka.base.uri}}&topic=sensordata&groupId=Cassandra_ConsumerGroup&consumerId=CassandraConsumer_Instance_1&clientId=adapter2" />
<unmarshal>
<custom ref="avroSensorData" />
</unmarshal>
In order to solve this problem you have to provide the keyDeserializer and valueDeserializer for camel kafka consumer as follows:
&keyDeserializer=org.apache.kafka.common.serialization.StringDeserializer &valueDeserializer=org.apache.kafka.common.serialization.ByteArrayDeserializer
http://camel.465427.n5.nabble.com/Camel-Kafka-Component-td5749525.html#a5769561
describes that Apache Camel version 2.16.0/2.15.3 will support various datatype and not only String messages.
As promised this has been fixed with Apache Camel 2.15.3 and was fixed with CAMEL-8790 (https://issues.apache.org/jira/browse/CAMEL-8790).
Related
The Documentation of Camel transport for CXF with blueprint
https://camel.apache.org/components/latest/cxf-transport.html
says, the configuration looks like:
client:
<camel:conduit id="*.camel-conduit" camelContextId="camel1" />
server:
<camel:destination id="*.camel-destination" camelContextId="camel1" />
But Blueprint complaints: '*.camel-destination' is not a valid value for 'NCname'. Same for '*.camel-conduit' .
If I leave out the id attribute, the CXF client or CXF server starts up.
But when called, it doesn't find the Camel context.
CXF client:
org.apache.camel.component.cxf.transport.CamelConduit says
IllegalAgumentException "CamelContext must be specified on: conduit:"
CXF server:
org.apache.camel.component.cxf.transport.CamelDestination says
IllegalAgumentException "CamelContext must be specified on:"
Running on Fuse 6.3.
Does anybody know how I must configure CXF transport for Camel in Blueprint?
Try using the "name" attribute instead of the "id" one.
It seems the documentation page(https://camel.apache.org/components/latest/cxf-transport.html) indicates that both the attributes can be used by I think that "name" is the correct one.
On 3 virtual machines running Zookeeper + ActiveMQ.
ActiveMQ Version 5.15.9 with Camel Version 2.19.5
Now, I'm using the embedded Camel EIP Framework in ActiveMQ.
I want to get a specific node from Zookeeper based on an event.
<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring">
<!-- You can use Spring XML syntax to define the routes here using the <route> element -->
<route>
<description>Read zookeeper variant 1</description>
<from uri="activemq:example.zoo1"/>
<pollEnrich>
<constant>zookeeper://localhost:2182/some/node/99</constant>
</pollEnrich>
<to uri="activemq:example.zooRead1"/>
</route>
<route>
<description>Read zookeeper variant 2</description>
<from uri="activemq:example.zoo2"/>
<pollEnrich>
<constant>zookeeper:znode1:2182,znode2:2182,znode3:2182/some/node/99</constant>
</pollEnrich>
<to uri="activemq:example.zooRead2"/>
</route>
</camelContext>
In the Log File following exception occured with variant 1
org.apache.camel.ResolveEndpointFailedException: Failed to resolve endpoint: zookeeper://localhost:2182/some/node/99 due to: No component found with scheme: zookeeper
at org.apache.camel.impl.DefaultCamelContext.getEndpoint(DefaultCamelContext.java:729)[camel-core-2.19.5.jar:2.19.5]
at org.apache.camel.util.CamelContextHelper.getMandatoryEndpoint(CamelContextHelper.java:80)[camel-core-2.19.5.jar:2.19.5]
at org.apache.camel.util.ExchangeHelper.resolveEndpoint(ExchangeHelper.java:92)[camel-core-2.19.5.jar:2.19.5]
at org.apache.camel.processor.PollEnricher.resolveEndpoint(PollEnricher.java:343)[camel-core-2.19.5.jar:2.19.5]
at org.apache.camel.processor.PollEnricher.process(PollEnricher.java:201)[camel-core-2.19.5.jar:2.19.5]
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)[camel-core-2.19.5.jar:2.19.5]
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:541)[camel-core-2.19.5.jar:2.19.5]
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:198)[camel-core-2.19.5.jar:2.19.5]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:120)[camel-core-2.19.5.jar:2.19.5]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:83)[camel-core-2.19.5.jar:2.19.5]
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:198)[camel-core-2.19.5.jar:2.19.5]
at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:97)[camel-core-2.19.5.jar:2.19.5]
at org.apache.camel.component.jms.EndpointMessageListener.onMessage(EndpointMessageListener.java:112)[camel-jms-2.19.5.jar:2.19.5]
at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:719)[spring-jms-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:679)[spring-jms-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:649)[spring-jms-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:317)[spring-jms-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:255)[spring-jms-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1168)[spring-jms-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1160)[spring-jms-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1057)[spring-jms-4.3.18.RELEASE.jar:4.3.18.RELEASE]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)[:1.8.0_222]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)[:1.8.0_222]
at java.lang.Thread.run(Thread.java:748)[:1.8.0_222]
2019-10-01 15:06:05,652 | DEBUG | Message exchange has failed: so breaking out of pipeline for exchange: Exchange[ID-Monkey-34387-1569942241018-0-1] Exception: org.apache.camel.ResolveEndpointFailedException: Failed to resolve endpoint: zookeeper://localhost:2182/some/node/99 due to: No component found with scheme: zookeeper | org.apache.camel.processor.Pipeline | Camel (camel) thread #1 - JmsConsumer[example.zoo1]
2019-10-01 15:06:05,690 | WARN | Execution of JMS message listener failed. Caused by: [org.apache.camel.ResolveEndpointFailedException - Failed to resolve endpoint: zookeeper://localhost:2182/some/node/99 due to: No component found with scheme: zookeeper] | org.apache.camel.component.jms.EndpointMessageListener | Camel (camel) thread #1 - JmsConsumer[example.zoo1]
org.apache.camel.ResolveEndpointFailedException: Failed to resolve endpoint: zookeeper://localhost:2182/some/node/99 due to: No component found with scheme: zookeeper
Camel Zookeeper Component
The solution works as #Bedla wrote.
Copy the Jar in the ACTIVEMQ_HOME/lib folder and it works as expceted.
I use servicemix. I'm trying to send messages from jetty to the queue, but I get the following error:
enter image description here
my blueprint:
<camelContext xmlns="http://camel.apache.org/schema/blueprint">
<route>
<from uri="jetty:http://localhost:8180/test"/>
<to uri="activemq://events_test" />
</route>
</camelContext>
if i use route from jetty to file - everything works as it should
Exchange pattern of your route is InOut, so your jetty consumer waiting response from activemq producer. Response will appear if you have activemq consumer that will process your jetty request. If activemq producer shouldn't make response and you need just save request in queue use "inOnly" instead if "to", like this:
<inOnly uri="activemq://events_test" />
I want to test camel spring integration example from the apache camel site
http://camel.apache.org/springintegration.html but I am getting the exception
org.apache.camel.RuntimeCamelException: org.springframework.integration.MessageDeliveryException: Dispatcher has no subscribers for channel outputchannel.
my short code is given below:
<channel id="inChannel"/>
<channel id="outputChannel"/>
<beans:bean id="greeting" class="com.javarticles.spring.integration.Greeting"/>
<service-activator input-channel="inChannel" ref="greeting" method="sayHello" output-channel="outputChannel"/>
<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="activemq:activemqsource"/>
<to uri="log:input"/>
<to uri="spring-integration:inChannel? inputChannel=outputChannel"/>
<to uri="log:output"/>
</route>
I tried to search related problems but I did not get with camel could any one tell me how to subscribe the outputchannel
According to the Camel documentation mentioned by you, the config should be like this:
<to uri="spring-integration:inputChannel?inOut=true&inputChannel=outputChannel"/>
Your issue is inOut=true:
The exchange pattern that the Spring integration endpoint should use. If inOut=true then a reply channel is expected, either from the Spring Integration Message header or configured on the endpoint.
I am trying to get ServiceMix+Camel talking to OpenMQ JMS Broker. I have a Camel route definition defined in Blueprint DSL which posts messages into an ActiveMQ JMS Queue. This is working fine, but I am now trying to convert this to post messages to an instance of OpenMQ running inside a Glassfish4 container.
I'm running the following configuration:
JDK 1.7.0_60
ServiceMix 5.1.1 (so Camel 2.13.2)
The documentation on this topic is somewhat minimal, but I've gathered various snippets from Google searches here and here and come up with the following (relevant parts only):
....
<!-- Post over JMS into Manager -->
<to uri="openmq:queue:resman"/>
</route>
</camelContext>
<bean id="openmq" class="org.apache.camel.component.jms.JmsComponent">
<property name="connectionFactory">
<bean class="com.sun.messaging.ConnectionFactory">
<property name="imqAddressList" value="localhost:7676"/>
</bean>
</property>
</bean>
I have the following JMS related "features" enabled in Servicemix/OSGI:
[installed ] [3.2.4.RELEASE ] spring-jms
[installed ] [2.13.2 ] camel-jms
and I have the following OSGI bundles installed and active:
[ 89] [Active ] [ ] [ ] [ 50] geronimo-jms_1.1_spec (1.1.1)
[ 126] [Active ] [ ] [ ] [ 50] camel-jms (2.13.2)
The problem I have is a ClassNotFoundException at the point I deploy the Came Route Blueprint into ServiceMix. Here are the relevant snippets from the Stacktrace logged:
org.osgi.service.blueprint.container.ComponentDefinitionException: Unable to instantiate components
Caused by: java.lang.NoClassDefFoundError: javax/jms/JMSContext
Caused by: java.lang.ClassNotFoundException: javax.jms.JMSContext not found by org.apache.geronimo.specs.geronimo-jms_1.1_spec [89]
So on the assumption this is an OSGI bundling issue, I've dug down into the Camel "features" just to confirm the dependencies:
karaf#root> features:info camel-jms
Description of camel-jms 2.13.2 feature
----------------------------------------------------------------
Feature has no configuration
Feature has no configuration files
Feature depends on:
spring [3.2,4)
spring-jms [3.2,4)
camel-core 2.13.2
Feature contains followed bundles:
mvn:org.apache.geronimo.specs/geronimo-jta_1.1_spec/1.1.1 start-level=50
mvn:commons-pool/commons-pool/1.6 start-level=50
mvn:org.apache.geronimo.specs/geronimo-jms_1.1_spec/1.1.1 start-level=50
mvn:org.apache.camel/camel-jms/2.13.2 start-level=50
So it appears that it is the "geronimo-jms_1.1_spec" that is causing the stack trace as it can't find JMSContext. I making a big assumption that as ApacheMQ must be using the same JMS class, so it must be in the environment somewhere. Also, JMSContext seems to be a JMS 2.0 API interface, so is the issue here that the geronimo-jms_1.1_spec bundle is trying to reference a JMS2 API?
I also have the issue of getting the client JARs for OpenMQ installed into the environment. The OpenMQ client JARs shipped with Glassfish4 are plain JARs and not OSGI bundles (see https://java.net/jira/browse/MQ-328), so I plan to either use the "file:wrap" feature to deploy this into ServiceMix, or otherwise unpack and re-pack into the JAR containing my Beans. I don't think this is related to this problem, but I could be wrong.
OK, managed to work this one out finally. The clue was in the reference to javax.jms.JMSContext which is new in the JMS 2.0 API. I was attempting to use the OpenMQ client libraries (imq.jar and jms.jar) taken from the Glassfish4 distribution, which is at JEE7 and hence JMS v1.1. ServiceMix 5.1.1 is still at JMS V1.1, hence the ClassNotFoundException.
Solution:
When connecting to an instance of OpenMQ running under Glassfish4, you need to download and use the OpenMQ client libraries from the previous Glassfish 3.2.2 release. You can obtain this release release from the Glassfish project archives here.
Tests so far show this to be working fine, so if I find anything of relevance going forward I'll come back and update this post.