camel-akka and response from actor - apache-camel

Dear Akka/Camel Masters!
I have following route:
(netty4:tcp) -> (akka:actor)
I'm using akka-camel module where:
akka:actor is of type UntypedConsumerActor
netty4:tcp is an endpoint defined in getEndopointUri method of akka:actor
netty4:tcp://localhost:8000?textline=true
When I send bytes to tcp socket I receive exception which tells that socket channel is closed:
Caused by: java.nio.channels.ClosedChannelException: null
at io.netty.channel.AbstractChannel$AbstractUnsafe.write(...)(Unknown Source) [netty-all-4.1.4.Final.jar:4.1.4.Final]
Message History
---------------------------------------------------------------------------------------------------------------------------------------
RouteId ProcessorId Processor Elapsed (ms)
[akka://FileDaemonS] [akka://FileDaemonS] [tcp://localhost:8000 ] [ 60061]
[akka://FileDaemonS] [to1 ] [akka://FileDaemonSystem/user/FileDaemonTcpEndpoint?autoAck=false&replyTimeout=] [ 60037]
java.util.concurrent.TimeoutException: Failed to get response from the actor [ActorEndpointPath(akka://FileDaemonSystem/user/FileDaemonTcpEndpoint)] within timeout [1 minute]. Check replyTimeout and blocking settings [Endpoint[akka://FileDaemonSystem/user/FileDaemonTcpEndpoint?autoAck=false&replyTimeout=60000+milliseconds]]
at akka.camel.internal.component.ActorProducer$$anonfun$1.applyOrElse(ActorComponent.scala:151) ~[akka-camel_2.11-2.4.9.jar:na]
at akka.camel.internal.component.ActorProducer$$anonfun$1.applyOrElse(ActorComponent.scala:148) ~[akka-camel_2.11-2.4.9.jar:na]
at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:36) ~[scala-library-2.11.8.jar:na]
at scala.PartialFunction$AndThen.apply(PartialFunction.scala:186) [scala-library-2.11.8.jar:na]
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32) [scala-library-2.11.8.jar:na]
What am I doing wrong?

I found a solution. Setting netty endpoint as a one-way solves the problem.
netty4:tcp://localhost:8000?textline=true&sync=false

Related

PAHO MQTT 5 throwing exception when using same clientId in routes

When using paho-mqtt5:test more than once with same clientId then it throw exception Client not connected but if i will use different clientId for each to and from then it will work fine
2021-10-05 19:25:28,650 ERROR [org.apa.cam.pro.err.DefaultErrorHandler] (Camel (camel-1) thread #0 - timer://test) Failed delivery for (MessageId: 871E4623819E4FB-000000000000001B on ExchangeId: 871E4623819E4FB-000000000000001B). Exhausted after delivery attempt: 1 caught: Client is not connected (32104)
Message History (complete message history is disabled)
---------------------------------------------------------------------------------------------------------------------------------------
RouteId ProcessorId Processor Elapsed (ms)
[route1 ] [route1 ] [from[timer://test?period=1000] ] [ 0]
...
[route1 ] [to1 ] [paho:test ] [ 0]
Stacktrace
---------------------------------------------------------------------------------------------------------------------------------------
: Client is not connected (32104)
at org.eclipse.paho.mqttv5.client.internal.ExceptionHelper.createMqttException(ExceptionHelper.java:32)
at org.eclipse.paho.mqttv5.client.internal.ClientComms.sendNoWait(ClientComms.java:231)
at org.eclipse.paho.mqttv5.client.MqttAsyncClient.publish(MqttAsyncClient.java:1530)
at org.eclipse.paho.mqttv5.client.MqttClient.publish(MqttClient.java:564)
at org.apache.camel.component.paho.mqtt5.PahoMqtt5Producer.process(PahoMqtt5Producer.java:55)
at org.apache.camel.support.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:66)
at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:172)
at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$SimpleTask.run(RedeliveryErrorHandler.java:463)
at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:179)
at org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:64)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:184)
at org.apache.camel.impl.engine.CamelInternalProcessor.process(CamelInternalProcessor.java:398)
at org.apache.camel.component.timer.TimerConsumer.sendTimerExchange(TimerConsumer.java:210)
at org.apache.camel.component.timer.TimerConsumer$1.run(TimerConsumer.java:76)
at java.base/java.util.TimerThread.mainLoop(Timer.java:556)
at java.base/java.util.TimerThread.run(Timer.java:506)
Here is my code which is throwing exception
#ApplicationScoped
class TestRouter : RouteBuilder() {
override fun configure() {
val mqtt5Component = PahoMqtt5Component()
mqtt5Component.configuration = PahoMqtt5Configuration().apply {
brokerUrl = "tcp://192.168.99.101:1883"
clientId = "paho123"
isCleanStart = true
}
context.addComponent("paho-mqtt5", mqtt5Component)
from("timer:test?period=1000").setBody(constant("Testing timer2")).to("paho-mqtt5:test")
from("paho-mqtt5:test").process { e ->
val body = (e.`in`?.body as? ByteArray)?.let { String(it) }
println("test body 1 => $body")
}
}
}
#William, this is expected behavior
The message broker uses the client id to differentiate between clients so it can perform housekeeping for a client connection that is no longer used
In addition, a client may have a "Last Will and Testament" that the broker keeps track of
It is acceptable to append a random number to the end of your current 'clientId' since it is likely no one but you will care about this
If you have access to the individuals login, you could use that as well but you would still want to make each session unique in case they run multiple sessions
Maybe I don't understand what your problem is
Each client must have a unique Id
What are you observing that makes you think that it is creating multiple connections for a single client?
Is there a chance you are opening multiple windows and each is generating a different clientId?
This is a good way to diagnose issues by monitoring what the server is seeing
My paho-mqtt client (Javascript) is connecting as "webclient" and I append a randome number (webclient173) to identify this client
To troubleshoot, I would suggest you close all connections on the client and monitor the log of the MQTT process
When the monitor is in place, open a connection from a client that currently has no connections
This is an example connection to my Mosquitto log file
$ tail -f /var/log/mosquitto/mosquitto.log
1635169943: No will message specified.
1635169943: Sending CONNACK to webclient173 (0, 0)
1635169943: Received SUBSCRIBE from webclient173
1635169943: testtopic (QoS 0)
1635169943: Sending SUBACK to webclient173
1635170003: Received PINGREQ from webclient173
1635170003: Sending PINGRESP to webclient173
1635170003: Received PINGREQ from webclient173
1635170003: Sending PINGRESP to webclient173
What does your log show?

Apache Atlas server not running,for check the reason 'zookeeper always send error message'

I am new for Atlas and had try to install it for 3 days.It's to hard for to install. So can anyone help me solve this question,thanks all.
WARN - 2021-02-02 07:49:29.543; [ ] org.apache.zookeeper.ClientCnxn$SendThread; Session 0x200ad9fa8f00005 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:714)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
ERROR - 2021-02-02 07:54:26.835; [ ] org.apache.zookeeper.ClientCnxn$EventThread; Error while calling watcher
java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1#5527a145 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor#10ea4804[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 21]
at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:135)
at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112)
at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:261)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)

Handle large messages with Apache Camel and AMQ Artemis

When I receive a large message (100KiB+) in a AMQ Artemis Queue and try to route this message to another AMQ and this message have the property _AMQ_LARGE_SIZE I got the follow error:
14:38:56.250 [Camel (CamelTestRoute) thread #1 - JmsConsumer[QUEUE.TEST]] WARN o.a.c.c.jms.EndpointMessageListener - Execution of JMS message listener failed. Caused by: [org.apache.camel.RuntimeCamelException - javax.jms.JMSRuntimeException: Invalid address QUEUE.TEST]
org.apache.camel.RuntimeCamelException: javax.jms.JMSRuntimeException: Invalid address QUEUE.TEST
I know if I set the property minLargeMessageSize in the Connection Factory that post the message in AMQ, this problem does not happens.
The problem is, I don't have control of the codes that create the Connection Factories, and some times they don't set the Large Message Size property.
Is there a way that can I handle this in Camel with my Connection Factory?
*EDIT
16:33:03.836 [Camel (CamelTestRoute) thread #1 - JmsConsumer[QUEUE.TEST]] WARN o.a.c.c.jms.EndpointMessageListener - Execution of JMS message listener failed. Caused by: [org.apache.camel.RuntimeCamelException - javax.jms.JMSRuntimeException: Invalid address QUEUE.TEST]
org.apache.camel.RuntimeCamelException: javax.jms.JMSRuntimeException: Invalid address QUEUE.TEST
at org.apache.camel.util.ObjectHelper.wrapRuntimeCamelException(ObjectHelper.java:1830)
at org.apache.camel.component.jms.EndpointMessageListener$EndpointMessageListenerAsyncCallback.done(EndpointMessageListener.java:196)
at org.apache.camel.component.jms.EndpointMessageListener.onMessage(EndpointMessageListener.java:117)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:719)
at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:679)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:649)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:317)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:255)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1168)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1160)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1057)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: javax.jms.JMSRuntimeException: Invalid address QUEUE.TEST
at org.apache.activemq.artemis.jms.client.ActiveMQDestination.fromAddress(ActiveMQDestination.java:119)
at org.apache.activemq.artemis.jms.client.ActiveMQMessage.getJMSDestination(ActiveMQMessage.java:386)
at org.apache.camel.component.jms.JmsBinding.extractHeadersFromJms(JmsBinding.java:187)
at org.apache.camel.component.jms.JmsMessage.populateInitialHeaders(JmsMessage.java:229)
at org.apache.camel.impl.DefaultMessage.createHeaders(DefaultMessage.java:257)
at org.apache.camel.component.jms.JmsMessage.ensureInitialHeaders(JmsMessage.java:214)
at org.apache.camel.component.jms.JmsMessage.getHeader(JmsMessage.java:164)
at org.apache.camel.impl.DefaultMessage.getHeader(DefaultMessage.java:93)
at org.apache.camel.impl.DefaultUnitOfWork.<init>(DefaultUnitOfWork.java:115)
at org.apache.camel.impl.MDCUnitOfWork.<init>(MDCUnitOfWork.java:54)
at org.apache.camel.impl.DefaultUnitOfWorkFactory.createUnitOfWork(DefaultUnitOfWorkFactory.java:32)
at org.apache.camel.processor.CamelInternalProcessor$UnitOfWorkProcessorAdvice.createUnitOfWork(CamelInternalProcessor.java:695)
at org.apache.camel.processor.CamelInternalProcessor$UnitOfWorkProcessorAdvice.before(CamelInternalProcessor.java:663)
at org.apache.camel.processor.CamelInternalProcessor$UnitOfWorkProcessorAdvice.before(CamelInternalProcessor.java:634)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:149)
at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:97)
at org.apache.camel.component.jms.EndpointMessageListener.onMessage(EndpointMessageListener.java:113)
... 11 common frames omitted
If you're using an Artemis 1.x client against an Artemis 2.x broker then you need to configure the acceptor that the client is connecting to with the appropriate anycastPrefix and multicastPrefix, e.g:
<acceptor name="artemis">tcp://0.0.0.0:61616?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=CORE,AMQP,STOMP,HORNETQ,MQTT,OPENWIRE;anycastPrefix=jms.queue.;multicastPrefix=jms.topic.;useEpoll=true;amqpCredits=1000;amqpLowCredits=300</acceptor>

Apache camel how to implement an optional consumer for a wire tap

I set up some routes (Camel 2.22.1) that uses wire tap to log some stuff into a Mongo db.
from(DIRECT_NEXT).process(sendFile)
.wireTap( "direct:count-fetch?failIfNoConsumers=false" )
as you see i am using failIfNoConsumers=false.
from(COUNT_FETCH)
.routeId( MONGO_COUNT_FETCH_ROUTEID )
.autoStartup( false )
.process(countFetchProcessor)
.to(persistenceEndpoints.updateImage())
.log(LoggingLevel.DEBUG, "Counted fetch.");
The mongo DB is an optional component, the whole application will run without it.
I am using Mongo'S ServerMonitorListener to check if Mongo is available. I suspend or resume the rout using Camel's ControlBus accordingly.
All is running fine!
My Problem is that Camel tries to send the exchanges to the not running routes for 30s:
...
[DEBUG] 2019-01-03 14:02:45.848 [Camel (camel-1) thread #23 - WireTap] DirectBlockingProducer - Waited 20025 for consumer to be ready
...
Why the producer blocks? The default value for "block" should be false?!
And after it we see of course an exception:
No consumers available on endpoint: direct://count-fetch?failIfNoConsumers=false
What is the best approach to let camel discard the exchange immediately (how to set the time out?) and don't throw any exception (because it is normal application behavior, exception will only slow down)?
UPDATE:
here is the complete exception:
[ERROR] 2019-01-07 10:21:22.702 [Camel (camel-1) thread #4 - WireTap] DefaultErrorHandler - Failed delivery for (MessageId: ID-moritz-1546852848013-0-3 on ExchangeId: ID-moritz-1546852848013-0-2). Exhausted after delivery attempt: 1 caught: org.apache.camel.component.direct.DirectConsumerNotAvailableException: No consumers available on endpoint: direct://update-all?failIfNoConsumers=false. Exchange[ID-moritz-1546852848013-0-2]
Message History
---------------------------------------------------------------------------------------------------------------------------------------
RouteId ProcessorId Processor Elapsed (ms)
[route4 ] [route4 ] [timer://updateAll ] [ 30065]
[route4 ] [log1 ] [log ] [ 1]
[route4 ] [to3 ] [direct:updateAll ] [ 19]
[route5 ] [process2 ] [Processor#0x4e92466a ] [ 9]
[route5 ] [process3 ] [Processor#0x1b29d52b ] [ 7]
[route5 ] [wireTap1 ] [wireTap[direct:update-all?failIfNoConsumers=false] ] [ 1]
Stacktrace
---------------------------------------------------------------------------------------------------------------------------------------
org.apache.camel.component.direct.DirectConsumerNotAvailableException: No consumers available on endpoint: direct://update-all?failIfNoConsumers=false. Exchange[ID-moritz-1546852848013-0-2]
at org.apache.camel.component.direct.DirectBlockingProducer.getConsumer(DirectBlockingProducer.java:67) ~[camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.component.direct.DirectBlockingProducer.process(DirectBlockingProducer.java:53) ~[camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.processor.SendDynamicProcessor$1.doInAsyncProducer(SendDynamicProcessor.java:178) ~[camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:445) ~[camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.processor.SendDynamicProcessor.process(SendDynamicProcessor.java:160) ~[camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548) [camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201) [camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:97) [camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.processor.WireTapProcessor$1.call(WireTapProcessor.java:160) [camel-core-2.22.1.jar:2.22.1]
at org.apache.camel.processor.WireTapProcessor$1.call(WireTapProcessor.java:155) [camel-core-2.22.1.jar:2.22.1]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
Make sure to check the document for the version of Camel you use, which is 2.22.x
There you can see the block is default enabled: https://github.com/apache/camel/blob/camel-2.22.x/camel-core/src/main/docs/direct-component.adoc

Camel HL7 - ClosedChannelException while sending ACK back to the client

I'm building a HL7 listener using netty4 and processing HL7 messages. Once succesfully processed an ACK is sent back.
from("hl7NettyListener")
.routeId("route_hl7listener")
.startupOrder(997)
.unmarshal()
.hl7(false)
.to("direct:a");
from("direct:a")
.doTry()
.to("bean:processHL7?method=process")
.doCatch(HL7Exception.class)
.to("direct:ErrorACK")
//.transform(ack())
.stop()
.end()
.transform(ack())
.wireTap("direct:b");
This is working fine in my local eclipse. I fire a HL7 message and I get a ACk back.
But i package this application into a jar and put it on my server and then try doing a
cat example.hl7 | netcat localhost 4444 (to fire a HL7 message to port 4444 on linux env)
I dont get an ACK back. I get a closedconnection exception.
DEBUG NettyConsumer - Channel: [id: 0xdf13b06b, L:0.0.0.0/0.0.0.0:4444] writing body: MSH|^~\&|Karisma||Kestral|Kestral|20180309144109.827+1300||ACK^R01|701||2.3.1
2018-03-09 14:41:09,838 [ad #3 - WireTap] DEBUG WireTapProcessor - >>>> (wiretap) direct:b Exchange[]
2018-03-09 14:41:09,839 [ServerTCPWorker] DEBUG NettyConsumer - Caused by: [org.apache.camel.CamelExchangeException - Cannot write response to null. Exchange[ID-annan06-56620-1520559639101-0-2]. Caused by: [java.nio.channels.ClosedChannelException - null]]
org.apache.camel.CamelExchangeException: Cannot write response to null. Exchange[ID-annan06-56620-1520559639101-0-2]. Caused by: [java.nio.channels.ClosedChannelException - null]
at org.apache.camel.component.netty4.handlers.ServerResponseFutureListener.operationComplete(ServerResponseFutureListener.java:54)
at org.apache.camel.component.netty4.handlers.ServerResponseFutureListener.operationComplete(ServerResponseFutureListener.java:36)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:514)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:488)
at io.netty.util.concurrent.DefaultPromise.access$000(DefaultPromise.java:34)
at io.netty.util.concurrent.DefaultPromise$1.run(DefaultPromise.java:438)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:418)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:440)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.nio.channels.ClosedChannelException
at io.netty.channel.AbstractChannel$AbstractUnsafe.write(...)(Unknown Source)
That worked. It was failing because netcat was immediately closing the connection. Had put a ”netcat -i 5 localhost” for netcat to wait for 5 secs and successfully received the ACK back.

Resources