I have a problem with camel in ServiceMix.
I made webservice through camel-jetty, camel-recipentlist in servicemix.
This package is good performance but resource lock and thread full occurred it. This system process 40 Call per second.
The problem is that pool threads aren't released properly sometimes. After
a few hours following the start of an application I can see using jstack
tool that some threads are stuck in a WAITING state:
configuration is as follows:
- servicemix 5.3.0
- camel 2.13.2
- using component (camel-jetty , camel-recipentlist based Spring DSL)
-SOURCE
<route customId="true" >
<from uri="direct:giop_addr_async">
<recipentList>
<simple>jetty://http://api.host.lm?x=${header.x}&y=${header.y}</simple>
</recipentList>
<bean ref="soapDecode" method="userDecode"/>
<to uri="direct:sendEndPoint">
</route>
<route customId="true>
<from uri="direct:sendEndPoint">
<to uri="jetty://http://resultMap?httpClient.soTimeout=80000"/>
</route>
-------------- LOG
ps -eLf | wc -l --> 32500
"CamelJettyClient(0x3d0b240d)-26916" damen prio=10 tid=0x000000000ff69800 nid =0x10ef wating on condition [0x00002b4b3ba3f0000]
java.lang.Thread.State: TIMED_WAITNG(parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x000000006f13f19b0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LocsSupport.parkNanos(LockSupport,java:226)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoss(QueuedThreadPool.java:526)
at org.eclipse.jetty.tuil.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
at java.lnag.thread.run(Thread.java:745)
The above log is more than 30000 lines.
Can you suggest what else can be checked? Am I missing something? Or may be
this is a bug in Camel?
Having threads in a waiting state is not abnormal, but 30.000 threads is a lot.
You are using a recipientList : For each combination of x and y, you are creating a new jetty producer. each jetty producer has his own thread pool, and camel keeps in a cache the last producers created: A large number of thread pools and thread are created here.
You don't need a recipientList in order to have a URL built dynamically. You can use the header Exchange.HTTP_URI :
for example:
<route customId="true" >
<from uri="direct:giop_addr_async">
<setHeader headerName="CamelHttpUri">
<simple>jetty://http://api.host.lm?x=${header.x}&y=${header.y}</simple>
</setHeader>
<to uri="jetty://http://api.host.lm"/>
<bean ref="soapDecode" method="userDecode"/>
<to uri="direct:sendEndPoint">
</route>
<route customId="true>
<from uri="direct:sendEndPoint">
<to uri="jetty://http://resultMap?httpClient.soTimeout=80000"/>
</route>
Related
I am using Apache Camel's AMQP component to listen to messages from an ActiveMQ Artemis topic.
This application is run on Kubernetes with two replicas.
I've configured a durable subscription with a unique clientId per pod and a common subscription name:
<route autoStartup=true" id="myRoute">
<from id="_amqp_topic" uri="amqp:topic:xxx?connectionFactory=#amqpCF&disableReplyTo=true&transacted=false&subscriptionDurable=true&clientId={{container-id}}&durableSubscriptionName=eventSubscription"/>
<log loggingLevel="INFO" message="Received event: ${body}"/>
...
</route>
The problem is both pods receive the message, when only one of them should. I'm trying to achieve something similar to Kafka's consumer groups, where only one member of the group receives each message.
If you want only one subscriber to receive the message then the subscribers must share the same subscription. Therefore, you need to:
set subscriptionShared=true in your _amqp_topic uri
use the same client ID (i.e. don't use {{container-id}})
For example:
<route autoStartup=true" id="myRoute">
<from id="_amqp_topic" uri="amqp:topic:xxx?connectionFactory=#amqpCF&disableReplyTo=true&transacted=false&subscriptionDurable=true&clientId=myClientID&durableSubscriptionName=eventSubscription&subscriptionShared=true"/>
<log loggingLevel="INFO" message="Received event: ${body}"/>
...
</route>
Another option would simply to use a queue instead of a topic, e.g.:
<route autoStartup=true" id="myRoute">
<from id="_amqp_queue" uri="amqp:queue:xxx?connectionFactory=#amqpCF&disableReplyTo=true&transacted=false"/>
<log loggingLevel="INFO" message="Received event: ${body}"/>
...
</route>
I am trying to use selectors on an amqp with Azure Service Bus consumer. However, for some reason the route is also consuming messages that do not match the selector.
Here's an example:
This route generates messages and append a header:
<route id="MessageGenerator">
<from uri="timer:generator?delay=5000&period=5000"/>
<setHeader headerName="INSTANCE_ID">
<simple>{{env:INSTANCE_ID}}</simple>
</setHeader>
<to uri="amqp:queue:external_queue" />
</route>
While this route should consume only those that contain INSTANCE_ID matching 2 possible values: env:INSTANCE_ID or Any.
<route id="ExternalConsumer">
<from uri="amqp:queue:external_queue?selector=INSTANCE_ID IN ('{{env:INSTANCE_ID}}', 'Any')"/>
<log message="{{env:INSTANCE_ID}} consumed message with Instance ID: ${header.INSTANCE_ID}" logName="AMQP_TEST" loggingLevel="INFO"/>
</route>
But the the log shows that it is consuming any message, regardless of the selector specifying which ones.
Am I missing something?
Thanks!
Issue here was that Azure Service Bus does NOT support selectors on queues. I switched to topics, which already have filters per subscription.
I have created multiple routes(say department, Employee) which takes input from file system folders(say department, Empployee) and process those files.
Now, I want to make them dependent. So, if I upload both emp.csv and dept.csv in those folders then it will process department file first and once complete it will start processing file for employee.
is there any way in camel to achieve this.
I looked at Route startupOrdering and AutoStartup feature, but it will work only for the first time when starting routes. However, I need same behavior for entire route life.
Thanks.
<route id="b" xmlns="http://camel.apache.org/schema/spring">
<from uri="file:/home/dev/code/Integration/RunCamleExample/src/main/resources/csv/Department?repeatCount=1&noop=true&delay=10000"/>
<log message="Department data is : ${body}"/>
</route>
<route id="employee" xmlns="http://camel.apache.org/schema/spring">
<from uri="file:/home/dev/code/Integration/RunCamleExample/src/main/resources/csv/Employee?noop=true&delay=10000"/>
<log message="Employee data is : ${body}"/>
</route>
I suggest to use other logic to handle the task. Two simple ways to go:
Use pollEnrich
Use pollEnrich to collect extra resource (e.g. a file with known name in file system) once at the middle of a route
Flow: Collect department files (From Endpoint) --(for each department file from file system) -> collect single employee file (trigger pollEnrich once with known name) ----> do anything else (if any)
Use ControlBus
Use ControlBus component to control the status of routes (only one of the route in 'start' status)
Flow: Start route A --(when route A complete its goal)-> Suspend route A ---> Start route B --(when route B complete its goal)-> Suspend route B ---> Start route A [loop back to head]
Dependent route execution first can be achieved in Camel using "RouteContext".
Example: If route 'A' is executed before route 'B' then route 'A' should be defined as 'RouteContext' and route be is defined inside "camelContext" like below:
<routeContext id="A" xmlns="http://camel.apache.org/schema/spring">
<route id="A">
<from uri="file:/home/dev/code/Integration/RunCamleExample/src/main/resources/csv/Department?repeatCount=1&noop=true&delay=10000"/>
<log message="Department data is : ${body}"/>
</route>
</routeContext>
Then regular "camelContext" should be defined with reference to this routeContext first.
<camelContext id="test" xmlns="http://camel.apache.org/schema/spring">
<routeContextRef ref="A"/>
<route id="B">
<from uri="file:/home/dev/code/Integration/RunCamleExample/src/main/resources/csv/Employee?noop=true&delay=10000"/>
<log message="Employee data is : ${body}"/>
</route>
</camelContext>
I have been playing around with camel-netty and camel-netty-http components, and trying to figure out what the setting maximumPoolSize does.
However, from what i observe based on this is that, the OrderPool always processes 16 concurrent requests. I am trying to change the maximumPoolSize to a value of 5 like the route as below,
<bean id="nettyBean" class="com.redhat.NettyTestImpl">
<property name="message" value="Netty maximumPoolSize test"/>
</bean>
<camelContext trace="false" xmlns="http://camel.apache.org/schema/blueprint" autoStartup="true">
<route>
<from uri="netty-http:http://localhost:8080/hello?maximumPoolSize=5&sync=true"/>
<log message="Forwarding to Netty component ....."/>
<setBody>
<method ref="nettyBean" method="sayHi"/>
</setBody>
<delay>
<constant>3000</constant>
</delay>
<log message="The body contains : ${body}"/>
</route>
</camelContext>
But it seems that i cannot get the maximumPoolSize to set to a value. So, what am i doing wrong ? How can i get the maximumPoolSize set ?
I check this by load testing with 20 concurrent requests and all are processed.
I just checked the code of camel-netty and camel-netty-http, maximumPoolSize is used for the OrderPool, you can use jconsoler to check the thread pool size.
As the OrderPool is used to handle the processor of camel, netty still has the work thread to handle the connection, which means it can still server 20 concurrent request, this time. But the TPS could be just about 5.
I have Thread pool profile like below:
<threadPoolProfile id="myThrottler"
poolSize="5"
maxPoolSize="20"
maxQueueSize="1000"
rejectedPolicy="CallerRuns"/>
I'm using this thread pool in route:
<route>
<from uri="stream:in"/>
<throttle timePeriodMillis="2000" asyncDelayed="true" executorServiceRef="myThrottler">
<constant>5</constant>
</throttle>
<log message="${threadName}"/>
</route>
I can get thread name by <log message="${threadName}"/>
Can I get the queue size using in thread pool?
You cannot get the thread pool queue size from the <log> which uses the Simple language.
But you can use JMX to get metrics from the thread pool, such as the queue size etc.
http://camel.apache.org/camel-jmx.html