I am trying to fetch the data from the file(size of 70 KB) and aggregate the data at the same time but it is getting delayed by 2 seconds for all the requests to finish the process. Below is the code.
Camel context file:
<route>
<from uri="http://localhost:6612/Services/EquationAdapter"/>
<log message="Request xml"/>
<enrich strategyRef="equationAdapterEnricher">
<constant>direct:loadEquationAdapterRulesXML</constant>
</enrich>
<convertBodyTo type="java.lang.String"/>
<log message="After enrich"/>
</route>
<route>
<from uri="direct:loadEquationAdapterRulesXML"/>
<to uri="language:simple:file:/tmp/test.xml"/>
</route>
Java code:
public class EquationAdapterConfigRulesEnricher implements AggregationStrategy {
Log logger = LogFactory.getLog(EquationAdapterConfigRulesEnricher.class);
public void enRichEquationConfigRules(Exchange exchg) {
}
#Override
public Exchange aggregate(Exchange oldExchg, Exchange newExchg) {
logger.info("Entering Equation Adapter Config Rules Enricher");
-----
written logic here
---
return oldExchg;
}
}
Below are the logs for the same.
12 Feb 2023 13:32:08,085 [ qtp797795250-29] -genericoperation-cxf-withxslt INFO ACTR -DPS-DPS-24113526-DI01 -4010-220724113526596 Request xml
12 Feb 2023 13:32:10,156 [ qtp797795250-29] tionAdapterConfigRulesEnricher INFO Entering Equation Adapter Config Rules Enricher
As we can see in the logs, we received the request at 13:32:08 and log in Java code printed after 2 seconds. It's the same issue for all the requests.
Can anyone please help here to rectify the issue and eliminate that 2 sec delay and also let me know how can we store the file in cache, so that will not poll the file everytime we get the request.
Related
I am developing a sample route
FROM: SOURCE ENDPOINT URI
TO: TRANS ENDPOINT URI // Error or Exception occurred at this TRANS endpoint
TO: TARGET ENDPOINT URI
Now I want to catch the Error Occured endpoint and pass it to my processor.
Could anyone please help me with this?
<route>
<from uri="file:C:/MINTS/Source/"/>
<to uri="file:C:/MINTS/TRANS/"/> <!-- EXCPECTION OCCURED -->
<to uri="file:C:/MINTS/TARGET/"/>
<onException>
<exception>java.lang.Exception</exception>
<handled>
<constant>true</constant>
</handled>
<!-- NEED TO CATCH FAILURE ENDPOINT URI AND PASS TO MY PROCESSOR BELOW-->
<process ref="MyExceptionProcessor" />
</onException>
</route>
You can use exchange properties:
CamelFailureEndpoint
example xml-dsl: <exchangeProperty>CamelFailureEndpoint</exchangeProperty>
example java: exchange.getProperty(Exchange.FAILURE_ENDPOINT, String.class);
CamelToEndpoint
example xml-dsl: <exchangeProperty>CamelToEndpoint</exchangeProperty>
example java: exchange.getProperty(Exchange.TO_ENDPOINT, String.class)
The first one should print uri for the consumer endpoint (from) that failed and the second one should show the previously called producer (to) endpoint.
One handy way to debug contexts of an failed exchange is to use:
<to uri="log:loggerName?showAll=true" />
This will log all exchange properties, headers, body and exception which can help to understand what information is available within exchange. Be careful where you use it as it might also log secrets and passwords if they're within the exchange so better use it only locally during development.
For more information you'd probably need to access CamelMessageHistory exchange property through processor, bean or something.
My application consists of many OSGi bundles running inside JBoss Fuse 6.2.1. Each bundle has a Camel route consuming from an ActiveMQ endpoint. Data is exchanged using VirtualTopics.
ProducerBundle publishes to topic VirtualTopic.MyTopic
ConsumerBundle A consumes from queue Consumer.A.VirtualTopic.MyTopic
ConsumerBundle B consumes from queue Consumer.B.VirtualTopic.MyTopic
ConsumerBundle C consumes from queue Consumer.C.VirtualTopic.MyTopic
At a certain moment in time Consumer C is closed, its bundle uninstalled and will never come back. Howewer, messages are still enqueued into Consumer.C.VirtualTopic.MyTopic queue.
How do I destroy such queue?
ActiveMQ pauses the Producer when the queue fills up, and I cannot set a small time to live on the message as other consumers may take a while to process each message. I cannot modify the VirtualTopic structure. I have full access to
ActiveMQ configuration and Camel routes.
Are there any other options to handle the situation?
<!-- producer route -->
<route id="ProducerRoute"/>
<from uri="direct:trigger"/>
<to uri="activemq:topic:VirtualTopic.MyTopic"/>
</route>
<!-- each consumer route -->
<route id="ConsumerARoute">
<from uri="activemq:Consumer.A.VirtualTopic.MyTopic"/>
<to uri="bean:myProcessor"/>
</route>
Look at the Apache ActiveMQ documentation how you can delete inactive queues/topics such as: http://activemq.apache.org/delete-inactive-destinations.html
I went for the aggressive solution: I hook into OSGi bundle lifecycle, when it is stopped I use JMX MBeanServer to destroy the now unneeded queues.
Since my bundle is managed using blueprint, I opted for a bean with a destroy method.
Here's an example implementation:
My bean
package org.darugna.osgi;
import javax.management.MBeanServer;
import javax.management.ObjectName;
public class QueueDestroyer {
private static final String[] QUEUES_TO_DESTROY = {
"Consumer.A.VirtualTopic.MyTopic"
};
private MBeanServer mBeanServer;
public void setMbeanServer(MBeanServer mBeanServer) {
this.mBeanServer = mBeanServer;
}
public void destroy() throws Exception {
ObjectName brokerName = new ObjectName("org.apache.activemq:type=Broker,brokerName=amq");
for (String queueName : QUEUES_TO_DESTROY) {
Object returnValue = mBeanServer.invoke(brokerName,
"removeQueue",
new Object[]{queueName},
new String[]{String.class.getName()});
}
}
}
Blueprint.xml
<blueprint>
<reference id="mbeanServer" interface="javax.management.MBeanServer"
availability="mandatory"/>
<bean id="queueDestroyer" class="org.darugna.osgi.QueueDestroyer"
destroy-method="destroy">
<property name="mbeanServer" ref="mbeanServer"/>
</bean>
<camelContext>
<route>
<from uri="activemq:Consumer.A.VirtualTopic.MyTopic"/>
<to uri="bean:myProcessor"/>
</route>
<camelContext>
</blueprint>
I had a similar situation, but I couldn't use the Claus's suggestion because in my broker there are other queues that have no consumer and I doesn't want to delete them.
In my case I'm running a JBoss Fuse 6.1.0 with fabric (I think that is the same with newer version of Fuse): I just removed the consumer (in my case I just removed the profile with the consumer) and after that I deleted the queue with the delete button in the hawtio console.
HTTP response code in range 100 to 299 is considered as success response in Camel. My requirement is that if response code is anything other than 200 then consider that as failure response. I'm trying to achieve this using custom load balancer. Problem is that when exchange is processed and if response code is 299 then for Camel it is success and it goes out of customLoadBalancer. I need a way to add custom failover in which I can check response header value and based on that route to different endpoints.
<route id="client_http" errorHandlerRef="noErrorHandler">
<from uri="direct:test"/>
<loadBalance inheritErrorHandler="false">
<custom ref="customLoadBalancer" />
<!-- TODO configurable failover attempts -->
<to ref="http4.1.to"/>
<to ref="http4.2.to"/>
<to ref="http4.3.to"/>
</loadBalance>
</route>
#Component
public class CustomLoadBalancer extends LoadBalancerSupport {
public boolean process(Exchange exchange, AsyncCallback callback) {
Logger log = Logger.getLogger(CustomLoadBalancer.class);
String body = exchange.getIn().getBody(String.class);
log.debug("Headers " + exchange.getIn().getHeaders()); //null because exchange is not processed
getProcessors().get(0).process(exchange);
log.debug("Headers " + exchange.getIn().getHeaders()); //line never executes because HTTP response was 299 and Camel considered that as success.
}
I want to use Camel to take a message from ActiveMQ and then, based on the message contents (a protobuf), send one or more messages to Twitter. I've written a bean that is called from within a route and which uses injection to send multiple messages to a "direct:xyz" endpoint.
However, Camel is complaining at runtime that:
2012-11-16 09:56:33,376 | WARN | ication.twitter] | DirectProducer | 160 - org.apache.camel.camel-core - 2.10.2 | No consumers available on endpoint: Endpoint[direct://twitter] to process: Exchange[Message: hello world]
If I instead inject directly to the Twitter endpoint from within the bean, it works fine. However, in order to ease testing, simplify configuration etc, I'd like to keep the actual Twitter config separate, hence wanting to send to a separate route.
The camel context config looks like:-
<camelContext id="NotificationTwitter"
trace="false" xmlns="http://camel.apache.org/schema/blueprint">
<dataFormats>
<protobuf id="notificationProto" instanceClass="org.abc.schemas.protobuf.NotificationDef$NotificationMsg" />
</dataFormats>
<route id="TwitterPreparation">
<from uri="activemq:notification.twitter" />
<unmarshal ref="notificationProto" />
<log logName="abc" loggingLevel="INFO"
message="Twitter request received: ${body}" />
<bean ref="NotificationTweeter" method="createTweets" />
</route>
<route id="Twitter">
<from uri="direct:twitter" />
<log logName="abc" loggingLevel="INFO"
message="Tweeting: ${body}" />
<to uri="twitter://timeline/user?consumerKey=itsasecret&consumerSecret=itsasecret&accessToken=itsasecret&accessTokenSecret=itsasecret" />
</route>
</camelContext>
The bean looks like:-
public class NotificationTweeter {
#EndpointInject(uri = "direct:twitter")
private ProducerTemplate producerTemplate;
public void createTweets(NotificationMsg notification) {
String tweet = notification.getMessageDetail().getTitle();
try {
// only send tweets where the notification message contains the Twitter mechanism
for (MechanismMsg mechanism : notification.getMechanismList()) {
if (mechanism.getType() == MechanismTypeEnum.TWITTER) {
// Cycle round the recipients
for (RecipientMsg recipient : mechanism.getRecipientList()) {
tweet = "#" + recipient.getIdentifier() + " " + tweet;
producerTemplate.sendBody(tweet);
}
// TODO exceptions if no recipients found, etc
}
}
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
I've had this problem in other routes (it's certainly not related to the Twitter feature) but have just worked around it. This time, however, I'd like to actually understand what the issue is! Any help gratefully received, thanks.
According to your setup, it might also depend on the CamelContext you have picked up. I got the same error message because I was sending messages on a route that existed in another CamelContext than the one I actually was using.
(Although the previous answer was already accepted, this might be the working solution for other people searching for that error message.)
It sounds like a problem with the startup ordering of your routes. See more detail here http://camel.apache.org/configuring-route-startup-ordering-and-autostartup.html
You can configure the "direct" route to start before the other route, then that issue should be resolved.
For others coming here, this error can also be caused by an OSGI error for a dependency that has not been deployed.
A bit late to the party but this error happened to me when I had two separate blueprint files, one for normal running and one for test. In my test I was referring to the test blueprint but noticed that the normal one was also automatically started which caused errors.
In the documentation http://camel.apache.org/blueprint-testing.html it says you can disable certain bundles from starting up. That helped me in my case.
This can also be caused by having a . in the route name. Replace my.Route.Name with myRouteName fixed the issue for me.
I would like to know if it's possible with Camel to do throttling based on the content of the exchange.
The situation is the following: I have to call a webservice via soap. Among, the parameters sent to that webservice there is a customerId. The problem is that the webservice send back an error if there are more than 1 request per minute for a given customerId.
I'm wondering if it would be possible to implement throttling per customerId with Camel. So the throttling should not be implemented for all messages but only for messages with the same customerId.
Let me know how I could implement this or if I need to clarify my question.
ActiveMQ Message Groups is designed to handle this case. So, if you can introduce a JMS queue hop in your route, then just set the JMSXGroupId header to the customerId. Then in another route, you can consume from this queue and send to your web service to get the behavior you described.
also see http://camel.apache.org/parallel-processing-and-ordering.html for more information...
While ActiveMQ Message Groups would definitely address the parallel processing of unique customer ID's, in my assessment Claus is correct that introducing a throttle for each unique group represents an unimplemented feature for Camel/ActiveMQ.
Message Groups alone will not meet the SLA described. While each group of messages (correlated by the customer ID) will be processed in order with one thread per group, as long as requests take less than a minute to receive a response, the requirement of one request per minute per customer would not be enforced.
That said, I would be very interested to know if it would be possible to combine Message Groups and a throttle strategy in a way that would simulate the feature request in JIRA. My attempts so far have failed. I was thinking something along these lines:
<route>
<from uri="activemq:pending?maxConcurrentConsumers=10"/>
<throttle timePeriodMillis="60000">
<constant>1</constant>
<to uri="mock:endpoint"/>
</throttle>
</route>
However, the throttle seems to be applied to the entire set of requests moving to the endpoint, and not to each individual consumer. I have to admit, I was a bit surprised to find that behavior. My expectation was that the throttle would apply to each consumer individually, which would satisfy the SLA in the original question, provided that the messages include the customer ID in the JMSXGroupId header.
I came across a similar problem and finally came up with the solution described here.
My assumptions are:
Order of messages is not important (though it can be solved by re-sequencer)
Total volume of messages per customer ID is not great so the runtime is not saturated.
The solution approach:
Run aggregator for 1 minute while using customerID to assemble messages with the same customer ID into a list
Use Splitter to split the list into individual messages
Send the first message from the splitter to the actual service
Re-route the rest of the list back into the aggregator.
Java DSL version is a bit easier to understand:
final AggregationStrategy aggregationStrategy = AggregationStrategies.flexible(Object.class)
.accumulateInCollection(ArrayList.class);
from("direct:start")
.log("Receiving ${body}")
.aggregate(header("customerID"), aggregationStrategy).completionTimeout(60000)
.log("Aggregate: releasing ${body}")
.split(body())
.choice()
.when(header(Exchange.SPLIT_INDEX).isEqualTo(0))
.log("*** Processing: ${body}")
.to("mock:result")
.otherwise()
.to("seda:delay")
.endChoice();
from("seda:delay")
.delay(0)
.to("direct:start");
Spring XML version looks like the following:
<!-- this is our aggregation strategy defined as a spring bean -->
<!-- see http://stackoverflow.com/questions/27404726/how-does-one-set-the-pick-expression-for-apache-camels-flexibleaggregationstr -->
<bean id="_flexible0" class="org.apache.camel.util.toolbox.FlexibleAggregationStrategy"/>
<bean id="_flexible2" factory-bean="_flexible0" factory-method="accumulateInCollection">
<constructor-arg value="java.util.ArrayList" />
</bean>
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="direct:start"/>
<log message="Receiving ${body}"/>
<aggregate strategyRef="_flexible2" completionTimeout="60000" >
<correlationExpression>
<xpath>/order/#customerID</xpath>
</correlationExpression>
<log message="Aggregate: releasing ${body}"/>
<split>
<simple>${body}</simple>
<choice>
<when>
<simple>${header.CamelSplitIndex} == 0</simple>
<log message="*** Processing: ${body}"/>
<to uri="mock:result"/>
</when>
<otherwise>
<log message="--- Delaying: ${body}"/>
<to uri="seda:delay" />
</otherwise>
</choice>
</split>
</aggregate>
</route>
<route>
<from uri="seda:delay"/>
<to uri="direct:start"/>
</route>
</camelContext>