I have a question regarding route management in Apache Camel. It seems to me that it should be handled, but I'm going to fire my question anyway.
If I have RouteA registered in the Camel Context and I execute it. While the route is being executed, another route with the same route id overwrites the original one (let's call it RouteA'). What will happen to the execution of the RouteA? What kind of behavior will I see?
I can imagine one of the following 3 cases:
RouteA Request --> Back end --> RouteA' overwrites RouteA -->
RouteA Response
RouteA Request --> Back end --> RouteA' overwrites RouteA -->
RouteA' Response
RouteA Request --> Back end --> RouteA' overwrites RouteA -->
Exception
Thank you!
Found my answer
when you add a route with the same routeId, it will attempt to do:
stopRoute(routeId)
Then apply the graceful shutdown strategy.
removeRoute(routeId)
After applying the algorithm above that it will add the new route with the same routeId
I was essentially trying to understand what was the shutdown strategy in camel, which is explained in details in the source code under DefaultShutdownStrategy.java see:
https://github.com/apache/camel/blob/3c0b7d0367c10379bbb09a0d0bb56197885b5a51/camel-core/src/main/java/org/apache/camel/impl/DefaultShutdownStrategy.java#L85
Related
I'm using Camel to implement the saga pattern using its Saga EIP, one of my requirements is having the route body in the compensation routes, to achieve this the documentation advises to use the option method:
from("direct:example")
.routeId("example")
.enrich("direct:getData", dataAggregationStrategy)
.process(somethingProcessor)
.setHeader("Body", body())
.saga()
.option("OptionBody", body())
.completion("direct:exampleDone")
.compensation("direct:exampleCompensated")
.to("direct:doSomething");
In my code I'm doing some enrich and process steps to manipulate the body and all is well in the routes called by to which receive the correct processed body.
In the compensation routes though I'm receiving an header OptionBody containing the original body passed to the direct:example route and not the processed one.
I tried to set an header with setHeader and the body is the correct version (the processed one).
I'm not sure why this is happening, is it because of lazy evaluation of the expressions?
I have two Camel routes, configured in XML and pasted below: -
Route 1:
<camel:route id="statementsArchivingPollRoute">
<camel:from uri="timer://tempQueue?fixedRate=true&period=30s&delay=30s"/>
<camel:transacted ref="PROPAGATION_REQUIRED">
<camel:process ref="statementsArchivingRequestZipProcessor"/>
<camel:choice>
<camel:when>
<camel:simple>${body.size} >= 1</camel:simple>
<camel:split>
<camel:simple>${body}</camel:simple>
<camel:marshal ref="archiveFileInterfaceMetadataMapper"/>
<camel:to pattern="InOnly"
uri="activemq:{{ccs.activemq.queue.prefix}}.sr.archive.bulk.ingestion.req?jmsMessageType=Text"/>
</camel:split>
<camel:log loggingLevel="INFO" message="Archiving content was processed"/>
</camel:when>
<camel:otherwise>
<camel:log loggingLevel="INFO" message="No archiving content to process"/>
</camel:otherwise>
</camel:choice>
</camel:transacted>
</camel:route>
Route 2:
<camel:route id="statementsArchivingBulkIngestionRequestRoute">
<camel:from uri="activemq:{{ccs.activemq.queue.prefix}}.sr.archive.bulk.ingestion.req"/>
<camel:throttle timePeriodMillis="4000">
<camel:constant>1</camel:constant>
<camel:setExchangePattern pattern="InOnly"/>
<camel:unmarshal ref="archiveFileInterfaceMetadataMapper"/>
<camel:bean ref="archiveFileEntryTransformer" method="transform"/>
<camel:setHeader headerName="CamelHttpMethod">
<camel:constant>POST</camel:constant>
</camel:setHeader>
<camel:toD uri="{{ccs.bulk.ingestion.service.ingest.archive.file}}"/>
</camel:throttle>
</camel:route>
The processor in the first route returns a list of request objects. The list is then split and each request is marshalled and placed on a queue.
The second route listens to this queue. When it de-queues a message, it unmarshals it, performs a transform and then uses it to send a post request to another service. I am throttling this route so that it only processes one message per second so as not to overwhelm the downstream service.
This all works fine when the list only contains a few requests and hence only a few messages enter the queue, but when there are many items in the list, route 2 times out and the log entry below appears:
Atomikos:12] c.a.icatch.imp.ActiveStateHandler : Timeout/setRollbackOnly of ACTIVE coordinator !
The timeout leads to the process repeating itself and the downstream service ends up getting called multiple times per message instead of just once.
I can't understand why the number of times route 2 is invoked should cause it to timeout. I thought that a single instance of the route would be launched for every message de-queued from the activemq. If a single message takes a long time to complete, then I would understand, but clearly the timeout is based on the cumulative times of all messages being de-queued.
I am fairly new to Camel and I have clearly misunderstood something from an architectural point of view. I would be extremely grateful for any guidance into how to stop these timeouts occurring. Thank you for reading.
Jon, you may want to try following to investigate
disable/comment the 2nd route. Purpose of using activemq would be to make process async which means 1st route should not have any impact due to 2nd route. If this works without route 2 then problem is elsewhere.
If you find 1st route is working fine without 2nd route then next would be to try setting fewer number of threads in 2nd route. May be put 1 or 2 threads and see if that helps. I am thinking it is contention on activemq rather than these route configurations.
Check the payload size that you are pushing to activemq. If you are publishing very large message to activemq that may also have an impact as large number of items and each item is large causing contention in activemq and transaction is taking longer than the timeout setting.
If you are pushing large data set to activemq, you may want to revisit the design. Save payload in some persistence area (db/file/cache) and sent notification events containing only reference to the payload and some metadata. 2nd route can then take the reference from event and retrieve the payload from where it was saved by route 1.
I am trying to understand the unit of work concept in Camel. I have a simple question hopefully someone here can help.
If there are multiple routes involved in routing the Excachange for example
from("aws-sqs:Q1").to("direct:processMe");//route1
from("direct:processMe").to("direct:aws-post");//route2
from("direct:aws-post").to("htt4:myservice");//route3
Is the unit of work invoked at the end of each routes? Or only at the end of route3? In my example, will the SQS message be deleted off of SQS once route1 completes? Or will it wait until my message reaches "myservice"?
Thanks.
FOLLOW UP:
I've modified the route slighty:
from("aws-sqs:Q1").to("direct:processMe");//route1
from("direct:processMe").process(
new Processor(){
public void process(Exchange exchange) throws Exception {
throw new RuntimeException("fail on purpose");
}
}
).to("direct:aws-post");//route2
from("direct:aws-post").to("http4:myservice");//route3
The thought process is this:
If the unit of work is invoked at end of each route then once the message is read from SQS Queue it will be acknowledged as read by the SQS component. On the other hand if unit of work is only invoked once the Exchange is done routing through all routes, then the exception in route 2 will result in the message not being acknowledged and will be available for redelivery once the visibility period expires.
The test showed that the message remains on the Q despite being read by first route. It is picked up again and again (until it ends up in dead letter q). As such I strongly believe that the unit boundary is defined by the end of Exchange being routed.
A unit of work is basically a transaction.
By default a message for a route (a camel Exchange) runs inside a single UnitOfWork context.
In your example there are 3 UnitOfWorks set up, starting at each from and finishing at the final to in each route.
I would expect the message from SQS to be consumed after the 1st route finishes. To test, you could add in a sleep to allow you to check the queue.
from("direct:processMe").process(new Processor()
{ void process() { try { Thread.sleep(60000L) } catch (Exception e) { } }
}).to("direct:aws-post")
If you want the message to remain on the queue until myservice gets the message then you need to put the processing in a single route.
I have found a great explanation as to the unit of work boundary:
"The OnCompletion DSL name is used to define an action that is to take place when a Unit of Work is completed.
A Unit of Work is a Camel concept that encompasses an entire exchange. See Section 43.1, “Exchanges”. The onCompletion command has the following features:
The scope of the OnCompletion command can be global or per route. A route scope overrides global scope.
OnCompletion can be configured to be triggered on success for failure.
The onWhen predicate can be used to only trigger the onCompletion in certain situations.
You can define whether or not to use a thread pool, though the default is no thread pool."
In case of SQS processing, the consumer defines onCompletion on the exchange. So it is invoked only after the exchange is done routing.
The whole answer can be found here: Apache Camel Development Guide2.11. OnCompletion
In Camel in Action book, it is written that default exception handling is for the life span of exchange object only. Suppose I am using following route -
from(sftp:{details}).to(file:{details})
1st query -> For the above route, I think the consumer is sftp and the producer is file. Am I right here?
2nd query -> How can we handle the exception at destination end point (which is outside the life span of exchange object).
1st You are correct.
2nd Any exception that will be raised before the file is successfully written to disk will trigger Camel's exception handling which will by default in file based components trigger rollback of the original file so that data won't be lost.
I have an camel route consuming on a JMS (activemq) queue targeted to be called in a request/reply manner. Inside this route I split the message and invoke another activemq queue (also in a request/reply manner).
Heres a minimal route showing the situation
<route>
<from uri="activemq:A" />
<split>
<xpath>/root/subpart</xpath>
<inOut uri="activemq:B" />
</split>
</route>
The problem is that Camel does not set a new JMSCorrelationId (since there is already one from the incoming message). If nothing is done, you get responses with unknown correlationId's and the exchanges never end.
I didn't go into details but my guess is that the same temporaryQueue is used for the hole splitter but that it (logically) expects different correlation id's for each of the messages. All using the same, it recieves the first and does not know what to do with the others.
What would be the best solution to handle the situation ?
The one I've found working is to save in another header the incoming JMSCorrelationId (not sure I need to though), and removing it. This is not really as clean as I would want it to be, but I couldn't think of something else. Any ideas ?
Essentially, your case is described in this Jira issue It seems there will be an addition in 2.11 where you can ask Camel to create a new corr-id.
So, in the meantime, why don't you continue what you had working - to remove the JMSCorrelationId header <removeHeader headerName="JMSCorrelationId" /> before you send it to "activemq:B"? I guess that is the best solution for now.
You could, of course, play with the "useMessageIDAsCorrelationID" option as well on the second endpoint.