Apache Camel - split and aggregation bug - apache-camel

Question

I tried to create a bug in Camel's issue tracker but it's not easy to get access there now. So maybe someone will be able to help me here.
I'm migrating gradually to the newest Camel version. Currently I'm going from the 3.7.3 to 3.11.7 but I checked that this bug happens also on 3.20.1.
Ok, so to the point.
When I have pipeline like this:
.to(SPLIT_WORKER_ROUTE_ID, OTHER_ROUTE_ID)
it should execute in sequence. But when somewhere inside SPLIT_WORKER_ROUTE_ID I have an aggregation code like this:
.split(body())
.process(splitWorkerProcessor)
.aggregate(exchangeProperty(CORRELATION_ID), new SplitAggregator()).completionSize(exchangeProperty(SPLIT_SIZE))
.to(AFTER_SPLIT_ROUTE_ID)
before it goes to AFTER_SPLIT_ROUTE_ID the OTHER_ROUTE_ID kicks in and starts to run in parallel with SPLIT_WORKER_ROUTE_ID.
When I rewrite the code like this (or go back to Camel 3.7.3):
.split(body(), new SplitAggregator()).parallelProcessing()
.process(splitWorkerProcessor)
.end()
.to(AFTER_SPLIT_ROUTE_ID)
everything runs as it should sequentially. Unfortunately, I have to use more complex aggregation conditions so I'm afraid I cannot use this workaround as aggregation configuration is not possible in this approach.
I guess that according to
https://camel.apache.org/manual/camel-3x-upgrade-guide-3_11.html#_aggregate_eip
something has changed in this area. (EDIT: I've just checked Camel 3.10 and it works properly so I'm 99.99% sure this change introduced this bug)
The problem leads to the situation that order of execution is disturbed and we have this:
.to(SPLIT_WORKER_ROUTE_ID, OTHER_ROUTE_ID)
the OTHER_ROUTE_ID can complete before this sequence SPLIT_WORKER_ROUTE_ID -> AFTER_SPLIT_ROUTE_ID.
Here is the log presenting the problem:
2023-02-02T18:45:41,229 [main] INFO direct://Main [...] [] [] [] [] - MAIN START
2023-02-02T18:45:41,230 [Camel (camel-1) thread #1 - Threads] INFO direct://splitWorker [...] [] [] [] [] - SPLIT_WORKER_ROUTE_ID START
2023-02-02T18:45:41,399 [Camel (camel-1) thread #1 - Threads] INFO direct://other [...] [] [] [] [] - OTHER_ROUTE_ID START
2023-02-02T18:45:41,399 [Camel (camel-1) thread #3 - Aggregator] INFO direct://splitWorker [...] [] [] [] [] - Aggregation just finished inside SPLIT_WORKER_ROUTE_ID START!
2023-02-02T18:45:41,399 [Camel (camel-1) thread #3 - Aggregator] INFO direct://splitWorker [...] [] [] [] [] - SPLIT_WORKER_ROUTE_ID FINISH
2023-02-02T18:45:41,400 [Camel (camel-1) thread #3 - Aggregator] INFO direct://afterSplit [...] [] [] [] [] - AFTER_SPLIT_ROUTE_ID START
2023-02-02T18:45:42,404 [Camel (camel-1) thread #1 - Threads] INFO direct://other [...] [] [] [] [] - OTHER_ROUTE_ID FINISH
2023-02-02T18:45:43,406 [Camel (camel-1) thread #3 - Aggregator] INFO direct://afterSplit [...] [] [] [] [] - AFTER_SPLIT_ROUTE_ID FINISH
2023-02-02T18:45:47,417 [Camel (camel-1) thread #6 - Delay] INFO direct://Main [...] [] [] [] [] - MAIN FINISH
I would appreciate any help, thanks a lot!

Answer 1

By default, the output of the aggregator is executed on a thread from the aggregator's thread pool. However, you can have the aggregator output run in the same thread as the calling route:
.aggregate(exchangeProperty(CORRELATION_ID), new SplitAggregator())
.completionSize(exchangeProperty(SPLIT_SIZE))
.executorService(new SynchronousExecutorService())
This technique is briefly described here.