Akka Http read entity fails with Substream Source cannot be materialized more than once - akka-stream

Question

When I am trying to consume the data on server into memory and later I want to create strict entity with it, but getting Substream Source cannot be materialized more than once error message.
As far as I know I am not consuming the data from the request more than once (from application layer not investigated akka-http library layer). Requests are successful with small chunk of data, failing with bigger (around 10 KB) chunk of data
First try:
if (!entity.isChunked()) {
entity.dataBytes.runFold(ByteString.empty)(_ ++ _).flatMap { encryptedBody =>
decrypt(encryptedBody).map { decrypted =>
HttpEntity(decrypted)
}
}
}
also tried following (suggestion in Akka http -ERROR :Substream Source cannot be materialized more than once If Payload size increased) but no luck
Second try (with strict):
entity.toStrict(4.seconds).flatMap { strictEntity =>
decrypt(strictEntity.data).map { decrypted =>
HttpEntity(decrypted)
}
}
AkkaHttp Version: 10.0.13 ( We are in the process of migrating to latest version, we are kind of stuck with older version)
Akka Version : 2.4.20
Any kind of help would be great.
Stack trace is:
[info] java.lang.IllegalStateException: Substream Source cannot be materialized more than once
[info] at akka.stream.impl.fusing.SubSource$$anon$4.setCB(StreamOfStreams.scala:725)
[info] at akka.stream.impl.fusing.SubSource$$anon$4.preStart(StreamOfStreams.scala:735)
[info] at akka.stream.impl.fusing.GraphInterpreter.init(GraphInterpreter.scala:520)
[info] at akka.stream.impl.fusing.GraphInterpreterShell.init(ActorGraphInterpreter.scala:380)
[info] at akka.stream.impl.fusing.ActorGraphInterpreter.tryInit(ActorGraphInterpreter.scala:538)
[info] at akka.stream.impl.fusing.ActorGraphInterpreter.preStart(ActorGraphInterpreter.scala:586)
[info] at akka.actor.Actor$class.aroundPreStart(Actor.scala:510)
[info] at akka.stream.impl.fusing.ActorGraphInterpreter.aroundPreStart(ActorGraphInterpreter.scala:529)
[info] at akka.actor.ActorCell.create(ActorCell.scala:590)
[info] at akka.actor.ActorCell.invokeAll$1_aroundBody2(ActorCell.scala:461)
[info] at akka.actor.ActorCell$AjcClosure3.run(ActorCell.scala:1)
[info] at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
[info] at akka.kamon.instrumentation.ActorSystemMessageInstrumentation$$anonfun$aroundSystemMessageInvoke$1.apply(ActorSystemMessageInstrumentation.scala:34)
[info] at kamon.trace.Tracer$.withContext(TracerModule.scala:58)
[info] at akka.kamon.instrumentation.ActorSystemMessageInstrumentation.aroundSystemMessageInvoke(ActorSystemMessageInstrumentation.scala:34)
[info] at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:1)
[info] at akka.actor.ActorCell.systemInvoke(ActorCell.scala:483)
[info] at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:282)
[info] at akka.dispatch.Mailbox.run(Mailbox.scala:223)
[info] at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
[info] at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
[info] at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
[info] at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
[info] at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

Answer 1

Failing because the entity was read more than once.
In the beginning, I could not figure it out because the reading was happening at the time validation, and it is deeply buried in the validations. (one of the validation depends on body content)
I tried strict entity conversion after the validations, so it did not work out.