How to ensure that test is executed after all tests in scalatest? - scalatest

Question

I want to test that all methods of a REST API are covered by tests.
All http calls are recorded into a mutable set, and I have a piece of code that checks correspondence between specification and the result set of recored api calls.
I can place this check in a separate test at the end of FunSuite and it would be executed after all other tests. However, there are two problems: I have to copy-paste it in every file that tests API and make sure it is at the end of file.
Using common trait does not work: tests from parent class are executed before tests from child class. Placing the test inside afterAll does not work either: scalatest swallows all exceptions (including test failures) thrown in it.
Is there a way to run some test after all others without boilerplate?

Answer 1

Personally I would go with dedicated coverage tool such as scoverage. One advantage would be avoiding global state.
Nevertheless, as per question, a way to execute test after all tests would be via Suites and BeforeAndAfterAll traits like so
import org.scalatest.{BeforeAndAfterAll, Suites, Matchers}
class AllSuites extends Suites(
new FooSpec,
new BarSpec,
) with BeforeAndAfterAll withy Matchers {
override def afterAll(): Unit = {
// matchers here as usual
}
}
Here is a toy example with global state as per question
AllSuites.scala
import org.scalatest.{BeforeAndAfterAll, Matchers, Suites}
object GlobalMutableState {
val set = scala.collection.mutable.Set[Int]()
}
class AllSuites extends Suites(
new HelloSpec,
new GoodbyeSpec
) with BeforeAndAfterAll with Matchers {
override def afterAll(): Unit = {
GlobalMutableState.set should contain theSameElementsAs Set(3,2)
}
}
HelloSpec.scala
#DoNotDiscover
class HelloSpec extends FlatSpec with Matchers {
"The Hello object" should "say hello" in {
GlobalMutableState.set.add(1)
"hello" shouldEqual "hello"
}
}
GoodbyeSpec.scala
#DoNotDiscover
class GoodbyeSpec extends FlatSpec with Matchers {
"The Goodbye object" should "say goodbye" in {
GlobalMutableState.set.add(2)
"goodbye" shouldEqual "goodbye"
}
}
Now executing sbt test gives something like
[info] example.AllSuites *** ABORTED ***
[info] HashSet(1, 2) did not contain the same elements as Set(3, 2) (AllSuites.scala:15)
[info] HelloSpec:
[info] The Hello object
[info] - should say hello
[info] GoodbyeSpec:
[info] The Goodbye object
[info] - should say goodbye
[info] Run completed in 377 milliseconds.
[info] Total number of tests run: 2
[info] Suites: completed 2, aborted 1
[info] Tests: succeeded 2, failed 0, canceled 0, ignored 0, pending 0
[info] *** 1 SUITE ABORTED ***
[error] Error during tests:
[error] example.AllSuites
[error] (Test / test) sbt.TestsFailedException: Tests unsuccessful