@FlakyTest similar to @RepeatedTest

[Bananeweizen]

Flaky tests have been a problem in every huge project I worked with. To overcome that, I have developed multiple approaches of automatically dealing with the flaky test by means of rules and extensions. My most recent approach is basically a variation of @RepeatedTest that doesn't have a fixed number of iterations, but rather adds more invocations depending on the test execution result monitored by being also an InvocationInterceptor. This works without any hacks of the test execution itself, deals fully with the lifecycle methods etc.

I was wondering if there are already other (better?) extensions for flaky tests, and whether something like dealing with flakiness as such would be something worth to consider in the JUnit project itself, especially if it can also be implemented extremely similar to the existing @RepeatedTest annotation.

[sbrannen]

This topic has been discussed in length in:

• Introduce support for retrying failed flaky tests #1558

Please continue the discussion there.

[Bananeweizen]

Sorry for completely missing that. I only searched in discussions. My fault. Thanks for pointing to the right direction.

[sbrannen]

No problem.

And now that I've noticed your username, I figured you must be from Baden-Württemberg or Bavaria, and I guessed correctly. 🍻