Flaky CI test execution issues after package updates

[gasparnagy]

Used Visual Studio

Visual Studio 2022

Are the latest Visual Studio updates installed?

Yes

Content of reqnroll.json (if present)

No response

Issue Description

When I worked on upgrading the packages locally, I also observed that after upgrading a few, some tests related to logging became flaky. At that time it seemed that the problem is with the ApprovalTests dependency (5.9.0 -> 6.0.0), so I left it as v5.9.0.

Now after the package upgrades are re-applied, the builds fail again with the same errors. (So probably was nothing to do with ApprovalTests.) This build I have even tried 3 times, all 3 failed.

The affected test class seems to be the Diagnostics.LoggingTests class, that is NOT an approval test. I reviewed the tests, but haven't found anything bad. They use a mock file system, so I don't understand how this problem is caused. But it seems we need to investigate this further or if nothing happens rewrite these tests with some other (?) method.

Steps to Reproduce

run CI or unit tests locally

Link to a project repository that reproduces the issue

No response

[UL-ChrisGlew]

Just to summarize the discussion @gasparnagy and I were having over at #39

It doesn't look like some of the unit test flakiness present is as a result of the package updates that occured recently in the extension - I am seeing some slight flakiness in the test executions before the package updates went through (commit #e221b):

[gasparnagy]

Examples from CI:

• https://github.com/reqnroll/Reqnroll.VisualStudio/actions/runs/11722251217 ("Find unused step definition with a single step definition attribute" in Features.Editor.Commands.FindUnusedStepDefinitionsCommandFeature)
• https://github.com/reqnroll/Reqnroll.VisualStudio/actions/runs/11722442228 (MessageIsLogged in Diagnostics.LoggingTests)