Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Randomize running of the test suite #2613

Open
4 tasks
mfocko opened this issue Oct 31, 2024 · 0 comments
Open
4 tasks

Randomize running of the test suite #2613

mfocko opened this issue Oct 31, 2024 · 0 comments
Labels
area/general Related to whole service, not a specific part/integration. complexity/single-task Regular task, should be done within days. kind/internal Doesn't affect users directly, may be e.g. infrastructure, DB related. testing Tests are impacted.

Comments

@mfocko
Copy link
Member

mfocko commented Oct 31, 2024

While working on #2590, I've uncovered some weird test failures that appeared after refactoring the GitHub events. I suspected that some of the mocks affect unrelated tests, so I've rebuilt the test image with pytest-randomly and ran the tests on main, they randomly fail (pun not intended…).

Based on the arch discussion:

  • add pytest-randomly to test deps
  • add another make target to run tests with and without randomizing
  • run both in parallel in Zuul
  • fix the tests…
@mfocko mfocko added testing Tests are impacted. complexity/single-task Regular task, should be done within days. area/general Related to whole service, not a specific part/integration. kind/internal Doesn't affect users directly, may be e.g. infrastructure, DB related. labels Oct 31, 2024
@mfocko mfocko moved this from new to priority-backlog in Packit Kanban Board Oct 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/general Related to whole service, not a specific part/integration. complexity/single-task Regular task, should be done within days. kind/internal Doesn't affect users directly, may be e.g. infrastructure, DB related. testing Tests are impacted.
Projects
Packit Kanban Board
Status: priority-backlog
Milestone
No milestone
Development

No branches or pull requests
1 participant
@mfocko