Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: prevent codecov from publishing coverage report early #4269

Merged
merged 2 commits into from
Aug 13, 2024

Conversation

StephenHulme
Copy link
Contributor

@StephenHulme StephenHulme commented Aug 13, 2024

Prevent CodeCov from publishing coverage report early

Changes proposed in this pull request

  • Wait for 12 reports to be uploaded before CodeCov comments on PR

Instructions for Reviewers

See equivalent change in Limber: sanger/limber#1806

[All PRs] - Confirm PR template filled
[Feature Branches] - Review code
[Production Merges to main]
    - Check story numbers included
    - Check for debug code
    - Check version

@StephenHulme StephenHulme changed the title fix: prevent CodeCov from publishing coverage report early fix: prevent codecov from publishing coverage report early Aug 13, 2024
Copy link

codecov bot commented Aug 13, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 87.45%. Comparing base (b4164ee) to head (73dc6b4).

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #4269      +/-   ##
===========================================
+ Coverage    87.41%   87.45%   +0.04%     
===========================================
  Files         1369     1369              
  Lines        29601    29601              
===========================================
+ Hits         25876    25889      +13     
+ Misses        3725     3712      -13     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@StephenHulme
Copy link
Contributor Author

Note how there are no edits to the Codecov report above - compare with the report in #4265

There is still variability in the coverage - I suspect due to flaky tests - but at least now we get one number per PR

Copy link
Contributor

@sdjmchattie sdjmchattie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could afford to be noisier about the warning in the ruby_test.yml file and make it more obvious what you mean in the warning, by using a substantial block of comment similar to:

########
## Warning! 
## CodeCov's config in `codecov.yml` is expecting an exact number of builds to complete before it will run.
## See the config value with key `after_n_builds` in that file.
## If the number of build steps changes, it is important to update the count there to match.
########

but I'm happy with the overall approach

@StephenHulme StephenHulme merged commit 65a8bac into develop Aug 13, 2024
21 checks passed
@StephenHulme StephenHulme deleted the sh51/prevent-multiple-codecov-reports branch August 13, 2024 13:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants