-
Notifications
You must be signed in to change notification settings - Fork 412
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore(ci_visibility): introduce experimental internal coverage collector #8727
Conversation
For posterity's sake, I insist on mentioning that the overwhelming majority of this work comes from @P403n1x87 , and I'm just piggybacking my changes on top. Code review should mainly focus on the safety of the feature flags (eg: that we don't break things unless people are using undocumented flags from unsupported features). |
BenchmarksBenchmark execution time: 2024-05-22 09:20:02 Comparing candidate commit d5fcbec in PR branch Found 0 performance improvements and 2 performance regressions! Performance is the same for 199 metrics, 9 unstable metrics. scenario:span-start
scenario:span-start-traceid128
|
Datadog ReportBranch report: ✅ 0 Failed, 117032 Passed, 58751 Skipped, 3h 54m 40.27s Total duration (5h 15m 39.44s time saved) ⌛ Performance Regressions vs Default Branch (6)
|
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #8727 +/- ##
===========================================
- Coverage 76.17% 10.46% -65.72%
===========================================
Files 1288 1262 -26
Lines 122505 121045 -1460
===========================================
- Hits 93321 12665 -80656
- Misses 29184 108380 +79196 ☔ View full report in Codecov by Sentry. |
…py into romainkomorn/add_clear
Introduces the
ModuleCodeCollector
which collects coverage and executable lines for imported modules.The collector has two modes, one that stores executed lines on the singleton instance, and one that uses context variables and a context manager.
This also introduces changes to the
pytest
integration as well as theCIVisibility
service's use of coverage to feature-flag using the new module collector.The features are gated behind
_DD_USE_INTERNAL_COVERAGE
and_DD_COVER_SESSION
(which introduces a newcoverage run
like behavior).There are no unit tests though the overall use of the feature flags has been tested quite extensively in the process of collecting performance data.
There are no release notes since this is an entirely undocumented feature for the moment.
Checklist
changelog/no-changelog
is set@DataDog/apm-tees
.@DataDog/security-design-and-guidance
.Reviewer Checklist