Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(ci_visibility): don't rely on git in pytest plugin [backport 2.8] #10024

Merged
merged 3 commits into from
Aug 2, 2024

Conversation

romainkomorndatadog
Copy link
Collaborator

Backport c460441 from #9989 to 2.8.

Fixes an issue introduced by #9586 and reported in #9981 where the pytest plugin would crash if the git binary was absent.

The workspace is instead grabbed from the CIVisibility service's tags (via a new getter classmethod).

In order for the variable to persist through to the pytest_terminal_summary hook, it is moved from being stashed on the pytest session object to the nested config object (which itself is passed to pytest_terminal_summary).

For improved testability, a function to get the location of the git binary's passed, using @cached() to avoid re-fetching the path each time we call git.

Checklist

  • PR author has checked that all the criteria below are met
  • The PR description includes an overview of the change
  • The PR description articulates the motivation for the change
  • The change includes tests OR the PR description describes a testing strategy
  • The PR description notes risks associated with the change, if any
  • Newly-added code is easy to change
  • The change follows the library release note guidelines
  • The change includes or references documentation updates if necessary
  • Backport labels are set (if applicable)

Reviewer Checklist

  • Reviewer has checked that all the criteria below are met
  • Title is accurate
  • All changes are related to the pull request's stated goal
  • Avoids breaking API changes
  • Testing strategy adequately addresses listed risks
  • Newly-added code is easy to change
  • Release note makes sense to a user of the library
  • If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
  • Backport labels are set in a manner that is consistent with the release branch maintenance policy

Fixes an issue introduced by #9586 and reported in #9981 where the
pytest plugin would crash if the `git` binary was absent.

The workspace is instead grabbed from the `CIVisibility` service's tags
(via a new getter classmethod).

In order for the variable to persist through to the
`pytest_terminal_summary` hook, it is moved from being stashed on the
pytest `session` object to the nested `config` object (which itself is
passed to `pytest_terminal_summary`).

For improved testability, a function to get the location of the `git`
binary's passed, using `@cached()` to avoid re-fetching the path each
time we call `git`.

- [x] PR author has checked that all the criteria below are met
- The PR description includes an overview of the change
- The PR description articulates the motivation for the change
- The change includes tests OR the PR description describes a testing
strategy
- The PR description notes risks associated with the change, if any
- Newly-added code is easy to change
- The change follows the [library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
- The change includes or references documentation updates if necessary
- Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

- [ ] Reviewer has checked that all the criteria below are met
- Title is accurate
- All changes are related to the pull request's stated goal
- Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- Testing strategy adequately addresses listed risks
- Newly-added code is easy to change
- Release note makes sense to a user of the library
- If necessary, author has acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment
- Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

---------

Co-authored-by: erikayasuda <[email protected]>
(cherry picked from commit c460441)
@datadog-dd-trace-py-rkomorn
Copy link

datadog-dd-trace-py-rkomorn bot commented Aug 1, 2024

Datadog Report

Branch report: backport-9989-to-2.8
Commit report: 57598d4
Test service: dd-trace-py

❌ 11 Failed (1 Known Flaky), 169635 Passed, 1487 Skipped, 12h 37m 36.79s Total duration (28m 3.06s time saved)
❄️ 1 New Flaky
⌛ 2 Performance Regressions

❌ Failed Tests (11)

This report shows up to 5 failed tests.

  • test_get_distributions - test_packages.py - ❄️ Known flaky - Details

    Expand for error
     assert {'attrs', 'au...overage', ...} == {'attrs', 'au...overage', ...}
       Extra items in the left set:
       'importlib-resources'
       Extra items in the right set:
       'importlib_resources'
       Full diff:
         {
          'attrs',
          'autocommand',
          'backports.tarfile',
     ...
    
  • test_span_creation_and_finished_metrics_otel - test_telemetry_metrics_e2e.py - Details

    Expand for error
     b'Failed to load context: ddcontextvars_context, fallback to contextvars_context
       Traceback (most recent call last):
       ... return tracer_provider.get_tracer(
       TypeError: get_tracer() takes from 2 to 4 positional arguments but 5 were given
       '
     assert 1 == 0
    
  • test_span_creation_and_finished_metrics_otel - test_telemetry_metrics_e2e.py - Details

    Expand for error
     b'Failed to load context: ddcontextvars_context, fallback to contextvars_context
       Traceback (most recent call last):
       ...^^^^^^^^^^^^^^^^^^^^
       TypeError: TracerProvider.get_tracer() takes from 2 to 4 positional arguments but 5 were given
       '
     assert 1 == 0
    
  • test_span_creation_and_finished_metrics_otel - test_telemetry_metrics_e2e.py - Details

    Expand for error
     b'Failed to load context: ddcontextvars_context, fallback to contextvars_context
       Traceback (most recent call last):
       ... return tracer_provider.get_tracer(
       TypeError: get_tracer() takes from 2 to 4 positional arguments but 5 were given
       '
     assert 1 == 0
    
  • test_span_creation_and_finished_metrics_otel - test_telemetry_metrics_e2e.py - Details

    Expand for error
     b'Failed to load context: ddcontextvars_context, fallback to contextvars_context
       Traceback (most recent call last):
       ...^^^^^^^^^^^^^^^^^^^^
       TypeError: TracerProvider.get_tracer() takes from 2 to 4 positional arguments but 5 were given
       '
     assert 1 == 0
    

New Flaky Tests (1)

  • test_send_timed_events - test_llmobs_writer.py - Last Failure

    Expand for error
     Calls not found.
     Expected: [call('sent %d LLMObs events to %r', 1, 'https://llmobs-intake.datad0g.com/api/v2/llmobs')]
    

⌛ Performance Regressions vs Default Branch (2)

  • test_schematized_operation_names[None] - test_djangorestframework.py 6.98s (+5.66s, +426%) - Details
  • test_schematized_service_name[None-None] - test_request.py 6.49s (+5.53s, +579%) - Details

@pr-commenter
Copy link

pr-commenter bot commented Aug 1, 2024

Benchmarks

Benchmark execution time: 2024-08-02 07:44:06

Comparing candidate commit 161b56f in PR branch backport-9989-to-2.8 with baseline commit 3f5a7c5 in branch 2.8.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 201 metrics, 9 unstable metrics.

@romainkomorndatadog romainkomorndatadog enabled auto-merge (squash) August 2, 2024 06:40
@romainkomorndatadog romainkomorndatadog merged commit edc6cac into 2.8 Aug 2, 2024
81 of 99 checks passed
@romainkomorndatadog romainkomorndatadog deleted the backport-9989-to-2.8 branch August 2, 2024 08:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants