From 370b175d40c4afaf6145eeae4a81aa97ac16789b Mon Sep 17 00:00:00 2001
From: Romain Komorn <136473744+romainkomorndatadog@users.noreply.github.com>
Date: Fri, 17 May 2024 10:33:56 +0100
Subject: [PATCH] ci: add ITR unskippable marker is subprocess marker is used
 (#9261)

This works around an issue where tests are improperly being skipped due
to issues collecting coverage data using coverage.py when subprocess is
used.

Whenever the subprocess marker is seen during collection, we add the
same skipif marker that's used to mark tests as unskippable, which makes
pytest run the tests no matter what.

This ends up with tests being forced to run and the
`test.itr.unskippable` tag being set to `true`:

![image](https://github.com/DataDog/dd-trace-py/assets/136473744/e189cacf-dbf5-4070-80c7-9584afe7edcc)


## Checklist

- [x] Change(s) are motivated and described in the PR description
- [x] Testing strategy is described if automated tests are not included
in the PR
- [x] Risks are described (performance impact, potential for breakage,
maintainability)
- [x] Change is maintainable (easy to change, telemetry, documentation)
- [x] [Library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
are followed or label `changelog/no-changelog` is set
- [x] Documentation is included (in-code, generated user docs, [public
corp docs](https://github.com/DataDog/documentation/))
- [x] Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))
- [x] If this PR changes the public interface, I've notified
`@DataDog/apm-tees`.

## Reviewer Checklist

- [x] Title is accurate
- [x] All changes are related to the pull request's stated goal
- [x] Description motivates each change
- [x] Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- [x] Testing strategy adequately addresses listed risks
- [x] Change is maintainable (easy to change, telemetry, documentation)
- [x] Release note makes sense to a user of the library
- [x] Author has acknowledged and discussed the performance implications
of this PR as reported in the benchmarks PR comment
- [x] Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)
---
 tests/conftest.py | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/tests/conftest.py b/tests/conftest.py
index e2f04d18b21..9a553821deb 100644
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -257,6 +257,18 @@ def _subprocess_wrapper():
         return _subprocess_wrapper()
 
 
+@pytest.hookimpl(tryfirst=True)
+def pytest_collection_modifyitems(session, config, items):
+    """Don't let ITR skip tests that use the subprocess marker because coverage collection in subprocesses is broken"""
+    for item in items:
+        if item.get_closest_marker("subprocess"):
+            if item.get_closest_marker("skipif"):
+                # Respect any existing skipif marker because they preempt ITR's decision-making
+                continue
+            unskippable = pytest.mark.skipif(False, reason="datadog_itr_unskippable")
+            item.add_marker(unskippable)
+
+
 @pytest.hookimpl(tryfirst=True)
 def pytest_runtest_protocol(item):
     if item.get_closest_marker("skip"):