No line number colours after running tests with pytest

[rzhao271]

Testing #24156

VS Code: latest Insiders on Windows
Virtual environment: conda with Python 3.11.9

I don't see red or green on the line numbers indicating which branches were covered when running test_unit_and_pytest_combo.py. I am using a Python profile, but the same issue happens on my default profile.

[eleanorjboyd]

in your test results view do you see a place to view test coverage?

[rzhao271]

Yes

Coverage is enabled, adding branch coverage as an argument.
Running pytest with args: ['-p', 'vscode_pytest', '--rootdir=c:\\Users\\raymondzhao\\work\\endgame\\inc_dec_example_repo\\inc_dec_example_test_suite', '--cov=.', '--cov-branch', 'c:\\Users\\raymondzhao\\work\\endgame\\inc_dec_example_repo\\inc_dec_example_test_suite\\test_unit_and_pytest_combo.py::test_class_unittest_combo_file::test_combo1_function_unittest']
============================= test session starts =============================
platform win32 -- Python 3.11.9, pytest-8.3.3, pluggy-1.5.0
rootdir: c:\Users\raymondzhao\work\endgame\inc_dec_example_repo\inc_dec_example_test_suite
plugins: cov-5.0.0
collected 1 item

test_unit_and_pytest_combo.py .                                          [100%]

---------- coverage: platform win32, python 3.11.9-final-0 -----------
Name                              Stmts   Miss Branch BrPart  Cover
-------------------------------------------------------------------
diff_pattern_test.py                  8      8      0      0     0%
first_test.py                         6      6      2      0     0%
inc_dec.py                            7      3      2      1    56%
test_logging.py                      25     25      0      0     0%
test_many_functions.py                7      7      0      0     0%
test_parameterized.py                 7      7      4      0     0%
test_single_function_failure.py       3      3      0      0     0%
test_subtests.py                      6      6      4      0     0%
test_two_classes.py                  22     22      2      0     0%
test_unit_and_pytest_combo.py        16      7      4      1    50%
test_wait_parameterized.py            6      6      2      0     0%
-------------------------------------------------------------------
TOTAL                               113    100     20      2    11%


============================== 1 passed in 0.12s ==============================
Finished running tests!

[eleanorjboyd]

ok it seems like on windows right now it is not working- still investigating this

[alanwilter]

Sorry if this sounds like I’m hijacking the conversation, but I beg to differ. The latest VS release mentions test coverage for Python:

https://code.visualstudio.com/updates/v1_94#_python

I assumed this was for the Production release, not just the Insider build. However, I don’t see it working.

Also, this page:

https://code.visualstudio.com/docs/python/testing#_run-tests-with-coverage

is missing an image.

Clearly, something is broken or, at the very least, poorly documented.

I see:

However, when I run it, I don't see the report coverage you mentioned @eleanorjboyd

Needless to say that pytest command in my terminal works all fine.
Since I use pyproject.toml, I'm wondering if this has anything to do with the issues I see.

The relevant part:

[tool.pytest.ini_options]
addopts = "tests --cov=process_dcm/ --cov-report=term-missing:skip-covered --cov-report=xml --dist=loadgroup -n 8 --durations=5"

[tool.coverage.report]
omit = ["__main__.py"]
exclude_lines = ["no cov", "if __name__ == .__main__.:", "if TYPE_CHECKING:"]

[eleanorjboyd]

Hi! This is for the production release- our docs do not mention it (we are adding it as it was an oversight) but you need to be on the rewrite to use it- maybe that is the problem?

To switch to the rewrite if you aren't in the experiment, you can do so adding this setting to your users settings.json "python.experiments.optInto": ["pythonTestAdapter"]. Then you can confirm its enabled by finding Experiment 'pythonTestAdapter' is active in your python logs. If you are unable to see the log for the experiment change your log level to trace and it should show up. To set to trace use theDeveloper: set log level command in the command palette. Thanks

[alanwilter]

Thanks @eleanorjboyd so I got it to work. Using vscode 1.94.2.

However it still have problems, serious problems.

The coverage report to start with, is wrong:

On terminal is the result when I run my pytest.

Second, this issue is back #23223 please, reopen it.

[alanwilter]

Ok, by fine-tuning "python.testing.pytestArgs" with "--cov=process_dcm/" I got the correct coverage results.

But then, the debugger stop working properly by not stopping at the breakpoints.

I think we need a different property for coverage, say "python.testing.pytestArgsWithCoverage", which could be used for coverage and other stuffs (like running tests in parallel). It's very slow BTW, running with coverage.

Perhaps I could create other Run and Debug profiles but I don't know how to do this, if this is really possible.

So, the way I work, I need vscode to help me debug so I have to keep "python.testing.pytestArgs" with just:

  "python.testing.pytestArgs": [
    "tests",
    "-v",
    "-c",
    "tests/pytest.ini" # empty dummy file
  ],

And when I want coverage etc, I use command line pytest, which reads my pyproject.toml:

[tool.pytest.ini_options]
addopts = "tests --cov=process_dcm/ --cov-report=term-missing:skip-covered --cov-report=xml --dist=loadgroup -n 8 --durations=5"

[tool.coverage.report]
omit = ["__main__.py"]
exclude_lines = ["no cov", "if __name__ == .__main__.:", "if TYPE_CHECKING:"]

Plus Coverage Gutters extension to visualise the coverage. I'm going back to my usual workflow. This vscode coverage is indeed very experimental.

[eleanorjboyd]

Hi! Sorry about the delay and glad you got it working- yes that makes sense that you need to coverage arg, wondering if you could try with a .coveragerc file instead of using the argument tho.

secondly yes we really want to have ways to specify args for different types of runs. We are working on it but it is a bit larger so is taking more time- you can follow along here: #23907. Could be useful to have a setting for coverage args in the meantime... let me give that some thought.