6.0: pytest is failing

[kloczek]

I'm trying to package your module as an rpm package. So I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

• python3 -sBm build -w --no-isolation
• because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
• install .whl file in </install/prefix>
• run pytest with PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>

I cannot find what provides data_filename and canonical_filename fixtures and I'm aasuming that I still have some modules missing in my build env.
May I ask for some hints about what I need to install?

[nitzmahone]

The problem you're running into is that there are no pytest unit tests in pyyaml- it uses its own bespoke test runner. See #588 for an attempt to wrap pytest's legacy test fixture in something that pytest can consume- some form of that will likely get merged for whatever our next release is. To get the tests to run today, just change your last step to run tests/lib/test_all.py with PYTHONPATH including the wheel's installed location. You might also want to have a look at https://github.com/yaml/pyyaml/blob/master/.github/workflows/ci.yaml to see the envvars that will force Cython and the extension build to run or fail under PEP517, since various frontends still don't agree on how to consume config data passed through the build command line, and without them, it'll be "best effort" and silently fall back to no Cython or extension build on any errors.

Since this is the current state of the world and fixes are already in-flight, closing this issue.

[kloczek]

The problem you're running into is that there are no pytest unit tests in pyyaml- it uses its own bespoke test runner.

Did you think about start using pytest? 🤔
Let me share of what I'm doing .. I'm working on Linux/Solaris distribution in which I'm using rpm spec files to build rpm packages on Linux and IPS packages on Solaris. On area of python modules I have so far 952 python modules. Each package in my distro needs to have executed test suite during build. That is obligatory requirement. I found that ~8/9 of all python modules can be used with pytest. Recently I've managed to push that further because pytest-dev/pytest#9976.
Result is for now:

[tkloczko@devel-g2v SPECS]$ grep ^%pytest python-* | wc -l; grep ^%tox python-* | wc -l; grep "package has no test suite" python-* | wc -l; ls -1 python-* | wc -l
910
1
16
952

My goal i to have 100% python modules tested using pytest. Why? two reasons:

• pytest result does not need to be perfect however however this will create kind of baseline for regression tests. If lets say new version flit-core or pytest will be released I'm going to perform kind of test builds to not produce new packages but to asses impact of that exact change on whole packages population
• using everywhere %pytest macro will allow me soon to perform additional set of test builds in which to explicit listed BuildRequires in each spec file will be added some pytest extension like black, flake8, cov and probably many more (randomly for example). Which would allow to create additional set of per package metrics showing compliance exact package with those additional tests.

Probably in longer term I would be able to provide above as CI to allow python modules developers asses exact impact in the python module code to show risk of introducing exact change on other modules which would be using future version of exact modulable.
At the moment I'm quire close to implant second point of that plan. To deliver results to python modules developer result and make those results public only one condition needs to be fulfilled: it needs to be possible to use pytest .. somehow.

No other approach can provide env to make above happen .. this is why I've choose pytest 😋
Maybe it is time to stop doing things-my-own-way and reuse power of the pytest? 😃

[nitzmahone]

Did you read my previous comment?

[kloczek]

Did you read my previous comment?

I see only one comment.

BTW part

since various frontends still don't agree on how to consume config data passed through the build command line, and without them, it'll be "best effort" and silently fall back to no Cython or extension build on any errors.

Passing configuration params is working in case of pep517.
Example from my python-zstd.spec

%build
%pyproject_wheel -- \
        -C=--global-option=--external \
        -C=--global-option=--legacy \
        %{nil}

[nitzmahone]

See #588 for an attempt to wrap pytest's [I meant pyyaml's] legacy test fixture in something that pytest can consume- some form of that will likely get merged for whatever our next release is

[kloczek]

OK, I've los that part 😃
Thank you 👍