Error in uv pip install when using a new build backend to build old wheels

[zmeir]

Hi,

First of all - thanks for this amazing tool! I've only been using it for a few weeks and it's already making my workflow so much faster and simpler!

I ran into this error trying to install some old version on numpy with uv pip install:
ModuleNotFoundError: No module named 'distutils.msvccompiler'

Full example:

$ uv --version
uv 0.4.15 (0d81bfbc6 2024-09-21)

$ uv pip install numpy==1.17.3
Resolved 1 package in 859ms
error: Failed to prepare distributions
  Caused by: Failed to fetch wheel: numpy==1.17.3
  Caused by: Build backend failed to build wheel through `build_wheel()` with exit status: 1
--- stdout:

--- stderr:
Running from numpy source directory.
Traceback (most recent call last):
  File "<string>", line 11, in <module>
  File "/Users/zmeir/Library/Caches/uv/builds-v0/.tmpT94fNH/lib/python3.10/site-packages/setuptools/build_meta.py", line 421, in build_wheel
    return self._build_with_temp_dir(
  File "/Users/zmeir/Library/Caches/uv/builds-v0/.tmpT94fNH/lib/python3.10/site-packages/setuptools/build_meta.py", line 403, in _build_with_temp_dir
    self.run_setup()
  File "/Users/zmeir/Library/Caches/uv/builds-v0/.tmpT94fNH/lib/python3.10/site-packages/setuptools/build_meta.py", line 503, in run_setup
    super().run_setup(setup_script=setup_script)
  File "/Users/zmeir/Library/Caches/uv/builds-v0/.tmpT94fNH/lib/python3.10/site-packages/setuptools/build_meta.py", line 318, in run_setup
    exec(code, locals())
  File "<string>", line 443, in <module>
  File "<string>", line 422, in setup_package
  File "/Users/zmeir/Library/Caches/uv/sdists-v4/pypi/numpy/1.17.3/qwWxgxjDba_GKoTWMyqpQ/numpy-1.17.3.zip/numpy/distutils/core.py", line 26, in <module>
    from numpy.distutils.command import config, config_compiler, \
  File "/Users/zmeir/Library/Caches/uv/sdists-v4/pypi/numpy/1.17.3/qwWxgxjDba_GKoTWMyqpQ/numpy-1.17.3.zip/numpy/distutils/command/config.py", line 20, in <module>
    from numpy.distutils.mingw32ccompiler import generate_manifest
  File "/Users/zmeir/Library/Caches/uv/sdists-v4/pypi/numpy/1.17.3/qwWxgxjDba_GKoTWMyqpQ/numpy-1.17.3.zip/numpy/distutils/mingw32ccompiler.py", line 34, in <module>
    from distutils.msvccompiler import get_build_version as get_build_msvc_version
ModuleNotFoundError: No module named 'distutils.msvccompiler'
---

I believe this is related to these issues:

• v0.4.14 seems to have resolver issues #7606
• Some outdated packages still execute on "from distutils.msvccompiler import MSVCCompiler, get_build_architecture" pypa/setuptools#4620

However, I don't know how to resolve this.

To elaborate, my full use-case is that I'm building a wheel for my project and I want to test it with the lowest and highest possible dependency versions.
In order to test the lowest dependency versions, I set up my test env with uv sync --resolution lowest-direct.
Since one of my dependencies has a transient dependency on numpy==1.17.3 in its lowest version range, I am faced with this error. I can add the latest numpy version to my project dependencies which would fix this error, but that will be too constricting for my package, since there is nothing preventing my code from working with lower numpy versions, it's just that's I'm unable to build them properly...

Would very much appreciate your assistance.

Thanks.

[konstin]

Hi!

I think the direct problem you are facing is that numpy 1.17.3 doesn't work with Python 3.10 (it has wheels for 3.5 to 3.8); on this Python version, you likely need to use are higher numpy version (or conversely, test on an older Python version).

I'm surprised that lowest-direct doesn't select a higher numpy, is one of your dependencies putting an upper bound on numpy? One option you could try is adding a constraint such numpy>=x.y.z; python_version >= "3.9" instead of a dependency, this way you're forcing a higher version that is compatible with Python 3.10 without constraining your dependents.

[zmeir]

I think the direct problem you are facing is that numpy 1.17.3 doesn't work with Python 3.10

Yep, that was it, thanks!

I'm surprised that lowest-direct doesn't select a higher numpy, is one of your dependencies putting an upper bound on numpy?

In my haste to submit this issue I may not have been accurate enough. Here is a more detailed representation of my issue:

1. My project depends on a package foo directly (real name is irrelevant since this is a package from an internal repository).
2. The package foo requires numpy<2.0.0 directly, but doesn't specify any lower version bound.
3. In my test environment, I want to check that a user that meets the very lowest dependency requirements could still use my package, so I run uv sync --resolution lowest-direct.
4. However, this causes the resolver to bring the latest version of numpy that satisfies numpy<2, which doesn't help me check a wide range of versions.
5. I could change my setup process to use --resolution lowest, but foo requires more packages than just numpy with unbound version constraints, so I add some select number of packages to my project's dependencies, for which I want to check the lower bound. Specifically: numpy>1.
6. Now when I run uv sync --resolution lowest-direct I expect to get the lowest possible version of numpy available for my python version (3.10) that satisfies numpy>1, but instead I get the aforementioned error.

If I understand correctly, the problem stems from the fact that these older versions of numpy did not have any lower bound on the required python version, and so the resolver chose them as possible candidates, but then failed to build the setup.py because, well, they are incompatible.
So that's not not an issue with a resolver but with the specification of the package being built. Still, I wonder if maybe at this point the resolver could "infer" that this version is not compatible and try the next possible version?

[konstin]

This a common problem, one that's about how the pypi-ecosystem handles metadata: When numpy 1.17.3 was released, the devs didn't know yet with which future Python version it may or may not be compatible, they could only say they know they need at least a certain version. Python 3.x makes only careful changes in each release, so a lot of packages that used to work on Python 3.x keep working on 3.(x+n). Since we projects append metadata for existing releases to specify the real compatibility after the original release, the most reasonable thing for us is to assume a packages works with all future python versions. This generally works, but sometimes we get those build failures.