uv exports incomplete requirements if using external packages

[Mateuscvieira]

This issue is linked to #7494 but doesn't have the same solution, or any solution I have found.

The problem

I can start my uv project with commands:

uv init
uv add "ceic-api-client<=2.7" --extra-index-url https://downloads.ceicdata.com/python

Then appending the extra index to the pyproject file like so:

[tool.uv]
extra-index-url = ["https://downloads.ceicdata.com/python"]

This creates an environment where I can freely add, remove, run, sync and lock. So, running uv export, I would expect a valid requirements file, but I get the following (using the no-hashes flag for legibility):

ceic-api-client==2.6.1.148
certifi==2024.8.30
python-dateutil==2.9.0.post0
six==1.16.0
urllib3==2.2.3

This, for obvious reasons, will not build.

Suggestions

Maybe this is unwarranted advice, but from what I understand from src/commands/project/export.rs, the export command reads from the lockfile, not pyproject. Well, as mentioned in #7494, the lockfile is in fact correct when it comes to extra indexes, so it wouldn't require adding full support for adding indexes in add to fix this issue with export (I think).

Versions

Running uv 0.4.11 in Windows 10

[zanieb]

Can you clarify what the expected behavior is? We should write the index URL into the exported file?

[Mateuscvieira]

Yes. The requirements file should have the --extra-index-url flag if one wishes to use pip install

[zanieb]

I don't think we should do this by default, but maybe we could include --emit-index-urls. I'm pretty hesitant, generally, because our index behavior doesn't match pip and I'm not sure we will be able to generate something that always works.

[Mateuscvieira]

Shouldn't export always export environments that work, though? As it stands, even using uv pip install, it's broken:

uv export > ../another_directory/requirements.txt
cd ../another_directory
uv venv .venv
.venv\Scripts\activate
uv pip install -r requirements.txt

Output:

  × No solution found when resolving dependencies:
  ╰─▶ Because ceic-api-client was not found in the package registry and you require ceic-api-client==2.6.1.148, we can conclude that your requirements are unsatisfiable.

The export help file currently specifies it as "Export the project's lockfile to an alternate format". IMO it's to be expected that the output should be as close a match of the lockfile as possible. The output I showed isn't, as it won't even generate a working environment using any form of index behavior, be it uv or pip, while the lockfile does.

[zanieb]

I don't think passing settings inside the requirements.txt file (which is not a standardized format, it's just a thing that pip does) is best practice and we will generally avoid it where we can. Additionally, the requirements.txt format doesn't allow specifying an index for specific packages, so even if we were to start emitting index URLs now, once we support #171 via #7481 we won't be able to generate a file that "works".

I understand this isn't ideal and we may be able to make it better somehow, but it's not a simple problem.

[Mateuscvieira]

I understand the concern. Well, keeping the behavior as is, this maybe warrants a warning of some kind, either when the engine detects a package only available in a non-default index or simply when there are extra indexes in the lockfile, so this behavior is more explicit.
Also, maybe we could make uv export --extra-index-url prepend the extra index into the requirements file, maybe with a warning about unsafe behavior. This makes the user explicitly opt in, and also uses a flag that is (apparently) currently not used.

[charliermarsh]

uv pip compile also does not export these by default; the behavior is intentional. Whether we want to support --emit-index-url is an open question though.

[Mateuscvieira]

uv pip compile also does not export these by default; the behavior is intentional

I understand, but both uv export and uv pip compile have --extra-index-url flags which don't actually add the extra index to the output, and neither of them provide feedback about using the extra indexes in resolution but not in output. As is, the functionality is pretty confusing for us using private packages