Replace bootstrap-conda by grayskull

[tobiasdiez]

Using the metadata added in #37446, we automatically generate the conda environment files. This no longer makes any reference to the conda.txt files contained in sage-the-distribution. Thus sage-on-top-of-conda is now completely independent of sage-the-distribution (only relying on information specified by sage-the-library). In particular, after this PR is merged the conda.txt files could be deleted from sage-the-distribution.

In particular, we no longer need to maintain the mapping of pypi packages to conda packages but instead can rely on the offical mappings maintained by the conda team (https://github.com/regro/cf-graph-countyfair/tree/master/mappings/pypi).

To test:

pip install https://github.com/conda/grayskull
python tools/update-conda.py

As a byproduct, this fixes #34626.

📝 Checklist

• The title is concise, informative, and self-explanatory.
• The description explains in detail what this PR is about.
• I have linked a relevant issue or discussion.
• I have created tests covering the changes.
• I have updated the documentation accordingly.

⌛ Dependencies

[dimpase]

I've removed disputed as it was added without a word of explanation. At least if you start a relabelling war, express what exactly you dispute.

[dimpase]

do you mean pip install git+https://github.com/conda/grayskull ?

Anyhow, after I do the latter, I get

$ python tools/update-conda.py .
Conda environment file written to src/environment-3.9.yml
Updating lock file for src/environment-3.9.yml at src/environment-3.9-linux
Traceback (most recent call last):
  File "/mnt/opt/Sage/sage-clang/tools/update-conda.py", line 150, in <module>
    update_conda(Path(options.sourcedir))
  File "/mnt/opt/Sage/sage-clang/tools/update-conda.py", line 75, in update_conda
    subprocess.run(
  File "/usr/lib/python3.11/subprocess.py", line 548, in run
    with Popen(*popenargs, **kwargs) as process:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/subprocess.py", line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.11/subprocess.py", line 1950, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'conda-lock'

What do I do wrong? Does it need to be run in a conda env?

[tobiasdiez]

What do I do wrong? Does it need to be run in a conda env?

Sorry, forgot that the script uses conda-lock as well. Please try again after pip install conda-lock. Later today, I'll add a bit of documentation to the script.

[dimpase]

Why do I see on an amd64 box

Locking dependencies for ['linux-aarch64']...

while running python tools/update-conda.py . ? Looks like an overkill, no?

[saraedum]

I took the liberty to add the "needs work" label since it's unclear what's disputed here. (Also, you could maybe check or remove the checklist at the top of the PR to make it clear that there's nothing from that checklist missing here.)

[mkoeppe]

@saraedum @roed314 @jhpalmieri I set "disputed" here because when the PR was opened, I was not able to comment because Tobias had blocked me without explanation or giving notice to the sage-abuse committee. It appears that now I can comment, so I am removing "disputed".

There has not been any meaningful review yet here, so conducting a vote would seem premature.

[mkoeppe]

It is correct that on my own PRs, @dimpase is blocked because of his persistent and aggressive code of conduct violations, including the type of denunciations that he has just written above.

[mkoeppe]

In any case, this PR here is based on #37446, which is in disputed status because of #37446 (comment)

[dimpase]

I am giving it a positive review; I tried it out, I pointed a problem, the problem got fixed - despite a rather abusive "There has not been any meaningful review yet here, so conducting a vote would seem premature" comment.

[github-actions]

Documentation preview for this PR (built with commit 55df9a9; changes) is ready! 🎉

[saraedum]

I understand that there are reservations about having the pyproject.toml in the root directory. Otherwise, this PR does seem like a reasonable idea to me (replace a shell script with Python and relay some of the heavy lifting to an external project.)

Is there a better place for the pyproject.toml that would make this work for everybody?

[saraedum]

I think it would be good to add an explanation at the top that explains what this script does. Or maybe add a --help to the argument parser that prints such documentation.

[saraedum]

Not super important but if we put all the global bits of this script in a def main() and do the usual if __name__ == "__main__" or similar, then this could be imported by others to use bits of the machinery for other purposes.

[mkoeppe]

Creating a new top-level tools/ directory should probably go through discussion in sage-devel.

And actually, there's no need for a new top-level directory: We already have a place for scripts of exactly this kind -- it is SAGE_ROOT/build/bin/.
In other words, the tools/update-conda.py script should just become part of the sage-bootstrap distribution package.

[saraedum]

Creating a new top-level tools/ directory should probably go through discussion in sage-devel.

I agree, that script should probably just go elsewhere. But otherwise, if we somehow move the pyproject.toml file, I don't see a problem here.

[mkoeppe]

otherwise, if we somehow move the pyproject.toml file, I don't see a problem here.

Same here. This could just be done on top of #37482.

[dimpase]

What's happening here?

[tobiasdiez]

What's happening here?

It still needs a bit of refinement, but is mostly working and I've used it to create #38878. I'll continue here after #38728 is merged.

In the meantime, it would be helpful if you (or someone else) could review #37446