Add dependency groups to pyproject.toml

[tobiasdiez]

Following PEP 735, add a few groups of dependencies that one usually needs while developing sage (e.g. for tests or docs).

This info will then be used to generate the conda lock files and once support for PEP 735 is more widespread other tools will be able to use the metadata as well. Eg, once pypa/pip#12963 is implemented one can install all test dependencies with pip install --groups test.

📝 Checklist

• The title is concise, informative, and self-explanatory.
• The description explains in detail what this PR is about.
• I have linked a relevant issue or discussion.
• I have created tests covering the changes.
• I have updated the documentation accordingly.

⌛ Dependencies

[dimpase]

#37482 - is a peculiar PR, just cherry-picking commits from here, a PR on which I can't even comment, and positively reviewed by the PR author himself...

[tobiasdiez]

#37482 - is a peculiar PR, just cherry-picking commits from here, a PR on which I can't even comment, and positively reviewed by the PR author himself...

It's not even a cherry-picked commit. Matthias copied a subset of the code changes of this PR, put's it into a different place that is more to his liking and slaps a "positive review" label on it. He did the same with #37452, and insisted that it's "positively review" even after I've removed that label (and explained in #36503 (comment) why the new location is not working for me).

@roed314 @williamstein sorry to bother you with this, but my understanding is that this is a clear misuse of the "positive review" label and, since done multiple times on purpose and not by accident, an abuse of his admin/triage rights. Speaking of abuse of admin rights, Matthias continues to flag my review comments as "resolved" - sometimes without even a response to the question (#37351 (review)). Normal users without admin rights cannot mark them as "unresolved" again, effectively hiding them from other reviewers. This problematic conduct has been mentioned to Matthias at multiple instances before (and if my memory serves me well even by members of the sage abuse team).

[dimpase]

@mkoeppe has blocked me on GH - while this is sorted out, we can proceed. And if it's not sorted out, we can block him, why not...

[roed314]

The Sage Code of Conduct Committee is setting this back to needs review for now based on it being part of @dimpase's reaction to our decision on #36181, #36561, #36676, #36951, #36964, #37351, #36999, and #36941. It can, of course, be granted positive review at a future date, and will be subject to the policy on disputed PRs as normal.

[dimpase]

is this still a disputed PR?

[mkoeppe]

This PR has obviously neither seen sufficient development, nor any meaningful review. It cannot be merged in this form.

It creates a pyproject.toml in the SAGE_ROOT, which I have objected to elsewhere.

• it declares a build-backend = "setuptools.build_meta", which obviously does not work -- there is nothing that backs it, neither a setup.cfg, setup.py, nor [tools.setuptools] information.
• it duplicates version information, violating our long-standing policy and practice of single-sourcing version information.

To summarize: -1, this is deliberate noise. please close.

[mkoeppe]

The part that declares external dependencies according to draft PEP 725 is valuable and interesting. I am hoping to use it with the skeleton generator for the modularized Sage packages in pyodide, #34539. I have split it out as

• src/pyproject.toml: Add 'external' section per draft PEP 725 (unbundled from #37446) #37482 -- where it is added in a pyproject.toml that has an actual function.

[dimpase]

we don't really practice "single source version information", it can't therefore "violate" it.

[github-actions]

Documentation preview for this PR (built with commit c7beb90; changes) is ready! 🎉
This preview will update shortly after each push to this PR.

[culler]

The primary objection to this PR is that is adds a pyproject.toml file in SAGE_ROOT. The explanation for why this was done is:

We add this information to a new pyproject.toml file in the root to be independent of the disputed PRs: ...

which seems to not have to do with the design, but rather with the PR review process.

There is some consensus that the ideas in this PR are useful. There is some evidence that they have not been fully worked out yet. And attempts to incorporate some of the ideas into a working PR without adding a pyproject.toml file in SAGE_ROOT have been summarily rejected and attacked on what seem to be personal grounds.

I think this is an unfinished proposal which might prove to be useful if people could manage to cooperate. But there seems to be little hope for that happening.

Because of this state of affairs, I vote -1. I don't see how merging this PR in the current context could possibly improve Sage.

[mkoeppe]

Thanks @culler for the clear analysis. Your vote brings us to:

• in favor: @tobiasdiez (author), @dimpase
• against: @mkoeppe, @culler

Setting back to "needs review".

[dimpase]

@orlitzky @tornaria - please have a look

[tobiasdiez]

The primary objection to this PR is that is adds a pyproject.toml file in SAGE_ROOT.

So you would prefer if I were to modify the pyproject.toml file in the src directory instead?

[tobiasdiez]

@kwankyu As the copyright holder, I'm objecting that my work is used in the form of #37482 and have hence removed the positive review label there. Instead, I propose to close #37482 until the discussion in this PR here came to an end.

If there are things that should be improved, I would like to do this in this PR. For example, I'm still waiting for a response to my earlier question #37446 (comment) to one of the reviewers here. Depending on the outcome, this may or may not make #37482 obsolete without creating additional friction for me.

[saraedum]

@kwankyu As the copyright holder, I'm objecting that my work is used in the form of #37482 and have hence removed the positive review label there. Instead, I propose to close #37482 until the discussion in this PR here came to an end.

I'm not a lawyer but I don't think that's how the GPL works. When you published your code here, you made it available under the GPL. You still hold the copyright but that does not give you the right to stop others to use your code within the terms of the GPL.

That said, I think it's obvious that everybody would like to see a consensus on the technical discussion here :)

[tobiasdiez]

You still hold the copyright but that does not give you the right to stop others to use your code within the terms of the GPL.

Sure, but the GPL requires that modifications and the original copyright holder are clearly marked in a prominent place. Perhaps, you would even need to state that sagemath is now a derived work of my fork - which is just absurd. But please not start a legal war on top of the already complicated enough situation.

That said, I think it's obvious that everybody would like to see a consensus on the technical discussion here :)

I agree. Maybe someone could start by explaining why creating a top-level file pyproject.toml is so controversial.
As a data point, the following projects have a pyproject file in their root only specifying metadata, ruff or other tools configs - and not being used for the installation of an actual python program. They are all reasonably popular (almost 300.000 stars combined), showing that such a metadata-pyproject file is not uncommon and normally doesn't lead to problems.

• https://github.com/TheAlgorithms/Python/blob/master/pyproject.toml
• https://github.com/nodejs/node/blob/main/pyproject.toml
• https://github.com/HypothesisWorks/hypothesis/blob/master/pyproject.toml
• https://github.com/vladmandic/automatic/blob/master/pyproject.toml

[roed314]

You still hold the copyright but that does not give you the right to stop others to use your code within the terms of the GPL.

Sure, but the GPL requires that modifications and the original copyright holder are clearly marked in a prominent place. Perhaps, you would even need to state that sagemath is now a derived work of my fork - which is just absurd. But please not start a legal war on top of the already complicated enough situation.

Here's what I can see about the current status of giving credit:

• Matthias marked you as the author on the single commit in src/pyproject.toml: Add 'external' section per draft PEP 725 (unbundled from #37446) #37482
• The text in the PR comment says "Author: @mkoeppe, based on @tobiasdiez's Add dependency groups to pyproject.toml #37446."
  What would you like to see in this regard?

I can't speak to the pyproject.toml question right now, though I hope others will be willing to.

[mkoeppe]

I can't speak to the pyproject.toml question right now, though I hope others will be willing to.

I have explained it in #37446 (comment)

[roed314]

I can't speak to the pyproject.toml question right now, though I hope others will be willing to.

I have explained it in #37446 (comment)

The comment you cite just says "It creates a pyproject.toml in the SAGE_ROOT, which I have objected to elsewhere," but does not include a pointer to where you gave reasons elsewhere.

[mkoeppe]

the bullet points below that sentence, no?

[roed314]

the bullet points below that sentence, no?

Ok, I was confused. Those looked like specific objections about the details of what went into the pyproject.toml, rather than objections to having it at all. For the version information that @tobiasdiez is including in pyproject.toml that you don't want to see duplicated, where is it stored now?

[mkoeppe]

This should be closed as a duplicate.

[tobiasdiez]

The scope of this PR is now largely reduced, only adding a few dependency groups at the end of the pyproject.toml file. I've thus removed the "disputed" flag. (I can also close this PR and open a new one if this is preferred)

[dimpase]

I'll be slow to respond in the coming 5-6 days or so, as we're moving across the pond tomorrow.