Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for app/library type to poetry new and poetry init #9668

Open
edmorley opened this issue Sep 3, 2024 · 3 comments
Open

Add support for app/library type to poetry new and poetry init #9668

edmorley opened this issue Sep 3, 2024 · 3 comments
Labels
kind/feature Feature requests/implementations status/triage This issue needs to be triaged

Comments

@edmorley
Copy link

edmorley commented Sep 3, 2024

Issue Kind

Brand new capability

Description

Currently if I use poetry new or poetry init, I end up with a pyproject.toml configuration that is more oriented towards a library than an app, with no way to tell the new / init commands otherwise (beyond --python, but that's only one of several things that would need overriding).

For example they generate:

[tool.poetry]
name = "example"
version = "0.1.0"
description = ""
authors = ["..."]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.12"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

...whereas for an app:

  • it's rare to want to publish the app (ie: I want package-mode = false from Introduce non-package-mode #8650, and don't want the name/version/description/authors/readme fields)
  • it's rare to need to build the app (ie: I don't need the [build-system])
  • it's safer to set a specific major Python version (eg 3.12.*) rather than allowing an unsafe range (such as >=3.12) - so prod vs staging vs each developer's machine is at least using the same major Python version.

And therefore for apps, a config like the following is more appropriate:

[tool.poetry]
package-mode = false

[tool.poetry.dependencies]
python = "3.12.*"

As such, it would be helpful if the poetry new and poetry init commands accepted something like a --type argument (and corresponding question prompt when using those commands in interactive mode), that accepted options like "app" or "library", that then generated config more appropriate for the specified use-case. The interactive prompts and/or existing CLI args would still allow users to mix and match if needed (for example, the user could select "app" mode, but when prompted for the python value, could override the app mode's suggested default of "3.N.*" to a wider range if they prefer.

Such a type option would be similar to:

Possible arg/option names:

  • --type {app,lib} (or --type {app,library}
  • --app / --lib (or --app / --library)

Note: I intentionally didn't include --package-mode {true,false} in the list, since this feature request is about more than just package-mode = false and about several other defaults that make less sense when using Poetry with an app.

Impact

  • Means users can more easily generate a config that contains best practices for app use-cases, rather than them having to discover additional config/best practices via the docs and change it retrospectively
  • Likely means fewer support tickets about things like "why is Poetry saying I have to have a README" etc, since not all users read the docs properly
  • Encourages projects which are apps to not use unsafe unbounded Python version ranges (which can cause breakage in CI/CD environments when new Python versions are released, or cause user confusion when production works differently from locally when they happen to be using different Python versions that are permitted by the wide range)

Workarounds

Users either:

  • avoid using poetry new / poetry init at all, and create their pyproject.toml from scratch using the docs
  • use poetry new / poetry init and fill out questions that aren't relevant, but then have to change the defaults afterwards

...but both of these rely upon the user knowing that the default configs generated by new/init are not ideal for app use-cases, and what they should change to make them more suitable.

@edmorley edmorley added kind/feature Feature requests/implementations status/triage This issue needs to be triaged labels Sep 3, 2024
@edmorley
Copy link
Author

edmorley commented Sep 3, 2024

Encourages projects which are apps to not use unsafe unbounded Python version ranges (which can cause breakage in CI/CD environments when new Python versions are released, or cause user confusion when production works differently from locally when they happen to be using different Python versions that are permitted by the wide range)

For more on the issues this causes, see:
heroku/buildpacks-python#260

@radoering
Copy link
Member

radoering commented Sep 3, 2024

In my opinion this distinction between app and lib is flawed for Python projects. (There are many apps that are packaged, e.g. Poetry itself, pipx, pre-commit, ...)

Setting package mode as in #9622 and maybe doing more based on this makes more sense to me, because:

it's safer to set a specific major Python version (eg 3.12.*) rather than allowing an unsafe range (such as >=3.12) - so prod vs staging vs each developer's machine is at least using the same major Python version.

I have not thought about that but we could consider to do this when creating a project in non-package mode, too.

@edmorley
Copy link
Author

edmorley commented Sep 17, 2024

Thank you for the reply! :-)

In my opinion this distinction between app and lib is flawed for Python projects. (There are many apps that are packaged, e.g. Poetry itself, pipx, pre-commit, ...)

Yeah I definitely agree that there are types of projects where it's a grey area between "app" and "library" (eg CLI tools, Poetry itself etc).

Naming is hard and all that...but by "app" I was instead meaning ~"end-user" projects that are typically not distributed for the world to use, but are used directly by the person or team maintaining them for a very specific purpose. Think Django webapps that people are deploying to production, or data analysis projects shared by a team of data scientists etc, rather than an open source project that's shared for consumption by others.

I would argue these types of "app" projects:

  1. Are much more common (by quantity of projects) than libraries or the "in-between an app and a library" type tools like CLIs/Poetry/pipx that you mention.
  2. Typically will only ever be run/tested using a single version of Python.

Given (1), I think it would be really beneficial for Poetry to support these types of projects better out of the box. (Plus, I would argue that the authors of libraries or of tools like pipx are much more capable of overriding defaults than most end users, so it makes sense to pick the defaults for the more-common and more-beginner-heavy app project type instead.)

The current default tool.poetry.dependencies.python value of e.g. >=3.10 is counter-productive for these projects, since they typically aren't tested against several Python versions in a CI matrix. So using anything other than the same version locally/across different developer machines/in production etc is asking for trouble.

I would also posit a big reason these types of projects use a package manager like Poetry (instead of using pip and requirements files), is because they want the determinism of a lockfile. However, without also ~"locking" the Python version to at least a major version (eg ==3.12.*) then they can still encounter the classic "works on your machine but not mine / in production".

As-is, all of the options for us (Heroku) adding support for reading the Python version from tool.poetry.dependencies.python aren't great - unless Poetry were to start encouraging "app" type projects to use safer/more-app-appropriate version ranges (such as ==3.X.*) by default. For more on why, see:
heroku/buildpacks-python#260

In contrast, uv's behaviour is already a lot more suitable out of the box for these app type projects: The uv init command creates .python-version file at the same time as populating pyproject.toml, and whenever any uv command is run validates that the .python-version file and requires-python fields are still compatible. There are still some improvements I'd like to see (xref astral-sh/uv#7429 (comment)), but right now uv is ahead in this area for ease of app deployment, and it would be great to close the gap.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature Feature requests/implementations status/triage This issue needs to be triaged
Projects
None yet
Development

No branches or pull requests

2 participants