Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up update-python3-dependencies #7234

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from
Open

Speed up update-python3-dependencies #7234

wants to merge 1 commit into from

Commits on Sep 23, 2024

  1. Speed up update-python3-dependencies

    The `make update-python3-dependencies` step is slow for two main
    reasons: container image layering and pip-compile.
    
    Currently, each `pip-compile` step is run in a separate dev-shell, which
    means after each one, if anything changed in the requirements/ folder,
    it needs to rebuild the slim image and reinstall all the pip
    dependencies.
    
    Now I've moved the steps to a separate script that execute in a single
    dev-shell, so no image rebuilding happens during the updates, it'll just
    be needed once afterwards.
    
    For an additional boost, switch to the new uv tool, which reimplements
    pip-compile in a much faster way. The output is basically the same,
    except the sorting is smarter (e.g. pytest comes before pytest-cov) and
    package names are properly normalized. We can also drop the
    `--allow-unsafe` because uv is entirely independent of setuptools and
    pip-tools.
    
    uv is still quite new to the Python ecosystem, but this allows us to
    begin using it without any lock-in, it should be trivial to swap back to
    pip-tools if needed.
    
    Overall `make update-python3-dependencies` now takes seconds to run
    instead of minutes \o/
    
    Refs <freedomofpress/securedrop-tooling#16>.
    legoktm committed Sep 23, 2024
    Configuration menu
    Copy the full SHA
    2ec947c View commit details
    Browse the repository at this point in the history