Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use poetry for bootstrap dependency mgmt #468

Closed
eloquence opened this issue Sep 28, 2023 · 3 comments · Fixed by #478
Closed

Use poetry for bootstrap dependency mgmt #468

eloquence opened this issue Sep 28, 2023 · 3 comments · Fixed by #478

Comments

@eloquence
Copy link
Member

Once #467 lands, we can migrate the bootstrap dependency management from pip-compile to poetry.

@eloquence
Copy link
Member Author

eloquence commented Dec 1, 2023

Took a first stab at this in e39d740. Our reqs are pretty outdated so I had to version-pin a couple more to avoid incompatibilities -- however nothing should be older than in the current bootstrap. I think a holistic refresh (#464) would be a good follow-up task.

Currently this is failing CI on Bookworm; as far as I can tell, it's attempting a source instead of wheel install for cython, and then rightly complaining that the sha256 for the wheel doesn't match the source tarball. Not sure why it's doing that, see https://app.circleci.com/jobs/github/freedomofpress/securedrop-builder/30477 for CI output. Will try to repro with Python 3.11 and if necessary with a Bookworm VM.

@eloquence
Copy link
Member Author

In hindsight it of course makes sense that Python 3.11 wants a different version of Cython. I've built one and added it in f7465d0, which make the bookworm jobs happy. Open Q: how much diff review do we want to apply to these core Python ecosystem dependencies? Will raise that in standup tomorrow.

Otherwise next up: making reprotest happy.

@eloquence
Copy link
Member Author

eloquence commented Dec 8, 2023

To recap, as part of migrating the bootstrap to Poetry, I tried to have Poetry update any bootstrap dependency that wouldn't cause immediate further breakage.

The reason reprotest-wheels is failing on f7465d0 is that the new bootstrap produces new wheels with different checksums. I suspect this is due to the Cython bump.

Additionally, the Cython bump won't work with securedrop-proxy currently due to freedomofpress/securedrop-client#1681.

I'm not sure anymore if it's a good idea to include any wheel updates in the initial Poetry migration.

I would suggest we migrate to Poetry for the bootstrap now by effectively pinning all versions in pyproject.toml to the versions used in the current bootstrap. This should enable us to finish this part of the Poetry migration, handling bootstrap updates separately.

Let me know if you have any concerns with that approach, if not I'll proceed along those lines on Monday.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant