ci: Use uv pip for faster build #2038
Conversation
|
Thank you for mentioning uv, the news haven't reached me yet. I think this is quite exciting and I will start using uv today, but I strongly discourage its use for mesa just yet. The first public release is less than 24 hours ago. This potentially decreases our workflows by a few seconds, but I don't think that justifies the possible hurdles that are attached to bleeding edge software. |
|
I have yet to measure by how much this shaves off the build time, because the CI still fails. According to the blog post:
If it builds and runs the code just fine, I see no reason not to use it. Besides, Ruff is considered bleeding edge, yet many huge projects use it in their CI. Not all "bleeding edge" public-released tools are the same in terms of reliability. |
rht
force-pushed
the
uv
branch
2 times, most recently
from
b07818d
to
7df218e
Compare
|
I have overcome astral-sh/uv#1374. Waiting for astral-sh/uv#1355 to be fixed, which should happen really soon. |
Yes but there have already been 2 bugfix releases in the first 12 hours of its existence and you already linked two issues that you had to overcome/waiting for a fix. And these are just the obvious bugs, I am more worried about the subtle bugs, where it appears to be working, but some details are broken. This is inevitable for such large projects. We don't know if it will affect us, but why risk it? |
|
Didn't you want to know how much this will speed up the macOS and Windows build? My sense is that it will be a lot, and to have a CI duration that is < 1min. |
|
I agree with @Corvince. Let's give UV a few weeks to sort out the initial problems before we consider shifting. Also, what is the problem with a CI that takes a bit over a minute? |
That sounds like a good timeline for Ruff developers. Ruff used to have a daily (if not more frequent) release cycle. And during this high frequency releases, it already had adopters by huge projects. Faster CI does matter, to the point they have been willing to adopt an alpha-release linter.
|
rht
force-pushed
the
uv
branch
6 times, most recently
from
fe7a5f2
to
a863186
Compare
Agreed. Let's wait a few week until uv stabilizes a bit more. |
rht
force-pushed
the
uv
branch
6 times, most recently
from
385eb56
to
b13ac39
Compare
.github/workflows/build_lint.yml
Outdated
| - name: Install dependencies | ||
| # Only if the cache misses | ||
| # Based on https://github.com/pypa/pip/issues/8049#issuecomment-633845028 | ||
| # read_requirements.py should be removed once | ||
| # https://github.com/pypa/pip/issues/11440 is resolved. | ||
| if: steps.cache.outputs.cache-hit != 'true' | ||
| run: | | ||
| pip install toml | ||
| rm -fr ${Python_ROOT_DIR}/lib/python${{ matrix.python-version}}/site-packages/pip-23.0.1.dist-info |
There was a problem hiding this comment.
Why is this line needed? Seems like it would break easily when the pip version changes.
There was a problem hiding this comment.
uv breaks if there are multiple pip installed. There isn't a bug report for this yet, though if I were to open one, would be resolved within days because of the use case in GH Actions.
There was a problem hiding this comment.
Are you sure about this one? Specifically I don't see how why multiple pips should be installed at all and why uv should interact with them at all. Also I haven't seen this in other workflow files that already use uv
There was a problem hiding this comment.
That's because they use uv venv, e.g. astral-sh/uv#1386 (comment).
There was a problem hiding this comment.
I was thinking of waiting until the non-venv solution has stabilized, but uv venv is blazing fast, so it shouldn't matter to create the venv.
There was a problem hiding this comment.
With uv venv, pytest failed https://github.com/projectmesa/mesa/actions/runs/7980718647/job/21790983916?pr=2038 because Solara has a hardcoded assumption on the virtualenv information.
|
I think its funny that pip takes 2 seconds to install just uv and then uv continues to install the dependencies in a few milliseconds. Curious how it will perform with outdated caches. |
rht
force-pushed
the
uv
branch
2 times, most recently
from
737f545
to
912e38e
Compare
|
Everything builds except for Windows, which is a matter of using I should be able to revert my advanced (but complex) caching back to |
rht
force-pushed
the
uv
branch
2 times, most recently
from
e6ce476
to
3a9265d
Compare
rht
force-pushed
the
uv
branch
13 times, most recently
from
cef4928
to
6f76bf5
Compare
|
Tests all passed. |
|
It's now even more concise with |
|
Nice work, I like the reduced complexity of our CI configuration. I think we can try it. |
|
There has been ~100 PRs opened that contain |
Simplify build_lint.yml caching temp Use uv venv uv: Use --system flag ci: Remove -e from pip install
uv is a fast drop-in pip, pip-compile, virtualenv etc replacement created by the creator of Ruff.
uvis a fast drop-inpip,pip-compile,virtualenvetc replacement created by the creator of Ruff.