Add GraalPy to manylinux images

[timfel]

This makes the open source GraalPy implementation available in manylinux images.

[timfel]

This was requested when adding GraalPy to cibuildwheels: pypa/cibuildwheel#1538 (comment)

[auvipy]

thanks lets wait for knowing the consensus

[mayeut]

This seems like a huge bump in size for a single version (around 200MB compressed so that's roughly between +25% and +50% increase for compressed image size depending on images). This is similar uncompressed (+30% on manylinux2014_aarch64).
Unless there's a similar huge demand for GraalPy, merging this would add a penalty on every other user.
IMHO, a starting middle ground would be to add a helper script in the image to install GraalPy properly (i.e. as it would be installed with this PR).

[timfel]

This seems like a huge bump in size for a single version (around 200MB compressed so that's roughly between +25% and +50% increase for compressed image size depending on images). This is similar uncompressed (+30% on manylinux2014_aarch64). Unless there's a similar huge demand for GraalPy, merging this would add a penalty on every other user. IMHO, a starting middle ground would be to add a helper script in the image to install GraalPy properly (i.e. as it would be installed with this PR).

The manylinux build scripts are not left in the image. Are you suggesting I change it so we leave them, and I add a script to run the commands that the image build runs? If we're not using the same build scripts, I'm worried we'll end up diverging with how we set up GraalPy from how we set up CPython.

Below are the steps that currently run during the build that are relevant for graalpy. Both the install and finalize scripts source helper scripts that are in /build_scripts; should I change the build to not delete those build scripts so we can execute them at runtime?

 RUN manylinux-entrypoint /build_scripts/install-graalpy.sh 3.10 graal 23.0.0 graalpython
 RUN manylinux-entrypoint /build_scripts/finalize.sh && rm -rf /build_scripts
 /opt/python/graalpy310-graalpy230_310_native/bin/python /tests/manylinux-check.py manylinux_2_28 x86_64

[timfel]

Hi @mayeut - I've updated the PR. I tried for a bit to bend the system so that I could create something in the image at runtime that is equivalent to what the build process produces currently, but due to the multi-stage nature of the current process it was a bit messy and fragile.

I settled instead for another approach: at the end of the build process in the finalize script, I delete most of the graalpy files except for the site-packages and launchers that were created in the build process (so the installed requirements stay in the image). I leave a script /usr/local/bin/install-graalpy${PY_VER} in the image that was generated at build time to ensure the exact same version (same sha256) of graalpy is later re-installed into the image. This way, the only footprint overhead is from the packages and stdlib that are installed for all python's the same way, so they weigh in about the same.

[timfel]

The CI failure seems unrelated to my PR, it is in a job that doesn't install graalpy

Didn't appear again after the rebase.

[timfel]

Friendly ping :) Anything I can help with or clarify?

[timfel]

The failure is again unrelated to this PR (it's on s390x, this failed before on this PR and then just passed after rebasing and rerunning)

[mayeut]

thanks for the follow-up @timfel.
I finally had a bit of time so took another approach in #1518

[mayeut]

superseded by #1520