Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Q: how to automatically detect platform_tag? #29

Open
ilya-lavrenov opened this issue Jun 4, 2024 · 5 comments
Open

Q: how to automatically detect platform_tag? #29

ilya-lavrenov opened this issue Jun 4, 2024 · 5 comments

Comments

@ilya-lavrenov
Copy link

ilya-lavrenov commented Jun 4, 2024

Question 1:
Currently, when I built wheel on Linux, platform tag is something like linux_x86_64
How to ensure that it detect current glibc version and set platform tag in manylinix_x_y_arch format?

I don't want to use auditwheel for it, because it's an extra step and not user friendly.

Question 2:
how to specify that my project does not contain Python extension and ensure py3-none is used instead of cp310-cp310

Question 3:
How to install several license files / custom license to the folder dist-info?

@ilya-lavrenov
Copy link
Author

For questions 1 and 2 I managed to partially WA the issue with:

[tool.py-build-cmake.cross]
implementation = "py"
version = "3"
abi = "none"
arch = "manylinux_2_31_x86_64"
toolchain_file = "cmake/native.cmake"

with empty cmake/native.cmake.

But I still have to hardcode manylinux_2_31_x86_64 which can be different depending on glibc version of specific Linux OS

@tttapa
Copy link
Owner

tttapa commented Jun 4, 2024

Question 1

I believe there are three common scenarios here:

  1. You (or a user) are building packages from source locally. In this case, py-build-cmake simply uses the current interpreter's platform tag (as provided by distlib, e.g. linux_x86_64). This is okay for local packages, but these packages should never be uploaded to PyPI.
  2. You're building a package for distribution on PyPI. This should really only be done in a CI/CD workflow, where you use standard tools such as cibuildwheel, which will process your package using auditwheel for you. This is necessary to make sure you don't upload incompatible packages with broken dependencies. Since this only happens in a CI workflow, you don't have to worry about user friendliness.
  3. You're cross-compiling. Here, you should know the glibc version of the toolchain you're using, and you can indeed use the cross.arch tag for this purpose. (In fact, you cannot even use auditwheel in this case.) If you use an appropriate toolchain and you handle dependencies carefully, these packages can also be uploaded to PyPI.

I'm open to adding an arch option to native builds, but IMHO, py-build-cmake should not try to deduce the glibc version automatically, because A. this is unnecessary for local builds, and B. it may result in people uploading these local wheels to PyPI by accident.

Question 2

Using the py3-none tags is not yet supported by the stable release. In the current pre-release I'm working on, you can set python_abi = 'none' in your [tool.py-build-cmake.cmake] config, see https://github.com/tttapa/py-build-cmake/blob/rework-0.2.0/docs/Config.md.

Question 3

You should be able to simply install them using an install(FILES ...) command in your CMakeLists.txt. For instance:

install(FILES LICENSE.custom
        EXCLUDE_FROM_ALL
        COMPONENT python_modules
        DESTINATION "${PY_BUILD_CMAKE_PACKAGE_NAME}-${PY_BUILD_CMAKE_PACKAGE_VERSION}.dist-info")

@ilya-lavrenov
Copy link
Author

ilya-lavrenov commented Jun 5, 2024

Hi @tttapa

Thank you for prompt response and for the project itself. I really find it very useful!

I'm open to adding an arch option to native builds, but IMHO, py-build-cmake should not try to deduce the glibc version automatically, because A. this is unnecessary for local builds, and B. it may result in people uploading these local wheels to PyPI by accident.

I'm open to adding an arch option to native builds, but IMHO, py-build-cmake should not try to deduce the glibc version automatically, because A. this is unnecessary for local builds, and B. it may result in people uploading these local wheels to PyPI by accident.

We, actually, don't differentiate these cases and want users to reproduce exactly the same results as CI service.
Are you OK to keep arch = 'none' by default to keep auditwheel as default path, but for users who knows what they are doing, add arch = 'auto' ? Auto can guess both architecture and glibc (or musl) version to automatically set arch (reference https://github.com/openvinotoolkit/openvino/blob/master/src/bindings/python/wheel/CMakeLists.txt#L17-L82)

python_abi = 'none'

I've tried and it works, thank you! Do you know some hints when can users leverage these new features?
BTW, I've tried to use your changes from branch:

[build-system]
requires = ["git+https://github.com/tttapa/[email protected]", "pybind11", "pybind11-stubgen", "cmake>=3.27"]
build-backend = "py_build_cmake.build"

and it looks like such syntax is not supported by frontends unfortunately.

@tttapa
Copy link
Owner

tttapa commented Jun 5, 2024

I'll have a more detailed look at this later this week, I'll need to see how other backends such as setuptools handle platform/architecture tags.

To answer your second question, I believe that the correct syntax would be:

[build-system]
requires = [
    "py-build-cmake@git+https://github.com/tttapa/py-build-cmake@sha1-hash",
]
build-backend = "py_build_cmake.build"

Since the development branches are by definition unstable, I'd strongly recommend referencing a specific commit rather than a branch name.

Case in point, I've created a new tools.py-build-cmake.wheel section, and moved the python_abi option to it, in preparation to maybe adding options like arch to it in the future: dc21bfe

@ilya-lavrenov
Copy link
Author

ilya-lavrenov commented Jun 6, 2024

Hi @tttapa

Maybe the better idea is to allow to call auditwheel during py-build-cmake build process? So, it will automatically fix wheel to reflect actual platform.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants