released from pypi do not have PPC architecture included

[ashwinik30]

Facing issue while running tests on anisble-lint https://github.com/ansible/ansible-lint repo
FATAL: For testing, we require pyyaml to be installed with its native extension, missing it would make testing 3x slower and risk missing essential bugs.

Required :
compile pyyaml for PPC architecture with libyaml

output:
[root@ibm-p9z-18-lp4 pyyaml]# python3 setup.py --with-libyaml install
running install
running bdist_egg
running egg_info
writing lib/PyYAML.egg-info/PKG-INFO
writing dependency_links to lib/PyYAML.egg-info/dependency_links.txt
writing top-level names to lib/PyYAML.egg-info/top_level.txt
reading manifest file 'lib/PyYAML.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README'
writing manifest file 'lib/PyYAML.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-ppc64le/egg
running install_lib
running build_py
running build_ext
building 'yaml._yaml' extension
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mcpu=power8 -mtune=power8 -funwind-tables -fstack-clash-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mcpu=power8 -mtune=power8 -funwind-tables -fstack-clash-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mcpu=power8 -mtune=power8 -funwind-tables -fstack-clash-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.9 -c yaml/_yaml.c -o build/temp.linux-ppc64le-3.9/yaml/_yaml.o
gcc: error: yaml/_yaml.c: No such file or directory
gcc: fatal error: no input files
compilation terminated.
error: command '/usr/bin/gcc' failed with exit code 1

[ssbarnea]

Pyyaml should look into using https://cibuildwheel.readthedocs.io/en/stable/ project for building multi-platform binaries as it supports a huge range of platforms, one that apparently includes ppc64le too.

[nitzmahone]

We've been using cibuildwheel for years, but building and testing under emulated arches is ridiculously slow, especially since we're providing our own libyaml to static link against. That means we need a much larger created-on-the-fly emulated build environment which takes even longer to set up, and we can't reasonably maintain cached build images for all those arches. Access to native builders for "non-commodity" architectures has not been workable long-term either- TMK Travis CI is the only public service that provides them in any capacity today. I had to drop the native ppc64le builds for cffi because the Travis native ppc64le builders were so unreliable. Similarly, from past experiences on other projects, relying on "handcrafted" build resources behind a firewall is inevitably broken every time we need to do a release (we're already doing that for aarch64 Mac builds, but at least I'm the one maintaining it).

I might consider adding "release-time only" emulated CI builds for ppc64le with the next batch of stuff, but we're absolutely not going to be testing those arches in the normal CI, and if it causes us problems, it's just getting shut off. The other problem of course is which Python/manylinux profiles to build against, since we can't currently do multi-Python builds. This conversation gets a lot easier if Cython's abi3 support actually gets usable, but last I tried it with PyYAML, not so much.