Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

deprecate unknown configure args in ConfigureMake easyblock #3026

Open
wants to merge 3 commits into
base: develop
Choose a base branch
from

Conversation

Flamefire
Copy link
Contributor

More compatible version of #3025 for 4.x to only deprecate and verbosely warn about unknown configure args

@boegel
Copy link
Member

boegel commented Nov 8, 2023

This makes sense to include in EasyBuild 4.x, but we should still try and do a sweep across all easyconfigs using ConfigureMake (or an easyblock that derives from ConfigureMake) so see how often this would start producing scary warnings...

That's a huge list of easyconfigs, so it's a pretty painful exercise, but a necessary one imho, unless we're willing to accept exposing people to these warnings (which may help in getting the problems fixed, I guess).

@Flamefire
Copy link
Contributor Author

That's a huge list of easyconfigs, so it's a pretty painful exercise, but a necessary one imho, unless we're willing to accept exposing people to these warnings (which may help in getting the problems fixed, I guess).

I'd opt for the latter. I actually run the EasyBlock which fails hard instead on our cluster such that those things actually make me fix it. A scary warning to convince more people to cleanup the ECs sounds like a good compromise to get this big task done.

Doing a bulk fix is going to be hard as even though you can use --stop=configure for this you'd need to have dependencies installed.

@boegel
Copy link
Member

boegel commented Nov 8, 2023

I did a quick scan for "unrecognized options" in my set of installations, and the installation logs for the modules below had hits.

Most of these were done as a part of the regression test for EasyBuild v4.8.2, so recent installations, but some are older (marked with (*)) and those may already be fixed.

  • BigDFT/1.9.1-foss-2021b: "unrecognized options: --disable-Werror, --with-yaml-path"
  • FriBidi/1.0.10-GCCcore-10.2.0: "unrecognized options: --disable-docs"
  • FriBidi/1.0.10-GCCcore-10.3.0 (same as above)
  • FriBidi/1.0.10-GCCcore-11.2.0 (same as above)
  • FriBidi/1.0.12-GCCcore-12.2.0 (same as above)
  • FriBidi/1.0.12-GCCcore-12.3.0 (same as above)
  • FriBidi/1.0.5-GCCcore-8.3.0 (*) (same as above)
  • FriBidi/1.0.9-GCCcore-9.3.0 (*) (same as above)
  • GDAL/3.0.2-foss-2019b-Python-3.7.4 (*): "unrecognized options: --with-libgeotiff"
  • GDAL/3.0.2-intel-2019b-Python-3.7.4 (*) (same as above)
  • GDAL/3.0.4-foss-2020a-Python-3.8.2 (*) (same as above)
  • GDAL/3.5.0-foss-2022a (*): "unrecognized options: --with-jasper"
  • GRASS/8.2.0-foss-2021b: "unrecognized options: --enable-64bit, --with-ffmpeg, --with-ffmpeg-libs, --with-ffmpeg-includes, --without-glw, --with-lapack-lib, --with-blas-lib, --with-proj, --with-python, --with-spatialite, --with-zlib, --with-ibzlib-includes"
  • groff/1.23.0-GCCcore-13.2.0: "unrecognized options: --with-doc"
  • GTK+/3.24.13-GCCcore-8.3.0 (*): "unrecognized options: --disable-visibility"
  • GTK+/3.24.23-GCCcore-10.2.0 (*) (same as above)
  • GTK3/3.24.29-GCCcore-10.3.0 (same as above)
  • GTK3/3.24.31-GCCcore-11.2.0 (same as above)
  • GTK3/3.24.35-GCCcore-12.2.0 (same as above)
  • hwloc/2.2.0-GCCcore-10.2.0 (*): "unrecognized options: --enable-libnuma"
  • hwloc/2.2.0-GCCcore-9.3.0 (*) (same as above)
  • hwloc/2.4.1-GCCcore-10.3.0 (same as above)
  • hwloc/2.5.0-GCCcore-11.2.0 (same as above)
  • hwloc/2.7.1-GCCcore-11.3.0 (same as above)
  • hwloc/2.8.0-GCCcore-12.2.0 (same as above)
  • hwloc/2.9.1-GCCcore-12.3.0 (same as above)
  • hwloc/2.9.2-GCCcore-13.2.0 (same as above)
  • libfdf/0.2.2-GCC-10.3.0-serial: "unrecognized options: --without-mpi"
  • libfdf/0.2.2-GCC-11.2.0-serial (same as above)
  • Libint/2.6.0-GCC-10.3.0-lmax-6-cp2k: "unrecognized options: --with-cxxgen-optflags"
  • Libint/2.6.0-iimpi-2021a-lmax-6-cp2k (same as above)
  • Libint/2.6.0-gompi-2020a-lmax-6-cp2k (*) (same as above)
  • LibTIFF/4.1.0-GCCcore-10.2.0 (*): "unrecognized options: --disable-libdeflate"
  • LibTIFF/4.1.0-GCCcore-9.3.0 (*) (same as above)
  • MEME/5.4.1-GCC-10.3.0: "unrecognized options: --with-python3"
  • MESS/0.1.6-foss-2019b (*): "unrecognized options: --enable-fortran"
  • OpenMPI/4.0.6-GCC-10.3.0: "unrecognized options: --with-ucc"
  • OpenMPI/4.0.7-GCC-10.3.0 (same as above)
  • OpenMPI/4.1.1-GCC-10.3.0 (same as above)
  • OpenMPI/4.1.1-GCC-11.2.0 (same as above)
  • OpenMPI/4.1.2-GCC-11.2.0 (same as above)
  • Tcl/8.6.10-GCCcore-10.2.0 (*): "unrecognized options: --with-tclinclude, --enable-shared"
  • Tcl/8.6.10-GCCcore-9.3.0 (*) (same as above)
  • Tcl/8.6.11-GCCcore-10.3.0 (same as above)
  • Tcl/8.6.11-GCCcore-11.2.0 (same as above)
  • Tcl/8.6.12-GCCcore-11.3.0 (same as above)
  • Tcl/8.6.12-GCCcore-12.2.0 (same as above)
  • Tcl/8.6.13-GCCcore-12.3.0 (same as above)
  • Tcl/8.6.13-GCCcore-13.2.0 (same as above)
  • Tkinter/3.11.3-GCCcore-12.3.0: "unrecognized options: --with-tcltk-includes, --with-tcltk-libs"
  • Tkinter/3.11.5-GCCcore-13.2.0 (same as above)

Although not a blocker for merging this PR, it would be good to have these easyconfigs updated so the warning won't present itself when installing these...

@Flamefire
Copy link
Contributor Author

I checked my recent tree and also found the Tcl issue. However it looks like it is some configure called by configure passing those flags as we only do ./configure --prefix=/software/Tcl/8.6.12-GCCcore-11.3.0 --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --enable-threads EXTRA_INSTALL="install-private-headers"

@boegel
Copy link
Member

boegel commented Jan 3, 2024

I did a scan for "unrecognized options" in all installations we have on generoso (9,991 installations), which revealed these:

BigDFT/1.9.1-foss-2021b
configure: WARNING: unrecognized options: --disable-Werror, --enable-mpi, --with-yaml-path, --with-ext-linalg
configure: WARNING: unrecognized options: --disable-Werror
configure: WARNING: unrecognized options: --disable-Werror, --with-yaml-path
configure: WARNING: unrecognized options: --disable-Werror, --with-yaml-path, --with-ext-linalg

FFLAS-FFPACK/2.5.0-gfbf-2022a
configure: WARNING: unrecognized options: --with-gmp, --with-givaro

FriBidi/1.0.10-GCCcore-10.2.0
FriBidi/1.0.10-GCCcore-10.3.0
FriBidi/1.0.10-GCCcore-11.2.0
FriBidi/1.0.12-GCCcore-11.3.0
FriBidi/1.0.12-GCCcore-12.2.0
FriBidi/1.0.12-GCCcore-12.3.0
FriBidi/1.0.13-GCCcore-13.2.0
FriBidi/1.0.5-GCCcore-7.3.0
FriBidi/1.0.5-GCCcore-8.2.0
FriBidi/1.0.5-GCCcore-8.3.0
FriBidi/1.0.9-GCCcore-9.3.0
configure: WARNING: unrecognized options: --disable-docs

GDAL/2.2.3-foss-2018b-Python-2.7.15
GDAL/2.2.3-foss-2018b-Python-3.6.6
GDAL/3.0.0-foss-2019a-Python-3.7.2
GDAL/3.0.2-foss-2019b-Python-3.7.4
GDAL/3.0.2-intel-2019b-Python-3.7.4
GDAL/3.0.4-foss-2020a-Python-3.8.2
configure: WARNING: unrecognized options: --with-libgeotiff

GDAL/3.5.0-foss-2022a
configure: WARNING: unrecognized options: --with-jasper

GRASS/8.2.0-foss-2021b
configure: WARNING: unrecognized options: --enable-64bit, --with-ffmpeg, --with-ffmpeg-libs, --with-ffmpeg-includes, --without-glw, --with-lapack-lib, --with-blas-lib, --with-proj, --with-python, --with-spatialite, --with-zlib, --with-ibzlib-includes

GTK+/3.24.13-GCCcore-8.3.0
GTK+/3.24.17-GCCcore-9.3.0
GTK+/3.24.23-GCCcore-10.2.0
GTK+/3.24.8-GCCcore-8.2.0
configure: WARNING: unrecognized options: --disable-visibility

GTK3/3.24.29-GCCcore-10.3.0
GTK3/3.24.31-GCCcore-11.2.0
GTK3/3.24.33-GCCcore-11.3.0
GTK3/3.24.35-GCCcore-12.2.0
configure: WARNING: unrecognized options: --disable-visibility

GnuTLS/3.7.2-GCCcore-10.3.0
configure: WARNING: unrecognized options: --without-tpm2

HDF-EOS2/3.0-GCCcore-11.3.0
configure: WARNING: unrecognized options: --enable-install-include

LibTIFF/4.1.0-GCCcore-10.2.0
LibTIFF/4.1.0-GCCcore-8.3.0
LibTIFF/4.1.0-GCCcore-9.3.0
configure: WARNING: unrecognized options: --disable-libdeflate

Libint/2.6.0-GCC-10.2.0-lmax-6-cp2k
Libint/2.6.0-GCC-10.3.0-lmax-6-cp2k
Libint/2.6.0-GCC-11.3.0-lmax-6-cp2k
Libint/2.6.0-gompi-2020a-lmax-6-cp2k
Libint/2.6.0-iccifort-2020.4.304-lmax-6-cp2k
Libint/2.6.0-iimpi-2021a-lmax-6-cp2k
configure: WARNING: unrecognized options: --with-cxxgen-optflags

LinBox/1.7.0-gfbf-2022a
configure: WARNING: unrecognized options: --with-fflas-ffpack, --with-givaro, --enable-openmp

MDSplus/7.96.12-GCCcore-9.3.0
configure: WARNING: unrecognized options: --disable-hdf5

MEME/5.4.1-GCC-10.3.0
configure: WARNING: unrecognized options: --with-python3
configure: WARNING: unrecognized options: --with-python3

MEME/5.4.1-GCC-11.2.0
configure: WARNING: unrecognized options: --with-python3
configure: WARNING: unrecognized options: --with-python3

MIRA/5.0rc2-foss-2020b
configure: WARNING: unrecognized options: --with-tcmalloc-dir

OpenMPI/4.1.1-intel-compilers-2021.4.0
configure: WARNING: unrecognized options: --with-ucc

Tcl/8.6.10-GCCcore-10.2.0
Tcl/8.6.10-GCCcore-9.3.0
Tcl/8.6.11-GCCcore-10.3.0
Tcl/8.6.11-GCCcore-11.2.0
Tcl/8.6.12-GCCcore-11.3.0
Tcl/8.6.12-GCCcore-12.2.0
Tcl/8.6.13-GCCcore-12.3.0
Tcl/8.6.13-GCCcore-13.1.0
Tcl/8.6.13-GCCcore-13.2.0
Tcl/8.6.9-GCCcore-8.2.0
Tcl/8.6.9-GCCcore-8.3.0
configure: WARNING: unrecognized options: --with-tclinclude, --enable-shared

Tkinter/3.11.3-GCCcore-12.3.0
configure: WARNING: unrecognized options: --with-tcltk-includes, --with-tcltk-libs

Yambo/5.2.dev-20230512-intel-2021b
configure: WARNING: unrecognized options: --with-netcdf-lib, --with-netcdf-include

bigdft-suite/1.9.1-foss-2021b
configure: WARNING: unrecognized options: --disable-Werror, --enable-mpi, --with-ext-linalg, --with-yaml-path
configure: WARNING: unrecognized options: --disable-Werror
configure: WARNING: unrecognized options: --disable-Werror, --with-yaml-path
configure: WARNING: unrecognized options: --disable-Werror, --with-ext-linalg, --with-yaml-path

gap/4.12.2-foss-2022a
configure: WARNING: unrecognized options: --with-gaproot

git-annex/10.20230802-GCCcore-12.2.0
configure: WARNING: unrecognized options: --with-compiler

groff/1.23.0-GCCcore-13.2.0
configure: WARNING: unrecognized options: --with-doc

hwloc/2.0.2-GCCcore-8.2.0
hwloc/2.0.3-GCCcore-8.3.0
hwloc/2.1.0-GCCcore-9.2.0
hwloc/2.2.0-GCCcore-10.2.0
hwloc/2.2.0-GCCcore-9.3.0
hwloc/2.4.1-GCCcore-10.3.0
hwloc/2.5.0-GCCcore-11.2.0
hwloc/2.7.1-GCCcore-11.3.0
hwloc/2.8.0-GCCcore-12.2.0
hwloc/2.9.1-GCCcore-12.3.0
hwloc/2.9.2-GCCcore-13.2.0
configure: WARNING: unrecognized options: --enable-libnuma

libfdf/0.2.2-GCC-10.2.0-serial
libfdf/0.2.2-GCC-10.3.0-serial
libfdf/0.2.2-GCC-11.2.0-serial
libfdf/0.2.2-iccifort-2020.4.304-serial
libfdf/0.2.2-intel-compilers-2021.2.0-serial
libfdf/0.2.2-intel-compilers-2021.4.0-serial
configure: WARNING: unrecognized options: --without-mpi

@Flamefire
Copy link
Contributor Author

That's quite a few and we may want to fix them (for the non-ancient and at least easy ones) for EB 5.
Given the list above and knowing that removing that param has no effect at all we might also just fix those without doing a build.

But: Do we still need to consider/handle EB 4.x PRs after 4.9.0 was released?

@Flamefire
Copy link
Contributor Author

@boegel I compiled as list of EC filenames from your 2 comments and mentioned in #3025 and am running tests building those using develop. The resulting PRs should be all findable by https://github.com/easybuilders/easybuild-easyconfigs/pulls?q=is%3Apr+author%3AFlamefire++configure

@Flamefire
Copy link
Contributor Author

Test report by @Flamefire

Overview of tested easyconfigs (in order)

  • SUCCESS BigDFT-1.9.1-foss-2021b.eb
  • SUCCESS FFLAS-FFPACK-2.5.0-gfbf-2022a.eb
  • SUCCESS FriBidi-1.0.10-GCCcore-10.2.0.eb
  • SUCCESS FriBidi-1.0.10-GCCcore-10.3.0.eb
  • SUCCESS FriBidi-1.0.10-GCCcore-11.2.0.eb
  • SUCCESS FriBidi-1.0.12-GCCcore-11.3.0.eb
  • SUCCESS FriBidi-1.0.12-GCCcore-12.2.0.eb
  • SUCCESS FriBidi-1.0.12-GCCcore-12.3.0.eb
  • SUCCESS FriBidi-1.0.13-GCCcore-13.2.0.eb
  • SUCCESS FriBidi-1.0.5-GCCcore-7.3.0.eb
  • SUCCESS FriBidi-1.0.5-GCCcore-8.2.0.eb
  • SUCCESS FriBidi-1.0.5-GCCcore-8.3.0.eb
  • SUCCESS FriBidi-1.0.9-GCCcore-9.3.0.eb
  • SUCCESS gap-4.12.2-foss-2022a.eb
  • SUCCESS GDAL-2.2.3-foss-2018b-Python-2.7.15.eb
  • SUCCESS GDAL-3.0.0-foss-2019a-Python-3.7.2.eb
  • SUCCESS GDAL-3.0.2-foss-2019b-Python-3.7.4.eb
  • SUCCESS GDAL-3.5.0-foss-2022a.eb
  • SUCCESS git-annex-10.20230802-GCCcore-12.2.0.eb
  • SUCCESS GnuTLS-3.7.2-GCCcore-10.3.0.eb
  • SUCCESS GRASS-8.2.0-foss-2021b.eb
  • SUCCESS groff-1.23.0-GCCcore-13.2.0.eb
  • SUCCESS GTK+-3.24.13-GCCcore-8.3.0.eb
  • SUCCESS GTK+-3.24.17-GCCcore-9.3.0.eb
  • SUCCESS GTK+-3.24.23-GCCcore-10.2.0.eb
  • SUCCESS GTK+-3.24.8-GCCcore-8.2.0.eb
  • SUCCESS GTK3-3.24.29-GCCcore-10.3.0.eb
  • SUCCESS GTK3-3.24.31-GCCcore-11.2.0.eb
  • SUCCESS GTK3-3.24.33-GCCcore-11.3.0.eb
  • SUCCESS GTK3-3.24.35-GCCcore-12.2.0.eb
  • SUCCESS HDF-EOS2-3.0-GCCcore-11.3.0.eb
  • SUCCESS hwloc-2.0.2-GCCcore-8.2.0.eb
  • SUCCESS hwloc-2.0.3-GCCcore-8.3.0.eb
  • SUCCESS hwloc-2.1.0-GCCcore-9.2.0.eb
  • SUCCESS hwloc-2.2.0-GCCcore-10.2.0.eb
  • SUCCESS hwloc-2.2.0-GCCcore-9.3.0.eb
  • SUCCESS hwloc-2.4.1-GCCcore-10.3.0.eb
  • SUCCESS hwloc-2.5.0-GCCcore-11.2.0.eb
  • SUCCESS hwloc-2.7.1-GCCcore-11.3.0.eb
  • SUCCESS hwloc-2.8.0-GCCcore-12.2.0.eb
  • SUCCESS hwloc-2.9.1-GCCcore-12.3.0.eb
  • SUCCESS hwloc-2.9.2-GCCcore-13.2.0.eb
  • SUCCESS libfdf-0.2.2-GCC-10.2.0-serial.eb
  • SUCCESS libfdf-0.2.2-GCC-10.3.0-serial.eb
  • SUCCESS libfdf-0.2.2-GCC-11.2.0-serial.eb
  • SUCCESS Libint-2.6.0-GCC-10.2.0-lmax-6-cp2k.eb
  • SUCCESS Libint-2.6.0-GCC-10.3.0-lmax-6-cp2k.eb
  • SUCCESS Libint-2.6.0-gompi-2020a-lmax-6-cp2k.eb
  • SUCCESS Libint-2.7.2-GCC-11.3.0-lmax-6-cp2k.eb
  • SUCCESS LibTIFF-4.1.0-GCCcore-10.2.0.eb
  • SUCCESS LibTIFF-4.1.0-GCCcore-8.3.0.eb
  • SUCCESS LibTIFF-4.1.0-GCCcore-9.3.0.eb
  • SUCCESS LinBox-1.7.0-gfbf-2022a.eb
  • SUCCESS MDSplus-7.96.12-GCCcore-9.3.0.eb
  • SUCCESS MEME-5.4.1-GCC-10.3.0.eb
  • SUCCESS MIRA-5.0rc2-foss-2020b.eb
  • SUCCESS OpenMPI-4.0.6-GCC-10.3.0.eb
  • SUCCESS OpenMPI-4.0.7-GCC-10.3.0.eb
  • SUCCESS OpenMPI-4.1.1-GCC-10.3.0.eb
  • SUCCESS OpenMPI-4.1.1-GCC-11.2.0.eb
  • SUCCESS OpenMPI-4.1.2-GCC-11.2.0.eb
  • SUCCESS Tcl-8.6.10-GCCcore-10.2.0.eb
  • SUCCESS Tcl-8.6.10-GCCcore-9.3.0.eb
  • SUCCESS Tcl-8.6.11-GCCcore-10.3.0.eb
  • SUCCESS Tcl-8.6.11-GCCcore-11.2.0.eb
  • SUCCESS Tcl-8.6.12-GCCcore-11.3.0.eb
  • SUCCESS Tcl-8.6.12-GCCcore-12.2.0.eb
  • SUCCESS Tcl-8.6.13-GCCcore-12.3.0.eb
  • SUCCESS Tcl-8.6.13-GCCcore-13.1.0.eb
  • SUCCESS Tcl-8.6.13-GCCcore-13.2.0.eb
  • SUCCESS Tcl-8.6.9-GCCcore-8.2.0.eb
  • SUCCESS Tcl-8.6.9-GCCcore-8.3.0.eb
  • SUCCESS Tkinter-3.11.3-GCCcore-12.3.0.eb
  • SUCCESS Tkinter-3.11.5-GCCcore-13.2.0.eb
  • SUCCESS GDAL-3.0.4-foss-2020a-Python-3.8.2.eb

Build succeeded for 75 out of 75 (75 easyconfigs in total)
n1358 - Linux RHEL 8.7 (Ootpa), x86_64, Intel(R) Xeon(R) Platinum 8470 (icelake), Python 3.8.13
See https://gist.github.com/Flamefire/7248cfe5c3a8bc6c24932231bc839e97 for a full test report.

@Flamefire Flamefire marked this pull request as ready for review March 11, 2024 08:07
@Flamefire
Copy link
Contributor Author

@boegel The above test report was done with a temporary change to issue an error instead of a warning. As you can see all ECs you identified are now fixed. So I'd say this is ready now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants