clangversion string style fails sanity check in latest AOCC

[matthewabrown]

easybuild-easyblocks/easybuild/easyblocks/a/aocc.py

Line 150 in cc5b95d

'dirs': ['include/llvm', 'lib/clang/%s/lib' % self.clangversion, 'lib32'],

I'm creating a new easyconfig AOCC-4.2.0-GCCcore-13.2.0.eb. I use the latest 4.0.0 easyconfig but

• provided a newer GCCcore: 13.2.0
• the latest version of AOCC: 4.2.0
• updated clangversion string in the easyconfig to match the version provided by GCCcore

The user supplied dotted decimal (16.0.3) is reused during sanity check but the AOCC installer uses major version (16) only.

# EasyConfigs have a clear pattern of using dotted decimals
$ grep clang *
AOCC-3.1.0-GCCcore-10.2.0.eb:clangversion = '12.0.0'
AOCC-3.1.0-GCCcore-10.3.0.eb:clangversion = '12.0.0'
AOCC-3.1.0-GCCcore-11.2.0.eb:clangversion = '12.0.0'
AOCC-3.2.0-GCCcore-11.2.0.eb:clangversion = '13.0.0'
AOCC-3.2.0-GCCcore-11.3.0.eb:clangversion = '13.0.0'
AOCC-4.0.0-GCCcore-11.3.0.eb:clangversion = '14.0.6'
AOCC-4.0.0-GCCcore-12.2.0.eb:clangversion = '14.0.6'
AOCC-4.0.0-GCCcore-12.3.0.eb:clangversion = '14.0.6'
AOCC-4.2.0-GCCcore-13.2.0.eb:clangversion = '16.0.3'

Installation failed during sanity checking:

== postprocessing...
== sanity checking...
== ... (took 1 secs)
== FAILED: Installation ended unsuccessfully (build directory: /dev/shm/brownm12/build/AOCC/4.2.0/GCCcore-13.2.0): build failed (first 300 chars): Sanity check failed: no file found at 'lib/clang/16.0.3/include/omp.h' in /apps/easybuild/software/owl-genoa/AOCC/4.2.0-GCCcore-13.2.0
no file found at 'lib/clang/16.0.3/include/stddef.h' in /apps/easybuild/software/owl-genoa/AOCC/4.2.0-GCCcore-13.2.0
no (non-empty) directory found at 'lib/clang/16. (took 52 secs)

In the build and install directories, I can confirm that I have the header files such as this one

$ find . -name "omp.h"
./lib/clang/16/include/omp.h

Perhaps this is different behavior from previous versions of AOCC. The easiest fix might be to add some logic to the sanity check in the easyblock to provide version strings that match the expected format.

[matthewabrown]

For what it's worth, if use the short version

clangversion = '16'

in the easyconfig, the installation succeeds with no errors. But not using the dotted decimal seems contrary to all other comments and logic around clangversion.

[Thyre]

Looking at my local (non-EasyBuild) LLVM/Clang installations, the same thing can be seen:

$ ls */lib/clang
12.0.1/lib/clang:
12.0.1

13.0.1/lib/clang:
13.0.1

14.0.6/lib/clang:
14.0.6

15.0.7/lib/clang:
15.0.7

16.0.6/lib/clang:
16

17.0.6/lib/clang:
17

18.1.8/lib/clang:
18

19.1.0-rc3/lib/clang:
19

WIth LLVM 16, they switched to just using the major version as the folder name. AOCC basically just copied the LLVM/Clang behavior.

[Thyre]

We could probably add some handling for this, like done in the clang.py EasyBlock:

diff --git a/easybuild/easyblocks/a/aocc.py b/easybuild/easyblocks/a/aocc.py
index ff33673af..71156fa4a 100644
--- a/easybuild/easyblocks/a/aocc.py
+++ b/easybuild/easyblocks/a/aocc.py
@@ -138,6 +138,12 @@ class EB_AOCC(PackedBinary):
 
     def sanity_check_step(self):
         """Custom sanity check for AOCC, based on sanity check for Clang."""
+
+        # Clang v16+ only use the major version number for the resource dir
+        resdir_version = self.clangversion
+        if self.clangversion >= '16':
+            resdir_version = LooseVersion(self.clangversion).version[0]
+
         shlib_ext = get_shared_lib_ext()
         custom_paths = {
             'files': [
@@ -147,7 +153,7 @@ class EB_AOCC(PackedBinary):
                 'lib/clang/%s/include/stddef.h' % self.clangversion, 'lib/libclang.%s' % shlib_ext,
                 'lib/libomp.%s' % shlib_ext,
             ],
-            'dirs': ['include/llvm', 'lib/clang/%s/lib' % self.clangversion, 'lib32'],
+            'dirs': ['include/llvm', 'lib/clang/%s/lib' % resdir_version, 'lib32'],
         }
 
         custom_commands = [

At the same time, I would suggest updating the version list in _aocc_guess_clang_version.

[Thyre]

This should be solved by #3458, right?

[boegel]

@matthewabrown Can you confirm this issue is fixed?

You can test using `eb --include-easyblocks-from-pr 3458 ..."

[matthewabrown]

It worked for me (transcript below). Thanks all for the help and the fix!

[brownm12@owl004 AOCC]$ eb ./AOCC-4.2.0-GCCcore-13.2.0.eb --include-easyblocks-from-pr 3458 --force --accept-eula-for=AOCC
== Temporary log file in case of crash /localscratch-nvme/60647/eb-yfol0jto/easybuild-u531rlw6.log
== easyblock aocc.py included from PR #3458
== resolving dependencies ...
== processing EasyBuild easyconfig /apps/easybuild/software/owl-genoa/EasyBuild/4.9.4/easybuild/easyconfigs/a/AOCC/AOCC-4.2.0-GCCcore-13.2.0.eb
== building and installing AOCC/4.2.0-GCCcore-13.2.0...
== fetching files...
== creating build dir, resetting environment...
== unpacking...
== ... (took 7 secs)
== patching...
== preparing...
== ... (took 1 secs)
== configuring...
== building...
== testing...
== installing...
== ... (took 34 secs)
== taking care of extensions...
== restore after iterating...
== postprocessing...
== sanity checking...
== ... (took 1 secs)
== cleaning up...
== creating module...
== permissions...
== ... (took 7 secs)
== packaging...
== COMPLETED: Installation ended successfully (took 53 secs)
== Results of the build can be found in the log file(s) /apps/easybuild/software/owl-genoa/AOCC/4.2.0-GCCcore-13.2.0/easybuild/easybuild-AOCC-4.2.0-20241009.102403.log
== Build succeeded for 1 out of 1
== Temporary log file(s) /localscratch-nvme/60647/eb-yfol0jto/easybuild-u531rlw6.log* have been removed.
== Temporary directory /localscratch-nvme/60647/eb-yfol0jto has been removed.