rocHPL Benchmark

[IllusiveAldebaran]

Trying to run rocHPL as described on Infinity Hub. Would appreciate some support.

Trying to get the binary in 3 different ways isn't working:

1. From the source code directly:
   ./install.sh

make[2]: Entering directory '/work1/zixian/henryfzh/rocHPL/tpl/ucx/build/src/tools/profile'
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/work1/zixian/henryfzh/rocHPL/tpl/ucx/build/src/tools/profile'
Making all in bindings/go
make[2]: Entering directory '/work1/zixian/henryfzh/rocHPL/tpl/ucx/build/bindings/go'
go env -w GO111MODULE=off ; \
cd /work1/zixian/henryfzh/rocHPL/tpl/ucx/contrib/../bindings/go/src/examples/perftest ;\
go build --tags= -o /work1/zixian/henryfzh/rocHPL/tpl/ucx/build/bindings/go/.libs/tmp/goperftest
cd /work1/zixian/henryfzh/rocHPL/tpl/ucx/contrib/../bindings/go/src/ucx ;\
go build
# examples/perftest
/usr/lib/golang/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/bin/ld: /work1/zixian/henryfzh/rocHPL/tpl/ucx/build/src/ucp/.libs/libucp.so: undefined reference to `uct_rkey_compare'
/usr/bin/ld: /opt/ohpc/pub/mpi/ucx-ohpc/1.14.0/lib/libuct.so.0: undefined reference to `ucs_callbackq_remove_if'
collect2: error: ld returned 1 exit status

make[2]: *** [Makefile:654: goperftest] Error 2
make[2]: Leaving directory '/work1/zixian/henryfzh/rocHPL/tpl/ucx/build/bindings/go'
make[1]: *** [Makefile:784: all-recursive] Error 1
make[1]: Leaving directory '/work1/zixian/henryfzh/rocHPL/tpl/ucx/build'
make: *** [Makefile:648: all] Error 2
ERROR: Unknown error

If you load the module ucx and the autotools and etc the error is a little different:

make  all-recursive
make[1]: Entering directory '/work1/zixian/pacolord/rocHPL/tpl/ucx/build'
Making all in src/ucm
make[2]: Entering directory '/work1/zixian/pacolord/rocHPL/tpl/ucx/build/src/ucm'
Making all in .
make[3]: Entering directory '/work1/zixian/pacolord/rocHPL/tpl/ucx/build/src/ucm'
make[3]: *** No rule to make target '/home1/pacolord/rocHPL/tpl/ucx/contrib/../src/ucm/event/event.c', needed by 'event/libucm_la-event.lo'.  Stop.
make[3]: Leaving directory '/work1/zixian/pacolord/rocHPL/tpl/ucx/build/src/ucm'
make[2]: *** [Makefile:939: all-recursive] Error 1
make[2]: Leaving directory '/work1/zixian/pacolord/rocHPL/tpl/ucx/build/src/ucm'
make[1]: *** [Makefile:784: all-recursive] Error 1
make[1]: Leaving directory '/work1/zixian/pacolord/rocHPL/tpl/ucx/build'
make: *** [Makefile:648: all] Error 2
ERROR: Unknown error

But basically the same.

2. Using the spack build script

bldRochpl.sh gives:

[myuser@login1 spack]$ ./bldRochpl.sh 
/home1/myuser/spack
==> Error: cannot initialize environment, spack.yaml already exists
/usr/lib/gcc/x86_64-redhat-linux/11/include/omp.h
find: ‘/usr/share/polkit-1/rules.d’: Permission denied
find: ‘/usr/share/empty.sshd’: Permission denied
find: ‘/usr/libexec/initscripts/legacy-actions/auditd’: Permission denied
==> Package rochpl was already added to /work1/zixian/pacolord/InfinityHub-CI/rochpl/spack
==> Error: variant "openmp" not found in package "llvm-amdgpu"

3. Via docker container
   The previous method wouldn't be too bad if we could get the rocHPL binary via docker, but it seems docker isn't installed, and "If needed, please consult with your system administrator or view official documentation." Though docker runs as root running the mpirun_rochpl binary with singularity is the only thing we're interested.

Singularity isn't giving us issues though.

• pacolord & henryfzh