Skip to content

Conference call notes 20210120

Jump to bottom
Kenneth Hoste edited this page Jan 20, 2021 · 4 revisions

(back to Conference calls)

Notes on the 165th EasyBuild conference call, Wednesday January 20 2021 (09:00 UTC)

Attendees

Alphabetical list of attendees (10):

  • Simon Branford (University of Birmingham, UK)
  • Alex Domingo (Vrije Universiteit Brussel, Belgium)
  • Alexander Grund (TU Dresden, Germany)
  • Kenneth Hoste (HPC-UGent, Belgium)
  • Adam Huffman (Big Data Institute, Oxford, UK)
  • Christian Kniep (AWS)
  • Terje Kvernes (University of Oslo, Norway)
  • Robert Mijakovic (LuxProvide)
  • Mikael Öhman (Chalmers University of Technology, Sweden)
  • Jörg Saßmannshausen (NIHR Biomedical Research Centre, UK)

Agenda

  • update on recent developments + outlook to next release
  • Q&A

Recent developments

  • next release (v4.3.3): before EUM'21? during EUM week?
  • recent changes
    • framework
      • bug fixes
        • don't clean up imported modules after verifying imports of included Python modules (PR #3544)
          • fix for obscure reported issue related to Python garbage collection (issue #3542)
      • enhancements
        • detect 'SYSTEM' toolchain as special case in easystack files (PR #3543)
        • enhance extract_cmd function to use 'cp -a' for shell scripts (.sh) (PR #3545)
          • we should also do this for *.whl "source" files (avoids the need for unpack_sources = False)
        • add support for specifying custom name to use when checking accepted EULA (PR #3546)
          • useful for Intel oneAPI components: can use 'Intel-oneAPI' to accept EULA for Intel compilers + impi + imkl
    • easyblocks
      • bug fixes
        • set $PYTHONNOUSERSITE in PythonBundle.extensions_step (PR #2289)
          • to avoid picking up Python packages installed in $HOME/.local (with pip install --user) when skipping/installing extensions
        • fix Boost sanity check on POWER (PR #2291)
        • unify handling of pylibdirs and don't add duplicated $PYTHONPATH in PythonBundle (PR #2281)
        • enhance Amber easyblock to fix running of update_amber script when 'python' command is not available in OS (PR #2282)
      • enhancements
        • update Clang easyblock to add support for building extra tools + leveraging hwloc and Z3 as optional dependencies (PR #2310)
      • new software
        • (none)
    • easyconfigs
      • over 50 merged easyconfig PRs since last conf call
      • bug fixes
        • added missing space in configopts in ParaView 5.8.0 easyconfigs using 2020a toolchain (PR #10989)
        • fix source URL in expat easyconfigs (and consistently add custom sanity_check_paths) (PR #11960)
        • add patch for Boost 1.74.0 to fix missing include file (PR #12007)
      • enhancements
        • (nothing major)
      • new software
        • (nothing major)
      • software updates
        • PyTorch 1.7.1 with fosscuda/2019b (PR #11636) and foss/2020b (PR #12008)
          • fosscuda/2020b is work-in-progress (PR #12003), failing tests?
          • issue with NCCL is fixed (multi-GPU test was running & failing on single GPU system)
        • SciPy-bundle v2020.03 with Python 2.7.18 and {foss,intel}/2020a (PR #11957)
        • ReFrame v3.3 (PR #11787)
      • changes
        • use libpng provided by EasyBuild in VTK to fix build issue on ppc64le (PR #11990)
  • to merge/fix/tackle soon
    • framework (v4.3.3 milestone)
      • support additional features in easystack files (see issues #3468, #3512, #3513, #3516)
      • add toolchain deps to template resolution (PR #3541)
        • added so %(cudaver)s template is defined when using fosscuda toolchain
        • not merged yet because it results in the Toolchain instance being created a lot earlier
        • also, it triggers a search for the easyconfig files for the toolchain...
        • Kenneth will try to find time to look into alternative approach based on $EBVERSIONCUDA...
        • or maybe we just need to be a bit more careful w.r.t. taking toolchain dependencies into account when defining the templates?
      • symlink lib to lib64 if only lib64 exists (issue #3549)
    • easyblocks (v4.3.3 milestone)
      • bug fixes
        • correctly determine path to active binutils in TensorFlow easyblock (PR #2218)
        • fix taking into account --sysroot when installing/using CMake
          • two options:
            • use toolchain file in CMakeMake (PR #2247)
            • patch CMake installation (PR #2248)
        • treat files/directories of unpacked sources equally in PackedBinary generic easyblock (PR #2306)
        • make sure the installation of libiberty.a in the binutils easyblock goes into a populated directory (PR #2308)
      • enhancements
        • enhance OpenBLAS easyblock to make it aware of optarch (PR #1946)
        • add support for statically linking Bazel (PR #2272)
        • improve Bazel easyblock: add support for running tests, enabl static linking, use build dir rather than tmpdir, verbose output (PR #2285)
        • add support for skipping steps in Python packages installed as extension (PR #2290)
        • update NEURON easyblock to use CMakeMake for recent versions (PR #2304)
        • add support for running TensorFlow CPU and GPU tests separately and enhance test failure reporting (PR #2312)
        • update impi easyblock for impi 2021.x (oneAPI) (PR #2313)
      • changes
        • create less temporary directories for TensorFlow by (only) using --output_user_root (PR #2293)
        • check for accepted EULA in custom easyblock for NVHPC (PR #2311)
      • new software
        • add custom easyblock for:
          • ADIOS (PR #2070)
          • JAX (PR #2262)
          • RELION (PR #2274)
            • does adding an EasyBuild configuration option to specify the system job scheduler used make sense?
            • yes, if it can help easyblocks to make proper decisions
            • can also be useful for OpenMPI to opt-in to Slurm integration
        • new generic easyblock for Intel oneAPI compilers (PR #2305)
          • using intel-compilers as software name, including icc/icpc/ifort (classic compilers) + icx/icpx/ifx/dpcpp (oneAPI compilers)
    • easyconfigs (v4.3.3 milestone)
      • bug fixes
        • fix for "error: 'runtime_error' is not a member of 'std'" in qtlocation (Qt5) (PR #12012)
      • enhancements
        • enable tests for most recent Bazel versions (PR #11894)
      • new software
        • Intel oneAPI
          • there will be no further updates to Intel Parallel Studio (apparently)
          • intel-compilers v2020.1 (PR #11982)
      • software updates

Q&A

  • Alex: Bazel patch to fix problem with new unzip command refusing to unpack old *.jar used by old Bazel versions
  • Mikael's flatbuffers PR (PR #11804)
    • included patch is required, but doesn't actually work yet
    • Robert has a working patch
  • Robert has a bunch of PRs (~90) to contribute back: TensorFlow (fosscuda/2020b), PyTorch, Horovod, flatbuffers (with fixed patch), ...
  • Robert: any experiences with building OpenMPI with Clang?
    • should ask Markus Geimer for feedback on this (@geimer in GitHub)
Clone this wiki locally