Telcon: 2016 06 30

Participants

1. Todd Gamblin (LLNL)
2. Elizabeth Fischer (NASA)
3. Greg Lee (LLNL)
4. Ben Boeckel (Kitware)
5. Jim Amundson (Fermi)
6. Jim Galarowicz (Krell)
7. KT Thompson (LANL)
8. Mario Melara (NERSC)
9. Matt Belhorn (ORNL)
10. Patrick Gartung (Fermi)
11. Peter Scheibel (LLNL)
12. Greg Becker (LLNL)
13. Robert French (ORNL)

Agenda

1. Hybrid Spack-Conda

News

1. Spack has 499 packages now (so close)

2. #579 - Download caching is now merged

• Spack will no longer download the same archive many times.
• Spack build a cache of all the tarballs you download in var/spack/cache.
• The archive can be copied around and used as a Spack mirror (it's in the same structure as a mirror)
• spack purge --cache will remove this.
• Suggestion to limit size of tarball cache and evict tarballs with LRU policy.

1. Other notable merges:

• Many updates to docs
• More module support from Massimiliano (@alalazo) in #984 - module files : explicit load and mnemonic suffixes
• @citibeth's recursive module support #721
  • prints out module load commands for a package and all its deps.

1. Soon:

• Build dependencies #378
• Faster virtuals #1015
• Binary caching #1013
• config.yaml #908
• #543

1. Planned:

• Rework compilers as dependencies
  1. Mixing C and Fortran compilers
  2. Manage hybrid builds with nvcc
  3. Manage icc + gcc builds.
  4. Virtual dependencies for language levels and OpenMP.
• LLNL hackathon on using Prolog engine to do concretization.
  • Thinking about using pyDatalog to do concretization solving.
  • Other packages to consider:
    • Ben Boeckel: consider libzyp
    • Patrick Gartung: networkx
    • Elizabeth Fischer: SAT solver slides

1. Interesting discussion on ways to install packages faster, without merging into the mainline (like Bower, other PMs)

• See Issue #1136

1. Spack/CMake integration.

• @gartung doing this for SpackDev at Fermi
• could leverage some of #543

1. glibc package

• Requires additional build support.
• Spack build model does not yet really support glibc.
• But we can work it in as a stopgap for now, and handle it better when we rework compilers as dependencies.
  • need different types of DAG constraints for glibc

1. LLNL/NERSC/ORNL Spackathon

• July 18-20, at NERSC.
• Will focus on stabilizing Cray packages.

Other discussions

1. @citibeth: Hybrid Spack/Conda?

• Spack manages combinatorial stuff well
• Unnamed company considering Conda and Spack
  • problems:
    • Windows (LLNL does actually care about this, as does Kitware)
    • Conda runs on windows but doesn't support combinatorial things
  • idea:
    • make spack generate combinatorial conda recipes
    • similar to what we do for RPM generation at LLNL.
      • try to leverage some of the RPM code for generating Conda recipes.

1. @gartung having some issues with autoloading gcc modules

• see autoloading modules above -- bug with the new feature.
• (@alalazo) followup from the google group : the issue seems to be due to an unquoted use of '%<compiler>' as a constraint in the YAML configuration file

1. @citibeth: Still having issues building Python 2.7

2. @citibeth: SC16 Tutorial

• Need to freeze some features before SC16
• Some gotchas might affect new users:
  • +mpi vs. ^mpich in concretization
  • variant propagation
    • look at #839
  • more stable version numbers
    • hierarchical packages.yaml (by architecture) are needed to get this working.

1. @gartung: what about hierarchical install directories?

• Allows physicists to depend on a central shared spack and also build other stuff around it.
• need better umask/gmask features as well.

1. ORNL folks:

• ORNL likes centrally managed installs.
• Allowing spack to write to multiple directories might confuse things a lot.

1. Mario: what does ORNL do with module files?

• Robert: Smithy stores templates for different module files, results in buggy modules.
  • NERSC uses straight Cray modules, with lmod semantics for compiler/MPI
  • ORNL does not use Cray module scheme -- but their modules attempt to deduce what to load based on the current environment.
    • Smithy templates generate case statements in the module file, which are stored in templates (instead of generated on the fly).
  • LANL depends on the user loading MPI, compiler, etc. to deduce what to load.