Skip to content

Commit

Permalink
Merge branch 'develop' of https://github.com/jcsda/spack-stack into f…
Browse files Browse the repository at this point in the history 
…eature/gsibec_121
  • Loading branch information
@climbfuji
climbfuji committed Mar 11, 2024
2 parents 2a9895b + 0fae4bd commit 420c254
Show file tree
Hide file tree
Showing 12 changed files with 54 additions and 45 deletions.
3 changes: 2 additions & 1 deletion .github/workflows/util-tests.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,11 @@ jobs:
- name: checkout
uses: actions/checkout@v2
with:
submodules: false
submodules: true

- name: run-util-tests
run: |
export SPACK_STACK_DIR=$PWD
source ./setup.sh
cd util/
./util_tests.sh
2 changes: 1 addition & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Full documentation with table of contents can be found at https://spack-stack.re
Spack-stack is a collaborative effort between:
* [NOAA Environmental Modeling Center (EMC)](https://www.emc.ncep.noaa.gov/emc_new.php): [Alex Richert](https://www.github.com/AlexanderRichert-NOAA), [Hang Lei](https://www.github.com/Hang-Lei-NOAA), [Ed Hartnett](https://www.github.com/edwardhartnett)
* [UCAR Joint Center for Satellite Data Assimilation (JCSDA)](https://www.jcsda.org/): [Dom Heinzeller](https://www.github.com/climbfuji), [Steve Herbener](https://github.com/srherbener)
* [Earth Prediction Innovation Center (EPIC)](https://epic.noaa.gov/): [Cam Book](https://github.com/ulmononian), [Natalie Perlin](https://github.com/natalie-perlin)
* [Earth Prediction Innovation Center (EPIC)](https://epic.noaa.gov/): [Cam Book](https://github.com/ulmononian), [Natalie Perlin](https://github.com/natalie-perlin), [Ratko Vasic](https://github.com/ratkovasic-noaa)

For more information about the organization of the spack-stack
project, see the [Project Charter](project_charter.md).
4 changes: 4 additions & 0 deletions configs/templates/skylab-dev/spack.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ spack:
- compilers: ['%aocc', '%apple-clang', '%gcc', '%intel']
- packages:
- ewok-env +ecflow +cylc
- ai-env
- geos-gcm-env
- jedi-fv3-env
- jedi-geos-env
Expand All @@ -30,3 +31,6 @@ spack:
- matrix:
- [$packages]
- [$compilers]
exclude:
# py-torch in ai-env doesn't build with Intel
- ai-env%intel
4 changes: 0 additions & 4 deletions configs/templates/unified-dev/spack.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,13 +21,9 @@ spack:
- jedi-tools-env
- jedi-ufs-env
- jedi-um-env
#- nceplibs-env
- soca-env
- ufs-srw-app-env
#- ufs-utils-env
- ufs-weather-model-env
#- upp-env
#- ww3-env

# Various fms tags (list all to avoid duplicate packages)
- fms@release-jcsda
Expand Down
4 changes: 2 additions & 2 deletions doc/source/Overview.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,11 @@ spack-stack is mainly a collection of Spack configuration files, but provides a

spack-stack is maintained by:

- Alex Richert (@AlexanderRichert-NOAA) and Ed Hartnett (@edwardhartnett), NOAA-EMC
- Alex Richert (@AlexanderRichert-NOAA), Hang Lei (@Hang-Lei-NOAA) and Ed Hartnett (@edwardhartnett), NOAA-EMC

- Dom Heinzeller (@climbfuji) and Steve Herbener (@srherbener), JCSDA

- Cameron Book (@ulmononian) and Mark Potts (@mark-a-potts), EPIC
- Cameron Book (@ulmononian), Natalie Perlin (@natalie-perlin) and Ratko Vasic (@ratkovasic-noaa), EPIC

===============
Getting started
Expand Down
40 changes: 20 additions & 20 deletions doc/source/PreConfiguredSites.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,49 +16,49 @@ Ready-to-use spack-stack 1.6.0 installations are available on the following, ful
On selected systems, developmental versions / release candidates are installed that are newer than spack-stack 1.6.0 (see following table). For information on the spack-stack 1.6.0 releases on this platforms, please revert to version 1.6.0 of the documentation (https://spack-stack.readthedocs.io/en/1.6.0/PreConfiguredSites.html#pre-configured-sites-tier-1).

+---------------------+----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
| Organization | System | Compilers | Location | Maintainers |
| Organization | System | Compilers | Location | Maintainers (principal/backup)|
+=====================+==================================+=================+=========================================================================================================+===============================+
| **HPC platforms** |
+---------------------+----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
| MSU | Hercules GCC+OpenMPI recommended | GCC | ``/work/noaa/epic/role-epic/spack-stack/hercules/spack-stack-1.6.0/envs/ue-gcc12-openmpi416`` | Dom Heinzeller |
| MSU | Hercules GCC+OpenMPI recommended | GCC | ``/work/noaa/epic/role-epic/spack-stack/hercules/spack-stack-1.6.0/envs/ue-gcc12-openmpi416`` | EPIC / JCSDA |
+---------------------+----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
| | Hercules | (GCC), Intel | ``/work/noaa/epic/role-epic/spack-stack/hercules/spack-stack-1.6.0/envs/unified-env`` | Cam Book / Dom Heinzeller |
| | Hercules | (GCC), Intel | ``/work/noaa/epic/role-epic/spack-stack/hercules/spack-stack-1.6.0/envs/unified-env`` | EPIC / JCSDA |
| MSU +----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
| | Orion | GCC, Intel | ``/work/noaa/epic/role-epic/spack-stack/orion/spack-stack-1.6.0/envs/unified-env`` | Cam Book / Dom Heinzeller |
| | Orion | GCC, Intel | ``/work/noaa/epic/role-epic/spack-stack/orion/spack-stack-1.6.0/envs/unified-env`` | EPIC / JCSDA |
+---------------------+----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
| | Discover SCU16 | GCC, Intel | ``/gpfsm/dswdev/jcsda/spack-stack/scu16/spack-stack-20240228/envs/unified-env-*`` | Dom Heinzeller / ??? |
| | Discover SCU16 | GCC, Intel | ``/gpfsm/dswdev/jcsda/spack-stack/scu16/spack-stack-20240228/envs/unified-env-*`` | JCSDA |
| NASA +----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
| | Discover SCU17 | GCC, Intel | ``/gpfsm/dswdev/jcsda/spack-stack/scu17/spack-stack-20240228/envs/unified-env-*`` | Dom Heinzeller / ??? |
| | Discover SCU17 | GCC, Intel | ``/gpfsm/dswdev/jcsda/spack-stack/scu17/spack-stack-20240228/envs/unified-env-*`` | JCSDA |
+---------------------+----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
| | Casper | GCC | ``/glade/work/epicufsrt/contrib/spack-stack/casper/spack-stack-1.6.0/envs/unified-env`` | Dom Heinzeller / ??? |
| | Casper | GCC | ``/glade/work/epicufsrt/contrib/spack-stack/casper/spack-stack-1.6.0/envs/unified-env`` | JCSDA / EPIC |
| NCAR-Wyoming +----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
| | Derecho | GCC, Intel | ``/glade/work/epicufsrt/contrib/spack-stack/derecho/spack-stack-1.6.0/envs/unified-env`` | Dom Heinzeller / Cam Book |
| | Derecho | GCC, Intel | ``/glade/work/epicufsrt/contrib/spack-stack/derecho/spack-stack-1.6.0/envs/unified-env`` | EPIC / JCSDA |
+---------------------+----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
| NOAA (NCEP) | Acorn | Intel | ``/lfs/h1/emc/nceplibs/noscrub/spack-stack/spack-stack-1.6.0/envs/unified-env-intel{19,2022}`` | Hang Lei / Alex Richert |
| NOAA (NCEP) | Acorn | Intel | ``/lfs/h1/emc/nceplibs/noscrub/spack-stack/spack-stack-1.6.0/envs/unified-env-intel{19,2022}`` | NOAA-EMC |
+---------------------+----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
| | Gaea C5 | Intel | ``/ncrc/proj/epic/spack-stack/spack-stack-1.6.0/envs/unified-env`` | Cam Book / Dom Heinzeller |
| | Gaea C5 | Intel | ``/ncrc/proj/epic/spack-stack/spack-stack-1.6.0/envs/unified-env`` | EPIC / NOAA-EMC |
| +----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
| NOAA (RDHPCS) | Hera | GCC, Intel | ``/scratch1/NCEPDEV/nems/role.epic/spack-stack/spack-stack-1.6.0/envs/unified-env`` | Mark Potts / Dom Heinzeller |
| NOAA (RDHPCS) | Hera | GCC, Intel | ``/scratch1/NCEPDEV/nems/role.epic/spack-stack/spack-stack-1.6.0/envs/unified-env`` | EPIC / NOAA-EMC |
| +----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
| | Jet | GCC, Intel | ``/mnt/lfs4/HFIP/hfv3gfs/role.epic/spack-stack/spack-stack-1.6.0/envs/unified-env`` | Cam Book / Dom Heinzeller |
| | Jet | GCC, Intel | ``/mnt/lfs4/HFIP/hfv3gfs/role.epic/spack-stack/spack-stack-1.6.0/envs/unified-env`` | EPIC / NOAA-EMC |
+---------------------+----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
| | Narwhal | GCC, Intel | ``/p/app/projects/NEPTUNE/spack-stack/spack-stack-1.6.0/envs/unified-env-*`` | Dom Heinzeller / Sarah King |
| | Narwhal | GCC, Intel | ``/p/app/projects/NEPTUNE/spack-stack/spack-stack-1.6.0/envs/unified-env-*`` | JCSDA / NRL |
| +----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
| U.S. Navy (HPCMP) | Nautilus | Intel | ``/p/app/projects/NEPTUNE/spack-stack/spack-stack-1.6.0/envs/unified-env`` | Dom Heinzeller / Sarah King |
| U.S. Navy (HPCMP) | Nautilus | Intel | ``/p/app/projects/NEPTUNE/spack-stack/spack-stack-1.6.0/envs/unified-env`` | JCSDA / NRL |
| +----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
| | Nautilus | AOCC | *currently not supported* | Dom Heinzeller / Sarah King |
| | Nautilus | AOCC | *currently not supported* | JCSDA / NRL |
+---------------------+----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
| | S4 | Intel | ``/data/prod/jedi/spack-stack/spack-stack-1.6.0/envs/unified-env`` | Dom Heinzeller / Mark Potts |
| | S4 | Intel | ``/data/prod/jedi/spack-stack/spack-stack-1.6.0/envs/unified-env`` | JCSDA |
| Univ. of Wisconsin +----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
| | S4 | GCC | *currently not supported* | Dom Heinzeller / Mark Potts |
| | S4 | GCC | *currently not supported* | JCSDA |
+---------------------+----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
| **Cloud platforms** |
+---------------------+----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
| | AMI Red Hat 8 | GCC | ``/home/ec2-user/spack-stack/spack-stack-1.6.0/envs/unified-env`` | Dom Heinzeller / ??? |
| | AMI Red Hat 8 | GCC | ``/home/ec2-user/spack-stack/spack-stack-1.6.0/envs/unified-env`` | JCSDA |
+ Amazon Web Services +----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
| | Parallelcluster JCSDA R&D | Intel | ``/mnt/experiments-efs/skylab-v8/spack-stack-20240207/envs/unified-env-*`` | Dom Heinzeller / ??? |
| | Parallelcluster JCSDA R&D | Intel | ``/mnt/experiments-efs/skylab-v8/spack-stack-20240207/envs/unified-env-*`` | JCSDA |
+---------------------+----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+
| NOAA (RDHPCS) | RDHPCS Cloud (Parallel Works) | Intel | ``/contrib/spack-stack/spack-stack-1.6.0/envs/unified-env`` | Mark Potts / Cam Book / Dom H |
| NOAA (RDHPCS) | RDHPCS Cloud (Parallel Works) | Intel | ``/contrib/spack-stack/spack-stack-1.6.0/envs/unified-env`` | EPIC / JCSDA |
+---------------------+----------------------------------+-----------------+---------------------------------------------------------------------------------------------------------+-------------------------------+

For more information about a specific platform, please see the individual sections below.
Expand Down
4 changes: 2 additions & 2 deletions doc/source/Utilities.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -46,10 +46,10 @@ The utility located at util/check_permissions.sh can be run inside any spack-sta
.. _LDD_Checker:

------------------------------
ldd_check.py
ldd_check.py (Linux only)
------------------------------

The util/ldd_check.py utility should be run for new installations to ensure that no shared library or executable that uses shared libraries is missing a shared library dependency. If the script returns a warning for a given file, this may indicate that Spack's RPATH substitution has not been properly applied. In some instances, missing library dependencies may not indicate a problem, such as a library that is intended to be found through $LD_LIBRARY_PATH after, say, a compiler or MPI environment module is loaded. Though these paths should probably also be RPATH-ified, such instances of harmless missing dependencies may be ignored with ldd_check.py's ``--ignore`` option by specifying a Python regular expression to be excluded from consideration (see example below), or can be permanently whitelisted by modifying the ``whitelist`` variable at the top of the ldd_check.py script itself (in which case please submit a PR). The script searches the 'install/' subdirectory of a given path and runs ``ldd`` on all shared objects. The base path to be search can be specified as a lone positional argument, and by default is the current directory. In practice, this should be ``$SPACK_ENV`` for the environment in question.
The util/ldd_check.py utility should be run for new installations to ensure that no shared library or executable that uses shared libraries is missing a shared library dependency. If the script returns a warning for a given file, this may indicate that Spack's RPATH substitution has not been properly applied. In some instances, missing library dependencies may not indicate a problem, such as a library that is intended to be found through $LD_LIBRARY_PATH after, say, a compiler or MPI environment module is loaded. Though these paths should probably also be RPATH-ified, such instances of harmless missing dependencies may be ignored with ldd_check.py's ``--ignore`` option by specifying a Python regular expression to be excluded from consideration (see example below), or can be permanently whitelisted by modifying the ``whitelist`` variable at the top of the ldd_check.py script itself (in which case please submit a PR). The script searches the 'install/' subdirectory of a given path and runs ``ldd`` on all shared objects. The base path to be search can be specified as a lone positional argument, and by default is the current directory. In practice, this should be ``$SPACK_ENV`` for the environment in question. This utility is available for Linux only.

.. code-block:: console
Expand Down
Loading

0 comments on commit 420c254

Please sign in to comment.