Update to compass v1.3.0-alpha.1 #793

xylar · 2024-03-14T11:05:52Z

Updates:

Albany and Trilinos to compass-2024-03-13 tags
- Updates landice tests to be compatible, see Remove "Build Type: Tpetra" from Landice tests #788
mache to 1.20.0, with updated support for several machines
ESMF to 8.6.0
mpas_tools to 0.32.0
SCORPIO to 2.6.0

Other Changes:

Add MOAB 5.5.1 (to spack build and conda env)
Removes pnetcdf conda recipe (has been on conda-forge for a long time)

Checklist

Document (in a comment titled Testing in this PR) any testing that was used to verify the changes

Testing

MPAS-Ocean with pr:

MALI with full_integration:

MPAS-Ocean with nonhydro - Failing due to problems detailed in #797

Deployed

MPAS-Ocean with pr:

MALI with full_integration:

xylar · 2024-03-14T11:09:54Z

@mcarlson801, @mperego,

When I try to build Albany with the compass-2024-03-13 tag on Chrysalis, I'm seeing:

CMake Error at tests/CMakeLists.txt:149 (IF):
  if given arguments:

    "VERSION_LESS" "2.7"

  Unknown arguments specified


-- Configuring incomplete, errors occurred!
See also "/lcrc/group/e3sm/ac.xylar/chrys_spack_tmp/ac.xasay-davis/spack-stage/spack-stage-albany-compass-2024-03-13-ee6kt34yhi6pk27i4r7gaojlye4sg2zp/spack-build-ee6kt34/CMakeFiles/CMakeOutput.log".
See also "/lcrc/group/e3sm/ac.xylar/chrys_spack_tmp/ac.xasay-davis/spack-stage/spack-stage-albany-compass-2024-03-13-ee6kt34yhi6pk27i4r7gaojlye4sg2zp/spack-build-ee6kt34/CMakeFiles/CMakeError.log".

Then, I'm seeing in CMakeError.log:

Performing C SOURCE FILE Test PARALLEL_EXODUS_SUPPORTED failed with the following output:
Change Dir: /lcrc/group/e3sm/ac.xylar/chrys_spack_tmp/ac.xasay-davis/spack-stage/spack-stage-albany-compass-2024-03-13-ee6kt34yhi6pk27i4r7gaojlye4sg2zp/spack-build-ee6kt34/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/gmake -f Makefile cmTC_cc8f4/fast && /usr/bin/gmake  -f CMakeFiles/cmTC_cc8f4.dir/build.make CMakeFiles/cmTC_cc8f4.dir/build
gmake[1]: Entering directory '/lcrc/group/e3sm/ac.xylar/chrys_spack_tmp/ac.xasay-davis/spack-stage/spack-stage-albany-compass-2024-03-13-ee6kt34yhi6pk27i4r7gaojlye4sg2zp/spack-build-ee6kt34/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_cc8f4.dir/src.c.o
/gpfs/fs1/soft/chrysalis/spack/opt/spack/linux-centos8-x86_64/gcc-9.2.0/openmpi-4.1.3-sxfyy4k/bin/mpicc -DPARALLEL_EXODUS_SUPPORTED -I/lcrc/group/e3sm/ac.xylar/chrys_spack_test/dev_compass_1_3_0_gnu_openmpi_albany/opt/spack/linux-rhel8-zen2/gcc-9.2.0/trilinos-for-albany-compass-2024-03-13-idguiqvi6zhx4rjckoxaxdpgnmlikjrh/include  -o CMakeFiles/cmTC_cc8f4.dir/src.c.o -c /lcrc/group/e3sm/ac.xylar/chrys_spack_tmp/ac.xasay-davis/spack-stage/spack-stage-albany-compass-2024-03-13-ee6kt34yhi6pk27i4r7gaojlye4sg2zp/spack-build-ee6kt34/CMakeFiles/CMakeTmp/src.c
/lcrc/group/e3sm/ac.xylar/chrys_spack_tmp/ac.xasay-davis/spack-stage/spack-stage-albany-compass-2024-03-13-ee6kt34yhi6pk27i4r7gaojlye4sg2zp/spack-build-ee6kt34/CMakeFiles/CMakeTmp/src.c:6:4: error: #error Missing pnetcdf
    6 |   #error Missing pnetcdf
      |    ^~~~~
gmake[1]: *** [CMakeFiles/cmTC_cc8f4.dir/build.make:78: CMakeFiles/cmTC_cc8f4.dir/src.c.o] Error 1
gmake[1]: Leaving directory '/lcrc/group/e3sm/ac.xylar/chrys_spack_tmp/ac.xasay-davis/spack-stage/spack-stage-albany-compass-2024-03-13-ee6kt34yhi6pk27i4r7gaojlye4sg2zp/spack-build-ee6kt34/CMakeFiles/CMakeTmp'
gmake: *** [Makefile:127: cmTC_cc8f4/fast] Error 2


Source file was:

  #include <exodusII_par.h>
  #if !NC_HAS_HDF5
  #error Missing hdf5
  #elif !NC_HAS_PNETCDF
  #error Missing pnetcdf
  #endif
  int main()
  {
    return 0;
  }

Is this something we need to somehow fix in the spack package for Albany?

xylar · 2024-03-14T11:42:35Z

Following up, I think the issue is that $PYTHON_EXECUTABLE and therefore $PYTHON_VERSION are not defined. I think the error from CMakeError.log above is not relevant.

mperego · 2024-03-14T18:05:32Z

Pinging @ikalash and @jewatkins for help with the spack builds.

mcarlson801 · 2024-03-14T18:06:53Z

@mcarlson801, @mperego,
Is this something we need to somehow fix in the spack package for Albany?

This isn't something I've run into or shown up on the nightlies and unfortunately I don't have access to Chrysalis to check it out. Is there somewhere we can take a look at how you are configuring? Maybe one of the other Albany developers who have run on Chrysalis can help. (I think this is probably Luca or Jerry but I'm not sure)

mperego · 2024-03-14T18:07:06Z

pinging @kliegeois as well.

ikalash · 2024-03-14T18:10:13Z

@xylar : are you having spack built everything including cmake and pnetcdf, the packages with the problem? Or you are pointing to existing ones? I agree with @mcarlson801 that it is difficult to troubleshoot this w/o having access to the machine, since these errors don't show up elsewhere.

xylar · 2024-03-14T18:15:49Z

This isn't something I've run into or shown up on the nightlies and unfortunately I don't have access to Chrysalis to check it out. Is there somewhere we can take a look at how you are configuring? Maybe one of the other Albany developers who have run on Chrysalis can help. (I think this is probably Luca or Jerry but I'm not sure)

@mcarlson801, this happend on Perlmutter the same way. It doesn't seem to be a machine issue.

xylar · 2024-03-14T18:18:01Z

@xylar : are you having spack built everything including cmake and pnetcdf, the packages with the problem? Or you are pointing to existing ones?

@ikalash, I think pnetcdf is a red herring. It's nothing to do with that. It's that things are failing when we pass the -DPYTHON_EXECUTABLE= argument (i.e. leaving it empty) when we don't want the python features.

Again, this does not seem to be machine specific. It happens to me on Perlmutter, too.

xylar · 2024-03-14T18:23:46Z

On Perlmutter, the failing Spack build of Albany is in:

/pscratch/sd/x/xylar/spack_tmp/spack-stage/spack-stage-albany-compass-2024-03-13-z5yq62pnbfqca4qb7oe6mu5abrd4oc3k

If you all have Perlmutter access, please have a look. Permissions should be open.

ikalash · 2024-03-14T18:30:23Z

I guess this is a PyAlbany thing, since that's the only place in Albany Python is needed, in which case @kliegeois may be the best one to take a look. You can try building w/o PyAlbany to see if there are any other errors:

spack --insecure install --dirty --keep-stage albany@develop%[email protected]+mpas+optimization+mesh_depends_on_params+omegah

(remote the +py).

xylar · 2024-03-14T18:44:49Z

Here's the full spec it's trying to build:

albany@compass-2024-03-13%[email protected]+64bit+confgui~debug+epetra~fpe~ipo+landice~mesh_depends_on_params+mpas~omegah~optimization~perf~py~sandybridge~sfad+unit_tests build_system=cmake build_type=Release generator=make sfadsize=4 arch=linux-sles15-zen3

It's not trying to build with +py, note the ~py.

xylar · 2024-03-14T18:46:04Z

I don't think it's related but I notices it's with +epetra even though all I explicitly specified was +mpas. That seems odd because epetra seems to be default disabled.

xylar · 2024-03-14T20:25:58Z

I was hoping you all had seen this elsewhere or had some quick intuition about it. I'll try to debug it some more tomorrow.

ikalash · 2024-03-15T17:34:18Z

I was hoping you all had seen this elsewhere or had some quick intuition about it. I'll try to debug it some more tomorrow.

Sorry @xylar this does not ring any bells. I will look at the PM output soon. I have been busy this week trying to get caught up after 2 weeks of foreign travel, while sick...

xylar · 2024-03-17T09:22:31Z

@ikalash, @mperego, @mcarlson801, @kliegeois, @matthewhoffman

The solution turns out to be pretty simple as far as I can tell:
E3SM-Project/spack#27
If PYTHON_EXECUTABLE is undefined, CMake tries to find python itself during testing. However, if it is defined as an empty string, this appears not to be the same as being undefined. It appears to attempt to find a version for python using an empty string as the executable, gets an empty string for the version, and fails to compare that version to 2.7.

Make sure to perform unit tests

This merge explicitly points to the libnetcdf that should be used in the Jigsaw build. It also breaks the jigsaw build into its own function.

This configuration can't be used to build MOAB.

xylar changed the title ~~Update to compass v1.3.0 alpha.1~~ Update to compass v1.3.0-alpha.1 Mar 14, 2024

xylar requested review from matthewhoffman and andrewdnolan March 14, 2024 11:55

xylar mentioned this pull request Mar 14, 2024

Remove "Build Type: Tpetra" from Landice tests #788

Closed

1 task

xylar force-pushed the update-to-1.3.0-alpha.1 branch from ebc9a6e to 96ea00d Compare March 14, 2024 17:26

This was referenced Mar 15, 2024

Update mpas_tools to 0.31.0 and compass to 1.2.0-alpha.10 #792

Closed

Only set Albany python exec. if +py E3SM-Project/spack#27

Merged

xylar force-pushed the update-to-1.3.0-alpha.1 branch from 81954bd to 6bab9e8 Compare March 17, 2024 09:25

xylar mentioned this pull request Mar 17, 2024

Update MALI-Dev submodule to latest develop #796

Merged

1 task

xylar force-pushed the update-to-1.3.0-alpha.1 branch from 858d073 to bd08435 Compare March 17, 2024 09:34

xylar mentioned this pull request Mar 18, 2024

nonhydro tests do not work with E3SM-Project submodule #797

Open

xylar added 4 commits March 20, 2024 11:09

Update to compass v1.3.0-alpha.1

a6e6116

Update mache to v1.20.0

2fe6255

Update albany to compass-2024-03-13

1641d02

Update spack library versions

9b2b85b

xylar and others added 13 commits March 20, 2024 11:09

Remove pnetcdf conda recipe

7333324

Update mpas_tools to v0.32.0

b7293f4

Remove Build Type: Tpetra from Landice tests

5bfd64e

Constrain ESMF from conda-forge

29e4e12

Add MOAB

02153b1

Update pyremap constrain

78bcbec

Don't install any dependencies from pip

62a4e5c

Install Albany without Python or EPetra support

cf788dd

Make sure to perform unit tests

Add a nonhydro test suite

d329868

Fix libnetcdf in jigsaw build

edcb7e4

This merge explicitly points to the libnetcdf that should be used in the Jigsaw build. It also breaks the jigsaw build into its own function.

Add docs on updating the Compass spack mirror

a04c20f

Look for spack mirror when creating a spack env

bc125a4

Add location of spack mirror on Chicoma

c4053e9

xylar force-pushed the update-to-1.3.0-alpha.1 branch from 33f67bc to c4053e9 Compare March 20, 2024 16:09

xylar and others added 2 commits March 20, 2024 11:36

Remove "~epetra" from the albany spack spec

db70967

Add chgrp cmd to documentation section about spack mirror.

db37615

xylar removed the request for review from matthewhoffman March 21, 2024 14:01

Remove gnu with mvapich from Anvil

45c0540

This configuration can't be used to build MOAB.

xylar removed the request for review from andrewdnolan March 24, 2024 10:27

xylar merged commit dd224f8 into MPAS-Dev:main Mar 24, 2024
4 checks passed

xylar deleted the update-to-1.3.0-alpha.1 branch March 24, 2024 10:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update to compass v1.3.0-alpha.1 #793

Update to compass v1.3.0-alpha.1 #793

xylar commented Mar 14, 2024 •

edited

Loading

xylar commented Mar 14, 2024

xylar commented Mar 14, 2024

mperego commented Mar 14, 2024

mcarlson801 commented Mar 14, 2024

mperego commented Mar 14, 2024

ikalash commented Mar 14, 2024

xylar commented Mar 14, 2024

xylar commented Mar 14, 2024

xylar commented Mar 14, 2024

ikalash commented Mar 14, 2024

xylar commented Mar 14, 2024

xylar commented Mar 14, 2024

xylar commented Mar 14, 2024

ikalash commented Mar 15, 2024

xylar commented Mar 17, 2024

Update to compass v1.3.0-alpha.1 #793

Update to compass v1.3.0-alpha.1 #793

Conversation

xylar commented Mar 14, 2024 • edited Loading

Updates:

Other Changes:

Testing

Deployed

xylar commented Mar 14, 2024

xylar commented Mar 14, 2024

mperego commented Mar 14, 2024

mcarlson801 commented Mar 14, 2024

mperego commented Mar 14, 2024

ikalash commented Mar 14, 2024

xylar commented Mar 14, 2024

xylar commented Mar 14, 2024

xylar commented Mar 14, 2024

ikalash commented Mar 14, 2024

xylar commented Mar 14, 2024

xylar commented Mar 14, 2024

xylar commented Mar 14, 2024

ikalash commented Mar 15, 2024

xylar commented Mar 17, 2024

xylar commented Mar 14, 2024 •

edited

Loading