Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow specifying compiler version for 'spack stack create env' #1294

Merged
merged 2 commits into from
Sep 11, 2024
Merged

Allow specifying compiler version for 'spack stack create env' #1294

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c8164ac
allow setting compiler version with --compiler
AlexanderRichert-NOAA Aug 1, 2024
c239551
allow compilers with version in matrix defs
AlexanderRichert-NOAA Aug 1, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 4 additions & 8 deletions spack-ext/lib/jcsda-emc/spack-stack/stack/stack_env.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ def __init__(self, **kwargs):

if not self.name:
# site = self.site if self.site else 'default'
self.name = "{}.{}.{}".format(self.template, self.site, self.compiler)
self.name = "{}.{}.{}".format(self.template, self.site, self.compiler.replace("@", "-"))

def env_dir(self):
"""env_dir is <dir>/<name>"""
Expand Down Expand Up @@ -170,7 +170,7 @@ def _copy_common_includes(self):
self._copy_or_merge_includes("modules", modules_yaml_path, modules_yaml_modulesys_path, destination)
# Merge or copy common package config(s)
packages_yaml_path = os.path.join(common_path, "packages.yaml")
packages_compiler_yaml_path = os.path.join(common_path, f"packages_{self.compiler}.yaml")
packages_compiler_yaml_path = os.path.join(common_path, f"packages_{self.compiler.split('@')[0]}.yaml")
destination = os.path.join(env_common_dir, "packages.yaml")
self._copy_or_merge_includes("packages", packages_yaml_path, packages_compiler_yaml_path, destination)

Expand Down Expand Up @@ -203,7 +203,7 @@ def _copy_site_includes(self):
self._copy_or_merge_includes("modules", modules_yaml_path, modules_yaml_modulesys_path, destination)
# Merge or copy site package config(s)
packages_yaml_path = os.path.join(env_path, "packages.yaml")
packages_compiler_yaml_path = os.path.join(env_path, f"packages_{self.compiler}.yaml")
packages_compiler_yaml_path = os.path.join(env_path, f"packages_{self.compiler.split('@')[0]}.yaml")
destination = os.path.join(env_site_dir, "packages.yaml")
self._copy_or_merge_includes("packages", packages_yaml_path, packages_compiler_yaml_path, destination)

Expand Down Expand Up @@ -263,11 +263,7 @@ def write(self):
target_compiler = f"%{self.compiler}"
for i in range(len(definitions)):
if "compilers" in definitions[i]:
j = len(definitions[i]["compilers"])-1
while j>=0:
if not definitions[i]["compilers"][j] == target_compiler:
definitions[i]["compilers"].pop(j)
j -= 1
definitions[i] = {"compilers": [target_compiler]}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is so much easier; do you remember why we put that complicated logic in place originally (I don't)?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No I don't remember... Maybe something to do with supporting multiple compilers in an environment..?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh right. This is something we need to test before we merge. The Intel or oneAPI environments require some packages to be built with gcc, therefore the compilers line in the spack config must contain both compilers. See

spack-stack/configs/sites/tier1/nautilus/packages_intel.yaml

Line 3 in a8bb4fb

compiler:: [[email protected],[email protected]]

We need to make sure that this still works (i.e. a spack module [tcl|lmod] refresh followed by spack stack setup-meta-modules still works as expected. I think it does, but better to check.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. I think we would expect it to work though, on account of there being only one value for target_compiler (based on self.compiler)? Unless users can specify multiple...

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It works. See "test env" step in https://github.com/JCSDA/spack-stack/actions/runs/10816758746/job/30008609091?pr=1294 (look for ecflow)

spack.config.set("definitions", definitions, scope=env_scope)

if self.install_prefix:
Expand Down
Loading