Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable module load, unload, ... invocations via wrapper #257

Conversation

SeanBryan51
Copy link
Collaborator

This change disables module commands that modify the environment via a wrapper when running a custom build script. This avoids preprocessing the custom build script for module statements which may produce an invalid bash script. This also allows for module commands that do not alter the environment to be executed as we only disable a subset of subcommands in the wrapper.

Fixes #249

@SeanBryan51 SeanBryan51 linked an issue Mar 1, 2024 that may be closed by this pull request
@SeanBryan51 SeanBryan51 force-pushed the 249-function-remove_module_lines-can-produce-an-invalid-bash-script branch 4 times, most recently from 859fa10 to 97407ef Compare March 1, 2024 02:50
Copy link

codecov bot commented Mar 1, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 72.38%. Comparing base (59910ed) to head (6d3436a).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #257      +/-   ##
==========================================
- Coverage   72.81%   72.38%   -0.43%     
==========================================
  Files          18       18              
  Lines         982      967      -15     
==========================================
- Hits          715      700      -15     
  Misses        267      267              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

benchcab/model.py Outdated Show resolved Hide resolved
@SeanBryan51 SeanBryan51 force-pushed the 249-function-remove_module_lines-can-produce-an-invalid-bash-script branch 3 times, most recently from c7d4787 to f799dce Compare March 21, 2024 21:49
@SeanBryan51 SeanBryan51 requested review from a team and ccarouge and removed request for a team March 21, 2024 21:55
This change disables module commands that modify the environment via a
wrapper when running a custom build script. This avoids preprocessing
the custom build script for module statements which may produce an
invalid bash script. This also allows for module commands that do not
alter the environment to be executed as we only disable a subset of
subcommands in the wrapper.

Fixes #249
@SeanBryan51 SeanBryan51 force-pushed the 249-function-remove_module_lines-can-produce-an-invalid-bash-script branch from f799dce to 6d3436a Compare March 22, 2024 00:06
@SeanBryan51
Copy link
Collaborator Author

Some problems that I've encountered:

  1. Scripts that use #!/bin/bash do not find the wrapper function but scripts that use #!/bin/sh do.
  2. Currently we source the environment_modules_wrapper.bash script to define the wrapper function when executing a custom build script. However custom build scripts may source system wide scripts and overwrite the wrapper function, e.g. the build script for the main branch of CABLE contains . /etc/bashrc which redefines the original module function.

@SeanBryan51
Copy link
Collaborator Author

After discussing with @ccarouge, we have decided to add support for spack and to deprecate support for legacy CABLE build systems in benchcab. This adds the requirement that users will only be able to test benchcab on branches that can be built by spack. ACCESS-NRI will help with transitioning legacy build systems to use the latest build system compatible with spack.

Moving to spack will solve existing issues around supporting bespoke build scripts (e.g. #244, #249). This is the best way forward considering support for legacy build systems is a temporary that will be deprecated sooner or later.

Closing this PR.

@SeanBryan51 SeanBryan51 deleted the 249-function-remove_module_lines-can-produce-an-invalid-bash-script branch March 24, 2024 22:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Function remove_module_lines can produce an invalid bash script
1 participant