Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

#140 - Alternate config_options style #137

Merged
merged 6 commits into from
Sep 26, 2023
Merged

#140 - Alternate config_options style #137

Changes from 5 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
1d1aa8e
#135 - Alternate config_options style
ccarouge Sep 11, 2023
390db68
#140 - Finish first reformatting
ccarouge Sep 15, 2023
b6a5980
Merge branch 'master' into 135-polish-documentation
ccarouge Sep 15, 2023
6688edf
#140 - remove a level of heading and reorganise options
ccarouge Sep 15, 2023
17df79b
140 - Add link to subheading for better readability
ccarouge Sep 15, 2023
efc9b36
#140- response to review
ccarouge Sep 22, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
290 changes: 218 additions & 72 deletions docs/user_guide/config_options.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ The different running modes of `benchcab` are solely dependent on the options us
For this test, you want to:

* Specify the details of two branches of CABLE
* Do not specify a [`patch`](#`patch`)
* Do not specify a [`patch`](#+realisation.patch)
* Use the default set of science options, i.e. do not specify [`science_configurations`](#`science_configurations`) in `config.yaml`
* Choose the [`experiment`](#`experiment`) suitable for your stage of development. A run with the `forty-two-site-test` will be required for submissions of new development to CABLE.

Expand All @@ -19,7 +19,7 @@ The different running modes of `benchcab` are solely dependent on the options us
For this test, you want to:

* Specify the details of two branches of CABLE
* Specify a [`patch`](#`patch`) for **one** of the branches
* Specify a [`patch`](#+realisation.patch) for **one** of the branches
* Use the default set of science options, i.e. do not specify [`science_configurations`](#`science_configurations`) in `config.yaml`
* Choose the [`experiment`](#`experiment`) suitable for your stage of development. A run with the `forty-two-site-test` will be required for submissions of new development to CABLE.

Expand All @@ -29,107 +29,144 @@ The different running modes of `benchcab` are solely dependent on the options us
This running mode is quite open to customisations:

* Specify the number of CABLE's branches you need
* Use [`patch`](#`patch`) on branches as required
* Specify the [science configurations](#`science_configurations`) you want to run. [`patch`](#`patch`) will be applied on top of the science configurations listed.
* Use [`patch`](#+realisation.patch) on branches as required
* Specify the [science configurations](#`science_configurations`) you want to run. [`patch`](#+realsation.patch) will be applied on top of the science configurations listed.


## Technical options
## project

### `project`
: **Default:** _required option, no default_. :octicons-dash-24: NCI project ID to charge the simulations to.

: NCI project ID to charge the simulations to.
``` yaml

### `modules`
project: nf33

: NCI modules to use for compiling CABLE
```

### `fluxsite`
: Contains settings specific to fluxsite tests.
: This key is **optional**. Default settings for the fluxsite tests will be used if it is not present
## modules

#### `pbs`
: **Default:** _required option, no default_. :octicons-dash-24: NCI modules to use for compiling CABLE

: Contains settings specific to the PBS scheduler at NCI for the PBS script running the CABLE simulations at FLUXNET sites and the bitwise comparison for these simulations.
: This key is **optional**. Default values for the PBS settings will apply if it is not specified.
``` yaml

##### `ncpus`
modules: [
intel-compiler/2021.1.1,
netcdf/4.7.4,
openmpi/4.1.0
]

: The number of CPU cores to allocate for the PBS job, i.e. the `-l ncpus=<4>` PBS flag (see [PBS Directives Explained][nci-pbs-directives]).
: This key is **optional** and can be omitted from the config file. By default `ncpus` is set to `18`.
```

##### `mem`
## experiment

: The total memory limit for the PBS job, i.e. the `-l mem=<10GB>` PBS flag (see [PBS Directives Explained][nci-pbs-directives]).
: This key is **optional** and can be omitted from the config file. By default `mem` is set to `30GB`.
: **Default:** _required option, no default_. :octicons-dash-24: Type of experiment to run. Experiments are defined in the **benchcab-evaluation** workspace on [modelevaluation.org][meorg]. This key specifies the met forcing files used in the test suite. To choose from:

##### `walltime`
- [`forty-two-site-test`][forty-two-me]: to run simulations using 42 FLUXNET sites
ccarouge marked this conversation as resolved.
Show resolved Hide resolved
- [`five-site-test`][five-me]: to run simulations using 5 FLUXNET sites
- `AU-Tum`: to run simulations at the Tumbarumba (AU) site
- `AU-How`: to run simulations at the Howard Spring (AU) site
- `FI-Hyy`: to run simulations at the Hyytiala (FI) site
- `US-Var`: to run simulations at the Vaira Ranch-Ione (US) site
- `US-Whs`: to run simulations at the Walnut Gulch Lucky Hills Shrub (US) site

: The wall clock time limit for the PBS job, i.e. `-l walltime=<HH:MM:SS>` PBS flag (see [PBS Directives Explained][nci-pbs-directives]).
: This key is **optional** and can be omitted from the config file. By default `walltime` is set to `6:00:00`.
```yaml

##### `storage`
experiment: "AU-How"

: A list of extra storage flags required for the PBS job, i.e. `-l storage=<scratch/a00>` (see [PBS Directives Explained][nci-pbs-directives]).
: This key is **optional** and can be omitted from the config file. By default `storage` is set to `[]`.
```

#### `multiprocess`
## fluxsite
Contains settings specific to fluxsite tests.

: Enables or disables multiprocessing for executing embarrassingly parallel tasks.
: This key is **optional** and can be omitted from the config file. By default `multiprocess` is set to `True`.
This key is _optional_. **Default** settings for the fluxsite tests will be used if it is not present

Example:
```yaml
fluxsite:
pbs:
ncpus: 16
mem: 64GB
walltime: 00:01:00
ncpus: 18
mem: 30GB
walltime: 06:00:00
storage: [scratch/a00, gdata/xy11]
multiprocess: True
```

## Simulations options
### [pbs](#pbs)

### `realisations`
Contains settings specific to the PBS scheduler at NCI for the PBS script running the CABLE simulations at FLUXNET sites and the bitwise comparison for these simulations.

: Entries for each CABLE branch to use. Each entry is a dictionary, `{}`, that contains the following keys:
This key is _optional_. **Default** values for the PBS settings will apply if it is not specified.

#### `path`
```yaml
fluxsite:
pbs:
ncpus: 18
mem: 30GB
walltime: 06:00:00
storage: [scratch/a00, gdata/xy11]
```

: The path of the branch relative to the SVN root of the CABLE repository (`https://trac.nci.org.au/svn/cable`).
: Example:
[`ncpus`](#+pbs.ncpus){ #+pbs.ncpus }

- to checkout `https://trac.nci.org.au/svn/cable/trunk`, set `path: "trunk"`
- to checkout `https://trac.nci.org.au/svn/cable/branches/Users/foo/my_branch`, set `path: "branches/Users/foo/my_branch"`
: **Default:** 18, _optional key_. :octicons-dash-24: The number of CPU cores to allocate for the PBS job, i.e. the `-l ncpus=<4>` PBS flag in [PBS Directives Explained][nci-pbs-directives].

#### `name`
```yaml

: An alias name used internally by `benchcab` for the branch. The `name` key also specifies the directory name when checking out the branch, that is, `name` is the optional `PATH` argument to `svn checkout`.
: This key is **optional** and can be omitted from the config file. By default `name` is set to the base name of [`path`](#`path`).
fluxsite:
pbs:
ncpus: 18

#### `build_script`
```

: This key is **optional**. The path to a custom shell script to build the code in that branch, relative to the name of the branch. **Note:** any lines in the provided shell script that call the [environment modules API][environment-modules] will be ignored. To specify modules to use for the build script, please specify them using the [`modules`](#`modules`) key.
: Example: `build_script: offline/build.sh` to specify a build script under `<name_of_branch>/offline/`.
[`mem`](#+pbs.mem){ #+pbs.mem }

#### `revision`
: **Default:** 30GB, _optional key_. :octicons-dash-24: The total memory limit for the PBS job, i.e. the `-l mem=<10GB>` PBS flag in [PBS Directives Explained][nci-pbs-directives].

: The revision number to use for the branch.
: This key is **optional** and can be omitted from the config file (in which case the HEAD of the branch is used).
```yaml

#### `patch`
fluxsite:
pbs:
mem: 30GB

: Branch-specific namelist settings for `cable.nml`. Settings specified in `patch` get "patched" to the base namelist settings used for both branches. Any namelist settings specified here will overwrite settings defined in the default namelist file and in the science configurations. This means these settings will be set as stipulated in the `patch` for this branch for all science configurations run by `benchcab`.
: The `patch` key must be a dictionary-like data structure that is compliant with the [`f90nml`][f90nml-github] python package.
: This key is **optional** and can be omitted from the config file (in which case `patch` does not modify the namelist file).
```

[`walltime`](#+pbs.walltime){ #+pbs.walltime }

: **Default:** `6:00:00`, _optional key_. :octicons-dash-24: The wall clock time limit for the PBS job, i.e. `-l walltime=<HH:MM:SS>` PBS flag in [PBS Directives Explained][nci-pbs-directives].

```yaml

fluxsite:
pbs:
walltime: 6:00:00

```
[`storage`](#+pbs.storage){ #+pbs.storage }

: **Default:** [], _optional key_. :octicons-dash-24: List of extra storage flags required for the PBS job, i.e. `-l storage=<scratch/a00>` in [PBS Directives Explained][nci-pbs-directives].

```yaml

fluxsite:
pbs:
storage: [scratch/a00, gdata/xy11]

#### `patch_remove`
```

### [multiprocess](#multiprocess)

: Specifies branch-specific namelist settings to be removed from the `cable.nml` namelist settings. When the `patch_remove` key is specified, the specified namelists are removed from all namelist files for this branch for all science configurations run by `benchcab`.
: The `patch_remove` key must be a dictionary-like data structure that is compliant with the [`f90nml`][f90nml-github] python package. Note, when specifying a namelist parameter in `patch_remove`, the value of the namelist parameter is ignored.
: This key is **optional** and can be omitted from the config file (in which case `patch_remove` does not modify the namelist file).
: **Default:** True, _optional key_. :octicons-dash-24: Enables or disables multiprocessing for executing embarrassingly parallel tasks.

```yaml

fluxsites:
multiprocess: True

```

## realisations

Entries for each CABLE branch to use. Each entry is a dictionary, `{}`, that contains the following keys.

Example:
```yaml
realisations: [
{ # head of the trunk
Expand All @@ -153,22 +190,131 @@ realisations: [
]
```

### `experiment`
[`path`](#+realisation.path){ #+realisation.path }

: Type of experiment to run. Experiments are defined in the **benchcab-evaluation** workspace on [modelevaluation.org][meorg]. This key specifies the met forcing files used in the test suite. To choose from:
: **Default:** _required option, no default_. :octicons-dash-24: The path of the branch relative to the SVN root of the CABLE repository (`https://trac.nci.org.au/svn/cable`).

- [`forty-two-site-test`][forty-two-me]: to run simulations using 42 FLUXNET sites
- [`five-site-test`][five-me]: to run simulations using 5 FLUXNET sites
- `AU-Tum`: to run simulations at the Tumbarumba (AU) site
- `AU-How`: to run simulations at the Howard Spring (AU) site
- `FI-Hyy`: to run simulations at the Hyytiala (FI) site
- `US-Var`: to run simulations at the Vaira Ranch-Ione (US) site
- `US-Whs`: to run simulations at the Walnut Gulch Lucky Hills Shrub (US) site
```yaml
realisations: [
{ # head of the trunk
path: "trunk", # (1)
},
{ # some development branch
path: "branches/Users/foo/my_branch", # (2)
}
]

```

1. To checkout `https://trac.nci.org.au/svn/cable/trunk`
2. To checkout `https://trac.nci.org.au/svn/cable/branches/Users/foo/my_branch`

[`name`](#+realisation.name){ #+realisation.name }

: **Default:** base name of [path](#+realisation.path), _optional key_. :octicons-dash-24: An alias name used internally by `benchcab` for the branch. The `name` key also specifies the directory name when checking out the branch, that is, `name` is the optional `PATH` argument to `svn checkout`.

```yaml
realisations: [
{ # head of the trunk
path: "trunk",
},
{ # some development branch
path: "branches/Users/foo/my_branch",
name: "my_feature", # (1)
}
]

```

1. Checkout the branch in the directory `src/my_feature`

[`build_script`](#+realisation.build_script){ #+realisation.build_script }

: **Default:** unset, _optional key_. :octicons-dash-24: The path to a custom shell script to build the code in that branch, relative to the name of the branch. **Note:** any lines in the provided shell script that call the [environment modules API][environment-modules] will be ignored. To specify modules to use for the build script, please specify them using the [`modules`](#modules) key.

```yaml
realisations: [
{ # head of the trunk
path: "trunk",
},
{ # some development branch
path: "branches/Users/foo/my_branch",
build_script: "offline/build.sh", # (1)
}
]
```

1. Uses the build script stored by `benchcab` as `src/my_branch/offline/build.sh`

[`revision`](#+realisation.revision){ #+realisation.revision }

: **Default:** HEAD of the branch is checked out, _optional key_. :octicons-dash-24: The revision number to checkout for the branch. This option can be used to ensure the reproducibility of the tests.

```yaml
realisations: [
{ # head of the trunk
path: "trunk",
},
{ # some development branch
path: "branches/Users/foo/my_branch",
revision: 439,
}
]
```

[`patch`](#+realisation.patch){ #+realisation.patch }

: **Default:** unset, _optional key_. :octicons-dash-24: Branch-specific namelist settings for `cable.nml`. Settings specified in `patch` get "patched" to the base namelist settings used for both branches. Any namelist settings specified here will overwrite settings defined in the default namelist file and in the science configurations. This means these settings will be set as stipulated in the `patch` for this branch for all science configurations run by `benchcab`.
: The `patch` key must be a dictionary-like data structure that is compliant with the [`f90nml`][f90nml-github] python package.

```yaml
realisations: [
{ # head of the trunk
path: "trunk",
},
{ # some development branch
path: "branches/Users/foo/my_branch",
patch: { # (1)
cable: {
cable_user: {
FWSOIL_SWITCH: "Lai and Ktaul 2000",
}
}
}
}
]
```

1. Sets FWSOIL_SWITCH to "Lai and Ktaul 2000" for all science configurations for this branch

[`patch_remove`](#+realisation.path_remove){#+realisation.patch_remove}

: **Default:** unset, no effect, _optional key. :octicons-dash-24: Specifies branch-specific namelist settings to be removed from the `cable.nml` namelist settings. When the `patch_remove` key is specified, the specified namelists are removed from all namelist files for this branch for all science configurations run by `benchcab`. When specifying a namelist parameter in `patch_remove`, the value of the namelist parameter is ignored.
: The `patch_remove` key must be a dictionary-like data structure that is compliant with the [`f90nml`][f90nml-github] python package.

```yaml
realisations: [
{ # head of the trunk
path: "trunk",
},
{ # some development branch
path: "branches/Users/foo/my_branch",
patch_remove: {
cable: {
soilparmnew: nil, # (1)
}
}
}
]
```

1. The value is ignored and does not have to be a possible value for the namelist option.


## science_configurations

### `science_configurations`
: **Default:** unset, no impact, _optional key_. :octicons-dash-24: User defined science configurations. Science configurations that are specified here will replace [the default science configurations](default_science_configurations.md). In the output filenames, each configuration is identified with S<N\> where N is an integer starting from 0 for the first listed configuration and increasing by 1 for each subsequent configuration.

: User defined science configurations. This key is **optional** and can be omitted from the config file. Science configurations that are specified here will replace [the default science configurations](default_science_configurations.md). In the output filenames, each configuration is identified with S<N\> where N is an integer starting from 0 for the first listed configuration and increasing by 1 for each subsequent configuration.
: Example:
```yaml
science_configurations: [
{ # S0 configuration
Expand Down