Skip to content

Commit

Permalink
move comments form config to readme
Browse files Browse the repository at this point in the history
  • Loading branch information
Haidong Yi committed Sep 27, 2024
1 parent 7059023 commit 62e4c8e
Show file tree
Hide file tree
Showing 2 changed files with 16 additions and 19 deletions.
20 changes: 2 additions & 18 deletions conf/stjude.config
Original file line number Diff line number Diff line change
Expand Up @@ -56,19 +56,7 @@ process {
}

// clusterOptions for gpu task:
// NOTE: gpu jobs are dispatched to HPCF's `gpu` queue
//
// To use GPU for your nextflow process:
// 1. GPU processes need to set the `accelerator` option in their directives. This can be configured by:
// 1. Adding directives in each process specifically
// 2. Using `process {}`-based label selectors (e.g. label 'process_gpu')
//
// 2. (optional) If you want to use docker (unsupported) or singularity (supported) on HPCF, you need to add the following container options:
// containerOptions = {
// workflow.containerEngine == "singularity" ? '--nv':
// ( workflow.containerEngine == "docker" ? '--gpus all': null )
// }

// NOTE: We use GPU exclusively in each job
clusterOptions = { task.accelerator ? "-gpu \"num=${task.accelerator}/host:mode=shared:j_exclusive=yes\"" : null }
}

Expand All @@ -79,11 +67,7 @@ singularity {
runOptions = '-p -B "$TMPDIR"'
pullTimeout = "3 hours"

// Directory to cache singularity images. Unavailable with an institution-level set-up.
// FIXME: This needs the support from the cluster administrator.
// Feel free to use your own directory by:
// 1. use `singularity.cacheDir = /path/to/your/singularity/cache` in your config.
// 2. set env variable: `export NXF_SINGULARITY_CACHE=/path/to/your/singularity/cache` in your shell's profile file (e.g. `~/<.bashrc/.zshrc>`).
// cacheDir = /path/to/your/singularity/cache
}

// clean the generated files in the working directory
Expand Down
15 changes: 14 additions & 1 deletion docs/stjude.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,14 @@ The configuration automatically selects the appropriate queue based on your task

## GPU Usage

In St. Jude's HPCF cluster, the GPU jobs will be dispatched the `gpu` queue.
To use GPUs in your pipeline:

1. Set the `accelerator` option in your process directives.
1. Set the `accelerator` option in your process directives. This can configured by:

- Adding directives in each process specifically, or
- Using `process {}`-based label selectors (e.g. label 'process_gpu')

2. (Optional) If you want to use docker (unsupported) or singularity (supported) on HPCF, you need to add the following container options:

```bash
Expand All @@ -66,6 +71,14 @@ Singularity is enabled by default with the following settings:
- Environment variables whitelist: SINGULARITY_TMPDIR, TMPDIR, CUDA_VISIBLE_DEVICES
- Pull timeout: 3 hours

### Singularity Cache Directory

There is no institution-level directory setup for caching singularity images. To set the cache directory, you
can try the following two ways:

1. Add `singularity.cacheDir = /path/to/your/singularity/cache` in your nextflow config file.
2. Set nextflow's environment variable: `export NXF_SINGULARITY_CACHE=/path/to/your/singularity/cache` in your shell's profile file (e.g. `~/<.bashrc/.zshrc>`).

> [!NOTE]
> You will need an account to use the HPC cluster at St. Jude Children's Research Hospital in order to run the pipeline. If in doubt, contact IT.
Expand Down

0 comments on commit 62e4c8e

Please sign in to comment.