Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update enkfgdas_update job resources in GFSv16.3.20 package #2972

Open
wants to merge 2 commits into
base: release/gfs.v16.3.20
Choose a base branch
from
Open

Update enkfgdas_update job resources in GFSv16.3.20 package #2972

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4b4cb01
Update enkfgdas_update job resources
KateFriedman-NOAA Oct 2, 2024
7058c3e
Update eupd resources for dev
KateFriedman-NOAA Oct 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion docs/Release_Notes.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ PRELUDE

The upstream OBSPROC package is updated to v1.3. Along with this, the GFS is switching to use the AFWA global snow file due to the hemispheric snow files being phased out.

Also included in this update is an improvement to the resource configuration for the enkfgdas_update job.

IMPLEMENTATION INSTRUCTIONS
---------------------------

Expand Down Expand Up @@ -95,7 +97,12 @@ Now ingest:
ENVIRONMENT AND RESOURCE CHANGES
--------------------------------

* No changes from GFS v16.3.19
The enkfgdas_update job resource settings and configuration are updated.
The `ecf/scripts/enkfgdas/analysis/create/jenkfgdas_update.ecf` script and
`parm/config/config.resources` config are updated to now use 320 tasks
and 12 threads, which results in 10 mpiprocs and 32 nodes (a reduction of
three nodes). The `env/WCOSS2.env` env file is also updated to use
`--cpu-bind core` instead of `--cpu-bind depth` for improved performance.

PRE-IMPLEMENTATION TESTING REQUIREMENTS
---------------------------------------
Expand Down
2 changes: 1 addition & 1 deletion ecf/scripts/enkfgdas/analysis/create/jenkfgdas_update.ecf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
#PBS -q %QUEUE%
#PBS -A %PROJ%-%PROJENVIR%
#PBS -l walltime=00:35:00
#PBS -l select=35:mpiprocs=9:ompthreads=14:ncpus=126
#PBS -l select=32:mpiprocs=10:ompthreads=12:ncpus=120
#PBS -l place=vscatter:exclhost
#PBS -l debug=true

Expand Down
3 changes: 2 additions & 1 deletion env/WCOSS2.env
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,13 +90,14 @@ elif [ $step = "eobs" ]; then

elif [ $step = "eupd" ]; then

export OMP_PROC_BIND=true
export OMP_PLACES=cores
export OMP_STACKSIZE=2G
export MPICH_COLL_OPT_OFF=1
export FI_OFI_RXM_SAR_LIMIT=3145728

export NTHREADS_ENKF=$nth_eupd
export APRUN_ENKF="$launcher -n ${npe_enkf:-${npe_eupd:-$PBS_NP}} -ppn $npe_node_eupd --cpu-bind depth --depth $NTHREADS_ENKF"
export APRUN_ENKF="$launcher -n ${npe_enkf:-${npe_eupd:-$PBS_NP}} -ppn $npe_node_eupd --cpu-bind core --depth $NTHREADS_ENKF"

export CFP_MP=${CFP_MP:-"NO"}
export USE_CFP=${USE_CFP:-"YES"}
Expand Down
8 changes: 4 additions & 4 deletions parm/config/config.resources.emc.dyn
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -367,8 +367,8 @@ elif [ $step = "eupd" ]; then
export npe_eupd=960
export nth_eupd=7
elif [[ "$machine" = "WCOSS2" ]]; then
export npe_eupd=315
export nth_eupd=14
export npe_eupd=320
export nth_eupd=12
fi
elif [ $CASE = "C384" ]; then
export npe_eupd=270
Expand All @@ -378,8 +378,8 @@ elif [ $step = "eupd" ]; then
elif [[ "$machine" = "HERA" ]]; then
export nth_eupd=8
elif [[ "$machine" = "WCOSS2" ]]; then
export npe_eupd=315
export nth_eupd=14
export npe_eupd=320
export nth_eupd=12
fi
elif [ $CASE = "C192" -o $CASE = "C96" -o $CASE = "C48" ]; then
export npe_eupd=42
Expand Down
4 changes: 2 additions & 2 deletions parm/config/config.resources.nco.static
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -287,8 +287,8 @@ elif [ $step = "ediag" ]; then
elif [ $step = "eupd" ]; then

export wtime_eupd="00:35:00"
export npe_eupd=315
export nth_eupd=14
export npe_eupd=320
export nth_eupd=12
export npe_node_eupd=$(echo "$npe_node_max / $nth_eupd" | bc)

elif [ $step = "ecen" ]; then
Expand Down