Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adjust C768 resources for Hera #2819

Merged
merged 26 commits into from
Sep 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
615350a
Add level-by-level fix for netcdf_io from gsi_utils
DavidHuber-NOAA Aug 9, 2024
9a024e3
Add more write groups to spread memory requirements
DavidHuber-NOAA Aug 9, 2024
fbd0660
Reduce eupd tasks/threads on Hera
DavidHuber-NOAA Aug 9, 2024
17efba2
Merge remote-tracking branch 'origin/develop' into fix/c768_hera
DavidHuber-NOAA Aug 9, 2024
49a4cd3
Renamed custom eupd resources
DavidHuber-NOAA Aug 9, 2024
f9fcc89
Correct resources for eupd, upp jobs @C768
DavidHuber-NOAA Aug 15, 2024
a961da1
Merge branch 'develop' into fix/c768_hera
DavidHuber-NOAA Aug 20, 2024
10f7c9b
Update parm/config/gfs/config.resources.HERA
DavidHuber-NOAA Aug 22, 2024
0491fca
Merge remote-tracking branch 'emc/develop' into fix/c768_hera
DavidHuber-NOAA Aug 26, 2024
80c5b55
Merge remote-tracking branch 'emc/develop' into fix/c768_hera
DavidHuber-NOAA Sep 4, 2024
6ba5131
Update gsi_utils hash
DavidHuber-NOAA Sep 6, 2024
7b0619e
Merge branch 'develop' into fix/c768_hera
DavidHuber-NOAA Sep 9, 2024
5376d2a
Merge remote-tracking branch 'origin/develop' into fix/c768_hera
DavidHuber-NOAA Sep 9, 2024
7099248
Point to NOAA-EMC gsi-utils
DavidHuber-NOAA Sep 9, 2024
f34bf73
Merge branch 'develop' into fix/c768_hera
DavidHuber-NOAA Sep 10, 2024
27414cc
Merge remote-tracking branch 'origin/develop' into fix/c768_hera
DavidHuber-NOAA Sep 11, 2024
857db41
Experimental update to gsi_utils
DavidHuber-NOAA Sep 11, 2024
8c9a1ad
Update to experimental gsi_utils to fix omga issue
DavidHuber-NOAA Sep 12, 2024
ec0803d
Merge remote-tracking branch 'emc/develop' into fix/c768_hera
DavidHuber-NOAA Sep 16, 2024
9db6eea
Fix Hera C384 GSI analysis resources
DavidHuber-NOAA Sep 16, 2024
6f21ec5
Fix analysis tasks/node variable name for S4
DavidHuber-NOAA Sep 16, 2024
6041793
Handle missing netcdf variables in gsi-utils
DavidHuber-NOAA Sep 17, 2024
112964a
Merge remote-tracking branch 'emc/develop' into fix/c768_hera
DavidHuber-NOAA Sep 17, 2024
1921288
Merge branch 'develop' into fix/c768_hera
DavidHuber-NOAA Sep 18, 2024
62ea389
Disable tracker and genesis for orion and hercules
DavidHuber-NOAA Sep 19, 2024
84f3c42
Merge branch 'develop' into fix/c768_hera
DavidHuber-NOAA Sep 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions jobs/JGFS_ATMOS_CYCLONE_GENESIS
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,6 @@
source "${HOMEgfs}/ush/preamble.sh"
source "${HOMEgfs}/ush/jjob_header.sh" -e "genesis" -c "base genesis"

# Hack to temporary skip this as the tracker has not been build
# on Hercules Rocky 9 yet
# TODO: Remove this after tracker has been built for Rocky 9 #2639
if [[ "${machine}" == 'HERCULES' ]]; then exit 0; fi

##############################################
# Set variables used in the exglobal script
##############################################
Expand Down
5 changes: 0 additions & 5 deletions jobs/JGFS_ATMOS_CYCLONE_TRACKER
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,6 @@
source "${HOMEgfs}/ush/preamble.sh"
source "${HOMEgfs}/ush/jjob_header.sh" -e "tracker" -c "base tracker"

# Hack to temporary skip this as the tracker has not been build
# on Hercules Rocky 9 yet
# TODO: Remove this after tracker has been built for Rocky 9 #2639
if [[ "${machine}" == 'HERCULES' ]]; then exit 0; fi

export COMPONENT="atmos"


Expand Down
7 changes: 7 additions & 0 deletions parm/config/gfs/config.base
Original file line number Diff line number Diff line change
Expand Up @@ -492,4 +492,11 @@ if [[ "${machine}" =~ "PW" ]]; then
export DO_WAVE="NO"
fi

# The tracker and genesis are not installed on Orion/Hercules yet; this requires spack-stack builds of the package.
# TODO: we should place these in workflow/hosts/[orion|hercules].yaml.
if [[ "${machine}" == "ORION" || "${machine}" == "HERCULES" ]]; then
export DO_TRACKER="NO"
export DO_GENESIS="NO"
fi

echo "END: config.base"
16 changes: 14 additions & 2 deletions parm/config/gfs/config.resources.HERA
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,9 @@
case ${step} in
"anal")
if [[ "${CASE}" == "C384" ]]; then
export ntasks=270
export threads_per_task_anal=8
export ntasks_gdas=270
export ntasks_gfs=270
export threads_per_task=8
export tasks_per_node=$(( max_tasks_per_node / threads_per_task ))
fi
;;
Expand All @@ -26,6 +27,10 @@ case ${step} in

"eupd")
case ${CASE} in
"C768")
export ntasks=80
export threads_per_task=20
;;
"C384")
export ntasks=80
;;
Expand All @@ -43,6 +48,13 @@ case ${step} in
export tasks_per_node=$(( max_tasks_per_node / threads_per_task ))
;;

"upp")
if (( "${CASE:1}" >= 768 )); then
WalterKolczynski-NOAA marked this conversation as resolved.
Show resolved Hide resolved
# Run fewer tasks per node for memory
tasks_per_node=20
fi
;;

*)
;;
esac
2 changes: 1 addition & 1 deletion parm/config/gfs/config.resources.S4
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ case ${step} in
*)
;;
esac
export tasks_node=$(( max_tasks_per_node / threads_per_task ))
export tasks_per_node=$(( max_tasks_per_node / threads_per_task ))
;;

"eobs")
Expand Down
2 changes: 1 addition & 1 deletion parm/config/gfs/config.ufs
Original file line number Diff line number Diff line change
Expand Up @@ -281,7 +281,7 @@ case "${fv3_res}" in
export rf_cutoff=100.0
export fv_sg_adj=450
export WRITE_GROUP_GDAS=2
export WRTTASK_PER_GROUP_PER_THREAD_PER_TILE_GDAS=10
export WRTTASK_PER_GROUP_PER_THREAD_PER_TILE_GDAS=15
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@junwang-noaa might have a recommendation for what some of the settings should be for this resolution, but it can be addressed in a separate PR.

export WRITE_GROUP_GFS=4
export WRTTASK_PER_GROUP_PER_THREAD_PER_TILE_GFS=20 #Note this should be 10 for WCOSS2
fi
Expand Down
Loading