From 4d1bf5266f00b35778ea47896f438e3ef612628d Mon Sep 17 00:00:00 2001 From: Kate Friedman Date: Wed, 20 Mar 2024 14:00:31 -0400 Subject: [PATCH] Updates to RTD documentation (#2418) Updates to the RTD documentation include: - Textual updates - Better definition of GDA subfolder structure - Added note about GDAur having been discontinued - Adjust copyright and author information - Fix Git version table and update its contents - Add "Table of Contents" header before table of contents on front page - Add AWS ICs path to init page - Add link to UFS_UTILS gdas_init RTD documentation - Add note about bash to `gw_setup.sh` section and added a warning block Refs #2395 --- docs/source/components.rst | 18 ++++++++++-------- docs/source/conf.py | 7 ++++--- docs/source/hpc.rst | 22 +++++++++++----------- docs/source/index.rst | 4 ++++ docs/source/init.rst | 8 +++++--- docs/source/setup.rst | 6 +++++- 6 files changed, 39 insertions(+), 26 deletions(-) diff --git a/docs/source/components.rst b/docs/source/components.rst index 98e76b467b..869ef89bab 100644 --- a/docs/source/components.rst +++ b/docs/source/components.rst @@ -28,7 +28,7 @@ Components included as submodules: * **GSI Monitor** (https://github.com/NOAA-EMC/GSI-Monitor): These tools monitor the GSI package's data assimilation, detecting and reporting missing data sources, low observation counts, and high penalty values * **GDAS** (https://github.com/NOAA-EMC/GDASApp): Jedi based Data Assimilation system. This system is currently being developed for marine Data Assimilation and in time will replace GSI for atmospheric data assimilation as well * **UFS UTILS** (https://github.com/ufs-community/UFS_UTILS): Utility codes needed for UFS-weather-model -* **wxflow** Collection of python utilities for weather workflows (https://github.com/NOAA-EMC/wxflow) +* **wxflow** (https://github.com/NOAA-EMC/wxflow): Collection of python utilities for weather workflows * **Verif global** (https://github.com/NOAA-EMC/EMC_verif-global): Verification package to evaluate GFS parallels. It uses MET and METplus. At this moment the verification package is limited to providing atmospheric metrics only .. note:: @@ -57,19 +57,20 @@ Data Observation data, also known as dump data, is prepared in production and then archived in a global dump archive (GDA) for use by users when running cycled experiments. The GDA (identified as ``$DMPDIR`` in the workflow) is available on supported platforms and the workflow system knows where to find the data. -* Hera: /scratch1/NCEPDEV/global/glopara/dump -* Orion/Hercules: /work/noaa/rstprod/dump -* Jet: /mnt/lfs4/HFIP/hfv3gfs/glopara/dump -* WCOSS2: /lfs/h2/emc/global/noscrub/emc.global/dump -* S4: /data/prod/glopara/dump +* Hera: ``/scratch1/NCEPDEV/global/glopara/dump`` +* Orion/Hercules: ``/work/noaa/rstprod/dump`` +* Jet: ``/mnt/lfs4/HFIP/hfv3gfs/glopara/dump`` +* WCOSS2: ``/lfs/h2/emc/global/noscrub/emc.global/dump`` +* S4: ``/data/prod/glopara/dump`` ----------------------------- Global Dump Archive Structure ----------------------------- -The global dump archive (GDA) mimics the structure of its production source: ``DMPDIR/CDUMP.PDY/[CC/atmos/]FILES`` +The global dump archive (GDA) mimics the structure of its production source: -The ``CDUMP`` is either gdas, gfs, or rtofs. All three contain production output for each day (``PDY``). The gdas and gfs folders are further broken into cycle (``CC``) and component (``atmos``). +* GDAS/GFS: ``DMPDIR/gdas[gfs].PDY/CC/atmos/FILES`` +* RTOFS: ``DMPDIR/rtofs.PDY/FILES`` The GDA also contains special versions of some datasets and experimental data that is being evaluated ahead of implementation into production. The following subfolder suffixes exist: @@ -81,6 +82,7 @@ The GDA also contains special versions of some datasets and experimental data th +--------+------------------------------------------------------------------------------------------------------+ | ur | Un-restricted versions of restricted files in production. Produced and archived on a 48hrs delay. | | | Some restricted datasets are unrestricted. Data amounts: restricted > un-restricted > non-restricted | +| | Limited availability. Discontinued producing mid-2023. | +--------+------------------------------------------------------------------------------------------------------+ | x | Experimental global datasets being evaluated for production. Dates and types vary depending on | | | upcoming global upgrades. | diff --git a/docs/source/conf.py b/docs/source/conf.py index 89526d9f69..81f231f6b0 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -13,13 +13,14 @@ import os import sys sys.path.insert(0, os.path.abspath('.')) - +from datetime import datetime # -- Project information ----------------------------------------------------- project = 'Global-workflow' -copyright = '2023, Kate Friedman, Walter Kolczynski, Rahul Mahajan, Lin Gan, Arun Chawla' -author = 'Kate Friedman, Walter Kolczynski, Rahul Mahajan, Lin Gan, Arun Chawla' +year = datetime.now().year +copyright = f"2015-{year} NOAA/NWS/NCEP/EMC" +author = 'Kate Friedman, Walter Kolczynski, Rahul Mahajan, Lin Gan, and numerous collaborators and contributors' # The full version, including alpha/beta/rc tags release = '0.1' diff --git a/docs/source/hpc.rst b/docs/source/hpc.rst index 3ce6a889d9..508597781d 100644 --- a/docs/source/hpc.rst +++ b/docs/source/hpc.rst @@ -72,23 +72,23 @@ Version It is advised to use Git v2+ when available. At the time of writing this documentation the default Git clients on the different machines were as noted in the table below. It is recommended that you check the default modules before loading recommended ones: -+---------+----------+---------------------------------------+ ++----------+----------+---------------------------------------+ | Machine | Default | Recommended | -+---------+----------+---------------------------------------+ ++----------+----------+---------------------------------------+ | Hera | v2.18.0 | default | -+---------+----------+---------------------------------------+ ++----------+----------+---------------------------------------+ | Hercules | v2.31.1 | default | -+---------+----------+---------------------------------------+ ++----------+----------+---------------------------------------+ | Orion | v1.8.3.1 | **module load git/2.28.0** | -+---------+----------+---------------------------------------+ ++----------+----------+---------------------------------------+ | Jet | v2.18.0 | default | -+---------+----------+---------------------------------------+ -| WCOSS2 | v2.26.2 | default or **module load git/2.29.0** | -+---------+----------+---------------------------------------+ ++----------+----------+---------------------------------------+ +| WCOSS2 | v2.35.3 | default | ++----------+----------+---------------------------------------+ | S4 | v1.8.3.1 | **module load git/2.30.0** | -+---------+----------+---------------------------------------+ -| AWS PW | v1.8.3.1 | default -+---------+----------+---------------------------------------+ ++----------+----------+---------------------------------------+ +| AWS PW | v1.8.3.1 | default | ++----------+----------+---------------------------------------+ ^^^^^^^^^^^^^ Output format diff --git a/docs/source/index.rst b/docs/source/index.rst index 4381488078..a5161789b3 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -27,6 +27,10 @@ General updates: NOAA employees and affiliates can join the gfs-announce distrib GitHub updates: Users should adjust their "Watch" settings for this repo so they receive notifications as they'd like to. Find the "Watch" or "Unwatch" button towards the top right of the `authoritative global-workflow repository page `_ and click it to adjust how you watch the repo. +================= +Table of Contents +================= + .. toctree:: :numbered: :maxdepth: 3 diff --git a/docs/source/init.rst b/docs/source/init.rst index 1b28d75584..f945494af8 100644 --- a/docs/source/init.rst +++ b/docs/source/init.rst @@ -51,6 +51,7 @@ Cold-start atmosphere-only cycled C96 deterministic C48 enkf (80 members) ICs ar Hera: /scratch1/NCEPDEV/global/glopara/data/ICSDIR/C96C48 Orion/Hercules: /work/noaa/global/glopara/data/ICSDIR/C96C48 WCOSS2: /lfs/h2/emc/global/noscrub/emc.global/data/ICSDIR/C96C48 + AWS: https://noaa-nws-global-pds.s3.amazonaws.com/index.html#data/ICSDIR/C96C48 Start date = 2021122018 @@ -111,6 +112,7 @@ Warm-start cycled w/ coupled (S2S) model C48 atmosphere C48 enkf (80 members) 5 Orion/Hercules: /work/noaa/global/glopara/data/ICSDIR/C48C48mx500 WCOSS2: /lfs/h2/emc/global/noscrub/emc.global/data/ICSDIR/C48C48mx500 Jet: /lfs4/HFIP/hfv3gfs/glopara/data/ICSDIR/C48C48mx500 + AWS: https://noaa-nws-global-pds.s3.amazonaws.com/index.html#data/ICSDIR/C48C48mx500 Start date = 2021032312 @@ -246,7 +248,7 @@ Automated Generation Cycled mode ----------- -Not yet supported. +Not yet supported. See the UFS_UTILS documentation on the gdas_init utility to generate your own ICs for cycled or forecast-only mode: https://noaa-emcufs-utils.readthedocs.io/en/latest/ufs_utils.html#gdas-init .. _forecastonly-coupled: @@ -319,14 +321,14 @@ Manual Generation The following information is for users needing to generate cold-start initial conditions for a cycled experiment that will run at a different resolution or layer amount than the operational GFS (C768C384L127). -The ``chgres_cube`` code is available from the `UFS_UTILS repository `_ on GitHub and can be used to convert GFS ICs to a different resolution or number of layers. Users should see the documentation to generation initial conditions in the UFS_UTILS repository. The ``chgres_cube`` code/scripts currently support the following GFS inputs: +The ``chgres_cube`` code is available from the `UFS_UTILS repository `_ on GitHub and can be used to convert GFS ICs to a different resolution or number of layers. Users should see the `documentation to generation initial conditions in the UFS_UTILS repository `_. The ``chgres_cube`` code/scripts currently support the following GFS inputs: * pre-GFSv14 * GFSv14 * GFSv15 * GFSv16 -See instructions in UFS_UTILS to clone, build and generate initial conditions. +See instructions in UFS_UTILS to clone, build and generate initial conditions: https://noaa-emcufs-utils.readthedocs.io/en/latest/ufs_utils.html#gdas-init .. _warmstarts-prod: diff --git a/docs/source/setup.rst b/docs/source/setup.rst index 0e87ade9a5..de5cfa099a 100644 --- a/docs/source/setup.rst +++ b/docs/source/setup.rst @@ -6,9 +6,13 @@ Experiment Setup :: - # Note: this will wipe your existing lmod environment source workflow/gw_setup.sh +.. warning:: + Sourcing gw_setup.sh will wipe your existing lmod environment + +.. note:: + Bash shell is required to source gw_setup.sh ^^^^^^^^^^^^^^^^^^^^^^^^ Forecast-only experiment