WRF Initialization Workflow Using NCSS

The WRF initialization process for running non-idealized simulations can be done using the WRF Preprocessing System (WPS). The goal of the WPS is to combine information about atmospheric conditions, as well as terrain information, for use in running the WRF model. Three programs make up the WPS:

• UNGRIB
• GEOGRID
• METGRID

METGRID uses output from both UNGRIB and GEOGRID to create a file that can be used to initialize WRF; UNGRIB and GEOGRID are independent of each other. More details concerning the WPS workflow can be found here.

The WRF model can be used to run simulations over a variety of spatial scales, and can use information about atmospheric conditions from a number of points in time. Because of this, it can often be the case that users end up downloading more data than needed to "spin up" the model. For example, some users download gridded data that covers the entire globe, only to run the model over a small portion of the continental United States.

More often than not, this data is in the GRIB format, as this is the most convenient format for getting data into WPS, at least as thing currently stand. There are very limited options for subsetting such data while keeping the end result in GRIB (thus allowing for existing tools to get the data ready for use by WRF).

Thankfully, the data encoded within the GRIB messages are quickly extracted and put into a temporary file early in the WRF initialization process using the UNGRIB program. The temporary file follows the "WRF Intermediate File Format", documented here and here.

In an attempt to circumvent the UNGRIB step of WRF initialization to enable easy spatial and temporal subsetting of initialization data, this new feature of the TDS would allow users to utilize the netCDF Subset Service to request and transfer only the data that is needed to spin-up WRF. These data would need to be returned in the Intermediate File Format.

A few extensions to the TDS codebase would need to be made to make this a reality.

1. Allow the use of varsets in the TDS NCSS Service

   As things currently stand with NCSS API, users need to request individual variables (for example, var=var1&var=var2&var=var3...). This means each user would need to know exactly what variables are needed to initialize WRF for every dataset served by the TDS (or at least those datasets appropriate for initializing WRF). It would be ideal if users could simply request a set of variables appropriate for the given task. This group of variables, called a varset, could easily be used outside of the context of WRF initialization, but would certainly be needed for this feature to work smoothly for the end user. Two things would be needed to make this work:

   1. We would need a way for TDS administrators, on a per-dataset level, to define a group of variables that are needed to initialize WRF for a given dataset. This would need to happen within the TDS Configuration Catalogs. Something like:

      <datasetScan ...>
        <varsets>
          <varset name="wrfinit">
            <variable name="var1" />
            <variable name="var2" />
            <variable name="var3" />
          </varset>
          <varset name="upper_air">
            <variable name="Temperature_isobaric" />
            <variable name="Relative_humidity_isobaric" />
            <variable name="u-component_of_wind_isobaric" />
            <variable name="v-component_of_wind_isobaric" />
          </varset>
        </varsets>
      </datasetScan>

      or for a feature collection:

      <dataset ...>
        <featureCollection ...>
          <collection ... />
          <varsets>
            <varset name="wrfinit">
              <variable name="var1" />
              <variable name="var2" />
              <variable name="var3" />
            </varset>
            <varset name="upper_air">
              <variable name="Temperature_isobaric" />
              <variable name="Relative_humidity_isobaric" />
              <variable name="u-component_of_wind_isobaric" />
              <variable name="v-component_of_wind_isobaric" />
            </varset>
          </varsets>
          ...
        </featureCollection>
      </dataset">

   2. The NCSS API would need to be extended to include the option for a user to request a varset (for example, varset=wrfinit) in lieu of individual variables.

   3. The NCSS dataset.xml document will need to advertise the available varsets.

2. Allow NCSS to return data in the WRF Intermediate File format

   We currently allow the NCSS service to return data in a variety of formats, and we would need one more to accommodate this feature. This means we need new java code to take data contained within a Coverage object and write it out to a file following the WRF Intermediate File Format.

Once users have their initialization data in the WRF Intermediate File Format, the rest of WRF initialization pipeline can be utilized (that is, METGRID). Happy days.

Notes:

The build script for ungrib is hard-coded to use -fconvert=big-endian. In theory users don't change this. Also, in theory, this can be overwritten by environmental variables, and I know these can get downright crazy and out of control on users systems as they try to get various programs to work. For now, let's assume the intermediate format should be written as big-endian. In the future, it might be useful to allow NCSS to return both (using two different accept parameters?).

ungrib produces an intermediate file for each time step. This means that if a user requests multiple times from NCSS, we will need to return multiple files. Perhaps, then, we should always return a tar file which contains one or more intermediate format files? Compression will likely take too long (each intermediate file produced from GFS 0.25 degree output is ~800MB), so tar is probably best? Or, so we skip all of that and only allow single time requests for WRF initialization files from NCSS?