Skip to content

Latest commit

 

History

History
123 lines (99 loc) · 5.19 KB

README.md

File metadata and controls

123 lines (99 loc) · 5.19 KB
Raw

Table of Contents

Test data

This directory contains data to be used by the tests in ../autotest. The generate/ directory provides programs to generate test data not included in the repository but which are critical to the autotests.

Domain descriptions

The following are general descriptions of the test data domain directories.

hru_1

This is the second (index 1 in python) HRU in the NHM CONUS domain. It is meant to be small spatially but long in time. The record is 40 years. It tests the NHM configuration with and without depression storage.

drb_2yr

A test domain of smaller spatial size and 2 years duration. It tests the NHM configuration with and without depression storage.

ucb_2yr

A test domain of larger spatial size and 2 years duration. It tests the NHM configuration with and without depression storage.

sagehen_5yr

A test domain of small size and a 5 year duration. It will eventually tests the cascades functionality in pywatershed. With cascades turned off (the sagehen_no_cascades.control file) the domain tests depression storage off, dunnian flow and sroff in the soilzone, preferential flow, and pptmix in the canopy (only depression storage off is tested by the other domains and configurations).

Generating data

Generating test data should now be done from the autotest/ directory with the following command:

python generate_test_data.py -n=auto

or as described in the autotest/README.md.Should problems arise with the above, additional details are provided below.

In general, autotest/generate_test_data.py is a wrapper that calls the scripts in test_data/generate. It as a few of its own arguments and passes all other arguments along to pytest. The only additional thing that generate_test_data.py handles are dot files like test_data/.test_data_version_{domain_name}.txt which indicate by their existence that test data has been generated for a domain and by their contents the version.txt file from the root of the repository which was available when created.

Scripts in the test_data/generate/ subdirectory, which create and destroy test data are:

  • remove_output_dirs.py
  • run_prms_domains.py
  • convert_prms_output_to_nc.py
  • remove_prms_csvs.py

The first and last remove data and are generally only used to support testing in CI. The "run" and "convert" are the core scripts for generating the test data.

The run_prms_domains.py scripts uses binaries provided in the pywatershed repository to run reference models, specifically the PRMS v5.2.1 binary. All domains included in the the 'test_data/' directory will be run with the binary (unless the --domain option is exercised) and the output generated by the binary then becomes test answers or reference data for pytest autotests. The convert_prms_output_to_nc.py simply converts the output from the first step to NetCDF format and also calculates a few derived variables, providing the actual data sources used by autotest.

It is possible that the included binaries will not work on your system. For a failing PRMS 5.2.1 binary, please see prms_src/prms5.2.1/README.md. If you compile yourself, you'll need to copy prms_src/prms5.2.1/bin/prms to bin/ while renaming it to one of the existing files found therein. Currently, supplied binaries are all compiled with gfortran except on Mac ARM architecture where ifort is used.

Domain directories

Additional domains may be supplied by the user for autotesting by pywatershed. Also note that evaluation of pywatershed against PRMS in the notebook examples/03_compare_pws_prms.ipynb uses this approach as well. To supply your own domain, you must generate the required files in test_data/your_domain given in this listing:

control.test  prcp.cbh      sf_data       tmax.nc       tmin.nc
myparam.param prcp.nc       tmax.cbh      tmin.cbh

The *.cbh files must be pre-converted to netcdf for prcp, tmin, and tmax. How to do this can be found near the top of notebook examples/02_prms_legacy_models.ipynb. The control.test and myparam.param files are used by both PRMS and PWS. The control.test files in the repository are specific for being able to run sub-models in pywatershed and include a nearly maximal amount of model output (time-inefficient for both PRMS and PWS but necessary or pywatershed autotests). The stock control files can be found in test_data/common. There is a file for single-HRU domains and multi-HRU domains and these are identical (as appropriate) for the domains included in the repository.

Domain YAML file

The contents of the domain/domain.yaml file is described in the autotest/README.md