Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hydrology model #6

Open
wants to merge 44 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 34 commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
b34e753
add link to readme
rccreswell Mar 25, 2021
d0c7e6e
change path in run tests
rccreswell Mar 25, 2021
d7e1ede
add data file and description
rccreswell Mar 25, 2021
ffb9ced
add load_data function
rccreswell Mar 25, 2021
f465408
add tests for load data
rccreswell Mar 25, 2021
14160a6
add ode function
rccreswell Mar 25, 2021
091ef16
add new forward model
rccreswell Mar 25, 2021
e27282a
add script to run model
rccreswell Mar 25, 2021
06b316c
add dependencies to setup
rccreswell Mar 25, 2021
9a4dfed
add sundials to actions
rccreswell Mar 25, 2021
d147afd
install extras in unit tests
rccreswell Mar 25, 2021
1eb4a65
try install again
rccreswell Mar 25, 2021
b2b1c69
add blas and lapack
rccreswell Mar 25, 2021
bb9a728
add ubuntu dependencies
rccreswell Mar 25, 2021
73c6c03
remove failing installs
rccreswell Mar 25, 2021
7e3fb8c
remove mkl
rccreswell Mar 25, 2021
122db4a
switch to sundials v5
rccreswell Mar 25, 2021
6f1bc2c
export paths
rccreswell Mar 25, 2021
e00907a
remove unusual install path
rccreswell Mar 25, 2021
cbcc6a6
add ld library path
rccreswell Mar 25, 2021
cb7f1e2
use github env variables
rccreswell Mar 25, 2021
6df1ee0
remove path from cmake
rccreswell Mar 25, 2021
a81fb72
print paths
rccreswell Mar 25, 2021
f725c31
print paths
rccreswell Mar 25, 2021
9ac6746
move install to yml
rccreswell Mar 25, 2021
d1a69a2
add sudo
rccreswell Mar 25, 2021
99eaa7e
repeat export
rccreswell Mar 25, 2021
7f72b4e
remove print and unused sh
rccreswell Mar 25, 2021
9499184
write model readme
rccreswell Mar 26, 2021
6e4b9e8
upload notebook
rccreswell Mar 29, 2021
e50b288
model changes
rccreswell Mar 29, 2021
926ff70
model readme and docs
rccreswell Mar 29, 2021
862df6a
actions yaml name
rccreswell Mar 29, 2021
c972fcf
repo readme name
rccreswell Mar 29, 2021
ec52ee5
add to readme
rccreswell Apr 19, 2021
a8c146a
readme references
rccreswell Apr 19, 2021
ae5c0aa
change model docstring
rccreswell Apr 19, 2021
d48fdf6
move notebook
rccreswell Apr 19, 2021
1f05aed
change readme link
rccreswell Apr 19, 2021
3359cfe
try to fix readme link
rccreswell Apr 19, 2021
1779171
change file name readme
rccreswell Apr 19, 2021
0919a05
fix link path
rccreswell Apr 19, 2021
3162862
changes to notebook
rccreswell Apr 19, 2021
6034755
add examples to readme
rccreswell Apr 19, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 18 additions & 1 deletion .github/workflows/unit-python-ver-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,30 @@ jobs:
python-version: ${{ matrix.python-version }}
architecture: x64

- name: Install SUNDIALS
run: |
sudo apt-get -qq update;
sudo apt-get install -y libopenblas-dev liblapack-dev
wget https://github.com/LLNL/sundials/releases/download/v5.1.0/sundials-5.1.0.tar.gz
tar xzf sundials-5.1.0.tar.gz
mkdir build-sundials-5.1.0
cd build-sundials-5.1.0/
cmake -DLAPACK_ENABLE=ON -DSUNDIALS_INDEX_SIZE=64 -DCMAKE_INSTALL_PREFIX=$GITHUB_WORKSPACE ../sundials-5.1.0/
make install
export SUNDIALS_INST=$GITHUB_WORKSPACE
export LD_LIBRARY_PATH=$GITHUB_WORKSPACE/lib:$LD_LIBRARY_PATH

- name: Install models
run: |
export SUNDIALS_INST=$GITHUB_WORKSPACE
export LD_LIBRARY_PATH=$GITHUB_WORKSPACE/lib:$LD_LIBRARY_PATH
python --version
python -m pip install --upgrade pip setuptools wheel
python -m pip install -e ./streamflow/
python -m pip install -e ./streamflow/[dev]
python -m pip install -e ./streamflow/[dev,cvode]

- name: run unit tests
run: |
export SUNDIALS_INST=$GITHUB_WORKSPACE
export LD_LIBRARY_PATH=$GITHUB_WORKSPACE/lib:$LD_LIBRARY_PATH
python run-tests.py --unit
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# pints-models
# bayesian-differential-equations-db

The purpose of pints-models is to present Python implementations of scientific time series models from various disciplines.
The purpose of bayesian-differential-equations-db is to present Python implementations of scientific time series models from various disciplines.
rccreswell marked this conversation as resolved.
Show resolved Hide resolved

While the `pints.toy` module already includes a number of simple models and distributions, this repository is intended for more complex models associated with real experimental data.

## Models and data

Hydrology
[Rainfall runoff model and river discharge data for the French Broad River at Asheville, North Carolina.](streamflow/)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we include citations for the model? Also, can we create a sort of table that says why each of the models is interesting? So, here, I guess it's because of potential model misspecification / a potentially complex noise process / numerical error in solving the system.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this idea, I started making a table which might work

20 changes: 20 additions & 0 deletions docs/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Minimal makefile for Sphinx documentation
#

# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = sphinx-build
SPHINXPROJ = differential-equations-inference-db
SOURCEDIR = source
BUILDDIR = build

# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

.PHONY: help Makefile

# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
36 changes: 36 additions & 0 deletions docs/make.bat
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
@ECHO OFF

pushd %~dp0

REM Command file for Sphinx documentation

if "%SPHINXBUILD%" == "" (
set SPHINXBUILD=sphinx-build
)
set SOURCEDIR=source
set BUILDDIR=build
set SPHINXPROJ=differential-equations-inference-db

if "%1" == "" goto help

%SPHINXBUILD% >NUL 2>NUL
if errorlevel 9009 (
echo.
echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
echo.installed, then set the SPHINXBUILD environment variable to point
echo.to the full path of the 'sphinx-build' executable. Alternatively you
echo.may add the Sphinx directory to PATH.
echo.
echo.If you don't have Sphinx installed, grab it from
echo.http://sphinx-doc.org/
exit /b 1
)

%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%
goto end

:help
%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS%

:end
popd
147 changes: 147 additions & 0 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
# -*- coding: utf-8 -*-
#
# This file is execfile()d with the current directory set to its
# containing dir.
#
# Note that not all possible configuration values are present in this
# autogenerated file.
#
# All configuration values have a default; values that are commented out
# serve to show the default.

# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
# import os
# import sys
# sys.path.insert(0, os.path.abspath('.'))
import sphinx

# -- General configuration ------------------------------------------------

# If your documentation needs a minimal Sphinx version, state it here.
#
# needs_sphinx = '1.0'

# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
'sphinx.ext.autodoc',
'sphinx.ext.doctest',
'sphinx.ext.mathjax',
'sphinx.ext.napoleon',
'sphinx.ext.viewcode',
]

# Autodoc defaults
if int(sphinx.__version__.split('.')[1]) < 8:
autodoc_default_flags = [
'members',
'inherited-members',
# 'show-inheritance',
]
else:
autodoc_default_options = {
'members': None,
'inherited-members': None,
}

# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']

# The suffix(es) of source filenames.
# You can specify multiple suffix as a list of string:
#
# source_suffix = ['.rst', '.md']
source_suffix = '.rst'

# The master toctree document.
master_doc = 'index'

# General information about the project.
project = u'model'

# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#

# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#
# This is also used if you do content translation via gettext catalogs.
# Usually you set "language" from the command line for these cases.
language = None

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This patterns also effect to html_static_path and html_extra_path
exclude_patterns = []

# Suppress warnings about unused footnotes (i.e. citations of papers)
suppress_warnings = [
'ref.footnote',
]

# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'

# If true, `todo` and `todoList` produce output, else they produce nothing.
todo_include_todos = False


# -- Options for HTML output ----------------------------------------------

# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
html_theme = 'alabaster'

# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
#
html_theme_options = {

# See: https://alabaster.readthedocs.io/en/latest/customization.html

# Fixed sidebar is unusable: Doesn't have its own scrollbar!
# 'fixed_sidebar': True,

'page_width': '1280px',
'sidebar_width': '320px',
}

# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = []


# -- Options for HTMLHelp output ------------------------------------------

# Output file base name for HTML help builder.
htmlhelp_basename = 'modelsdoc'


# -- Options for LaTeX output ---------------------------------------------

latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#
# 'papersize': 'letterpaper',

# The font size ('10pt', '11pt' or '12pt').
#
# 'pointsize': '10pt',

# Additional stuff for the LaTeX preamble.
#
# 'preamble': '',

# Latex figure (float) alignment
#
# 'figure_align': 'htbp',
}
9 changes: 9 additions & 0 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
Welcome to the differential-equations-inference-db documentation
================================================================

Contents
========

.. toctree::

pystreamflow
22 changes: 22 additions & 0 deletions docs/source/pystreamflow.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
************
pystreamflow
************

The pystreamflow model can be used to study river discharge data.

This package provides a PINTS Forward Model for simulation and inference, and a
function for loading precipitation, evaporation, and discharge data in Pandas
format.

.. currentmodule:: pystreamflow

Forward model
*************

.. autoclass:: pystreamflow.RiverModel
:members: __init__, simulate, set_model_data

Data
****

.. autofunction:: pystreamflow.load_data
2 changes: 1 addition & 1 deletion run-tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ def run_unit_tests():
"""
Runs unit tests (without subprocesses).
"""
model_dirs = ['streamflow/pystreamflow']
model_dirs = [os.path.join('streamflow', 'pystreamflow')]

for dir in model_dirs:
tests = os.path.join(dir, 'tests')
Expand Down
36 changes: 36 additions & 0 deletions streamflow/README.md
Original file line number Diff line number Diff line change
@@ -1 +1,37 @@
# pystreamflow

pystreamflow enables efficient simulation and inference for rainfall runoff differential equation models of the sort that arise in hydrological modelling of river basins.

## Installation

1. Navigate to `differential-equations-inference-db/streamflow/` (this directory)
1. For a simple install, run `pip install .`
1. To install with support for the CVODE solver via scikits.odes, run `pip install .[cvode]`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, I tried to install without doing this step but I ran into the following error:

RuntimeError: scikit solver could not be imported

Is scikit.odes required? Or does it just make things faster?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, when exactly did this error appear? Was it when running the notebook? Or did it appear during pip install?

scikit.odes is optional and just makes solving faster. However, if you try to instantiate the model object with the non-default optional argument solver='scikit', it will raise this error if scikits.odes is not available. So if this happened when running the notebook cell which compares the two different solvers, I would consider that the expected behavior.

Another option would be to raise a warning and revert to the scipy solver in this case.


### Installing with CVODE

The CVODE solver enables faster model evaluations and inference, but it requires you to install the SUNDIALS C library and the scikits.odes Python package. The following is a typical install process for the 5.1.0 version of SUNDIALS:

```
sudo apt-get install libopenblas-dev liblapack-dev
wget https://github.com/LLNL/sundials/releases/download/v5.1.0/sundials-5.1.0.tar.gz
tar xzf sundials-5.1.0.tar.gz
mkdir build-sundials-5.1.0
cd build-sundials-5.1.0/
cmake -DLAPACK_ENABLE=ON -DSUNDIALS_INDEX_SIZE=64 ../sundials-5.1.0/
make install
```

scikits.odes is pip installable, and an attempt will be made to install it when `pip install .[cvode]` is executed. However, system-specific troubleshooting is often required; see the scikit.odes [installation documentation](https://scikits-odes.readthedocs.io/en/latest/installation.html) for details.

If you have conda, an alternative is to use conda forge:

```
conda install -c conda-forge scikits.odes
```

## Usage

Once installed, the streamflow model can be accessed using the `pystreamflow.RiverModel` class and the data can be accessed using the `pystreamflow.load_data` function. The raw data files are available [here](pystreamflow/data/), and the model code is [here](pystreamflow/model.py).

The [notebook](examples/data_and_inference.ipynb) in the examples directory contains code for loading the data and running inference with MCMC.
Loading