Skip to content

Commit

Permalink
group references
Browse files Browse the repository at this point in the history
  • Loading branch information
nicholasloveday authored and tennlee committed May 14, 2024
1 parent 9620c40 commit b1b009e
Show file tree
Hide file tree
Showing 2 changed files with 14 additions and 15 deletions.
25 changes: 12 additions & 13 deletions docs/paper.bib
Original file line number Diff line number Diff line change
Expand Up @@ -121,15 +121,6 @@ @Article{Brown:2021
year = {2021},
doi = {10.1175/bams-d-19-0093.1}
}
@Article{loveday2024jive,
title={The Jive Verification System and its Transformative Impact on Weather Forecasting Operations},
author={Nicholas Loveday and Deryn Griffiths and Tennessee Leeuwenburg and Robert Taggart and Thomas C. Pagano and George Cheng and Kevin Plastow and Elizabeth Ebert and Cassandra Templeton and Maree Carroll and Mohammadreza Khanarmuei and Isha Nagpal},
year={2024},
eprint={2404.18429},
archivePrefix={arXiv},
primaryClass={physics.ao-ph},
doi={10.48550/arXiv.2404.18429}
}
@book{Griffiths:2017,
title = {Advice for automation of forecasts: a framework},
author = {Griffiths, Deryn and Jack, Harry and Foley, Michael and Ioannou, Ioanna and Liu, Maoyuan},
Expand Down Expand Up @@ -230,7 +221,15 @@ @article{griffiths2021circular
publisher={CSIRO Publishing},
doi={10.1071/es21010}
}
@article{Ferro_2013,
@misc{loveday2024jive,
title={The Jive Verification System and its Transformative Impact on Weather Forecasting Operations},
author={Nicholas Loveday and Deryn Griffiths and Tennessee Leeuwenburg and Robert Taggart and Thomas C. Pagano and George Cheng and Kevin Plastow and Elizabeth Ebert and Cassandra Templeton and Maree Carroll and Mohammadreza Khanarmuei and Isha Nagpal},
year={2024},
eprint={2404.18429},
archivePrefix={arXiv},
primaryClass={physics.ao-ph}
}
@article{Ferro_2013,
title={Fair scores for ensemble forecasts: Fair Scores for Ensemble Forecasts},
volume={140},
ISSN={0035-9009},
Expand All @@ -243,8 +242,8 @@ @article{Ferro_2013
year={2013},
month=dec,
pages={1917–1923}
}
@article{Gneiting_2007,
}
@article{Gneiting_2007,
title={Strictly Proper Scoring Rules, Prediction, and Estimation},
volume={102}, ISSN={1537-274X},
url={http://dx.doi.org/10.1198/016214506000001437},
Expand All @@ -256,5 +255,5 @@ @article{Gneiting_2007
year={2007},
month=mar,
pages={359–378}
}
}

4 changes: 2 additions & 2 deletions docs/paper.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ In order to meet the needs of researchers, `scores`:

## Roadmap and Future Development

At the time of writing, the scores contained in this package are: Mean Squared Error (MSE), Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), the Fixed Risk Multicategorical (FIRM) score [@Taggart:2022a], Continuous Ranked Probability Score (CRPS) for Cumulative Distribution Functions (CDFs) (including threshold-weighting, see [@Gneiting:2011]), CRPS for ensembles [@Gneiting_2007], [@Ferro_2013], the Flip-Flop Index [@Griffiths:2019; @griffiths2021circular], Receiver Operating Characteristic (ROC) curves, the quantile score, and the Murphy score [@Ehm:2016]. It also includes the Diebold-Mariano statistical test [@Diebold:1995] with both the [@Harvey:1997] and [@Hering:2011] modifications. Additionally it contains isotonic regression which is becoming an increasingly important tool in forecast verification and can be used to generate stable reliability diagrams [@dimitriadis2021stable]. We anticipate more scores, metrics and statistical techniques will be added over time.
At the time of writing, the scores contained in this package are: Mean Squared Error (MSE), Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), the Fixed Risk Multicategorical (FIRM) score [@Taggart:2022a], Continuous Ranked Probability Score (CRPS) for Cumulative Distribution Functions (CDFs) (including threshold-weighting, see [@Gneiting:2011]), CRPS for ensembles [@Gneiting_2007; @Ferro_2013], the Flip-Flop Index [@Griffiths:2019; @griffiths2021circular], Receiver Operating Characteristic (ROC) curves, the quantile score, and the Murphy score [@Ehm:2016]. It also includes the Diebold-Mariano statistical test [@Diebold:1995] with both the [@Harvey:1997] and [@Hering:2011] modifications. Additionally it contains isotonic regression which is becoming an increasingly important tool in forecast verification and can be used to generate stable reliability diagrams [@dimitriadis2021stable]. We anticipate more scores, metrics and statistical techniques will be added over time.

The `scores` roadmap includes:

Expand All @@ -69,7 +69,7 @@ The `scores` roadmap includes:

In 2015, the Australian Bureau of Meteorology began developing a new verification system called Jive. For a description of Jive see [@loveday2024jive].

The Jive verification metrics have been used by [@Griffiths:2017], [@Foley:2020], [@Taggart:2022b], [@Taggart:2022c] and [@Taggart:2022d].
The Jive verification metrics have been used to support several publications [@Griffiths:2017; @Foley:2020; @Taggart:2022b; @Taggart:2022c; @Taggart:2022d].

`scores` has arisen from the Jive verification system. `scores` includes mathematical functions from Jive and is intended to modularise these functions and make them available as an open source package.

Expand Down

0 comments on commit b1b009e

Please sign in to comment.