Skip to content

Commit

Permalink
added references for CRPS for ensembles
Browse files Browse the repository at this point in the history
  • Loading branch information
nicholasloveday authored and tennlee committed May 14, 2024
1 parent bcb5984 commit 9620c40
Show file tree
Hide file tree
Showing 2 changed files with 30 additions and 2 deletions.
30 changes: 29 additions & 1 deletion docs/paper.bib
Original file line number Diff line number Diff line change
Expand Up @@ -229,4 +229,32 @@ @article{griffiths2021circular
year={2021},
publisher={CSIRO Publishing},
doi={10.1071/es21010}
}
}
@article{Ferro_2013,
title={Fair scores for ensemble forecasts: Fair Scores for Ensemble Forecasts},
volume={140},
ISSN={0035-9009},
url={http://dx.doi.org/10.1002/qj.2270},
DOI={10.1002/qj.2270},
number={683},
journal={Quarterly Journal of the Royal Meteorological Society},
publisher={Wiley},
author={Ferro, C. A. T.},
year={2013},
month=dec,
pages={1917–1923}
}
@article{Gneiting_2007,
title={Strictly Proper Scoring Rules, Prediction, and Estimation},
volume={102}, ISSN={1537-274X},
url={http://dx.doi.org/10.1198/016214506000001437},
DOI={10.1198/016214506000001437},
number={477},
journal={Journal of the American Statistical Association},
publisher={Informa UK Limited},
author={Gneiting, Tilmann and Raftery, Adrian E},
year={2007},
month=mar,
pages={359–378}
}

2 changes: 1 addition & 1 deletion docs/paper.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ In order to meet the needs of researchers, `scores`:

## Roadmap and Future Development

At the time of writing, the scores contained in this package are: Mean Squared Error (MSE), Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), the Fixed Risk Multicategorical (FIRM) score [@Taggart:2022a], Continuous Ranked Probability Score (CRPS) for Cumulative Distribution Functions (CDFs) (including threshold-weighting, see [@Gneiting:2011]), CRPS for ensembles, the Flip-Flop Index [@Griffiths:2019; @griffiths2021circular], Receiver Operating Characteristic (ROC) curves, the quantile score, and the Murphy score [@Ehm:2016]. It also includes the Diebold-Mariano statistical test [@Diebold:1995] with both the [@Harvey:1997] and [@Hering:2011] modifications. Additionally it contains isotonic regression which is becoming an increasingly important tool in forecast verification and can be used to generate stable reliability diagrams [@dimitriadis2021stable]. We anticipate more scores, metrics and statistical techniques will be added over time.
At the time of writing, the scores contained in this package are: Mean Squared Error (MSE), Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), the Fixed Risk Multicategorical (FIRM) score [@Taggart:2022a], Continuous Ranked Probability Score (CRPS) for Cumulative Distribution Functions (CDFs) (including threshold-weighting, see [@Gneiting:2011]), CRPS for ensembles [@Gneiting_2007], [@Ferro_2013], the Flip-Flop Index [@Griffiths:2019; @griffiths2021circular], Receiver Operating Characteristic (ROC) curves, the quantile score, and the Murphy score [@Ehm:2016]. It also includes the Diebold-Mariano statistical test [@Diebold:1995] with both the [@Harvey:1997] and [@Hering:2011] modifications. Additionally it contains isotonic regression which is becoming an increasingly important tool in forecast verification and can be used to generate stable reliability diagrams [@dimitriadis2021stable]. We anticipate more scores, metrics and statistical techniques will be added over time.

The `scores` roadmap includes:

Expand Down

0 comments on commit 9620c40

Please sign in to comment.