24.8.0 (2024-08-29)

This release contains a large number of bug fixes and small improvements, and sets the stage for the major cycling improvement (#750) coming in the next release in a few days.

What's Changed

• Parse recipe variables as python literals by @jfrost-mo in #683
• Fixed metplus config issue by @dasha-shchep in #693
• Clarify error message on missing files by @jfrost-mo in #663
• Update testing.rst by @Sylviabohnenstengel in #696
• Correct rst syntax added in #696 by @jfrost-mo in #700
• Suggest descriptive PR titles instead of changelog entry by @jfrost-mo in #701
• Improve PR title documentation in developer's guide by @jfrost-mo in #707
• Remove Fixes placeholder in PR template by @jfrost-mo in #705
• Update section header to match PR checklist item by @jfrost-mo in #704
• Minor wording tweak in marking PR as ready to review docs by @jfrost-mo in #702
• Add canonical URL links to documentation by @jfrost-mo in #650
• Clarify running specific tests by @jfrost-mo in #703
• Fix SET_SUBAREA being required to run some LFRIC recipes by @dasha-shchep in #717
• Remove deprecated postage stamp contour plot operator by @jfrost-mo in #710
• Remove unneeded fallback code for old recipe step keys by @jfrost-mo in #711
• Test vertical plotting with a filename specified by @jfrost-mo in #712
• Test running recipes with no collate steps and running with a specified style file by @jfrost-mo in #713
• Test handling of masked arrays in convection operators, and load convection test files with fixtures by @jfrost-mo in #714
• Ignore cartopy DownloadWarning in pytest by @jfrost-mo in #716
• LFRic extension to vertical profile by @Sylviabohnenstengel in #638
• Remove unneeded rose edit setting by @jfrost-mo in #722
• Test higher dimentional orography handling in convection operators by @jfrost-mo in #715
• Support STASH codes in generate_var_constraint by @jfrost-mo in #723
• Add cross-section transect functionality by @jwarner8 in #531
• Extend UM vertical plotting to model level by @Sylviabohnenstengel in #697
• UM spatial plot on model levels by @Sylviabohnenstengel in #699
• Enable spatial fields on full and half levels by @Sylviabohnenstengel in #695
• Fix bug in transect operator by @jwarner8 in #731
• Unquote double quoted shell values by @jfrost-mo in #729
• Remove invalid rose meta trigger for removed variable by @jfrost-mo in #733
• Add _utils operators to internal function documentation by @jfrost-mo in #735
• Don't check documentation hyperlinks in commit CI by @jfrost-mo in #749
• Generate Histograms for 2D field by @Sylviabohnenstengel in #594
• Tidy up documentation around recipes and cset bake command, introducing examples by @Sylviabohnenstengel in #641
• Create operator to combine Cubes/CubeList into single CubeList by @jwarner8 in #738
• Update copyright attribution per legal advice by @jfrost-mo in #753
• Regrid to take CubeList and Cubes by @jwarner8 in #734
• Enhance level filter operator to return all vertical levels by @jwarner8 in #728
• Mass retrieval fix by @jwarner8 in #759
• Copy source files from any named folder when installing locally by @jfrost-mo in #472
• Add missing brackets to TemporaryDirectory call when installing local CSET by @jfrost-mo in #760
• Move all website files under the workflow shared directory by @jfrost-mo in #764
• Fix overwriting when using transect on multiple variables by @jwarner8 in #766
• Add Q-Q plot functionality by @daflack in #642
• Add nc-time-axis to dependencies by @jwarner8 in #767
• Fix plot frames jumping around by @jwarner8 in #772
• Surface field histogram by @Sylviabohnenstengel in #640
• Preload plot images on web page by @jfrost-mo in #781
• Extract single point data by @JKPShonk in #577
• Filter irrelevant warning raised by regrid test by @jfrost-mo in #796
• Merge and concatinate cubes on load by @jfrost-mo in #790
• Allow Point cell methods for empty constraint, making it possible to unify UM and LFRic recipes by @jfrost-mo in #778
• Merge install_local_cset into build_conda so environment is setup in a single cylc task by @jfrost-mo in #791
• Convert time AuxCoord to DimCoord for LFRic data by @jfrost-mo in #789
• Cancel running PR checks if new commit is pushed by @jfrost-mo in #793
• Stop ruff warning about ignore-init-module-imports by @jfrost-mo in #800
• Parsing of float in workflow by @jwarner8 in #802
• Configurable plotting resolution by @jfrost-mo in #803
• General small plot improvements and website organisation by @jwarner8 in #801
• Set figsize to consistent 8 by 8 inches, and reduce resolution to 100 dpi by @jfrost-mo in #786
• Remove LFRic specific recipes now recipes can handle both UM and LFRic data by @jfrost-mo in #805
• Move workflow utility code into unstable module within CSET package by @jfrost-mo in #792
• Add pcolormesh plotting operator by @jfrost-mo in #787
• Use pcolormesh for Quicklook surface spatial plots by @jfrost-mo in #788
• Create the plot index in finish_website to avoid a data race between concurrent index writers by @jfrost-mo in #794
• Add release notes for CSET v24.8.0 by @jfrost-mo in #816

Full Changelog: v24.6.0...v24.8.0

24.6.0 (2024-06-17)

This release contains a quite a number of small improvements, increasing the reliability of CSET significantly, and paving the way for further improvements to come.

• @jfrost-mo replaced how the encoding of subprocess output is determined in PR #604. This adds support for python before 3.11, and more accurately reflects the encoding.
• @jwarner8 add intelligent determination of whether to plot country lines in PR #655
• @daflack fixed inflow properties recipe in PR #662
• @daflack added science review guidance to the documentation in PR #649
• @jfrost-mo ensured cartopy data files are included in the GitHub Actions cache in PR #647
• @jfrost-mo grouped the package install logs in GitHub Actions in PR #645
• @daflack added an inflow layer properties diagnostic in PR #353
• @jfrost-mo fixed LFRic cube metadata on load in PR #627. This means that loading LFRic data no longer requires special steps in the recipe.
• @jfrost-mo made the install_restricted_files.sh script non-interactive in PR #606. This should make it less confusing to use.
• @Sylviabohnenstengel added information to rose-meta for colorbar selection and provided path to example JSON file in PR #632
• @jfrost-mo added a Generative AI policy in PR #624
• @jfrost-mo linked to the CSET discussion forum in PR #625
• @jwarner8 use common operator to identify x/y coord names in regrid operator PR #626
• @jwarner8 added generic cube util for common functions so all operators can use to reduce repetition in PR #620
• @JorgeBornemann added METPlus GridStat functionality (NIWA) in PR #629
• @jfrost-mo added a code of conduct in PR #618
• @jfrost-mo fixed some rose edit metadata so the subarea selectors show up when enabled in PR #612
• @jfrost-mo removed some old recipes that are now redundant in PR #512
• @jfrost-mo added a git cheat sheet to the documentation in PR #598
• @jfrost-mo added a warning when input files don’t exist in PR #518. This makes it easier to see if configuration mistakes were made.
• @jfrost-mo improved the documentation for adding a new diagnostic in PR #603
• @dasha-shchep added generate_area_constraint operator and added to LFRic recipes in PR #522. This was their first contribution 🎉
• @Sylviabohnenstengel added a vertical line plotting operator that plots vertical profiles using an optional series coordinate and an optional sequence coordinate in PR #567. The series coordinate is currently tested for pressure and the sequence coordinate allows displaying vertical profiles over time using the time slider functionality. Furthermore added a recipe to plot vertical profiles and test functions for the vertical plot operator.
• @jfrost-mo dropped python 3.9 support in PR #448 The minimum required python is now 3.10.
• @jfrost-mo fixed some outdated documentation examples in PR #546
• @jfrost-mo added setuptools as an explicit dependency of the workflow in PR #543
• @cjohnson-pi added support for custom plotting styles in PR #570. This avoids many issues of side-by-side plots having different scales, or extreme values causing plots to saturate. This was their first contribution 🎉
• @Sylviabohnenstengel documentation: removed necessity to add new recipe to flow.cylc.
• @Ashfinn fixed a typo in the documentation in PR #573. This was their first contribution 🎉
• @jfrost-mo fixed how arguments from the CSET_ADDOPTS environment variable are parsed in PR #569. This fixes issues with passing lists into recipes.
• @jfrost-mo added a dead link checker to the documentation in PR #556
• @Sylviabohnenstengel documented the common error of operating on a CubeList instead of a Cube in PR #541
• @Sylviabohnenstengel documented how to update your conda environment in PR #519
• @Sylviabohnenstengel documented the common error of no cubes being loaded in PR #513
• @Sylviabohnenstengel and @jfrost-mo redid the rose-meta sort orders so that workflow configuration makes more sense in PR #504
• @jfrost-mo updated the example rose-suite.conf to reflect what a modern version should look like in PR #508

v24.4.1 (2024-04-19)

This release contains a large generalisation of the CSET workflow, allowing use of templating to use the same recipe for multiple variables. It also adds cycling to the workflow, so a long workflow can be efficiently processed in parallel.

v24.4.0 doesn't exist, I just slipped up on the version numbering when releasing...

What's Changed

@jfrost-mo added GitHub Issue and Pull Request templates, and a detailed contribution checklist to the documentation in PR #465
@jfrost-mo added a changelog in PR #468
@jfrost-mo documented the category recipe key in PR #499
@jfrost-mo renamed the steps and post-steps keys to parallel and collate in PR #484. This makes them more meaningful, but is a breaking change.
@daflack added some generic arithmetic operators in PR #452
@jfrost-mo made the log output of the read operator nicer in PR #461
@jfrost-mo added links to share feedback to the output page in PR #442
@jfrost-mo documented some common errors in PR #443
@jfrost-mo documented the deprecation policy in PR #444
@jfrost-mo fixed an iris deprecation warning for save_split_attrs in PR #459
@jfrost-mo added LFRic specific recipes in PR #462. This allows CSET to read in structured LFRic data.
@jfrost-mo fixed a memory leak when plotting in PR #482
@jfrost-mo included the recipe title in the plot title, giving more context to the output. This was PR #462
@Sylviabohnenstengel added the capability to loop over model levels in PR #441
@Sylviabohnenstengel and @jfrost-mo renamed and better linked up the Developer’s Guide in PR #434 and PR #435
@jfrost-mo updated the documentation Actions workflow to make it simpler and faster in PR #449
@jfrost-mo added a cycling to the cylc workflow so recipes can be run in parallel across multiple nodes. This was PR #395
@jfrost-mo added looping inside an include file for generalisation in PR #387
@jwarner8 added a basic regridding operator in PR #405
@jfrost-mo made conda-lock update PRs use a GitHub App in PR #415
@Sylviabohnenstengel retitled code and tooling setup page in PR #433
@Sylviabohnenstengel updated git terminology in PR #436
@jfrost-mo added sequential plot display with unified postage stamp plots in PR #379
@JorgeBornemann fixed IFS in build conda in PR #447
@jfrost-mo added a licence header to convection tests in PR #450

New Contributors

• @JorgeBornemann made their first contribution in #447

Full Changelog: v24.2.1...v24.4.1

v24.2.1 Portability bug fixes

A small bug fix release containing several fixes that ensure portability on Australia's NCI system.

What's Changed

• Graceful error when graphing without xdg-open by @jfrost-mo in #394
• Docs update by @jfrost-mo in #392
• Update workflow conda lockfiles automatically by @jfrost-mo in #410
• Handle LD_LIBRARY_PATH being unset by @jfrost-mo in #404

Full Changelog: v24.2.0...v24.2.1

CSET v24.2.0 (2024-02-13)

With this release we have switched to CalVer for our versioning scheme. This makes it easier to know if you are using an out-of-date version,and saves us from always being on version 0.X. We are using the format YY.MM.patch, e.g.: 24.2.0.

PyPI | conda-forge

This release open sources the cylc workflow, allowing for much easier running of CSET over large datasets. It also includes support for parametrising recipes to allow a single recipe to work for many cases. Information on how to use both of these things can be found in the documentation.

Finally, a bug where the recipes were not being packaged has also been fixed (9c6f05d).

Happy pancake day! 🥞

What's Changed

Along with updating dependencies the following PRs are included in this release.

• Open source workflow by @jfrost-mo in #247
• CAPE ratio diagnostic by @daflack in #325
• CAPE ratio rose edit tweak by @daflack in #332
• Minor bug fix to cape ratio documentation by @daflack in #336
• Use cached conda environment for CI by @jfrost-mo in #351
• Single cube read operator by @jfrost-mo in #323
• Hash updated config ensuring unique branch by @jfrost-mo in #350
• Add filter_multiple_cubes operator by @jfrost-mo in #362
• Test exception for invalid output directory by @jfrost-mo in #364
• Test no constraints given to filter_multiple_cubes by @jfrost-mo in #363
• Update workflow-installation.rst by @Sylviabohnenstengel in #365
• Recipe parametrisation by @jfrost-mo in #337
• Fix crash when running recipe from env var by @jfrost-mo in #384

Full Changelog: v0.5.0...v0.6.0

v0.5.0 Somewhat nicer plots

Small update featuring some better looking plots (though still a work-in-progress, see #240) and a documentation fix.

What's Changed

• Bump version to 0.5.0 by @jfrost-mo in #278
• Improve contour plot by @jfrost-mo in #282

Full Changelog: v0.4.0...v0.5.0

v0.4.0 CSET was a static site generator all along!?

Containing many months of work, this release contains many usability improvements, new generic operators, and a big change to the output, where it is now generated as handily viewable HTML pages.

What's Changed

• Update version to 0.4.0 by @jfrost-mo in #180
• Postage stamp plots by @jfrost-mo in #160
• Add collapse operator with corresponding yaml file and changes by @Sylviabohnenstengel in #168
• Make plot.contour_plot and write.write_cube_to_nc return a cube by @jfrost-mo in #183
• Postage stamp plot fix by @jfrost-mo in #181
• Document collapse operator by @jfrost-mo in #185
• Refactor tests to use PyTest helpers by @jfrost-mo in #177
• Document installing CSET into its own environment by @jfrost-mo in #198
• Update README.md by @Sylviabohnenstengel in #206
• Use hash of updated lock files in branch name by @jfrost-mo in #201
• Add note on updating a cloned repository by @jfrost-mo in #190
• Skip build-docs on push to main by @jfrost-mo in #200
• Python 3.12 support by @jfrost-mo in #202
• Update README.md by @Sylviabohnenstengel in #225
• Update README.md by @Sylviabohnenstengel in #226
• Update why-cset.rst by @Sylviabohnenstengel in #227
• Fix globbing for lock file hashing by @jfrost-mo in #229
• Update index.rst by @Sylviabohnenstengel in #228
• Update index.rst by @Sylviabohnenstengel in #230
• Use static branch name while updating lock files by @jfrost-mo in #245
• Swap out flake8 for Ruff by @jfrost-mo in #218
• Including aggregate operator. by @Sylviabohnenstengel in #241
• Fix filter operator for filtering cube by @daflack in #258
• Fix pre-commit mangling test data by @jfrost-mo in #273
• Improve tutorials by @jfrost-mo in #209
• Model level constraint operator by @Sylviabohnenstengel in #272
• Plot generation improvements by @jfrost-mo in #274

New Contributors

• @daflack made their first contribution in #258

Full Changelog: v0.3.0...v0.4.0

v0.3.0 Now with a baking theme! 🧑‍🍳

This release contains some major changes to the user experience. This includes many of the CLI commands changing names, and the documentation being completely restructured. Hopefully this should be the last major reshuffle of the user experience, as we are getting closers to being feature complete for our MVP.

Other highlights include the addition of the graph command for visualising recipes, and the cookbook command for dumping the built in recipes to disk.

What's Changed

• Operator runner improvements by @jfrost-mo in #128
• Add Codespell pre-commit hook by @jfrost-mo in #135
• Add graph command to visualise recipe files by @jfrost-mo in #136
• Pin version of tox used in environment by @jfrost-mo in #142
• Increase version number by @jfrost-mo in #124
• Update description of CSET by @jfrost-mo in #141
• Refactoring by @jfrost-mo in #144
• Rename run command to bake by @jfrost-mo in #143
• Add command to create recipes on disk by @jfrost-mo in #140
• Documentation restructure by @jfrost-mo in #151
• Add version command by @jfrost-mo in #156
• General cleanup by @jfrost-mo in #158
• Remove Python 3.8 support by @jfrost-mo in #173
• Fix install instructions in docs by @jfrost-mo in #176
• Allow PR checks to be run manually by @jfrost-mo in #179
• Ensemble ingestation with read operator by @jfrost-mo in #157
• Update working practices link to point to contributing docs by @jfrost-mo in #175

Full Changelog: v0.2.0...v0.3.0

v0.2.0: The path towards a stable Recipe format

Lots of good work in the release towards making the recipe format more usable.

What's Changed

• Update installation instructions to use conda and add missing operators to documentation by @jfrost-mo in #94
• Update index.rst by @Sylviabohnenstengel in #95
• Improve installation instructions by @jfrost-mo in #97
• Use speedy libmamba when resolving conda environments by @jfrost-mo in #105
• Add documentation on rational by @jfrost-mo in #102
• Relax version requirement for sphinx by @jfrost-mo in #108
• Run PR checks on push to main by @jfrost-mo in #109
• Move to YAML recipe format by @jfrost-mo in #119
• Lock pre-commit config to specific SHA by @jfrost-mo in #118
• Use recipes from environment variable by @jfrost-mo in #122

Full Changelog: v0.1.1...v0.2.0

v0.1.0 First Release

The first release of CSET! 🎉 This release contains basic operators to do reading, writing, filtering, and plotting of data. It is however still quite limited in each of them, and still doesn't promise much in the way of API stability, with things undoubtedly going to undergo significant change in the near future.

This release also serves as a basis for packaging CSET out into the wider world; packages will be released on PyPI, and conda-forge.

What's Changed

• Reenable testing on python 3.11 by @jfrost-mo in #61
• Operator runner improvements by @jfrost-mo in #56
• Move METplus tasks out of command line repository by @jfrost-mo in #76
• Remove extra punctuation from conda lock CI commit message by @jfrost-mo in #78
• Measure test coverage by @jfrost-mo in #68
• Improve test coverage by @jfrost-mo in #81
• Fix link to Git tutorial by @jfrost-mo in #83
• Fix description of a git tag by @jfrost-mo in #84
• Add basic plotting capabilities by @jfrost-mo in #85
• Make PR coverage reports edit last comment by @jfrost-mo in #92
• Package on PyPI by @jfrost-mo in #90

Full Changelog: https://github.com/MetOffice/CSET/commits/v0.1.0