Skip to content

Latest commit

 

History

History
736 lines (479 loc) · 32.4 KB

CHANGES.md

File metadata and controls

736 lines (479 loc) · 32.4 KB

Changelog

List of notable changes, for a complete list of changes see the closed milestones for each release.

cylc-8.3.1 (Released 2024-07-04)

🔧 Fixes

#6130 - Prevent commands accepting job IDs where it doesn't make sense.

#6170 - Fix an issue where the Cylc logo could appear in the workflow log.

#6176 - Fix bug where jobs which fail to submit are not shown in GUI/TUI if submission retries are set.

cylc-8.3.0 (Released 2024-06-18)

⚠ Breaking Changes

#5600 - The cylc dump command now only shows active tasks (e.g. running & queued tasks). This restores its behaviour of only showing the tasks which currently exist in the pool as it did in Cylc 7 and earlier versions of Cylc 8.

#5727 - Cylc now ignores PYTHONPATH to make it more robust to task environments which set this value. If you want to add to the Cylc environment itself, e.g. to install a Cylc extension, use CYLC_PYTHONPATH.

#5794 - Remove cylc report-timings from automatic installation with pip install cylc-flow[all]. If you now wish to install it use pip install cylc-flow[report-timings]. cylc report-timings is incompatible with Python 3.12.

#5836 - Removed the 'CYLC_TASK_DEPENDENCIES' environment variable

#5956 - cylc lint: deprecated [cylc-lint] section in favour of [tool.cylc.lint] in pyproject.toml

#6046 - The submit-fail and expire task outputs must now be optional and can no longer be required.

🚀 Enhancements

#5571 - Make workflow CYLC_ variables available to the template processor during parsing.

#5658 - New "cylc set" command for setting task prerequisites and outputs.

#5709 - Forward arbitrary environment variables over SSH connections

#5721 - Allow task simulation mode settings to be changed dynamically using cylc broadcast.

#5731 - Major upgrade to cylc tui which now supports larger workflows and can browse installed workflows.

#5738 - Optionally spawn parentless xtriggered tasks sequentially - i.e., one at a time, after the previous xtrigger is satisfied, instead of all at once out to the runahead limit. The wall_clock xtrigger is now sequential by default.

#5769 - Include task messages and workflow port as appropriate in emails configured by "mail events".

#5803 - Updated 'reinstall' functionality to support multiple workflows

#5809 - The workflow-state command and xtrigger are now flow-aware and take universal IDs instead of separate arguments for cycle point, task name, etc. (which are still supported, but deprecated).

#5831 - Add capability to install xtriggers via a new cylc.xtriggers entry point

#5864 - Reimplemented the suite-state xtrigger for interoperability with Cylc 7.

#5872 - Improvements to cylc clean remote timeout handling.

#5873 - cylc lint improvements:

  • Allow use of #noqa: S001 comments to skip checks for a single line.
  • Stop cylc lint objecting to %include <file> syntax.

#5879 - cylc lint now warns of use of old templated items such as %(suite)s

#5890 - Lint: Warn users that setting CYLC_VERSION, ROSE_VERSION or FCM_VERSION in the workflow config is deprecated.

#5943 - The stop after cycle point can now be specified as an offset from the inital cycle point.

#5955 - Support xtrigger argument validation.

#6029 - Workflow graph window extent is now preserved on reload.

#6046 - The condition that Cylc uses to evaluate task output completion can now be customized in the [runtime] section with the new completion configuration. This provides a more advanced way to check that tasks generate their required outputs when run.

🔧 Fixes

#5809 - Fix bug where the "cylc workflow-state" command only polled for task-specific status queries and custom outputs.

#6008 - Fixed bug where the [scheduler][mail]to/from settings did not apply as defaults for task event mail.

#6036 - Fixed bug in simulation mode where repeated submissions were not displaying correctly in TUI/GUI.

#6067 - Fixed a bug that sometimes allowed suicide-triggered or manually removed tasks to be added back later.

#6109 - Fixed bug affecting job submission where the list of bad hosts was not always reset correctly.

#6123 - Allow long-format datetime cycle points in IDs used on the command line.

cylc-8.2.7 (Released 2024-05-15)

🔧 Fixes

#6096 - Fixed bug that caused graph arrows to go missing in the GUI when suicide triggers are present.

#6102 - Fixed bug introduced in 8.2.6 in cylc vip & cylc vr when using cylc-rose options (-S, -D, -O).

cylc-8.2.6 (Released 2024-05-02)

⚠ Breaking Changes

#6068 - Removed the Rose Options (-S, -O, -D) from cylc play. If you need these use them with cylc install.

🚀 Enhancements

#6072 - Nano Syntax Highlighting now available.

🔧 Fixes

#6071 - cylc config now shows xtrigger function signatures.

#6078 - Fixed bug where cylc lint could hang when checking inherit settings in flow.cylc.

cylc-8.2.5 (Released 2024-04-04)

🔧 Fixes

#5924 - Validation: a cycle offset can only appear on the right of a dependency if the task's cycling is defined elsewhere with no offset.

#5933 - Fixed bug in cylc broadcast (and the GUI Edit Runtime command) where everything after a # character in a setting would be stripped out.

#5959 - Fix an issue where workflow "timeout" events were not fired in all situations when they should have been.

#6011 - Fixed a cylc vip bug causing remote re-invocation to fail if using --workflow-name option.

#6031 - Fixed workflow-state command and xtrigger for alternate cylc-run directory.

cylc-8.2.4 (Released 2024-01-11)

🚀 Enhancements

#5772 - cylc lint: added a check for indentation being 4N spaces.

#5838 - cylc lint: added rule to check for rose date usage (should be replaced with isodatetime).

🔧 Fixes

#5789 - Prevent the run mode from being changed on restart.

#5801 - Fix traceback when using parentheses on right hand side of graph trigger.

#5821 - Fixed issue where large uncommitted changes could cause cylc install to hang.

#5841 - cylc lint: improved handling of S011 to not warn if the # is #$ (e.g. shell base arithmetic).

#5885 - Fixed bug in using a final cycle point with chained offsets e.g. 'final cycle point = +PT6H+PT1S'.

#5893 - Fixed bug in computing a time interval-based runahead limit when future triggers are present.

#5902 - Fixed a bug that prevented unsetting execution time limit by broadcast or reload.

#5908 - Fixed bug causing redundant DB updates when many tasks depend on the same xtrigger.

#5909 - Fix a bug where Cylc VIP did not remove --workflow-name= from Cylc play arguments.

cylc-8.2.3 (Released 2023-11-02)

🔧 Fixes

#5660 - Re-worked graph n-window algorithm for better efficiency.

#5753 - Fixed bug where execution time limit polling intervals could end up incorrectly applied

#5776 - Ensure that submit-failed tasks are marked as incomplete (so remain visible) when running in back-compat mode.

#5791 - fix a bug where if multiple clock triggers are set for a task only one was being satisfied.

cylc-8.2.2 (Released 2023-10-05)

🚀 Enhancements

#5237 - Back-compat: allow workflow-state xtriggers (and the cylc workflow-state command) to read Cylc 7 databases.

🔧 Fixes

#5693 - Log command issuer, if not the workflow owner, for all commands.

#5694 - Don't fail config file parsing if current working directory does not exist. (Note however this may not be enough to prevent file parsing commands failing elsewhere in the Python library).

#5704 - Fix off-by-one error in automatic upgrade of Cylc 7 "max active cycle points" to Cylc 8 "runahead limit".

#5708 - Fix runahead limit at start-up, with recurrences that start beyond the limit.

#5755 - Fixes an issue where submit-failed tasks could be incorrectly considered as completed rather than causing the workflow to stall.

cylc-8.2.1 (Released 2023-08-14)

🔧 Fixes

#5631 - Fix bug in remote clean for workflows that generated flow.cylc files at runtime.

#5650 - Fix a bug preventing clean-up of finished tasks in the GUI and TUI.

#5685 - Fix "cylc pause" command help (it targets workflows, not tasks, but was printing task-matching documentation as well).

cylc-8.2.0 (Released 2023-07-21)

Breaking Changes

#5600 - The CYLC_TASK_DEPENDENCIES environment variable will no longer be exported in job environments if there are more than 50 dependencies. This avoids an issue which could cause jobs to fail if this variable became too long.

Enhancements

#5992 - Before trying to reload the workflow definition, the scheduler will now wait for preparing tasks to submit, and pause the workflow. After successful reload the scheduler will unpause the workflow.

#5605 - Added -z shorthand option for defining a list of strings:

  • Before: cylc command -s "X=['a', 'bc', 'd']"
  • After: cylc command -z X=a,bc,d.

#5537 - Allow parameters in family names to be split, e.g. <foo>FAM<bar>.

#5589 - Move to workflow directory during file parsing, to give the template processor access to workflow files.

#5405 - Improve scan command help, and add scheduler PID to the output.

#5461 - preserve colour formatting when starting workflows in distributed mode using run hosts.

#5291 - re-implement old-style clock triggers as wall_clock xtriggers.

#5439 - Small CLI short option chages: Add the -n short option for --workflow-name to cylc vip; rename the -n short option for --no-detach to -N; add -r as a short option for --run-name.

#5231 - stay up for a timeout period on restarting a completed workflow, to allow for manual triggering.

#5549, #5546 - Various enhancements to cylc lint:

  • cylc lint will provide a non-zero return code if any issues are identified. This can be overridden using the new --exit-zero flag.
  • Fix numbering of lint codes (n.b. lint codes should now be permenantly unchanging, but may have changed since Cylc 8.1.4, so pyproject.toml files may need updating).
  • Check for suicide triggers in .cylc files.
  • Check for platform = $(rose host-select).
  • Check for use of deprecated Cylc commands (and rose suite-hook).
  • Check for zero prefixed Jinja2 integers.
  • Only check for missing Jinja2 shebangs in flow.cylc and suite.rc files.

#5525 - Jobs can use scripts in share/bin and Python modules in share/lib/python.

Fixes

#5328 - Efficiency improvements to reduce task management overheads on the Scheduler.

#5611 - Improve the documentation of the GraphQL schema.

#5616 - Improve PBS support for job IDs with trailing components.

#5619 - Fix an issue where the task_pool table in the database wasn't being updated in a timely fashion when tasks completed.

#5606 - Task outputs and messages are now validated to avoid conflicts with built-in outputs, messages, qualifiers and Cylc keywords.

#5614 - Fix a bug in Cylc 7 compatibility mode where tasks running in the none flow (e.g. via cylc trigger --flow=none) would trigger downstream tasks.

#5604 - Fix a possible issue where workflows started using cylc play --start-cycle-point could hang during startup.

#5573 - Fix bug that ran a queued waiting task even after removal by cylc remove.

#5524 - Logging includes timestamps for cylc play when called by cylc vip or cylc vr.

#5228 - Enabled the "stop", "poll", "kill" and "message" commands to be issued from the UI whilst the workflow is in the process of shutting down.

#5582 - Set Cylc 7 compatibility mode before running pre-configure plugins.

#5587 - Permit commas in xtrigger arguments and fix minor issues with the parsing of xtrigger function signatures.

#5618 - Fix a bug when rapidly issuing the same/opposite commands e.g. pausing & resuming a workflow.

#5625 - Exclude setuptools version (v67) which results in dependency check failure with editable installs.

cylc-8.1.4 (Released 2023-05-04)

Fixes

#5514 - Ensure cylc cat-log directory listings always include the job-activity.log file when present and are able to list submit-failed jobs.

#5506 - Fix bug introduced in 8.1.3 where specifying a subshell command for flow.cylc[runtime][<namespace>][remote]host (e.g. $(rose host-select)) would always result in localhost.

cylc-8.1.3 (Released 2023-04-27)

Enhancements

#5475 - much faster computation of the visualization window around active tasks (at the cost, for now, of not showing non-active "cousin" nodes).

#5453 - cylc cat-log can now list and view workflow log files including install logs and workflow configuration files.

Fixes

#5495 - Fix bug that could cause invalid parent tasks to appear in the UI datastore.

#5334 - Apply graph prerequisite changes to already-spawned tasks after reload or restart.

5466 - Don't generate duplicate prerequisites from recurrences with coincident points.

5450 - Validation provides better error messages if [sections] and settings are mixed up in a configuration.

5445 - Fix remote tidy bug where install target is not explicit in platform definition.

5398 - Fix platform from group selection order bug.

#5395 - Fix bug where workflow shuts down if all hosts for all platforms in a platform group are unreachable.

#5384 - Fixes cylc set-verbosity.

#5479 - Fixes cylc help license

#5394 - Fixes a possible scheduler traceback observed with remote task polling.

#5386 - Fix bug where absence of job name length maximum in PBS platform settings would cause Cylc to crash when preparing the job script.

#5343 - Fix a bug causing platform names to be checked as if they were hosts.

#5359 - Fix bug where viewing a workflow's log in the GUI or using cylc cat-log would prevent cylc clean from working.

cylc-8.1.2 (Released 2023-02-20)

Fixes

#5349 - Bugfix: cylc vip --workflow-name only worked when used with a space, not an =.

#5367 - Enable using Rose options (-O, -S & -D) with cylc view.

#5363 Improvements and bugfixes for cylc lint.

cylc-8.1.1 (Released 2023-01-31)

Fixes

#5313 - Fix a bug causing Cylc to be unable to parse previously played Cylc 7 workflows.

#5312 - task names must be comma-separated in queue member lists. Any implicit tasks (i.e. with no task definition under runtime) assigned to a queue will generate a warning.

#5314 - Fix broken command option: cylc vip --run-name.

#5319, #5321, #5325 - Various efficiency optimisations to the scheduler which particularly impact workflows with many-to-many dependencies (e.g. <a> => <b>).

cylc-8.1.0 (Released 2023-01-16)

Breaking Changes

  • Workflows started with Cylc 8.0 which contain multiple "flows" cannot be restarted with Cylc 8.1 due to database changes.

Enhancements

#5229 -

  • Added a single command to validate a previously run workflow against changes to its source and reinstall a workflow.
  • Allows Cylc commands (including validate, list, view, config, and graph) to load template variables configured by cylc install and cylc play.

#5121 - Added a single command to validate, install and play a workflow.

#5184 - Scan for active runs of the same workflow at install time.

#5084 - Assign the most recent previous flow numbers to tasks triggered when no flows are present (e.g. on restarting a finished workflow).

#5032 - Set a default limit of 100 for the "default" queue.

#5055 and #5086 - Upgrades to cylc lint

  • Allow users to ignore Cylc Lint issues using --ignore <Issue Code>.
  • Allow settings for cylc lint to be recorded in a pyproject.toml file.
  • Allow files to be excluded from cylc lint checks.

#5081 - Reduced amount that gets logged at "INFO" level in scheduler logs.

#5259 - Add flow_nums to task_jobs table in the workflow database.

Fixes

#5286 - Fix bug where [scheduling][special tasks]clock-trigger would skip execution retry delays.

#5292 - Fix an issue where polling could be repeated if the job's platform was not available.

cylc-8.0.4 (Released 2022-12-14)

Maintenance release.

Fixes

##5205 - Fix bug which caused orphaned running tasks to silently skip remote file installation at scheduler restart.

#5224 - workflow installation: disallow reserved names only in the top level source directory.

#5211 - Provide better explanation of failure if icp = next (T-02, T-32) when list should be semicolon separated.

#5196 - Replace traceback with warning, for scan errors where workflow is stopped.

#5199 - Fix a problem with the consolidation tutorial.

#5195 - Fix issue where workflows can fail to shutdown due to unavailable remote platforms and make job log retrieval more robust.

cylc-8.0.3 (Released 2022-10-17)

Maintenance release.

Fixes

#5192 - Recompute runahead limit after use of cylc remove.

#5188 - Fix task state selectors in cylc trigger and other commands.

#5125 - Allow rose-suite.conf changes to be considered by cylc reinstall.

#5023, #5187 - tasks force-triggered after a shutdown was ordered should submit to run immediately on restart.

#5137 - Install the ana/ directory to remote platforms by default.

#5146 - no-flow tasks should not retrigger incomplete children.

#5104 - Fix retriggering of failed tasks after a reload.

#5139 - Fix bug where cylc install could hang if there was a large uncommitted diff in the source dir (for git/svn repos).

#5131 - Infer workflow run number for workflow_state xtrigger.

cylc-8.0.2 (Released 2022-09-12)

Maintenance release.

Fixes

#5115 - Updates rsync commands to make them compatible with latest rsync releases.

#5119 - Fix formatting of deprecation warnings at validation.

#5067 - Datastore fix for taskdefs removed before restart.

#5066 - Fix bug where .cylcignore only found if cylc install is run in source directory.

#5091 - Fix problems with tutorial workflows.

#5098 - Fix bug where final task status updates were not being sent to UI before shutdown.

#5114 - Fix bug where validation errors during workflow startup were not printed to stderr before daemonisation.

#5110 - Fix bug where reloading a stalled workflow would cause it stall again.

cylc-8.0.1 (Released 2022-08-16)

Maintenance release.

Fixes

#5025 - Fix a bug where polling causes a failed task to be shown as submitted when the workflow is reloaded.

#5045 - Fix issue where unsatisfied xtriggers could be wiped on reload.

#5031 - Fix bug where specifying multiple datetime offsets (e.g. final cycle point = +P1M-P1D) would not obey the given order.

#5033 - Running cylc clean on a top level dir containing run dir(s) will now remove that top level dir in addition to the run(s) (if there is nothing else inside it).

#5007 - Fix for cylc broadcast cycle point validation in the UI.

#5037 - Fix bug where the workflow restart number would get wiped on reload.

#5049 - Fix several small bugs related to auto restart.

#5062 - Fix bug where preparing tasks could sometimes get orphaned when an auto restart occurred.

cylc-8.0.0 (Released 2022-07-28)

Cylc 8 production-ready release.

Major Changes

  • Python 2 -> 3.
  • Internal communications converted from HTTPS to ZMQ (TCP).
  • PyGTK GUIs replaced by:
    • Terminal user interface (TUI) included in cylc-flow.
    • Web user interface provided by the cylc-uiserver package.
  • A new scheduling algorithm with support for branched workflows.
  • Command line changes:
    • cylc run -> cylc play
    • cylc restart -> cylc play
    • rose suite-run -> cylc install; cylc play <id>
  • The core package containing Cylc scheduler program has been renamed cylc-flow.
  • Cylc review has been removed, the Cylc 7 version remains Cylc 8 compatible.
  • New documentation.

See the migration guide for a full list of changes.

Enhancements

#4964 - cylc reinstall now displays the changes it would make when run interactively and has improved help / documentaiton.

#4836 - The log directory has been tidied. Workflow logs are now found in log/scheduler rather than log/workflow, filenames now include start/restart. Other minor directory changes. Remote file installation logs are now per install target.

#4938 - Detect bad Platforms config: background and at job runners should have a single host.

#4877 - Upgrade the version of Jinja2 used by Cylc from 2.11 to 3.0.

#4896 - Allow the setting of default job runner directives for platforms.

#4900 - Added a command to assist with upgrading Cylc 7 workflows to Cylc 8: Try cylc lint <workflow-dir>.

#5009 - Added new job environment variable $CYLC_WORKFLOW_NAME_BASE as the basename of $CYLC_WORKFLOW_NAME.

#4993 - Remove the few remaining uses of a configured text editor (via cylc view and cylc cat-log options). The primary uses of it (cylc trigger --edit and cylc edit in Cylc 7) have already been removed from Cylc 8.

Fixes

#5011 - Removes preparing jobs appearing in UI, and reuse submit number on restart for preparing tasks.

#5008 - Autospawn absolute-triggered tasks exactly the same way as parentless tasks.

#4984 - Fixes an issue with cylc reload which could cause preparing tasks to become stuck.

#4976 - Fix bug causing tasks to be stuck in UI due to discontinued graph of optional outputs.

#4975 - Fix selection of platforms from [job] and [remote] configs.

#4948 - Fix lack of errors/warnings for deprecated [runtime][<task>][remote]retrieve job logs * settings.

#4970 - Fix handling of suicide triggers in back-compat mode.

#4887 - Disallow relative paths in global.cylc[install]source dirs.

#4906

  • Fix delayed spawning of parentless tasks that do have parents in a previous cycle point.
  • Make integer-interval runahead limits consistent with time-interval limits: P0 means just the runahead base point; P1 the base point and the point (i.e. one cycle interval), and so on.

#4936 - Fix incorrect error messages when workflow CLI commands fail.

#4941 - Fix job state for platform submit-failures.

#4931 - Fix cylc install for installing workflows from multi-level directories.

#4926 - Fix a docstring formatting problem presenting in the UI mutation flow argument info.

#4891 - Fix bug that could cause past jobs to be omitted in the UI.

#4860 - Workflow validation now fails if owner setting is used, as that setting no longer has any effect.

#4978 - cylc clean: fix occasional failure to clean on remote hosts due to leftover contact file.

#4889 - cylc clean: don't prompt if no matching workflows.

#4890 - cylc install: don't overwrite symlink dir targets if they were not cleaned properly before.

#4881 - Fix bug where commands targeting a specific cycle point would not work if using an abbreviated cycle point format.

Older Releases