Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

2023 Cylc Meetings #163

Closed
hjoliver opened this issue Jan 9, 2023 · 6 comments
Closed

2023 Cylc Meetings #163

hjoliver opened this issue Jan 9, 2023 · 6 comments
Assignees
@hjoliver

Comments

@hjoliver
Copy link
Member

hjoliver commented Jan 9, 2023 •
edited by oliver-sanders
Loading

Following on from #143

10 January

Recent activity

Attended:

  • TP
  • MH
  • RD
  • DM
  • HO
  • OS

Discussion Topics

cylc-flow 8.1.0: 96 closed issues, 7 open

  • release end of week?
  • cylc vro nearly done

cylc-uiserver 1.1.1 and 1.2.0 both in progress

  • go straight to 1.2.0 but make sure the 1.x bug fixes are merged to master (include J-Hub 3.0 bug workaround)

cylc-ui: 1.4.0: 62 closed, 3 open

  • release soon with 8.1.0
  • job log view (MH) very close to being done

Master branch protection

  • HO to do: merge via PR only, no force-push or deletion
  • (Met O has daily back-ups just in case)

OS to present a webinar to IS-ENES3 in mid March. Should end up on you tube.

UK team to start work on rose edit and rosie soon, less time on Cylc

And will start pushing all users on to Cylc 8 very soon; lots of support likely needed.

HO to report on DS and AC hours left on UMP contract etc.
@oliver-sanders oliver-sanders pinned this issue Jan 11, 2023
@hjoliver hjoliver self-assigned this Mar 15, 2023
@hjoliver
Copy link
Member Author

hjoliver commented Mar 15, 2023 •
edited
Loading

16/17 March

All present.

Bye Mel! 💐 🎖️

Discussion Topics

  • Prioritize bug fixes and current PRs.
    • HO: 13 scheduler PRs up, need to finish to allow full user intervention powers
    • (this includes what to do about expire triggers, cylc set proposal)
  • Quick update on workflow analysis view progress.
    • Not quite ready for review yet, but we can look a the approach being taken
  • Making it easier to re-run a workflow:
    • Automatically specify start-tasks when re-running workflows (cylc play --start-tasks='*'?).
    • Reduce number of commands required? cylc play <id>; cylc trigger --flow=<flow> <id>//...; cylc play <id>.
    • Also, should we be able to warm-restart a workflow?
    • agreed useful; should be easy to implement; must be a new flow so we don't lose provenance
  • share/lib and share/bin directories?
    • agreed; avoids the nuke-on-reinstall problem
  • Should the expired output be included in the compulsory/optional output system?
    • needs further thought; it should work within the optional outputs system, but needs further thought because: success may not be optional IF the task does not expire
  • How best to handle large log files in the UI?

If possible, agree cylc set proposal:
https://cylc.github.io/cylc-admin/proposal-cylc-set.html

  • (Not found the time yet).
  • (Possibly needs updating now to not use the term "expire" for forcibly telling the scheduler to forget incomplete tasks)

@hjoliver
Copy link
Member Author

hjoliver commented Jun 11, 2023 •
edited
Loading

12 June

Updates

HO:

  • flood-forecasting sub-sub-workflows
    • motivations include separately developed subsystems, as well as dynamic graph structure
    • we need Cylc 9 already! 😱
    • That's probably several years off, without additional resources, so let's not promise anything yet
  • DR
    • workflow-driven (in-workflow, or via watcher-worfkow)
      • disk-failover vs full DR
      • full DR: next restart enhancements (we have start-tasks with pre-initial-ignore, but, also triggering, cylc set etc., and clock-triggered tasks)
      • gentle failure (when possible) via workflow migration?
      • DM and HO to meet and discuss how Met Office does it, at some point soon
  • BoM priorities:
    • DR stuff (above)
    • cross-user (with or without hub admin)
    • UIS remote spawn
    • Hub with pseudo-spawner (plus remote spawn)
    • Platform "cross product" fix
    • Monitoring ~10 oper accounts
    • Note NIWA and MO not using J-Hub in anger yet, for Cylc; will do in next year
    • Cylc UI and UIS can handle multi-user subscriptions, but we need to decide how to request that and how gscan should display it

Discussion Topics

General:

  • getting the current ~30 PRs finished off
  • (some previous meeting topics still in the wild - above)
  • Another Cylc 7 final release is needed [DONE]
  • Cylc 8.2.0 release before next meeting ~17 July; need restart-finished-flow at the least
  • HO to shift 8.1.x issue to 8.2.0; we'll need to do some triage and bump a bunch of issues back
  • TP offered to help HO with tests and reviews for current PRs

@wxtim
Copy link
Member

wxtim commented Jul 17, 2023 •
edited
Loading

17 Jul

Updates

HO

TP

  • 🐛 🐛 🐛 👞 (Squashing bugs).
  • Add cylc play -z=f,g,h option.
  • Reviewing
  • Failing to add tests for HO's PR's. Sorry. :(

OS

RD

  • Mainly UI stuff

Discussion Topics

Notes

Present HO, DM, RD, DS, TP

Priorities

8.2

Release blockers?

  • Can HO hand over incomplete PRs?
  • OS to highlight PRs in question for HO.
  • HO to bump bumpable items.

UIS JupyterHub 2.0

Need to have a release shortly after JupyterHub 2.0 upgrade branch merged.
OS says it's nearly ready, but that careful review required. Documentation done.

8.3

  • Intervention proposals
  • Collapsing GScan

Discussions

5085 - Don't strip whitespace in Parsec

OS - will break MO workflows.
HO - Agreed to close.

5237 - Workflow state command Cylc 7 ←→ Cylc 8 compat

HO Blocker at NIWA, why not reviewed
OS Still marked draft

Clock Trigger Dependencies

Issue number @dwsutherland ?

OS If it's just clock triggers depending on earlier clock triggers it's ok.

Data Workflows

OS - Further work on the analysis view is ongoing.

Rose Stem

OS - Is starting to become a priority and may involve adding some requirements to Cylc which seem "odd" without reference to rose stem.

@wxtim
Copy link
Member

wxtim commented Sep 11, 2023 •
edited by dwsutherland
Loading

11 September

Updates

Note

UI Server 1.3.1 Released last Friday, with UI 2.1.0.

HO

  • cylc set command (almost done)
  • cylc hold flow-specific (done, needs one more review)
  • optional outputs (trying to resolve what I see as problem with the extended functionality)

TP

  • Skip Mode Precursors
    • Testing Sim Mode (found some 🐛 )
    • Making Sim Mode changeable by broadcast.
    • Holiday.

OS

RD

MD

DS

  • Graph no-rewalk and reloaded flag PRs ready to go/be-reviewed
  • Moving onto the named clock/trigger sole prerequisite sequential spawning.

Discussion Topics

Super-high priorities for 8.3.0:

  • implement intervention proposals, document
  • avoid graph rewalk (done, needs review?)

BOM priorities:

  • "workflow-specific" run-dir symlinking (agreed on the rose-like approach?)
  • (needs custom environment variables passed to run-hosts)
  • UIS log location!

After that, triage the rest and get what we can in.

UI:

  • BUG completed tasks never removed if graph if graph - or table? - is default view
  • new Gantt view? (HO tried to test...)

7.8 release

  • need cylc suite-state forward compat (the sibling PR is merged) - testing Cylc 7??

Notes

Present

Priorities

@hjoliver
Copy link
Member Author

hjoliver commented Oct 30, 2023 •
edited by oliver-sanders
Loading

30/31 October 🎃

Updates

  • 8.2.2 released 🎉
  • FYI: Status page now lists issue counts (auto-generated)
  • Workflow auto-collapsing in the workflows view now implemented.

HO

  • cylc set command (almost done - again 😬 )
  • (Singapore workshop)
  • (BOM training and demo session)

TP

  • Working out what simulation mode is actually doing, and testing it: Notably the evaluation of simulation mode needs to happen at runtime, not on config parsing, else we cannot broadcast to it or skip mode. There are still some outstanding discussions about what skip mode should do with optional outputs.
  • Helped a colleague (Joe Hickson) deliver a bout of Cylc training. (So did Mark, Oliver and Ronnie)
  • Stopped PYTHONPATH affecting Cylc. If you want it use CYLC_PYTHONPATH
  • Fixed a mod large pile of bugs.

OS

  • Clearing review backlog.
  • Looking into making "rose stem" a Cylc plugin (removing rose stem wrapper, unlocking other Cylc commands).

RD

MD

DS

Discussion Topics

High priorities for 8.3.0:

  • implement intervention proposals, document
  • avoid graph rewalk (done, needs review?) - Note this is 8.2.x
  • Tui 1.0

BOM priorities:

  • Hub cross-user demo?
  • "workflow-specific" run-dir symlinking (agreed on the rose-like approach?)
  • (needs custom environment variables passed to run-hosts) - MERGED
  • UIS log location?

cylc rewind?

(if there's time)

cylc/cylc-flow#5416

# it works but it ain't snappy, ideas?
cylc set <cycle> --pre=all [--flow=new]

After that, triage the rest and get what we can in.

7.8 release

  • need cylc suite-state forward compat (the sibling PR is merged) - testing Cylc 7??

Notes

Present

Priorities

This was referenced Feb 19, 2024
Open
Closed
@oliver-sanders oliver-sanders unpinned this issue Feb 19, 2024
@oliver-sanders
Copy link
Member

This is now closed, the conversation continues in #188

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@hjoliver hjoliver

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@hjoliver @oliver-sanders @wxtim