Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft: Allow invoked pipelines to complete, mitigate blocking on status channel #113

Closed
wants to merge 2 commits into from

Conversation

peteski22
Copy link
Contributor

DRAFT: IN PROGRESS

This PR is intended to be used initially for discussion of potential changes/improvements to go-eventlogger.

Perceived issues:

  • graph.process The Status channel is changed from unbuffered (default size) to buffered, the size will be the estimated number of nodes within all pipelines (number of root nodes * 3). This is to prevent occurrence of sending messages to a blocked channel is no longer being received on
  • graph.process While ranging over the root nodes, do not invoke roots when we know the context is already Done
  • graph.doProcess parameter tweaked so the channel supplied can only be used for sending
  • broker.SetRequireAllPipelinesComplete Added broker-level setting (per event type) to decide whether process should allow all invoked pipelines to complete and return their status before gathering/fanning-in

…atus to the channel when we've essentially stopped listening, add ability to wait for invoked pipelines to complete before returning status+err
@peteski22
Copy link
Contributor Author

UPDATE:

Most of this was a waste of time 😄 as doProcess doesn't allow returning the status if the context is cancelled.

Side note: there was a bug in this anyway when we came to Store as we'd need to have checked if the pipeline already exists in the map and only increment if it didn't.

TL;DR: Not really relevant and I had the wrong end of the stick, I should read the Go docs next time!

doProcess: // # No Status is sent when a request is cancelled by the context

@peteski22 peteski22 closed this Jul 25, 2024
@peteski22 peteski22 reopened this Jul 25, 2024
@peteski22 peteski22 closed this Aug 11, 2024
@peteski22 peteski22 deleted the peteski22/channels-and-completed-pipelines branch August 11, 2024 20:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant