[Feature Request] Partially Sequential PipelineRuns/Temporary Task Halting Guards

[RobertDiebels]

Feature request

For some Pipelines it may be useful to temporarily halt a PipelineRun when it reaches a Task T that:

• Performs a transformation on a data-source.
• Uses an earlier output of that transformation (from a previous PipelineRun) as an input to the current execution of the transformation.
• May behave incorrectly when multiple TaskRuns of Task T are running in Parallel.

In this case a PipelineRun should halt execution until all PipelineRuns that were initiated earlier have completed Task T (either successfully or unsuccessfully).

This would create a situation where PipelineRuns halt execution (behave sequentially) only for certain tasks, and execute the rest in parallel (default behavior). In turn creating Partially Sequential PipelineRuns or Partially Parallel PipelineRuns (whichever you prefer).

Use case

Assume we have a Task V in Pipeline P, that reads the current version from a VCS and based on other data determines a new version.
Furthermore, we have 2 PipelineRuns X and Y for Pipeline P that are running in parallel, where PipelineRun X is initiated earlier than PipelineRun Y. Hence, we expect PipelineRun to write a (new) version to VCS and PipelineRun Y to use X's version as its current version.

If PipelineRun X is stalled, for instance due to waiting on scheduling, then PipelineRun Y can potentially catch up to it, in terms of tasks executed.
As a result it may occur that, for both PipelineRun X and Y, Task V is initiated at the same moment in time. Since we're dependent on our CVS and data existing within it, this may cause the output of incorrect versions. As both TaskRuns of Task T will use the same "current version", which is highly undesirable.

Additional Information

In essence, this feature is similar to an execution guard, however for this feature execution would be temporarily halted/postponed (Temporary Task Halting Guard), whereas execution guards prevent execution altogether.