Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Backport 2.x] Add WorkflowStep Factory and implement XContent-based Template Parsing #60

Merged
merged 1 commit into from
Sep 28, 2023

Conversation

opensearch-trigger-bot[bot]
Copy link

Backport 734f9c2 from #47.

#47)

* Add WorkflowStepFactory class

Signed-off-by: Daniel Widdis <[email protected]>

* Add XContent classes representing Template JSON

Signed-off-by: Daniel Widdis <[email protected]>

* Add parse methods for the Template XContent

Signed-off-by: Daniel Widdis <[email protected]>

* Cleanup parsing, javadocs, and demo output

Signed-off-by: Daniel Widdis <[email protected]>

* Refactor to use field name constants, get tests working again

Signed-off-by: Daniel Widdis <[email protected]>

* Separate WorkflowNode and ProcessNode functionality

Signed-off-by: Daniel Widdis <[email protected]>

* Fix demos to align with template field names

Signed-off-by: Daniel Widdis <[email protected]>

* Add workflow, node, and edge tests

Signed-off-by: Daniel Widdis <[email protected]>

* Add Template tests

Signed-off-by: Daniel Widdis <[email protected]>

* Refactor TemplateParser to WorkflowProcessSorter

Signed-off-by: Daniel Widdis <[email protected]>

* Test exceptional cases

Signed-off-by: Daniel Widdis <[email protected]>

* Finish up exceptional cases

Signed-off-by: Daniel Widdis <[email protected]>

* Fix a template field name bug in demo

Signed-off-by: Daniel Widdis <[email protected]>

* Rebase with #34

Signed-off-by: Daniel Widdis <[email protected]>

* Rebase changes from #54

Signed-off-by: Daniel Widdis <[email protected]>

* Integrate thread pool executor service

Signed-off-by: Daniel Widdis <[email protected]>

* Fix flaky ProcessNodeTests by removing orTimeout

Signed-off-by: Daniel Widdis <[email protected]>

* Rebase and refactor with #44

Signed-off-by: Daniel Widdis <[email protected]>

* Fix demos and remove DataDemo

Signed-off-by: Daniel Widdis <[email protected]>

* Use non-deprecated mapping method for CreateIndexStep

Signed-off-by: Daniel Widdis <[email protected]>

* Eliminate casting and deprecation warnings on test classes

Signed-off-by: Daniel Widdis <[email protected]>

* Remove unused/leftover demo class

Signed-off-by: Daniel Widdis <[email protected]>

* Typo

Signed-off-by: Daniel Widdis <[email protected]>

* Don't offer steps as an alternative to nodes

Signed-off-by: Daniel Widdis <[email protected]>

* Move Workflow into package with all the other parsing classes

Signed-off-by: Daniel Widdis <[email protected]>

* Move process sequencing classes into workflow package

Signed-off-by: Daniel Widdis <[email protected]>

* Add PipelineProcessor class and XContent parsing, rename package

Signed-off-by: Daniel Widdis <[email protected]>

---------

Signed-off-by: Daniel Widdis <[email protected]>
(cherry picked from commit 734f9c2)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@codecov
Copy link

codecov bot commented Sep 28, 2023

Codecov Report

Merging #60 (a629cba) into 2.x (6f3b720) will increase coverage by 13.05%.
The diff coverage is 90.14%.

@@              Coverage Diff              @@
##                2.x      #60       +/-   ##
=============================================
+ Coverage     73.87%   86.93%   +13.05%     
- Complexity       65      163       +98     
=============================================
  Files             8       13        +5     
  Lines           245      574      +329     
  Branches         22       75       +53     
=============================================
+ Hits            181      499      +318     
+ Misses           54       52        -2     
- Partials         10       23       +13     
Files Coverage Δ
...nsearch/flowframework/model/PipelineProcessor.java 100.00% <100.00%> (ø)
...a/org/opensearch/flowframework/model/Workflow.java 100.00% <100.00%> (ø)
...search/flowframework/workflow/CreateIndexStep.java 84.84% <100.00%> (ø)
...owframework/workflow/CreateIngestPipelineStep.java 90.62% <ø> (ø)
...opensearch/flowframework/workflow/ProcessNode.java 58.69% <88.88%> (ø)
.../opensearch/flowframework/FlowFrameworkPlugin.java 0.00% <0.00%> (ø)
...g/opensearch/flowframework/model/WorkflowEdge.java 84.84% <84.84%> (ø)
.../flowframework/workflow/WorkflowProcessSorter.java 92.85% <92.85%> (ø)
...g/opensearch/flowframework/model/WorkflowNode.java 90.54% <90.54%> (ø)
...a/org/opensearch/flowframework/model/Template.java 92.36% <92.36%> (ø)
... and 1 more

@dbwiddis dbwiddis merged commit 5161294 into 2.x Sep 28, 2023
16 checks passed
@github-actions github-actions bot deleted the backport/backport-47-to-2.x branch September 28, 2023 02:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant