Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates exception handling with FlowFrameworkException and fix Global Context mapping #106

Closed
wants to merge 8 commits into from

Conversation

joshpalis
Copy link
Member

@joshpalis joshpalis commented Oct 19, 2023

Description

This PR updates the workflow parsing, workflow execution and rest action exception handling to throw FlowFrameworkExceptions with the correct REST Status code.

Example invalid create workflow request with missing name :

curl -i -XPOST "localhost:9200/_plugins/_flow_framework/workflow?pretty" -H "Content-Type:application/json" --data '{"description":"My semantic search use case","use_case":"SEMANTIC_SEARCH","operations":["PROVISION"],"version":{"template":"1.0.0","compatibility":["2.9.0","3.0.0"]},"user_inputs":{"index_name":"my-knn-index","index_settings":{}},"workflows":{"provision":{"nodes":[{"id":"create_index","type":"create_index","inputs":{"type":"knn","index-name":"my-knn-index","settings":""}},{"id":"create_ingest_pipeline","type":"create_ingest_pipeline","inputs":{"id":"my-ingest-pipeline","description":"some description","processors":[{"type":"text_embedding","params":{"model_id":"my-existing-model-id","input_field_name":"text_passage","output_field_name":"text_embedding"}}]}}],"edges":[{"source":"create_index","dest":"create_ingest_pipeline"}]}}}'

HTTP/1.1 400 Bad Request
X-OpenSearch-Version: OpenSearch/3.0.0-SNAPSHOT (opensearch)
content-type: application/json; charset=UTF-8
content-length: 86

{
  "error" : "Request failed with exception: [A template object requires a name.]"
}

Note : Opening this PR up for second opinions regarding the various rest status codes for each exception. I've opted to return 400 for parsing, rest request format errors. Client exception rest status codes are determined via the OpenSearch ExceptionsHelper::status method

Additionally this PR modified the global context mapping so that user_outputs, workflows, and resourcesCreated are of type object instead of text. This allows us to use Template::toXContent and Template::parse directly when reading and writing to the global context index

Issues Resolved

Part of #88

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@joshpalis joshpalis added the backport 2.x backport PRs to 2.x branch label Oct 19, 2023
@codecov
Copy link

codecov bot commented Oct 19, 2023

Codecov Report

Merging #106 (d67a7c1) into main (63ef780) will decrease coverage by 2.42%.
The diff coverage is 40.78%.

@@             Coverage Diff              @@
##               main     #106      +/-   ##
============================================
- Coverage     81.46%   79.04%   -2.42%     
+ Complexity      285      263      -22     
============================================
  Files            30       30              
  Lines          1122     1026      -96     
  Branches        126      103      -23     
============================================
- Hits            914      811     -103     
- Misses          162      179      +17     
+ Partials         46       36      -10     
Files Coverage Δ
...lowframework/exception/FlowFrameworkException.java 45.45% <100.00%> (+5.45%) ⬆️
...ework/transport/CreateWorkflowTransportAction.java 100.00% <100.00%> (ø)
...opensearch/flowframework/workflow/ProcessNode.java 93.47% <100.00%> (ø)
...a/org/opensearch/flowframework/model/Template.java 89.54% <0.00%> (+2.26%) ⬆️
.../flowframework/workflow/WorkflowProcessSorter.java 97.10% <83.33%> (ø)
...ch/flowframework/indices/GlobalContextHandler.java 72.22% <50.00%> (ø)
...rk/transport/ProvisionWorkflowTransportAction.java 45.61% <42.85%> (ø)
...search/flowframework/workflow/CreateIndexStep.java 56.63% <8.33%> (-2.08%) ⬇️
...h/flowframework/rest/RestCreateWorkflowAction.java 45.83% <31.57%> (-20.84%) ⬇️
...lowframework/rest/RestProvisionWorkflowAction.java 48.00% <35.00%> (-24.73%) ⬇️

... and 1 file with indirect coverage changes

…ng all invocations of toDocumentSource/parseFromDocumentSource and using toXContent/parse instead

Signed-off-by: Joshua Palis <[email protected]>
@joshpalis joshpalis changed the title Updates exception handling with FlowFrameworkException Updates exception handling with FlowFrameworkException and fix Global Context mapping Oct 19, 2023
Copy link
Member

@dbwiddis dbwiddis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

I'm okay with 400 (Bad Request) but technically it means bad syntax. From RFC 2068:

The request could not be understood by the server due to malformed
syntax. The client SHOULD NOT repeat the request without
modifications.

Consider using 422 (Unprocessable Entity) for the cases where it's valid JSON but a bad value (equivalent of IllegalArgumetnException). From RFC 2518:

The 422 (Unprocessable Entity) status code means the server
understands the content type of the request entity (hence a
415(Unsupported Media Type) status code is inappropriate), and the
syntax of the request entity is correct (thus a 400 (Bad Request)
status code is inappropriate) but was unable to process the contained
instructions. For example, this error condition may occur if an XML
request body contains well-formed (i.e., syntactically correct), but
semantically erroneous, XML instructions.

For this PR I'd suggest 422 for the cases where you parse a valid edge source/destination but it doesn't match a node, for example (but based on ExceptionsHelper I'm still OK with 400.)

@joshpalis
Copy link
Member Author

Closing this PR in favor of #137

@joshpalis joshpalis closed this Oct 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x backport PRs to 2.x branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants