Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Complete advanced output transform (ingest) #236

Merged
merged 6 commits into from
Jul 24, 2024

Conversation

ohltyler
Copy link
Member

@ohltyler ohltyler commented Jul 23, 2024

Description

This PR completes the advanced output transform flow on the ingest side. Specifically:

  • fully implements OutputTransformModal and the logic for fetching the input and generating the output. The notable difference is on fetching the input - for transforming the output, we run the simulate ingest pipeline API up to and including the currently-configured processor, but also removing any currently-configured output transforms. That way, we get a consistent and expected input to the output transform every time. It's also possible we refactor both of these modals into a single generic one - since the UI is not finalized, we wait to handle that
  • refactors some logic and helper fns into utils to reuse in the input / output transform modals
  • refactors and cleans up the form in MLProcessorInputs - moves the modal buttons under Preview buttons in-line with the form
  • cleans up the transform modals, and changes the source input / generated output to use EuiCodeEditor to easily handle overflow/scrolling, as this data may be very long and complex (e.g., 1000-dimension embeddings)
  • minor: adds a 'Test' button on query form to test a query outside of any search pipeline context

Demo video, showing the refactored form components, transform modals, a working output transform modal, and finally, ingest:

screen-capture.11.webm

Issues Resolved

Makes progress on #23

Check List

  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@ohltyler ohltyler merged commit 2fcb819 into opensearch-project:main Jul 24, 2024
4 of 6 checks passed
@ohltyler ohltyler deleted the output-transform branch July 24, 2024 15:17
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jul 24, 2024
Signed-off-by: Tyler Ohlsen <[email protected]>
(cherry picked from commit 2fcb819)
ohltyler added a commit that referenced this pull request Jul 24, 2024
Signed-off-by: Tyler Ohlsen <[email protected]>
(cherry picked from commit 2fcb819)

Co-authored-by: Tyler Ohlsen <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants