Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate with updated Flow Framework APIs #222

Merged
merged 5 commits into from
Jul 19, 2024

Conversation

ohltyler
Copy link
Member

@ohltyler ohltyler commented Jul 18, 2024

Description

This PR integrates with 2 new enhancements to Flow Framework APIs:

  1. Ability to force delete indices when deprovisioning Add allow_delete parameter to Deprovision API flow-framework#763
  2. Ability to update certain fields (e.g., ui_metadata, name) on a provisioned workflow Support editing of certain workflow fields on a provisioned workflow flow-framework#757

With enhancement 1, we no longer have errors when deprovisioning / reprovisioning a workflow when the index config does not change. Before, this would produce exceptions on provision, due to the fact the index still exists, due to the fact we couldn't delete the index.

With enhancement 2, we can easily update and persist the UI form configuration (stored via ui_metadata field) without users explicitly clicking on an Ingest/Search button, which was doing the deep & full deprovision / provision actions - essentially giving us a "lightweight" save. Because of this, we can have some autosave functionality as users fill out their form. Current implementation is to perform a save behind-the-scenes when users touch any form field that is persisted. It is debounced (rate-limited) to save at a maximum of every 10s. We can tune this or change the scope of how/when we save later on as well.

Implementation details:

  • audit all form fields to ensure Formik's touched is updated on every type of input (model dropdown, map, JSON)
  • add autosave hooks in the base WorkflowInputs component to listen on & execute autosave in a debounced fashion when conditions are met
  • improve deprovision to optionally handle a list of resource_ids to be force deleted. This is utilized on the 2 places we call deprovision on WorkflowInputs - in these cases, we want to force delete and re-create any created index as users test. On the search side, we will not want this. TODO comment has been added, and will be dealt with once fine-grained provisioning is supported
  • improve update to optionally handle an update_fields parameter, specifying to only update metadata fields like ui_metadata

Demo video, showing 1/ multiple creations of the same index after adding an ingest pipeline with no errors, and 2/ autosaving. Input is entered in the form, and after refreshing the page, it is still visible and populated where the user last left off.

screen-capture.5.webm

Issues Resolved

Makes progress on #23

Check List

  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Tyler Ohlsen <[email protected]>
@ohltyler ohltyler merged commit 9216efc into opensearch-project:main Jul 19, 2024
6 checks passed
@ohltyler ohltyler deleted the onboard-ff-updates branch July 19, 2024 15:23
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jul 19, 2024
Signed-off-by: Tyler Ohlsen <[email protected]>
(cherry picked from commit 9216efc)
ohltyler added a commit that referenced this pull request Jul 19, 2024
Signed-off-by: Tyler Ohlsen <[email protected]>
(cherry picked from commit 9216efc)

Co-authored-by: Tyler Ohlsen <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants