[META] Resolve TODOs in code prior to release

[dbwiddis]

The following TODO comments should be resolved prior to release by one or more of:

• doing the thing that needs to be done (if quick, write a PR, if it will take more than a day, write an issue)
• deciding that the TODO can be handled in a future release and documenting it in an issue
• disabling a functionality that is incomplete (and write an issue)

TODOs:

• Workflow steps that create resources but don't have a corresponding deletion should be disabled until fully implemented. This includes register model group, create ingest pipeline, and create index. (lines 32, 36, 38) [FEATURE/DOCS] Properly handle model group deletion #365
• FlowFrameworkIndicesHandler has comments on updating/concurrency/retry (lines 443, 473, 477)
• Provisioning progress detail (line 14 and WorkflowState line 54) - For later - [FEATURE] Enhance provisioning progress field #385
• WorkflowState access control error line 111
• Get Workflow State get the full template?
• Provision Workflow error message improvement Refactor RetryableWorkflowStep to avoid recursion and subclass specifics #298
• Search Workflow access controller question - we need an access controller for the plugin, this can be done later.
• EncryptorUtils improve handling of which fields are encrypted
• RestHandlerUtils special handling for UI Metadata
• Create Connector Step comment on retrying
• Create Index Step and Create Ingest Pipeline Steps, same refactoring of input data already done for most other steps - these are incomplete workflow steps and not needed for the first use case.
• ProcessNode better getCause handling Clean up miscellaneous code smells, eliminate most Guava use #327
• ProcessNode still using generic thread pool executor [FEATURE] Update ProcessNode to use the plugin's thread pool #366
• FlowFrameworkPlugin executorBuilder thread pool settings
• Template writeable comment called out on line 245 [FEATURE] Make Template Writeable #391

[joshpalis]

Get Workflow State get the full template?

This doesnt seem like a priority to me, @amitgalitz wdyt?

EncryptorUtils improve handling of which fields are encrypted

Currently the only field we need to encrypt is the credential field of the Create Connector Step. I think we can hold off on this item until we have a new field that requires encryption

Template writeable comment called out on line 245

Perhaps we can just write the Template json string to the output stream and then parse the Template when reading it in. We can avoid having to write a complex writeTo method

[amitgalitz]

Get Workflow State get the full template?

This doesnt seem like a priority to me, @amitgalitz wdyt?

Yeah don't think its a must even for GA, I also made an issue for this earlier so can probably be removed from the TODO list as its tracked there. #171

EncryptorUtils improve handling of which fields are encrypted

Currently the only field we need to encrypt is the credential field of the Create Connector Step. I think we can hold off on this item until we have a new field that requires encryption

agree on this, this is the only field that currently needs to be encrypted for us.

[dbwiddis]

Perhaps we can just write the Template json string to the output stream and then parse the Template when reading it in.

I think we actually do this, in which case we should remove the TODO. Can you confirm?

We could get fancy and use a byte array with CBOR or SMILE 😁

[jackiehanyang]

FlowFrameworkIndicesHandler has comments on updating/concurrency/retry (lines 443, 473, 477)

I believe we can address this in a future release, as we are not heavily relying on the update template operation currently. We can resolve this issue after thoroughly discussing the type of retry strategy that benefits us later.

[amitgalitz]

FlowFrameworkIndicesHandler has comments on updating/concurrency/retry (lines 443, 473, 477)

Right now this is has been fixed by updating the state index correctly and adding retries. Some improvements could be made but okay to not have in GA in my opinion. I will write an issue for all these TODOs

[dbwiddis]

All todos have their own issue assigned.