Delete workflow state when template is deleted and no resources exist

[dbwiddis]

Description

Deletes workflow state when:

• template is deleted and no resources exist
• workflow is deprovisioned and template was already deleted

Issues Resolved

Fixes #688

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

Attention: Patch coverage is 54.68750% with 29 lines in your changes are missing coverage. Please review.

Project coverage is 73.78%. Comparing base (3e24e22) to head (6770be9).
Report is 1 commits behind head on main.

Files	Patch %	Lines
...framework/indices/FlowFrameworkIndicesHandler.java	60.00%	17 Missing and 1 partial ⚠️
.../transport/DeprovisionWorkflowTransportAction.java	50.00%	5 Missing and 1 partial ⚠️
...ework/transport/DeleteWorkflowTransportAction.java	28.57%	5 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main     #689      +/-   ##
============================================
+ Coverage     72.56%   73.78%   +1.21%     
- Complexity      697      722      +25     
============================================
  Files            82       82              
  Lines          3605     3662      +57     
  Branches        309      312       +3     
============================================
+ Hits           2616     2702      +86     
+ Misses          845      818      -27     
+ Partials        144      142       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[dbwiddis]

Provisioned a no-op template:

POST localhost:9200/_plugins/_flow_framework/workflow
{
  "name": "test",
  "description": "Flow template",
  "use_case": "TESTING",
  "version": {
    "template": "1.0.0",
    "compatibility": ["2.12.0", "3.0.0"]
  },
  "workflows": {
    "provision": {
      "user_params": {},
      "nodes": [
        {
          "id": "no op",
          "type": "noop"
        }
      ]
    }
  }
}

{
    "workflow_id": "vZAyLY8BaPsQZP-UDULE"
}

State exists:

GET localhost:9200/_plugins/_flow_framework/workflow/vZAyLY8BaPsQZP-UDULE/_status?all=true

{
    "workflow_id": "vZAyLY8BaPsQZP-UDULE",
    "state": "NOT_STARTED",
    "provisioning_progress": "NOT_STARTED"
}

Deleted template:

DELETE localhost:9200/_plugins/_flow_framework/workflow/vZAyLY8BaPsQZP-UDULE

{
    "_index": ".plugins-flow-framework-templates",
    "_id": "vZAyLY8BaPsQZP-UDULE",
    "_version": 2,
    "result": "deleted",
    "_shards": {
        "total": 1,
        "successful": 1,
        "failed": 0
    },
    "_seq_no": 1,
    "_primary_term": 1
}

State entry is gone!

GET localhost:9200/_plugins/_flow_framework/workflow/vZAyLY8BaPsQZP-UDULE/_status?all=true

{
    "error": "Fail to find workflow vZAyLY8BaPsQZP-UDULE"
}

Provisioned a template with resources (full template omitted):

{
    "workflow_id": "vpA5LY8BaPsQZP-UJkIb"
}

State exists with resources

localhost:9200/_plugins/_flow_framework/workflow/vpA5LY8BaPsQZP-UJkIb/_status?all=true

{
    "workflow_id": "vpA5LY8BaPsQZP-UJkIb",
    "state": "COMPLETED",
    "provisioning_progress": "DONE",
    "provision_start_time": 1714450671376,
    "provision_end_time": 1714450677712,
    "resources_created": [
        {
            "workflow_step_name": "create_connector",
            "resource_id": "v5A5LY8BaPsQZP-UK0Ia",
            "resource_type": "connector_id",
            "workflow_step_id": "create_connector_1"
        },
        {
            "workflow_step_name": "register_remote_model",
            "resource_id": "wpA5LY8BaPsQZP-UNUIO",
            "resource_type": "model_id",
            "workflow_step_id": "register_model_2"
        },
        {
            "workflow_step_name": "deploy_model",
            "resource_id": "wpA5LY8BaPsQZP-UNUIO",
            "resource_type": "model_id",
            "workflow_step_id": "register_model_2"
        },
        {
            "workflow_step_name": "register_agent",
            "resource_id": "xJA5LY8BaPsQZP-UPULJ",
            "resource_type": "agent_id",
            "workflow_step_id": "sub_agent"
        },
        {
            "workflow_step_name": "register_agent",
            "resource_id": "xZA5LY8BaPsQZP-UP0Ij",
            "resource_type": "agent_id",
            "workflow_step_id": "root_agent"
        }
    ]
}

Deleting the template:

localhost:9200/_plugins/_flow_framework/workflow/vpA5LY8BaPsQZP-UJkIb

{
    "_index": ".plugins-flow-framework-templates",
    "_id": "vpA5LY8BaPsQZP-UJkIb",
    "_version": 3,
    "result": "deleted",
    "_shards": {
        "total": 1,
        "successful": 1,
        "failed": 0
    },
    "_seq_no": 4,
    "_primary_term": 1
}

State is not deleted!

localhost:9200/_plugins/_flow_framework/workflow/vpA5LY8BaPsQZP-UJkIb/_status?all=true

{
    "workflow_id": "vpA5LY8BaPsQZP-UJkIb",
    "state": "COMPLETED",
    "provisioning_progress": "DONE",
    "provision_start_time": 1714450671376,
    "provision_end_time": 1714450677712,
    "resources_created": [
        {
            "workflow_step_name": "create_connector",
            "resource_id": "v5A5LY8BaPsQZP-UK0Ia",
            "resource_type": "connector_id",
            "workflow_step_id": "create_connector_1"
        },
        {
            "workflow_step_name": "register_remote_model",
            "resource_id": "wpA5LY8BaPsQZP-UNUIO",
            "resource_type": "model_id",
            "workflow_step_id": "register_model_2"
        },
        {
            "workflow_step_name": "deploy_model",
            "resource_id": "wpA5LY8BaPsQZP-UNUIO",
            "resource_type": "model_id",
            "workflow_step_id": "register_model_2"
        },
        {
            "workflow_step_name": "register_agent",
            "resource_id": "xJA5LY8BaPsQZP-UPULJ",
            "resource_type": "agent_id",
            "workflow_step_id": "sub_agent"
        },
        {
            "workflow_step_name": "register_agent",
            "resource_id": "xZA5LY8BaPsQZP-UP0Ij",
            "resource_type": "agent_id",
            "workflow_step_id": "root_agent"
        }
    ]
}

Deprovisioning:

localhost:9200/_plugins/_flow_framework/workflow/vpA5LY8BaPsQZP-UJkIb/_deprovision

{
    "workflow_id": "vpA5LY8BaPsQZP-UJkIb"
}

State document deleted!

localhost:9200/_plugins/_flow_framework/workflow/vpA5LY8BaPsQZP-UJkIb/_status?all=true

{
    "error": "Fail to find workflow vpA5LY8BaPsQZP-UJkIb"
}

[amitgalitz]

Can we add one integration test here also (we might already have one with delete/deprovision but we should now check this new logic is applied)

[owaiskazi19]

Overall the fix looks good. IMO, we should also include a force flag in the delete API to delete the template from all the indices.

[dbwiddis]

Overall the fix looks good. IMO, we should also include a force flag in the delete API to delete the template from all the indices.

Can we make that a suggestion for 2.15? I'm keeping this PR to the context of bug fix, and a new API parameter is a new feature.

#692