[BUG] ISM rollover explain does not report action.failed flag correctly

[peterzxu-crm]

Describe the bug
ISM rollover explain does not report action.failed flag correctly

{
  "test-000169": {
    "index.plugins.index_state_management.policy_id": "test",
    "index.opendistro.index_state_management.policy_id": "test",
    "index": "test-000169",
    "index_uuid": "yPHItytKTNO8OI8EBrHnyQ",
    "policy_id": "test",
    "policy_seq_no": -2,
    "policy_primary_term": 0,
    "rolled_over": false,
    "index_creation_date": 1684830592835,
    "state": {
      "name": "rollover",
      "start_time": 1684830951513
    },
    "action": {
      "name": "rollover",
      "start_time": null,
      "index": 0,
      "failed": false,
      "consumed_retries": 0,
      "last_retry_time": null
    },
    "retry_info": {
      "failed": false,
      "consumed_retries": 0
    },
    "info": {
      "cause": "Validation Failed: 1: this action would add [8] total shards, but this cluster currently has [2997]/[3000] maximum shards open;",
      "message": "Pending retry of failed managed index"
    },
    "enabled": false
  },
  "total_managed_indices": 1
}

To Reproduce
Steps to reproduce the behavior:

1. Limit the max shards per node
2. Then create indices with rollover policy to hit max shard per node configuration

Expected behavior
The action.failed flag should be set to true

Plugins

Screenshots

Host/Environment (please complete the following information):

• OS: AWS OpenSearch
• Version 2.5

Additional context

[minalsha]

Issue needs triaging from ISM SMEs.

[bowenlan-amzn]

It seems you called retry API, which would reset action and retry_info.

[peterzxu-crm]

@bowenlan-amzn Not sure what you mean by calling retry API, I am use this ISM explain API https://opensearch.org/docs/2.5/im-plugin/ism/api/#explain-index

[bowenlan-amzn]

After you called retry API https://opensearch.org/docs/2.5/im-plugin/ism/api/#retry-failed-index
The metadata will be update, code part

index-management/src/main/kotlin/org/opensearch/indexmanagement/indexstatemanagement/transport/action/retryfailedmanagedindex/TransportRetryFailedManagedIndexAction.kt

Lines 272 to 282 in a0f611d

	managedIndexMetaData.copy(
	stepMetaData = null,
	policyRetryInfo = PolicyRetryInfoMetaData(false, 0),
	actionMetaData = managedIndexMetaData.actionMetaData?.copy(
	failed = false,
	consumedRetries = 0,
	lastRetryTime = null,
	startTime = null
	),
	transitionTo = request.startState,
	info = mapOf("message" to "Pending retry of failed managed index")

[peterzxu-crm]

The code snippet says it all, thanks!