[DOC] Search Pipeline Metrics

[macohen]

What do you want to do?

• Request a change to existing documentation
• Add new documentation
• Report a technical problem with the documentation
• Other

Tell us about your request. Provide a summary of the request and all versions that are affected.
Relating to Search pipeline metrics (at least on par w/ ingestion pipeline metrics).#6723, we want to add a page under Search Pipelines that show what metrics we are exposing.

This affects 2.9 only as this feature is new to 2.9.

What other resources are available? Provide links to related issues, POCs, steps for testing, etc.

[noCharger]

Metrics overview opensearch-project/OpenSearch#6723 (comment)
PR opensearch-project/OpenSearch#8053

[noCharger]

Example request / response

% curl -XGET 'localhost:9200/_nodes/stats/search_pipeline?pretty'
{
  "_nodes" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "cluster_name" : "runTask",
  "nodes" : {
    "CpvTK7KuRD6Oww8TTp8g2Q" : {
      "timestamp" : 1689007282929,
      "name" : "runTask-0",
      "transport_address" : "127.0.0.1:9300",
      "host" : "127.0.0.1",
      "ip" : "127.0.0.1:9300",
      "roles" : [
        "cluster_manager",
        "data",
        "ingest",
        "remote_cluster_client"
      ],
      "attributes" : {
        "testattr" : "test",
        "shard_indexing_pressure_enabled" : "true"
      },
      "search_pipeline" : {
        "total_request" : {
          "count" : 5,
          "time_in_millis" : 158,
          "current" : 0,
          "failed" : 0
        },
        "total_response" : {
          "count" : 2,
          "time_in_millis" : 1,
          "current" : 0,
          "failed" : 0
        },
        "pipelines" : {
          "public_info" : {
            "request" : {
              "count" : 3,
              "time_in_millis" : 71,
              "current" : 0,
              "failed" : 0
            },
            "response" : {
              "count" : 0,
              "time_in_millis" : 0,
              "current" : 0,
              "failed" : 0
            },
            "request_processors" : [
              {
                "filter_query:abc" : {
                  "type" : "filter_query",
                  "stats" : {
                    "count" : 1,
                    "time_in_millis" : 0,
                    "current" : 0,
                    "failed" : 0
                  }
                }
              },
              {
                "filter_query" : {
                  "type" : "filter_query",
                  "stats" : {
                    "count" : 4,
                    "time_in_millis" : 2,
                    "current" : 0,
                    "failed" : 0
                  }
                }
              }
            ],
            "response_processors" : [ ]
          },
          "guest_pipeline" : {
            "request" : {
              "count" : 2,
              "time_in_millis" : 87,
              "current" : 0,
              "failed" : 0
            },
            "response" : {
              "count" : 2,
              "time_in_millis" : 1,
              "current" : 0,
              "failed" : 0
            },
            "request_processors" : [
              {
                "script" : {
                  "type" : "script",
                  "stats" : {
                    "count" : 2,
                    "time_in_millis" : 86,
                    "current" : 0,
                    "failed" : 0
                  }
                }
              },
              {
                "filter_query:abc" : {
                  "type" : "filter_query",
                  "stats" : {
                    "count" : 1,
                    "time_in_millis" : 0,
                    "current" : 0,
                    "failed" : 0
                  }
                }
              },
              {
                "filter_query" : {
                  "type" : "filter_query",
                  "stats" : {
                    "count" : 3,
                    "time_in_millis" : 0,
                    "current" : 0,
                    "failed" : 0
                  }
                }
              }
            ],
            "response_processors" : [
              {
                "rename_field" : {
                  "type" : "rename_field",
                  "stats" : {
                    "count" : 2,
                    "time_in_millis" : 1,
                    "current" : 0,
                    "failed" : 0
                  }
                }
              }
            ]
          }
        }
      }
    }
  }
}

Note:

1. filter_query:abc refering to <type>:<tag>, if the "tag" field is not specified, stats of multiple different processors with same type will be aggregated, hense the stats depends on perf of each individual processor.
2. No change on metrics exposed in ingest group.

[noCharger]

1. Add one row in metric groups https://opensearch.org/docs/latest/api-reference/nodes-apis/nodes-stats/#ingest

search_pipeline |  Statistics about search pipelines.

2. Create a table similiar to ingest metrics one https://opensearch.org/docs/latest/api-reference/nodes-apis/nodes-stats/#ingest

Field	Field type	Description
total_request.count	Integer	The total number of search request processor executions.
total_request.time_in_millis	Integer	The total amount of time for search request processor executions, in milliseconds.
total_request.current	Integer	The total number of search request processor executions currently executing in progress.
total_request.failed	Integer	The total number of failed search request processor executions.
total_response.count	Integer	The total number of search response processor executions.
total_response.time_in_millis	Integer	The total amount of time for search response processor executions, in milliseconds.
total_response.current	Integer	The total number of search response processor executions currently in progress.
total_response.failed	Integer	The total number of failed search response processor executions.
pipelines	Object	Search pipeline statistics. Each pipeline is a nested object specified by its ID with the properties listed below.
pipelines.id.request.count	Integer	The number of search request processor executions by the search pipeline.
pipelines.id.request.time_in_millis	Integer	The total amount of time for search request processor executions in the search pipeline, in milliseconds.
pipelines.id.request.current	Integer	The total number of search request processor executions currently in progress for the search pipeline.
pipelines.id.request.failed	Integer	The total number of failed search request processor executions for the search pipeline.
pipelines.id.response.count	Integer	The number of search response processor executions by the search pipeline.
pipelines.id.response.time_in_millis	Integer	The total amount of time for search response processor executions in the search pipeline, in milliseconds.
pipelines.id.response.current	Integer	The total number of search response processor executions currently in progress for the search pipeline.
pipelines.id.response.failed	Integer	The total number of failed search response processor executions for the search pipeline.
pipelines.id.request_processors	Array of objects	Statistics for the search request processors. Includes the total number of executions, the total amount of time of executions, the total number of executions currently in progress. the number of failed executions.
pipelines.id.response_processors	Array of objects	Statistics for the search response processors. Includes the total number of executions, the total amount of time of executions, the total number of executions currently in progress. the number of failed executions.