feat: search index tool

[yuye-aws]

Description

Implement SearchIndexTool to support agent framework. It takes in two string parameters: index and query. Here is the sample usage of this tool. Please remember to replace the model id and agent id.

POST /_plugins/_ml/agents/_register
{
  "name": "Test_Agent_For_ReAct",
  "type": "conversational",
  "description": "this is a test agent",
  "llm": {
    "model_id": "xxxxx",
    "parameters": {
      "max_iteration": 10,
      "stop_when_no_tool_found": true,
      "response_filter": "$.completion"
    }
  },
  "memory": {
    "type": "conversation_index"
  },
  "tools": [
    {
      "type": "SearchIndexTool",
      "description": "Use this tool to search index with a query. You should pass in two parameters: index and query. Index is the index name and the query is an OpenSearch DSL query."
    }
  ],
  "app_type": "my app"
}


POST /_plugins/_ml/agents/xxxxx/_execute
{
  "parameters": {
    "question": "Show me all the name field of all documents in index .plugins-ml-agent",
    "verbose": true
  }
}

The expected successful response should be

{
  "inference_results": [
    {
      "output": [
        {
          "name": "memory_id",
          "result": "xxxxx"
        },
        {
          "name": "response",
          "result": "Thought: To get the name field of documents in the .plugins-ml-agent index, I should search that index using the SearchIndexTool."
        },
        {
          "name": "response",
          "result": "\nObservation: \"{\\\"_index\\\":\\\".plugins-ml-agent\\\",\\\"_source\\\":{\\\"name\\\":\\\"Test_Agent_For_ReAct\\\"},\\\"_id\\\":\\\"xxxxx\\\",\\\"_score\\\":1.0}\\n\""
        },
        {
          "name": "response",
          "result": "Based on the information I was previously provided from searching the index, the name field of the documents in the .plugins-ml-agent index contains 'Test_Agent_For_ReAct'."
        },
        {
          "name": "response",
          "result": "The name field of the documents in the .plugins-ml-agent index is 'Test_Agent_For_ReAct'."
        }
      ]
    }
  ]
}

If there is error when parsing the input, the expected error response could be:

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "[query] is null or empty, can not process it."
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "[query] is null or empty, can not process it."
  },
  "status": 400
}

Issues Resolved

1. Implement search index tool
2. Fix bugs in unt tests

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

Attention: 38 lines in your changes are missing coverage. Please review.

Comparison is base (e662249) 69.59% compared to head (d0c9843) 69.75%.
Report is 1 commits behind head on feature/agent_framework_dev.

❗ Current head d0c9843 differs from pull request most recent head 9d1bab8. Consider uploading reports for the commit 9d1bab8 to get more accurate results

Files	Patch %	Lines
...rg/opensearch/ml/engine/tools/SearchIndexTool.java	66.66%	18 Missing and 2 partials ⚠️
...n/java/org/opensearch/ml/engine/tools/RAGTool.java	0.00%	6 Missing ⚠️
...opensearch/ml/engine/tools/PainlessScriptTool.java	0.00%	4 Missing ⚠️
...a/org/opensearch/ml/engine/tools/CatIndexTool.java	50.00%	2 Missing ⚠️
...g/opensearch/ml/engine/tools/IndexMappingTool.java	50.00%	2 Missing ⚠️
...g/opensearch/ml/engine/tools/SearchAlertsTool.java	50.00%	2 Missing ⚠️
.../java/org/opensearch/ml/engine/tools/MathTool.java	0.00%	1 Missing ⚠️
...a/org/opensearch/ml/engine/tools/VectorDBTool.java	0.00%	1 Missing ⚠️

Additional details and impacted files

@@                        Coverage Diff                        @@
##             feature/agent_framework_dev    #1750      +/-   ##
=================================================================
+ Coverage                          69.59%   69.75%   +0.16%     
- Complexity                          2662     2666       +4     
=================================================================
  Files                                242      243       +1     
  Lines                              13017    13020       +3     
  Branches                            1305     1311       +6     
=================================================================
+ Hits                                9059     9082      +23     
+ Misses                              3340     3318      -22     
- Partials                             618      620       +2     

Flag	Coverage Δ
ml-commons	69.75% <63.10%> (+0.16%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ylwu-amzn]

Please don't do force push. Otherwise, reviewers can't see the previous comments and don't know what's the latest change.

[yuye-aws]

This PR will be closed because we will move SearchIndexTool to skills repo, here is the link for new PR: opensearch-project/skills#61.