Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document new Split and Sort SearchResponseProcessors #7767

Merged
2 changes: 1 addition & 1 deletion _ingest-pipelines/processors/split.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ Parameter | Required/Optional | Description
:--- | :--- | :---
`field` | Required | The field containing the string to be split.
`separator` | Required | The delimiter used to split the string. This can be a regular expression pattern.
`preserve_field` | Optional | If set to `true`, preserves empty trailing fields (for example, `''`) in the resulting array. If set to `false`, empty trailing fields are removed from the resulting array. Default is `false`.
`preserve_trailing` | Optional | If set to `true`, preserves empty trailing fields (for example, `''`) in the resulting array. If set to `false`, empty trailing fields are removed from the resulting array. Default is `false`.
dbwiddis marked this conversation as resolved.
Show resolved Hide resolved
`target_field` | Optional | The field where the array of substrings is stored. If not specified, then the field is updated in-place.
`ignore_missing` | Optional | Specifies whether the processor should ignore documents that do not contain the specified field. If set to `true`, then the processor ignores missing values in the field and leaves the `target_field` unchanged. Default is `false`.
`description` | Optional | A brief description of the processor.
Expand Down
7 changes: 5 additions & 2 deletions _search-plugins/search-pipelines/search-processors.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,13 +37,16 @@ The following table lists all supported search response processors.

Processor | Description | Earliest available version
:--- | :--- | :---
[`collapse`]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/collapse-processor/)| Deduplicates search hits based on a field value, similarly to `collapse` in a search request. | 2.12
[`personalize_search_ranking`]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/personalize-search-ranking/) | Uses [Amazon Personalize](https://aws.amazon.com/personalize/) to rerank search results (requires setting up the Amazon Personalize service). | 2.9
[`retrieval_augmented_generation`]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/rag-processor/) | Used for retrieval-augmented generation (RAG) in [conversational search]({{site.url}}{{site.baseurl}}/search-plugins/conversational-search/). | 2.10 (generally available in 2.12)
[`rename_field`]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/rename-field-processor/)| Renames an existing field. | 2.8
[`rerank`]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/rerank-processor/)| Reranks search results using a cross-encoder model. | 2.12
[`collapse`]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/collapse-processor/)| Deduplicates search hits based on a field value, similarly to `collapse` in a search request. | 2.12
[`retrieval_augmented_generation`]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/rag-processor/) | Used for retrieval-augmented generation (RAG) in [conversational search]({{site.url}}{{site.baseurl}}/search-plugins/conversational-search/). | 2.10 (generally available in 2.12)
[`sort`]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/sort-processor/)| Sorts an array of items in either ascending or descending order. | 2.16
[`split`]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/split-processor/)| Splits a string field into an array of substrings based on a specified delimiter. | 2.16
[`truncate_hits`]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/truncate-hits-processor/)| Discards search hits after a specified target count is reached. Can undo the effect of the `oversample` request processor. | 2.12


## Search phase results processors

A search phase results processor runs between search phases at the coordinating node level. It intercepts the results retrieved from one search phase and transforms them before passing them to the next search phase.
Expand Down
251 changes: 251 additions & 0 deletions _search-plugins/search-pipelines/sort-processor.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,251 @@
---
layout: default
title: Sort
nav_order: 32
has_children: false
parent: Search processors
grand_parent: Search pipelines
---

# Sort processor

The `sort` processor sorts an array of items in either ascending or descending order. Numeric arrays are sorted numerically, while string or mixed arrays (strings and numbers) are sorted lexicographically. The processor throws an error if the input is not an array.

## Request fields

The following table lists all available request fields.

Field | Data type | Description
:--- | :--- | :---
`field` | String | The field to be sorted. Must be an array. Required.
`order` | String | The sort order to apply. Accepts `asc` for ascending or `desc` for descending. Default is `asc`.
`target_field` | String | The name of the field in which the sorted array is stored. If not specified, then the sorted array is stored in the same field as the original array (the `field` variable).
`tag` | String | The processor's identifier.
`description` | String | A description of the processor.
`ignore_failure` | Boolean | If `true`, OpenSearch [ignores any failure]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/creating-search-pipeline/#ignoring-processor-failures) of this processor and continues to run the remaining processors in the search pipeline. Optional. Default is `false`.
dbwiddis marked this conversation as resolved.
Show resolved Hide resolved

## Example

The following example demonstrates using a search pipeline with a `sort` processor.

### Setup

Create an index named `my_index` and index a document with the field `message` that contains an array of strings:

```json
POST /my_index/_doc/1
{
"message": ["one", "two", "three", "four"],
"visibility": "public"
}
```
{% include copy-curl.html %}

### Creating a search pipeline

Create a search pipeline with a `sort` response processor that sorts the `message` field and stores the sorted results in the `sorted_message` field:

```json
PUT /_search/pipeline/my_pipeline
{
"response_processors": [
{
"sort": {
"field": "message",
"target_field": "sorted_message"
}
}
]
}
```
{% include copy-curl.html %}

### Using a search pipeline

Search for documents in `my_index` without a search pipeline:

```json
GET /my_index/_search
```
{% include copy-curl.html %}

The response contains the field `message`:

<details open markdown="block">
<summary>
Response
</summary>
{: .text-delta}

```json
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 1,
"hits": [
{
"_index": "my_index",
"_id": "1",
"_score": 1,
"_source": {
"message": [
"one",
"two",
"three",
"four"
],
"visibility": "public"
}
}
]
}
}
```
</details>

To search with a pipeline, specify the pipeline name in the `search_pipeline` query parameter:

```json
GET /my_index/_search?search_pipeline=my_pipeline
```
{% include copy-curl.html %}

The `sorted_message` field contains the strings from the `message` field sorted alphabetically:

<details open markdown="block">
<summary>
Response
</summary>
{: .text-delta}

```json
{
"took": 3,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 1,
"hits": [
{
"_index": "my_index",
"_id": "1",
"_score": 1,
"_source": {
"visibility": "public",
"sorted_message": [
"four",
"one",
"three",
"two"
],
"message": [
"one",
"two",
"three",
"four"
]
}
}
]
}
}
```
</details>

You can also use the `fields` option to search for specific fields in a document:

```json
POST /my_index/_search?pretty&search_pipeline=my_pipeline
{
"fields": ["visibility", "message"]
}
```
{% include copy-curl.html %}

In the response, the `message` field is sorted and the results are stored in the `sorted_message` field:

<details open markdown="block">
<summary>
Response
</summary>
{: .text-delta}

```json
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 1,
"hits": [
{
"_index": "my_index",
"_id": "1",
"_score": 1,
"_source": {
"visibility": "public",
"sorted_message": [
"four",
"one",
"three",
"two"
],
"message": [
"one",
"two",
"three",
"four"
]
},
"fields": {
"visibility": [
"public"
],
"sorted_message": [
"four",
"one",
"three",
"two"
],
"message": [
"one",
"two",
"three",
"four"
]
}
}
]
}
}
```
</details>
Loading
Loading