Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Radial Search #1166

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

alex-keeler
Copy link

Add minScore, maxDistance parameters to KnnQuery

Description

OpenSearch 2.14 added support for Radial Search. This change added the ability to specify a max_distance or min_score parameter on the knn query, which act as a relevancy threshold for neighboring documents. Exactly one of k, max_distance, or min_score must be provided to the knn query.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@dblock
Copy link
Member

dblock commented Aug 27, 2024

Thanks @alex-keeler.

Please amend with -s for DCO to pass.

Please check that these fields are present in https://github.com/opensearch-project/opensearch-api-specification, and maybe even add a test to use these queries as we are working on generating much of this code from spec.

@alex-keeler
Copy link
Author

It appears the specification (schemas/_common.yaml#/components/schemas/KnnQuery) is inconsistent with the current implementation of the knn query. It specifies a similarity parameter, which from what I can tell appears to be based on an iteration of the proposal prior to its final release. I can open an issue and perhaps take a crack at a pull request when I get the chance.

Xtansia and others added 6 commits August 29, 2024 10:04
* Generate ml.register_model_group

Signed-off-by: Thomas Farr <[email protected]>

* Start neural search sample

Signed-off-by: Thomas Farr <[email protected]>

* Re-generate ShardStatistics

Signed-off-by: Thomas Farr <[email protected]>

* Re-generate ShardFailure

Signed-off-by: Thomas Farr <[email protected]>

* Re-generate Result

Signed-off-by: Thomas Farr <[email protected]>

* Re-generate WriteResponseBase

Signed-off-by: Thomas Farr <[email protected]>

* Generate ml.delete_model_group

Signed-off-by: Thomas Farr <[email protected]>

* Generate ml.register_model

Signed-off-by: Thomas Farr <[email protected]>

* Exclude legacy license from ml namespace

Signed-off-by: Thomas Farr <[email protected]>

* Generate ml.get_task

Signed-off-by: Thomas Farr <[email protected]>

* Generate ml.delete_task

Signed-off-by: Thomas Farr <[email protected]>

* Generate ml.delete_model

Signed-off-by: Thomas Farr <[email protected]>

* Generate ml.deploy_model

Signed-off-by: Thomas Farr <[email protected]>

* Generate ml.undeploy_model

Signed-off-by: Thomas Farr <[email protected]>

* Complete neural search sample

Signed-off-by: Thomas Farr <[email protected]>

* Generate ml.get_model

Signed-off-by: Thomas Farr <[email protected]>

* Add changelog entry

Signed-off-by: Thomas Farr <[email protected]>

* note

Signed-off-by: Thomas Farr <[email protected]>

* Fix tests

Signed-off-by: Thomas Farr <[email protected]>

---------

Signed-off-by: Thomas Farr <[email protected]>
Signed-off-by: Alex Keeler <[email protected]>
Add minScore, maxDistance parameters to KnnQuery in order to support Radial Search, which was introduced in OpenSearch 2.14
https://opensearch.org/docs/latest/search-plugins/knn/radial-search-knn/

Signed-off-by: Alex Keeler <[email protected]>
Signed-off-by: alex-keeler <[email protected]>
Signed-off-by: Alex Keeler <[email protected]>
@alex-keeler
Copy link
Author

I'm not entirely sure why the WhiteSource Security Check failed, any ideas why it wasn't able to locate the branch?

On that note, is there anything else I should do before my pull request is ready to merge? The API spec is up to date and all of the checks have been passing (minus this one). Following up on this since it's been inactive for a couple of weeks.

@reta
Copy link
Collaborator

reta commented Sep 12, 2024

I'm not entirely sure why the WhiteSource Security Check failed, any ideas why it wasn't able to locate the branch?

The check is very unstable, sorry about that @alex-keeler

@dblock
Copy link
Member

dblock commented Sep 13, 2024

I'm hoping this gets (eventually) covered by the code generator, cc: @Xtansia.

@alex-keeler
Copy link
Author

alex-keeler commented Sep 17, 2024

Given the somewhat odd nature of the API spec for this particular query (see opensearch-project/opensearch-api-specification#538 (comment)), I'm a tad skeptical that the code generator will be able to handle this case. The java code had already deviated slightly from the API implementation before I touched it. That being said, I haven't looked at the code generator, so I could be wrong. Just a heads up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants