Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Support searches across raw data indices and rollup indices together #1213

Closed
sarthakaggarwal97 opened this issue Jul 26, 2024 · 3 comments · Fixed by #1268
Closed
Labels
enhancement New request

Comments

@sarthakaggarwal97
Copy link
Contributor

Is your feature request related to a problem?
Currently, we do not support searches if the same request has both raw indices and the indices with associated rollup job. We return the exception upon validation if both types of indices are present in the request.

What solution would you like?
Support searches across both all indices in order to query / aggregate results irrespective of whether the rollup job is assigned or not to a particular index.

@sarthakaggarwal97 sarthakaggarwal97 changed the title [FEATURE] Support searches across raw data indices and rollup indices together. [FEATURE] Support searches across raw data indices and rollup indices together Jul 26, 2024
@mgodwan
Copy link
Member

mgodwan commented Jul 31, 2024

Is there any historical reason around why we are not supporting this? Is this just because of query rewrite we trigger via rollup interceptor?

Also, we would need to test this flow and verify that the aggregations across indices (raw, rolled up) yield correct results.

@dblock dblock removed the untriaged label Aug 19, 2024
@dblock
Copy link
Member

dblock commented Aug 19, 2024

Catch All Triage - 1, 2, 3

@manojfaria
Copy link

manojfaria commented Sep 18, 2024

In the absence of this feature, users of opensearch have to evaluate the option of issuing two separate queries i.e. one query to the raw index, and one query to the rollup-indices and then merge the two results in their application code.

Some customers/users of OpenSearch are reporting this double querying (once to raw, and additional query to rollup indices) as a major hurdle, as this workaround of separately querying raw and rollup indices requires significant code change to their product/app.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants