Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] ISM shrunken index auto manage experience #882

Open
Kruzerson opened this issue Aug 2, 2023 · 3 comments
Open

[BUG] ISM shrunken index auto manage experience #882

Kruzerson opened this issue Aug 2, 2023 · 3 comments
Labels
ism action question Further information is requested

Comments

@Kruzerson
Copy link

Describe the bug
I am using a data flow to get the log data and also ISM policies to rollover the data flow. In this ISM policy, after rollover the index, I want to shrink the shards using the ISM shrink policy.
The shrink action creates a new index with the suffix _shrunken. After the shrink is successfully executed, there are 2 indexes with the same data:

  1. Old original index in the data stream
  2. Another newly created index with the _shrunken suffix with fewer shards.

At this point, 2 problems arise:

  1. During the shrink, aliases are not added to the new index, which is why the index is not used in the discovery.
  2. It is not possible to manage the new _shrunken index because it is not covered by ISM

To Reproduce
Steps to reproduce the behavior:

  1. Create new Data stream
  2. Create new ISM with shrink action
  3. Get result
@Kruzerson Kruzerson added bug Something isn't working untriaged labels Aug 2, 2023
@KrooshalUX
Copy link

@opensearch-project/admin - can you please move this issue to the proper repository? thanks!

@peternied peternied transferred this issue from opensearch-project/oui Aug 2, 2023
@peternied peternied transferred this issue from opensearch-project/index-management-dashboards-plugin Aug 2, 2023
@bowenlan-amzn
Copy link
Member

@Kruzerson for your point1, shrink action support add new alias to the new shrunken index, https://opensearch.org/docs/latest/im-plugin/ism/policies/#shrink
Do you meet a problem it's not working correctly, or you are looking forward this feature #676

for point2: ISM can auto manage an index pattern using ism_template in policy, example https://opensearch.org/docs/latest/im-plugin/ism/policies#sample-policy-with-ism-template-for-auto-rollover
But I guess the blocker here is that shrink action only add a suffix _shrunken, but not a prefix so that you can distinguish your source index with shrunken index
Will adding prefix to shrunken index name help you in this case? like add shrunken_ prefix to the shrunken index

@bowenlan-amzn bowenlan-amzn changed the title [BUG] [BUG] shrink action usability Aug 30, 2023
@bowenlan-amzn bowenlan-amzn added the good first issue Good for newcomers label Sep 14, 2023
@bowenlan-amzn bowenlan-amzn added ism action and removed good first issue Good for newcomers labels Sep 22, 2023
@bowenlan-amzn bowenlan-amzn changed the title [BUG] shrink action usability [BUG] ISM shrunken index auto manage experience Oct 3, 2023
@bowenlan-amzn bowenlan-amzn added question Further information is requested and removed bug Something isn't working labels Nov 7, 2023
@disaster37
Copy link

Hi, I have the same matter. I doesn't found a workaround on Opensearch 2.14.0.
I can conclue that ISM shrink step can not be used on data stream index ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ism action question Further information is requested
Projects
None yet
Development

No branches or pull requests

4 participants