Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Remote store] Replication handling for remote translog for primary & replicas #3786

Closed
Tracked by #5671
ashking94 opened this issue Jul 6, 2022 · 0 comments
Closed
Tracked by #5671
Labels
distributed framework enhancement Enhancement or improvement to existing feature or request Storage:Durability Issues and PRs related to the durability framework

Comments

@ashking94
Copy link
Member

Is your feature request related to a problem? Please describe.
Segment replication along with remote store for segments and translog makes creation of translog optional on replicas since translog is added for durability on the replicas. Now if we store the translog on remote store (like s3) durably, we can eliminate the need for translog on the replicas. On shard failures or failovers, the translog can be replayed from remote store. Irrespective of what approach we finalise in #3706, we need to implement a new engine for either no op replication or not supporting index/update/delete altogether. Since there would be changes in how checkpoints are perceived, we would also need to handle how the global checkpoint and local checkpoints are handled on primary and replica both. We also need to ensure that peer recovery and failovers are happening as currently.

Describe the solution you'd like
Creation of new engine that would run on replica especially. We would try to avoid making changes to Internal Engine which would run on the primary shard. This requires further brainstorming.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

@ashking94 ashking94 added enhancement Enhancement or improvement to existing feature or request untriaged labels Jul 6, 2022
@ashking94 ashking94 changed the title [Remote store] Replication handling for remote translog for replicas [Remote store] Replication handling for remote translog for primary & replicas Jul 6, 2022
@mch2 mch2 added distributed framework Storage:Durability Issues and PRs related to the durability framework and removed untriaged labels Jul 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
distributed framework enhancement Enhancement or improvement to existing feature or request Storage:Durability Issues and PRs related to the durability framework
Projects
None yet
Development

No branches or pull requests

2 participants