-
Notifications
You must be signed in to change notification settings - Fork 605
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(v2): implement ReadIndex for linearizable reads #3619
Conversation
fc05998
to
910451b
Compare
3a1a54b
to
17673e1
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for taking care of this @marcsanmi!
@korniltsev could you please take a look as well? Just to make 100% sure the approach is valid
We might need it for deployments outside k8s
a2b5e4e
to
49c6249
Compare
I've reverted the min ready check:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
This PR implements the ReadIndex in the Metastore to ensure linearizable reads, as described in the Raft consensus algorithm. The implementation leverages the Hashicorp Raft library's existing features while adding necessary checks to guarantee read consistency.
Key changes:
ReadIndex
method to verify leadership and return current commit index.readIndex
method to ensure the node is up-to-date before serving reads.QueryMetadata
andGetProfileStats
to usereadIndex
before querying data.CheckReady
method to usereadIndex
for consistency checks.Raft Algorithm Considerations:
runLeader
is called for the leader to be up to the latest commit index, which aligns with the Raft paper's recommendation for ensuring a committed entry from the current term (thanks @kolesnikovae for the heads up).Testing Considerations: