Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[log] optimize get term #6

Merged
merged 1 commit into from
Apr 10, 2024
Merged

[log] optimize get term #6

merged 1 commit into from
Apr 10, 2024

Conversation

ehds
Copy link
Owner

@ehds ehds commented Apr 9, 2024

pick: baidu#438

leader 上 _logs_in_memory 中缓存的 LogEntry 会在 apply 完成且写到盘上后被清理掉。
leader 在 append_entries rpc 中需要获取 prev_log_term。
如果请求的并发比较小,发送新Log时,前一个LogEntry 很有可能已经从缓存清理掉了,prev_log_term 只能从 LogStorage 获取。(场景: 一个server上,node 数量特别多,但是单个node qps 比较小,能观察到 bvar raft_read_term_from_storage_second 很高)
这个 pr 在 clear_memory_logs 时多保留 log,使得上面场景总能通过缓存查询到 prev_log_term。
如果长时间不写,最后这个保留的 log 会在打快照的时候被清理掉。

@ehds ehds added the enhancement New feature or request label Apr 9, 2024
@ehds
Copy link
Owner Author

ehds commented Apr 9, 2024

TODO: 进一步的优化点: 当机器/进程重启后,可以将最新的日志放入 memory log 中提高读取最后一条日志的速度。

当机器/进程包含很多raft group时,重启后会突然获得大量的请求,需要读取最后一条日志(主要是获得term),都会触发磁盘读取操作。

@ehds ehds merged commit fab82e8 into master Apr 10, 2024
8 checks passed
ehds added a commit to ehds/mbraft that referenced this pull request Apr 16, 2024
ehds added a commit to ehds/mbraft that referenced this pull request Apr 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants