Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] LMQ Optimization: Revamp on top of RocksDB #8829

Open
lizhanhui opened this issue Oct 17, 2024 · 0 comments
Open

[Feature] LMQ Optimization: Revamp on top of RocksDB #8829

lizhanhui opened this issue Oct 17, 2024 · 0 comments
Assignees
@lizhanhui
Labels
type/new feature

Comments

@lizhanhui
Copy link
Contributor

Is Your Feature Request Related to a Problem?

LMQ is supposed to be light-weight in terms of resource cost and lifecycle. Current implementation is still based on file queue and does not scale as expected

Describe the Solution You'd Like

Provide a key-value based implementation

Describe Alternatives You've Considered

In-place optimization is not a viable option in the long run.

Additional Context

No response
@lizhanhui lizhanhui self-assigned this Oct 17, 2024
This was referenced Oct 17, 2024
Merged
Merged
Merged
Merged
Merged
fuyou001 pushed a commit that referenced this issue Oct 23, 2024
@lizhanhui
[ISSUE #8829]feat: add test case to RocksDBConsumeQueueOffsetTable (#…
738c9f3 
…8857)

* feat: add test case to RocksDBConsumeQueueOffsetTable, verifying forEach works properly for long topic-name

Signed-off-by: Li Zhanhui <[email protected]>

* fix: change OpenJDK distribution from adopt to corretto as the previous one is not updated anymore

Signed-off-by: Li Zhanhui <[email protected]>

* fix: unify set-java version and distribution

Signed-off-by: Li Zhanhui <[email protected]>

---------

Signed-off-by: Li Zhanhui <[email protected]>
lizhanhui added a commit that referenced this issue Oct 28, 2024
@lizhanhui
[ISSUE #8829] feat: provide ConfigManagerV2 to make best uses of Rock…
c8938e0 
…sDB (#8856)

* feat: provide ConfigManagerV2 to make best uses of RocksDB

Signed-off-by: Li Zhanhui <[email protected]>

* fix: release RocksDB objects using try-with-resource

Signed-off-by: Li Zhanhui <[email protected]>

---------

Signed-off-by: Li Zhanhui <[email protected]>
@lizhanhui lizhanhui mentioned this issue Nov 5, 2024
Merged
RongtongJin pushed a commit that referenced this issue Nov 6, 2024
@lizhanhui
[ISSUE #8829] Keep data version while reload and XXXConfigManagerV2 t…
311b831 
…urns off sync

Signed-off-by: Li Zhanhui <[email protected]>
@lizhanhui lizhanhui mentioned this issue Nov 13, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@lizhanhui lizhanhui

Labels
type/new feature
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@lizhanhui