Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Apply a percentage when calculating memory limit on control plane deployment #11274

Open
jijiechen opened this issue Sep 2, 2024 · 0 comments
Labels
kind/improvement Improvement on an existing feature triage/accepted The issue was reviewed and is complete enough to start working on it

Comments

@jijiechen
Copy link
Member

Description

We've implemented GOMEMLIMIT based on resources.memory.limit on CP deployoment by this PR #10879

It's suggested to leave some headroom to make the GC run more frequently so that we get a smaller chance of being OOM-killed.

We may use 15% headroom for our case since the memory can grow quickly sometimes when new DPs are rolling out quickly. GOMEMLIMIT is not a hard limit to the process, it'll only impact the behaviour of the GC.

In this case, a good rule of thumb is to leave an additional 5-10% of headroom to account for memory sources the Go runtime is unaware of.
Ref: https://tip.golang.org/doc/gc-guide#Suggested_uses

We may use a helm helper like this to covert user provided values dynamitically:
kubernetes/kubernetes#91514 (comment)

Other charts are doing similar handling:
https://github.com/bitnami/charts/blob/main/bitnami/scylladb/templates/_helpers.tpl#L224

@jijiechen jijiechen added triage/pending This issue will be looked at on the next triage meeting kind/feature New feature labels Sep 2, 2024
@jakubdyszkiewicz jakubdyszkiewicz added triage/accepted The issue was reviewed and is complete enough to start working on it kind/improvement Improvement on an existing feature and removed triage/pending This issue will be looked at on the next triage meeting kind/feature New feature labels Sep 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/improvement Improvement on an existing feature triage/accepted The issue was reviewed and is complete enough to start working on it
Projects
None yet
Development

No branches or pull requests

2 participants