Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory optimizations: remove BTF and kallsyms caches #2937

Merged
merged 2 commits into from
Sep 20, 2024

Conversation

tpapagian
Copy link
Member

Do not cache large objects like the BTF and ksyms. Reload them when needed.

Before:
Screenshot 2024-09-19 at 11 09 15 AM

After:
Screenshot 2024-09-19 at 11 31 21 AM

@tpapagian tpapagian added the release-note/misc This PR makes changes that have no direct user impact. label Sep 19, 2024
@tpapagian tpapagian marked this pull request as ready for review September 20, 2024 11:44
@tpapagian tpapagian requested a review from a team as a code owner September 20, 2024 11:44
We use ksyms for checking the proper exit hooks when loading the base
sensor. Furthermore, we use it when we add a kprobe policy for a
function that is part of a kernel module.

Having that always in memory, uses a lot of memory. This patch makes the
read of ksyms when we need that.

Signed-off-by: Anastasios Papagiannis <[email protected]>
@tpapagian tpapagian force-pushed the pr/apapag/memory_opts branch 2 times, most recently from 247482f to 59386c6 Compare September 20, 2024 12:05
@tpapagian tpapagian marked this pull request as draft September 20, 2024 12:11
@tpapagian tpapagian marked this pull request as ready for review September 20, 2024 12:41
Copy link
Member

@mtardy mtardy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!! I converged to writing this independently today, thanks for doing it :)

Comment on lines -22 to 23
var (
kernelSymbols *Ksyms
setKernelSymbols sync.Once
)

func KernelSymbols() (*Ksyms, error) {
var err error
setKernelSymbols.Do(func() {
kernelSymbols, err = NewKsyms(option.Config.ProcFS)
})
return kernelSymbols, err
return NewKsyms(option.Config.ProcFS)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Could we add a comment there? it might be a little bit confusing why we had this function in the first place.

@@ -124,6 +125,9 @@ func (s *Sensor) Load(bpfDir string) error {
progsAdd(s.Progs)
AllMaps = append(AllMaps, s.Maps...)

// cleanup the BTF once we have loaded all sensor's program
btf.FlushKernelSpec()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😻 perfect!

pkg/sensors/program/loader.go Outdated Show resolved Hide resolved
@mtardy mtardy changed the title Memory optimizations Memory optimizations: remove BTF and kallsyms caches Sep 20, 2024
This patch flushes the BTF when we complete the loading of a sensor.

Signed-off-by: Anastasios Papagiannis <[email protected]>
@tpapagian tpapagian merged commit 6f70926 into main Sep 20, 2024
40 checks passed
@tpapagian tpapagian deleted the pr/apapag/memory_opts branch September 20, 2024 19:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-note/misc This PR makes changes that have no direct user impact.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants