Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add goroutine watcher #16

Merged
merged 10 commits into from
Jul 24, 2024
Merged

feat: add goroutine watcher #16

merged 10 commits into from
Jul 24, 2024

Conversation

jake-shin0
Copy link
Contributor

@jake-shin0 jake-shin0 commented Apr 24, 2024

  • queryer was managed as a separate dir and its implementation was divided. (cgroup & runtime)
  • Queryer interface method to public.
  • As I was dividing queryer, I also wanted to manage profiler in dir, but I did not consider this part.
  • Since both monitor and profile are managed through the profile singleton object in pprof lib, I considered implementing it by injecting and managing a common singleton object in queryer and profiler, but did not do so.
    • The singleton object inside the pprof lib is managed as a map, making its usability convenient, but the public function it returns is a list, making its usability a bit ambiguous.
    • if creating an internal singleton object, it must be taken care of concurrency issues.
    • It is not a frequently called object, and its call frequency is determined by ticker, so pprof lib is used in both queryer and profiler.

@@ -53,8 +64,9 @@ type autoPprof struct {
reportBoth bool
Copy link
Contributor Author

@jake-shin0 jake-shin0 Apr 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This field is an option that is only valid when monitoring only two things, CPU and memory, and as the runtime monitor increases, its meaning becomes less meaningful.

@mingrammer Please give me your opinion on how to remove it in a separate version!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should introduce a new option, reportAll, which reports all profile results when any threshold is exceeded. And deprecate the reportBoth.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea! I will create this idea as a separate PR.

@jake-shin0 jake-shin0 marked this pull request as ready for review April 25, 2024 06:50
@jake-shin0 jake-shin0 changed the title [FEED-8337] feat: add goroutine watcher feat: add goroutine watcher Apr 25, 2024
@@ -858,13 +1020,13 @@ func BenchmarkLightJob(b *testing.B) {

func BenchmarkLightJobWithWatchCPUUsage(b *testing.B) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think we need to benckmark goroutine watcher too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I artificially created a lot of goroutines and ran a benchmark test. This is my first time doing a test like this. What should I fix?

BenchmarkLightAsyncJob-5                                  376760             32056 ns/op            7040 B/op        352 allocs/op
BenchmarkLightAsyncJobWithWatchGoroutineCount-5           350994             33689 ns/op            7040 B/op        351 allocs/op
BenchmarkHeavyAsyncJob-5                                     414          29095456 ns/op         6001969 B/op     300096 allocs/op
BenchmarkHeavyAsyncJobWithWatchGoroutineCount-5              403          29474789 ns/op         5972177 B/op     298607 allocs/op

go.sum Outdated Show resolved Hide resolved
autopprof.go Show resolved Hide resolved
func (s *SlackReporter) ReportGoroutineProfile(
ctx context.Context, r io.Reader, gi GoroutineInfo,
) error {
hostname, _ := os.Hostname() // Don't care about this error.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

p999; I think this comment is vague, how about just removing it?

queryer/cgroupv1.go Show resolved Hide resolved
queryer/cgroupv2.go Show resolved Hide resolved
@@ -53,8 +64,9 @@ type autoPprof struct {
reportBoth bool
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should introduce a new option, reportAll, which reports all profile results when any threshold is exceeded. And deprecate the reportBoth.

autopprof.go Outdated Show resolved Hide resolved
queryer/cgroupv1.go Outdated Show resolved Hide resolved
queryer/cgroupv1.go Outdated Show resolved Hide resolved
queryer/cgroupv1.go Outdated Show resolved Hide resolved
queryer/cgroupv1_test.go Outdated Show resolved Hide resolved
queryer/cgroupv2_test.go Outdated Show resolved Hide resolved
queryer/queue.go Outdated Show resolved Hide resolved
queryer/runtime_test.go Outdated Show resolved Hide resolved
queryer/runtime_test.go Outdated Show resolved Hide resolved
@jake-shin0
Copy link
Contributor Author

jake-shin0 commented Jul 18, 2024

Copy link
Contributor

@mingrammer mingrammer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@jake-shin0 jake-shin0 merged commit b91e419 into main Jul 24, 2024
16 checks passed
@jake-shin0 jake-shin0 deleted the FEED-8337 branch July 24, 2024 06:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants