You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
tsdb benchmarks at a rate of 100 per s or so works fine for me, but anything significantly more (say 200) and graphite-api just starts timeouting, cpu usage is fairly low but I think the way its concurrency model works is it can only do a very limited amount of requests at the same time (by default num_cpus times 2), even though it actually spends much of its time waiting.
this means that we can't actually stresstest kairos or NMT, because graphite-api runs in trouble much earlier.
i see that we can override the concurrency but this looks very expensive (each process takes about 35MB RSS on my system)
is there an easy fix? probably not.
anyway, right now nmt/kairos both perform at around 4ms. hopefully + probably once we can test with more data (see https://github.com/raintank/ops/issues/170#issuecomment-151926833) that will be sufficient to compare both stacks at fairly low rates and this isn't really issue.
btw for dev/prod it would be nice if we could somehow monitor the "busyness" of each process, to anticipate how close we are to saturating our capacity.
The text was updated successfully, but these errors were encountered:
tsdb benchmarks at a rate of 100 per s or so works fine for me, but anything significantly more (say 200) and graphite-api just starts timeouting, cpu usage is fairly low but I think the way its concurrency model works is it can only do a very limited amount of requests at the same time (by default num_cpus times 2), even though it actually spends much of its time waiting.
this means that we can't actually stresstest kairos or NMT, because graphite-api runs in trouble much earlier.
i see that we can override the concurrency but this looks very expensive (each process takes about 35MB RSS on my system)
is there an easy fix? probably not.
anyway, right now nmt/kairos both perform at around 4ms. hopefully + probably once we can test with more data (see https://github.com/raintank/ops/issues/170#issuecomment-151926833) that will be sufficient to compare both stacks at fairly low rates and this isn't really issue.
btw for dev/prod it would be nice if we could somehow monitor the "busyness" of each process, to anticipate how close we are to saturating our capacity.
The text was updated successfully, but these errors were encountered: