Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix the ingest rate displayed in the CLI. #4682

Merged
merged 3 commits into from
May 10, 2024
Merged

Fix the ingest rate displayed in the CLI. #4682

merged 3 commits into from
May 10, 2024

Commits on May 10, 2024

  1. Fix the ingest rate displayed in the CLI. 

    The CLI was showing the average rate since the CLI was started.

This PR changes it to an estimation over the last 2 seconds.
    @fulmicoton
    fulmicoton committed May 10, 2024
    Configuration menu
    Copy the full SHA
    c857732 View commit details
    Browse the repository at this point in the history

  2. Bugfix in the rate estimator 

    The rate estimator was returning inflated results whenever
we did not record any work during a bucket period.

The fix does two things:
- It replace the 1-bit color logic `(bucket_ord / num_buckets) % 2`` to a more generic and more accurate 8 bit hash `bucket_ord % 243`.

Collision may only happen if no write targetted this specific bucket for `num_buckets x 243 x bucket_period`.

Upon read, it checks for the hash and dismisses buckets that are not matching the hash.
    @fulmicoton
    fulmicoton committed May 10, 2024
    Configuration menu
    Copy the full SHA
    6449f2f View commit details
    Browse the repository at this point in the history

  3. Update quickwit/quickwit-common/src/tower/rate_estimator.rs 

    Co-authored-by: trinity-1686a <[email protected]>
    @fulmicoton @trinity-1686a
    fulmicoton and trinity-1686a committed May 10, 2024
    Configuration menu
    Copy the full SHA
    718553a View commit details
    Browse the repository at this point in the history