Fixing bug that was causing the Indexer to stay busy in cooperative m… #4452
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
…ode.
Removing the difference between Idle and Processing.
It was originally introduced with the idea of measure how long actors
where busy or idle, but tokio-console does that job very well now.
The code was relying on the enforcement of the non trivial state machine:
calling .idle() in the pause state had no effect for instance.
The real bug however showed up with the following sequence of event:
but only consuming high priority messages. The
inbox.is_empty()
exitcondiition is never satisfied, so that this loop would keep going,
yielding at each iteration, until the Resume Command is received.
This is done on the indexer runtime.
The task would always be busy, but yield all of the time, consuming the
whatever room is available on the thread it is consuming.
Closes #4448
Test
The bug was doing a using yielding loop, so unit testing is a difficult.
I tested locally and on k8s.
A small pipeline of 100KB/s now takes <1% cpu vs 1 full core before.
Outside of merges, indexing 100200kb=20MB on a single node takes 1 cpu instead of 4.
Outside of merges, indexing 1000200kb=20MB on a 10 nodes takes 1 cpu instead of 4.