-
Notifications
You must be signed in to change notification settings - Fork 434
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[GLUTEN-7800][VL] Add config for max reclaim wait time to avoid dead lock when memory arbitration #7799
[GLUTEN-7800][VL] Add config for max reclaim wait time to avoid dead lock when memory arbitration #7799
Conversation
Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues? https://github.com/apache/incubator-gluten/issues Then could you also rename commit message and pull request title in the following format?
See also: |
Run Gluten Clickhouse CI |
@zhztheplayer would you help take a look? |
#include "velox/common/memory/SharedArbitrator.h" | ||
#include "velox/exec/MemoryReclaimer.h" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why we need to include SharedArbitrator.h
? Thought we should avoid this kind of dependency. One of the reason is, code of the shared arbitrator is usually updated frequently so we may easily find our code doesn't work after a Velox rebase.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just want reuse velox::memory::SharedArbitrator::ExtraConfig
, would you give some input?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Only one question about the header dependency. Thanks.
A bit dramatical that upstream Velox really updated the function name in It causes main CI build failure and let's revert this PR |
What changes were proposed in this pull request?
Add config
spark.gluten.sql.columnar.backend.velox.reclaimMaxWaitMs
to avoid dead lock when memory arbitration has bug, default timeout is 60min.