-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use DorisVector to avoid memory usage from not being traced #41557
Conversation
Thank you for your contribution to Apache Doris. Since 2024-03-18, the Document has been moved to doris-website. |
be/src/vec/common/custom_allocator.h
Outdated
@@ -23,8 +23,8 @@ | |||
template <class T, typename MemoryAllocator = Allocator<true>> | |||
class CustomStdAllocator; | |||
|
|||
template <typename T> | |||
using DorisVector = std::vector<T, CustomStdAllocator<T>>; | |||
template <typename T, bool clean_memory = false> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
rename to clear_memory
@@ -53,6 +53,9 @@ Status AggLocalState::init(RuntimeState* state, LocalStateInfo& info) { | |||
_hash_table_size_counter = | |||
ADD_COUNTER_WITH_LEVEL(Base::profile(), "HashTableSize", TUnit::UNIT, 1); | |||
|
|||
_container_memory_usage = ADD_COUNTER(profile(), "ContainerMemoryUsage", TUnit::BYTES); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
rename to MemoryUsageContainer MemoryUsageArena
c79bf06
to
b44b7a1
Compare
} | ||
} | ||
Base::keys = input_keys.data(); | ||
} | ||
|
||
size_t serialized_keys_size(bool is_build) const override { | ||
return is_build ? build_arena.size() : Base::arena.size(); | ||
return is_build ? (build_stored_keys.size() * sizeof(StringRef)) | ||
: (stored_keys.size() * sizeof(StringRef)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
感觉这个不对吧,这个计算的偏小了。应该加上arena
b44b7a1
to
211f359
Compare
Proposed changes
And merge hash table of agg in thread of pipeline tasks.