issue: 3884801 Check for resident hugepages to avoid SIGBUS #204
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pasis
changed the title
pasis
added
enhancement
| if (rc < 0) { | ||
| __log_info_dbg("mincore() failed to verify hugepages (errno=%d)", errno); | ||
| } else if (resident_nr != pages_nr) { | ||
| __log_info_dbg("Not all hugepages are resident (allocated=%zu resident=%zu)", |
There was a problem hiding this comment.
I guess we need to add it as info or at least some information that hugepages will not be used.
There was a problem hiding this comment.
higher level logic (xlio_allocator) iterates over available and suitable hugepage sizes in some order until allocation succeeds or all the hugepages fail. The warning message about falling back to regular pages is printed by xlio_allocator. So it's completely ok to fail here and try the next hugepage size.
AlexanderGrissik
force-pushed
the
vNext
branch
from
cfce2e8
to
51c2340
Compare
In a containerized environment mmap() beyond the hugepages limit can be successful, but a further memory access triggers SIGBUS and terminates the process. Check that all the allocated hugepages are resident with mincore() to avoid the SIGBUS issue. Cost of the feature depends on the allocation size and hugepage size: - 32GB with 2MB pages takes 4400 us - 32GB with 1GB pages takes 11 us - 32MB with 2MB pages takes 5-6 us We expect a big memory preallocation at the start (2GB by default) and rare 32MB allocations during warmup period. Therefore, this feature can add several additional 5us latency spikes during warmup period. Signed-off-by: Dmytro Podgornyi <[email protected]>
pasis
force-pushed
the
hugepage-mincore
branch
from
b04003d
to
437bb39
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
In a containerized environment mmap() beyond the hugepages limit can be successful, but a further memory access triggers SIGBUS and terminates the process. Check that all the allocated hugepages are resident with mincore() to avoid the SIGBUS issue.
Cost of the feature depends on the allocation size and hugepage size:
We expect a big memory preallocation at the start (2GB by default) and
rare 32MB allocations during warmup period. Therefore, this feature can
add several additional 5us latency spikes during warmup period.
What
Check for resident hugepages after allocation.
Why ?
Avoid SIGBUS in containerized environments.
Change type
What kind of change does this PR introduce?
Check list