Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[criteo] repair: Improve estimated_partitions to reduce memory usage
Currently, we use the sum of the estimated_partitions from each participant node as the estimated_partitions for sstable produced by repair. This way, the estimated_partitions is the biggest possible number of partitions repair would write. Since repair will write only the difference between repair participant nodes, using the biggest possible estimation will overestimate the partitions written by repair, most of the time. The problem is that overestimated partitions makes the bloom filter consume more memory. It is observed that it causes OOM in the field. This patch changes the estimation to use a fraction of the average partitions per node instead of sum. It is still not a perfect estimation but it already improves memory usage significantly. Fixes scylladb#18140 Criteo: cherry-picked from scylladb#18141 to reduce probability of bad_alloc during repair
- Loading branch information