Fix concurrent map read/write in partition management #454
+110
−56
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
The current implementation of
partitionMap
is not thread-safe, leading to potential race conditions during concurrent read and write operations. This issue affects various parts of the codebase, including cluster management, node operations, and partition parsing.Issue References
#446
#399
Solution
Implement a thread-safe
partitionMap
usingsync.Map
and update all related functions to use the new thread-safe methods. This change ensures safe concurrent access to partition data across multiple goroutines.Technical Notes
partitionMap
is a thread-safe map that stores partition information for different namespaces. It uses async.Map
internally to provide concurrent read/write access without explicit locking. The keys are namespace names (strings), and the values are pointers to Partitions structs. This structure allows for efficient, concurrent operations on partition data across multiple goroutines.Changes
partitions.go
partitionMap
to wrap async.Map
partition.go
PartitionForWrite
andPartitionForRead
to use new get method ofpartitionMap
partition_parser.go
partitionMap
partitionMap
node.go
refreshPartitions
method to work with the newpartitionMap
structurecluster.go
partitionWriteMap
in Cluster struct to use a pointer topartitionMap
setPartitions
,getPartitions
, andfindNodeInPartitionMap
methodsNewCluster
function to initializepartitionWriteMap
correctlytend
method to use thread-safe operations onpartitionMap