Skip to content

Commit

Permalink
default_policy: Builder: forbid preferring racks without DCs
Browse files Browse the repository at this point in the history
This is an API restriction continuing of what has already been done
in bb09756: as the case of providing
preferred rack without specifying the preferred datacenter is invalid,
in spirit of the principle “Make illegal states unrepresentable”,
`DefaultBuilder`'s API is changed accordingly to disallow creating
such configuration.
A stray backslash is deleted from a docstring.

Docs are updated to reflect the API change.
  • Loading branch information
wprzytula committed Jul 31, 2023
1 parent 9d76db1 commit 2118631
Show file tree
Hide file tree
Showing 2 changed files with 53 additions and 57 deletions.
47 changes: 24 additions & 23 deletions docs/source/load-balancing/default-policy.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,7 @@ for queries with non-local consistency mode is also supported.
`builder()` method of `DefaultPolicy` returns a new instance of
`DefaultPolicyBuilder` with the following default values:

- `preferred_datacenter`: `None`
- `preferred_rack`: `None`
- `preferences`: no particular datacenter/rack preference
- `is_token_aware`: `true`
- `permit_dc_failover`: `false`
- `latency_awareness`: `None`
Expand All @@ -25,8 +24,7 @@ You can use the builder methods to configure the desired settings and create a
use scylla::load_balancing::DefaultPolicy;

let default_policy = DefaultPolicy::builder()
.prefer_datacenter("dc1".to_string())
.prefer_rack("rack1".to_string())
.prefer_datacenter_and_rack("dc1".to_string(), "rack1".to_string())
.token_aware(true)
.permit_dc_failover(true)
.build();
Expand All @@ -35,36 +33,39 @@ let default_policy = DefaultPolicy::builder()

### Semantics of `DefaultPolicy`

#### Preferred Datacenter
#### Preferences

The `preferences` field in `DefaultPolicy` allows the load balancing
policy to prioritize nodes based on their location. It has three modes:

- no preference
- preferred datacenter
- preferred datacenter and rack

The `preferred_datacenter` field in `DefaultPolicy` allows the load balancing
policy to prioritize nodes based on their location. When a preferred datacenter
is set, the policy will treat nodes in that datacenter as "local" nodes, and
nodes in other datacenters as "remote" nodes. This affects the order in which
nodes are returned by the policy when selecting replicas for read or write
operations. If no preferred datacenter is specified, the policy will treat all
nodes as local nodes.
When a datacenter `"my_dc"` is preferred, the policy will treat nodes in `"my_dc"`
as "local" nodes, and nodes in other datacenters as "remote" nodes. This affects
the order in which nodes are returned by the policy when selecting replicas for
read or write operations. If no datacenter is preferred, the policy will treat
all nodes as local nodes.

`preferences` allow the load balancing policy to prioritize nodes based on their
availability zones (racks) in the preferred datacenter, too. When a datacenter
and a rack are preferred, the policy will first return replicas in the local rack
in the preferred datacenter, and then the other replicas in the datacenter
(followed by remote replicas).

When datacenter failover is disabled (`permit_dc_failover` is set to
false), the default policy will only include local nodes in load balancing
plans. Remote nodes will be excluded, even if they are alive and available to
serve requests.

#### Preferred Rack

The `preferred_rack` field in `DefaultPolicy` allows the load balancing policy to
prioritize nodes based on their availability zones in the preferred datacenter.
When a preferred rack is set, the policy will first return replicas in the local rack
in the preferred datacenter, and then the other replicas in the datacenter.
When a preferred datacenter is not set, setting preferred rack will not have any effect.

#### Datacenter Failover

In the event of a datacenter outage or network failure, the nodes in that
datacenter may become unavailable, and clients may no longer be able to access
the data stored on those nodes. To address this, the `DefaultPolicy` supports datacenter
failover, which allows to route requests to nodes in other datacenters if the
local nodes are unavailable.
the data stored on those nodes. To address this, the `DefaultPolicy` supports
datacenter failover, which allows to route requests to nodes in other datacenters
if the local nodes are unavailable.

Datacenter failover can be enabled in `DefaultPolicy` by `permit_dc_failover`
setting in the builder. When this flag is set, the policy will prefer to return
Expand Down
63 changes: 29 additions & 34 deletions scylla/src/transport/load_balancing/default.rs
Original file line number Diff line number Diff line change
Expand Up @@ -636,8 +636,7 @@ impl Default for DefaultPolicy {
/// # }
#[derive(Clone, Debug)]
pub struct DefaultPolicyBuilder {
preferred_datacenter: Option<String>,
preferred_rack: Option<String>,
preferences: ReplicaLocationPreference,
is_token_aware: bool,
permit_dc_failover: bool,
latency_awareness: Option<LatencyAwarenessBuilder>,
Expand All @@ -648,8 +647,7 @@ impl DefaultPolicyBuilder {
/// Creates a builder used to customise configuration of a new DefaultPolicy.
pub fn new() -> Self {
Self {
preferred_datacenter: None,
preferred_rack: None,
preferences: ReplicaLocationPreference::Any,
is_token_aware: true,
permit_dc_failover: false,
latency_awareness: None,
Expand All @@ -668,24 +666,8 @@ impl DefaultPolicyBuilder {
Box::new(DefaultPolicy::is_alive)
};

// As the case of providing preferred rack without providing datacenter is invalid, the rack is then ignored.
// According to the principle “Make illegal states unrepresentable”, in the next major release we will
// alter the `DefaultPolicyBuilder`'s API so that it is impossible for the user to create such state.
let preferences = match (self.preferred_datacenter, self.preferred_rack) {
(None, None) => ReplicaLocationPreference::Any,
(None, Some(_)) => {
// This a is case that the user shouldn't be able to represent.
warn!("Preferred rack has effect only if a preferred datacenter is set as well. Ignoring the preferred rack.");
ReplicaLocationPreference::Any
}
(Some(datacenter), None) => ReplicaLocationPreference::Datacenter(datacenter),
(Some(datacenter), Some(rack)) => {
ReplicaLocationPreference::DatacenterAndRack(datacenter, rack)
}
};

Arc::new(DefaultPolicy {
preferences,
preferences: self.preferences,
is_token_aware: self.is_token_aware,
permit_dc_failover: self.permit_dc_failover,
pick_predicate,
Expand All @@ -694,22 +676,28 @@ impl DefaultPolicyBuilder {
})
}

/// Sets the rack to be preferred by this policy
/// Sets the datacenter to be preferred by this policy.
///
/// Allows the load balancing policy to prioritize nodes based on their availability zones
/// in the preferred datacenter.
/// When a preferred rack is set, the policy will first return replicas in the local rack
/// in the preferred datacenter, and then the other replicas in the datacenter.
/// Allows the load balancing policy to prioritize nodes based on their location.
/// When a preferred datacenter is set, the policy will treat nodes in that
/// datacenter as "local" nodes, and nodes in other datacenters as "remote" nodes.
/// This affects the order in which nodes are returned by the policy when
/// selecting replicas for read or write operations. If no preferred datacenter
/// is specified, the policy will treat all nodes as local nodes.
///
/// When a preferred datacenter is not set, setting preferred rack will not have any effect.
pub fn prefer_rack(mut self, rack_name: String) -> Self {
self.preferred_rack = Some(rack_name);
/// When datacenter failover is disabled (`permit_dc_failover` is set to false),
/// the default policy will only include local nodes in load balancing plans.
/// Remote nodes will be excluded, even if they are alive and available
/// to serve requests.
pub fn prefer_datacenter(mut self, datacenter_name: String) -> Self {
self.preferences = ReplicaLocationPreference::Datacenter(datacenter_name);
self
}

/// Sets the datacenter to be preferred by this policy.
/// Sets the datacenter and rack to be preferred by this policy.
///
/// Allows the load balancing policy to prioritize nodes based on their location.
/// Allows the load balancing policy to prioritize nodes based on their location
/// as well as their availability zones in the preferred datacenter.
/// When a preferred datacenter is set, the policy will treat nodes in that
/// datacenter as "local" nodes, and nodes in other datacenters as "remote" nodes.
/// This affects the order in which nodes are returned by the policy when
Expand All @@ -720,8 +708,15 @@ impl DefaultPolicyBuilder {
/// the default policy will only include local nodes in load balancing plans.
/// Remote nodes will be excluded, even if they are alive and available
/// to serve requests.
pub fn prefer_datacenter(mut self, datacenter_name: String) -> Self {
self.preferred_datacenter = Some(datacenter_name);
///
/// When a preferred rack is set, the policy will first return replicas in the local rack
/// in the preferred datacenter, and then the other replicas in the datacenter.
pub fn prefer_datacenter_and_rack(
mut self,
datacenter_name: String,
rack_name: String,
) -> Self {
self.preferences = ReplicaLocationPreference::DatacenterAndRack(datacenter_name, rack_name);
self
}

Expand All @@ -740,7 +735,7 @@ impl DefaultPolicyBuilder {
/// In the case of `DefaultPolicy`, token awareness is enabled by default,
/// meaning that the policy will prefer to return alive local replicas
/// if the token is available. This means that if the client is requesting data
/// that falls within the token range of a particular node, the policy will try\
/// that falls within the token range of a particular node, the policy will try
/// to route the request to that node first, assuming it is alive and responsive.
///
/// Token awareness can significantly improve the performance and scalability
Expand Down

0 comments on commit 2118631

Please sign in to comment.