Does ebs-csi-driver have impact on the AZ chosen when EKS replace a worker node?

[ensean]

Hello, we are planning to deploy a stateful application(Clickhouse) to EKS with the support of ebs-csi-driver.

But we are concerning that during the process of worker node replacement(may be due to hardware failure etc.), is there a chance that an instance in a different AZ is launched, causing EBS volume mount failure?

/triage support

[k8s-ci-robot]

@ensean: The label(s) triage/support cannot be applied, because the repository doesn't have them.

In response to this:

Hello, we are planning to deploy a stateful application(Clickhouse) to EKS with the support of ebs-csi-driver.

But we are concerning that during the process of worker node replacement(may be due to hardware failure etc.), is there a chance that an instance in a different AZ is launched, causing EBS volume mount failure?

/triage support

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[ConnorJC3]

Hi @ensean - EBS volumes are a zonal resource, so any instances using a volume must be in the same zone as the volume (for more info see the AWS docs for EBS volumes). The EBS CSI Driver will automatically populate a CSI Topology label with the key topology.kubernetes.io/zone containing the AZ of the associated volume/node.

Most node scalers such as cluster-autoscaler or Karpenter either automatically are aware of CSI topology labels or can be configured to do so. For example, here is the Karpenter documentation about PV-aware scheduling.

[ensean]

Hi @ConnorJC3 , thanks a lot for your explanation.

Suppose the AZ where the PV belongs fails, will ebs-csi-driver try to copy the PV to another AZ through snapshot?

[AndrewSirenko]

Suppose the AZ where the PV belongs fails, will ebs-csi-driver try to copy the PV to another AZ through snapshot?

No, the ebs-csi-driver will not try to copy a failed PV to another AZ through snapshot.

You may be able to write your own Kubernetes Operator which would do this.