Weekly Release 2023-10-26

This is the weekly bundle and EKS-A CLI release from main.

Date: 2023-10-26
Build-tooling repo commit: 02d13b176542e42ef75efa3fab8e19ce0f590c2a
CLI repo commit: 36a789f

v0.17.4

Supported OS version details

	vSphere	Bare Metal	Nutanix	CloudStack	Snow
Ubuntu	20.04	20.04	20.04	Not supported	20.04
	22.04	22.04	22.04	Not supported	Not supported
Bottlerocket	1.14.3	1.14.3	Not supported	Not supported	Not supported
RHEL	8.7	8.7	Not supported	8.7	Not supported

Added

• Enabled audit logging for kube-apiserver on baremetal provider (#6779).

v0.17.3

Supported OS version details

	vSphere	Bare Metal	Nutanix	CloudStack	Snow
Ubuntu	20.04	20.04	20.04	Not supported	20.04
	22.04	22.04	22.04	Not supported	Not supported
Bottlerocket	1.13.1	1.13.1	Not supported	Not supported	Not supported
RHEL	8.7	8.7	Not supported	8.7	Not supported

Fixed

• Fixed cli upgrade mgmt kubeconfig flag (#6679)
• Bare Metal: Prevent machine config references from changing to existing machine configs (#6674)

v0.17.2

Supported OS version details

	vSphere	Bare Metal	Nutanix	CloudStack	Snow
Ubuntu	20.04	20.04	20.04	Not supported	20.04
	22.04	22.04	22.04	Not supported	Not supported
Bottlerocket	1.13.1	1.13.1	Not supported	Not supported	Not supported
RHEL	8.7	8.7	Not supported	8.7	Not supported

Fixed

• Bare Metal: Ensure new worker node groups can reference new machine configs (#6615)
• Bare Metal: Fix writefile action to ensure Bottlerocket configs write content or error (#2441)

Added

• Added support for configuring healthchecks on EtcdadmClusters using etcdcluster.cluster.x-k8s.io/healthcheck-retries annotation (aws/etcdadm-controller#44)
• Add check for making sure quorum is maintained before deleting etcd machines (aws/etcdadm-controller#46)

Changed

• Only delete one etcd machine at a time when multiple are failing healthchecks (aws/etcdadm-controller#46)

v0.17.1

Supported OS version details

	vSphere	Bare Metal	Nutanix	CloudStack	Snow
Ubuntu	20.04	20.04	20.04	Not supported	20.04
	22.04	22.04	22.04	Not supported	Not supported
Bottlerocket	1.13.1	1.13.1	Not supported	Not supported	Not supported
RHEL	8.7	8.7	Not supported	8.7	Not supported

Fixed

• Fix worker node groups being rolled when labels adjusted #6330
• Fix worker node groups being rolled out when taints are changed #6482
• Fix vSphere template tags validation to run on the control plane and etcd VSpherMachinesConfig #6591
• Fix Bare Metal upgrade with custom pod CIDR #6442

Added

• Add validation for missing management cluster kubeconfig during workload cluster operations #6501

Weekly Release 2023-08-24

This is the weekly bundle and EKS-A CLI release from main.

Date: 2023-08-24
Build-tooling repo commit: 69e65d2a0062918863e201017186e96c5fd457ca
CLI repo commit: bdfefc8

Weekly Release 2023-08-17

This is the weekly bundle and EKS-A CLI release from main.

Date: 2023-08-17
Build-tooling repo commit: fe5d9a28d9dbd4198a0d1bbe12d3b6a2470723b0
CLI repo commit: 2907104

v0.17.0

Supported OS version details

	vSphere	Bare Metal	Nutanix	CloudStack	Snow
Ubuntu	20.04	20.04	20.04	Not supported	20.04
	22.04	22.04	22.04	Not supported	Not supported
Bottlerocket	1.13.1	1.13.1	Not supported	Not supported	Not supported
RHEL	8.7	8.7	Not supported	8.7	Not supported

Note: We have updated the image-builder docs to include the latest enhancements. Please refer to the image-builder docs for more details.

Added

• Add support for AWS CodeCommit repositories in FluxConfig with git configuration #4290
• Add new information to the EKS Anywhere Cluster status #5628:
  • Add the ControlPlaneInitialized, ControlPlaneReady, DefaultCNIConfigured, WorkersReady, and Ready conditions.
  • Add the observedGeneration field.
  • Add the failureReason field.
• Add support for different machine templates for control plane, etcd, and worker node in vSphere provider #4255
• Add support for different machine templates for control plane, etcd, and worker node in Cloudstack provider #6291
• Add support for Kubernetes version 1.25, 1.26, and 1.27 to CloudStack provider #6167
• Add bootstrap cluster backup in the event of cluster upgrade error #6086
• Add support for organizing virtual machines into categories with the Nutanix provider #6014
• Add support for configuring egressMasqueradeInterfaces option in Cilium CNI via EKS Anywhere cluster spec #6018
• Add support for a flag for create and upgrade cluster to skip the validation --skip-validations=vsphere-user-privilege
• Add support for upgrading control plane nodes separately from worker nodes for vSphere, Nutanix, Snow, and Cloudstack providers #6180
• Add preflight validation to prevent skip eks-a minor version upgrades #5688
• Add preflight check to block using kindnetd CNI in all providers except Docker #6097

Upgraded

• Cluster API Provider vSphere: v1.6.1 to v1.7.0
• Cluster API Provider CloudStack: v0.4.9-rc5 to v0.4.9-rc6
• Cluster API Provider Nutanix: v1.2.1 to v1.2.3

Cilium Upgrades

• Cilium: v1.11.15 to v1.12.11 (changelog)

  Note: If you are using the vSphere provider with the Redhat OS family, there is a known issue with VMWare and the new Cilium version that only affects our Redhat variants. To prevent this from affecting your upgrade from EKS Anywhere v0.16 to v0.17, we are adding a temporary daemonset to disable UDP offloading on the nodes before upgrading Cilium. After your cluster is upgraded, the daemonset will be deleted. This note is strictly informational as this change requires no additional effort from the user.

Changed

• Change the default node startup timeout from 10m to 20m in Bare Metal provider #5942
• EKS Anywhere now fails on pre-flights if a user does not have required permissions. #5865
• eksaVersion field in the cluster spec is added for better representing CLI version and dependencies in EKS-A cluster #5847
• vSphere datacenter insecure and thumbprint is now mutable for upgrades when using full lifecycle API 6143

Fixed

• Fix cluster creation failure when the <Provider>DatacenterConfig is missing apiVersion field #6096
• Allow registry mirror configurations to be mutable for Bottlerocket OS #2336
• Patch an issue where mutable fields in the EKS Anywhere CloudStack API failed to trigger upgrades #5910
• image builder: Fix runtime issue with git in image-builder v0.16.2 binary #2360
• Bare Metal: Fix issue where metadata requests that return non-200 responses were incorrectly treated as OK #2256

Known Issues

• Upgrading Docker clusters from previous versions of EKS Anywhere may not work on Linux hosts due to an issue in the Cilium 1.11 to 1.12 upgrade. Docker clusters is meant solely for testing and not recommended or support for production use cases. There is currently no fixed planned.
• If you are installing EKS Anywhere Packages, Kubernetes versions 1.23-1.25 are incompatible with Kubernetes versions 1.26-1.27 due to an API difference. This means that you may not have worker nodes on Kubernetes version <= 1.25 when the control plane nodes are on Kubernetes version >= 1.26. Therefore, if you are upgrading your control plane nodes to 1.26, you must upgrade all nodes to 1.26 to avoid failures.
• There is a known bug with systemd >= 249 and all versions of Cilium. This is currently known to only affect Ubuntu 22.04. This will be fixed in future versions of EKS Anywhere. To work around this issue, run one of the follow options on all nodes.

Option A

# Does not persist across reboots.
sudo ip rule add from all fwmark 0x200/0xf00 lookup 2004 pref 9  
sudo ip rule add from all fwmark 0xa00/0xf00 lookup 2005 pref 10  
sudo ip rule add from all lookup local pref 100  

Option B

# Does persist across reboots.
# Add these values /etc/systemd/networkd.conf
[Network]
ManageForeignRoutes=no
ManageForeignRoutingPolicyRules=no

Deprecated

• The bundlesRef field in the cluster spec is now deprecated in favor of the new eksaVersion field. This field will be deprecated in three versions.

Removed

• Installing vSphere CSI Driver as part of vSphere cluster creation. For more information on how to self-install the driver refer to the documentation here

⚠️ Breaking changes

• CLI: --force-cleanup has been removed from create cluster, upgrade cluster and delete cluster commands. For more information on how to troubleshoot issues with the bootstrap cluster refer to the troubleshooting guide (1 and 2). #6384

v0.16.5

Changed

• Bump up the worker count for etcdadm-controller from 1 to 10 #34
• Add 2X replicas hard limit for rolling out new etcd machines #37

Fixed

• Fix code panic in healthcheck loop in etcdadm-controller #41
• Fix deleting out of date machines in etcdadm-controller #40

v0.16.4

Fixed

• Fix support for having management cluster and workload cluster in different namespaces #6414