Skip to content

Commit

Permalink
Remove StorageExtensions
Browse files Browse the repository at this point in the history
It's deprecated and marked to be ignored in 1.30.

Signed-off-by: Tom Wieczorek <[email protected]>
  • Loading branch information
twz123 committed Jul 3, 2024
1 parent c7a2fe9 commit 3e4e576
Show file tree
Hide file tree
Showing 14 changed files with 29 additions and 743 deletions.
8 changes: 4 additions & 4 deletions cmd/config/validate_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,9 @@ func validConfig() []byte {

func invalidConfig() []byte {
config := v1beta1.DefaultClusterConfig()
config.Spec.Extensions.Storage.Type = "external_storage"
// CreateDefaultStorageClass is not supported with external_storage
config.Spec.Extensions.Storage.CreateDefaultStorageClass = true
// NLLB cannot be used with external address
config.Spec.Network.NodeLocalLoadBalancing.Enabled = true
config.Spec.API.ExternalAddress = "k0s.example.com"
cfg, _ := yaml.Marshal(config)
return cfg
}
Expand All @@ -50,7 +50,7 @@ func TestValidateCmd(t *testing.T) {
errOut := bytes.NewBuffer(nil)
cmd.SetErr(errOut)
assert.Error(t, cmd.Execute())
assert.Contains(t, errOut.String(), "default storage class for external_storage")
assert.Contains(t, errOut.String(), "node-local load balancing cannot be used in conjunction with an external Kubernetes API server address")
errOut.Reset()
cmd.SetIn(bytes.NewReader(validConfig()))
assert.NoError(t, cmd.Execute())
Expand Down
17 changes: 0 additions & 17 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,9 +63,6 @@ spec:
concurrencyLevel: 5
charts: null
repositories: null
storage:
create_default_storage_class: false
type: external_storage
installConfig:
users:
etcdUser: etcd
Expand Down Expand Up @@ -526,20 +523,6 @@ In the runtime the image names are calculated as `my.own.repo/calico/kube-contro
`spec.extensions.helm` is the config file key in which you configure the list of [Helm](https://helm.sh) repositories and charts to deploy during cluster bootstrap (for more information, refer to [Helm Charts](helm-charts.md)).
### `spec.extensions.storage`
`spec.extensions.storage` controls bundled storage provider.
The default value `external` makes no storage deployed.
To enable [embedded host-local storage provider](examples/openebs.md) use the following configuration:
```yaml
spec:
extensions:
storage:
type: openebs_local_storage
```
### `spec.konnectivity`
The `spec.konnectivity` key is the config file key in which you configure Konnectivity-related settings.
Expand Down
77 changes: 3 additions & 74 deletions docs/examples/openebs.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,8 @@
# OpenEBS

This tutorial covers the installation of OpenEBS as a Helm extension, both from
scratch and how to migrate it from a storage extension.

## Installing OpenEBS from scratch

**WARNING**: Do not configure OpenEBS as both a storage extension and a Helm
extension. It's considered an invalid configuration and k0s will entirely ignore
the configuration to prevent accidental upgrades or downgrades. The chart
objects defined in the API will still behave normally.

OpenEBS can be installed as a helm chart by adding it as an extension to your configuration:
This tutorial covers the installation of OpenEBS as a Helm extension. OpenEBS
can be installed as a helm chart by adding it as an extension to your
configuration:

```yaml
extensions:
Expand All @@ -33,69 +25,6 @@ OpenEBS can be installed as a helm chart by adding it as an extension to your co
If you want OpenEBS to be your default storage class, set `isDefaultClass` to `true`.

## Migrating bundled OpenEBS to helm extension

The bundled OpenEBS extension is already a helm extension installed as a
`chart.helm.k0sproject.io`. For this reason, all we have to do is to remove the
manifests and to clean up the object. However, this must be done in a specific order
to prevent data loss.

**WARNING**: Not following the steps in the precise order presented by the
documentation may provoke data loss.

The first step to perform the migration is to disable the `applier-manager`
component on all controllers. For each controller, restart the controller
with the flag `--disable-components=applier-manager`. If you already had this flag,
set it to `--disable-components=<previous value>,applier-manager`.

Once the `applier-manager` is disabled in every running controller, you need to modify
the configuration to use `external_storage` instead of `openebs_local_storage`.

If you are using [dynamic configuration](../dynamic-configuration.md), you can
change it with this command:

```shell
kubectl patch clusterconfig -n kube-system k0s --patch '{"spec":{"extensions":{"storage":{"type":"external_storage"}}}}' --type=merge
```

If you are using a static configuration file, replace `spec.extensions.storage.type`
from `openebs_local_storage` to `external_storage` in all control plane nodes and
restart all the control plane nodes one by one.

When the configuration is set to `external_storage` and the servers are
restarted, you must manage the it as a chart object in the API:

```shell
kubectl get chart -n kube-system k0s-addon-chart-openebs -o yaml
```

First, remove the labels and annotations related to the stack applier:

```shell
k0s kc annotate -n kube-system chart k0s-addon-chart-openebs k0s.k0sproject.io/stack-checksum-
k0s kc label -n kube-system chart k0s-addon-chart-openebs k0s.k0sproject.io/stack-
```

After the annotations and labels are removed, remove the manifest file **on each
controller**. This file is located in
`<k0s-data-dir>/manifests/helm/<number>_helm_extension_openebs.yaml`, which in
most installations defaults to
`/var/lib/k0s/manifests/helm/0_helm_extension_openebs.yaml`.

**WARNING**: Not removing the old manifest file from all controllers may cause
the manifest to be reapplied, reverting your changes and potentially casuing
data loss.

Finally, we want to re-enable the `applier-manager` and restart all controllers
without the `--disable-components=applier-manager` flag.

Once the migration is coplete, you'll be able to update the OpenEBS chart.
Let's take v3.9.0 as an example:

```shell
kubectl patch chart -n kube-system k0s-addon-chart-openebs --patch '{"spec":{"version":"3.9.0"}}' --type=merge
```

## Usage

Once installed, the cluster will have two storage classes available for you to use:
Expand Down
107 changes: 0 additions & 107 deletions docs/storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,110 +26,3 @@ Different Kubernetes storage solutions are explained in the [official Kubernetes
- Portworx

If you are looking for a fault-tolerant storage with data replication, you can find a k0s tutorial for configuring Ceph storage with Rook [in here](examples/rook-ceph.md).

## Bundled OpenEBS storage (deprecated)

Bundled OpenEBS was deprecated in favor of running it [as a helm extension](./examples/openebs.md),
this documentation is maintained as a reference for existing installations.

This was done for three reasons:

1. By installing it as a helm extension, users have more control and flexibility without adding complexity.
2. It allows users to choose the OpenEBS version independent of their k0s version.
3. It makes the k0s configuration more consistent.

For new installations or to migrate existing installations, please refer to the [OpenEBS extension page](./examples/openebs.md).

The OpenEBS extension is enabled by setting [`spec.extensions.storage.type`](configuration.md#specextensionsstorage) to``openebs_local_storage`:

```yaml
spec:
extensions:
storage:
type: openebs_local_storage
```
The cluster will have two storage classes available for you to use:
```shell
k0s kubectl get storageclass
```

```shell
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
openebs-device openebs.io/local Delete WaitForFirstConsumer false 24s
openebs-hostpath openebs.io/local Delete WaitForFirstConsumer false 24s
```

The `openebs-hostpath` is the storage class that maps to the `/var/openebs/local`

The `openebs-device` is not configured and could be configured by [manifest deployer](manifests.md) accordingly to the [OpenEBS documentation](https://docs.openebs.io/)

### Example usage

Use following manifests as an example of pod with mounted volume:

```yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: nginx-pvc
namespace: default
spec:
accessModes:
- ReadWriteOnce
storageClassName: openebs-hostpath
resources:
requests:
storage: 5Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
namespace: default
labels:
app: nginx
spec:
selector:
matchLabels:
app: nginx
strategy:
type: Recreate
template:
metadata:
labels:
app: nginx
spec:
containers:
- image: nginx
name: nginx
volumeMounts:
- name: persistent-storage
mountPath: /var/lib/nginx
volumes:
- name: persistent-storage
persistentVolumeClaim:
claimName: nginx-pvc
```
```shell
k0s kubectl apply -f nginx.yaml
```

```shell
persistentvolumeclaim/nginx-pvc created
deployment.apps/nginx created
bash-5.1# k0s kc get pods
NAME READY STATUS RESTARTS AGE
nginx-d95bcb7db-gzsdt 1/1 Running 0 30s
```

```shell
k0s kubectl get pv
```

```shell
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pvc-9a7fae2d-eb03-42c3-aaa9-1a807d5df12f 5Gi RWO Delete Bound default/nginx-pvc openebs-hostpath 30s
```
2 changes: 0 additions & 2 deletions inttest/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -107,8 +107,6 @@ check-metricsscraper-singlenode: TEST_PACKAGE=metricsscraper

check-nllb: TIMEOUT=15m

check-openebs: TIMEOUT=7m

.PHONY: $(smoketests)
include Makefile.variables

Expand Down
2 changes: 0 additions & 2 deletions inttest/Makefile.variables
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@ smoketests := \
check-customdomain \
check-customports \
check-customports-dynamicconfig \
check-defaultstorage \
check-disabledcomponents \
check-dualstack \
check-dualstack-dynamicconfig \
Expand All @@ -55,7 +54,6 @@ smoketests := \
check-noderole \
check-noderole-no-taints \
check-noderole-single \
check-openebs\
check-psp \
check-reset \
check-singlenode \
Expand Down
Loading

0 comments on commit 3e4e576

Please sign in to comment.