v1.28.2+k0s.0

What's Changed

This is the first release in 1.28 series. Below you'll find some highlights of the release, read more in our release blog post.

Kubernetes 1.28

k0s now builds with Kubernetes 1.28.2. All the Kubernetes are updated to 1.28.2.

Enhanced autopilot

k0s autopilot now allows the cluster to follow a specific update channel on an update server. The general idea is that each minor version will have it’s own dedicated update channel. That allows you to very easily keep up-to-date with frequent patch updates both upstream Kubernetes and k0s provides.

SBOM generation

We now generate full signed SBOM for each release going forward. You'll find the SPDX formatted SBOM in the release artefacts.

Extended OS testing matrix

We've extended our OS testing matrix to cover 22 OS and version combinations:

• Alpine: 3.17
• Centos: 7, 8 and 9
• Debian: 10, 11 and 12
• Fedora CoreOS: 38
• Fedora: 38
• Flatcar
• Oracle: 7.9, 8.7, 9.1
• RHEL: 7, 8 and 9
• Rocky: 8 and 9
• Ubuntu: 20.04, 22.04 and 23.04

Not only do we test different OS versions, but we also test different configurations of k0s. The matrix “dimensions” also cover different CNI providers and different kube-proxy modes (iptables vs. IPVS).

Component versions

• Kubernetes: 1.28.2
• ContainerD: 1.7.6
• RunC: 1.1.9
• Etcd: 3.5.9
• Kine: 0.10.3
• Konnectivity: 0.1.4
• Kube-router: 1.6.0
• Calico: 3.26.1
• CoreDNS: 1.11.1

All changes

• Bump github.com/otiai10/copy from 1.10.0 to 1.11.0 by @dependabot in #3021
• Add kubeproxy unittest file which was forgotten to merge with PR #2987 by @mikhail-sakhnov in #3023
• Bump pip from 23.1 to 23.1.1 in /docs by @dependabot in #3027
• Bump termcolor from 2.2.0 to 2.3.0 in /docs by @dependabot in #3028
• Bump mkdocs-material from 9.1.6 to 9.1.7 in /docs by @dependabot in #3026
• Add WaitForStatefulSet to inttest util by @juanluisvaladas in #3025
• Small improvements for k0s status client by @twz123 in #3030
• Bump mkdocs-material from 9.1.7 to 9.1.8 in /docs by @dependabot in #3033
• Bump controller-tools to v0.11.4 by @twz123 in #3034
• Let sysinfo look for more ipvs modules by @ncopa in #3031
• Bump pip from 23.1.1 to 23.1.2 in /docs by @dependabot in #3035
• Bump golangci-lint version from 1.51.1 to 1.52.2 by @juanluisvaladas in #3036
• Fix mkdoc-material to use custom colours from our CSS by @jnummelin in #3037
• Fix racy leader pool test by @twz123 in #3038
• Bump requests from 2.28.2 to 2.29.0 in /docs by @dependabot in #3043
• Unify clientset generation by @twz123 in #2851
• Use .DELETE_ON_ERROR in Makefile by @twz123 in #3046
• Bump github.com/urfave/cli from 1.22.12 to 1.22.13 by @dependabot in #3049
• Bump github.com/go-playground/validator/v10 from 10.12.0 to 10.13.0 by @dependabot in #3050
• Bump mkdocs-material from 9.1.8 to 9.1.9 in /docs by @dependabot in #3052
• Bump Go to v1.20.4 by @twz123 in #3058
• Bump mkdocs from 1.4.2 to 1.4.3 in /docs by @dependabot in #3062
• Bump regex from 2023.3.23 to 2023.5.4 in /docs by @dependabot in #3063
• Let addons inttest respect the suite's context by @twz123 in #3055
• Clear error in Helm chart status after successful reconciliation by @twz123 in #3053
• Bump regex from 2023.5.4 to 2023.5.5 in /docs by @dependabot in #3066
• Bump requests from 2.29.0 to 2.30.0 in /docs by @dependabot in #3074
• Bump google.golang.org/grpc from 1.54.0 to 1.55.0 by @dependabot in #3075
• Remove unused constant WinDataDirDefault by @twz123 in #3078
• Bump golang.org/x/sync from 0.1.0 to 0.2.0 by @dependabot in #3076
• Bump github.com/opencontainers/image-spec to v1.1.0-rc3 by @twz123 in #3080
• Helm upgrade bug fix by @makhov in #3082
• Use verbose logging in conformance tests run to qualify for k8s-conformance repository requirements by @mikhail-sakhnov in #3090
• Bump golang.org/x/sys from 0.7.0 to 0.8.0 by @dependabot in #3085
• Bump certifi from 2022.12.7 to 2023.5.7 in /docs by @dependabot in #3086
• Bump golang.org/x/tools from 0.8.0 to 0.9.0 by @dependabot in #3093
• Bump mkdocs-material from 9.1.9 to 9.1.11 in /docs by @dependabot in #3095
• Remove unused inttest util function ConfigGetter.FakeAPIConfig() by @twz123 in #3079
• Bump golang.org/x/crypto from 0.8.0 to 0.9.0 by @dependabot in #3094
• Fix copyright script by @ncopa in #3087
• Bump containerd to v1.7.1 by @twz123 in #3099
• Bump golang.org/x/tools from 0.9.0 to 0.9.1 by @dependabot in #3102
• Bump runc to v1.1.7 by @twz123 in #3040
• Bump calico image version (vuln fixes) by @makhov in #3065
• Bump etcd to v3.5.9 by @twz123 in #3108
• Bump github.com/docker/distribution from 2.8.1+incompatible to 2.8.2+incompatible by @dependabot in #3110
• Bump mkdocs-material from 9.1.11 to 9.1.12 in /docs by @dependabot in #3116
• Bump pymdown-extensions from 9.11 to 10.0 in /docs by @dependabot in #3123
• Bump pymdown-extensions from 10.0 to 10.0.1 in /docs by @dependabot in #3124
• Bump github.com/estesp/manifest-tool/v2 from 2.0.6 to 2.0.8 by @dependabot in #2856
• Update otelhttp to v0.39.0 by @twz123 in #3068
• Don't be fail-fast on errors in addons inttest by @twz123 in #3054
• Bump controller-tools to v0.12.0 by @twz123 in #3056
• Make multicontroller inttest be fail-fast by @twz123 in #3100
• Bump Alpine to v3.18 by @twz123 in #3115
• Bump kine to v0.10.1 by @twz123 in #3041
• Bump mkdocs-material from 9.1.12 to 9.1.13 in /docs by @dependabot in #3128
• Bump github.com/sirupsen/logrus from 1.9.0 to 1.9.2 by @dependabot in #3131
• Bump Kubernetes to v1.27.2 by @juanluisvaladas in #3136
• Bump Helm to v3.12.0 by @twz123 in #3057
• Use filepath.Join directly for Windows UNC paths by @twz123 in #3137
• Bump github.com/stretchr/testify from 1.8.2 to 1.8.3 by @dependabot in #3145
• Bump troubleshoot to v0.64.0 by @twz123 in #3042
• Add missing prerequisites to footlose Docker image by @twz123 in #3148
• Bump cri-dockerd to v0.3.2 by @twz123 in #3147
• Bump github.com/go-playground/validator/v10 from 10.13.0 to 10.14.0 by @dependabot in #3152
• Bump urllib3 from 1.26.15 to 1.26.16 in /docs by @dependabot in #3159
• Bump sonobuoy to v0.56.16 by @juanluisvaladas in #3157
• Bump requests from 2.30.0 to 2.31.0 in /docs by @dependabot in #3155
• Delete empty non used file by @mikhail-sakhnov in #3167
• Bump github.com/zcalusic/sysinfo from 0.9.5 to 1.0.0 by @dependabot in #3169
• Bump github.com/imdario/mergo from 0.3.15 to 0.3.16 by @dependabot in #3168
• Bump mkdocs-material from 9.1.13 to 9.1.15 in /docs by @dependabot in #3173
• Bump github.com/stretchr/testify from 1.8.3 to 1.8.4 by @dependabot in #3172
• Bump github.com/k0sproject/k0sctl from 0.15.0 to 0.15.1 in /hack/tool by @dependabot in #3177
• Bump github.com/BurntSushi/toml from 1.2.1 to 1.3.0 by @dependabot in #3176
• Bump golang.org/x/tools from 0.9.1 to 0.9.3 by @dependabot in #3181
• Bump zeebe-io/backport-action from 1.2.0 to 1.3.0 by @dependabot in #3171
• Bump markupsafe from 2.1.2 to 2.1.3 in /docs by @dependabot in #3187
• Add Apparmor probe for sysinfo by @ncopa in #3048
• Refactor configuration loading by @kke in #3072
• Bump go 1.20.5 by @ncopa in #3197
• Bump github.com/k0sproject/k0sctl from 0.15.1 to 0.15.2 in /hack/tool by @dependabot in #3196
• Correctly join PATH on windows by @kke in #3184
• Bump github.com/go-playground/validator/v10 from 10.14.0 to 10.14.1 by @dependabot in #3186
• Bump zeebe-io/backport-action from 1.3.0 to 1.3.1 by @dependabot in #3207
• Enhance release&support info on docs by @jnummelin in #3198
• Bump regex from 2023.5.5 to 2023.6.3 in /docs by @dependabot in #3188
• Bump github.com/BurntSushi/toml from 1.3.0 to 1.3.2 by @dependabot in #3203
• Bump github.com/urfave/cli from 1.22.13 to 1.22.14 by @dependabot in #3210
• Bump golang.org/x/sys from 0.8.0 to 0.9.0 by @dependabot in #3212
• Bump golang.org/x/crypto from 0.9.0 to 0.10.0 by @dependabot in #3211
• Fix calico stack applier error handling by @mikhail-sakhnov in #3201
• Issue3183/pause container configuration by @mikhail-sakhnov in #3194
• Bump golang.org/x/mod from 0.10.0 to 0.11.0 by @dependabot in #3222
• Bump helm.sh/helm/v3 from 3.12.0 to 3.12.1 by @dependabot in #3221
• Bump sonobuoy to v0.56.17 by @twz123 in #3220
• Bump golang.org/x/sync from 0.2.0 to 0.3.0 by @dependabot in #3224
• Bump kubernetes to 1.27.3 (CVE-2023-2728) by @ncopa in #3213
• Bump cri-dockerd to v0.3.3 by @twz123 in #3225
• Bump mkdocs-material from 9.1.15 to 9.1.16 in /docs by @dependabot in #3229
• Bump pyparsing from 3.0.9 to 3.1.0 in /docs by @dependabot in #3231
• Bump github.com/otiai10/copy from 1.11.0 to 1.12.0 by @dependabot in #3230
• Replace Slack with Lens Forums in the rest of the docs by @twz123 in #3232
• Clean up embedded-bins stamp files even when Docker fails by @twz123 in #3226
• Bump golang.org/x/tools from 0.9.3 to 0.10.0 by @dependabot in #3227
• Bump mkdocs-macros-plugin from 0.7.0 to 1.0.1 in /docs by @dependabot in #3166
• Increase golangci-lint timeout by @twz123 in #3238
• Bump google.golang.org/grpc from 1.55.0 to 1.56.1 by @dependabot in #3237
• Refactor GitHub workflows for reusability by @twz123 in #3234
• Bump mkdocs-material from 9.1.16 to 9.1.18 in /docs by @dependabot in #3252
• Bump github.com/zcalusic/sysinfo from 1.0.0 to 1.0.1 by @dependabot in #3251
• Bump github.com/opencontainers/image-spec from 1.1.0-rc3 to 1.1.0-rc4 by @dependabot in #3253
• Update 'High Availability' and 'Custom CA' docs by @Friz-zy in #3245
• Add a script to tag and push k0s releases by @twz123 in #3244
• Remove special-case for setup-go on ARMv7 by @twz123 in #3193
• correct typo in metallb helm override causing failure by @kengz in #3254
• Bump golang.org/x/mod from 0.11.0 to 0.12.0 by @dependabot in #3257
• Bump containerd to 1.7.2 by @ncopa in #3255
• Sort inttests alphabetically by @twz123 in #3261
• Bump golang.org/x/sys from 0.9.0 to 0.10.0 by @dependabot in #3256
• Bump golang.org/x/crypto from 0.10.0 to 0.11.0 by @dependabot in #3258
• Bump shogo82148/actions-upload-release-asset from 1.6.4 to 1.6.5 by @dependabot in #3266
• Bump shogo82148/actions-create-release from 1.4.1 to 1.4.2 by @dependabot in #3265
• Bump google.golang.org/grpc from 1.56.1 to 1.56.2 by @dependabot in #3267
• DNS fixes in docker entrypoint for both legacy and nft by @twz123 in #3268
• Bump golang.org/x/tools from 0.10.0 to 0.11.0 by @dependabot in #3264
• Bump github.com/Microsoft/hcsshim from 0.10.0-rc.8 to 0.10.0-rc.9 by @dependabot in #3263
• Don't mention latest Docker tag in docs by @twz123 in #3269
• Bump click from 8.1.3 to 8.1.4 in /docs by @dependabot in #3271
• Fix typo in function name by @twz123 in #3277
• Bump cri-dockerd to v0.3.4 by @twz123 in #3278
• Bump troubleshoot to v0.69.3 by @twz123 in #3279
• Bump mkdocs-macros-plugin from 1.0.1 to 1.0.2 in /docs by @dependabot in #3284
• Canonical spelling of AppArmor by @twz123 in #3288
• Update to Alpine 3.18.2 by @twz123 in #3287
• Bump click from 8.1.4 to 8.1.5 in /docs by @dependabot in #3290
• Bump pymdown-extensions from 10.0.1 to 10.1 in /docs by @dependabot in #3291
• Bump helm.sh/helm/v3 from 3.12.1 to 3.12.2 by @dependabot in #3292
• Bump github.com/go-openapi/jsonpointer from 0.19.6 to 0.20.0 by @dependabot in #3294
• Exclude cluster config status from config create by @ncopa in #3262
• Add bird-live/ready for live/readinessProbe in bird mode by @ncopa in #3285
• Bump pip from 23.1.2 to 23.2 in /docs by @dependabot in #3297
• Bump pyyaml from 6.0 to 6.0.1 in /docs by @dependabot in #3298
• Disable VXLAN checksum offloading by default by @juanluisvaladas in #3296
• Bump go to 1.20.6 by @ncopa in #3293
• Bump click from 8.1.5 to 8.1.6 in /docs by @dependabot in #3301
• Bump runc to 1.1.8 by @ncopa in #3302
• Kubernetes 1.27.4 by @ncopa in #3305
• Bump pip from 23.2 to 23.2.1 in /docs by @dependabot in #3319
• Bump mkdocs-material from 9.1.18 to 9.1.19 in /docs by @dependabot in #3299
• Bump wheel from 0.40.0 to 0.41.0 in /docs by @dependabot in #3320
• Bump certifi from 2023.5.7 to 2023.7.22 in /docs by @dependabot in #3321
• Bump mkdocs from 1.4.3 to 1.5.1 in /docs by @dependabot in #3331
• Bump mkdocs-material from 9.1.19 to 9.1.21 in /docs by @dependabot in #3332
• Bump google.golang.org/grpc from 1.56.2 to 1.57.0 by @dependabot in #3328
• helm-charts.md: add example helm repo with auth by @shoce in #3325
• Add etcd and kine metrics collection with --enable-metrics-scraper by @jnummelin in #3322
• Bump pyparsing from 3.1.0 to 3.1.1 in /docs by @dependabot in #3335
• Bump github.com/k0sproject/k0sctl from 0.15.2 to 0.15.3 in /hack/tool by @dependabot in #3336
• Bump golang.org/x/tools from 0.11.0 to 0.11.1 by @dependabot in #3334
• Bump go to 1.20.7 (CVE-2023-29409) by @ncopa in #3338
• Bump go.uber.org/zap from 1.24.0 to 1.25.0 by @dependabot in #3339
• Bump containerd 1.7.3 by @ncopa in #3344
• Bump kine to 0.10.2 by @ncopa in #3343
• Bump mkdocs from 1.5.1 to 1.5.2 in /docs by @dependabot in #3349
• Calico 3.26 by @ncopa in #3286
• Prune docker data on arm builders by @ncopa in #3350
• Bump github.com/k0sproject/k0sctl from 0.15.3 to 0.15.4 in /hack/tool by @dependabot in #3346
• Bump github.com/go-playground/validator/v10 from 10.14.1 to 10.15.0 by @dependabot in #3355
• Bump pygments from 2.15.1 to 2.16.1 in /docs by @dependabot in #3353
• Bump wheel from 0.41.0 to 0.41.1 in /docs by @dependabot in #3354
• Bump golang.org/x/crypto from 0.11.0 to 0.12.0 by @dependabot in #3357
• Bump golang.org/x/tools from 0.11.1 to 0.12.0 by @dependabot in #3359
• Bump shogo82148/actions-upload-release-asset from 1.6.5 to 1.6.6 by @dependabot in #3352
• Bump mkdocs-macros-plugin from 1.0.2 to 1.0.4 in /docs by @dependabot in #3361
• Bump regex from 2023.6.3 to 2023.8.8 in /docs by @dependabot in #3363
• Introduce Terraform module for OS testing by @twz123 in #3249
• Bump shogo82148/actions-create-release from 1.4.2 to 1.4.3 by @dependabot in #3360
• Fix workerprofile data types on the generated CRD by @jnummelin in #3369
• Bump runc to 1.1.9 by @ncopa in #3371
• added debian_12 for ostests by @walf443 in #3375
• Bump helm.sh/helm/v3 from 3.12.2 to 3.12.3 by @dependabot in #3373
• Typo fix for external-runtime-deps.md by @kke in #3379
• Bump zeebe-io/backport-action from 1.3.1 to 1.4.0 by @dependabot in #3383
• Bump github.com/k0sproject/k0sctl from 0.15.4 to 0.15.5 in /hack/tool by @dependabot in #3387
• Bump metrics-server to v0.6.4 by @twz123 in #3395
• Bump click from 8.1.6 to 8.1.7 in /docs by @dependabot in #3399
• Bump github.com/go-playground/validator/v10 from 10.15.0 to 10.15.1 by @dependabot in #3398
• Bump controller runtime to v0.15 by @twz123 in #3393
• Bump controller-tools to v0.13.0 by @twz123 in #3396
• Bump wheel from 0.41.1 to 0.41.2 in /docs by @dependabot in #3405
• Bump mkdocs-material from 9.1.21 to 9.2.3 in /docs by @dependabot in #3411
• Add sysprobe and some docs for required mount/umount by @jnummelin in #3409
• Add NodePortAddresses to kubeProxy configuration by @juanluisvaladas in #3407
• Distinguish if feature gate false because it is set as false or by default by @mikhail-sakhnov in #3380
• Add GitHub workflows for OS testing by @twz123 in #3388
• Use image pinning via labels to prevent GC removing system images. by @jnummelin in #3408
• Bump containerd to 1.7.4 by @jnummelin in #3420
• Build k0s Docker image on GitHub managed runners by @twz123 in #3419
• Actually merge CRI configs from drop-ins instead of concatenating them by @jnummelin in #3412
• Bump containerd 1.7.5 by @ncopa in #3427
• Bump mkdocs-material from 9.2.3 to 9.2.5 in /docs by @dependabot in #3431
• Fix hanging start-stop-daemon in footloose Alpine by @twz123 in #3436
• Bump pymdown-extensions from 10.1 to 10.2 in /docs by @dependabot in #3440
• Bump github.com/go-playground/validator/v10 from 10.15.1 to 10.15.2 by @dependabot in #3437
• Bump Alpine to v3.18.3 by @twz123 in #3439
• Bump github.com/go-playground/validator/v10 from 10.15.2 to 10.15.3 by @dependabot in #3442
• Bump pymdown-extensions from 10.2 to 10.2.1 in /docs by @dependabot in #3446
• Bump mkdocs-material from 9.2.5 to 9.2.6 in /docs by @dependabot in #3447
• Bump k0sctl to v0.15.5 by @twz123 in #3432
• Bump golang.org/x/crypto from 0.12.0 to 0.13.0 by @dependabot in #3458
• Bump Go to v1.20.8 by @twz123 in #3459
• Use Docker iid files consequently in builds by @twz123 in #3241
• Bump github.com/hashicorp/terraform-exec from 0.18.1 to 0.19.0 by @dependabot in #3453
• Drop tunneled networking mode support by @mikhail-sakhnov in #3441
• Bump github.com/hashicorp/terraform-exec from 0.18.1 to 0.19.0 in /hack/tool by @dependabot in #3465
• Bump github.com/cyphar/filepath-securejoin from 0.2.3 to 0.2.4 by @dependabot in #3460
• Bump k0sproject/version to v0.4.1 by @twz123 in #3462
• Bump google.golang.org/grpc from 1.57.0 to 1.58.0 by @dependabot in #3463
• Bump golang.org/x/tools from 0.12.0 to 0.13.0 by @dependabot in #3464
• Bump pymdown-extensions from 10.2.1 to 10.3 in /docs by @dependabot in #3451
• Lower disk space eviction limits for network conformance inttest by @twz123 in #3470
• Windows containerd compilation by @mikhail-sakhnov in #3174
• Bump markdown from 3.3.7 to 3.4.4 in /docs by @dependabot in #3326
• Add sysinfo probe for name resolution of localhost by @twz123 in #3370
• Bump mkdocs-material from 9.2.6 to 9.2.8 in /docs by @dependabot in #3457
• Bump Kubernetes to v1.28.1 by @twz123 in #3281
• Bump mkdocs-material from 9.2.8 to 9.3.1 in /docs by @dependabot in #3475
• Bump shogo82148/actions-create-release from 1.4.3 to 1.4.4 by @dependabot in #3474
• Remove cid file before invoking docker build by @twz123 in #3469
• Windows calico host containers by @mikhail-sakhnov in #3209
• Bump github.com/go-playground/validator/v10 from 10.15.3 to 10.15.4 by @dependabot in #3477
• Bump kine to 0.10.3 by @ncopa in #3481
• Bump containerd to v1.7.6 by @twz123 in #3485
• Bump sigs.k8s.io/controller-runtime from 0.15.0 to 0.16.2 by @dependabot in #3479
• Capture target variable in loop body by @twz123 in #3483
• Bump iptables to v1.8.9 by @twz123 in #2680
• Remove unused error channel by @twz123 in #3491
• Fixes for kernel >= 6.0 by @twz123 in #3510
• Bump mkdocs from 1.5.2 to 1.5.3 in /docs by @dependabot in #3507
• Bump go.uber.org/zap from 1.25.0 to 1.26.0 by @dependabot in #3497
• Bump google.golang.org/grpc from 1.58.0 to 1.58.1 by @dependabot in #3496
• Fix log collection in cli inttest by @twz123 in #3495
• Bump github.com/opencontainers/image-spec from 1.1.0-rc4 to 1.1.0-rc5 by @dependabot in #3498
• kube-router 1.6.0 by @jnummelin in #3509
• Bump github.com/Microsoft/hcsshim from 0.11.0 to 0.11.1 by @dependabot in #3514
• Initialize global controller-runtime logger by @twz123 in #3499
• Bump mkdocs-material-extensions from 1.1.1 to 1.2 in /docs by @dependabot in #3516
• Add new dimension for kube-proxy mode to OS testing matrix by @twz123 in #3501
• SBOM generation by @makhov in #3482
• Add missing default for kube-proxy-mode to OS testing workflow by @twz123 in #3517
• Also remove required for kube-proxy-mode from OS testing workflow by @twz123 in #3519
• Bump google.golang.org/grpc from 1.58.1 to 1.58.2 by @dependabot in #3521
• Generate release asset sha256 checksums.txt by @kke in #3392
• Bump CoreDNS to v1.11.1 by @twz123 in #3525
• Fix wrong controller port used for pre-generated tokens command by @fangpenlin in #3524
• Standalone autopilot update logic by @jnummelin in #3406
• Change grab user-agent to mitigate GH blocking it by @jnummelin in #3527
• Bump mkdocs-material from 9.3.1 to 9.3.2 in /docs by @dependabot in #3506
• Bump Kubernetes to v1.28.2 by @twz123 in #3528
• Fix checksums job in release workflow by @twz123 in #3529
• Bump github.com/k0sproject/version from 0.3.1-0.20220411075111-0270bb85e7f8 to 0.4.2 by @dependabot in #3526
• Add GitHub token to checksums step in release workflow by @twz123 in #3532
• Bump mkdocs-material from 9.3.2 to 9.4.2 in /docs by @dependabot in #3530
• Update kubernetes pause image to v3.9 by @twz123 in #3533
• Use release ID in checksums step of release pipeline by @twz123 in #3534

New Contributors

• @Friz-zy made their first contribution in #3245
• @kengz made their first contribution in #3254
• @shoce made their first contribution in #3325
• @walf443 made their first contribution in #3375
• @fangpenlin made their first contribution in #3524

Full Changelog: v1.27.6+k0s.0...v1.28.2+k0s.0