[BUG] Update SidecarSet(Not just the image field) will not trigger sidecar version update.

[jzeng4]

What happened:
We have a SidecarSet deployed in Minikube.

• Changed SidecarSet.spec.containers[x].resources.limits.cpu.
• Upgrade the SidecarSet.spec.containers[x].image to a new version.

What you expected to happen:
The sidecar container version should be updated to the new version without restarting the pod.

How to reproduce it (as minimally and precisely as possible):

• Install openKruise in minikube: https://openkruise.io/docs/installation/
• kubectl apply -f sidecarset.yaml

sidecarset.yaml
apiVersion: apps.kruise.io/v1alpha1
kind: SidecarSet
metadata:
  name: test-sidecarset3
spec:
  selector:
    matchLabels:
      sidecar-envoy: envoy
  updateStrategy:
    type: RollingUpdate
    maxUnavailable: 1
  containers:
  - name: sidecar12
    image: centos:6.7
    resources:
      requests:
        memory: "100Mi"
        cpu: "250m"
      limits:
        memory: "128Mi"
        cpu: "1000m"
    command: ["sleep", "998d"] # do nothing at all
    volumeMounts:
    - name: log-volume
      mountPath: /var/log
  volumes: # this field will be merged into pod.spec.volumes
  - name: log-volume
    emptyDir: {}

• kubectl apply -f cloneset.yaml

apiVersion: apps.kruise.io/v1alpha1
kind: CloneSet
metadata:
  labels:
    app: sample
  name: sample
spec:
  replicas: 2
  selector:
    matchLabels:
      app: sample
  template:
    metadata:
      labels:
        app: sample
        label1: v1
        label2: v2
    spec:
      containers:
      - name: nginx
        image: nginx:alpine
        resources:
          limits:
            cpu: 251m
            memory: 100Mi
          requests:
            cpu: 250m
            memory: 100Mi

• Changed SidecarSet.spec.containers[x].resources.limits.cpu.
• Upgrade the SidecarSet.spec.containers[x].image to a new version.

Anything else we need to know?:
I did found the "kruise-daemon" is failed.

kubectl get pod -n kruise-system
NAME                                         READY   STATUS             RESTARTS         AGE
kruise-controller-manager-5bcc8987fd-bskn6   1/1     Running            1 (7d7h ago)     8d
kruise-controller-manager-5bcc8987fd-tttdr   1/1     Running            2 (4d9h ago)     8d
kruise-daemon-w2pwr                          0/1     CrashLoopBackOff   1204 (98s ago)   8d

Environment:

• Kruise version: 1.4.0
• Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.4", GitCommit:"872a965c6c6526caa949f0c6ac028ef7aff3fb78", GitTreeState:"clean", BuildDate:"2022-11-09T13:36:36Z", GoVersion:"go1.19.3", Compiler:"gc", Platform:"darwin/amd64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"26", GitVersion:"v1.26.1", GitCommit:"8f94681cd294aa8cfd3407b8191f6c70214973a4", GitTreeState:"clean", BuildDate:"2023-01-18T15:51:25Z", GoVersion:"go1.19.5", Compiler:"gc", Platform:"linux/amd64"}

• Install details (e.g. helm install args): follow https://openkruise.io/docs/installation/
• Others:

[zmberg]

@jzeng4 Current sidecarSet only support in-place update pod with 'Image' filed. Because you change the resources filed, so SidecarSet can't in-place update sidecar container.

[shuyingliang]

@zmberg true that SidecarSet support in-place update. but can you help us understand

1. why does the kruise-daemonset fail on this?
2. when users update the resource in the sidecar container, if that is not supported, should there be some API to control if the the pod will be recreated in fallback or just reject the request if in-place update is enforced? cc @FillZpp

[jzeng4]

@zmberg

Besides, I saw that the CPU/Memory was changed in SidecarSet objects and I don't expect SidecarSets to restart the pods. But how users know they should do a non InplaceUpdate to make the resource changes? Without the notifications, users may still keep bumping image versions. But those bumpings will not succeed.

[zmberg]

@zmberg

Besides, I saw that the CPU/Memory was changed in SidecarSet objects and I don't expect SidecarSets to restart the pods. But how users know they should do a non InplaceUpdate to make the resource changes? Without the notifications, users may still keep bumping image versions. But those bumpings will not succeed.

Yes, we will add status to show the why Pods can't be inplace update.

[MarkLux]

@zmberg can you assign this to me?

[zmberg]

/assign @MarkLux

[MarkLux]

@zmberg made a pull request for this, maybe you can give a review on it. #1309