Clean up "object has already been modified" warnings #700

blampe · 2024-10-01T17:29:58Z

No description provided.

cleverguy25 · 2024-10-01T17:30:00Z

Added to epic #586

### Proposed changes  This PR optimizes the watches to reduce the likelihood of a race between the cache and the reconciliation loop. The general idea is to use predicates to filter on the 'edge' events that would allow the reconciler to make forward progress. For example, the workspace reconciler applies a StatefulSet, then waits for it to be ready. Only when the StatefulSet is indeed ready should reconciliation be queued. Before this PR, the watch on StatefulSet was triggering upon any revision change (way too eager). Overall, a number of optimizations were implemented: - workspace controller should wait for the statefulset to be ready - workspace controller need not watch the Service - stack controller should wait for flux source to be ready - stack controller should not trigger reconciliation upon adding its finalizer (which increments the generation). - stack controller should wait for the update to complete - update controller should wait for workspace readiness - update controller should not enqueue the completed updates of a given workspace Also, some debug statements were introduced to reason about the revision changes over time, since the "conflicts" are detected based on revision. In particular, we emit a debug statement when reconciliation is triggered by a given watch, to understand 'why'. ### Related issues (optional)  Closes #700 Supercedes #679 ### Example Here's the result of applying [operator/examples/random-yaml/stack.yaml](https://github.com/pulumi/pulumi-kubernetes-operator/blob/78836991b2691034b2c029df4040ea2016240cd8/operator/examples/random-yaml/stack.yaml). #### Before ```log controller-manager-6f849dcb6b-4wscw manager 2024-10-16T01:47:26.032Z ERROR updating status {"controller": "workspace", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "841b4d5b-029c-4457-a2b7-8c1821475adf", "error": "Operation cannot be fulfilled on workspaces.auto.pulumi.com \"random-yaml\": the object has been modified; please apply your changes to the latest version and try again"} controller-manager-6f849dcb6b-4wscw manager github.com/pulumi/pulumi-kubernetes-operator/v2/operator/internal/controller/auto.(*WorkspaceReconciler).Reconcile.func1 controller-manager-6f849dcb6b-4wscw manager /go/operator/internal/controller/auto/workspace_controller.go:110 controller-manager-6f849dcb6b-4wscw manager github.com/pulumi/pulumi-kubernetes-operator/v2/operator/internal/controller/auto.(*WorkspaceReconciler).Reconcile controller-manager-6f849dcb6b-4wscw manager /go/operator/internal/controller/auto/workspace_controller.go:181 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222 controller-manager-6f849dcb6b-4wscw manager 2024-10-16T01:47:26.033Z ERROR Reconciler error {"controller": "workspace", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "841b4d5b-029c-4457-a2b7-8c1821475adf", "error": "Operation cannot be fulfilled on workspaces.auto.pulumi.com \"random-yaml\": the object has been modified; please apply your changes to the latest version and try again"} controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:324 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222 controller-manager-6f849dcb6b-4wscw manager 2024-10-16T01:47:26.047Z ERROR unable to save object status {"controller": "stack-controller", "namespace": "default", "name": "random-yaml", "reconcileID": "6a99e260-ca2d-4376-b95f-ab490acbe32a", "error": "Operation cannot be fulfilled on stacks.pulumi.com \"random-yaml\": the object has been modified; please apply your changes to the latest version and try again"} controller-manager-6f849dcb6b-4wscw manager github.com/pulumi/pulumi-kubernetes-operator/v2/operator/internal/controller/pulumi.(*StackReconciler).Reconcile.func1 controller-manager-6f849dcb6b-4wscw manager /go/operator/internal/controller/pulumi/stack_controller.go:445 controller-manager-6f849dcb6b-4wscw manager github.com/pulumi/pulumi-kubernetes-operator/v2/operator/internal/controller/pulumi.(*StackReconciler).Reconcile controller-manager-6f849dcb6b-4wscw manager /go/operator/internal/controller/pulumi/stack_controller.go:741 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222 controller-manager-6f849dcb6b-4wscw manager 2024-10-16T01:47:26.047Z ERROR Reconciler error {"controller": "stack-controller", "namespace": "default", "name": "random-yaml", "reconcileID": "6a99e260-ca2d-4376-b95f-ab490acbe32a", "error": "Operation cannot be fulfilled on stacks.pulumi.com \"random-yaml\": the object has been modified; please apply your changes to the latest version and try again"} controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:324 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222 controller-manager-6f849dcb6b-4wscw manager 2024-10-16T01:47:26.091Z ERROR updating status {"controller": "workspace", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "d397d7d1-c74b-43fe-b4a0-80c3d39ee3ea", "error": "Operation cannot be fulfilled on workspaces.auto.pulumi.com \"random-yaml\": the object has been modified; please apply your changes to the latest version and try again"} controller-manager-6f849dcb6b-4wscw manager github.com/pulumi/pulumi-kubernetes-operator/v2/operator/internal/controller/auto.(*WorkspaceReconciler).Reconcile.func1 controller-manager-6f849dcb6b-4wscw manager /go/operator/internal/controller/auto/workspace_controller.go:110 controller-manager-6f849dcb6b-4wscw manager github.com/pulumi/pulumi-kubernetes-operator/v2/operator/internal/controller/auto.(*WorkspaceReconciler).Reconcile controller-manager-6f849dcb6b-4wscw manager /go/operator/internal/controller/auto/workspace_controller.go:188 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222 controller-manager-6f849dcb6b-4wscw manager 2024-10-16T01:47:26.091Z ERROR Reconciler error {"controller": "workspace", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "d397d7d1-c74b-43fe-b4a0-80c3d39ee3ea", "error": "Operation cannot be fulfilled on workspaces.auto.pulumi.com \"random-yaml\": the object has been modified; please apply your changes to the latest version and try again"} controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:324 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222 controller-manager-6f849dcb6b-4wscw manager 2024-10-16T01:47:26.111Z ERROR updating status {"controller": "workspace", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "b94710c8-8165-4937-a99c-a4e065f63788", "error": "Operation cannot be fulfilled on workspaces.auto.pulumi.com \"random-yaml\": the object has been modified; please apply your changes to the latest version and try again"} controller-manager-6f849dcb6b-4wscw manager github.com/pulumi/pulumi-kubernetes-operator/v2/operator/internal/controller/auto.(*WorkspaceReconciler).Reconcile.func1 controller-manager-6f849dcb6b-4wscw manager /go/operator/internal/controller/auto/workspace_controller.go:110 controller-manager-6f849dcb6b-4wscw manager github.com/pulumi/pulumi-kubernetes-operator/v2/operator/internal/controller/auto.(*WorkspaceReconciler).Reconcile controller-manager-6f849dcb6b-4wscw manager /go/operator/internal/controller/auto/workspace_controller.go:188 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222 controller-manager-6f849dcb6b-4wscw manager 2024-10-16T01:47:26.111Z ERROR Reconciler error {"controller": "workspace", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "b94710c8-8165-4937-a99c-a4e065f63788", "error": "Operation cannot be fulfilled on workspaces.auto.pulumi.com \"random-yaml\": the object has been modified; please apply your changes to the latest version and try again"} controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:324 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222 controller-manager-6f849dcb6b-4wscw manager 2024-10-16T01:47:32.082Z ERROR updating status {"controller": "workspace", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "7cb6660a-508c-4392-80cd-749fe72def9f", "error": "Operation cannot be fulfilled on workspaces.auto.pulumi.com \"random-yaml\": the object has been modified; please apply your changes to the latest version and try again"} controller-manager-6f849dcb6b-4wscw manager github.com/pulumi/pulumi-kubernetes-operator/v2/operator/internal/controller/auto.(*WorkspaceReconciler).Reconcile.func1 controller-manager-6f849dcb6b-4wscw manager /go/operator/internal/controller/auto/workspace_controller.go:110 controller-manager-6f849dcb6b-4wscw manager github.com/pulumi/pulumi-kubernetes-operator/v2/operator/internal/controller/auto.(*WorkspaceReconciler).Reconcile controller-manager-6f849dcb6b-4wscw manager /go/operator/internal/controller/auto/workspace_controller.go:252 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222 controller-manager-6f849dcb6b-4wscw manager 2024-10-16T01:47:32.082Z ERROR Reconciler error {"controller": "workspace", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "7cb6660a-508c-4392-80cd-749fe72def9f", "error": "Operation cannot be fulfilled on workspaces.auto.pulumi.com \"random-yaml\": the object has been modified; please apply your changes to the latest version and try again"} controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:324 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222 controller-manager-6f849dcb6b-4wscw manager 2024-10-16T01:47:36.114Z ERROR updating status {"controller": "workspace", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "480ad23a-2d02-40a6-b4a4-198fb9df01c4", "error": "Operation cannot be fulfilled on workspaces.auto.pulumi.com \"random-yaml\": the object has been modified; please apply your changes to the latest version and try again"} controller-manager-6f849dcb6b-4wscw manager github.com/pulumi/pulumi-kubernetes-operator/v2/operator/internal/controller/auto.(*WorkspaceReconciler).Reconcile.func1 controller-manager-6f849dcb6b-4wscw manager /go/operator/internal/controller/auto/workspace_controller.go:110 controller-manager-6f849dcb6b-4wscw manager github.com/pulumi/pulumi-kubernetes-operator/v2/operator/internal/controller/auto.(*WorkspaceReconciler).Reconcile controller-manager-6f849dcb6b-4wscw manager /go/operator/internal/controller/auto/workspace_controller.go:316 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:114 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:311 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222 controller-manager-6f849dcb6b-4wscw manager 2024-10-16T01:47:36.114Z ERROR Reconciler error {"controller": "workspace", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "480ad23a-2d02-40a6-b4a4-198fb9df01c4", "error": "Operation cannot be fulfilled on workspaces.auto.pulumi.com \"random-yaml\": the object has been modified; please apply your changes to the latest version and try again"} controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:324 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:261 controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 controller-manager-6f849dcb6b-4wscw manager /go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:222 ``` #### After ```log controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:45.212Z INFO Reconciling Stack {"controller": "stack-controller", "namespace": "default", "name": "random-yaml", "reconcileID": "585499fc-fb33-4505-8594-6a9fd8c810b2", "revision": "893372"} controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:45.225Z INFO KubeAPIWarningLogger metadata.finalizers: "finalizer.stack.pulumi.com": prefer a domain-qualified finalizer name to avoid accidental conflicts with other finalizer writers controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:45.225Z INFO installing watcher for newly seen source kind {"GroupVersionKind": "source.toolkit.fluxcd.io/v1, Kind=GitRepository"} controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:45.225Z INFO Starting EventSource {"controller": "stack-controller", "source": "kind source: *unstructured.Unstructured"} controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:45.238Z INFO Reconciling Workspace {"controller": "workspace-controller", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "d1595cfb-0fe6-46d5-9e9c-2227f334eb7f", "revision": "893374"} controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:45.238Z INFO Applying StatefulSet {"controller": "workspace-controller", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "d1595cfb-0fe6-46d5-9e9c-2227f334eb7f", "revision": "893374", "hash": "608b7e170ccd95754972cde1178ab426", "source": {"Generation":1,"ForceRequest":"","Git":null,"Flux":{"Url":"http://source-controller.flux-system.svc.cluster.local./gitrepository/default/pulumi-examples/1432c2ce26299516223c0ad228e92294eba4b2c1.tar.gz","Digest":"sha256:44d554df090dcdeb9bae908cd38155c8c93db05f64dc72982027e8e1294af0d3","Dir":"random-yaml/"}}} controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:48.609Z INFO Reconciling Workspace {"controller": "workspace-controller", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "5e7b9b87-fab5-4da0-b025-65dadefc097e", "revision": "893380"} controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:48.609Z INFO Applying StatefulSet {"controller": "workspace-controller", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "5e7b9b87-fab5-4da0-b025-65dadefc097e", "revision": "893380", "hash": "608b7e170ccd95754972cde1178ab426", "source": {"Generation":1,"ForceRequest":"","Git":null,"Flux":{"Url":"http://source-controller.flux-system.svc.cluster.local./gitrepository/default/pulumi-examples/1432c2ce26299516223c0ad228e92294eba4b2c1.tar.gz","Digest":"sha256:44d554df090dcdeb9bae908cd38155c8c93db05f64dc72982027e8e1294af0d3","Dir":"random-yaml/"}}} controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:48.719Z INFO Connecting to workspace pod {"controller": "workspace-controller", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "5e7b9b87-fab5-4da0-b025-65dadefc097e", "revision": "893380", "addr": "random-yaml-workspace.default.svc.cluster.local:50051"} controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:48.724Z INFO Connected to workspace pod {"controller": "workspace-controller", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "5e7b9b87-fab5-4da0-b025-65dadefc097e", "revision": "893380", "addr": "random-yaml-workspace.default.svc.cluster.local:50051"} controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:48.724Z INFO Running pulumi install {"controller": "workspace-controller", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "5e7b9b87-fab5-4da0-b025-65dadefc097e", "revision": "893380"} controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:51.509Z INFO Creating Pulumi stack(s) {"controller": "workspace-controller", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "5e7b9b87-fab5-4da0-b025-65dadefc097e", "revision": "893413"} controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:55.250Z INFO workspace pod initialized {"controller": "workspace-controller", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "5e7b9b87-fab5-4da0-b025-65dadefc097e", "revision": "893422"} controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:55.250Z INFO Ready {"controller": "workspace-controller", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "5e7b9b87-fab5-4da0-b025-65dadefc097e", "revision": "893422"} controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:55.255Z INFO Reconciling Stack {"controller": "stack-controller", "namespace": "default", "name": "random-yaml", "reconcileID": "910a2faa-5b49-4590-a23a-d9ef8cab0023", "revision": "893375"} controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:55.271Z INFO Reconciling Update {"controller": "update", "controllerGroup": "auto.pulumi.com", "controllerKind": "Update", "Update": {"name":"random-yaml-b7kng2l5","namespace":"default"}, "namespace": "default", "name": "random-yaml-b7kng2l5", "reconcileID": "5b9ced73-989f-46fa-a485-746686519832"} controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:55.271Z INFO Updating the status {"controller": "update", "controllerGroup": "auto.pulumi.com", "controllerKind": "Update", "Update": {"name":"random-yaml-b7kng2l5","namespace":"default"}, "namespace": "default", "name": "random-yaml-b7kng2l5", "reconcileID": "5b9ced73-989f-46fa-a485-746686519832"} controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:55.275Z INFO Connecting {"controller": "update", "controllerGroup": "auto.pulumi.com", "controllerKind": "Update", "Update": {"name":"random-yaml-b7kng2l5","namespace":"default"}, "namespace": "default", "name": "random-yaml-b7kng2l5", "reconcileID": "5b9ced73-989f-46fa-a485-746686519832", "addr": "random-yaml-workspace.default.svc.cluster.local:50051"} controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:55.278Z INFO Selecting the stack {"controller": "update", "controllerGroup": "auto.pulumi.com", "controllerKind": "Update", "Update": {"name":"random-yaml-b7kng2l5","namespace":"default"}, "namespace": "default", "name": "random-yaml-b7kng2l5", "reconcileID": "5b9ced73-989f-46fa-a485-746686519832", "stackName": "dev"} controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:57.333Z INFO Applying the update {"controller": "update", "controllerGroup": "auto.pulumi.com", "controllerKind": "Update", "Update": {"name":"random-yaml-b7kng2l5","namespace":"default"}, "namespace": "default", "name": "random-yaml-b7kng2l5", "reconcileID": "5b9ced73-989f-46fa-a485-746686519832", "type": "up"} controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:57.334Z INFO Executing update operation {"controller": "update", "controllerGroup": "auto.pulumi.com", "controllerKind": "Update", "Update": {"name":"random-yaml-b7kng2l5","namespace":"default"}, "namespace": "default", "name": "random-yaml-b7kng2l5", "reconcileID": "5b9ced73-989f-46fa-a485-746686519832", "request": "message:\"Stack Update (up)\" target_dependents:false refresh:true"} controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:02:05.421Z INFO Update complete {"controller": "update", "controllerGroup": "auto.pulumi.com", "controllerKind": "Update", "Update": {"name":"random-yaml-b7kng2l5","namespace":"default"}, "namespace": "default", "name": "random-yaml-b7kng2l5", "reconcileID": "5b9ced73-989f-46fa-a485-746686519832", "result": "stdout:\"Updating (dev)\\n\\nView Live: https://app.pulumi.com/eron-pulumi-corp/random/dev/updates/4536\\n\\n+ pulumi:pulumi:Stack: (create)\\n [urn=urn:pulumi:dev::random::pulumi:pulumi:Stack::random-dev]\\n + random:index/randomPassword:RandomPassword: (create)\\n [urn=urn:pulumi:dev::random::random:index/randomPassword:RandomPassword::randomPassword]\\n length : 16\\n overrideSpecial: \\\"_%@\\\"\\n special : true\\n --outputs:--\\n bcryptHash : [secret]\\n id : \\\"none\\\"\\n lower : true\\n minLower : 0\\n minNumeric : 0\\n minSpecial : 0\\n minUpper : 0\\n number : true\\n numeric : true\\n result : [secret]\\n upper : true\\n --outputs:--\\n password: [secret]\\nResources:\\n + 2 created\\n\\nDuration: 3s\\n\" summary:{start_time:{seconds:1729044119} end_time:{seconds:1729044122} result:\"succeeded\" message:\"\\\"Stack Update (up)\\\"\"} permalink:\"https://app.pulumi.com/eron-pulumi-corp/random/dev/updates/4536\" outputs:{key:\"password\" value:{value:\"\\\"fAMcmXI2DfW7KCey\\\"\" secret:true}}"} controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:02:05.438Z INFO Reconciling Stack {"controller": "stack-controller", "namespace": "default", "name": "random-yaml", "reconcileID": "a100c4f3-c5b9-4894-8644-fabc1f71d008", "revision": "893437"} controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:02:05.546Z INFO Commit hash unchanged. Will wait for Source update or resync. {"controller": "stack-controller", "namespace": "default", "name": "random-yaml", "reconcileID": "a100c4f3-c5b9-4894-8644-fabc1f71d008", "revision": "893437"} ```

pulumi-bot added the needs-triage Needs attention from the triage team label Oct 1, 2024

cleverguy25 mentioned this issue Oct 1, 2024

[Epic] Kubernetes Operator Core Functionality Enhancements (PKOv2) #586

Open

5 tasks

blampe added this to the 0.111 milestone Oct 1, 2024

blampe removed the needs-triage Needs attention from the triage team label Oct 1, 2024

blampe assigned EronWright Oct 1, 2024

blampe mentioned this issue Oct 1, 2024

[V2] Fix log spamming from cache misses when updating status repeatedly #679

Closed

mikhailshilkov added the kind/enhancement Improvements or new features label Oct 8, 2024

EronWright mentioned this issue Oct 15, 2024

Clean up "object has already been modified" warnings #700 #717

Merged

EronWright added the resolution/fixed This issue was fixed label Oct 17, 2024

EronWright closed this as completed Oct 17, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Clean up "object has already been modified" warnings #700

Clean up "object has already been modified" warnings #700

blampe commented Oct 1, 2024

cleverguy25 commented Oct 1, 2024

Clean up "object has already been modified" warnings #700

Clean up "object has already been modified" warnings #700

Comments

blampe commented Oct 1, 2024

cleverguy25 commented Oct 1, 2024