feat: finer grained wait proposal

hips/hip-0999.md

@@ -0,0 +1,102 @@
+---
+hip: "0999"
+title: "Finer grained --wait ressources healthchecks"
+authors: [ "Jimmy Toubert <[email protected]>" ]
+created: "2023-03-03"
+type: "feature"
+status: "draft"
+---
+
+## Abstract
+
+Currently the --wait arguments will wait for specific ressources, handled by helm, and there is now way of declaratively check each ressource status when releasing a chart, it is really unconvenient for crds, and currently needs to be implemented in specific jobs, or in specific tools features like in argocd or flux, where you can declare what an healthy crd should be.
+
+The first proposal is to have a declarative yet simple way to handle ressources states depending on whatever is in the final ressource when the --wait argument is specified, it can be done through annotations, like the terraform kubernetes provider, and like the already existing helm hooks system.
+
+The second proposal is to have add a `helm.sh/weight` annotation (not a hooked one), to be able to define the ressources deployment order, combined with the first proposal, we would be able to make a proxy "depends on this other ressource" feature, since you can be sure that the previous ressources are healthy, if the next ressource depends on the current ressource, then it would be handled.
+
+## Motivation
+
+The primary motivation is to have a helm native way to handle custom ressources and track if it's really healthy or not, as declared in the following issue: https://github.com/helm/helm/issues/11848.
+
+To note a couple of them:
+- An helm --wait flag should wait for the release to really be healthy, even with crds, which means I can't be sure with only helm that my release was really running well.
+- The helm engine should have a generic way to handle the healthyness of a ressource, which can be used to handle default kubernetes ressources as well.
+- Users should have the same result when they use the `helm install` command as well as using tools like argocd to deploy their charts.
+- Having a `depends on` like feature in the helm engine, without only hooks to handle the order in which each ressource should be handled.
+
+## Rationale
+
+Annotations handling are already there, and there's already a way to handle resource statuses in the helm engine.
+The terraform kubernetes provider already have a feature to wait for ressource healthiness: https://www.hashicorp.com/blog/wait-conditions-in-the-kubernetes-provider-for-hashicorp-terraform
+
+## Specification
+
+From the user's perspective, their old helm charts whould not be impacted, if they want to add checks on their ressources they can do it, and provide a mode mature/idempotent helm chart.
+
+The current idea for this implementation is to add specific helm annotations, following the json patch spec, it can be any syntax spec if it cover the following specs:
+```yaml
+# first ressource spec
+annotations:
+    helm.sh/health/spec/someattribute: "==meh"                      # if spec.someattribute equal "meh"
+    helm.sh/health/status/somelist/-/: "==OK"                       # if any (or all?) status.somelist[*] values are equal to "OK"
+    helm.sh/health/status/somelist/each/: "==OK"                    # if all status.somelist[*] values are equal to "OK"
+    helm.sh/health/status/somelist/any/: "==OK"                     # if any status.somelist[*] values are equal to "OK"
+    helm.sh/health/status/somelist/-/someattribute: "==meh"         # if any (or all?) status.somelist.*.someattribute are equal to "meh"
+    helm.sh/health/status/somelist/-/someattribute: "<=1"           # if any status.somelist.*.someattribute are inferior or equal to 1
+    helm.sh/health/status/somelist/each/someattribute: "<1"         # if all status.somelist.*.someattribute are inferior to 1
+    helm.sh/health/status/somelist/any/someattribute: "<1"          # if all status.somelist.*.someattribute are inferior to 1
+    helm.sh/weight: 1                                   # will sync in the wave '1' of the deployment
+    ...
+```
+
+```yaml
+# second ressource spec
+annotations:
+    helm.sh/health/spec/someattribute: "==meh"                      # if spec.someattribute equal "meh"
+    helm.sh/health/status/somelist/-/: "==OK"                       # if any (or all?) status.somelist[*] values are equal to "OK"
+    helm.sh/health/status/somelist/each/: "==OK"                    # if all status.somelist[*] values are equal to "OK"
+    helm.sh/health/status/somelist/any/: "==OK"                     # if any status.somelist[*] values are equal to "OK"
+    helm.sh/health/status/somelist/-/someattribute: "==meh"         # if any (or all?) status.somelist.*.someattribute are equal to "meh"
+    helm.sh/health/status/somelist/-/someattribute: "<=1"           # if any status.somelist.*.someattribute are inferior or equal to 1
+    helm.sh/health/status/somelist/each/someattribute: "<1"         # if all status.somelist.*.someattribute are inferior to 1
+    helm.sh/health/status/somelist/any/someattribute: "<1"          # if all status.somelist.*.someattribute are inferior to 1
+    helm.sh/weight: 2                                   # will sync in the wave '2' of the deployment, after the first ressource if the first ressource is healthy
+    ...
+```
+
+This way, it would be compatible with the kubernetes annotations:
+- annotations specs: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
+- json patch specs: https://jsonpatch.com/
+
+I would prefer to have something nicer like the hcl syntax, but the kubernetes annotations specs do not permit special characters like '*' for splat operators things.
+
+## Backwards compatibility
+
+No impact since it's new helm annotations used for this.
+
+## Security implications
+
+No security implications here.
+
+## How to teach this
+
+The same way as the helm hooks.
+
+## Reference implementation
+
+None for now.
+
+## Rejected ideas
+
+None for now.
+
+## Open issues
+
+- https://github.com/helm/helm/issues/11848
+
+## References
+
+- kubernetes annotations specs: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
+- json patch specs: https://jsonpatch.com/
+- terraform kubernetes provider wait for fields example: https://www.hashicorp.com/blog/wait-conditions-in-the-kubernetes-provider-for-hashicorp-terraform