operator: generate webhook certs without cert-manager, enable webhook…

… by default (#1272)

* operator: generate webhook certs without cert-manager, remove empty lines from artifacthub changes

* updated argocd notes

* added changelog

charts/victoria-metrics-k8s-stack/README.md

@@ -60,6 +60,39 @@ vmagent:
       - url: "https://insert.vmcluster.domain.com/insert/0/prometheus/api/v1/write"
 ```
 
+### ArgoCD issues
+
+When deploying K8s stack using ArgoCD without Cert Manager (`.Values.victoria-metrics-operator.admissionWebhooks.certManager.enabled: false`)
+it will rerender operator's webhook certificates on each sync since Helm `lookup` function is not respected by ArgoCD.
+To prevent this please update you K8s stack Application `spec.syncPolicy` and `spec.ignoreDifferences` with a following:
+
+```yaml
+apiVersion: argoproj.io/v1alpha1
+kind: Application
+...
+spec:
+  ...
+  syncPolicy:
+    syncOptions:
+    # https://argo-cd.readthedocs.io/en/stable/user-guide/sync-options/#respect-ignore-difference-configs
+    # argocd must also ignore difference during apply stage
+    # otherwise it ll silently override changes and cause a problem
+    - RespectIgnoreDifferences=true
+  ignoreDifferences:
+    - group: ""
+      kind: Secret
+      name: <fullname>-validation
+      namespace: kube-system
+      jsonPointers:
+        - /data
+    - group: admissionregistration.k8s.io
+      kind: ValidatingWebhookConfiguration
+      name: <fullname>-admission
+      jqPathExpressions:
+      - '.webhooks[]?.clientConfig.caBundle'
+```
+where `<fullname>` is output of `{{ include "vm-operator.fullname" }}` for your setup
+
 ### Rules and dashboards
 
 This chart by default install multiple dashboards and recording rules from [kube-prometheus](https://github.com/prometheus-operator/kube-prometheus)

charts/victoria-metrics-k8s-stack/README.md.gotmpl

@@ -62,6 +62,39 @@ vmagent:
       - url: "https://insert.vmcluster.domain.com/insert/0/prometheus/api/v1/write"
 ```
 
+### ArgoCD issues
+
+When deploying K8s stack using ArgoCD without Cert Manager (`.Values.victoria-metrics-operator.admissionWebhooks.certManager.enabled: false`)
+it will rerender operator's webhook certificates on each sync since Helm `lookup` function is not respected by ArgoCD.
+To prevent this please update you K8s stack Application `spec.syncPolicy` and `spec.ignoreDifferences` with a following:
+
+```yaml
+apiVersion: argoproj.io/v1alpha1
+kind: Application
+...
+spec:
+  ...
+  syncPolicy:
+    syncOptions:
+    # https://argo-cd.readthedocs.io/en/stable/user-guide/sync-options/#respect-ignore-difference-configs
+    # argocd must also ignore difference during apply stage
+    # otherwise it ll silently override changes and cause a problem
+    - RespectIgnoreDifferences=true
+  ignoreDifferences:
+    - group: ""
+      kind: Secret
+      name: <fullname>-validation
+      namespace: kube-system
+      jsonPointers:
+        - /data
+    - group: admissionregistration.k8s.io
+      kind: ValidatingWebhookConfiguration
+      name: <fullname>-admission
+      jqPathExpressions:
+      - '.webhooks[]?.clientConfig.caBundle'
+```
+where `<fullname>` is output of `{{"{{"}} include "vm-operator.fullname" {{"}}"}}` for your setup
+
 ### Rules and dashboards
 
 This chart by default install multiple dashboards and recording rules from [kube-prometheus](https://github.com/prometheus-operator/kube-prometheus)

charts/victoria-metrics-operator/CHANGELOG.md

@@ -8,6 +8,8 @@
 - Use static container names in a pod
 - Updated operator service scrape config
 - Added `.Values.vmstorage.service.ipFamilies` and `.Values.vmstorage.service.ipFamilyPolicy` for service IP family management
+- Enabled webhook by default
+- Generate webhook certificate when Cert Manager is not enabled
 
 ## 0.34.0
 

charts/victoria-metrics-operator/README.md

@@ -10,6 +10,38 @@ Victoria Metrics Operator
 * Install the follow packages: ``git``, ``kubectl``, ``helm``, ``helm-docs``. See this [tutorial](../../REQUIREMENTS.md).
 * PV support on underlying infrastructure.
 
+## ArgoCD issues
+
+When running operator using ArgoCD without Cert Manager (`.Values.admissionWebhooks.certManager.enabled: false`) it will rerender webhook certificates
+on each sync since Helm `lookup` function is not respected by ArgoCD. To prevent this please update you operator Application `spec.syncPolicy` and `spec.ignoreDifferences` with a following:
+
+```yaml
+apiVersion: argoproj.io/v1alpha1
+kind: Application
+...
+spec:
+  ...
+  syncPolicy:
+    syncOptions:
+    # https://argo-cd.readthedocs.io/en/stable/user-guide/sync-options/#respect-ignore-difference-configs
+    # argocd must also ignore difference during apply stage
+    # otherwise it ll silently override changes and cause a problem
+    - RespectIgnoreDifferences=true
+  ignoreDifferences:
+    - group: ""
+      kind: Secret
+      name: <fullname>-validation
+      namespace: kube-system
+      jsonPointers:
+        - /data
+    - group: admissionregistration.k8s.io
+      kind: ValidatingWebhookConfiguration
+      name: <fullname>-admission
+      jqPathExpressions:
+      - '.webhooks[]?.clientConfig.caBundle'
+```
+where `<fullname>` is output of `{{ include "vm-operator.fullname" }}` for your setup
+
 ## Upgrade guide
 
  During release an issue with helm CRD was discovered. So for upgrade from version less then 0.1.3 you have to two options:
@@ -139,11 +171,11 @@ Change the values according to the need of the environment in ``victoria-metrics
 
 | Key | Type | Default | Description |
 |-----|------|---------|-------------|
-| admissionWebhooks | object | `{"caBundle":"","certManager":{"enabled":false,"issuer":{}},"enabled":false,"enabledCRDValidation":{"vmagent":true,"vmalert":true,"vmalertmanager":true,"vmalertmanagerConfig":true,"vmauth":true,"vmcluster":true,"vmrule":true,"vmsingle":true,"vmuser":true},"policy":"Fail"}` | Configures resource validation |
-| admissionWebhooks.caBundle | string | `""` | with keys: tls.key, tls.crt, ca.crt |
+| admissionWebhooks | object | `{"certManager":{"enabled":false,"issuer":{}},"enabled":true,"enabledCRDValidation":{"vlogs":true,"vmagent":true,"vmalert":true,"vmalertmanager":true,"vmalertmanagerconfig":true,"vmauth":true,"vmcluster":true,"vmrule":true,"vmsingle":true,"vmuser":true},"keepTLSSecret":false,"policy":"Fail","tls":{"caCert":null,"cert":null,"key":null}}` | Configures resource validation |
+| admissionWebhooks.certManager | object | `{"enabled":false,"issuer":{}}` | with keys: tls.key, tls.crt, ca.crt |
 | admissionWebhooks.certManager.enabled | bool | `false` | Enables cert creation and injection by cert-manager. |
 | admissionWebhooks.certManager.issuer | object | `{}` | If needed, provide own issuer. Operator will create self-signed if empty. |
-| admissionWebhooks.enabled | bool | `false` | Enables validation webhook. |
+| admissionWebhooks.enabled | bool | `true` | Enables validation webhook. |
 | admissionWebhooks.policy | string | `"Fail"` | What to do in case, when operator not available to validate request. |
 | affinity | object | `{}` | Pod affinity |
 | annotations | object | `{}` | Annotations to be added to the all resources |
@@ -161,6 +193,7 @@ Change the values according to the need of the environment in ``victoria-metrics
 | extraVolumeMounts | list | `[]` | Extra Volume Mounts for the container |
 | extraVolumes | list | `[]` | Extra Volumes for the pod |
 | fullnameOverride | string | `""` | Overrides the full name of server component |
+| global.cluster.dnsDomain | string | `"cluster.local"` |  |
 | global.image.registry | string | `""` |  |
 | global.imagePullSecrets | list | `[]` |  |
 | image.pullPolicy | string | `"IfNotPresent"` | Image pull policy |
@@ -215,4 +248,4 @@ Change the values according to the need of the environment in ``victoria-metrics
 | serviceMonitor | object | `{"annotations":{},"basicAuth":{},"enabled":false,"extraLabels":{},"interval":"","relabelings":[],"scheme":"","scrapeTimeout":"","tlsConfig":{}}` | configures monitoring with serviceScrape. VMServiceScrape must be pre-installed |
 | tolerations | list | `[]` | Array of tolerations object. Ref: [https://kubernetes.io/docs/concepts/configuration/assign-pod-node/](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/) |
 | topologySpreadConstraints | list | `[]` | Pod Topology Spread Constraints. Ref: [https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/](https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/) |
-| watchNamespace | string | `""` |  |
+| watchNamespace | string | `""` |  |

charts/victoria-metrics-operator/README.md.gotmpl

@@ -10,6 +10,38 @@
 * Install the follow packages: ``git``, ``kubectl``, ``helm``, ``helm-docs``. See this [tutorial](../../REQUIREMENTS.md).
 * PV support on underlying infrastructure.
 
+## ArgoCD issues
+
+When running operator using ArgoCD without Cert Manager (`.Values.admissionWebhooks.certManager.enabled: false`) it will rerender webhook certificates
+on each sync since Helm `lookup` function is not respected by ArgoCD. To prevent this please update you operator Application `spec.syncPolicy` and `spec.ignoreDifferences` with a following:
+
+```yaml
+apiVersion: argoproj.io/v1alpha1
+kind: Application
+...
+spec:
+  ...
+  syncPolicy:
+    syncOptions:
+    # https://argo-cd.readthedocs.io/en/stable/user-guide/sync-options/#respect-ignore-difference-configs
+    # argocd must also ignore difference during apply stage
+    # otherwise it ll silently override changes and cause a problem
+    - RespectIgnoreDifferences=true
+  ignoreDifferences:
+    - group: ""
+      kind: Secret
+      name: <fullname>-validation
+      namespace: kube-system
+      jsonPointers:
+        - /data
+    - group: admissionregistration.k8s.io
+      kind: ValidatingWebhookConfiguration
+      name: <fullname>-admission
+      jqPathExpressions:
+      - '.webhooks[]?.clientConfig.caBundle'
+```
+where `<fullname>` is output of `{{"{{"}} include "vm-operator.fullname" {{"}}"}}` for your setup
+
 ## Upgrade guide
 
  During release an issue with helm CRD was discovered. So for upgrade from version less then 0.1.3 you have to two options:
@@ -137,4 +169,4 @@ The following tables lists the configurable parameters of the chart and their de
 
 Change the values according to the need of the environment in ``victoria-metrics-operator/values.yaml`` file.
 
-{{ template "chart.valuesTable" . }}
+{{ template "chart.valuesTable" . }}

charts/victoria-metrics-operator/templates/_helpers.tpl

@@ -62,9 +62,49 @@ helm.sh/chart: {{ include "vm-operator.chart" . }}
 app.kubernetes.io/managed-by: {{ .Release.Service | trunc 63 | trimSuffix "-" }}
 {{- end -}}
 
+{{/*
+Create unified annotations for vm-operator components
+*/}}
+{{- define "vm-operator.annotations" -}}
+helm.sh/resource-policy: keep
+meta.helm.sh/release-namespace: {{ .Release.Namespace }}
+meta.helm.sh/release-name: {{ .Release.Name }}
+{{- end -}}
+
 {{/*
 Create the name of service account and clusterRole for cleanup-hook
 */}}
 {{- define "vm-operator.cleanupHookName" -}}
 {{- printf "%s-%s" (include "vm-operator.fullname" .) "cleanup-hook" | replace "+" "_" | trunc 63 | trimSuffix "-" }}
 {{- end }}
+
+{{/*
+Generate certificates for webhook
+*/}}
+{{- define "vm-operator.certs" -}}
+{{- $webhook := .Values.admissionWebhooks -}}
+{{- $tls := $webhook.tls -}}
+{{- $serviceName := (include "vm-operator.fullname" .) -}}
+{{- $secretName := (printf "%s-validation" $serviceName) -}}
+{{- $secret := lookup "v1" "Secret" .Release.Namespace $secretName -}}
+{{- if (and $tls.caCert $tls.cert $tls.key) -}}
+caCert: {{ $tls.caCert | b64enc }}
+clientCert: {{ $tls.cert | b64enc }}
+clientKey: {{ $tls.key | b64enc }}
+{{- else if and $webhook.keepTLSSecret $secret -}}
+caCert: {{ index $secret.data "ca.crt" }}
+clientCert: {{ index $secret.data "tls.crt" }}
+clientKey: {{ index $secret.data "tls.key" }}
+{{- else -}}
+{{- $altNames := default list -}}
+{{- $namePrefix := (printf "%s.%s" $serviceName .Release.Namespace) -}}
+{{- $altNames = append $altNames $namePrefix -}}
+{{- $altNames = append $altNames (printf "%s.svc" $namePrefix) -}}
+{{- $altNames = append $altNames (printf "%s.svc.%s" $namePrefix .Values.global.cluster.dnsDomain) -}}
+{{- $ca := genCA "vm-operator-ca" 3650 -}}
+{{- $cert := genSignedCert $serviceName nil $altNames 3650 $ca -}}
+caCert: {{ $ca.Cert | b64enc }}
+clientCert: {{ $cert.Cert | b64enc }}
+clientKey: {{ $cert.Key | b64enc }}
+{{- end -}}
+{{- end -}}

charts/victoria-metrics-operator/templates/crd.yaml

@@ -3,13 +3,13 @@
 {{- /* see this pull request https://github.com/VictoriaMetrics/helm-charts/pull/771 for details */ -}}
 {{- if .Values.createCRD }}
 {{- $files := .Files }}
-{{- $fileContentsList := $files.Get "crd.yaml" | splitList "---" }}
-{{- range $fileContentsList }}
-    {{- $fileContents := . | fromYaml }}
-    {{- $newAnnotations := dict "helm.sh/resource-policy" "keep" | merge $fileContents.metadata.annotations }}
-    {{- $newMetadata := dict "annotations" $newAnnotations | merge $fileContents.metadata }}
-    {{- $newFileContents := dict "metadata" $newMetadata | merge $fileContents }}
-    {{- $newFileContents | toYaml }}
+{{- $crds := $files.Get "crd.yaml" | splitList "---" }}
+{{- $labels := (include "vm-operator.labels" .) | fromYaml -}}
+{{- $annotations := (include "vm-operator.annotations" .) | fromYaml -}}
+{{- $extra := dict "metadata" (dict "annotations" $annotations "labels" $labels) -}}
+{{- range $crds }}
+    {{- $crd := . | fromYaml }}
+    {{- toYaml (merge $crd $extra) }}
     {{- print "\n---\n" }}
 {{- end }}
-{{- end }}
+{{- end }}

charts/victoria-metrics-operator/templates/service.yaml

@@ -38,9 +38,9 @@ spec:
   ports:
     - name: http
       port: 8080
-      targetPort: 8080
+      targetPort: http
       protocol: TCP
     - name: webhook
       port: 443
-      targetPort: 9443
+      targetPort: webhook
   selector: {{ include "vm-operator.selectorLabels" . | nindent 4 }}