Helm chart checks are a set of checks against which the Red Hat Helm chart-verifier tool verifies and validates whether a Helm chart is qualified for a certification from Red Hat. These checks contain metadata that have certain parameters and values with which a Helm chart must comply to be certified by Red Hat. A Red Hat-certified Helm chart qualifies in terms of readiness for distribution in the OpenShift Helm Chart Repository.
- Key features
- Types of Helm chart checks
- Default set of checks for a Helm chart
- Run Helm chart checks
- Profiles
- Chart Testing
- Signed Charts
- You can execute all of the mandatory checks.
- Include or exclude individual checks during the verification process by using the command line options.
- Each check is independent, and there is no specific execution order.
- Executing the checks includes running the
helm lint,helm template,helm install, andhelm testcommands against the chart. - The checks are configurable. For example, if a chart requires additional values to be compliant with the checks, configure the values using the available options. The options are similar to those used by the
helm lintandhelm templatecommands. - When there are no error messages, the
helm-lintcheck passes the verification and is successful. Messages such asWarningandinfodo not cause the check to fail. - Profiles define the checks needed based on the chart type: partner, redhat or community.
- Profiles are versioned. Each new version can include updated checks, new checks, new annotations, or changed annotations.
- The generated report, by default, is written to stdout.
- Alternatively the
--write-to-fileflag can be used to write to areport.yamlfile.
- Alternatively the
- An error log is created for all verify commands but can be optionally suppressed.
- You can indicate that a chart is not to be published in the OpenShift catalog.
- From chart verifier version 1.9.0 the generated report includes a sha value based on the report content. This is used during the submission process to verify the integrity of the report.
- You can verify a signed chart. See: Signed Charts.
Helm chart checks are categorized into the following types:
| Check type | Description |
|---|---|
| Mandatory | Checks are required to pass and be successful for certification. |
| Recommended | Checks are about to become mandatory; we recommend fixing any check failures. |
| Optional | Checks are ready for customer testing. Checks can fail and still pass the verification for certification. |
| Experimental | New checks introduced for testing purposes or beta versions. |
NOTE: The current release of the chart-verifier includes only the mandatory and optional type of checks.
The following table lists the set of checks for each profile version with details including the name and version of the check, and a description of the check.
| Porfile v1.2 | Profile v1.1 | Profile v1.0 | Description |
|---|---|---|---|
| is-helm-v3 v1.0 | is-helm-v3 v1.0 | is-helm-v3 v1.0 | Checks that the given uri points to a Helm v3 chart. |
| has-readme v1.0 | has-readme v1.0 | has-readme v1.0 | Checks that the Helm chart contains the README.md file. |
| contains-test V1.0 | contains-test V1.0 | contains-test v1.0 | Checks that the Helm chart contains at least one test file. |
| has-kubeversion v1.1 | has-kubeversion v1.1 | has-kubeversion v1.0 | Checks that the Chart.yaml file of the Helm chart includes the kubeVersion field (v1.0) and is a valid semantic version (v1.1). |
| contains-values-schema v1.0 | contains-values-schema v1.0 | contains-values-schema v1.0 | Checks that the Helm chart contains a JSON schema file (values.schema.json) to validate the values.yaml file in the chart. |
| not-contains-crds v1.0 | not-contains-crds v1.0 | not-contains-crds v1.0 | Checks that the Helm chart does not include custom resource definitions (CRDs). |
| not-contain-csi-objects v1.0 | not-contain-csi-objects v1.0 | not-contain-csi-objects v1.0 | Checks that the Helm chart does not include Container Storage Interface (CSI) objects. |
| images-are-certified v1.0 | images-are-certified v1.0 | images-are-certified v1.0 | Checks that the images referenced by the Helm chart are Red Hat-certified. |
| helm-lint v1.0 | helm-lint v1.0 | helm-lint v1.0 | Checks that the chart is well formed by running the helm lint command. |
| chart-testing v1.0 | chart-testing v1.0 | chart-testing v1.0 | Installs the chart and verifies it on a Red Hat OpenShift Container Platform cluster. |
| contains-values v1.0 | contains-values v1.0 | contains-values v1.0 | Checks that the Helm chart contains the values¹ file. |
| required-annotations-present v1.0 | required-annotations-present v1.0 | - | Checks that the Helm chart contains the annotation: charts.openshift.io/name. |
| signature-is-valid v1.0 | - | - | Verifies a signed chart based on a provided public key |
¹ For more information on the values file, see values and Best Practices for using values.
There are two ways to run Helm chart checks, either through containers with podman/docker command, or run the binary directly (Linux only).
This section provides help on the basic usage of Helm chart checks with the podman or docker command.
- A container engine and the Podman or Docker CLI installed.
- Internet connection to check that the images are Red Hat certified.
- GitHub profile to submit the chart to the OpenShift Helm Charts Repository.
- Red Hat OpenShift Container Platform cluster.
-
Run all the available checks for a remotely available chart using a
uri, assuming the kube config file is available in ${HOME}/.kube:$ podman run --rm -i \ -e KUBECONFIG=/.kube/config \ -v "${HOME}/.kube":/.kube \ "quay.io/redhat-certification/chart-verifier" \ verify \ <chart-uri> -
Run all the available checks for a chart local to your file system, assuming the chart is in the current directory and the kube config file is available in ${HOME}/.kube:
$ podman run --rm \ -e KUBECONFIG=/.kube/config \ -v "${HOME}/.kube":/.kube \ -v $(pwd):/charts \ "quay.io/redhat-certification/chart-verifier" \ verify \ /charts/<chart> -
Get the list of options for the
verifycommand:$ podman run -it --rm quay.io/redhat-certification/chart-verifier verify helpThe output is similar to the following example:
Verifies a Helm chart by checking some of its characteristics Usage: chart-verifier verify <chart-uri> [flags] Flags: -S, --chart-set strings set values for the chart (can specify multiple or separate values with commas: key1=val1,key2=val2) -G, --chart-set-file strings set values from respective files specified via the command line (can specify multiple or separate values with commas: key1=path1,key2=path2) -X, --chart-set-string strings set STRING values for the chart (can specify multiple or separate values with commas: key1=val1,key2=val2) -F, --chart-values strings specify values in a YAML file or a URL (can specify multiple) --debug enable verbose output -x, --disable strings all checks will be enabled except the informed ones -e, --enable strings only the informed checks will be enabled --helm-install-timeout duration helm install timeout (default 5m0s) -h, --help help for verify --kube-apiserver string the address and the port for the Kubernetes API server --kube-as-group stringArray group to impersonate for the operation, this flag can be repeated to specify multiple groups. --kube-as-user string username to impersonate for the operation --kube-ca-file string the certificate authority file for the Kubernetes API server connection --kube-context string name of the kubeconfig context to use --kube-token string bearer token used for authentication --kubeconfig string path to the kubeconfig file -n, --namespace string namespace scope for this request -V, --openshift-version string set the value of certifiedOpenShiftVersions in the report -o, --output string the output format: default, json or yaml -k, --pgp-public-key string file containing gpg public key of the key used to sign the chart -d, --provider-delivery chart provider will provide the chart delivery mechanism (default: false) --registry-config string path to the registry config file (default "/home/baiju/.config/helm/registry.json") --repository-cache string path to the file containing cached repository indexes (default "/home/baiju/.cache/helm/repository") --repository-config string path to the file containing repository names and URLs (default "/home/baiju/.config/helm/repositories.yaml") -s, --set strings overrides a configuration, e.g: dummy.ok=false -f, --set-values strings specify application and check configuration values in a YAML file or a URL (can specify multiple) -E, --suppress-error-log suppress the error log (default: written to ./chartverifier/verifier-<timestamp>.log) --timeout duration time to wait for completion of chart install and test (default 30m0s) -w, --write-to-file write report to ./chartverifier/report.yaml (default: stdout) Global Flags: --config string config file (default is $HOME/.chart-verifier.yaml) -
Run a subset of the checks:
$ podman run --rm -i \ -e KUBECONFIG=/.kube/config \ -v "${HOME}/.kube":/.kube \ "quay.io/redhat-certification/chart-verifier" \ verify -e images-are-certified,helm-lint \ <chart-uri> -
Run all the checks except a subset:
$ podman run --rm -i \ -e KUBECONFIG=/.kube/config \ -v "${HOME}/.kube":/.kube \ "quay.io/redhat-certification/chart-verifier" \ verify -x images-are-certified,helm-lint \ <chart-uri> -
Provide chart-override values:
$ podman run --rm -i \ -e KUBECONFIG=/.kube/config \ -v "${HOME}/.kube":/.kube \ "quay.io/redhat-certification/chart-verifier" \ verify -S default.port=8080 \ <chart-uri> -
Provide chart-override values from a file in the current directory:
$ podman run --rm -i \ -e KUBECONFIG=/.kube/config \ -v "${HOME}/.kube":/.kube \ -v $(pwd):/values \ "quay.io/redhat-certification/chart-verifier" \ verify -F /values/overrides.yaml \ <chart-uri>
Increase the timeout value if chart-testing is going to take more time, default value is 30m.
$ podman run --rm -i \
-e KUBECONFIG=/.kube/config \
-v "${HOME}/.kube":/.kube \
-v $(pwd):/values \
"quay.io/redhat-certification/chart-verifier" \
verify --timeout 40m \
<chart-uri>
Note: In case chart-testing takes more time, it is advised to submit the report for certification since the certification process will use the default value of 30m.
By default the report is written to stdout which can be redirected to a file. For example:
$ podman run --rm -i \
-e KUBECONFIG=/.kube/config \
-v "${HOME}/.kube":/.kube \
"quay.io/redhat-certification/chart-verifier" \
verify -e images-are-certified,helm-lint \
<chart-uri> > report.yaml
Alternatively, use the -w option to write the report directly to the file ./chartverifier/report.yaml. To get this file a volume mount is required to /app/chartverifer. For example:
$ podman run --rm -i \
-e KUBECONFIG=/.kube/config \
-v "${HOME}/.kube":/.kube \
-v $(pwd)/chartverifier:/app/chartverifier \
-w \
"quay.io/redhat-certification/chart-verifier" \
verify -e images-are-certified,helm-lint \
<chart-uri>
If the file already exists it is overwritten.
By default an error log is written to file ./chartverifier/verify-<timestamp>.yaml. It includes any error messages, the results of each check and additional information around chart testing. To get a copy of the error log a volume mount is required to /app/chartverifer. For example:
$ podman run --rm -i \
-e KUBECONFIG=/.kube/config \
-v "${HOME}/.kube":/.kube \
-v $(pwd)/chartverifier:/app/chartverifier \
"quay.io/redhat-certification/chart-verifier" \
verify -e images-are-certified,helm-lint \
<chart-uri> > report.yaml
If multiple logs are added to the same directory a maximum of 10 will be kept. The files will be deleted oldest first.
Use the -E flag to suppress error log output.
Note: Error and warning messages are also output to stderr and are not suppressed by the -E option.
Alternatively, download chart-verifier binary from the release page, unzip the tarball with tar zxvf <tarball>, and run ./chart-verifier verify under the unzipped directory to perform Helm chart checks. Refer to the procedures in the podman/docker section, for example,
$ podman run --rm -i \
-e KUBECONFIG=/.kube/config \
-v "${HOME}/.kube":/.kube \
"quay.io/redhat-certification/chart-verifier" \
verify \
<chart-uri>
will become
$ ./chart-verifier verify <chart-uri>
By default, chart-verifier will assume kubeconfig is under $HOME/.kube, set environment variable KUBECONFIG for different kubeconfig file.
A profile defines a set of checks to run and an indication of whether each check is mandatory or optional. Following profiles are currently available:
- partner
- Defines the requirements for a partner chart to pass helm chart certfication.
- All checks are mandatory, that is they must all pass for a partner helm chart to be certified.
- redhat
- Defines the requirements for a red hat internal chart to pass helm chart certfication.
- All checks are mandatory, that is they must all pass for a Red Hat helm chart to be certified.
- community
- Defines the requirements for a community chart to pass helm chart certfication.
- The
helm-lintcheck is the only mandatory check with all other checks optional.
- developer-console
- Defines the requirements for a developer-console chart to be validated.
- The checks which are enabled for this profile are
helm-lint,is-helm-v3,contains-values,contains-values-schema,has-kubeversionandhas-readme. All these checks are mandatory.
- default
- The default is the same as the partner profile and is used if a specific one is not specified.
- All checks are mandatory.
Each profile also has a version and currently there are three profile versions: v1.0, v1.1 and v1.2. The developer-console just has one profile version v1.0.
Compared to profile v1.1, adds a new check:
| check | partner | RedHat | community | default |
|---|---|---|---|---|
| signature-is-valid v1.0 | mandatory | mandatory | optional | mandatory |
Annotations added to a v1.1 profile report are common to all profile types: partner, RedHat, community and default
| annotation | description |
|---|---|
| digests.chart | The sha value of the chart as calculated from the copy loaded into memory. |
| digests.package | The sha value of the chart tarball if used to create the report. |
| testedOpenShiftVersion | The Open Shift version that was used by the chart-testing check. |
| lastCertifiedTimestamp | The time that the report was created by the chart verifier |
| supportedOpenShiftVersions | The Open Shift versions supported by the chart based on the kuberVersion attrinute in chart.yaml |
This table shows which checks are preformed and whether or not they ar mnandatory or optional for each profile type.
| check | partner | RedHat | community | default |
|---|---|---|---|---|
| is-helm-v3 v.1.0 | mandatory | mandatory | optional | mandatory |
| has-readme v1.0 | mandatory | mandatory | optional | mandatory |
| contains-test v1.0 | mandatory | mandatory | optional | mandatory |
| has-kubeversion v1.1 | mandatory | mandatory | optional | mandatory |
| contains-values-schema v1.0 | mandatory | mandatory | optional | mandatory |
| not-contains-crds v1.0 | mandatory | mandatory | optional | mandatory |
| not-contain-csi-objects v1.0 | mandatory | mandatory | optional | mandatory |
| images-are-certified v1.0 | mandatory | mandatory | optional | mandatory |
| helm-lint v1.0 | mandatory | mandatory | optional | mandatory |
| chart-testing v1.0 | mandatory | mandatory | optional | mandatory |
| contains-values v1.0 | mandatory | mandatory | optional | mandatory |
| required-annotations-present v1.0 | mandatory | mandatory | optional | mandatory |
Annotations added to a v1.0 profile report are common to all profile types: partner, RedHat, community and default
| annotation | description |
|---|---|
| digests.chart | The sha value of the chart as calculated from the copy loaded into memory. |
| digests.package | The sha value of the chart tarball if used to create the report. |
| certifiedOpenShiftVersion | The Open Shift version that was used by the chart-testing check. |
| lastCertifiedTimestamp | The time that the report was created by the chart verifier |
This table shows which checks are preformed and whether or not they ar mnandatory or potion for each profile type.
| check | partner | RedHat | community | default |
|---|---|---|---|---|
| is-helm-v3 v.1.0 | mandatory | mandatory | optional | mandatory |
| has-readme v1.0 | mandatory | mandatory | optional | mandatory |
| contains-test v1.0 | mandatory | mandatory | optional | mandatory |
| has-kubeversion v1.0 | mandatory | mandatory | optional | mandatory |
| contains-values-schema v1.0 | mandatory | mandatory | optional | mandatory |
| not-contains-crds v1.0 | mandatory | mandatory | optional | mandatory |
| not-contain-csi-objects v1.0 | mandatory | mandatory | optional | mandatory |
| images-are-certified v1.0 | mandatory | mandatory | optional | mandatory |
| helm-lint v1.0 | mandatory | mandatory | optional | mandatory |
| chart-testing v1.0 | mandatory | mandatory | optional | mandatory |
| contains-values v1.0 | mandatory | mandatory | optional | mandatory |
To specify which profile to use the --set flag:
--set profile.vendorType=partner
valid values based on current profiles: partner, community, redhat, default
default is same as partner.
If value specified is not specified or not recognized, default will be assumed.
The flag name is case insensitive.
--set profile.version=v1.1
Valid values based on current profiles: v1.0, v1.0
If value specified is not specified or not recognized, v1.1 will be assumed.
The flag name is case insensitive.
For example:
$ podman run --rm -i \
-e KUBECONFIG=/.kube/config \
-v "${HOME}/.kube":/.kube \
"quay.io/redhat-certification/chart-verifier" \
verify --set profile.vendorType=partner, profile.version=v1.1 \
<chart-uri>
You can configure the chart-testing check by performing one of the following steps:
-
Option 1: Through the
--setcommand line option:$ chart-verifier \ verify \ --enable chart-testing \ --set chart-testing.buildId=${BUILD_ID} \ --set chart-testing.upgrade=true \ --set chart-testing.skipMissingValues=true \ --set chart-testing.namespace=${NAMESPACE} \ --set chart-testing.releaseLabel="app.kubernetes.io/instance" \ --set chart-testing.release=${RELEASE} \ some-chart.tgz -
Option 2: Create a YAML file (config.yaml) similar to the following example:
chart-testing: buildId: <BUILD_ID> upgrade: true skipMissingValues: true namespace: <NAMESPACE> releaseLabel: "app.kubernetes.io/instance" release: <RELEASE>Specify the file using the
--set-valuescommand line option:$ chart-verifier verify --enable chart-testing --set-values config.yaml some-chart.tgzAll settings are optional, if not set default values will be used.
If the chart requires overrides values, these can be set using through the --chart-set command line options:
-S, --chart-set strings set values for the chart (can specify multiple or separate values with commas: key1=val1,key2=val2)
-G, --chart-set-file strings set values from respective files specified via the command line (can specify multiple or separate values with commas: key1=path1,key2=path2)
-X, --chart-set-string strings set STRING values for the chart (can specify multiple or separate values with commas: key1=val1,key2=val2)
-F, --chart-values strings specify values in a YAML file or a URL (can specify multiple)
The chart-testing check performs the following actions, keeping the semantics provided by github.com/helm/chart-testing:
- Install: the chart being verified will be installed in the available OpenShift cluster utilizing the same semantics client-go uses to find the current context:
--kubeconfig flagKUBECONFIGenvironment variable$HOME/.kube/config.
- Test: once a release is installed for the chart being verified, performs the same actions as helm test would, which installing all chart resources containing the "helm.sh/hook": test annotation.
The check will be considered successful when the chart's installation and tests are all successful.
For the chart install and test check there are two configurable timeout options:
--helm-install-timeout- limits how long the
chart-testingcheck waits for chart install to complete. - default is 5 minutes
- set for example:
--helm-install-timeout 10m0s
- limits how long the
--timeout- limits how long the check waits for the
chart-testingcheck to complete:- chart install
- wait for deployments to be available
- run the test
- default is 30 minutes
- set for example:
--timeout 60m0s
- limits how long the check waits for the
Notes:
- The timeouts are independent.
- Changing one timeout does not impact the other.
- If
helm-install-timeoutis increased, consider also increasingtimeout
- The helm chart certification process uses default timeout values.
- If a helm chart can only pass the chart testing check with modified timeouts a verifier report must be included in the chart submission.
In profile v1.2 a new mandatory check is added for signed charts. For information on signed charts see helm provenance and integrity.
- For a signed chart:
- The check requires a pgp public key file to run.
- Ensures a signed chart is validly signed for the public key which will be provided to users to verify the chart.
- Specify the public key file using the flag:
--pgp-public-key <public-key-file> - The check runs
helm verifyusing the public key- If
helm verifyfails the check will fail. - For information on
helm verifysee helm verify
- If
- To create the pgp public key file:
- run:
gpg --export -a <User-Name> > <public-key-file>- User-Name is the user name of the secret key used to sign the chart.
- run:
- If a pgp public key is not provided the check result will be "SKIPPED" which is considered a PASS for chart certification purposes.
- The check requires a pgp public key file to run.
- For a non-signed chart:
- the check result will be "SKIPPED" which is considered a PASS for chart certification purposes.
For troubleshooting this check see: signature-is-valid v1.0.