Values yaml schema expects String for oidcConfiguration.customParams which causes upgrades to fail

[ArcanElement]

Checks

• I have checked for existing issues.
• This report is about the Openmetadata Helm Chart.
• This report is about the Openmetadata Dependencies Helm Chart.

Description

While upgrading from 1.2.1 to 1.4.1, I configured the new oidcConfiguration section in the authorizer section of the values file (we use KeyCloak to provide SSO). I am using the default value of "" for customParams. On upgrade, the run-db-migrations init pod of the new container fails due to an unexpected type of String for the customParams field (see the full exception in the attached logs).

I have tried changing the customParams field to {} to match the expected type of "LinkedHashMap", however the helm upgrade command fails because the values schema expects a string for customParams. A value of "{}" yields the same error as a value of "".

Steps to reproduce

Install version 1.2.1 of OpenMetadata on Kubernetes with a "custom-oidc" authentication provider and a clientType of "public".
Update to version 1.4.1 with a client type of "confidential" and a configured oidcConfiguration section (at least "enabled: true").

Chart Version

1.4.1

App Version

1.4.1

Kubernetes Version

1.26.7

Helm Version

3.14.4

Relevant Logs

|||||||
 ||||   ||||      ____
||||     ||||    / __ \
||||     ||||   | |  | | _ __    ___  _ __
|||||   |||||   | |  | || '_ \  / _ \| '_ \
|||||||||||||   | |__| || |_) ||  __/| | | |
|||||||||||||    \____/ | .__/  \___||_| |_|
||| ||||| |||    __  __ | |    _              _         _
|||  |||  |||   |  \/  ||_|   | |            | |       | |
|||   |   |||   | \  / |  ___ | |_  __ _   __| |  __ _ | |_  __ _
|||       |||   | |\/| | / _ \| __|/ _` | / _` | / _` || __|/ _` |
||| || || |||   | |  | ||  __/| |_| (_| || (_| || (_| || |_| (_| |
||| ||||| |||   |_|  |_| \___| \__|\__,_| \__,_| \__,_| \__|\__,_|
 |||||||||||
   |||||||
Migrating the OpenMetadata Schema.
Failed to db migration due to
io.dropwizard.configuration.ConfigurationParsingException: /opt/openmetadata/bootstrap/../conf/openmetadata.yaml has an error:
  * Incorrect type of value at: authenticationConfiguration.oidcConfiguration.customParams; is of type: String, expected: LinkedHashMap

        at io.dropwizard.configuration.ConfigurationParsingException$Builder.build(ConfigurationParsingException.java:277)
        at io.dropwizard.configuration.BaseConfigurationFactory.build(BaseConfigurationFactory.java:170)
        at org.openmetadata.service.util.OpenMetadataOperations.parseConfig(OpenMetadataOperations.java:497)
        at org.openmetadata.service.util.OpenMetadataOperations.migrate(OpenMetadataOperations.java:215)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at picocli.CommandLine.executeUserObject(CommandLine.java:2066)
        at picocli.CommandLine.access$1500(CommandLine.java:148)
        at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2461)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2453)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2415)
        at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2273)
        at picocli.CommandLine$RunLast.execute(CommandLine.java:2417)
        at picocli.CommandLine.execute(CommandLine.java:2170)
        at org.openmetadata.service.util.OpenMetadataOperations.main(OpenMetadataOperations.java:632)
Caused by: com.fasterxml.jackson.databind.exc.InvalidFormatException: Cannot coerce empty String ("") to element of `java.util.LinkedHashMap` (but could if coercion was enabled using `CoercionConfig`)
 at [Source: UNKNOWN; byte offset: #UNKNOWN] (through reference chain: org.openmetadata.service.OpenMetadataApplicationConfig["authenticationConfiguration"]->org.openmetadata.schema.api.security.AuthenticationConfiguration["oidcConfiguration"]->org.openmetadata.schema.security.client.OidcClientConfig["customParams"])
        at com.fasterxml.jackson.databind.exc.InvalidFormatException.from(InvalidFormatException.java:67)
        at com.fasterxml.jackson.databind.DeserializationContext.reportBadCoercion(DeserializationContext.java:1832)
        at com.fasterxml.jackson.databind.deser.std.StdDeserializer._checkCoercionFail(StdDeserializer.java:1662)
        at com.fasterxml.jackson.databind.deser.std.StdDeserializer._deserializeFromEmptyString(StdDeserializer.java:325)
        at com.fasterxml.jackson.databind.deser.std.StdDeserializer._deserializeFromString(StdDeserializer.java:270)
        at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:454)
        at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:32)
        at com.fasterxml.jackson.module.blackbird.deser.SettableObjectProperty.deserializeAndSet(SettableObjectProperty.java:44)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:273)
        at com.fasterxml.jackson.module.blackbird.deser.SuperSonicBeanDeserializer.deserialize(SuperSonicBeanDeserializer.java:155)
        at com.fasterxml.jackson.module.blackbird.deser.SettableObjectProperty.deserializeAndSet(SettableObjectProperty.java:44)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:273)
        at com.fasterxml.jackson.module.blackbird.deser.SuperSonicBeanDeserializer.deserialize(SuperSonicBeanDeserializer.java:155)
        at com.fasterxml.jackson.module.blackbird.deser.SettableObjectProperty.deserializeAndSet(SettableObjectProperty.java:44)
        at com.fasterxml.jackson.module.blackbird.deser.SuperSonicBeanDeserializer.deserialize(SuperSonicBeanDeserializer.java:155)
        at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342)
        at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:4875)
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3033)
        at io.dropwizard.configuration.BaseConfigurationFactory.build(BaseConfigurationFactory.java:148)
        ... 16 common frames omitted

Custom Helm Values

authentication:
      enabled: true
      clientType: confidential
      provider: "custom-oidc"
      publicKeys:
      - "http://<keycloak realm URL>/protocol/openid-connect/certs"
      - "http://localhost:8585/api/v1/system/config/jwks"
      authority: "https://<keycloak realm URL>/"
      clientId: "open-metadata"
      callbackUrl: "https://<OpenMetadata URL>/callback"
      jwtPrincipalClaims:
      - "email"
      - "preferred_username"
      - "sub"
      enableSelfSignup: true
      oidcConfiguration:
        enabled: true
        oidcType: "custom-oidc"
        clientId:
          secretRef: keycloak-client-credentials
          secretKey: openmetadata-client-id
        clientSecret:
          secretRef: keycloak-client-credentials
          secretKey: openmetadata-client-secret
        scope: "openid email profile"
        discoveryUri: "https://<keycloak realm URL>/.well-known/openid-configuration"
        useNonce: true
        preferredJwsAlgorithm: RS256
        responseType: code
        disablePkce: true
        callbackUrl: "https://<OpenMetadata URL>/callback"
        serverUrl: "https://<OpenMetadata URL>"
        clientAuthenticationMethod: client_secret_post
        tenant: ""
        maxClockSkew: ""
        customParams: ""

Have you joined Openmetadata Slack community?

Yes

[baptistegh]

Same error for me. I can't deploy OpenMetadata with oidc authentication enabled.

[marlenekoh]

Bump @akash-jain-10 this breaking change prevents openmetadata from being deployed with oidc.

ref: https://github.com/open-metadata/openmetadata-helm-charts/blob/main/charts/openmetadata/templates/secrets.yaml#L242

possible to allow default null value for the OIDC_CUSTOM_PARAMS environment variable instead? not sure what are possible values that will go in, will be great if there's examples

[TomaszSyc]

as a workaround I've set extraEnvs like below:

values.yaml

  extraEnvs:
  - name: OIDC_CUSTOM_PARAMS
    value: null