Cannot upgrade from 0.12.3 to 0.16.3

[paymog]

Expected Behavior

I should not be spammed with issues about connecting to my kubernetes cluster

Actual Behavior

I get the following error message printed for most (all?) of my resources

│ Error: Failed to configure client: exec plugin: invalid apiVersion "client.authentication.io/v1beta1"
│
│   with kubernetes_cluster_role_binding.datadog-agent-clusterrole-binding,
│   on cdk.tf.json line 2030, in resource.kubernetes_cluster_role_binding.datadog-agent-clusterrole-binding:
│ 2030:       },
│
goldsky-infra-dev  ╷
                   │ Error: Failed to configure client: exec plugin: invalid apiVersion "client.authentication.io/v1beta1"
                   │
                   │   with kubernetes_cluster_role_binding.datadog-agent-clusterrole-binding (datadog-agent-clusterrole-binding),
                   │   on cdk.tf.json line 2030, in resource.kubernetes_cluster_role_binding.datadog-agent-clusterrole-binding (datadog-agent-clusterrole-binding):
                   │ 2030:       },

Steps to Reproduce

Not sure how to reliably reproduce this. I was on cdktf 0.12.3 and it worked flawlessly. I decided to upgrade to 0.16.3 and worked my way through all the build time issues with import reorganization. But now when I try to diff my stack, I get the above output for almost all of my resources.

I found this issue but it's not clear exactly what I need to do to fix my situation.

Versions

❯❯❯ npx cdktf debug
cdktf debug
language: typescript
cdktf-cli: 0.16.3
node: v17.9.1
cdktf: 0.16.3
constructs: 10.0.36
jsii: null
terraform: 1.3.5
arch: x64
os: linux 5.15.0-1036-aws

Providers

❯❯❯ npx cdktf provider list
┌─────────────────────┬──────────────────┬─────────┬────────────┬────────────────────────────┬─────────────────┐
│ Provider Name       │ Provider Version │ CDKTF   │ Constraint │ Package Name               │ Package Version │
├─────────────────────┼──────────────────┼─────────┼────────────┼────────────────────────────┼─────────────────┤
│ aws                 │ 3.76.1           │         │ ~> 3.42    │                            │                 │
├─────────────────────┼──────────────────┼─────────┼────────────┼────────────────────────────┼─────────────────┤
│ kubernetes          │ 2.7.1            │         │  ~> 2.7.0  │                            │                 │
├─────────────────────┼──────────────────┼─────────┼────────────┼────────────────────────────┼─────────────────┤
│ http                │ 2.1.0            │         │  ~> 2.1.0  │                            │                 │
├─────────────────────┼──────────────────┼─────────┼────────────┼────────────────────────────┼─────────────────┤
│ tls                 │ 3.1.0            │         │  ~> 3.1.0  │                            │                 │
├─────────────────────┼──────────────────┼─────────┼────────────┼────────────────────────────┼─────────────────┤
│ helm                │ 2.4.1            │         │  ~> 2.4.1  │                            │                 │
├─────────────────────┼──────────────────┼─────────┼────────────┼────────────────────────────┼─────────────────┤
│ random              │ 3.1.3            │         │  ~> 3.1.0  │                            │                 │
├─────────────────────┼──────────────────┼─────────┼────────────┼────────────────────────────┼─────────────────┤
│ gavinbunney/kubectl │ 1.14.0           │         │ ~> 1.14.0  │                            │                 │
├─────────────────────┼──────────────────┼─────────┼────────────┼────────────────────────────┼─────────────────┤
│ DopplerHQ/doppler   │ 1.1.6            │         │ 1.1.6      │                            │                 │
├─────────────────────┼──────────────────┼─────────┼────────────┼────────────────────────────┼─────────────────┤
│ aws                 │ 4.66.1           │ ^0.16.0 │            │ @cdktf/provider-aws        │ 14.0.4          │
├─────────────────────┼──────────────────┼─────────┼────────────┼────────────────────────────┼─────────────────┤
│ kubernetes          │ 2.21.1           │ ^0.16.3 │            │ @cdktf/provider-kubernetes │ 6.0.4           │
└─────────────────────┴──────────────────┴─────────┴────────────┴────────────────────────────┴─────────────────┘

Gist

No response

Possible Solutions

this issue mentions this problem but it's not clear how to fix it.

My kubeconfig uses the beta1 api version:

users:
- name: arn:aws:eks:us-west-2:<redacted>:cluster/mercury-prod
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1beta1
      args:
      - --region
      - us-west-2
      - eks
      - get-token
      - --cluster-name
      - mercury-prod
      command: aws
      env:
      - name: AWS_PROFILE
        value: prod
      interactiveMode: IfAvailable
      provideClusterInfo: false
- name: arn:aws:eks:us-west-2:<redacted>:cluster/mercury-dev
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1beta1
      args:
      - --region
      - us-west-2
      - eks
      - get-token
      - --cluster-name
      - mercury-dev
      command: aws
      env: null
      interactiveMode: IfAvailable
      provideClusterInfo: false

My kubernetes provider uses beta1

    const kubernetesProvider = new KubernetesProvider(
      this,
      "kubernetesProvider",
      {
        host: eksCluster.endpoint,
        exec: {
          apiVersion: "client.authentication.io/v1beta1",
          args: ["eks", "get-token", "--cluster-name", eksCluster.name],
          command: "aws",
        },
        clusterCaCertificate: `\${base64decode(aws_eks_cluster.eksCluster.certificate_authority.0.data)}`,
      }
    );

My aws cli version is

 ❯❯❯ aws --version
aws-cli/2.8.13 Python/3.9.11 Linux/5.15.0-1036-aws exe/x86_64.ubuntu.20 prompt/off

I've tried updating my kubeconfig with

 ❯❯❯ aws eks update-kubeconfig --name mercury-dev --region us-west-2
Updated context arn:aws:eks:us-west-2:716091368358:cluster/mercury-dev in /home/ubuntu/.kube/config

but that still resulted in the same errors.

Workarounds

No response

Anything Else?

No response

References

hashicorp/terraform-provider-helm#893 (comment)

Help Wanted

• I'm interested in contributing a fix myself

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[paymog]

I just realized the aws cli has two versions with the same binary name. I just updated to the latest version of the cli using and now have version

 ❯❯❯ aws --version
aws-cli/2.11.27 Python/3.11.3 Linux/5.15.0-1036-aws exe/x86_64.ubuntu.20 prompt/off

but I still get the same error as above.

[paymog]

Well, turns out I wasn't paying enough attention. My kubernetes provider was using api version client.authentication.io/v1beta1 when it should have been using client.authentication.k8s.io/v1beta1 (note the extra .k8s)

[github-actions]

I'm going to lock this issue because it has been closed for 30 days. This helps our maintainers find and focus on the active issues. If you've found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.