Resource id mismatch after updating cdktf & cdktf-cli from v0.16.3 to v0.17.0

[ApexCaptain]

Expected Behavior

I want all ids of resources generated from previous version(v0.16.3) of cdktf & cdktf-cli
matches after updating to v0.17.0.

// This is one of my `stack` code.
export class KeysStack extends TerraformStack {
  ...
  masterKey = new PrivateKey(this, 'master-key', {
    algorithm: 'RSA',
    rsaBits: 4096,
  });
  ...
}

This is cdktf diff output when the version of cdktf/cdtkf-cli is v0.16.3

...
keys-stack  tls_private_key.keysstack_masterkey_280F69D3 (master-key): Refreshing state... [id=56110643fa7c8a4b78d866a81affd2520f9d9e8d]
...

keys-stack  No changes. Your infrastructure matches the configuration.

            Terraform has compared your real infrastructure against your configuration
            and found no differences, so no changes are needed.
Done in 11.77s

Actual Behavior

After updating cdktf & cdktf-cli version, cdktf diff output has changed as follow

...
keys-stack  Terraform used the selected providers to generate the following execution
            plan. Resource actions are indicated with the following symbols:
              + create
              - destroy

            Terraform will perform the following actions:
keys-stack    # tls_private_key.keysstack_masterkey_280F69D3 will be destroyed
              # (because tls_private_key.keysstack_masterkey_280F69D3 is not in configuration)
              - resource "tls_private_key" "keysstack_masterkey_280F69D3" {
                  - algorithm                     = "RSA" -> null
                  - ecdsa_curve                   = "P224" -> null
                  - id                            = "56110643fa7c8a4b78d866a81affd2520f9d9e8d" -> null
                  - private_key_openssh           = (sensitive value) -> null
                  - private_key_pem               = (sensitive value) -> null
                  - private_key_pem_pkcs8         = (sensitive value) -> null
                  - public_key_fingerprint_md5    = "<Some Secret Info>" -> null
                  - public_key_fingerprint_sha256 = "<Some Secret Info>" -> null
                  - public_key_openssh            = <<-EOT
                        <Some Secret Info>
                    EOT -> null
                  - public_key_pem                = <<-EOT
                         <Some Secret Info>
                    EOT -> null
                  - rsa_bits                      = 4096 -> null
                }

              # tls_private_key.master-key (master-key) will be created
              + resource "tls_private_key" "master-key" {
                  + algorithm                     = "RSA"
                  + ecdsa_curve                   = "P224"
                  + id                            = (known after apply)
                  + private_key_openssh           = (sensitive value)
                  + private_key_pem               = (sensitive value)
                  + private_key_pem_pkcs8         = (sensitive value)
                  + public_key_fingerprint_md5    = (known after apply)
                  + public_key_fingerprint_sha256 = (known after apply)
                  + public_key_openssh            = (known after apply)
                  + public_key_pem                = (known after apply)
                  + rsa_bits                      = 4096
                }

            Plan: 1 to add, 0 to change, 1 to destroy.

This is cdktf.output file from cdktf v0.16.3

"tls_private_key": {
  "keysstack_masterkey_280F69D3": {
    "//": {
      "metadata": {
        "path": "keys-stack/master-key",
        "uniqueId": "keysstack_masterkey_280F69D3"
      }
    },
    "algorithm": "RSA",
    "rsa_bits": 4096
  }
}

And this is cdktf.output file from cdktf v0.17.0

"tls_private_key": {
  "master-key": {
    "//": {
      "metadata": {
        "path": "keys-stack/master-key",
        "uniqueId": "master-key"
      }
    },
    "algorithm": "RSA",
    "rsa_bits": 4096
  }
}

One thing I can say is that unique id of the resource master-key
has changed from keysstack_masterkey_280F69D3 to just simply, master-key, which is
far more intuitive, however, does not equal to remote state.

That is why cdktf is trying to destroy and recreate all my precious resources.
Noy only just simple tls key but also other resources such as EC2 instances,
databases, k8s clusters, etc.

Did I do something terribly wrong?
Are there any additional steps when updating cdktf?
If so, plz let me know.

Thank you 😊

p.s
Of course, it's not urgent.
Sticking to previous version, I can still manage my infra.

Steps to Reproduce

1. Generate any new resource on cdktf version v0.16.3
2. Deploy reosurce
3. Update cdktf & cdktf-cli to v0.17.0
4. Plan

Versions

language: typescript
cdktf-cli: 0.17.0
cdktf: 0.17.0
node: v18.16.0
constructs: 10.2.52
terraform: 1.5.0
arch: amd64
os: Debian GNU/Linux 5.15.90.1-microsoft-standard-WSL2

Providers

No response

Gist

No response

Possible Solutions

No response

Workarounds

No response

Anything Else?

No response

References

No response

Help Wanted

• I'm interested in contributing a fix myself

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[ansgarm]

Hi @ApexCaptain 👋

Could you share your cdktf.json config file? I suspect that you have some feature flags in there that we deprecated in v0.15.0 and removed in CDKTF v0.17.0.

How many resources are affected?

[ApexCaptain]

Thanks for your response @ansgarm

This is my cdktf.json file

{
  "output": "cdktf.out",
  "codeMakerOutput": "src/generated",
  "app": "yarn ts-node -r tsconfig-paths/register src/main.ts",
  "language": "typescript",
  "projectId": "6d8affb3-b3ac-48a3-a593-680005e6eb3a",
  "sendCrashReports": false,
  "terraformProviders": [
    {
      "name": "aws",
      "source": "hashicorp/aws",
      "version": "~> 4.0"
    },
    {
      "name": "cloudinit",
      "source": "hashicorp/cloudinit",
      "version": "~> 2.3"
    },
    {
      "name": "local",
      "source": "hashicorp/local",
      "version": "~> 2.3"
    },
    {
      "name": "oci",
      "source": "oracle/oci",
      "version": "~> 4.122"
    },
    {
      "name": "tls",
      "source": "hashicorp/tls",
      "version": "~> 4.0"
    },
    {
      "name": "null",
      "source": "hashicorp/null",
      "version": "~> 3.2"
    },
    {
      "name": "kubernetes",
      "source": "hashicorp/kubernetes",
      "version": "~> 2.20"
    },
    {
      "name": "github",
      "source": "integrations/github",
      "version": "~> 5.0"
    },
    {
      "name": "external",
      "source": "hashicorp/external",
      "version": "~> 2.3"
    }
  ],
  "terraformModules": [],
}

I think this feature affects only the resources that are generated by terraform and cross-stack-references.
In other words, Data resources are not in the case.
There are, 86 resources affected in total.

[DanielMSchmidt]

Hey, it seems like you have been using cdktf without the feature flags. We deprecated them in 0.15 and removed them in 0.17. The result is that the resource ids change the way you are experiencing right now. This feature was introduced over 2 years ago to make them more readable and e.g. variables or outputs more accessible.

To move forward you can either move your state (see the refactoring guide) or you can use resource.overrideLogicalId("keysstack_masterkey_280F69D3") to pin the IDs.

[jsteinich]

It should also be possible to override allocateLogicalId on the stack to replicate the old behavior

[github-actions]

I'm going to lock this issue because it has been closed for 30 days. This helps our maintainers find and focus on the active issues. If you've found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.