Provider Cannot Assume Role From EKS

[CSimpiFoN]

When the provider is run in an EKS pod, with the IAM assume role setup, that works with the AWS provider too, the provider cannot use the assumed role, the following error is dropped:
* failed to load credentials: unable to assume role, {role_arn}: operation error STS: AssumeRole, https response error StatusCode: 403, RequestID: 285fca36-2088-4448-b083-b61f3bc85cfe, api error AccessDenied: User: {temp_role_user} is not authorized to perform: sts:AssumeRole on resource: {role_arn}

Provider version: 0.7.1

Configuration:

provider "kafka" {
  bootstrap_servers    = try(split(",", aws_msk_cluster.kafka[0].bootstrap_brokers_sasl_iam), [])
  tls_enabled          = true
  sasl_mechanism       = "aws-iam"
  sasl_aws_region      = var.region
  sasl_aws_creds_debug = var.aws_creds_debug
}

AWS provider is able to assume the role without any issue.

I suspect the provider should work the same way as the AWS SDK, that uses the attached ServiceAccount to gather the temporary tokens to assume the role, and then to connect to MSK.

[benjefferies]

@CSimpiFoN did you find a workaround?

[lebohnes]

We also faced this error inside a EKS pod. I am not fully sure, if its the same error, but we kept getting the error message:
Error: kafka: client has run out of available brokers to talk to: dial tcp: missing address
The iam auth worked locally, but we retrieved this error for different aws-iam configs for the provider.

[iamnotabout]

it's cuz the pod has AWS_ARN_ROLE env var set and by default it uses that value for sasl_aws_role_arn.

as seen here

terraform-provider-kafka/kafka/provider.go

Line 79 in 5de3f1e

DefaultFunc: schema.EnvDefaultFunc("AWS_ROLE_ARN", nil),

workaround is set sasl_aws_role_arn = "" inside your provider "kafka" { , then it will use your pod's service account's IAM Role

[lebohnes]

works for me now for 0.8.1