-
Notifications
You must be signed in to change notification settings - Fork 92
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cluster Recreate Causes Kustomize Module To Fail #156
Comments
This needs more investigation, but I've also seen this myself. The module receives the credentials the cluster resources output as an input. My preliminary investigation of this issue suggests that somehow when creating a plan to create cluster and cluster services, Terraform has the dependency graph correct. Also on destroy, the order seems correct, K8s resources first, then the cluster. But if the cluster gets destroyed and recreated, the graph does not first destroy the K8s resources, then destroy the cluster, then recreate the cluster and finally recreate the resources. That means the resources stay in the state but there are no cluster credentials to refresh them during plan. |
To make it easier to understand, I created a simple config to reproduce the issue. https://github.com/pst/debugrecreateplan The example repo shows the behaviour with both the official kubernetes provider as well as my kustomize provider on top of a KinD cluster. So it's also not Google provider specific. And so far it seems to support my theory. Create and destroy plans correctly handle resources and clusters. But destroy and re-create plans do not handle the K8s resources on the cluster at all. Create plan
Destroy plan
Destroy & recreate plan Triggered by changing
|
Likely related upstream issue: hashicorp/terraform#22572 |
This seems to be fixed in the last couple of times I've used it, I'll do a test to make sure |
This is definitly still an issue and it's not Kubestack specific, but generally an issue with Terraform. I hope moving away from the in-module manifests and towards the new native modules may make the issue less frequent. But even then, e.g. the auth-configmap for EKS in the module may still cause this. The only real workaround is a |
Ah right, I recreated a GKE cluster that had a couple of manifests and it didn't break, was hoping that meant it was fixed |
Problem
We are recreating our cluster to enable the Private Node Pools, the issue seems to be that because we are recreating the cluster, the Kustomize Provider is trying to communicate with the Kubernetes Cluster on the default Localhost
Logs
Steps To Reproduce
Create a cluster with the setting:
Then Once created change the value:
and run on that TF workspace:
Workaround
Currently there is a workaround by using:
This will update the cluster which should then fix the problem
The text was updated successfully, but these errors were encountered: