Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shouldn't need to run update-kubeconfig manually #28

Open
jaytmiller opened this issue May 7, 2020 · 6 comments
Open

Shouldn't need to run update-kubeconfig manually #28

jaytmiller opened this issue May 7, 2020 · 6 comments

Comments

@jaytmiller
Copy link

jaytmiller commented May 7, 2020

I ran into this:

Error: stat /Users/jmiller/.kube/config: no such file or directory on autoscaler.tf line 63, in resource "helm_release" "cluster-autoscaler": 63: resource "helm_release" "cluster-autoscaler" {

and @yuvipanda suggested I write an issue about it.

The manual work-around was:

aws eks update-kubeconfig --name <CLUSTER-NAME>

@salvis2
Copy link
Member

salvis2 commented Aug 10, 2020

Yeah, I've had to run

aws eks update-kubeconfig --name=<name> --region=<region> --profile=<profile>

after every cluster's creation. I have a feeling Terraform doesn't want to change your local kubeconfig.

There are options for the terraform-aws-eks module that can output a kubeconfig file, but there would still be a step to switch to that file instead of ~/.kube/config I think.

@yuvipanda
Copy link
Member

Yeah, might be useful to output a kubeconfig file locally, and then set the KUBECONFIG env var to point to it? Terraform definitely doesn't wanna change your ~/.kube/config...

@salvis2
Copy link
Member

salvis2 commented Aug 11, 2020

I think by default the terraform-aws-eks module outputs a kubeconfig file into the directory where you ran Terraform commands and names it kubeconfig_${var.cluster_name} (see https://github.com/terraform-aws-modules/terraform-aws-eks/blob/master/kubectl.tf). Maybe there is a good way to automatically switch to the KUBECONFIG env var to it?

@jaytmiller
Copy link
Author

We're wondering about maybe adding this to hubploy? (Maybe as an option which defaults to on?) How "expert" is the scenario where automatic kubeconfig is bad?

@yuvipanda
Copy link
Member

hubploy doesn't depend on KUBECONFIG - it gets credentials from the keys set in hubploy.yaml. Current environment variables shouldn't matter, and I'd prefer to not have hubploy modify anything in ~/.kube/config.

If you can find a way to set KUBECONFIG env var after terraform completes, that's not a bad idea. However, I think changing what KUBECONFIG points to without the user explicitly asking for it is recipe for trouble, as you can then accidentally perform operations in the wrong cluster! So I'd generally recommend against it, nad instead document what people can do.

@jaytmiller
Copy link
Author

OK, sounds totally reasonable to me. We're in the process of documenting the end-to-end process internally so this is easy to add to those docs explicitly. If we ever do make a comprehensive wrapper script to automate installing all the pieces end-to-end, we could also add this step to that script.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants