Skip to content

Using this template

Jump to bottom
Kevin Mack edited this page Aug 29, 2018 · 2 revisions

A template for building a kubernetes environment with a master and worker nodes with terraform and packer.

Intention:

The purpose of this template is to provide an easy-to-use approach to using an Infrastructure-as-a-service deployment to deploy a kubernetes cluster on Microsoft Azure. The goal being that you can start fresh with a standardized approach and preconfigured master and worker nodes.

How it works?

This template create a master node, and as many worker nodes as you specify, and during creation will automatically execute the scripts required to join those nodes to the cluster. The benefit of this being that once your cluster is created, all that is required to add additional nodes is to increase the count of the "lkwn" vm type, and reapply the template. This will cause the newe VMs to be created and the cluster will start orchestrating them automatically.

This template can also be built into a CI/CD pipeline to automatically provision the kubernetes cluster prior to pushing pods to it.

Naming Conventions used:

For this template, several naming conventions have been used, and they are outlined below:

  • Deployment Code / Location Code: You will see references to these codes throughout the template, and them specified as part of the packer and terraform environment configuration. if you run this as is, you will see the resources start with the name "SUS1AZ1-" this denotes S = staging, US1 = United States first deployment, Arizona Data Center first deployment.
  • lkma: Denotes the Linux Kubernetes MAster.
  • lkwn: Denotes the Linux Kubernetes Worker Node.

Steps for use:

To begin with please follow the guide in our wiki for configuring your terraform environment, which can be found here.

Once you have your terraform environment configured, the next step is to configure this template to point to your Azure Environment.

Create a "secrets.tfvars" file within "sus1"

Create a new file called "secrets.tfvars" and provide the following information as variables on separate lines.

  • ssh_key: This is your ssh public key which will be used for each new machine created in this template.
  • keyvault_tenantid: This is the directory id for the Azure AD tenant where the security principal for keyvault will be created.
  • kube_keyvault_password: This is the password for the security principal that is being created to access keyvault.

Update environment.tfvars for terraform

Go to sus1\environment.tfvars and update the the following information at the top to be relevant to your environment that you wish to deploy to:

  • subscription_id = ""
  • environment_code = "s"
  • deployment_code = "us1"
  • location_code = ""
  • azure_location = ""
  • azure_network_octets = "10.8"
  • name_servers = ["8.8.8.8","16.16.16.16"]

Update Packer Configuration

Go to packer\ubuntu\sus1.json and update this for the information relevant to your azure environment.

{ "deployment_code": "us1", "subscription_id": "", "location": "", "cloud_environment_name": "", "resource_group": "SUS1AZ1-Packer" }

Open your VS Code Terminal, and login for Azure

Go to VS Code, and execute the following command to login to azure. Make sure your terminal is set to leverage bash.

az login

If you are logging into Azure Government, execute the following commands

az cloud set --name AzureUSGovernment
az login

Create the packer image

The next step is to create the packer images for the vm, which can be done using the following commands, in the terminal:

cd packer
cd ubuntu
packer build -var-file sus1.json ubuntu.json

This will take a while to execute but will ultimately create an image in a resource group for use when creating your virtual machines.

Executing Terraform Template

First, go into sus1\environment.tfvars and increase the count of lkma to 1. This will tell it to create one master.

Then execute the terraform template, you need to return to the top level and execute the following.

./tf-deploy.sh -c azure -e sus1 -a apply -t azurerm_key_vault.kub-kv

This will create the key-vault required, it must be present before the master can be created.

After that, run the following command to perform a "plan" for the rest of the terraform template.

Note: The keyvault command also created a service principal that will need at minimum "read" rights on the target subscription. Make sure you grant these permissions

./tf-deploy.sh -c azure -e sus1 -a plan

This will generate the plan for your review. Once you are comfortable with this plan.

./tf-deploy.sh -c azure -e sus1 -a apply

This will apply the template and create the master node.

Next go into sus1\environment.tfvars and increase the count of lkwn to the desired number of worker nodes. This will create the vms required.

./tf-deploy.sh -c azure -e sus1 -a apply

This will finish the deployment of the nodes.

Confirming the cluster

You then can access your lkma server using SSH and the uadmin username, and check that your cluster is working by executing the following commands. This will grant access to uadmin to the kubernetes apis.

sudo mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown uadmin:uadmin $HOME/.kube/config

Execute the following command to see the nodes in the cluster and their status:

sudo kubectl get nodes

Common issue = File Encoding

One common issue people have with this template is that they do not make sure the .sh files are encoded with "LF" instead of "CRLF". This is important because of how linux coding works. Make sure you change the encoding before executing the template if it appears differently on your machine.

Useful Links:

Clone this wiki locally