Skip to content

Latest commit

 

History

History
161 lines (107 loc) · 11 KB

CONTRIBUTING.md

File metadata and controls

161 lines (107 loc) · 11 KB
Raw

Contributing on Windows Node support

Contains a list of common resources when contributing in the effort to support Windows Node and Windows Server containers in Kubernetes.

Joining the community of other contributors

The best way to get in contact with the contributors working on Windows support is through the Kubernetes Slack. To get a Slack invite, visit http://slack.k8s.io/ . Once you're logged in, join us in the #SIG-Windows channel.

To get access to shared documents, meeting calendar, and additional discussions, be sure to also join the SIG-Windows Google Group.

View the leadership team in SIG-Windows and other subprojects in the getting started guide.

Find work in progress

View the SIG-Windows project boards:

Items are added to these boards automatically and these boards are reviewed during our bi-weekly backlog refinement meeting.

Building Kubernetes for Windows from Source

The Kubernetes build scripts have not been ported to Windows, so it's best to develop in a Linux VM or WSL2 environment where you can run the same Docker container used in the official Kubernetes builds.

It's best to read through the Building Kubernetes guide if you have never built Kubernetes before to get the latest info. These steps are a summary focused on cross-building the Windows node binaries (kubectl, kubelet, and kube-proxy).

Build Prerequisites

At least 60GB of disk space is required, and 16GB of memory (or memory + swap).

In your developer environment, install Git, Docker-CE, and make. The build scripts will pull a Docker container with the required version of golang and other needed tools preinstalled.

If you're using Ubuntu, then install the following packages: git, build-essential, Docker-CE.

Building Kubernetes binaries for Windows

You can build individual components such as kubelet, kube-proxy, or kubectl by running ./build/run.sh make <binary name> KUBE_BUILD_PLATFORMS=windows/amd64 such as ./build/run.sh make kubelet KUBE_BUILD_PLATFORMS=windows/amd64

If you would like to build all binaries at once, then run ./build/run.sh make cross KUBE_BUILD_PLATFORMS=windows/amd64

Once the build completes, the files will be in _output/dockerized/bin.

Testing your changes

Build a local cluster

The easist way to test changes is to use the community maintained SIG Windows developer environment project. You can use this to build a fully functional cluster locally with binaries built from source.

Updating the Node binaries

If you already have an existing cluster you also test your changes by swapping the binaries on a node. To update the binaries on an existing node, follow these steps:

  1. Drain & cordon a node with kubectl drain <nodename>.
  2. Connect to the node with SSH or Windows Remote Desktop, and start PowerShell
  3. On the node, run Stop-Service kube-proxy -Force followed by Stop-Service kubelet -Force.
  4. Copy kubelet.exe and kube-proxy.exe to the node.
  5. Overwrite the existing kubelet & kube-proxy binaries. If you don't know where they are, run sc.exe qc kubelet or sc.exe qc kube-proxy and look at the BINARY_PATH_NAME returned.
  6. Start the updated kubelet & kube-proxy with Start-Service kubelet.

Creating a PR

Congratulations on contributing to the SIG-Windows ecosystem. If there is a PR you would like to build, it's easy. You can create a working branch, pull the changes from GitHub in a patch, apply, then build.

Please follow the Kubernetes contribute guide for pull-requests when submitting your changes.

Additionally please perform the following:

  1. Add the sig/windows label to your PR by adding /sig windows in the pull request description or a comment.
  2. Trigger windows specific e2e test by adding /test pull-kubernetes-e2e-aks-engine-windows-containerd in a comment.

API Considerations

If you modifying an API in the SIG-Windows codebase, make sure you are aware of the API guidelines and conventions used in Kubernetes. This document offers guidelines for API reviewers that API developers should always have in consideration.

Running Tests

For the most up-to-date steps on how to build and run tests, please go to https://github.com/kubernetes-sigs/windows-testing. It has everything you need to build and run tests, as well as links to the SIG-Windows configurations used on TestGrid.

Specifically, you need to build the e2e test binary, using https://github.com/kubernetes-sigs/windows-testing/blob/master/README.md#how-do-i-build-the-e2etest-binary, in order to run these tests.

Troubleshooting

Intro

If you are having issues with network dependent services coming up with your Windows based container there is a workaround that may help.

We can use the Container Lifecycle Hooks to help ensure the service starts after the network is available. Specifically we will be using the PostStart hook.

Examples

The first example will execute after the pod is up and in this particular case will restart the dbconnect service once dbhost.example.com can be resolved via DNS. This command could also be modified to require said host to be reachable before exiting.

        lifecycle:
          postStart:
            exec:
              command: ["powershell.exe","-command","do { $Result = @(ping -n 1 dbhost.example.com) } while ( $Result -notcontains 'Approximate round trip times in milli-seconds:' ); Restart-Service -Name dbconnect"]

This second example is used in pods where GMSA is being used. This will restart the netlogon service until we can affirm that the pod has logged on to the domain correctly.

        lifecycle:
          postStart:
            exec:
              command: ["powershell.exe","-command","do { Restart-Service -Name netlogon } while ( $($Result = (nltest.exe /query); if ($Result -like '*0x0 NERR_Success*') {return $true} else {return $false}) -eq $false)"]

Reporting Issues and Feature Requests

If you have what looks like a bug, or you would like to make a feature request, please use the Github issue tracking system. You can open issues on GitHub and assign them to SIG-Windows. You should first search the list of issues in case it was reported previously and comment with your experience on the issue and add additional logs. SIG-Windows Slack is also a great avenue to get some initial support and troubleshooting ideas prior to creating a ticket.

If filing a bug, please include detailed information about how to reproduce the problem, such as:

  • Kubernetes version: kubectl version
  • Environment details: Cloud provider, OS distro, networking choice and configuration, and Docker version
  • Detailed steps to reproduce the problem
  • Relevant logs
  • Tag the issue sig/windows by commenting on the issue with /sig windows to bring it to a SIG-Windows member's attention

Gathering Logs

Logs are an important element of troubleshooting issues in Kubernetes. Make sure to include them any time you seek troubleshooting assistance from other contributors.

Collecting kubelet and kube-proxy Logs

There are a few different ways to run the node binaries (kubelet, kube-proxy, etc) and the execution method also dictates how logs will be collected. Issue 75319 tracks pending work for better log management on Windows (for example using the Windows Event Log for higher log throughput and log rotation). If you end up logging to a file, you can use fluentd or Splunk to ship logs to a syslog server for search and analytics.

  1. Windows Service Manager services - You may have to introduce your own log consumption and log rotation service if you are logging to a file. We will investigate if journald is an option
  2. nssm.exe services - nssm.exe provides support for forwarding stdout/stderr logs to a file (use the AppStdout and AppStderr options) and also supports logs file rotation (See the File rotation and I/O redirection sections in the documentation). You can see more examples on using nssm.exe for the Kubernetes components of the Windows node in the services and background processes section under troubleshooting
# Example nssm command line for the kubelet
nssm set kubelet AppStdout C:\k\kubelet.log
nssm set kubelet AppStderr C:\k\kubelet.log

Collecting Networking Logs

  1. On the node before creating the pod for the first time.
  2. start-bitstransfer https://raw.githubusercontent.com/Microsoft/SDN/master/Kubernetes/windows/debug/collectlogs.ps1
  3. Execute collectlogs.ps1 in a PowerShell window 4 Start the trace by running C:\k\debug\starthnstrace.cmd
  4. Reproduce the issue
  5. Run netsh trace stop
  6. Execute collectlogs.ps1 in a PowerShell window again
  7. Include in your ticket C:\server.etl