multi-networkpolicy implementation using Linux Traffic Control (TC)
Kubernetes provides Network Policies for network security. MultiNetworkPolicy defines an API similar to Kubernetes built-in NetworkPolicy API for secondary kubernetes networks defined via NetworkAttachmentDefinition CRD. multi-networkpolicy-tc implements MultiNetworkPolicy API using Linux TC, providing network security for net-attach-def networks.
multi-networkpolicy-tc is intended to be used with networks provided via accelerated bridge cni. it is currently not compatible with other CNIs however support may be extended for additional CNIs.
multi-networkpolicy-tc relies on the fact that a Pod has an SRIOV VF allocated for the network with a corresponding VF representor netdev which follows the kernel switchdev model.
given a MultiNetworkPolicy it generates and programs TC rules to enforce the policy.
for more information refer to docs/tc-rule-pipeline.md.
- Linux kernel 5.17.9 or newer
- NIC supporting switchdev and TC hardware offload such as:
- Nvidia Mellanox ConnectX-6Dx
This project uses go modules for dependency management and requires Go 1.18 to build.
to build binary run:
$ make buildBinary executable is located under build folder
Install MultiNetworkPolicy CRD into Kubernetes.
$ git clone https://github.com/k8snetworkplumbingwg/multi-networkpolicy-tc
$ cd multi-networkpolicy-tc
$ kubectl create -f deploy/crds/multi-net-crd.yaml
customresourcedefinition.apiextensions.k8s.io/multi-networkpolicies.k8s.cni.cncf.io created
Deploy multi-networkpolicy-tc into Kubernetes.
$ git clone https://github.com/k8snetworkplumbingwg/multi-networkpolicy-tc
$ cd multi-networkpolicy-tc
$ kubectl create -f deploy/deploy.yml
clusterrole.rbac.authorization.k8s.io/multi-networkpolicy created
clusterrolebinding.rbac.authorization.k8s.io/multi-networkpolicy created
serviceaccount/multi-networkpolicy created
daemonset.apps/multi-networkpolicy-ds-amd64 created
multi-network-policy-tc runs as a daemonset on each node.
multi-networkpolicy-tc watches MultiNetworkPolicy object and creates TC rules on VF representor to filters packets
to/from interface, based on MultiNetworkPolicy.
The following configuration flags are supported by multi-networkpolicy-tc:
--kubeconfig string Path to kubeconfig file with authorization information (the master location is set by the master flag).
--master string The address of the Kubernetes API server (overrides any value in kubeconfig)
--hostname-override string If non-empty, will use this string as identification instead of the actual hostname.
--network-plugins strings List of network plugins to be be considered for network policies. (default [accelerated-bridge])
--pod-rules-path string If non-empty, will use this path to store pod's rules for troubleshooting.
--tc-driver string TC driver to use for interacting with linux Traffic Class subsystem. [cmdline, netlink]. (default "cmdline")
--add_dir_header If true, adds the file directory to the header of the log messages
--alsologtostderr log to standard error as well as files (no effect when -logtostderr=true)
--log_backtrace_at traceLocation when logging hits line file:N, emit a stack trace (default :0)
--log_dir string If non-empty, write log files in this directory (no effect when -logtostderr=true)
--log_file string If non-empty, use this log file (no effect when -logtostderr=true)
--log_file_max_size uint Defines the maximum size a log file can grow to (no effect when -logtostderr=true). Unit is megabytes. If the value is 0, the maximum file size is unlimited. (default 1800)
--logtostderr log to standard error instead of files (default true)
--one_output If true, only write logs to their native severity level (vs also writing to each lower severity level; no effect when -logtostderr=true)
--skip_headers If true, avoid header prefixes in the log messages
--skip_log_headers If true, avoid headers when opening log files (no effect when -logtostderr=true)
--stderrthreshold severity logs at or above this threshold go to stderr when writing to files and stderr (no effect when -logtostderr=true or -alsologtostderr=false) (default 2)
-v, --v Level number for the log level verbosity
--vmodule moduleSpec comma-separated list of pattern=N settings for file-filtered logging
--log-flush-frequency duration Maximum number of seconds between log flushes (default 5s)
-h, --help help for multi-networkpolicy-tc
As this project is under active development, there are several limitations which are planned to be addressed in the near future.
- MultiNetworkPolicy Ingress rules are not supported. Ingress policy will not be enforced
- QinQ traffic is not supported network policy will not be enforced
To report a bug or request a feature, open an issue in this repository.
to contribute to the project please refer to CONTRIBUTING.md doc