"device or resource busy" error from netlink when creating macvlan device

[dudeisbrendan03]

Hi there,
I have multiple NetworkAttachmentDefinitions, mostly using IPVLAN L2 bridges.

I tried to convert one of my IPVLAN bridges to MACVLAN, as there is a workload that functions as a gateway for another workload that terminates some tunnelled traffic.

Issue is I need to utilise MACVLAN for L2 routing at the gateway, when I changed it from IPVLAN/L2 to MACVLAN/Bridge I got the error "device or resource busy" from netlink when trying to attack the interface, I assumed this was because I modified an existing NAD- so I deleted and recreated it as an ipvlan resource.

apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
  name: br-n6
  namespace: phoenix
spec:
  config: >-
    {"cniVersion":"0.3.1","type":"ipvlan","master":"ens20","mode":"l2","ipam":{"type":"host-local","ranges":[[{"subnet":"192.168.15.0/24","rangeStart":"192.168.15.10","rangeEnd":"192.168.15.253","gateway":"192.168.15.1"}]]}}

apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
  name: br-n6
  namespace: phoenix
spec:
  config: >-
    {"cniVersion":"0.3.1","type":"macvlan","master":"ens20","mode":"bridge","ipam":{"type":"host-local","ranges":[[{"subnet":"192.168.15.0/24","rangeStart":"192.168.15.10","rangeEnd":"192.168.15.253","gateway":"192.168.15.1"}]]}}

This did not resolve the issue, and I am still unable to create any MACVLAN devices in my cluster.

Each node in the cluster is identical, including interface names and the networks behind them.

I'm wondering if modifying the resource meant that something wasn't cleaned up properly, or is there a restriction on utilising different types of bridges (since existing IPVLAN bridges exist on the same interface).

Attempting to create a macvlan device myself on any of the nodes in the cluster results in the same issue

❯ sudo ip link add testmacvlan1 link ens20 type macvlan mode bridge
RTNETLINK answers: Device or resource busy

[dougbtv]

RTNETLINK answers: Device or resource busy

This is an issue on the host side, that the netlink api is complaining about something going on with the underlying devices (virtual or otherwise) -- so, likely it's some incompability there, naturally the netlink replies are often.... vague kind of.

You might also file an issue to the CNI plugins repo, e.g. for macvlan/ipvlan