No firmware-level SRIOV enablement for e810-based NICs

[iancoolidge]

In the Mellanox plugin, if the CX6 firmware does not have VirtualizationMode=SRIOV and NumVFSAdvertised != 0, the mellanox glue code will call 'mstconfig' to set the firmware to enable SRIOV.

However, for the Intel e810 case, no such firmware settings management happens -- the SRIOV network operator just fails to detect SRIOV support.

[adrianchiris]

@iancoolidge does the NIC in question (device ID) appear in [1] ?

[1] https://github.com/k8snetworkplumbingwg/sriov-network-operator/blob/master/doc/supported-hardware.md

[iancoolidge]

Yes,

each of

Intel E810-CQDA2/2CQDA2 Family | 8086 | 1592
Intel E810-XXVDA4 Family | 8086 | 1593
Intel E810-XXVDA2 Family | 8086 | 159b

are supported and applicable here, and have a similar firmware control for enabling SRIOV

[SchSeba]

Hi @iancoolidge for intel there is no need to configure anything in the firmware that I am aware of.

please run lspci -vvv -s <PF-pci> and check if the sriov capability exist and also print the cmdline. you can also check your bios configuration

[iancoolidge]

@SchSeba lspci simply shows that the firmware-level setting is enabled, and one such setting does exist for e810-based cards.

In the Mellanox case, prior to invocing mstconfig, no SRIOV support exists in lspci, but the operator handles that by calling mstconfig.

For e810, on the servers that I am using, you can enable SRIOV through BIOS option, but maybe we need support from Intel to add this feature to their 'ethernet port configuration tool' or something.

[SchSeba]

Hi @iancoolidge I don't think e810 as the same case as MLX cards.

@Eoghan1232 can you please check this issue?

[Eoghan1232]

Hi @iancoolidge @SchSeba

Just catching up, the ethernet port configuration tool offers the configuration ability for SIOV not SRIOV.

You can use Intel's Ethernet Port Configuration Tool (EPCT) to enable Intel
Scalable IOV. If the EPCT tool is not available, you can also enable Intel
Scalable IOV through your system's HII interface (if it has one). 

As for SRIOV capability of the E800 series NICs, you would simply enable 2 BIOS settings:

Intel(R) VT for Directed I/O
Intel(R) Virtualization Technology

Other than that, then the device is SRIOV capable, and requires no firmware configuration.
You would then use sysfs to enable virtual functions (VF).

 echo 4 > /sys/class/net/<PF_Dev>/device/sriov_numvfs

[iancoolidge]

thanks @Eoghan1232 -- that BIOS configuration is exactly what I'm after here.

With MLX, you can configure the NIC firmware in a system BIOS interface, or with the mstconfig tool. sriov-network-operator uses the mstconfig tool.

Are there any plans to expose this for e810 cards in a command line tool similarly? That would let sriov-network-operator have feature parity between MLX and e810 (automatic management of firmware settings)

[SchSeba]

Hi @iancoolidge I don't think the bios flag and the mstconfig doing the same thing...

[SchSeba]

btw you can use something like redfish to do with an API but again that redfish API call will not be implemented as part of the operator. as must of the production deployments the out of band management port (what redfish will do the API call) is not connected to the workload/host network to avoid security issues.

[iancoolidge]

Hi @iancoolidge I don't think the bios flag and the mstconfig doing the same thing...

Can you please elaborate?

[iancoolidge]

btw you can use something like redfish to do with an API but again that redfish API call will not be implemented as part of the operator. as must of the production deployments the out of band management port (what redfish will do the API call) is not connected to the workload/host network to avoid security issues.

yes, I agree that is not a good suggestion

[adrianchiris]

Hi @iancoolidge I don't think the bios flag and the mstconfig doing the same thing...

Can you please elaborate?

i believe @iancoolidge means there are some FW parameters exposed in UEFI and can be configured during node boot.
these are not related to the general SRIOV (CPU related?) parameters like Intel Virtualization technology.

[SchSeba]

no new comments closing this issue for housekeeping feel free you reopen if needed