FAQ¶

Q: Is there any existing open source project with similar goals or with which this project may conflict or overlap? If yes, please identify the project and explain why we should create the ARM open source project anyway?

A: Yes, there are some open source projects serve similar purpose, e.g., OPNFV. However, compared with the software we would like to create, OPNFV has a much larger scope for us to cover. Moreover, we would like to simplify the reference solutions, focus on combining some currently being contributed networking projects, e.g., DPDK, VPP, ODP, and provide reference solutions aligned with existing marketing requirements.

Inside the project, we will apply some optimizations applicable on Arm platform only, to achieve better performance, and those architecture specific optimizations will not be likely accepted by upstream community.
Q: Could you please provide the project description of functionality or purpose?

A: The network functions this software provided serves multiple purposes of,
1. Showcase the integration of various components and act as poof of concept to all stakeholders.
2. Allow for performance analysis/optimization with a solution that is close to customers’ production deployment.
3. Provide customers with a out-of-the-box reference design for rapid design modeling.

Q: How to install Mellanox ConnectX-5 OFED driver and update NIC firmware?

A: To use Mellanox NIC, firstly install the OFED driver MLNX_OFED, and then update NIC Firmware.

The key steps are:

Download the OFED driver

Install OFED driver:

$ sudo mount -o ro,loop MLNX_OFED_LINUX-5.4-3.1.0.0-ubuntu20.04-aarch64.iso /mnt
$ sudo /mnt/mlnxofedinstall --upstream-libs --dpdk

Update firmware after OFED installation:

$ wget https://www.mellanox.com/downloads/MFT/mft-4.20.0-34-arm64-deb.tgz
$ tar xvf mft-4.20.0-34-arm64-deb.tgz
$ cd mft-4.20.0-34-arm64-deb/
$ sudo ./install.sh
$ sudo mst start
$ sudo mlxfwmanager --online -u -d <device PCIe address>

Q: How to enable SR-IOV VFs for Mellanox ConnectX-5?

A: First, ensure the OFED driver and firmware tools are installed by following the steps above. Next, use this guide to enable SR-IOV on your platform.

The key steps are:
- Enable SR-IOV in BIOS settings
- Enable SR-IOV on NIC firmware and set maximum number of VFs to a non-zero number:
```
$ sudo mst status # lists NIC under "MST Devices"
$ sudo mlxconfig -d /dev/mst/mt4121_pciconf0 set SRIOV_EN=True NUM_OF_VFS=4 # replace /dev/mst/... with your MST Device
```
- Reboot the machine
Q: How to update firmware for Intel 700 Series NICs?

A: To update the NIC firmware, first install the NIC on an x86 host. Then, install the NVM Update Utility on the x86 host and follow the included readme.

The key steps are:
- Insert the NIC into an x86 host
- Download the NVM Update Utility
- Unzip the package & extract the Linux version:
```
$ unzip 700Series_NVMUpdatePackage_v8_70.zip
$ tar xvf 700Series_NVMUpdatePackage_v8_70_Linux.tar.gz
```
- Run the NVM Update Utility:
```
$ cd 700Series/Linux_x64/
$ sudo ./nvmupdate64e
```
- Press A to update all NICs, or specify a list of cards to update
- Reboot the machine

Q: How to setup SSH keys?

A: SSH keys can be setup on Ubuntu 20.04 by following this guide.

The key steps are:

Check whether SSH key already exists in ~/.ssh/id_rsa.pub. If not, create new authentication key pairs for SSH:

$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/<remote_user>/.ssh/id_rsa):
Created directory '/home/<remote_user>/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/<remote_user>/.ssh/id_rsa
Your public key has been saved in /home/<remote_user>/.ssh/id_rsa.pub

Copy SSH public key to Arm machine. remote_user is the user name on Arm machine and dut.arm.com is the FQDN of Arm machine:
```
$ ssh-copy-id [email protected]
```
Make sure remote_user can login Arm machine from management node without entering password:
```
$ ssh [email protected]
```

Q: Can Ansible use SSH keys protected with a passphrase?

A: Yes, by using the ssh-agent utility. Follow these steps to set it up.

Q: How to setup DUT with isolated CPUs and 1G hugepages?

The key steps are:

Edit variable GRUB_CMDLINE_LINUX_DEFAULT in file /etc/default/grub to reserve hugepages using parameters default_hugepagesz=1G hugepagesz=1G hugepages=4 and reserve isolated CPU cores using parameters isolcpus=1-3 nohz_full=1-3 rcu_nocbs=1-3 cpuidle.off=1 cpufreq.off=1
```
$ grep GRUB_CMDLINE_LINUX_DEFAULT /etc/default/grub | grep -v "#"
GRUB_CMDLINE_LINUX_DEFAULT="default_hugepagesz=1G hugepagesz=1G hugepages=4 iommu.passthrough=1 isolcpus=1-3 nohz_full=1-3 rcu_nocbs=1-3 cpuidle.off=1 cpufreq.off=1"
```
Enable above kernel parameters by executing sudo update-grub and rebooting the system:
```
$ sudo update-grub
$ sudo reboot now
```

Make sure kernel parameters take effect after reboot

$ cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-5.19.10-051910-generic-64k root=/dev/mapper/ubuntu--vg-ubuntu--lv ro default_hugepagesz=1G hugepagesz=1G hugepages=4 iommu.passthrough=1 isolcpus=1-3 nohz_full=1-3 rcu_nocbs=1-3 cpuidle.off=1 cpufreq.off=1
$ grep Huge /proc/meminfo
HugePages_Total:       4
HugePages_Free:        4
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:    1048576 kB
$ grep Cpus_allowed_list /proc/self/status
Cpus_allowed_list:     0,4-191

Q: Why isolate CPUs with these parameters?

A: The combination of the kernel parameters mentioned above ensure that the CPUs run only the desired application. They never process kernel RCU callbacks (rcu_nocbs), don’t generate scheduling ticks (nohz_full) when there is only one process running on them, and are isolated from running any processes other than the ones pinned to them (isolcpus). Additionally, the cpufreq and cpuidle subsystems are disabled.
Q: How do I make python3 -c 'import socket; print(socket.getfqdn())' show the FQDN?

A: If the above python command fails to print out the correct FQDN, then your system may be affected by this python bug. The solution is to modify /etc/hosts to hardcode the correct FQDN. To do so, follow the steps in the Debian manual to modify /etc/hosts.

Q: How to build HTML rendered document?

A: Install following dependencies and run make doc:

sudo apt-get update
sudo apt-get install python3-venv libenchant-dev -y
cd <nw_cra_workspace>/cnf-reference-arch
make doc

Q: How to resolve create VF task failure echo 2 > /sys/bus/pci/devices/<pcie_addr>/sriov_numvfs ... echo: I/O error?

A: CRA is most easily deployed with exclusive control over a PF. Ensure no other applications like DPDK or QEMU are using the PF. Additionally, ensure the PF is bound to the default kernel driver such as i40e or mlx5_core.
1. Check NIC port is used by other applications like QEMU, DPDK, stop them if possible.
2. Examine NIC is bind with vfio-pci driver, this can be done with driverctl -v list-devices | grep vfio.
3. Bind with NIC’s default driver, take i40e as example, bind with driverctl set-override <pcie_addr> i40e.
4. Re-deploy CRA solution.
CRA can also be deployed onto specific VFs on a machine. This can be used to share underlying PFs with other applications like DPDK or QEMU. To leverage this deployment model, specify the VF PCIe addresses in the pcie_addr for the worker node, and set the pcie_addr_is_vf flag. See the user guide for more information.