Configuring the Data Plane Development Kit (DPDK) Integrated with Contrail vRouter

DPDK Support in Contrail​

Contrail Networking supports the Data Plane Development Kit (DPDK). DPDK is an open source set of libraries and drivers for fast packet processing. DPDK enables fast packet processing by allowing network interface cards (NICs) to send direct memory access (DMA) packets directly into an application’s address space, allowing the application to poll for packets, and thereby avoiding the overhead of interrupts from the NIC.

Integrating with DPDK allows a Contrail vRouter to process more packets per second than is possible when running as a kernel module.

In Contrail Networking, before you use DPDK the DPDK configuration should be defined in instances.yaml for ansible based provision, or in host.yaml for helm-based provision. The AGENT_MODE configuration specifies whether the hypervisor is provisioned in the DPDK mode of operation.

When a Contrail compute node is provisioned with DPDK, the corresponding yaml file specifies the number of CPU cores to use for forwarding packets, the number of huge pages to allocate for DPDK, and the UIO driver to use for DPDK.

Preparing the Environment File for Provisioning a Cluster Node with DPDK

The environment file is used at provisioning to specify all of the options necessary for the installation of a Contrail cluster, including whether any node should be configured to use DPDK.

Each node to be configured with the DPDK vRouter must be listed in the provisioning file with a dictionary entry, along with the percentage of memory for DPDK huge pages and the CPUs to be used.

The following are descriptions of the required entries for the server configuration. :

• HUGE_PAGES—Specify the percentage of host memory to be reserved for the DPDK huge pages. The reserved memory will be used by the vRouter and the Quick Emulator (QEMU) for allocating memory resources for the virtual machines (VMs) spawned on that host.

  Note:

  The percentage allocated to HUGE_PAGES should not be too high, because the host Linux kernel also requires memory.

  When using huge pages with DPDK, ensure that:

  • By default, 2MB huge pages are provisioned.

  • If 1GB huge pages are required for your node, set the 1GB huge page setting to your preference and disable 2MB hugepages. We strongly recommend setting the 2M huge page setting to 0 instead of deleting the variable to ensure the node does not attempt to use 2M hugepages.

  • Huge pages are set differently depending on environment, but you can accomplish setting the 2M huge pages to 0 in most environments by setting 2M huge pages to 0 in the ContrailDpdkParameters hierarchy within the contrail_services.yaml file.

• CPU_CORE_MASK—Specify a CPU affinity mask with which vRouter will run. vRouter will use only the CPUs specified for its threads of execution. These CPU cores will be constantly polling for packets, and they will be displayed as 100% busy in the output of “top”.

  The supported format is hexadecimal (for example, 0xf).

• DPDK_UIO_DRIVER—Specify the UIO driver to be used with DPDK.

  The supported values include:

  • vfio-pci—Specify that the vfio module in the Linux kernel should be used instead of uio. The vfio module protects memory access using the IOMMU when a SR-IOV virtual function is used as the physical interface of vrouter.

  • uio_pci_generic—Specify that the UIO driver built into the Linux kernel should be used. This option does not support the use of SR-IOV VFs. This is the default option if DPDK_UIO_DRIVER is not specified.

  • mlnx – For Mellanox ConnectX-4 and ConnectX-5 NICs.

Use the standard Ansible or helm-based provision procedure. Upon completion, your cluster, with specified nodes using the DPDK vRouter implementation, is ready to use.

Sample configuration in instances.yml for ansible-based provision

Sample configuration in host.yml for helm-based provision

Creating a Flavor and Understanding Huge Pages for DPDK

To launch a VM in a DPDK-enabled vRouter hypervisor, the flavor for the VM should be set to use huge pages. The use of huge pages is a requirement for using a DPDK vRouter.

Use the following command to add the flavor, where m1.large is the name of the flavor. When a VM is created using this flavor, OpenStack ensures that the VM will only be spawned on a compute node that has huge pages enabled.

Huge pages are enabled for compute nodes where vRouter is provisioned with DPDK.

If a VM is spawned with a flavor that does not have huge pages enabled, the VM should not be created on a compute node on which vRouter is provisioned with DPDK.

You can use OpenStack availability zones or host aggregates to exclude the hosts where vRouter is provisioned with DPDK.

Note the following when using huge pages with DPDK:

• The size of the flow table on your node should dictate the size of your huge page configuration. Use 1GB huge pages in environments with more flow table entries.

• By default, 2MB huge pages are provisioned.

• If 1GB huge pages are required for your node, set the 1GB huge page setting to your preference and disable 2M hugepages. We strongly recommend setting the 2M huge page setting to 0 instead of deleting the variable to ensure the environment does not attempt to use 2M hugepages.

  Huge pages are set differently depending on environment, but you can accomplish setting the 2M huge pages to 0 in most environments by setting 2M huge pages to 0 in the ContrailDpdkParameters hierarchy within the contrail_services.yaml file.

Configuring and Verifying MTU for DPDK vRouter

This section describes how you configure the maximum transmission unit (MTU) for DPDK vRouter. To set MTU, you need to specify the desired value for mtu in the contrail_vrouter_dpdk_bond.yaml file.

You can verify the configured value from hypervisor by running the following command:

You can use the vif -g or vif --get command to view the status of the bond interfaces in a DPDK vRouter.

For example,