Rate and give feedback:  Feedback Received. Thank You!

Rate and give feedback: 

X

This document helped resolve my issue

Yes No

Additional Comments

800 characters remaining

May we contact you if necessary?

Name:  

E-mail: 

Enter a valid Email ID

Need product assistance? Contact Juniper Support

Submit

This site is protected by hCaptcha and its Privacy Policy and Terms of Service apply.

English  

English

Configuring the Data Plane Development Kit (DPDK) Integrated with Contrail vRouter

DPDK Support in Contrail​

Contrail 3.0 and later supports the Data Plane Development Kit (DPDK) on Ubuntu systems, only.

DPDK is an open source set of libraries and drivers for fast packet processing. DPDK enables fast packet processing by allowing network interface cards (NICs) to send direct memory access (DMA) packets directly into an application’s address space, allowing the application to poll for packets, and thereby avoiding the overhead of interrupts from the NIC.

Integrating with DPDK allows a Contrail vRouter to process more packets per second than is possible when running as a kernel module.

When using DPDK with Contrail, one or more Contrail compute nodes are provisioned with DPDK during installation. An entry in the testbed.py file specifies which nodes are to be configured to use the DPDK vRouter mode instead of the regular kernel mode. This allows for a mixed setup, where different nodes use different modes of the vRouter.

Upon installation, when a Contrail compute node is provisioned with DPDK, the testbed.py file specifies which physical interface(s) to use, how many CPU cores to use for forwarding packets, and the number of huge pages to allocate for DPDK.

Preparing the testbed.py File for Provisioning a Contrail Cluster Node with DPDK

The testbed.py file is a Python file that is configured to specify all of the options necessary for the installation of a Contrail cluster, including whether any node should be configured to use DPDK.

An env.dpdk entry in the testbed.py file is used to specify which node(s) (hosts) will be configured upon installation to use the DPDK vRouter mode instead of the regular kernel module.

Each node to be configured with the DPDK vRouter must be listed in the env.dpdk dictionary entry, along with the percentage of memory for DPDK huge pages and the CPUs to be used, as in the following example:

env.dpdk = {
    host1: { 'huge_pages' : '50', 'coremask' : '0xf'},
    host2: { 'huge_pages' : '50', 'coremask' : '0xf'},
}

• hostx—Specify the numerical name of each node host (host1, host2, host3, etc.) that is to be configured with a DPDK vRouter. Any nodes not listed here will use the kernel vRouter module.
• huge_pages—Specify the percentage of host memory to be reserved for the DPDK huge pages. The reserved memory will be used by the vRouter and the Quick Emulator (QEMU) for allocating memory resources for the virtual machines (VMs) spawned on that host.

  Note: The percentage allocated to huge_pages should not be too high, because the host Linux kernel also requires memory.

• coremask—Specify a CPU affinity mask with which vRouter will run. vRouter will use only the CPUs specified for its threads of execution. These CPU cores will be constantly polling for packets, and they will be displayed as 100% busy in the output of “top”.

  Supported formats include:

  • Hexadecimal (for example, 0xf)
  • Comma-separated list of CPUs (1,2,4...)
  • Dash-separated range of CPUs (for example, 1-4)

The testbed.py file is configured prior to the installation of Contrail.

Use the standard Contrail installation procedure with fab tools, and upon completion, your cluster with specified nodes using the DPDK vRouter implementation is ready to use.

Support for Multiple UIO Drivers

Starting with Contrail Release 3.2, support for optionally specifying the userspace IO (UIO) driver to use in a DPDK-enabled compute node is available.

Specify UIO in the env.dpdk section of the testbed file, by using the optional attribute uio_driver:

env.dpdk = {
  host1:{'huge_pages':'50','coremask':'0xf','uio_driver':’igb_uio’}, 
  host2:{'huge_pages':'50','coremask':'3-7','uio_driver':'vfio-pci'}
}

• igb_uio—specify that the igb_uio module from the DPDK library should be used.
• vfio-pci—specify that the vfio module in the Linux kernel should be used instead of uio, which protects memory accesses using the IOMMU when a SR-IOV virtual function is used as the physical interface of vrouter.
• uio_pci_generic—specify that the UIO driver that is built into the Linux kernel should be used. This option does not support the use of SR-IOV VFs.

If the uio_driver is not specified in the testbed file, igb_uio is used by default.

Provisioning a Contrail Cluster with DPDK

Use the same process for provisioning a regular kernel vRouter module using fab tools, Server Manager, or Server Manager Lite to provision a cluster with DPDK vRouter mode.

The DPDK nodes are specified in the testbed.py before cluster installation with fab tools.

1. Copy the contrail-install-packages*.deb file into the host that has been configured as a host_build in your testbed.py file.
2. Log in to the host_build node and install the package (assuming it is in your $HOME directory).

   # dpkg -i ~/contrail-install-packages_3.0-*~kilo_all.deb

3. Run the setup.sh script from /opt/contrail/contrail_packages.

   # cd /opt/contrail/contrail_packages/

   # ./setup.sh

4. Copy your testbed.py file into the /opt/contrail/utils/fabfile/testbeds/ directory.
5. From the /opt/contrail/utils directory, issue the following fab commands:

   # cd /opt/contrail/utils

   # fab install_pkg_all:~/contrail-install-packages_3.0-*~icehouse_all.deb

   # fab upgrade_kernel_all

   # fab install_contrail

   # fab setup_interface

   # fab setup_all

Upon completion, the cluster with nodes using the DPDK vRouter implementation is ready to use.

Creating a Flavor for DPDK in OpenStack Kilo

OpenStack Kilo has a feature called flavors, which are virtual hardware templates that define sizes for RAM, disk, and so on. Contrail 3.0 and later supports the OpenStack Kilo flavor that specifies that a VM should use huge pages. The use of huge pages is a requirement for using a DPDK vRouter.

Use the following command to add the flavor, where m1.large is the name of the flavor. When a VM is created using this flavor, OpenStack ensures that the VM will only be spawned on a compute node that has huge pages enabled.

$nova flavor-key m1.large set hw:mem_page_size=large  

Huge pages are enabled for compute nodes where vRouter is provisioned with DPDK.

If a VM is spawned with a flavor that does not have huge pages enabled, the VM should not be created on a compute node on which vRouter is provisioned with DPDK.

You can use OpenStack availability zones or host aggregates to exclude the hosts where vRouter is provisioned with DPDK.