vRouter Command Line Utilities

vRouter is the component that takes packets from VMs and forwards them to their destinations. In this effort, vRouter depends on the vRouter agent to make sense of the overall topology, understand the various policies that govern the communication between VMs, and program them in vRouter in a way vRouter understands.

vRouter has a few fundamental data structures that abstracts out the various communication paths. There is "interface," "flow," "route," and "nexthop" that enables vRouter to push packets to their eventual destinations. In addition, vRouter also has good statistics that can help understand and debug packet paths. Various command line utilities provided by the vRouter can be used to display these data structures and better understand the behavior that one sees in a compute node.

This section describes the shell prompt utilities available for examining the state of the vRouter kernel module in Contrail.

The most useful commands for inspecting the Contrail vRouter module are summarized in the following table.

The following sections describe each of the vRouter utilities in detail.

The vRouter requires vRouter interfaces (vif) to forward traffic. Use the vif command to see the interfaces that are known by the vRouter.

Example: vif --list

vif Options

Use vif –-help to display all options available for the vif command. Following is a brief description of each option.

Contrail Networking Release 2008 supports clearing of vif statistics counters for all interfaces by using the --clear command. For more information on --clear command options, see Table 2.

Use the flow command to display all active flows in a system.

Example: flow -l

Use -l to list everything in the flow table. The -1 is the only relevant debugging option.

​Each record in the flow table listing displays the index of the record, the source IP: source port, the destination ip: destination port, the inet protocol, and the source VRF (V) to which the flow belongs.

Each new flow has to be approved by the vRouter agent. The agent does this by setting actions for each flow. There are three main actions associated with a flow table entry: Forward (‘F’), Drop (‘D’), and Nat (‘N’).

For NAT, there are additional flags indicating the type of NAT to which the flow is subject, including: SNAT (S), DNAT (D), source port translation (Ps), and destination port translation (Pd).

S(nh) indicates the source nexthop index used for the RPF check to validate that the traffic is from a known source. If the packet must go to an ECMP destination, E:X is also displayed, where ‘X’ indicates the destination to be used through the index within the ECMP next hop.

The Statistics field indicates the Packets/Bytes that hit this flow entry.

There is a Mirror Index field if the traffic is mirrored, listing the indices into the mirror table (which can be dumped by using mirror –-dump).

If there is an explicit association between the forward and the reverse flows, as is the case with NAT, you will see a double arrow in each of the records with either side of the arrow displaying the flow index for that direction.

Example: flow -r

Use -r to view all of the flow setup rates.

Example: flow --help

Use --help to display all options available for the flow command.

Use vrfstats to display statistics per next hop for a vrf. It is typically used to determine if packets are hitting the expected next hop.

Example: vrfstats --dump

The —dump option displays the statistics for all VRFs that have seen traffic. In the following example, there was traffic only in Vrf 0 (the public VRF). Receives shows the number of packets that came in the fabric destined to this location. Encaps shows the number of packets destined to the fabric.

If there is VM traffic going out on the fabric, the respective tunnel counters will increment. ​

Example: vrfstats --get 0​

Use --get 0 to retrieve statistics for a particular vrf.

​Example: ​vrfstats --help

Use the rt command to display all routes in a VRF.

Example: rt --dump

The following example displays inet family routes for vrf 0.

In this example output, the first line displays the routing table that is being dumped. In 0/0/unicast, the first 0 is for the router ID, the next 0 is for the VRF ID, and unicast identifies the unicast table. The vRouter maintains separate tables for unicast and multicast routes. ​ By default, if the —table option is not specified, only the unicast table is dumped.

Each record in the table output specifies the destination prefix length, the parent route prefix length from which this route has been expanded, the flags for the route, the MPLS label if the destination is a VM in another location, and the next hop ID. To understand the second field “PPL”, it is good to keep in mind that the unicast routing table is internally implemented as an ‘mtrie’.

The Flags field can have two values. L indicates that the label field is valid, and H indicates that vroute should proxy arp for this IP.

The Nexthop field indicates the next hop ID to which the route points.

Example: rt --dump --table mcst

To dump the multicast table, use the —table option with mcst as the argument.

Use the dropstats command to see packet drop counters in vRouter. Use the dropstats --debug command to view the Cloned Original counters.

Example: dropstats

dropstats ARP Block

GARP packets from VMs are dropped by vRouter, an expected behavior. In the example output, the first counter GARP indicates how many packets were dropped.

ARP requests that are not handled by vRouter are dropped, for example, requests for a system that is not a host. These drops are counted by ARP notme counters.

The Invalid ARPs counter is incremented when the Ethernet protocol is ARP, but the ARP operation was neither a request nor a response.

dropstats Interface Block

Invalid IF counters are incremented normally during transient conditions, and should not be a concern.

Trap No IF counters are incremented when vRouter is not able to find the interface to trap the packets to vRouter agent, and should not happen in a working system.

IF TX Discard and IF RX Discard counters are incremented when vRouter is not in a state to transmit and receive packets, and typically happens when vRouter goes through a reset state or when the module is unloaded.

IF Drop counters indicate packets that are dropped in the interface layer. The increase can typically happen when interface settings are wrong.

dropstats Flow Block

When packets go through flow processing, the first packet in a flow is cached and the vRouter agent is notified so it can take actions on the packet according to the policies configured. If more packets arrive after the first packet but before the agent makes a decision on the first packet, then those new packets are dropped. The dropped packets are tracked by the Flow unusable counter.

The Flow No Memory counter increments when the flow block doesn't have enough memory to perform internal operations.

The Flow Table Full counter increments when the vRouter cannot install a new flow due to lack of available slots. A particular flow can only go in certain slots, and if all those slots are occupied, packets are dropped. It is possible that the flow table is not full, but the counter might increment.

The Flow NAT no rflow counter tracks packets that are dropped when there is no reverse flow associated with a forward flow that had action set as NAT. For NAT, the vRouter needs both forward and reverse flows to be set properly. If they are not set, packets are dropped.

The Flow Action Drop counter tracks packets that are dropped due to policies that prohibit a flow.

The Flow Action Invalid counter usually does not increment in the normal course of time, and can be ignored.

The Flow Invalid Protocol usually does not increment in the normal course of time, and can be ignored.

The Flow Queue Limit Exceeded usually does not increment in the normal course of time, and can be ignored.

dropstats Miscellaneous Operational Block

The Discard counter tracks packets that hit a discard next hop. For various reasons interpreted by the agent and during some transient conditions, a route can point to a discard next hop. When packets hit that route, they are dropped.

The TTL Exceeded counter increments when the MPLS time-to-live goes to zero.

The Mcast Clone Fail happens when the vRouter is not able to replicate a packet for flooding.

The Cloned Original is an internal tracking counter. It is harmless and can be ignored.

The Invalid NH counter tracks the number of packets that hit a next hop that was not in a state to be used (usually in transient conditions) or a next hop that was not expected, or no next hops when there was a next hop expected. Such increments happen rarely, and should not continuously increment.

The Invalid Label counter tracks packets with an MPLS label unusable by vRouter because the value is not in the expected range.

The Invalid Protocol ​typically increments when the IP header is corrupt.

The Rewrite Fail counter tracks the number of times vRouter was not able to write next hop rewrite data to the packet.

The Invalid Mcast Source tracks the multicast packets that came from an unknown or unexpected source and thus were dropped.

The Duplicated counter tracks the number of duplicate packets that are created after dropping the original packets. An original packet is duplicated when generic send offload (GSO) is enabled in the vRouter or the original packet is unable to include the header information of the vRouter agent.

The Invalid Source counter tracks the number of packets that came from an invalid or unexpected source and thus were dropped.

The remaining counters are of value only to developers.

The mpls utility command displays the input label map that has been programmed in the vRouter.

Example: mpls --dump

The —dump command dumps the complete label map. The output is divided into two columns. The first field is the label and the second is the next hop corresponding to the label. When an MPLS packet with the specified label arrives in the vRouter, it uses the next hop corresponding to the label to forward the packet.

You can inspect the operation on nh 9 as follows:

The nh output shows that the next hop directs the packet to go out on the interface with index 3 (Oif:3) with the given rewrite data.

To check the index of 3, use the following:

The -get 3 output shows that the index of 3 corresponds to a tap interface that goes to a VM.

You can also dump individual entries in the map using the —get option, as follows:

Example: mpls -help

Use the mirror command to dump the mirror table entries.

Example: Inspect Mirroring

The following example inspects a mirror configuration where traffic is mirrored from network vn1 (1.1.1.0/24) to network vn2 (2.2.2.0/24). A ping is run from 1.1.1.253 to 2.2.2.253, where both IPs are valid VM IPs, then the flow table is listed:

In the example output, Mirror Index:0 is listed, it is the index to the mirror table. The mirror table can be dumped with the —dump option, as follows:

The mirror table entries point to next hops. In the example, the index 0 points to next hop 18. The References indicate the number of flow entries that point to this entry.

A next hop get operation on ID 18 is performed as follows:

The nh --get output shows that mirrored packets go to a system with IP 250.250.2.253. The packets are tunneled as a UDP datagram and sent to the destination. Vrf:-1 indicates that a lookup has to be done in the source Vrf for the destination.

You can also get an individual mirror table entry using the —get option, as follows:

Example: mirror --help

The vxlan command can be used to dump the VXLAN table. The vxlan table maps a network ID to a next hop, similar to an MPLS table.

If a packet comes with a VXLAN header and if the VNID is one of those in the table, the vRouter will use the next hop identified to forward the packet.

Example: vxlan --dump​

Example: vxlan --get

You can use the —get option to dump a specific entry, as follows:

Example: vxlan --help

The nh command enables you to inspect the next hops that are known by the vRouter. Next hops tell the vRouter the next location to send a packet in the path to its final destination. The processing of the packet differs based on the type of the next hop. The next hop types are described in the following table.

Example: nh --list

Example: nh --get

Use the --get option to display information for a single next hop.

Example: nh --help

In Contrail Networking Release 2008, the dpdkinfo command enables you to see the details of the internal data structures of a DPDK enabled vRouter.

dpdkinfo Options

Use dpdkinfo –-help to display all options available for the dpdkinfo command. The dpdkinfo command options are described in the following table:

Example: dpdkinfo --bond

The dpdkinfo --bond displays the following information for primary and backup devices: actor/partner status, actor/partner key, actor/partner system priority, actor/partner MAC address, actor/partner port priority, actor/partner port number, and so on.

Example: dpdkinfo --lacp all

The dpdkinfo --lacp all command displays the following information for primary devices: LACP rate and LACP configuration details, which include Fast periodic (ms), Slow periodic (ms), Short timeout (ms), Long timeout (ms), LACP packet statistics for Tx and Rx counters, and so on. Also, dpdkinfo --lacp all displays actor and partner port status details of all the backup devices.

Example: dpdkinfo --mempool all and dpdk --mempool <mempool-name>

The dpdkinfo --mempool all displays a summary of the memory pool information of the primary and backup devices, which include number of available memory pools, size of the memory pool, and so on.

The dpdk --mempool <mempool-name> displays detailed information of the memory pool you have specified in the command.

Example: dpdkinfo --stats eth

The dpdkinfo --stats eth command reads Rx and Tx packets statistics from the NIC card and displays the information.

Example: dpdkinfo --xstats

The dpdkinfo --xstats command reads the Rx and Tx from the NIC cards and displays the packet statistics in detail.

Example: dpdkinfo --lcore

The dpdkinfo --lcore displays Logical core (lcore) information, which includes number of forwarding lcores, the interfaces mapped to the lcore, and queue-ID of the interfaces.

dpdkinfo --app

The dpdkinfo --app command displays the following information:

• Application related information about number of lcores, the names of the existing​ backup interfaces, and so on.

• For VLAN configured devices the command displays VLAN​ name, tag, and vlan_vif name.

• For bond interfaces the command displays ethdev information, which include Max rx queues, Max tx queues, Reta size, Port id, number of ethdev slaves, Tapdev information, and so on.

• Monitoring interface names (if available) and SR-IOV information, which includes logical core, ethdev port ID, and driver name.

Example: dpdkinfo --ddp list

In Contrail Networking Release 2011, you can use the dpdkinfo --ddp list command to display the list of DDP profiles added in the vRouter.

The dpdkinfo --ddp list displays a summary of the DDP profile added in the vRouter. The summary of the profile information includes tracking ID of the profile, version number, and profile name.

In Contrail Networking Release 2011, the dpdkconf command enables you to configure a DPDK enabled vRouter. In release 2011, you can use the dpdkconf command to enable or delete a DDP profile in vRouter.

Example: dpdkconf --ddp add

Use the dpdkconf --ddp add command during runtime to enable a DDP profile in a DPDK enabled vRouter.

Example: dpdkconf --ddp delete

Use the dpdkconf --ddp delete command to delete a DDP profile, which is already loaded in the vRouter.