CoS Support on EVPN VXLANs
You can configure class of service (CoS) features on VXLAN interfaces. VXLAN traffic from different tenants traverses network boundaries over the same physical underlay network. To ensure fairness in the treatment of traffic for all tenants in the VXLAN, and to prioritize higher priority traffic, apply CoS features to the VXLAN interfaces.
Understanding CoS on VXLAN Interfaces
This section describes how classification and rewrite rules are applied to packets in a VXLAN instance. Figure 1 shows a simple VXLAN with two leaf nodes and one spine node.
Refer to Figure 1 to understand the packet flow with DSCP/ToS fields in a VXLAN:
-
CE 1 sends a packet with Layer3 DSCP/ToS bit programmed to the Leaf 1 node.
-
Leaf 1 receives the original packet and appends the VXLAN header on top of the original packet. The outer VXLAN Layer3 header uses the original packet DSCP/Tos bit. You can create classifiers based on the original packet DSCP/802.1p bit. The ingress interface on the ingress leaf supports DSCP and 802.1p classifiers.
-
If rewrite is configured on Leaf 1, the inner header will have the DSCP/802.1p bit set by CE 1 and the outer header will have the rewrite bit. Only DSCP rewrite rules are supported, except on QFX10000 switches where 802.1p rewrite is also supported if the underlay is tagged.
-
The Spine node receives the VXLAN packet and can use ingress classification using these DSCP bits and forward the packet to the egress interface with the appropriate forwarding class.
-
The Spine egress interface can rewrite these bits using rewrite rules. These Spine rewrite rules only affects the outer Layer3 DSCP field. The inner/original packet still holds the DSCP/802.1p bit that was set by CE 1.
-
Leaf 2 receives the packet, processes the tunnel termination, and remove the outer VXLAN header.
-
Leaf 2 classification and rewrite functionality works on the inner header.
-
The original packet arrives on CE 2.
On the leaf nodes, if the packet is multicast,
you can use multi-destination
classification to create appropriate multicast
classification and rewrite rules.
Configuring CoS on VXLAN Interfaces
This section shows sample configurations of classifiers and rewrite rules for the leaf and spine nodes in VXLAN using Figure 1 as a reference. You can create schedulers as normal for the classifiers on each node.
Sample configuration of classifiers and rewrite rules on Leaf 1.
Sample configuration of classifiers and rewrite rules on the Spine.
-
Create a classifier based on the outer VXLAN DSCP/ToS bits:
[edit class-of-service classifiers] user@spine#set dscp dscp_cf forwarding-class best-effort loss-priority low code-points af22 user@spine#set dscp dscp_cf forwarding-class network-control loss-priority high code-points af31 user@spine#set dscp dscp_cf forwarding-class expedited-forwarding loss-priority low code-points af13 user@spine#set dscp dscp_cf forwarding-class assured-forwarding loss-priority high code-points cs3
-
Apply the classier to the ingress Spine interfaces:
[edit class-of-service interfaces] user@spine#set ge-0/0/3 unit 0 classifiers dscp dscp_cf user@spine#set ge-0/0/5 unit 0 classifiers dscp dscp_cf
-
Create a rewrite rule for the outer VXLAN DSCP/ToS bits:
[edit class-of-service rewrite-rules] user@spine#set dscp dscp_rw forwarding-class best-effort loss-priority low code-points af22 user@spine#set dscp dscp_rw forwarding-class network-control loss-priority high code-points af31 user@spine#set dscp dscp_rw forwarding-class expedited-forwarding loss-priority low code-points af13 user@spine#set dscp dscp_rw forwarding-class assured-forwarding loss-priority high code-points cs3
-
Apply the rewrite rule to the egress Spine interfaces:
[edit class-of-service interfaces] user@spine#set ge-0/0/4 unit 0 rewrite-rules dscp dscp_rw user@spine#set ge-0/0/6 unit 0 rewrite-rules dscp dscp_rw
Sample configuration of classifiers and rewrite rules on Leaf 2.
-
Create a classifier based on the original DSCP/ToS bits, as the VXLAN header is removed at tunnel termination before forwarding classes are applied:
[edit class-of-service classifiers] user@leaf2#set dscp dscp_cf forwarding-class best-effort loss-priority low code-points 100000 user@leaf2#set dscp dscp_cf forwarding-class network-control loss-priority high code-points 110000 user@leaf2#set dscp dscp_cf forwarding-class expedited-forwarding loss-priority low code-points 011010 user@leaf2#set dscp dscp_cf forwarding-class assured-forwarding loss-priority high code-points 001010
-
Apply the classier to the ingress Leaf 2 interfaces:
[edit class-of-service interfaces] user@leaf2#set ge-0/0/7 unit 0 classifiers dscp dscp_cf user@leaf2#set ge-0/0/8 unit 0 classifiers dscp dscp_cf
-
Create a rewrite rule for the original DSCP/ToS bits:
[edit class-of-service rewrite-rules] user@leaf2#set dscp dscp_rw forwarding-class best-effort loss-priority low code-points 100000 user@leaf2#set dscp dscp_rw forwarding-class network-control loss-priority high code-points 110000 user@leaf2#set dscp dscp_rw forwarding-class expedited-forwarding loss-priority low code-points 011010 user@leaf2#set dscp dscp_rw forwarding-class assured-forwarding loss-priority high code-points 001010
-
Apply the rewrite rule to the egress Leaf 2 interface:
[edit class-of-service interfaces] user@leaf2#set ge-0/0/9 unit 0 rewrite-rules dscp dscp_rw
To check the CoS configuration on one of the interfaces:
user@node#show class-of-service interface interface-name
To check the queue statistics on one of the interfaces:
user@node#show interfaces queue interface-name
Implementing CoS on VXLAN Interfaces (Junos OS Evolved)
CoS for EVPN VXLAN traffic is supported using a combination of classifiers, schedulers, and rewrite rules. This section describes how these components are implemented across different nodes on devices running Junos OS Evolved to apply CoS on the EVPN VXLAN traffic.
- Classification at User Network Interface (UNI)/Ingress PE — Traffic classification based on IEEE 802.1p and Differentiated Services code point (DSCP) are supported on the ingress PE where the EVPN VXLAN tunnel is initiated. BA and MF classifiers can be applied to Enterprise style (EP) or Service Provider (SP) style access interfaces.
- Classification at Network Node Interface (NNI)/Egress PE — Traffic classification based on IEEE 802.1p and Differentiated Services code point (DSCP) are supported on the egress PE where the EVPN VXLAN tunnel is terminated. BA classifiers can be applied to the underlying logical interface or unit. MF classifiers are not supported in tunnel terminations.
- Rewrite at NNI — After the
encapsulation of the VXLAN tunnel, the rewrites on
the outer/tunnel header are configured using the
rewrite rules on the underlying logical interface
or unit. Based on the configured rewrite rules,
the VXLAN traffic is classified in the
Spine/Network.
DSCP rewrites on the outer/tunnel header of VXLAN packets is supported on the NNI interface.
Rewrite rules are supported in the following EVPN VXLAN scenarios:- Intra-VNI L2 gateway — Rewrite rules are applied to both unicast and broadcast, unknown unicast, and multicast (BUM) traffic.
- Inter-VNI L3 gateway — Centrally-routed bridging (CRB) and edge-routed bridging (ERB).
- EVPN Type 5 routes.
- Rewrite at UNI — After the termination
of the VXLAN tunnel, the rewrites on the inner
headers are configured using rewrite rules on the
Enterprise style (EP) or Service Provider (SP)
style access interfaces. Based on the configured
rewrite rules, the decapsulated packets are
classified in the CE side network. The following
rewrite rules are supported on the UNI interface
for the decapsulated packets:
- DSCP rewrites on the inner IPv4/IPv6 header
- IEEE 802.1p rewrites on the inner Ethernet header (if tagged)
- Intra-VNI L2 gateway — Rewrite rules are applied to both unicast and broadcast, unknown unicast, and multicast (BUM) traffic.
- Inter-VNI L3 gateway — Centrally-routed bridging (CRB) and edge-routed bridging (ERB).
- EVPN Type 5 routes.
- Scheduling — Traffic prioritization and bandwidth reservation are achieved by using schedulers. The schedulers are associated with a forwarding class set via classifiers.
CoS Limitations on VXLANs
The following limitations apply to PTX routers:
- DSCP rewrite rules are not supported on Integrated Routing and Bridging (IRB) (L3 gateway scenarios).
- IEEE 802.1p rewrite rules are not supported on the NNI interface.
- Explicit congestion notification (ECN) rewrites are not supported on either UNI or NNI interfaces.
- Priority-based flow control (PFC) is not supported.
- No support for CoS classification and rewrite mechanism for IPv6 or IRB underlay.
The CoS functionality on EVPN VXLAN is the same as on QFX5K platforms. All VXLAN CoS features already supported on the QFX5120 are also supported on the QFX5130 and QFX5700 platforms.
The following limitations apply to the QFX5130 and QFX5700 platforms:
- HQoS is not supported due to hardware limitations.
- Classifier, rewrite and scheduler on IRB interface is not supported.
- DOT1P rewrite and classifier on the NNI port is not supported.
- DOT1P and DSCP rewrite on the UNI port is not supported.
- DSCP rewrite on the NNI port is supported with the following
conditions:
-
DSCP rewrite on the NNI port is supported by default and does not work if
vxlan-tos-copy-filterat[edit forwarding-options]is enabled. -
Inner ECN bits are copied to the outer VXLAN header regardless of whether
vxlan-tos-copy-filterat[edit forwarding-options]is enabled. -
Do not enable
vxlan-tos-copy-filterandvxlan-disable-copy-tos-encapat[edit forwarding-options]at the same time as this causes in-deterministic behavior.
-
- PFC configuration will cause momentary traffic drops of up to 10ms.
- DSCP IPV6 classifiers and rewrites are not supported. Use DSCP classifier and rewrite instead.
- TOS copy feature does not work for Type-5 EVPN VXLANs.
The following limitation applies to QFX10000 platforms:
-
Because IRB interfaces do not support dscp rewrite rules, you can apply rewrite rules on underlying L2 interfaces. 802.1p/dscp values in a VXLAN tunneled packet are written using underlying L2 interface rules.