Assisted Replication Multicast Optimization in EVPN Networks

Benefits of Assisted Replication

• In an EVPN network with a significant volume of broadcast, unknown unicast, and multicast (BUM) traffic, AR helps to optimize BUM traffic flow by passing replication and forwarding tasks to other devices that have more capacity to better handle the load.

• In an EVPN-VXLAN environment, AR reduces the number of hardware next hops to multiple remote virtual tunnel endpoints (VTEPs) over traditional multicast ingress replication.

• AR can operate with other multicast optimizations such as IGMP snooping, MLD snooping, and selective multicast Ethernet tag (SMET) forwarding.

• AR operates transparently with devices that do not support AR for backward compatibility in existing EVPN networks.

AR Device Roles

To enable AR, you configure devices in the EVPN network into AR replicator and AR leaf roles. For backward compatibility, your EVPN network can also include devices that don’t support AR. Devices that don't support AR operate in a regular network virtualization edge (NVE) device role.

Table 1 summarizes these roles:

If you enable other supported multicast optimization features in your EVPN network, such as IGMP snooping, MLD snooping, and SMET forwarding, AR replicator and AR leaf devices forward traffic in the EVPN core or on the access side only toward interested receivers.

If you want to enable AR in a fabric with OISM, see AR with Optimized Intersubnet Multicast (OISM) for considerations when planning which devices to configure as AR replicators and AR leaf devices in that case.

AR Route Advertisements

Devices in an EVPN network advertise EVPN Type 3 (IMET) routes for regular ingress replication and AR.

AR replicator devices advertise EVPN Type 3 routes for regular ingress replication that include:

• The AR replicator device ingress replication tunnel IP address

• Tunnel type—IR

• AR device role—AR-REPLICATOR

AR replicators also advertise EVPN Type 3 routes for the special AR overlay tunnels that include:

• An AR IP address you configure on a loopback interface on the AR replicator device (see the replicator configuration statement).

• Tunnel type—AR.

• AR device role—AR-REPLICATOR.

• EVPN multicast flags extended community with the Extended-MH-AR flag when operating in extended AR mode (see Extended AR Mode for Multihomed Ethernet Segments).

AR leaf devices advertise EVPN Type 3 routes for regular ingress replication that include:

• The AR leaf device ingress replication tunnel IP address

• Tunnel type—IR

• AR device role—AR-LEAF

Regular NVE devices don’t support AR or are devices you didn't configured as AR leaf devices. Regular NVE devices ignore AR route advertisements and forward multicast traffic using the usual EVPN network ingress replication rules.

Extended AR Mode for Multihomed Ethernet Segments

EVPN networks employ split horizon and local bias rules to enable multicast optimizations when the architecture includes multihomed Ethernet segments (see EVPN-over-VXLAN Supported Functionality). These methods include information about the ingress AR leaf device in forwarded packets to ensure that the traffic isn’t unnecessarily forwarded or looped back to the source by way of the ingress device’s multihomed peers.

Some devices serving in the AR replicator role can’t retain the source IP address or Ethernet segment identifier (ESI) label on behalf of the ingress AR leaf device when forwarding the traffic to other overlay tunnels. AR replicator devices with this limitation operate in extended AR mode, where the AR replicator forwards traffic toward other devices in the EVPN network with the AR replicator’s ingress replication IP address as the source IP address.

AR replicators include the EVPN multicast flags extended community in the EVPN Type 3 AR route advertisement. The Extended-MH-AR flag indicates the AR operating mode.

When AR devices operate in extended AR mode:

• Besides forwarding the traffic to the AR replicator device, an AR leaf device with multihomed sources also handles replicating the traffic to its multihoming peers.

• An AR replicator device receiving source traffic from an AR leaf device with multihomed sources skips forwarding to the AR leaf device’s multihoming peers.

See Source behind an AR Leaf Device (Multihomed Ethernet Segment) with Extended AR Mode, which shows the traffic flow in extended AR mode for an EVPN network with a multihomed source.

Extended AR mode behavior applies only when AR leaf devices have multihomed Ethernet segments with other AR leaf devices, not with AR replicators from which the AR leaf requests replication assistance. When AR replicator and AR leaf devices share multihomed Ethernet segments in environments that require extended AR mode, AR can’t work correctly. As a result, ingress AR leaf devices don’t use the AR tunnels and default to using only ingress replication. In this case, AR show commands display the AR operational mode as Misconfiguration/Ingress Replication.

AR replicators that can retain the source IP address or ESI label of the ingress AR leaf device operate in regular AR mode.

See the show evpn multicast-snooping assisted-replication replicators command for more information about AR operational modes.

Default AR Leaf Load-Balancing When Detecting Multiple AR Replicators

For QFX Series switches in an EVPN-VXLAN network:

• AR leaf devices that are EX4650 switches or switches in the QFX5000 line designate a particular AR replicator device for a VLAN or VXLAN network identifier (VNI) to load-balance among the available AR replicators. This default method on these devices uses a hash mod algorithm based on the VLAN or VNI and the number of AR replicators.

• AR leaf devices in the QFX10000 line of switches actively load-balance among the available AR replicators based on traffic flow levels (packets per second) within a VLAN or VNI. This method on these devices uses a flow level hash mod algorithm based on the multicast group and the number of AR replicators.

Deterministic Load Balancing and Traffic Steering to AR Replicators

For AR leaf devices that don't use a flow level load balancing mechanism by default (see Default AR Leaf Load-Balancing When Detecting Multiple AR Replicators), we support a feature with which you can deterministically configure AR leaf devices to send multicast flows to particular AR replicators.

With this deterministic AR replicator load balancing method, you:

1. Define routing policies to match one or more multicast flows in the policy from conditions.

   For example, you can set a condition to match a particular multicast group address using the route-filter multicast-group-address statement at the [edit policy-options policy-statement name from] hierarchy.

2. In the then clause of each routing policy, for the policy action, specify the AR replicator(s) to which the AR leaf device sends the matching flows.

   You set the policy action with the assisted-replication replicator-ip replicator-ip-addr statement at the [edit policy-options policy-statement name then] hierarchy. (See assisted-replication (Deterministic AR Replicator Policy Actions).)

   You have options to strictly steer matching flows only to a preferred AR replicator, or also include a fallback AR replicator to use in case the preferred one goes down.

3. Assign the routing policy or policies to an EVPN instance on the AR leaf device using the deterministic-ar-policy statement at the [edit routing-instances name protocols evpn assisted-replication leaf] hierarchy level. (See deterministic-ar-policy.)

An example use case where you might want to deterministically steer AR leaf traffic to particular replicators is when an AR leaf device serves a source device where you know particular multicast flows are usually significantly larger or smaller. Figure 2 shows how the AR leaf device might load-balance the flows using the default load balancing method that results in a very unbalanced traffic distribution.

Figure 2: Default AR Load Balancing with Multicast Flows of Different Sizes

Figure 3 illustrates how you can configure deterministic load balancing to more equally distribute (steer) the flows among the available replicators.

Figure 3: Deterministic AR Load Balancing to Handle Multicast Flows of Significantly Different Sizes

• AR Load Balancing Routing Policy Options
• Behavior Notes on Deterministic AR Load Balancing Policies
• Assign a Deterministic AR Policy to an EVPN Instance on an AR Leaf Device
• Sample Deterministic AR Policy Configurations

AR and OISM Device Roles

With AR, you configure devices to function in the AR replicator or AR leaf role. When you don't assign an AR role to a device, the device functions as a regular NVE device. See Table 1 for more on these roles.

With OISM, you configure devices in an ERB overlay fabric to function in one of the following roles:

• OISM border leaf role.

• OISM server leaf role.

• No OISM role—These devices are usually lean spine devices in the fabric.

See Optimized Intersubnet Multicast in EVPN Networks for details on how OISM works and how to configure OISM devices.

When you integrate AR and OISM functions, we support the AR replicator role in the following modes:

• Collocated: You configure the AR replicator role on the same device as the OISM border leaf role.

• Standalone: The AR replicator function is not collocated with the OISM border leaf role on the same device. In this case, the AR replicator is usually a lean spine device in an ERB fabric running OISM.

  Note:

  On QFX5130-32CD and QFX5700 switches, we only support standalone mode. You can configure the AR replicator role only on a device in the fabric that isn't also an OISM border leaf device.

Guidelines to Integrate AR and OISM Device Roles

Use these guidelines to integrate AR and OISM roles on the devices in the fabric:

• You can configure any OISM leaf devices (border leaf or server leaf) with the AR leaf role.

• Use the following rules when you configure the AR replicator role:

  1. We support the AR replicator role on any of following devices: QFX5130-32CD, QFX5700, QFX10002, QFX10008, and QFX10016.

     Note:

     QFX5130-32CD and QFX5700 devices support the AR replicator role only in standalone mode.

  2. The devices you use as AR replicators must be devices that support OISM, even if you configure an AR replicator in standalone mode (see AR and OISM Device Roles). The AR replicator devices must have the same tenant virtual routing and forwarding (VRF) and VLAN information as the OISM devices (see rule 3).

  3. Even if the AR replicator device is in standalone mode, you must configure the device with the same tenant VRF instances, corresponding IRB interfaces, and member VLANs as the OISM leaf devices. The AR replicator requires this information to install the correct L2 multicast states to properly forward the multicast traffic.

     See Configure Common OISM Elements on Border Leaf Devices and Server Leaf Devices for details on the OISM configuration steps you duplicate on standalone AR replicators for these elements.

How AR and OISM Work Together

When you enable AR and OISM together, you can configure higher capacity devices in the fabric to do the replication and lessen the load on the OISM leaf devices. With OISM, the ingress leaf device replicates and sends copies of the packets on the ingress VLAN toward interested receivers, where:

• The ingress leaf device might be an OISM server leaf device or an OISM border leaf device.

• For multicast traffic sourced within the fabric, the ingress VLAN is a revenue VLAN.

• For multicast traffic coming into the fabric from an external multicast source, the ingress VLAN is the supplemental bridge domain (SBD).

When you configure the AR leaf role on the ingress OISM leaf device, the device sends one copy of the traffic toward the AR replicator. The AR replicator takes care of replicating and forwarding the traffic toward the other OISM server leaf or border leaf devices. The OISM leaf devices do local routing to minimize the multicast traffic in the EVPN core.

See the following illustrations of AR and OISM working together:

• AR and OISM with an Internal Multicast Source

• AR and OISM with an External Multicast Source

See Configure Assisted Replication for details on how to configure AR, including when you integrate AR and OISM device roles.

Source behind a Regular NVE Device

Figure 4 shows an EVPN network with multicast traffic from a source attached to Regular NVE, a device that doesn’t support AR.

Figure 4: Multicast Source Traffic Ingress at a Regular NVE Device

In this case, forwarding behavior is the same whether you enable AR or not, because the ingress device, Regular NVE, is not an AR leaf device sending traffic for replication to an AR replicator. Regular NVE employs the usual ingress replication forwarding rules as follows:

• Without IGMP snooping or MLD snooping, and without SMET forwarding enabled, Regular NVE floods the traffic to all the other devices in the EVPN network.

• With IGMP snooping or MLD snooping, and with SMET forwarding enabled, Regular NVE only forwards the traffic to other devices in the EVPN network with active listeners. In this case, Regular NVE forwards to all the other devices except AR Leaf 2 and AR Leaf 4.

Spine 1 and Spine 2 replicate and forward the traffic toward their local single-homed or multihomed receivers or external gateways using the usual EVPN network ingress replication local bias or DF behaviors. In this case, Figure 4 shows that Spine 2 is the elected DF and forwards the traffic to a multihomed receiver on an Ethernet segment that the two spine devices share.

Source behind an AR Replicator Device

Figure 5 shows an EVPN network with multicast traffic from a source attached to an AR replicator device called Spine 1.

Figure 5: Multicast Source Traffic Ingress at an AR Replicator Device

In this case, forwarding behavior is the same whether you enable AR or not, because the ingress device, Spine 1, is not an AR leaf device sending traffic for replication to an AR replicator. Spine 1, although it’s configured as an AR replicator device, does not act as an AR replicator. Instead, it employs the usual ingress replication forwarding rules as follows:

• Without IGMP snooping or MLD snooping, and without SMET forwarding enabled, Spine 1, the ingress device, floods the traffic to all the other devices in the EVPN network.

• With IGMP snooping or MLD snooping, and with SMET forwarding enabled, Spine 1 only forwards the traffic to other devices in the EVPN network with active listeners. In this case, Spine 1 forwards to the other devices except AR Leaf 2 and AR Leaf 4, and to Spine 2, which has a local interested receiver.

• Spine 1 also replicates the traffic toward local single-homed or multihomed receivers, or external gateways, using the usual EVPN network ingress replication local bias or DF behaviors. In this case, Spine 1 uses local bias and forwards the traffic to a multihomed receiver due to local bias rules (whether it is the DF on that Ethernet segment or not).

Spine 2 forwards the traffic it received from Spine 1 to its local receiver. Spine 2 also does a local bias check for its multihomed receiver, and skips forwarding to the multihomed receiver even though it is the DF for that Ethernet segment.

Source behind an AR Leaf Device (Single-Homed Ethernet Segment)

Figure 6 shows an EVPN network with multicast traffic from a source attached to the AR leaf device called AR Leaf 4. Traffic flow is the same whether the ingress AR leaf device and AR replicator are operating in extended AR mode or not, because the source isn’t multihomed to any other leaf devices.

Figure 6: Source Traffic Ingress at an AR Leaf Device (Single-Homed Source)

Without IGMP snooping or MLD snooping, and without SMET forwarding enabled:

• AR Leaf 4 forwards one copy of the multicast traffic to an advertised AR replicator device, in this case Spine 1, using the AR overlay tunnel secondary IP address that you configured on the loopback interface, lo0, for Spine 1.

• AR Leaf 4 also applies local bias forwarding rules and replicates the multicast traffic to its locally attached receiver.

• Spine 1 receives the multicast traffic on the AR overlay tunnel and replicates and forwards it on behalf of Leaf 4 using the usual ingress replication overlay tunnels to all the other devices in the EVPN network, including Spine 2. Spine 1 skips forwarding the traffic back to the ingress device AR Leaf 4 (according to split horizon rules).

With IGMP snooping or MLD snooping, and with SMET forwarding enabled:

• AR replicator Spine 1 further optimizes replication by only forwarding the traffic toward other devices in the EVPN network with active listeners. In this case, Spine 1 skips forwarding to AR Leaf 2.

Spine 1 also replicates and forwards the traffic to its local receivers and toward multihomed receivers or external gateways using the usual EVPN network ingress replication local bias or DF behaviors. In this case, Spine 1 forwards the traffic to a local receiver. Spine 1 uses local bias to forward to a multihomed receiver (although it is not the DF for that Ethernet segment).

Spine 2 forwards the traffic it received from Spine 1 to its local receiver. Spine 2 does a local bias check for its multihomed receiver, and skips forwarding to the multihomed receiver even though it is the DF for that Ethernet segment.

Source behind an AR Leaf Device (Multihomed Ethernet Segment) with Extended AR Mode

Figure 7 shows an EVPN network with multicast traffic from a source on a multihomed Ethernet segment operating in extended AR mode. The source is multihomed to three AR leaf devices and might send the traffic to any one of them. In this case, AR Leaf 1 is the ingress device.

Figure 7: Source Traffic Ingress at an AR Leaf Device (Multihomed Source)

Without IGMP snooping or MLD snooping, and without SMET forwarding enabled:

• AR Leaf 1 forwards one copy of the multicast traffic to one of the advertised AR replicator devices, in this case, Spine 1, using the AR overlay tunnel secondary IP address that you configured on the loopback interface, lo0, for Spine 1.

• Operating in extended AR mode:

  • AR Leaf 1 also replicates and forwards the multicast traffic to all of its multihoming peers (AR Leaf 2 and AR Leaf 3) for the source Ethernet segment.

  • Spine 1 receives the multicast traffic on the AR overlay tunnel and replicates and forwards it using the usual ingress replication overlay tunnels to the other devices in the EVPN network except AR Leaf 1 and its multihoming peers AR Leaf 2 and AR Leaf 3.

With IGMP snooping or MLD snooping, and with SMET forwarding enabled, the AR leaf and AR replicator devices further optimize multicast replication in extended AR mode, as follows:

• Besides sending to AR replicator Spine 1, AR Leaf 1 also replicates and forwards the multicast traffic only to its multihoming peers that have active listeners (AR Leaf 3).

• Spine 1 replicates traffic for AR Leaf 1 only to other devices in the EVPN network with active listeners. In this case, that includes only the regular NVE device and Spine 2.

Spine 1 also replicates and forwards the traffic to its local receivers and toward multihomed receivers or external gateways using the usual EVPN network ingress replication local bias or DF behaviors. In this case, Spine 1 uses local bias to forward to a multihomed receiver although it is not the DF for that Ethernet segment. Spine 2 receives the traffic from Spine 1 and forwards the traffic to its local receiver. Spine 2 also does a local bias check for its multihomed receiver, and skips forwarding to the multihomed receiver even though it is the DF for that Ethernet segment.