Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
ON THIS PAGE
 

Fast Reroute for Egress Link Protection with EVPN-VXLAN Multihoming

You can enable the fast reroute egress link protection (ELP) feature on multihoming peer provider edge (PE) devices in an EVPN-VXLAN network. This feature improves route convergence times and avoids load-balanced traffic loss toward a multihomed customer edge (CE) device if a PE device link to the CE device fails.

Benefits

  • Avoids loss of load-balanced traffic toward a multihomed CE device when a link to the CE device fails.

  • Improves route convergence time for multihoming Ethernet segment routes in higher scaled EVPN-VXLAN fabrics.

How Fast Reroute ELP Works with Multihoming in an Ethernet Segment

In EVPN fabrics, to improve reliability and performance, customers often multihome CE devices to two or more PE devices in an Ethernet segment (ES). Multihoming helps to:

  • Safeguard against traffic loss if a PE device link to the multihomed CE device fails.

  • Enable load balancing among multiple paths to the destination.

The connections for a multihomed CE to different PE devices share an ES identifier (ESI). EVPN devices use ES routes (EVPN Type 4 routes) to advertise their local multihoming peer PE connections. The PE devices also use the ES routes to elect designated forwarders (DFs) among the peer devices to avoid sending duplicate traffic on the ESI.

With EVPN-VXLAN, a PE device connected to a multihomed CE device advertises ES tunnel routes (VXLAN tunnel routes) for each locally attached ESI. Remote PE devices receive these advertisements and create load-balancing next-hop tables for the ES. The remote PE devices use the ES next hops to load-balance traffic that is destined for a multihomed CE device among the peer PE devices associated with the ESI.

When one of the ESI links fails while a remote PE is sending traffic to the multihomed CE device:

  1. The PE device with the failed link withdraws the corresponding ES route.

  2. A remote PE device continues load balancing traffic toward the ESI using routes that correspond to the failed link until the device receives the route withdrawal and adjusts its load-balancing next hops.

  3. The PE device with the failed link drops the excess (residual) load-balanced traffic destined for the failed link.

  4. The remote PE devices receive and process the route withdrawal. The remote PE devices update their MAC address routing information and next-hop tables. At that point, the remote PE devices stop sending load-balanced traffic to the failed link.

Until the remote PE routing tables converge (which might take seconds in scaled environments), the destination CE device loses some of the traffic.

To help avoid this load-balanced traffic loss on the ESI when a member link goes down, you can enable the fast reroute ELP feature on the multihoming peer PE devices. This feature also helps improve ESI route convergence time, especially in highly-scaled networks.

ELP with Fast Reroute in an Ethernet Segment

When you enable this feature on a PE device, the device creates a backup VXLAN tunnel called an ELP tunnel to each of its peer PE devices for an ES. Upon ES link failure, the PE device uses the ELP tunnel to reroute traffic to one or more of its peer PE devices.

Note:

For this feature to work, you enable it uniformly on all peer PE devices associated with an ESI. Each PE device on the ES has an ELP tunnel to all of the peer PE devices.

The following figures illustrate how the ELP tunnel reroute process works in a simple EVPN-VXLAN fabric where:

  • CE1 is multihomed to peer PE devices PE1 and PE2.

  • PE1 and PE2 have ELP tunnels to each other.

  • PE3 is a remote source that load-balances traffic toward CE1 using VXLAN tunnel endpoints (VTEPs) on PE1 and PE2.

In Figure 1, all the ES links toward CE1 are up, so PE1 and PE2 each forward their portions of the load-balanced traffic to CE1. PE1 and PE2 don't need to use their ELP tunnels.

Figure 1: Load-Balanced Traffic to Multihomed CE with All ESI Links Up Load-Balanced Traffic to Multihomed CE with All ESI Links Up

In Figure 2, the link from PE1 toward CE1 fails. PE1 withdraws the ES route for the failed link. However, until PE3 achieves route convergence, PE3 still sends some excess load-balanced traffic to PE1. Instead of dropping its portion of the load-balanced traffic, PE 1 uses its ELP tunnel to reroute that traffic to PE2. PE2 forwards both portions of the load-balanced traffic to CE1.

Figure 2: ES Link Down with Fast Reroute to Peer PE ES Link Down with Fast Reroute to Peer PE

In Figure 3, PE3 achieves route convergence and stops sending any load-balanced traffic for the ES to PE1. PE3 sends all traffic for the ES only to PE2, and PE2 sends the traffic to CE1. PE1 doesn't need to use its ELP tunnel to PE2 anymore.

Figure 3: ELP Tunnel Not Used After ES Route Convergence ELP Tunnel Not Used After ES Route Convergence

ELP Tunnels with Multiple Peer PE Devices

When you enable this feature on a PE device with two or more peer PE devices on the ES, the PE device sets up a backup VXLAN ELP tunnel to each peer PE device. See Figure 4. Each PE device maintains a next hops group for load balancing traffic destined for the ES. The remote source PE device distributes the traffic toward the ES among the next hops in the group. Each peer PE device on the ES also maintain a load-balancing next hops group with all of the ELP tunnels for the ES, so the device can also load balance rerouted traffic.

Figure 4 shows what happens if one of the links on the ES fails. In this case, the link from PE1 to CE1 fails. As a result:

  • Until PE4 achieves route convergence, PE4 still sends some excess load-balanced traffic to PE1.

  • Instead of dropping its portion of the load-balanced traffic, PE1 reroutes its excess load-balanced traffic on both of its ELP tunnels toward peer PE devices PE2 and PE3.

  • PE2 and PE3 send all of the traffic to CE1, including the load-balanced traffic they receive from PE4 and the traffic from PE1’s ELP tunnels.

  • (Not shown) PE4's routing tables converge. PE4 stops sending any traffic toward PE1 and load-balances the traffic only to PE2 and PE3.

Figure 4: ELP Tunnels for Two or More Peer PE Devices ELP Tunnels for Two or More Peer PE Devices

Fast Reroute ELP Requirements and Limitations

To enable an ELP tunnel on a PE device, configure the reroute-address statement at the [edit forwarding-options evpn-vxlan] hierarchy level. Specify the IP address the device uses for the ELP tunnel using the inet ip-address option to the reroute-address statement. Configure Fast Reroute ELP on a PE Device describes more about this statement and the other steps required to configure this feature.

With this feature:

  • You must configure this statement on all of the peer PE devices for a particular ES for the ELP tunnels on any peer PE device to work.

  • Multihoming peer PE devices do not flush MAC address table entries for aggregated Ethernet (AE) interface bundle links in active/active state when an AE ESI interface goes down. This behavior helps implement the fast reroute actions and improves route convergence times.

Fast reroute ELP has the following limitations:

  • This feature applies only with multihoming configured on AE logical interfaces.

  • We support this feature only for IPv4 client traffic.

  • This feature works for unicast traffic and VLAN flooding only, not multicast traffic.

  • We support this feature only with enterprise style interface configurations. (See Flexible Ethernet Services Encapsulation for more on this type of interface configuration.)

  • This feature handles rerouting traffic upon ES link failure on only one multihoming peer at a time that is associated with a particular ESI.

Note:

Fast reroute ELP is not supported for planned administrative link-down events such as when you administratively disable a PE-CE interface locally on the PE device by running the set interface interface-name disable command. For a planned interface-down maintenance event on the PE device, if needed, take other administrative actions to prevent traffic loss during the maintenance. However, if a PE-CE link goes down due to an administrative link-down command on the CE device, the PE device detects the link failure and invokes the fast reroute operation in that case.

Configure Fast Reroute ELP on a PE Device

You configure fast reroute ELP feature elements on supported multihoming peer PE devices for an ES. For this feature to work, you must enable it on all of the multihoming peer PE devices for the ES. You configure the steps that are listed here uniformly on each peer PE device.

You don't need to configure any corresponding elements on the multihomed CE device in the ES or on the remote source PE devices that send traffic to the ES. The CE devices and remote source PE devices can be devices that don't support this feature.

To enable the fast reroute ELP feature on a multihoming peer PE device for an ES:

  1. Configure the IP address you want to use as the ELP tunnel source VTEP address for the ELP ES. You configure this address as a secondary address on the device loopback interface (lo0). The ELP tunnel address must be different from the device router ID address (which is the primary, preferred address).

    For example, on a PE device PE1 with router ID 192.168.0.1, you might use 192.168.102.1 as the IP address for the ELP tunnel to peer PE device PE2:

  2. Enable fast reroute ELP using the secondary lo0 address you configured in step 1.

    When you commit this configuration, the PE device creates a corresponding ELP tunnel.

    For example, using the secondary lo0 interface IP address from step 1:

    Note:

    The ELP tunnel is idle until a member link on the PE device to a multihomed CE device fails. When the link fails, the remote source device continues to send some load-balanced traffic toward the failed link until route convergence. The PE device uses the ELP tunnel only to reroute any excess load-balanced traffic to its peer PE devices until the remote source stops sending traffic to it. After that point, the ELP tunnel is idle again.

Verify Fast Reroute ELP Tunnel Creation and Operation

When you enable the fast reroute ELP tunnel feature, you can use the CLI show commands in this section to see ELP tunnel IP addresses, next hop IDs, and source and remote VTEP information.

We explain some sample show command outputs here based on a topology similar to Figure 2, which we include again below for reference:

Figure 5: Sample Topology for Show Command Output Sample Topology for Show Command Output

Table 1 shows the sample topology primary lo0 IP addresses for the PE devices and secondary lo0 IP addresses for their ELP tunnels:

Table 1: Sample Topology IP Addresses for Multihoming Peer PE Devices
Device Device IP Address ELP Tunnel IP Address

PE1

192.168.0.1/32

192.168.102.1/32

ELP tunnel from PE1 to PE2

PE2

192.168.0.2/32

192.168.201.1/32

ELP tunnel from PE2 to PE1

The sample topology also has these hosts (not shown) behind the CE devices in Figure 5:

Table 2: Sample Topology Host Information at Layer 2
Host Description VLAN name (VLAN ID) MAC Address

Receiver behind CE1

Vlan2 (VLAN ID 2)

00:00:94:00:00:01

Remote source behind CE2

Vlan3 (VLAN ID 3)

00:01:95:00:00:01

Finally, in this sample topology, the ESI assigned to the multihomed links from CE1 to PE1 and PE2 is 00:00:00:00:20:00:00:00:00:00.

show ethernet-switching table

Use the show ethernet-switching table mac-address extensive command to see ELP tunnel next hops for a particular MAC route.

For example, this output shows that PE1 will use the ELP tunnel next hop (ELP-NH field) to send any traffic it receives to the host behind CE1 on Vlan2 (MAC address 00:00:94:00:00:01):

show ethernet-switching vxlan-tunnel-end-point esi

Use the show ethernet-switching vxlan-tunnel-end-point esi esi-identifier esi-value command to see the ELP tunnel VTEP IP address, next hops and VTEP interface name.

For example, this output shows PE1 has an ELP tunnel and ELP next hop (ELP-NH field) to remote VTEP PE2 (RVTEP-IP field) for ESI 00:00:00:00:20:00:00:00:00:00:

show ethernet-switching vxlan-tunnel-end-point remote

Use the show ethernet-switching vxlan-tunnel-end-point remote command to see ELP tunnel source VTEP and remote VTEP information about a device, including:

  • VTEP IP addresses and interface names

  • Corresponding next hop ID and ELP tunnel IP address

If the ELP-IP output field contains an IP address, that means the remote VTEP in theRVTEP-IP column has a corresponding ELP tunnel. Also, the ELP-IP column in the RVTEP-IP row contains “ELP” if that RVTEP-IP address is an ELP tunnel address.

See the sample output below and Table 3 for details on what this output means:

Table 3: Sample show ethernet-switching vxlan-tunnel-end-point remote Output Fields
Output Fields Meaning

SVTEP-IP—192.168.0.1

ELP-SVTEP-IP—192.168.102.1

PE1 (192.168.0.1) is the source VTEP and has an ELP tunnel with IP address 192.168.102.1 to PE2.

First row under RVTEP-IP heading with:

RVTEP-IP—192.168.0.2

ELP-IP—192.168.201.1

PE1 has PE2 (192.168.0.2) as a remote VTEP. PE2 has an ELP tunnel with IP address 192.168.201.1 to PE1.

Third row under RVTEP-IP heading with:

RVTEP-IP—192.168.201.1

ELP-IP—ELP

The remote VTEP with IP address 192.168.201.1 is an ELP tunnel remote VTEP. From the other output fields in this table, we know this remote VTEP is the ELP tunnel from PE2 to PE1.

show ethernet-switching vxlan-tunnel-end-point source

Use the show ethernet-switching vxlan-tunnel-end-point source command to see the device ELP tunnel source VTEP IP address in the ELP-SVTEP-IP output field.

For example, the following sample output on PE1 shows that PE1 (SVTEP -IP 192.168.0.1) has an ELP tunnel with IP address 192.168.102.1 (ELP-SVTEP-IP) on the lo0 interface:

show route table __default_evpn__.evpn.0 extensive

The show route table __default_evpn__.evpn.0 extensive routing table command includes the Reroute address field, which shows the local reroute ELP tunnel address.

For example, the Reroute address field below shows that the ELP tunnel on PE1 (192.168.0.1) has IP address 192.168.102.1:

show route table bgp.evpn.0 extensive

The show route table bgp.evpn.0 extensive command includes the Reroute address field showing the device ELP tunnel address.

For example:

show interfaces (with ELP Tunnel VTEP Interfaces)

Use the show interfaces command with the ELP tunnel VTEP interface name to see when the device reroutes traffic on the ELP tunnel.

For example, the sample command output we saw earlier in show ethernet-switching vxlan-tunnel-end-point remote shows that the ELP tunnel VTEP interface name is vtep.32831 (the Interface column with the remote VTEP for the tunnel to PE2).

If you enter the show interfaces vtep.32831 command, before the link between CE1 and PE1 fails, the Output packets field shows that the traffic for the VTEP doesn't go over the ELP tunnel:

After the link between CE1 and PE1 fails, the same command shows that the traffic takes the ELP tunnel path for a short time (until source routing table convergence happens):

See Also