Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
ON THIS PAGE
 

Best Practices and Usage Notes

Address Resolution Protocol Active-Active MC-LAG Support Methodology

ARP and MAC address tables normally stay synchronized in MC-LAG configurations, but might get out of sync under certain network conditions (such as link flapping). To ensure these tables remain in sync while those conditions are being resolved, we recommend enabling the arp-l2-validate statement on IRB interfaces in an MC-LAG configuration.

This option turns on validation of ARP and MAC table entries, automatically applying updates if they become out of sync. You might want to enable this option as a workaround when the network is experiencing other issues that also cause loss of ARP and MAC synchronization, but disable it during normal operation because this option might impact performance in scale configurations.

  • In some cases, ARP messages received by one MC-LAG peer are replicated to the other MC-LAG peer through ICCP. This optimization feature is applicable only for ARP replies, not ARP requests, received by the MC-LAG peers.

  • Dynamic ARP resolution over the ICL interface is not supported. Consequently, incoming ARP replies on the ICL are discarded. However, ARP entries can be populated on the ICL interface through ICCP exchanges from a remote MC-LAG peer.

  • During graceful Routing Engine switchover (GRES), ARP entries that were learned remotely are purged and then learned again.

DHCP Server

Note:

We do not support configuring DHCP server on MC-LAG peers if DHCP service redundancy is required.

DHCP Relay with Option 82

Note:

DHCP relay is not supported with MAC address synchronization. If DHCP relay is required, configure VRRP over IRB or RVI for Layer 3 functionality.
Best Practice:

In an MC-LAG active-active environment, we recommend that you use the bootp relay agent by configuring the DHCP relay agent with the forwarding options helpers bootp command to avoid stale session information issues that might arise for clients when the router is using the extended DHCP relay agent (jdhcp) process.

Failure Handling

Table 1 describes the different ICCP failure scenarios for EX9200 switches. The dash means that the item is not applicable.

Table 1: ICCP Failure Scenarios for EX9200 Switches

ICCP Connection Status

ICL Status

Backup Liveness Peer Status

Action on Multichassis Aggregated Ethernet Interface with Status Set to Standby

Action on Multichassis Aggregated Ethernet Interface with Status Set to Standby and Prefer Status Control Set to Active

Down

Down or Up

Not configured

LACP system ID is changed to default value.

Not applicable. Liveness detection must be configured.

Down

Down or Up

Active

LACP system ID is changed to default value.

No change in LACP system ID.

Down

Down or Up

Inactive

No change in LACP system ID.

No change in LACP system ID.

Up

Down

–

LACP state is set to standby. MUX state moves to waiting state.

LACP status is set to standby. MUX state moves to waiting status.

Table 2 describes the different ICCP failure scenarios for QFX Series switches. The dash means that the item is not applicable.

Table 2: ICCP Failure Scenarios for QFX Series Switches

ICCP Connection Status

ICL Status

Backup Liveness Peer Status

Action on Multichassis Aggregated Ethernet Interface with Status Set to Standby

Down

Down or Up

Not configured

LACP system ID is changed to default value.

Down

Down or Up

Active

.LACP system ID is changed to default value for both active and standby MC-AE interfaces.

Down

Down or Up

Inactive

No change in LACP system ID.

Up

Down

–

LACP state is set to standby. MUX state moves to waiting state.

Configure the master-only statement on the IP address of the fxp0 interface for backup liveness detection on both the primary and backup Routing Engines. This ensures that the connection is not reset during GRES in the remote peer.

For example, on the primary Routing Engine:

For example, on the backup Routing Engine:

The primary Routing Engine services both 10.8.2.31 and 10.8.2.33. Configure 10.8.2.33 in a backup-liveness-detection configuration on the peer node.

For example, on the backup Routing Engine:

ICCP and ICL

Best Practice:

We recommend that you use separate ports and choose different Flexible PIC Concentrators (FPCs) for the interchassis link (ICL) and Inter-Chassis Control Protocol (ICCP) interfaces. Although you can use a single link for the ICCP interface, an aggregated Ethernet interface is preferred.

When configuring ICCP and ICL, we recommend that you:

  • Configure ICCP and the ICL on different interfaces, and configure prefer status control as active on the active MC-LAG peer.

    For example, issue the prefer-status-control-active statement on the active MC-LAG peer.

  • Configure the IP address for the management port (fxp0).

    When you configure backup liveness detection, this out-of-band channel is established between the peers through the management network.

  • Use the peer loopback address to establish ICCP peering. Doing so avoids any direct link failure between MC-LAG peers. As long as the logical connection between the peers remains up, ICCP stays up.

  • Configure the ICCP liveness-detection interval (the Bidirectional Forwarding Detection (BFD) timer) to be at least 8 seconds if you have configured ICCP connectivity through an IRB interface. A liveness-detection interval of 8 seconds or more allows for graceful Routing Engine switchover (GRES) to work seamlessly. By default, ICCP liveness detection uses multihop BFD, which runs in centralized mode.

    This recommendation does not apply if you configured ICCP connectivity through a dedicated physical interface. In this case, you can configure single-hop BFD.

  • Configure a session establishment hold time for ICCP. Doing this results in faster ICCP connection between the MC-LAG peers and also prevents any delay during convergence.

    Note:

    On QFX Series switches, the default session establishment hold time is 300 seconds. However, the session establishment time must be at least 100 seconds higher than the init delay time. You can optionally update the session establishment time to be 340 seconds and the init delay time to be 240 seconds.

  • For better convergence during a MC-LAG peer (status-control of active) reboot, we recommend that you configure the icl-down-delay time instead of the hold-time. The icl-down-delay time is the number of seconds that elapse between when the interchassis link (ICL) goes down and the multichassis aggregated Ethernet interfaces (MCAEs) change to standby mode.

  • Starting with Junos OS Release 15.1 on MX Series routers, configure the backup liveness detection feature to implement faster failover of data traffic during an MC-LAG peer reboot. Configure the backup-liveness-detection statement on the management interface (fxp0) only.

    Note:

    Liveness detection does not work if the management interface is within the management instance.
Note:

DHCP snooping, dynamic ARP inspection (DAI), and IP source guard are not supported on the ICL or MC-LAG interfaces. Consequently, incoming address resolution protocol replies on the ICL are discarded. However, ARP entries can be populated on the ICL interface through ICCP exchanges from a remote MC-LAG peer.

IGMP Snooping on an Active-Active MC-LAG

You must enable Protocol Independent Multicast (PIM) on the IRB interface to avoid multicast duplication.

You must configure the ICL interface as a router-facing interface (by configuring the multicast-router-interface statement) for multicast forwarding to work in an MC-LAG environment. For the scenario in which traffic arrives by way of a Layer 3 interface, PIM and IGMP must be enabled on the IRB or RVI interface configured on the MC-LAG peers. You must enable PIM on the IRB or RVI interface to avoid multicast duplication.

Init Delay Time

On QFX and EX Series switches, the default session establishment hold time is 300 seconds. However, the session establishment time must be at least 100 seconds higher than the init delay time. You can optionally update the session establishment time to be 340 seconds and the init delay time to be 240 seconds.

Label Swap Function

Note:

QFX Series switches configured as MC-LAG peers do not support vlans label swap function.

Miswiring Detection Guidelines

You can use STP to detect miswiring loops within the peer or across MC-LAG peers. An example of miswiring is when a port of a network element is accidentally connected to another port of the same network element. Using STP to detect loops on MC-LAG interfaces, however, is not supported.

Note:

Do not use Multiple Spanning Tree Protocol (MSTP) or VLAN Spanning Tree Protocol (VSTP). There could be a loop if MSTP or VSTP is enabled in an MC-AE topology without enabling MSTP or VSTP on the MC-AE logical interfaces. Also, there could be a loop if an alternate path exists from access nodes to MC-AE nodes.
Best Practice:

To detect miswirings, we recommend that you do the following:

  • Configure STP globally so that STP can detect local miswiring within and across MC-LAG peers.

  • Disable STP on ICL links, however, because STP might block ICL interfaces and disable protection.

  • Disable STP on interfaces that are connected to aggregation switches.

  • Configure MC-LAG interfaces as edge ports.

  • Enable bridge protocol data unit (BPDU) block on edge.

  • Do not enable BPDU block on interfaces connected to aggregation switches.

Configuration Guidelines and Caveats

  • Configure the IP address on the active MC-LAG peer with a high IP address or a high DR priority. To ensure that the active MC-LAG peer retains the DR membership designation if PIM neighborship with the peer goes down.

  • Using Bidirectional Forwarding Detection (BFD) and RVI or IRB MAC synchronization together is not supported because ARP fails.

  • When using RVI or IRB MAC synchronization, make sure that you configure the primary IP address on both MC-LAG peers. Doing this ensures that both MC-LAG peers cannot become assert winners.

  • The number of BFD sessions on RVIs or IRBs with PIM enabled is restricted to 100. Also, If you have more than 100 RVIs or IRBs configured, do not configure BFD, and make sure that the hello interval is 2 seconds.

Redundancy Groups

Best Practice:

We recommend that you configure only one redundancy group between MC-LAG nodes. The redundancy group represents the domain of high availability between the MC-LAG nodes. One redundancy group is sufficient between a pair of MC-LAG nodes. If you are using logical systems, then configure one redundancy group between MC-LAG nodes in each logical system.

Segment Routing

Note:

QFX Series switches configured as MC-LAG peers do not support segment routing.

Status Control

Best Practice:

You can configure prefer-status-control-active statement with the status-control standby configuration to prevent the LACP mc-ae system ID from reverting to the LACP default system ID during an ICCP failure. You should only configure this option if ICCP will never go down unless the remote peer goes down.

In the event that the peer configured with status-control active goes down abruptly, such as during a power off, we recommended that you configure the hold-time interval for the interchassis control link configured as status-control standby with a value greater than the ICCP BFD timeout. Doing this will prevent momentary traffic loss on the peer configured as status-control standby. Without the hold-time interval configuration on the ICL, the MC-AE interface on the peer configured as status-control standby will momentarily move to standby during a power-off of the remote peer.

Virtual Router Redundancy Protocol (VRRP) over IRB and MAC Address Synchronization

Note:

On EX9200 and QFX Series switches, routing protocols are not supported on the downstream clients.
Best Practice:

On EX9200 and QFX Series switches, we recommend that you use MAC address synchronization for the downstream clients. For the upstream routers, we recommend that you use VRRP over IRB or RVI method.

Note:

On EX9200 and QFX Series switches, you cannot configure both VRRP over IRB and MAC synchronization, because processing MAC addresses might not work.
Note:

Use MAC synchronization if you require more than 1,000 VRRP instances.
Note:

Here are some caveats with configuring MAC address synchronization:

  • Use MAC address synchronization if you are not planning to run routing protocols on the IRB interfaces.

    MAC address synchronization does not support routing protocols on IRB interfaces, and routing protocols are not supported with downstream MC-LAG clients. If you need routing capability, configure both VRRP and routing protocols on each MC-LAG peer. Routing protocols are supported on upstream routers.

  • DHCP relay is not supported with MAC address synchronization.

    If you need to configure DHCP relay, configure VRRP over IRB.

  • Gratuitous ARP requests are not sent when the MAC address on the IRB interface changes.

VLANs

Best Practice:

We recommend that you limit the scope of VLANs and configure them only where they are necessary. Configure the MC-AE trunk interfaces with only the VLANs that are necessary for the access layer.

On the QFX Series, the all option at the [edit interfaces name unit number ethernet-switching vlan] hierarchy is not supported on multichassis aggregated Ethernet (MC-AE), aggregated Ethernet (AE), Gigabit Ethernet (GE), 10-Gigabit Ethernet (XE), 40-Gigabit Ethernet (ET), and 100-Gigabit Ethernet (ET) Layer 2 Ethernet interfaces. Instead, specify a range of VLAN IDs or VLAN names. VLAN names or VLAN IDs for ICCP purposes are not supported on MC-AE, single-homed AE, GE, XE, and ET interfaces connected to servers or other network devices. VLAN names or VLAN IDs used for ICCP purposes are not supported on MC-AE multi-homed access and trunk Layer 2 interfaces, single-homed AE, GE, XE, and ET Layer 2 Ethernet access or trunk interfaces. When you configure VLANs on an MC-AE or other revenue interfaces on the device, the VLAN range is supported but without the ICCP-dedicated VLAN.

QFX Series switches (except QFX10002, QFX10008, and QFX10016), EX4600 switches, and EX4650 switches do not support service provider style of configuration for MC-LAG.

VRRP Active-Standby Support

You must configure VRRP on both MC-LAG peers for both the active and standby members to accept and route packets. Additionally, you must configure the VRRP backup device to send and receive ARP requests.

If you are using the VRRP over IRB or RVI method to enable Layer 3 functionality, you must configure static ARP entries for the IRB or RVI interface of the remote MC-LAG peer to allow routing protocols to run over the IRB or RVI interfaces.

Starting in Junos OS Release 15.2R1, you do not need to configure a static ARP or ND entry for the remote IRB IP address. If you have already manually configured a static ARP or ND entry and upgrade to a later release, the static entry is deleted when ICCP goes down. If you configured ICCP on the IRB static entry, then ICCP might not come up. As a workaround, you can disable the automatic creation of static ARP and ND entries by issuing the following command: set protocols l2-learning no-mclag-ifa-sync.

Change History Table

Feature support is determined by the platform and release you are using. Use Feature Explorer to determine if a feature is supported on your platform.

Release
Description
15.1
Starting with Junos OS Release 15.1 on MX Series routers, configure the backup liveness detection feature to implement faster failover of data traffic during an MC-LAG peer reboot.