Rate and give feedback:  Feedback Received. Thank You!

Rate and give feedback: 

X

This document helped resolve my issue

Yes No

Additional Comments

800 characters remaining

May we contact you if necessary?

Name:  

E-mail: 

Enter a valid Email ID

Need product assistance? Contact Juniper Support

Submit

This site is protected by hCaptcha and its Privacy Policy and Terms of Service apply.

English  

English

Example: Configuring Multichassis Link Aggregation for Layer 3 Unicast Using VRRP

Requirements

• Junos OS Release 12.3 or later for the QFX Series
• Two QFX3500 or QFX3600 or QFX5100 switches

• Configure aggregated Ethernet interfaces on a switch. See Example: Configuring Link Aggregation Between a QFX Series Product and an Aggregation Switch.
• Configure the Link Aggregation Control Protocol (LACP) on aggregated Ethernet interfaces on a switch. See Example: Configuring Link Aggregation with LACP Between a QFX Series Product and an Aggregation Switch.
• Configure Virtual Router Redundancy Protocol (VRRP) on a switch. See Configuring Basic VRRP Support.

Overview

In this example, you configure an MC-LAG across two switches by including interfaces from both switches in an aggregated Ethernet interface (ae1). To support the MC-LAG, create a second aggregated Ethernet interface (ae0) for the interchassis control link-protection link (ICL-PL). Configure a multichassis protection link for the ICL-PL, the Inter-Chassis Control Protocol (ICCP) for the peers hosting the MC-LAG, and Layer 3 connectivity between MC-LAG peers.

Note: Layer 3 connectivity is required for ICCP.

To complete the configuration, enable VRRP by completing the following steps:

• Create a routed VLAN interface (RVI)
• Create a VRRP group and assign a virtual IP address that is shared between each switch in the VRRP group
• Enable a member of a VRRP group to accept all packets destined for the virtual IP address if it is the master in the VRRP group
• Configure Layer 3 connectivity between the VRRP groups

Topology

The topology used in this example consists of two switches hosting an MC-LAGs. The two switches are connected to a server. Figure 1 shows the topology of this example.

Figure 1: Configuring a Multichassis LAG Between Switch A and Switch B

Table 1 details the topology used in this configuration example.

Configuration

CLI Quick Configuration

To quickly configure this example, copy the following commands, paste them in a text file, remove any line breaks, change any details necessary to match your network configuration, copy and paste the commands into the CLI at the [edit] hierarchy level , and then enter commit from configuration mode.

Note: This example shows how to configure MC-LAG using both the original CLI and Enhanced Layer 2 Software (ELS). In ELS, there are three different statements and one different option from the original CLI: The port-mode statement in the [edit interfaces interface-name unit number family ethernet-switching] hierarchy is not supported. Use the interface-mode statement instead. The vlan statement in the [edit interfaces interface-name] hierarchy is not supported. Use the irb statement instead. The vlan.logical-interface-number option in the [edit vlans vlan-name l3-interface] option is not supported. Use the irb.logical-interface-number option instead. The service-id statement in the [edit switch-options] hierarchy is required in the ELS CLI.

Configuring MC-LAG on Two Switches

Step-by-Step Procedure

1. Configure the number of LAGs on both Switch A and Switch B.
   [edit chassis]
   user@switch# set aggregated-devices ethernet device-count 2
2. Add member interfaces to the aggregated Ethernet interfaces on both Switch A and Switch B.

   Switch A and Switch B:

   [edit interfaces]
   user@switch# set xe-0/0/12 ether-options 802.3ad ae0[edit interfaces]
   user@switch# set xe-0/0/13 ether-options 802.3ad ae0

   Switch A:

   [edit interfaces]
   user@switch# set xe-0/0/44 ether-options 802.3ad ae1

   Switch B:

   [edit interfaces]
   user@switch# set xe-0/0/46 ether-options 802.3ad ae1
3. Configure a trunk interface between Switch A and Switch B using the original CLI.
   [edit interfaces]
   user@switch# set ae0 unit 0 family ethernet-switching port-mode trunk
4. Configure a trunk interface between Switch A and Switch B using ELS.
   [edit interfaces]
   user@switch# set ae0 unit 0 family ethernet-switching interface-mode trunk
5. Configure a multichassis protection link between Switch A and Switch B.

   Switch A:

   [edit]
   user@switch# set multi-chassis multi-chassis-protection 3.3.3.1 interface ae0

   Switch B:

   [edit]
   user@switch# set multi-chassis multi-chassis-protection 3.3.3.2 interface ae0

Step-by-Step Procedure

1. Configure the local IP address to be in the ICCP connection on Switch A and Switch B.

   Switch A:

   [edit protocols]
   user@switch# set iccp local-ip-addr 3.3.3.2

   Switch B:

   [edit protocols]
   user@switch# set iccp local-ip-addr 3.3.3.1
2. Configure the peer IP address and minimum receive interval for a Bidirectional Forwarding Detection (BFD) session for ICCP on Switch A and Switch B.

   Switch A:

   [edit protocols]
   user@switch# set iccp peer 3.3.3.1 liveness-detection minimum-receive-interval 1000

   Switch B:

   [edit protocols]
   user@switch# set iccp peer 3.3.3.2 liveness-detection minimum-receive-interval 1000
3. Configure the peer IP address and minimum transmit interval for a BFD session for ICCP on Switch A and Switch B.

   Switch A:

   [edit protocols]
   user@switch# set iccp peer 3.3.3.1 liveness-detection transmit-interval minimum-interval 1000

   Switch B:

   [edit protocols]
   user@switch# set iccp peer 3.3.3.2 liveness-detection transmit-interval minimum-interval 1000
4. (Optional) Configure the time during which an ICCP connection must succeed between MC-LAG peers on Switch A and Switch B.

   Note: Configuring session establishment hold time helps to establish a faster ICCP connection . The recommended value is 50 seconds.

   Switch A:

   [edit protocols]
   user@switch# set iccp peer 3.3.3.1 session-establishment-hold-time 50

   Switch B:

   [edit protocols]
   user@switch# set iccp peer 3.3.3.2 session-establishment-hold-time 50
5. (Optional) Configure the backup IP address to be used for backup liveness detection on both Switch A and Switch B.

   Note: By default, backup liveness detection is not enabled. Configuring a backup IP address helps achieve sub-second traffic loss during an MC-LAG peer reboot.

   Switch A:

   [edit protocols]
   user@switch# set iccp peer 3.3.3.1 backup-liveness-detection backup-peer-ip 10.207.64.233

   Switch B:

   [edit protocols]
   user@switch# set iccp peer 3.3.3.2 backup-liveness-detection backup-peer-ip 10.207.64.234
6. Configure Layer 3 connectivity between the MC-LAG peers on both Switch A and Switch B using the original CLI.
   [edit vlans]
   user@switch# set v500 vlan-id 500
   [edit vlans]
   user@switch# set v500 l3-interface vlan.500
   [edit interfaces]
   user@switch# set ae0 unit 0 family ethernet-switching port-mode trunk vlan members v500 v100
7. Configure Layer 3 connectivity between the MC-LAG peers on both Switch A and Switch B using ELS.
   edit vlans]
   user@switch# set v500 vlan-id 500
   [edit vlans]
   user@switch# set v500 l3-interface irb.500
   [edit interfaces]
   user@switch# set ae0 unit 0 family ethernet-switching interface-mode trunk vlan members v500 v100

Step-by-Step Procedure

1. Enable LACP on the MC-LAG interface on Switch A and Switch B.

   Note: At least one end needs to be active. The other end can be either active or passive.

   [edit interfaces]
   user@switch# set ae1 aggregated-ether-options lacp active
2. Specify the same multichassis aggregated Ethernet identification number on both MC-LAG peers on Switch A and Switch B.
   [edit interfaces]
   user@switch# set ae1 aggregated-ether-options mc-ae mc-ae-id 3
3. Specify the same service ID on Switch A and Switch B.

   ELS:

   [edit]
   user@switch# set switch-options service-id 10
4. Specify a unique chassis ID for the MC-LAG on the MC-LAG peers on Switch A and Switch B.

   Switch A:

   [edit interfaces]
   user@switch# set ae1 aggregated-ether-options mc-ae chassis-id 0

   Switch B:

   [edit interfaces]
   user@switch# set ae1 aggregated-ether-options mc-ae chassis-id 1
5. Specify the operating mode of the MC-LAG on both Switch A and Switch B.

   Note: Only active-active mode is supported at this time.

   [edit interfaces]
   user@switch# set ae1 aggregated-ether-options mc-ae mode active-active
6. Specify the status control for MC-LAG on Switch A and Switch B.

   Note: You must configure status control on both Switch A and Switch B hosting the MC-LAG. If one peer is in active mode, the other must be in standby mode.

   Switch A:

   [edit interfaces]
   user@switch# set ae1 aggregated-ether-options mc-ae status-control active

   Switch B:

   [edit interfaces]
   user@switch# set ae1 aggregated-ether-options mc-ae status-control standby
7. Specify the number of seconds by which the bring-up of the multichassis aggregated Ethernet interface should be deferred after you reboot Switch A and Switch B.

   Note: The recommended value for maximum VLAN configuration (for example, 4,000 VLANS) is 240 seconds. If IGMP snooping is enabled on all of the VLANs, the recommended value is 420 seconds.

   [edit interfaces]
   user@switch# set ae1 aggregated-ether-options mc-ae init-delay-time 240
8. Specify the same LACP system ID for the MC-LAG on Switch A and Switch B.
   [edit interfaces]
   user@switch# set ae1 aggregated-ether-options lacp system-ID 00:01:02:03:04:05
9. Specify the same LACP administration key on both Switch A and Switch B.
   [edit interfaces]
   user@switch# set ae1 aggregated-ether-options lacp admin-key 3
10. Enable a VLAN on the MC-LAG on Switch A and Switch B using the original CLI.
    [edit interfaces]
    user@switch# set ae1 unit 0 family ethernet-switching port-mode trunk
    [edit]
    user@switch# set vlans v100 vlan-id 100
    [edit interfaces]
    user@switch# set ae1 unit 0 family ethernet-switching vlan members v100
11. Enable a VLAN on the MC-LAG on Switch A and Switch B using ELS.
    [edit interfaces]
    user@switch# set ae1 unit 0 family ethernet-switching interface-mode trunk
    [edit]
    user@switch# set vlans v100 vlan-id 100
    [edit interfaces]
    user@switch# set ae1 unit 0 family ethernet-switching vlan members v100
12. Enable VRRP on the MC-LAG on Switch A and Switch B:
    • Create a routed VLAN interface (RVI), assign a virtual IP address that is shared between each switch in the VRRP group, and assign an individual IP address for each switch in the VRRP group:

      Switch A:

      [edit interfaces]
      user@switch# set vlan unit 100 family inet address 100.1.1.11/24 vrrp-group 1 virtual-address 100.1.1.1

      Switch B:

      [edit interfaces]
      user@switch# set vlan unit 100 family inet address 100.1.1.10/24 vrrp-group 1 virtual-address 100.1.1.1
    • Assign the priority for each switch in the VRRP group:

      Note: The switch configured with the highest priority is the master.

      Switch A:

      [edit interfaces]
      user@switch# set vlan unit 100 family inet address 100.1.1.11/24 vrrp-group 1 priority 200

      Switch B:

      [edit interfaces]
      user@switch# set vlan unit 100 family inet address 100.1.1.10/24 vrrp-group 1 priority 150
    • Enable the switch to accept all packets destined for the virtual IP address if it is the master in the VRRP group:

      Switch A:

      [edit interfaces]
      user@switch# set vlan unit 100 family inet address 100.1.1.11/24 vrrp-group 1 accept-data

      Switch B:

      [edit interfaces]
      user@switch# set vlan unit 100 family inet address 100.1.1.10/24 vrrp-group 1 accept-data
    • Configure Layer 3 connectivity between Switch A and Switch B.
      [edit interfaces]
      user@switch# set vlans v100 l3-interface vlan.100
13. Enable VRRP on the MC-LAG on Switch A and Switch B using ELS:
    • Create a routed VLAN interface (RVI), assign a virtual IP address that is shared between each switch in the VRRP group, and assign an individual IP address for each switch in the VRRP group:

      Switch A:

      [edit interfaces]
      user@switch# set irb unit 100 family inet address 100.1.1.11/24 vrrp-group 1 virtual-address 100.1.1.1

      Switch B:

      [edit interfaces]
      user@switch# set irb unit 100 family inet address 100.1.1.10/24 vrrp-group 1 virtual-address 100.1.1.1
    • Assign the priority for each switch in the VRRP group:

      Note: The switch configured with the highest priority is the master.

      Switch A:

      [edit interfaces]
      user@switch# set irb unit 100 family inet address 100.1.1.11/24 vrrp-group 1 priority 200

      Switch B:

      [edit interfaces]
      user@switch# set irb unit 100 family inet address 100.1.1.10/24 vrrp-group 1 priority 150
    • Enable the switch to accept all packets destined for the virtual IP address if it is the master in the VRRP group:

      Switch A:

      [edit interfaces]
      user@switch# set irb unit 100 family inet address 100.1.1.11/24 vrrp-group 1 accept-data

      Switch B:

      [edit interfaces]
      user@switch# set irb unit 100 family inet address 100.1.1.10/24 vrrp-group 1 accept-data
    • Configure Layer 3 connectivity between Switch A and Switch B.
      [edit interfaces]
      user@switch# set irb v100 l3-interface irb.100

Step-by-Step Procedure

1. Enable RSTP globally on all interfaces on Switch A and Switch B.

   Note: The all option is not available on ELS, so you cannot issue this command on ELS.

   [edit]
   user@switch# set protocols rstp interface all mode point-to-point

   ELS:

   [edit]
   user@switch# set protocols rstp interface ae1.0 mode point-to-point
2. Disable RSTP on the ICL-PL interfaces on Switch A and Switch B.

   Note: This command is not needed on ELS.

   [edit]
   user@switch# set protocols rstp interface ae0.0 disable
3. Configure the MC-LAG interfaces as edge ports on Switch A and Switch B.

   Note: The ae1 interface is a downstream interface. This is why RSTP and bpdu-block-on-edge need to be configured.

   [edit]
   user@switch# set protocols rstp interface ae1.0 edge
4. Enable BPDU blocking on all interfaces except for the ICL-PL interfaces on Switch A and Switch B.

   Note: The ae1 interface is a downstream interface. This is why RSTP and bpdu-block-on-edge need to be configured.

   [edit]
   user@switch# set protocols rstp bpdu-block-on-edge

Results

From configuration mode , confirm your configuration by entering the show chassis, show interfaces, show protocols, show multi-chassis, and show vlans commands. If the output does not display the intended configuration, repeat the instructions in this example to correct the configuration.

Switch A—Original CLI

Switch A—ELS

Switch B—Original CLI

Switch B—ELS

Verification

Verify that the configuration is working properly.

• Verifying That ICCP Is Working on Switch A
• Verifying That ICCP Is Working on Switch B
• Verifying That LACP Is Active on Switch A
• Verifying That LACP Is Active on Switch B
• Verifying That the multichassis aggregated Ethernet and ICL-PL Interfaces Are Up on Switch A
• Verifying That the Multichassis Aggregated Ethernet and ICL-PL Interfaces Are Up on Switch B
• Verifying that MAC Learning Is Occurring on Switch A
• Verifying that MAC Learning Is Occurring on Switch B
• Verifying that Switch A is the Master in the VRRP Group
• Verifying that Switch B is the Backup Member in the VRRP Group
• Verifying that the Virtual IP Address is Attached to an Individual Address on Switch A
• Verifying that the Virtual IP Address is Attached to an Individual Address on Switch B

Verifying That ICCP Is Working on Switch A

Purpose

Verify that ICCP is running on Switch A.

Action

Redundancy Group Information for peer 3.3.3.1
  TCP Connection       : Established
  Liveliness Detection : Up

 Client Application: MCSNOOPD

 Client Application: eswd

Meaning

This output shows that the TCP connection between the peers hosting the MC-LAG is up, liveness detection is up, and MCSNOOPD and ESWD client applications are running.

Verifying That ICCP Is Working on Switch B

Purpose

Verify that ICCP is running on Switch B.

Action

show iccp

Redundancy Group Information for peer 3.3.3.2
  TCP Connection       : Established
  Liveliness Detection : Up

 Client Application: MCSNOOPD

 Client Application: eswd

Meaning

This output shows that the TCP connection between the peers hosting the MC-LAG is up, liveness detection is up, and MCSNOOPD and ESWD client applications are running.

Verifying That LACP Is Active on Switch A

Purpose

Verify that LACP is active on Switch A.

Action

Aggregated interface: ae1
    LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
      xe-0/0/46      Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
      xe-0/0/46    Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
    LACP protocol:        Receive State  Transmit State          Mux State
      xe-0/0/46                 Current   Fast periodic Collecting distributing

Meaning

This output shows that Switch A is participating in LACP negotiation.

Verifying That LACP Is Active on Switch B

Purpose

Verify that LACP is active on Switch B.

Action

Aggregated interface: ae1
    LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
      xe-0/0/44      Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
      xe-0/0/44    Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
    LACP protocol:        Receive State  Transmit State          Mux State
      xe-0/0/44                 Current   Fast periodic Collecting distributing

Meaning

This output shows that Switch B is participating in LACP negotiation.

Verifying That the multichassis aggregated Ethernet and ICL-PL Interfaces Are Up on Switch A

Purpose

Verify that the multichassis aggregated Ethernet and Inter-chassis Link Protection (ICL-PL) interfaces are up on Switch A.

Action

Member Link                  : ae1
 Current State Machine's State: mcae active state
 Local Status                 : active
 Local State                  : up
 Peer Status                  : active
 Peer State                   : up
     Logical Interface        : ae1.0
     Topology Type            : bridge
     Local State              : up
     Peer State               : up
     Peer Ip/MCP/State        : 3.3.3.1 ae0.0 up

Meaning

This output shows that the multichassis aggregated Ethernet and ICL-PL on Switch A is up and active.

Verifying That the Multichassis Aggregated Ethernet and ICL-PL Interfaces Are Up on Switch B

Purpose

Verify that the multichassis aggregated Ethernet and ICL-PL interfaces are up on Switch B.

Action

Member Link                  : ae1
 Current State Machine's State: mcae active state
 Local Status                 : active
 Local State                  : up
 Peer Status                  : active
 Peer State                   : up
     Logical Interface        : ae1.0
     Topology Type            : bridge
     Local State              : up
     Peer State               : up
     Peer Ip/MCP/State        : 3.3.3.2 ae0.0 up

Meaning

This output shows that the multichassis aggregated Ethernet and ICL-PL interface on Switch B is up and active.

Verifying that MAC Learning Is Occurring on Switch A

Purpose

Verify that MAC learning is working on Switch A.

Action

Ethernet-switching table: 6 entries, 1 learned, 0 persistent entriesC
  VLAN              MAC address       Type         Age Interfaces
  v100              *                 Flood          - All-members
  v100              00:00:5e:00:01:01 Static         - Router
  v100              78:fe:3d:5a:07:42 Static         - Router
  v100              78:fe:3d:5b:ad:c2 Learn(R)        0 ae0.0
  v500              *                 Flood          - All-members
  v500              78:fe:3d:5a:07:42 Static         - Router

Meaning

The output shows two static MAC address in VLAN v100 and one static MAC address in VLAN v500. These addresses belong to the Layer 3 RVI addresses on both Switch A and Switch B that you configured in the MC-LAG. The ICL-PL interface configured on the VRRP master member learned the VLAN v100 Learn (R) MAC address of the VRRP backup member.

Verifying that MAC Learning Is Occurring on Switch B

Purpose

Verify that MAC learning is working on Switch B.

Action

Ethernet-switching table: 7 entries, 1 learned, 0 persistent entries
  VLAN              MAC address       Type         Age Interfaces
  v100              *                 Flood          - All-members
  v100              00:00:5e:00:01:01 Static         - Router
  v100              78:fe:3d:5a:07:42 Learn(R)        0 ae0.0
  v100              78:fe:3d:5b:ad:c2 Static         - Router
  v200              78:fe:3d:5b:ad:c2 Static         - Router
  v500              *                 Flood          - All-members
  v500              78:fe:3d:5b:ad:c2 Static         - Router

Meaning

The output shows two static MAC address in VLAN v100 and one static MAC address in VLAN v500. These addresses belong to the Layer 3 RVI addresses on both Switch A and Switch B that you configured in the MC-LAG. The ICL-PL interface configured on the VRRP backup member learned the VLAN v100 Learn (R) MAC address of the VRRP master member.

Verifying that Switch A is the Master in the VRRP Group

Purpose

Verify that Switch A is the master member in the VRRP group.

Action

Interface     State       Group   VR state VR Mode   Timer    Type   Address
vlan.100      up              1   master   Active      A  0.605 lcl    100.1.1.11     
                                                                vip    100.1.1.1     

Meaning

The output shows that Switch A is the master member in the VRRP group.

Verifying that Switch B is the Backup Member in the VRRP Group

Purpose

Verify that Switch B is the backup member in the VRRP group.

Action

Interface     State       Group   VR state VR Mode   Timer    Type   Address
vlan.100      up              1   backup   Active      A  0.605 lcl    100.1.1.10     
                                                                vip    100.1.1.1     

Meaning

The output shows that Switch B is the backup member in the VRRP group.

Verifying that the Virtual IP Address is Attached to an Individual Address on Switch A

Purpose

Action

Interface               Admin Link Proto    Local                 Remote
vlan                    up    up  
vlan.100                up    up   inet     100.1.1.1/24    
                                            100.1.1.11/24   
vlan.500                up    up   inet     3.3.3.2/24      

Meaning

The output shows that the virtual IP address (100.1.1.1/24) is bound to the individual IP address (100.1.1.11/24) on Switch A.

Verifying that the Virtual IP Address is Attached to an Individual Address on Switch B

Purpose

Action

Interface               Admin Link Proto    Local                 Remote
vlan                    up    up  
vlan.100                up    up   inet     100.1.1.1/24    
                                            100.1.1.10/24   
vlan.500                up    up   inet     3.3.3.1/24      

Meaning

The output shows that the virtual IP address (100.1.1.1/24) is bound to the individual IP address (100.1.1.10/24) on Switch B.

Troubleshooting

Troubleshooting a LAG That Is Down

Problem

The show interfaces terse command shows that the MC-LAG is down.

Solution

Check the following:

1. Verify that there is no configuration mismatch.
2. Verify that all member ports are up.
3. Verify that the MC-LAG is part of family Ethernet switching (Layer 2 LAG).
4. Verify that the MC-LAG member is connected to the correct MC-LAG member at the other end.