Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

TechLibrary
Navigation
Table of Contents
Guide That Contains This Content
[+] Expand All
[-] Collapse All

Example: Configuring Multichassis Link Aggregation for Layer 3 Unicast using MAC Address Synchronization

There are two methods for enabling Layer 3 unicast functionality across a multichassis link aggregation group (MC-LAG). You can choose either to synchronize the MAC addresses for the Layer 3 interfaces of the switches participating in the MC-LAG , or you can configure Virtual Router Redundancy Protocol (VRRP). The procedure to configure MAC address synchronization is included in this example. For more information on configuring VRRP for use in a Layer 3 unicast MC-LAG, see Example: Configuring Multichassis Link Aggregation for Layer 3 Unicast Using VRRP.

Requirements

This example uses the following hardware and software components:

  • Junos OS Release 12.3 or later for the QFX Series
  • Two QFX3500 or QFX3600 switches

Before you configure an MC-LAG for Layer 3 unicast, be sure that you understand how to:

Overview

In this example, you configure an MC-LAG across two switches by including interfaces from both switches in an aggregated Ethernet interface (ae1). To support the MC-LAG, create a second aggregated Ethernet interface (ae0) for the interchassis control link-protection link (ICL-PL). Configure a multichassis protection link for the ICL-PL, Interchassis Control Protocol (ICCP) for the peers hosting the MC-LAG, and Layer 3 connectivity between MC-LAG peers.

Note: Layer 3 connectivity is required for ICCP.

To complete the configuration, configure MAC address synchronization between the peers and specify the same IP address on both Layer 3 interface members (also known as the routed VLAN interface [RVI] or the integrated routing and bridging (IRB) interface) in the MC-LAG VLAN.

Topology

The topology used in this example consists of two switches hosting an MC-LAG. The two switches are connected to a server. Figure 1 shows the topology of this example.

Figure 1: Configuring a Multichassis LAG Between Switch A and Switch B

Configuring a Multichassis LAG Between Switch A and Switch B

Table 1 details the topology used in this configuration example.

Table 1: Components of the Topology for Configuring a Multichassis LAG Between Two Switches

HostnameBase HardwareMultichassis Link Aggregation Group

Switch A

Switch B

QFX3500 switch or QFX3600 switch

QFX3500 switch or QFX3600 switch

ae0 is configured as an aggregated Ethernet interface, and is used as an ICL-PL. The following interfaces are part of ae0: xe-0/0/12 and xe-0/0/13 on Switch A and
xe-0/0/12 and xe-0/0/13 on Switch B. These interfaces are included in VLAN v500.

ae1 is configured as an MC-LAG, and the following two interfaces are part of ae1:
xe-0/0/44 on Switch A and
xe-0/0/46 on Switch B. These interfaces are included in VLAN v100.

Configuration

CLI Quick Configuration

To quickly configure this example, copy the following commands, paste them in a text file, remove any line breaks, change any details necessary to match your network configuration, and paste the commands into the CLI at the [edit] hierarchy level of Switch A.

set chassis aggregated-devices ethernet device-count 2
set interfaces xe-0/0/12 ether-options 802.3ad ae0
set interfaces xe-0/0/13 ether-options 802.3ad ae0
set interfaces xe-0/0/44 ether-options 802.3ad ae1
set interfaces ae0 unit 0 family ethernet-switching port-mode trunk
set interfaces ae0 unit 0 family ethernet-switching vlan members v500
set interfaces ae0 unit 0 family ethernet-switching vlan members v100
set interfaces ae1 aggregated-ether-options lacp active
set interfaces ae1 aggregated-ether-options lacp system-id 00:01:02:03:04:05
set interfaces ae1 aggregated-ether-options lacp admin-key 3
set interfaces ae1 aggregated-ether-options mc-ae mc-ae-id 3
set interfaces ae1 aggregated-ether-options mc-ae chassis-id 0
set interfaces ae1 aggregated-ether-options mc-ae mode active-active
set interfaces ae1 aggregated-ether-options mc-ae status-control active
set ae1 aggregated-ether-options mc-ae init-delay-time 240
set interfaces ae1 unit 0 family ethernet-switching port-mode trunk
set interfaces ae1 unit 0 family ethernet-switching vlan members v100
set interfaces vlan unit 100 family inet address 100.1.1.10
set interfaces vlan unit 500 family inet address 3.3.3.2/24
set vlans v100 vlan-id 100
set vlans v100 l3-interface vlan.100
set vlans v100 mcae-mac-synchronize
set vlans v500 vlan-id 500
set vlans v500 l3-interface vlan.500
set protocols iccp local-ip-addr 3.3.3.2
set protocols iccp peer 3.3.3.1 session-establishment-hold-time 50
set protocols iccp peer 3.3.3.1 backup-liveness-detection backup-peer-ip 10.207.64.233
set protocols iccp peer 3.3.3.1 liveness-detection minimum-receive-interval 60
set protocols iccp peer 3.3.3.1 liveness-detection transmit-interval minimum-interval 60
set protocols rstp interface ae0.0 disable
set protocols rstp interface ae1.0 edge
set protocols rstp interface all mode point-to-point
set protocols rstp bpdu-block-on-edge
set multi-chassis multi-chassis-protection 3.3.3.1 interface ae0

To quickly configure this example, copy the following commands, paste them in a text file, remove any line breaks, change any details necessary to match your network configuration, and paste the commands into the CLI at the [edit] hierarchy level of Switch B.

set chassis aggregated-devices ethernet device-count 2
set interfaces xe-0/0/12 ether-options 802.3ad ae0
set interfaces xe-0/0/13 ether-options 802.3ad ae0
set interfaces xe-0/0/46 ether-options 802.3ad ae1
set interfaces ae0 unit 0 family ethernet-switching port-mode trunk
set interfaces ae0 unit 0 family ethernet-switching vlan members v500
set interfaces ae0 unit 0 family ethernet-switching vlan members v100
set interfaces ae1 aggregated-ether-options lacp active
set interfaces ae1 aggregated-ether-options lacp system-id 00:01:02:03:04:05
set interfaces ae1 aggregated-ether-options lacp admin-key 3
set interfaces ae1 aggregated-ether-options mc-ae mc-ae-id 3
set interfaces ae1 aggregated-ether-options mc-ae chassis-id 1
set interfaces ae1 aggregated-ether-options mc-ae mode active-active
set interfaces ae1 aggregated-ether-options mc-ae status-control standby
set ae1 aggregated-ether-options mc-ae init-delay-time 240
set interfaces ae1 unit 0 family ethernet-switching port-mode trunk
set interfaces ae1 unit 0 family ethernet-switching vlan members v100
set interfaces vlan unit 100 family inet address 100.1.1.10
set interfaces vlan unit 500 family inet address 3.3.3.1/24
set vlans v100 vlan-id 100
set vlans v100 l3-interface vlan.100
set vlans v100 mcae-mac-synchronize
set vlans v500 vlan-id 500
set vlans v500 l3-interface vlan.500
set protocols iccp local-ip-addr 3.3.3.1
set protocols iccp peer 3.3.3.2 session-establishment-hold-time 50
set protocols iccp peer 3.3.3.2 backup-liveness-detection backup-peer-ip 10.207.64.234
set protocols iccp peer 3.3.3.2 liveness-detection minimum-receive-interval 60
set protocols iccp peer 3.3.3.2 liveness-detection transmit-interval minimum-interval 60
set protocols rstp interface ae0.0 disable
set protocols rstp interface ae1.0 edge
set protocols rstp interface all mode point-to-point
set protocols rstp bpdu-block-on-edge
set multi-chassis multi-chassis-protection 3.3.3.2 interface ae0

Configuring MC-LAG on Two Switches

Step-by-Step Procedure

To enable multichassis protection link between MC-LAG peers:

The following example requires you to navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode in the CLI User Guide.

  1. Configure the number of LAGs on both Switch A and Switch B.
    [edit chassis]
    user@switch# set aggregated-devices ethernet device-count 2
  2. Add member interfaces to the aggregated Ethernet interfaces on both Switch A and Switch B.

    Switch A and Switch B:

    [edit interfaces]
    user@switch# set xe-0/0/12 ether-options 802.3ad ae0
    [edit interfaces]
    user@switch# set xe-0/0/13 ether-options 802.3ad ae0

    Switch A

    [edit interfaces]
    user@switch# set xe-0/0/44 ether-options 802.3ad ae1

    Switch B

    [edit interfaces]
    user@switch# set xe-0/0/46 ether-options 802.3ad ae1
  3. Configure a trunk interface between Switch A and Switch B.
    [edit interfaces]
    user@switch# set ae0 unit 0 family ethernet-switching port-mode trunk
  4. Configure a multichassis protection link between Switch A and Switch B.

    Switch A:

    [edit]
    user@switch# set multi-chassis multi-chassis-protection 3.3.3.2 interface ae0

    Switch B:

    [edit]
    user@switch# set multi-chassis multi-chassis-protection 3.3.3.1 interface ae0

Step-by-Step Procedure

To enable ICCP:

The following example requires you to navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode in the CLI User Guide.

  1. Configure the local IP address to be in the ICCP connection on Switch A and Switch B.

    Switch A:

    [edit protocols]
    user@switch# set iccp local-ip-addr 3.3.3.2

    Switch B:

    [edit protocols]
    user@switch# set iccp local-ip-addr 3.3.3.1
  2. Configure the peer IP address and minimum receive interval for a (BFD) session for ICCP on Switch A and Switch B.

    Switch A:

    [edit protocols]
    user@switch# set iccp peer 3.3.3.1 liveness-detection minimum-receive-interval 60

    Switch B:

    [edit protocols]
    user@switch# set iccp peer 3.3.3.2 liveness-detection minimum-receive-interval 60
  3. Configure the peer IP address and minimum transmit interval for Bidirectional Forwarding Detection (BFD) session for ICCP on Switch A and Switch B.

    Switch A:

    [edit protocols]
    user@switch# set iccp peer 3.3.3.1 liveness-detection transmit-interval minimum-interval 60

    Switch B:

    [edit protocols]
    user@switch# set iccp peer 3.3.3.2 liveness-detection transmit-interval minimum-interval 60
  4. (Optional) Configure the time during which an ICCP connection must succeed between MC-LAG peers on Switch A and Switch B.

    Note: Configuring session establishment hold time helps in faster ICCP connection establishment. The recommended value is 50 seconds.

    Switch A:

    [edit protocols]
    user@switch# set iccp peer 3.3.3.1 session-establishment-hold-time 50

    Switch B:

    [edit protocols]
    user@switch# set iccp peer 3.3.3.2 session-establishment-hold-time 50
  5. (Optional) Configure the backup IP address to be used for backup liveness detection on both Switch A and Switch B.

    Note: By default, backup liveness detection is not enabled. Configuring a backup IP address helps achieve sub-second traffic loss during a MC-LAG peer reboot.

    Switch A:

    [edit protocols]
    user@switch# set iccp peer 3.3.3.1 backup-liveness-detection backup-peer-ip 10.207.64.233

    Switch B:

    [edit protocols]
    user@switch# set iccp peer 3.3.3.2 backup-liveness-detection backup-peer-ip 10.207.64.234
  6. Configure Layer 3 connectivity across the ae0 ICCP link by adding a Layer 3 interface on both Switch A and Switch B.
    [edit vlans]
    user@switch# set v500 vlan-id 500
    [edit vlans]
    user@switch# set v500 l3-interface vlan.500
    [edit interfaces]
    user@switch# set ae0 unit 0 family ethernet-switching port-mode trunk vlan members v500
    [edit interfaces]
    user@switch# set ae0 unit 0 family ethernet-switching port-mode trunk vlan members v100

Step-by-Step Procedure

To enable the MC-LAG interface:

The following example requires you to navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode in the CLI User Guide.

  1. Enable LACP on the MC-LAG interface on Switch A and Switch B.

    Note: At least one end needs to be active. The other end can be either active or passive.

    [edit interfaces]
    user@switch# set ae1 aggregated-ether-options lacp active
  2. Specify the same multichassis aggregated Ethernet identification number on both MC-LAG peers on Switch A and Switch B.
    [edit interfaces]
    user@switch# set ae1 aggregated-ether-options mc-ae mc-ae-id 3
  3. Specify a unique chassis ID for the MC-LAG on the MC-LAG peers on Switch A and Switch B.

    Switch A:

    [edit interfaces]
    user@switch# set ae1 aggregated-ether-options mc-ae chassis-id 0

    Switch B:

    [edit interfaces]
    user@switch# set ae1 aggregated-ether-options mc-ae chassis-id 1
  4. Specify the operating mode of the MC-LAG on both Switch A and Switch B.

    Note: Only active-active mode is supported at this time.

    [edit interfaces]
    user@switch# set ae1 aggregated-ether-options mc-ae mode active-active
  5. Specify the status control for MC-LAG on Switch A and Switch B.

    Note: You must configure status control on both Switch A and Switch B hosting the MC-LAG. If one peer is in active mode, the other must be in standby mode.

    Switch A:

    [edit interfaces]
    user@switch# set ae1 aggregated-ether-options mc-ae status-control active

    Switch B:

    [edit interfaces]
    user@switch# set ae1 aggregated-ether-options mc-ae status-control standby
  6. Specify the number of seconds by which the bring-up of the MC-AE interface should be deferred after you reboot Switch A and Switch B.

    Note: The recommended value for maximum VLAN configuration (for example, 4,000 VLANS) is 240 seconds. If IGMP snooping is enabled on all of the VLANs, the recommended value is 420 seconds.

    [edit interfaces]
    user@switch# set ae1 aggregated-ether-options mc-ae init-delay-time 240
  7. Specify the same LACP system ID for the MC-LAG on Switch A and Switch B.
    [edit interfaces]
    user@switch# set ae1 aggregated-ether-options lacp system-ID 00:01:02:03:04:05
  8. Specify the same LACP administration key on both Switch A and Switch B.
    [edit interfaces]
    user@switch# set ae1 aggregated-ether-options lacp admin-key 3
  9. Enable a VLAN on the MC-LAG on Switch A and Switch B.
    [edit interfaces]
    user@switch# set ae1 unit 0 family ethernet-switching port-mode trunk
    [edit]
    user@switch# set vlans v100 vlan-id 100
    [edit interfaces]
    user@switch# set ae1 unit 0 family ethernet-switching vlan members v100
  10. Create a Layer 3 interface for the MC-LAG VLAN and assign the same IP address on both Switch A and Switch B.
    [edit]
    user@switch# set vlans v100 l3-interface vlan.100
    [edit interfaces]
    user@switch# set vlan unit 100 family inet address 100.1.1.10
  11. Configure MAC address synchronization in the MC-LAG VLAN on both Switch A and Switch B.
    [edit]
    user@switch# set vlans v100 mcae-mac-synchronize

Step-by-Step Procedure

To enable RSTP:

The following example requires you to navigate various levels in the configuration hierarchy. For information about navigating the CLI, see Using the CLI Editor in Configuration Mode in the CLI User Guide.

  1. Enable RSTP globally on all interfaces on Switch A and Switch B.
    [edit]
    user@switch# set protocols rstp interface all mode point-to-point
  2. Disable RSTP on the ICL-PL interfaces on Switch A and Switch B:
    [edit]
    user@switch# set protocols rstp interface ae0.0 disable
  3. Configure the MC-LAG interfaces as edge ports on Switch A and Switch B.

    Note: The ae1 interface is a downstream interface. This is why RSTP and bpdu-block-on-edge need to be configured.

    [edit]
    user@switch# set protocols rstp interface ae1.0 edge
  4. Enable BPDU blocking on all interfaces except for the ICL-PL interfaces on Switch A and Switch B.

    Note: The ae1 interface is a downstream interface. This is why RSTP and bpdu-block-on-edge need to be configured.

    [edit]
    user@switch# set protocols rstp bpdu-block-on-edge

Results

Display the results of the configuration on Switch A.

chassis {aggregated-devices {ethernet {device-count 2;}}}
interfaces {xe-0/0/12 {ether-options {802.3ad ae0;}}xe-0/0/13 {ether-options {802.3ad ae0;}}xe-0/0/44 {ether-options {802.3ad ae1;}}ae0 {unit 0 {family ethernet-switching {port-mode trunk;vlan {members v500;}}}}ae1 {aggregated-ether-options {lacp {active;system-id 00:01:02:03:04:05;admin-key 3;}mc-ae {mc-ae-id 3;chassis-id 0;mode active-active;status-control active;init-delay-time 240}}unit 0 {family ethernet-switching {port-mode trunk;vlan {members v100;}}}}vlan {unit 100 {family inet {address 100.1.1.10/32;}}unit 500 {family inet {address 3.3.3.2/24;}}}}
protocols {iccp {local-ip-addr 3.3.3.2;peer 3.3.3.1 {session-establishment-hold-time 50;backup-liveness-detection {backup-peer-ip 10.207.64.233;}liveness-detection {minimum-receive-interval 60;transmit-interval {minimum-interval 60;}}}}rstp {interface ae0.0 {disable;}interface ae1.0 {edge;}interface all {mode point-to-point;}bpdu-block-on-edge;}}
multi-chassis {multi-chassis-protection 3.3.3.1 {interface ae0;}}
vlans {v100 {vlan-id 100;l3-interface vlan.100;mcae-mac-synchronize;}v500 {vlan-id 500;l3-interface vlan.500;}}

Display the results of the configuration on Switch B.

chassis {aggregated-devices {ethernet {device-count 2;}}}
interfaces {xe-0/0/12 {ether-options {802.3ad ae0;}}xe-0/0/13 {ether-options {802.3ad ae0;}}xe-0/0/46 {ether-options {802.3ad ae1;}}ae0 {unit 0 {family ethernet-switching {port-mode trunk;vlan {members v500;}}}}ae1 {aggregated-ether-options {lacp {active;system-id 00:01:02:03:04:05;admin-key 3;}mc-ae {mc-ae-id 3;chassis-id 1;mode active-active;status-control standby;init-delay-time 240}}unit 0 {family ethernet-switching {port-mode trunk;vlan {members v100;}}}}vlan {unit 100 {family inet {address 100.1.1.10/32;}}unit 500 {family inet {address 3.3.3.1/24;}}}}
protocols {iccp {local-ip-addr 3.3.3.1;peer 3.3.3.2 {session-establishment-hold-time 50;backup-liveness-detection {backup-peer-ip 10.207.64.234;}liveness-detection {minimum-receive-interval 60;transmit-interval {minimum-interval 60;}}}}rstp {interface ae0.0 {disable;}interface ae1.0 {edge;}interface all {mode point-to-point;}bpdu-block-on-edge;}}
multi-chassis {multi-chassis-protection 3.3.3.2 {interface ae0;}}
vlans {v100 {vlan-id 100;l3-interface vlan.100;mcae-mac-synchronize;}v500 {vlan-id 500;l3-interface vlan.500;}}

Verification

To verify that the MC-LAG group has been created and is working properly, perform these tasks:

Verifying That ICCP Is Working on Switch A

Purpose

Verify that ICCP is running on Switch A.

Action

[edit]
user@switch# show iccp
Redundancy Group Information for peer 3.3.3.1
  TCP Connection       : Established
  Liveliness Detection : Up

 Client Application: MCSNOOPD

 Client Application: eswd

Meaning

This output shows that the TCP connection between the peers hosting the MC-LAG is up, liveness detection is up, and MCSNOOPD and ESWD client applications are running.

Verifying That ICCP Is Working on Switch B

Purpose

Verify that ICCP is running on Switch B.

Action

show iccp

[edit]
user@switch# show iccp
Redundancy Group Information for peer 3.3.3.2
  TCP Connection       : Established
  Liveliness Detection : Up

 Client Application: MCSNOOPD

 Client Application: eswd

Meaning

This output shows that the TCP connection between the peers hosting the MC-LAG is up, liveness detection is up, and MCSNOOPD and ESWD client applications are running.

Verifying That LACP Is Active on Switch A

Purpose

Verify that LACP is active on Switch A.

Action

[edit]
user@switch# show lacp interfaces
Aggregated interface: ae1
    LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
      xe-0/0/46      Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
      xe-0/0/46    Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
    LACP protocol:        Receive State  Transmit State          Mux State
      xe-0/0/46                 Current   Fast periodic Collecting distributing

Meaning

This output shows that Switch A is participating in LACP negotiation.

Verifying That LACP Is Active on Switch B

Purpose

Verify that LACP is active on Switch B

Action

[edit]
user@switch# show lacp interfaces
Aggregated interface: ae1
    LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
      xe-0/0/44      Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
      xe-0/0/44    Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
    LACP protocol:        Receive State  Transmit State          Mux State
      xe-0/0/44                 Current   Fast periodic Collecting distributing

Meaning

This output shows that Switch B is participating in LACP negotiation.

Verifying That the MC-AE and ICL-PL Interfaces Are Up on Switch A

Purpose

Verify that the MC-AE and ICL-PL interfaces are up on Switch A.

Action

[edit]
user@switch# show interfaces mc-ae
Member Link                  : ae1
 Current State Machine's State: mcae active state
 Local Status                 : active
 Local State                  : up
 Peer Status                  : active
 Peer State                   : up
     Logical Interface        : ae1.0
     Topology Type            : bridge
     Local State              : up
     Peer State               : up
     Peer Ip/MCP/State        : 3.3.3.1 ae0.0 up

Meaning

This output shows that the MC-AE interface on Switch A is up and active.

Verifying That the MC-AE and ICL-PL Interfaces Are Up on Switch B

Purpose

Verify that the MC-AE and ICL-PL interfaces are up on Switch B.

Action

[edit]
user@switch# show interfaces mc-ae
Member Link                  : ae1
 Current State Machine's State: mcae active state
 Local Status                 : active
 Local State                  : up
 Peer Status                  : active
 Peer State                   : up
     Logical Interface        : ae1.0
     Topology Type            : bridge
     Local State              : up
     Peer State               : up
     Peer Ip/MCP/State        : 3.3.3.2 ae0.0 up

Meaning

This output shows that the MC-AE interface on Switch B is up and active.

Verifying MAC Address Synchronization on Switch A and Switch B

Purpose

Confirm that MAC address synchronization is working on both Switch A and Switch B.

Action

[edit]
user@switch# show ethernet-switching table vlan v100
Ethernet-switching table: 3 unicast entries
  VLAN	            MAC address       Type         Age Interfaces
  v100              *                 Flood          - All-members
  v100              84:18:88:df:35:36 Static         - Router
  v100              84:18:88:df:83:0a Static         - Router

Meaning

The output shows two static MAC addresses in VLAN v100. The addresses belong to the Layer 3 IRB/RVI interfaces of both Switch A and Switch B that you configured in the MC-LAG VLAN. Appearance of both addresses indicates that MAC address synchronization is working.

Troubleshooting

Troubleshooting a LAG That Is Down

Problem

The show interfaces terse command shows that the MC-LAG is down

Solution

Check the following:

  • Verify that there is no configuration mismatch.
  • Verify that all member ports are up.
  • Verify that the MC-LAG is part of family Ethernet switching (Layer 2 LAG).
  • Verify that the MC-LAG member is connected to the correct MC-LAG member at the other end.
 

Related Documentation

 

Published: 2015-03-16

Previous Page Next Page