close

keyboard_arrow_left

Table of Contents Expand all

play_arrow How to Perform a QFX Series MC-LAG Fabric Upgrade
- QFX Series MC-LAG Fabric Upgrade Procedure

list Table of Contents

English

QFX Series MC-LAG Fabric Upgrade Procedure

date_range 02-Aug-23

arrow_backward

arrow_forward

About This Network Configuration Example

This network configuration example (NCE) shows how to manually upgrade an MC-LAG pair of QFX series devices. This process minimizes service disruption and has minimal impact on data center workloads.

Use Case Overview

To eliminate the access switch as a single point of failure in a data center environment, multichassis link aggregation groups (MC-LAGs) enable a client device to form a logical LAG interface between two MC-LAG peers. An MC-LAG provides redundancy and load balancing between the two MC-LAG peers, multihoming support, and a loop-free Layer 2 network without running STP. This example uses a basic MC-LAG configuration, but you can use this process for many different use cases.

This example does not cover how to perform a non-stop software upgrade (NSSU).

Technical Overview

Manually upgrading MC-LAG peers is similar to an NSSU. The manual upgrade process uses a high-availability design to systematically remove one device from service in order to perform the upgrade and then reboot. When servers are dual-homed to each MC-LAG peer, the network can handle the removal of one of the MC-LAG peers during the upgrade window. There’s a reduction of overall network bandwidth during the process, but the network remains available.

The MC-LAG is in active-active state and uses the ICCP protocol to keep the device state synchronized between the members of the MC-LAG. While one peer handles the traffic, the other peer is taken offline to upgrade the software.

Figure 1 illustrates a basic MC-LAG topology.

Figure 1: Basic MC-LAG Topology

Here’s the sequence of events that occur during an upgrade between two MC-LAG peers (Node 1 and Node 2):

All traffic is shifted from Node 1 to Node 2.
Node 1 is no longer handling traffic, so the MC-LAG is no longer operational.
Software is installed on Node 1 and then reboots.
Node 1 comes online, and all traffic is shifted from Node 2 to Node 1.
Software is installed on Node 2 and then reboots.
When Node 2 is online, the MC-LAG interfaces are re-enabled between the Node 1 and Node 2.

How to Perform a QFX Series MC-LAG Fabric Upgrade

Requirements
Overview
QFX Series MC-LAG Fabric Upgrade Configuration
Upgrade the QFX Series MC-LAG Fabric
Conclusion
Device Configuration Details

Requirements

This example uses the following hardware and software components:

Two QFX5100 devices running Junos OS Release 18.2R3-S3
Junos OS Release 18.4R3.3
A test server running Ubuntu Linux 16.04

Overview

To ensure a minimum of downtime, upgrading between software releases requires a sequence of steps coordinated among all of the network elements This topology uses servers with redundant connections to the MC-LAG to achieve high-availability during the switch over between MC-LAG peers.

To upgrade the fabric to a new version of Junos OS with minimal traffic disruption, you need to disable the MC-LAG and upgrade the MC-LAG peers as standalone units. After the software has been upgraded on both MC-LAG peers, you will re-connect them and re-establish the MC-LAG.

Topology

Figure 2 illustrates the MC-LAG topology referred to in this example.

Figure 2: Topology

QFX Series MC-LAG Fabric Upgrade Configuration

Prepare for the Upgrade

Step-by-Step Procedure

Use this procedure to upgrade both peers of a MC-LAG fabric consisting of QFX5100 switches to the same Junos OS Release version. We strongly recommend that both members of the MC-LAG are the same platform.

This configuration example shows how to manually upgrade MC-LAG peers from Junos OS Release 18.2R3-S3 to Junos OS Release 18.4R3.3.

Verify that the MC-LAG state is operational between both MC-LAG peers by checking the MC-LAG parameters.

content_copy zoom_out_map
user@QFX5100-A> show interfaces mc-ae 
 Member Link                  : ae1
 Current State Machine's State: mcae active state
 Local Status                 : active
 Local State                  : up
 Peer Status                  : active
 Peer State                   : up
     Logical Interface        : ae1.0
     Topology Type            : bridge
     Local State              : up
     Peer State               : up
     Peer Ip/MCP/State        : 10.3.3.1 ae0.0 up

content_copy zoom_out_map
user@QFX5100-A> show iccp
Redundancy Group Information for peer 10.3.3.1
  TCP Connection       : Established
  Liveliness Detection : Up
  Backup liveness peer status: Up
  Redundancy Group ID          Status
    1                           Up   
Client Application: lacpd
  Redundancy Group IDs Joined: 1 

Client Application: l2ald_iccpd_client
  Redundancy Group IDs Joined: 1

content_copy zoom_out_map
user@QFX5100-A> show lacp interfaces ae0 
Aggregated interface: ae0
    LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
      xe-0/0/8       Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
      xe-0/0/8     Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
      xe-0/0/9       Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
      xe-0/0/9     Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
    LACP protocol:        Receive State  Transmit State          Mux State 
      xe-0/0/8                  Current   Fast periodic Collecting distributing
      xe-0/0/9                  Current   Fast periodic Collecting distributing

content_copy zoom_out_map
user@QFX5100-A> show lacp interfaces ae1
Aggregated interface: ae1
    LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
      xe-0/0/10      Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
      xe-0/0/10    Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
    LACP protocol:        Receive State  Transmit State          Mux State 
      xe-0/0/10                 Current   Fast periodic Collecting distributing

Upgrade the QFX Series MC-LAG Fabric

Procedure

Step-by-Step Procedure

Copy the new Junos OS software image to the /var/tmp directories on both peers.

Copying the software on both MC-LAG peers stages the software for the upgrade procedure. The copy operation takes some time to complete while it transfers the Junos OS software images from the server to the MC-LAG peers.
```
user@QFX5100-A> file copy http://server.juniper.net/volume/download/docroot/software/junos/18.4R3.3/jinstall-host-qfx-5-18.4R3.3-signed.tgz /var/tmp/
 
user@QFX5100-B> file copy http://server.juniper.net/volume/download/docroot/software/junos/18.4R3.3/jinstall-host-qfx-5-18.4R3.3-signed.tgz /var/tmp/
 
```
Disable the server-facing interfaces on QFX5100-A to minimize disruption during the switch over to QFX5100-B.
```
user@QFX5100-A# set interfaces xe-0/1/10 disable
 
user@QFX5100-A# commit and-quit
```
Figure 3: Disabling the Server-Facing Interface on QFX5100-A
Disable the uplink interfaces on QFX5100-A.
```
user@QFX5100-A# set interfaces et-0/0/52 disable 

user@QFX5100-A# commit and-quit
```
Figure 4: Disabling the Uplink Interface on QFX5100-A

Disable the interfaces between the QFX5100-A and QFX5100-B.

This breaks up the MC-LAG.

content_copy zoom_out_map
user@QFX5100-A# set interfaces xe-0/0/8 disable
user@QFX5100-A# set interfaces xe-0/0/9 disable
user@QFX5100-A# commit and-quit

content_copy zoom_out_map
user@QFX5100-B# set interfaces xe-0/0/8 disable
user@QFX5100-B# set interfaces xe-0/0/9 disable
user@QFX5100-B# commit and-quit

Figure 5: Disabling Interfaces Between QFX5100-A and QFX5100-B

Upgrade QFX5100-A.

content_copy zoom_out_map
user@QFX5100-A> request system software add /var/tmp/jinstall-host-qfx-5-18.4R3.3-signed.tgz reboot 

Figure 6: Upgrading QFX5100-A

To redirect the traffic from QFX5100-B to QFX5100-A, re-enable the server-facing and uplink interfaces on QFX5100-A.
```
user@QFX5100-A# delete interfaces xe-0/0/10 disable
user@QFX5100-A# delete interfaces et-0/0/52 disable
user@QFX5100-A# commit and-quit
```
Figure 7: Re-enabling Server-Facing and Uplink Interfaces
Disable the server-facing interfaces on QFX5100-B.
```
user@QFX5100-B# set interfaces xe-0/0/10 disable
user@QFX5100-B# commit and-quit
```
Figure 8: Disabling Server-Facing Interfaces on QFX5100-B
Disable the uplink interfaces on QFX5100-B, so that the traffic goes through QFX5100-A.
```
user@QFX5100-B# set interfaces et-0/0/52 disable
user@QFX5100-B# commit and-quit
```
Figure 9: Disabling Uplink Interfaces on QFX5100-B

Upgrade QFX5100-B.

content_copy zoom_out_map
user@QFX5100-B> request system software add /var/tmp/jinstall-host-qfx-5-18.4R3.3-signed.tgz reboot 

Figure 10: Upgrading QFX5100-B

Re-enable the ICCP-PL interface between QFX5100-A and QFX5100-B.

content_copy zoom_out_map
user@QFX5100-A# delete interfaces xe-0/0/8 disable
user@QFX5100-A# delete interfaces xe-0/0/9 disable
user@QFX5100-A# commit and-quit

content_copy zoom_out_map
user@QFX5100-B# delete interfaces xe-0/0/8 disable
user@QFX5100-B# delete interfaces xe-0/0/9 disable
user@QFX5100-B# commit and-quit

Re-enable the server-facing and uplink interfaces on QFX5100-B.

content_copy zoom_out_map
user@QFX5100-B# delete interfaces xe-0/0/10 disable
user@QFX5100-B# delete interfaces et-0/0/52 disable
user@QFX5100-B# commit and-quit

Verification

Verify that the MC-LAG Fabric is Operational
Verify that the New Version of Junos OS is Installed

Verify that the MC-LAG Fabric is Operational

Purpose
Action
Meaning

Purpose

Verify that the MC-LAG Fabric is operational.

Action

content_copy zoom_out_map
user@QFX5100-A> show interfaces mc-ae 
 Current State Machine's State: mcae active state
 Local Status                 : active
 Local State                  : up
 Peer Status                  : active
 Peer State                   : up
     Logical Interface        : ae1.0
     Topology Type            : bridge
     Local State              : up
     Peer State               : up
     Peer Ip/MCP/State        : 10.3.3.1 ae0.0 up

content_copy zoom_out_map
user@QFX5100-A> show iccp
Redundancy Group Information for peer 10.3.3.1
  TCP Connection       : Established
  Liveliness Detection : Up
  Backup liveness peer status: Up
  Redundancy Group ID          Status
    1                           Up   

Client Application: lacpd
  Redundancy Group IDs Joined: 1 

Client Application: l2ald_iccpd_client
  Redundancy Group IDs Joined: 1

content_copy zoom_out_map
user@QFX5100-A> show lacp interfaces ae0
Aggregated interface: ae0
    LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
      xe-0/0/8       Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
      xe-0/0/8     Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
      xe-0/0/9       Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
      xe-0/0/9     Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
    LACP protocol:        Receive State  Transmit State          Mux State 
      xe-0/0/8                  Current   Fast periodic Collecting distributing
      xe-0/0/9                  Current   Fast periodic Collecting distributing

content_copy zoom_out_map
user@QFX5100-A> show lacp interfaces ae1
Aggregated interface: ae1
    LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
      xe-0/0/10      Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
      xe-0/0/10    Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
    LACP protocol:        Receive State  Transmit State          Mux State 
      xe-0/0/10                 Current   Fast periodic Collecting distributing

Meaning

You can see that the MC-LAG is operational because the MC-AE interface and ICCP connections are up.

Verify that the New Version of Junos OS is Installed

Purpose
Action
Meaning

Purpose

Verify that the new version of Junos OS is installed on QFX5100-A and QFX5100-B.

Action

content_copy zoom_out_map
user@QFX5100-A> show version
fpc0:
--------------------------------------------------------------------------
Hostname: QFX5100-A
Model: qfx5100-48s-6q
Junos: 18.4R3.3
JUNOS Base OS Software Suite [18.4R3.3]
JUNOS Base OS boot [18.4R3.3]
JUNOS Crypto Software Suite [18.4R3.3]
JUNOS Crypto Software Suite [18.4R3.3]
JUNOS Online Documentation [18.4R3.3]
JUNOS Kernel Software Suite [18.4R3.3]
JUNOS Packet Forwarding Engine Support (qfx-ex-x86-32) [18.4R3.3]
JUNOS Routing Software Suite [18.4R3.3]
JUNOS jsd [i386-18.4R3.3-jet-1]
JUNOS SDN Software Suite [18.4R3.3]
JUNOS Enterprise Software Suite [18.4R3.3]
JUNOS Web Management Platform Package [18.4R3.3]
JUNOS Openconfig [18.4R3.3]
JUNOS py-base-i386 [18.4R3.3]
JUNOS py-extensions-i386 [18.4R3.3]
JUNOS Host Software [17.3R3.9]

content_copy zoom_out_map
user@QFX5100-B> show version
fpc0:
--------------------------------------------------------------------------
Hostname: QFX5100-B
Model: qfx5100-48s-6q
Junos: 18.4R3.3
JUNOS Base OS Software Suite [18.4R3.3]
JUNOS Base OS boot [18.4R3.3]
JUNOS Crypto Software Suite [18.4R3.3]
JUNOS Crypto Software Suite [18.4R3.3]
JUNOS Online Documentation [18.4R3.3]
JUNOS Kernel Software Suite [18.4R3.3]
JUNOS Packet Forwarding Engine Support (qfx-ex-x86-32) [18.4R3.3]
JUNOS Routing Software Suite [18.4R3.3]
JUNOS jsd [i386-18.4R3.3-jet-1]
JUNOS SDN Software Suite [18.4R3.3]
JUNOS Enterprise Software Suite [18.4R3.3]
JUNOS Web Management Platform Package [18.4R3.3]
JUNOS Openconfig [18.4R3.3]
JUNOS py-base-i386 [18.4R3.3]
JUNOS py-extensions-i386 [18.4R3.3]
JUNOS Host Software [17.3R3.9]

Meaning

You can see that Junos OS 18.4R3.3 is installed on QFX5100-A and QFX5100-B.

Conclusion

Manually upgrading QFX Series MC-LAG Fabric

Step-by-Step Procedure

Device Configuration Details

Step-by-Step Procedure

This is the MC-LAG configuration used in this example.

QFX5100-A

content_copy zoom_out_map
set chassis aggregated-devices ethernet device-count 2
set interfaces xe-0/0/8 ether-options 802.3ad ae0
set interfaces xe-0/0/9 ether-options 802.3ad ae0
set interfaces xe-0/0/10 ether-options 802.3ad ae1
set interfaces ae0 aggregated-ether-options lacp active
set interfaces ae0 unit 0 family ethernet-switching interface-mode trunk
set interfaces ae0 unit 0 family ethernet-switching vlan members v500
set interfaces ae0 unit 0 family ethernet-switching vlan members v100
set interfaces ae1 aggregated-ether-options lacp active
set interfaces ae2 aggregated-ether-options lacp periodic fast
set interfaces ae1 aggregated-ether-options lacp system-id 00:01:02:03:04:05
set interfaces ae1 aggregated-ether-options lacp admin-key 3
set interfaces ae1 aggregated-ether-options mc-ae mc-ae-id 3
set interfaces ae1 aggregated-ether-options mc-ae chassis-id 0
set interfaces ae1 aggregated-ether-options mc-ae mode active-active
set interfaces ae1 aggregated-ether-options mc-ae status-control active
set interfaces ae1 aggregated-ether-options mc-ae init-delay-time 240
set interfaces ae1 unit 0 family ethernet-switching interface-mode access
set interfaces ae1 unit 0 family ethernet-switching vlan members v100
set interfaces irb unit 100 family inet address 10.1.1.9/24 vrrp-group 100 virtual-address 10.1.1.1
set interfaces irb unit 100 family inet address 10.1.1.9/24 vrrp-group 100 priority 200
set interfaces irb unit 100 family inet address 10.1.1.9/24 vrrp-group 100 accept-data
set interfaces irb unit 500 family inet address 10.3.3.2/24
set multi-chassis multi-chassis-protection 10.3.3.1 interface ae0
set protocols iccp local-ip-addr 10.3.3.2
set protocols iccp peer 10.3.3.1 session-establishment-hold-time 340
set protocols iccp peer 10.3.3.1 redundancy-group-id-list 1
set protocols iccp peer 10.3.3.1 backup-liveness-detection backup-peer-ip 10.92.71.88
set protocols iccp peer 10.3.3.1 liveness-detection minimum-receive-interval 60
set protocols iccp peer 10.3.3.1 liveness-detection transmit-interval minimum-interval 60
set protocols rstp interface ae0 disable
set protocols rstp interface ae1 edge
set protocols rstp interface all mode point-to-point
set protocols rstp bpdu-block-on-edge
set switch-options service-id 6
set vlans v100 vlan-id 100
set vlans v100 l3-interface irb.100
set vlans v100 mcae-mac-synchronize
set vlans v500 vlan-id 500
set vlans v500 l3-interface irb.500

Step-by-Step Procedure

QFX5100-B

content_copy zoom_out_map
set chassis aggregated-devices ethernet device-count 2
set interfaces xe-0/0/8 ether-options 802.3ad ae0
set interfaces xe-0/0/9 ether-options 802.3ad ae0
set interfaces xe-0/0/10 ether-options 802.3ad ae1
set interfaces ae0 aggregated-ether-options lacp active
set interfaces ae0 unit 0 family ethernet-switching interface-mode trunk
set interfaces ae0 unit 0 family ethernet-switching vlan members v500
set interfaces ae0 unit 0 family ethernet-switching vlan members v100
set interfaces ae1 aggregated-ether-options lacp active
set interfaces ae2 aggregated-ether-options lacp periodic fast
set interfaces ae1 aggregated-ether-options lacp system-id 00:01:02:03:04:05
set interfaces ae1 aggregated-ether-options lacp admin-key 3
set interfaces ae1 aggregated-ether-options mc-ae mc-ae-id 3
set interfaces ae1 aggregated-ether-options mc-ae chassis-id 1
set interfaces ae1 aggregated-ether-options mc-ae mode active-active
set interfaces ae1 aggregated-ether-options mc-ae status-control standby
set interfaces ae1 aggregated-ether-options mc-ae init-delay-time 240
set interfaces ae1 unit 0 family ethernet-switching interface-mode access
set interfaces ae1 unit 0 family ethernet-switching vlan members v100
set interfaces irb unit 100 family inet address 10.1.1.10/24 vrrp-group 100 virtual-address 10.1.1.1
set interfaces irb unit 100 family inet address 10.1.1.10/24 vrrp-group 100 priority 100
set interfaces irb unit 100 family inet address 10.1.1.10/24 vrrp-group 100 accept-data
set interfaces irb unit 500 family inet address 10.3.3.1/24
set multi-chassis multi-chassis-protection 10.3.3.2 interface ae0
set protocols iccp local-ip-addr 10.3.3.1
set protocols iccp peer 10.3.3.2 session-establishment-hold-time 340
set protocols iccp peer 10.3.3.2 redundancy-group-id-list 1
set protocols iccp peer 10.3.3.2 backup-liveness-detection backup-peer-ip 10.92.71.87
set protocols iccp peer 10.3.3.2 liveness-detection minimum-receive-interval 60
set protocols iccp peer 10.3.3.2 liveness-detection transmit-interval minimum-interval 60
set protocols rstp interface ae0 disable
set protocols rstp interface ae1 edge
set protocols rstp interface all mode point-to-point
set protocols rstp bpdu-block-on-edge
set switch-options service-id 6
set vlans v100 vlan-id 100
set vlans v100 l3-interface irb.100
set vlans v100 mcae-mac-synchronize
set vlans v500 vlan-id 500
set vlans v500 l3-interface irb.500

arrow_backward PREVIOUS QFX Series MC-LAG Fabric Upgrade Procedure

NEXT arrow_forward