Upgrading Software on a Virtual Chassis and Mixed Virtual Chassis Using Nonstop Software Upgrade
Nonstop software upgrade (NSSU) enables you to upgrade the software running on all member switches of supported Virtual Chassis with minimal traffic disruption during the upgrade.
NSSU works only on some Virtual Chassis with certain from and to Junos OS Releases. Use
the request system software add
command to upgrade the
member switches in the Virtual Chassis individually if the Virtual
Chassis is running a software version that does not support NSSU or
does not support the combination of from and to releases.
You can also refer to Two-Member QFX Series Virtual Chassis Upgrade Procedure, a network configuration example on how to manually upgrade a two-member QFX Series Virtual Chassis with minimal impact to traffic flow when NSSU is not supported.
Preparing the Switch for Software Installation
Before you begin installing the new software using NSSU:
Ensure that the Virtual Chassis is connected and configured correctly to support the NSSU process. See Requirements for Performing an NSSU.
Verify that the members are running the same version of the software:
user@switch>
show version
If the Virtual Chassis or mixed Virtual Chassis members are not running the same version of the software, use the
request system software add
command to upgrade the software on the inconsistent members.Ensure that graceful Routing Engine switchover (GRES) is enabled, or for applicable platforms, make sure nonstop active routing (NSR) is enabled, which also enables graceful Routing Engine switchover. See Configuring Nonstop Active Routing on Switches for more information.
To check the nonstop active routing state to verify both NSR and GRES are enabled:
user@switch>
show task replication
(Optional for applicable platforms) Enable nonstop bridging (NSB), which ensures that all NSB-supported Layer 2 protocols operate seamlessly during the Routing Engine switchover that is part of the NSSU. See Configuring Nonstop Bridging on Switches (CLI Procedure) for details.
For a two-member Virtual Chassis, make sure you configured
no-split-detection
so the Virtual Chassis does not split when NSSU upgrades one of the members. See Disabling Split and Merge in a Virtual Chassis.In an EX4300 Virtual Chassis running a Junos OS 13.2X50 release, you should set the
vcp-no-hold-time
option at the [edit virtual-chassis
] hierarchy level before performing a software upgrade using NSSU, otherwise the Virtual Chassis might split during the upgrade. A split Virtual Chassis can disrupt your network, and you might need to manually reconfigure your Virtual Chassis after the NSSU if the split-and-merge feature was disabled. For more information about a Virtual Chassis split, see Understanding Split and Merge in a Virtual Chassis. This statement only affects EX4300 Virtual Chassis or mixed Virtual Chassis that contain EX4300 switches.To configure this option:
user@switch#
set virtual-chassis vcp-no-hold-time
On a QFX5100 Virtual Chassis with line-card upgrade groups configured, you should enable the
lc-reboot-delay
option to configure a delay for when adjacent members in a line card group reboot. Without this option, when the next member reboots, approximately two minutes after the previous member reboots and joins the Virtual Chassis, the previous rebooted member might not be ready to carry traffic. This delay helps prevent dropping traffic when there are two adjacent line card members with interfaces that are part of a common link aggregation group (LAG).We recommend setting a 200-second delay (the allowable range is 0 to 600 seconds). To configure this delay:
[edit chassis] user@switch#
set chassis nssu lc-reboot-delay 200
(Optional) Back up the system software (Junos OS, the active configuration, and log files) on each member to an external storage device as desired using the
request system snapshot
command.
Upgrading the Software Using NSSU
This procedure describes how to upgrade the software running on all Virtual Chassis or mixed Virtual Chassis members using NSSU. When the upgrade completes, all members are running the new version of the software. The upgrade includes a graceful Routing Engine switchover, so the original Virtual Chassis backup member switch becomes the new primary.
During NSSU, the primary copies the new software image to all the members in the Virtual Chassis and reboots them in turn. If copying the new software to a member fails or rebooting a member fails, NSSU terminates the upgrade process and logs the error. In this case, you must manually perform recovery measures for members left in an incompatible state to restore all members to running the same version of the software. Starting in Junos OS Release 14.1X53-D40, NSSU automatically invokes recovery measures after either of these failures, as follows:
if NSSU terminates due to a copy error, the primary removes the new image from any members to which it was already copied.
If any member fails to reboot, NSSU automatically initiates a clean Virtual Chassis restart by bringing down and rebooting the entire Virtual Chassis. All members come up running the new software at the same time. This action cleanly recovers correct Virtual Chassis operation more quickly than having an unstable Virtual Chassis running different versions of the software trying to converge.
Junos OS software images with enhanced automation are only supported on a non-mixed Virtual Chassis with QFX5100 switches. Also, you can’t perform an NSSU from a standard Junos OS software image to a Junos OS software image with enhanced automation, or from a Junos OS software image with enhanced automation to a standard Junos OS software image.
To upgrade all members in a Virtual Chassis using NSSU:
Download the software package as described in Installing Software Packages on QFX Series Devices. If you are upgrading a mixed Virtual Chassis, download the software packages for the different switch types.
Copy the software package or packages to the Virtual Chassis. We recommend that you copy the file or files to the
/var/tmp
directory on the primary.Use the console connection or the virtual management Ethernet (VME) interface to log in to the Virtual Chassis or mixed Virtual Chassis. You can monitor the progress of the primary switch reboot if you use a console connection.
Start the NSSU:
On a Virtual Chassis where all members use the same software image, enter:
user@switch> request system software nonstop-upgrade force-host /var/tmp/package-name.tgz
where
package-name.tgz
is the software package name, for example,jinstall-qfx-3-13.2X50-D15.3-domestic-signed.tgz
.On a mixed Virtual Chassis where members might use different software images, enter the
request system software nonstop-upgrade
command with theset
option to specify more than one software package name:user@switch> request system software nonstop-upgrade set [/var/tmp/package-name1.tgz /var/tmp/package-name2.tgz]
For example,
/var/tmp/package-name1.tgz
and/var/tmp/package-name2.tgz
might specify software packages for EX4200 and EX4500 switches in a mixed EX Series Virtual Chassis with EX4200 and EX4500 switches.
The switch displays status messages similar to the following messages as the upgrade executes:
Chassis ISSU Check Done NSSU: Validating Image NSSU: Preparing Backup RE Installing image on other FPC's along with the backup Checking pending install on fpc1 Pushing bundle to fpc1 WARNING: A reboot is required to install the software WARNING: Use the 'request system reboot' command immediately Completed install on fpc1 Checking pending install on fpc2 Pushing bundle to fpc2 WARNING: A reboot is required to install the software WARNING: Use the 'request system reboot' command immediately Completed install on fpc2 Rebooting fpc1 NSSU: Backup RE Prepare Done Waiting for Backup RE reboot GRES operational Initiating Chassis In-Service-Upgrade Chassis NSSU Started NSSU: Preparing Daemons NSSU: Daemons Ready for NSSU NSSU: Starting Upgrade for FRUs NSSU: Preparing for Switchover NSSU: Ready for Switchover Checking In-Service-Upgrade status Item Status Reason FPC 0 Online FPC 1 Online FPC 2 Online (ISSU) Going to install image on master WARNING: A reboot is required to install the software WARNING: Use the 'request system reboot' command immediately relinquish mastership NSSU: IDLE *** FINAL System shutdown message from user@switch *** System going down IMMEDIATELY Shutdown NOW! [pid 9336]
Log in after the original primary switch reboot completes. To verify that the software is upgraded on all Routing Engines in the Virtual Chassis, enter the following command:
user@switch>
show version
To ensure the resilient dual-root partitions feature operates correctly, copy the new Junos OS image into the alternate root partitions of all members:
user@switch>
request system snapshot slice alternate all-members
With resilient dual-root partitions, the switch can boot transparently from the alternate root partition if the system fails to boot from the primary root partition.
After an upgrade is complete, please verify syslog, show chassis fabric errors, show
chassis fabric fpcs
, and show system alarms
.
If the FPCs or fabric display any errors, set alarms for specific errors. Configure
pfe-offline
as error action to mitigate outages.