Troubleshooting Chassis Cluster ISSU Failures
This topic includes the following sections:
Stopping ISSU Process When it Halts During an Upgrade
Problem
The ISSU process halts in the middle of an upgrade.
Solution
If the ISSU fails to complete and only one device in the cluster has been upgraded, you can roll back to the previous OS on that device alone by using the following commands on the upgraded device:
- Abort ISSU on both nodes using the request chassis cluster in-service-upgrade abort command.
- Rollback the image using the request system software rollback command with node option.
- Reboot the rolled back node using the request system reboot command.
Recovering the Node in Case of a Failed ISSU
Problem
The ISSU procedure stops progressing.
Solution
Open a new session on the primary device and issue the request chassis cluster in-service-upgrade abort command.
This step aborts an in-progress ISSU . This command must be issued from a session other than the one on which you issued the request system in-service-upgrade command that launched the ISSU. If the node is being upgraded, this command cancels the upgrade. The command is also helpful in recovering the node in case of a failed ISSU.
When an ISSU encounters an unexpected situation that necessitates an abort, the system message provides you with detailed information about when and why the upgrade stopped and recommendations for the next steps to take.
For example, the following message is issued when a node fails to become RG-0 secondary when it boots up:
Rebooting Secondary Node
Shutdown NOW!
[pid 2120]
ISSU: Backup RE Prepare Done
Waiting for node1 to reboot.
node1 booted up.
Waiting for node1 to become secondary
error: wait for node1 to become secondary failed (error-code: 5.1)
ISSU aborted. But, both nodes are in ISSU window.
Please do the following:
1. Log on to the upgraded node.
2. Rollback the image using rollback command with node option
Note: Not using the 'node' option might cause
the images on both nodes to be rolled back
3. Make sure that both nodes (will) have the same image
4. Ensure the node with older image is primary for all RGs
5. Abort ISSU on both nodes
6. Reboot the rolled back node
{primary:node0} | Note: If you attempt to upgrade a device pair running a Junos OS image earlier than Release 9.6, the ISSU will fail without changing anything about either device in the cluster. Devices running Junos OS Releases earlier than 9.6 must be upgraded separately using individual device upgrade procedures. If the secondary device experiences a power-off condition before it boots up using the new image specified when the ISSU is initiated, when power is restored the newly upgraded device will still be waiting to end the ISSU. To end the ISSU, issue the request chassis cluster in-service-upgrade abort command. |
