Upgrade Apstra on Same VM (In-Place)

Note:

If you upgrade in-place you won't receive Ubuntu Linux OS fixes, including security vulnerability updates. To receive these updates you must upgrade on a new VM. To upgrade in-place instead, keep reading.

To upgrade the Apstra server you need Apstra OS admin user privileges and Apstra admin user group permissions.

Step 1: Pre-Upgrade Validation

Refer to Upgrade Paths to confirm that you're upgrading to a supported version. (You can find your current Apstra version by navigating to Platform > About in the Apstra GUI. As of Apstra version 4.2.1, the Apstra version is also shown in the left navigation menu of the GUI under the Juniper Apstra logo.)
By default, the Apstra upgrade creates the additional Docker subnet 172.18.0.1/16. If you're using this subnet elsewhere in your network, the Apstra upgrade could fail. If this applies to you, see Docker Network and Apstra Upgrades for how to use a different subnet.
Log in to the Apstra server as admin (For example, if your Apstra server IP address were 10.28.105.3, the command would be ssh admin@10.28.105.3).

Confirm that the VM has enough memory to hold two versions of the Apstra software at the same time. Run the command free -h to check if memory utilization is less than 50%.

If utilization is greater than 50%, gracefully shut down the Apstra server, add resources, then restart the Apstra server.

Run the command service aos status to check that server is active and has no issues.

Check the new Apstra version release notes for configuration-rendering changes that could impact the data plane. Update configlets, as needed.
Review each blueprint to confirm that all Service Config has succeeded. If necessary, undeploy and remove devices from the blueprint to resolve any pending or failed service config.
Review each blueprint for probe anomalies, and resolve them as much as possible. Take notes of any remaining anomalies.
Refer to Qualified Devices and NOS Versions to verify that your NOS versions are qualified on the new Apstra version. Upgrade or downgrade, as needed, to one of the supported versions.
If you're using Junos devices and upgrading to Apstra version 4.2.1, the pristine configuration must include the essential mgmt_junos VRF. See Juniper Support Knowledge Base article KB77094 for more information.

CAUTION:

If the pristine configuration doesn't include the mgmt_junos VRF, then deployment will fail.
Remove any Device AAA configuration. During device upgrade, configured device agent credentials are required for SSH access.
Remove any configlets used to configure firewalls. If you use FW's Routing Engine filters on devices, you'll need to update them to include the IP address of the new controller and worker VMs.
To upgrade device system agents, Apstra must be able to SSH to all devices using the credentials that were configured when creating the agents. To check this from the Apstra GUI, navigate to Devices > Agents, select the check box(es) for the device(s) to check, then click the Check button in the Agent menu. Verify that all job states are in the SUCCESS state. If any check job fails, resolve the issue before proceeding with the Apstra upgrade.
As root user, run the command sudo aos_backup to back up the Apstra server.
Note:
CAUTION:

The upgraded Apstra server doesn't include any time voyager revisions, so if you need to revert back to a past state, this backup is required. Previous states are not included due to the tight coupling with the reference designs which may change between Apstra versions.
Copy the backup files from /var/lib/aos/snapshot/<shapshot_name> to an external location.

Step 2: Deploy New Apstra Server

Download the Apstra installer package for your platform from Juniper Support Downloads (for example, aos_server_4.1.2-269) and transfer it to the Apstra server.

Unzip the Apstra installer package.

If you're using an Apstra cluster (offbox agents, IBA probes), download the installer package to the worker nodes as well. You'll upgrade the worker nodes in a later step.
Log in to the Apstra server as admin.

Run the sudo bash aos_<aos_version>.run command, where <aos_version> is the version of the run file. For example, if the version is 4.0.1-1045 the command would be sudo bash aos_4.1.1-287.run as shown below.

When you run this command, if any previous Apstra versions are detected, the script enters upgrade mode instead of new installation mode. The new Docker container installs next to the Docker containers from the previous version. The script imports the data from the previous version and migrates it to Apstra SysDB on the new version.

The upgrade script presents a summary view of the devices within the fabric that will receive configuration changes during the upgrade. As of Apstra version 4.1.2, a warning appears on the screen recommending that you read Release Notes and Upgrade Paths documentation before proceeding. The release notes include a category for Configuration Rendering Changes, as of Apstra version 4.1.2. Configuration rendering changes are clearly documented at the top explaining the impact of each change on the network.

As of Apstra version 4.0.1, the Apstra Upgrade Summary shows information separated by device roles (superspine, spine, leaf, leaf pair, and access switch for example). If an incremental config was applied instead of a full config, more details are displayed about the changes.

After you've reviewed the summary, enter q to exit the summary. The AOS Upgrade: Interactive Menu appears where you can review the exact configuration change on each device. If you're using configlets, verify that the new configuration pushed by the upgrade does not conflict with any existing configlets.
CAUTION:

The Apstra Reference Design in the new Apstra release may have changed in a way that invalidates configlets. To avoid unexpected outcomes, verify that your configlets don’t conflict with the newly rendered config. If you need to update your configlets, quit the upgrade, update your configlets, then run the upgrade again.
If you want to continue with the upgrade after reviewing pending changes, enter c. The older Apstra version is deleted and the new Apstra version is activated on the server. When the upgrade is complete, navigate to Platform > About in the Apstra GUI to check the version.

CAUTION:

Upgrading the Apstra server is a disruptive process. When you upgrade in-place (same VM) and continue with the upgrade from this point, you cannot roll back the upgrade. The only way to return to the previous version is to reinstall a new VM with the previous version and restore the database from the backup that you previously made. You made a backup, right?
If you want to stop the upgrade, enter q to abort the process. If you quit at this point and later decide to upgrade, you must start the process from the beginning.
If you're using an Apstra cluster, the worker nodes disconnect from the Apstra controller and change to the FAILED state. This state means that offbox agents and the IBA probe containers that are on the worker nodes are not available; devices that are managed by the offbox agents do remain in service though. After you upgrade the agents in a later step, you'll upgrade the worker nodes in your Apstra cluster and the agents and/or probes will become available.

Step 3: Change Operation Mode to Normal

When you initiate an Apstra server upgrade, the operation mode changes from Normal to Maintenance automatically. After you've completed the upgrade you must manually change the mode back to Normal.

From the left navigation menu in the Apstra GUI, navigate to Platform > Apstra Cluster > Cluster Management.
Click the Change Operation Mode button, select Normal, then click Update. When you change the mode to Normal, any configured offbox agents are activated, but you must initiate the upgrade of any onbox agents (in the next section).

You can also access the Cluster Management page from the lower left section of any page. You have continuous platform health visibility from here as well, based on colors.

From the bottom of the left navigation menu, click one of the dots, then click Operation Mode to go to Cluster Management. Click the Change Operation Mode button, select Normal, then click Update.

Because they're still in the process of upgrading, the agents won't be connected. When the upgrade finishes, the agents reconnect to the server and come back online. On the blueprint dashboard the Liveness anomalies for spine and leaf will also resolve.

Step 4: Upgrade Onbox Agents

CAUTION:

When you initiate agent upgrade you cannot roll back to the previous version. The only way to return to the previous version is to reinstall a new VM with the previous version and restore the database from the backup that you previously made.

Log in to the Apstra GUI as user admin.
From the left navigation menu, navigate to Devices > System Agents > Agents and select the device(s) to upgrade (up to 100 devices at a time as of Apstra version 4.0.1). You can upgrade multiple onbox agents at the same time, but the order of device upgrade is important.
- Upgrade agents for superspines first.
- Upgrade agents for spines second.
- Upgrade agents for leafs third.
Click the Install button to initiate the install process. The job state changes to IN PROGRESS. If agents are using a previous version of the Apstra software, they are automatically upgraded to the new version. Then they connect to the server and push any pending configuration changes to the devices. Telemetry also resumes, and the job states change to SUCCESS.
In the Liveness section on the blueprint dashboard, confirm that you don't have any device anomalies .

Note:
If you need to roll back to the previous Apstra version after initiating agent upgrade, you must build a new VM with the previous Apstra version and restore the configuration to that VM. For assistance, contact Juniper Technical Support.

Step 5: Upgrade Worker Nodes (Apstra Cluster Only)

If you're using an Apstra cluster (for offbox agents and/or IBA probes), you need to upgrade the worker nodes as well as the controller node that you have already upgraded.

If you didn't download the Apstra installer package to the worker nodes when you downloaded it to the Apstra server, do that now.

From each Apstra worker node, run the

sudo bash
                            aos_<aos_version>.run

command, where <aos_version> is the version of the run file. For example, if the version is 4.1.1-287 the command would be sudo bash aos_4.1.1-287.run (no options). This is the same file you used to upgrade the controller. There are no prompts during the worker node upgrade.

Next Steps:

If the NOS versions of your devices are not qualified on the new Apstra version, upgrade them to a qualified version. (See the Juniper Apstra User Guide for details.)