Prepare for the Upgrade
SUMMARY This section describes important tasks to perform and upgrade considerations before upgrading to Contrail Cloud v16.2.
Upgrade Considerations
- Check Your Cloud Health
- Back up Your Undercloud and Overcloud
- Pause and Shutdown Business Services
- External NFS Storage
- Undercloud Directory and Disk Space
Check Your Cloud Health
Make sure that your cloud is functional, healthy, and that all services are active. Any problems in your cloud health can cause errors during the upgrade. To check your cloud health, see the Contrail Cloud Deployment Guide. Scroll to the "Verify Quorum and Node Health" section in the Node Reboot and Health Check appendix for instructions. See also, Validating Red Hat OpenStack Platform 13 before the upgrade.
Back up Your Undercloud and Overcloud
For back up instructions, see: Backup and Restore the Director Undercloud, and Backing up the Overcloud control plane services.
Pause and Shutdown Business Services
We recommend that you pause or shutdown external business services to ensure a smooth upgrade. Doing so helps prevent data loss or a workload error. The services are dependent on the specific business service or virtual machine (VM) that is running. It is important to know that there are no guarantees for the integrity of API calls during the upgrade. See the documentation for your specific service for instructions.
Do the following:
- Quiesce all external API requests, for example, Horizon.
- Perform a graceful shutdown on any vulnerable workloads.
External NFS Storage
You must provide an external NFS storage for the upgrade. Your NFS is used during the Contrail Cloud upgrade to back up the Contrail configuration database and the OpenStack controllers. External NFS storage is outside of the scope of Contrail Cloud and unique to your environment. Certain considerations need to be taken when preparing for an upgrade to Contrail Cloud 16.2. See below to understand how NFS storage functions with Contrail Cloud and the upgrade process.
- NFS storage must be provided to store backup information during the upgrade process.
- The Contrail configuration database and virtual machine images for the OpenStack overcloud controller role will be copied to the NFS mount.
- The NFS server must have sufficient space to hold the configuration database and the raw virtual machine image files. See the size calculation section below for more information.
- The NFS server must be reachable by the jump host and each control host, and the NFS server host name must be resolvable by their DNS. You configure DNS in the config/site.yml file under global['dns'] list.
- You must ensure that the NFS server authorizes export storage to the jump host and the
control hosts.
- Depending on the network topology for a particular cluster deployment, it can be common for the jump host to NAT traffic from the control hosts to destinations outside of the cluster. When using NAT, the NFS server sees traffic as originating from the jump host. You may not need explicit exports for the control hosts.
- Files on NFS will not be cleaned automatically after the upgrade. The files written to your NFS are kept for manual recovery purposes. You will need to remove these files manually after all steps have been completed.
Verify You Have NFS Connectivity
Verify that your jump host and control hosts can access your NFS share for the upgrade process.
Use the following CLI example on the jump host and each control host to verify you have a proper NFS path and permissions:
# Create the mount directory for the test mkdir ~/nfs_test_mount # Mount the share sudo mount <ip/fqdn_of_nfs_server>:/<nfs_dir> ~/nfs_test_mount # Check if the share is properly mounted and permissions allow the host to create files ls -al ~/nfs_test_mount touch ~/nfs_test_mount/test-file rm ~/nfs_test_mount/test-file # Unmount and remove the mount directory sudo umount ~/nfs_test_mount rmdir ~/nfs_test_mount
NFS Storage Calculation
Make sure that you have enough external NFS storage before you upgrade. To determine how much storage that you need use the following calculation:
NFS size must be at least X + (Y * N) GB where:
X = The configuration database size.
Y = Is the configured image size from: control_hosts['vm']['control']['disk']['vda']['size']
in the config/site.yml file.
N = The number of control hosts.
Undercloud Directory and Disk Space
During the upgrade process the undercloud virtual disk is adjusted to fit the requirements of RHEL 8.2. An automated backup of the VM is performed as part of this process. For this purpose, a dedicated directory with enough space is required on the jump host.
The default directory is "/home/{{ global['service_user']['name'] }}
", but
this can be changed in the upgrade_plan by adding:
upgrade_plan: upgrade_backup_directory: <your_backup_directory>
Calculate the VM Disk Size
You need to adjust for the VM disk to meet the required minimal size. The size is a sum of fixed undercloud VM disk size and the current disk space consumed by the root partition on undercloud VM. You can access these values by using the following commands:
du -hs /var/lib/libvirt/images/undercloud.qcow2
and
ssh undercloud 'df -h /'
Request an Upgrade
- Include your Contrail Cloud activation key in the email request. State that you are upgrading from v13 to v16.
- Specify the time and date for your upgrade.
Juniper receives your request and activates your new upgrade key.
During the Upgrade Process
Do not make any configuration changes during the upgrade (such as enabling or disabling features, changing names, and so forth).
Do not attempt to change resources or cluster properties as part of the upgrade process. These activities should be handled as part of the deployment process, and is outside of the upgrade process. Making such changes during the upgrade process will likely lead to failures.