Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
ON THIS PAGE
 

Upgrading Contrail Networking Release 1912.L4 or 2011.L3 with RHOSP 13 or RHOSP 16.1 to Contrail Networking Release 21.4 with RHOSP 16.2

The goal of this topic is to provide a combined procedure to upgrade Red Hat OpenStack Platform (RHOSP) from RHOSP 13 or RHOSP 16.1 to RHOSP 16.2 by leveraging Red Hat Fast Forward Upgrade (FFU) procedure while simultaneously upgrading Contrail Networking from Release 1912.L4 or 2011.L3 to Release 21.4.

The downtime will be reduced by not requiring extra server reboots in addition to the ones that the RHOSP FFU procedure already requires for Kernel/RHEL upgrades.

Before upgrading overcloud, refer Chapter 20—Speeding up an overcloud upgrade.

Refer to Framework for Upgrades (13 to 16.2) documentation for details on RHOSP 13 to RHOSP 16.2 Fast Forward Upgrade (FFU) procedure of OpenStack Platform environment from one long life version to the next long life version.

Refer to Keeping Red Hat OpenStack Platform Updated documentation for details on RHOSP 16.1 to RHOSP 16.2 to perform minor updates of Red Hat OpenStack Platform.

When to Use This Procedure

The procedure in this document has been validated for the following Contrail Networking upgrade scenarios:

Table 1: Validated Upgrade Scenarios

Current Version

Target Version

RHOSP 13

RHOSP 16.2

RHOSP 16.1

RHOSP 16.2

Contrail Networking Release 1912.L4

Contrail Networking Release 21.4

Contrail Networking Release 2011.L3

Contrail Networking Release 21.4

Prerequisites

This document makes the following assumptions about your environment:

  • A Contrail Networking deployment using Red Hat OpenStack version 13 (RHOSP 13) or RHOSP 16.1 as the orchestration platform is already operational.

  • The overcloud nodes in the RHOSP 13 or RHOSP 16.1 environment have an enabled Red Hat Enterprise Linux (RHEL) subscription.

  • Your environment is running Contrail Release 1912.L4 or 2011.L3 and upgrading to Contrail Release 21.4.

  • If you are updating Red Hat OpenStack simultaneously with Contrail Networking, we assume that the undercloud node is updated to the latest minor version and that new overcloud images are prepared for an upgrade. See the Chapter 2—Updating the undercloud section of the Keeping Red Hat OpenStack Platform Updated guide from Red Hat. If the undercloud has been updated and a copy of the heat templates are used for the deployment, update the copy of the heat template from the Red Hat’s core heat template collection at /usr/share/openstack-tripleo-heat-templates. For more information on this process, see the Understanding Heat Templates chapter from Red Hat.

  • Per Red Hat OpenStack support guidelines, do not change IP addresses during this upgrade.

  • New Contrail Control plane is deployed on K8s/OpenShift cluster with a self-signed root CA:

    • You should generate a self-signed root CA and a key: k8s-root-ca.pem and k8s-root-ca-key.pem.

    • You must deploy Contrail with the generated self-signed root CA and key as a Contrail root CA.

      Example of input environment variables for deploying new Contrail Control plane on K8s/OpenShift cluster with a self-signed root CA:

  • Use CA bundle, if RHOSP uses IPA for the certificate management cluster. Example of how to prepare CA bundle and use as Contrail root CA:

Before You Begin

We recommend performing these procedures before you start the update:

Upgrade Contrail Networking Release 1912.L4 or 2011.L3 with RHOSP 13 or RHOSP 16.1 to Contrail Networking Release 21.4 with RHOSP 16.2

To perform the upgrade:

  1. Deploy new Contrail control plane on Kubernetes/Openshift cluster. See Prerequisites.
  2. To prepare the cluster for ISSU update:
    1. Ensure that overcloud VIP FQDNs and all overcloud nodes FQDNs are resolvable on the selected ISSU node (modify /etc/hosts). For example:
      Note:

      Overcloud node FQDNs can be taken from /etc/hosts of one of the overcloud node.

    2. Distribute the self signed root CA of the Kubernetes cluster on Contrail Controller nodes as trusted CA and switch Contrail RabbitMQ to use the CA bundle.

      1. Make a CA bundle file.

      2. Prepare the environment file ca-bundle.yaml.

      3. Distribute the CA bundle. Add a new environment file and make Contrail services to use the CA bundle to run the deploy command. For example:

  3. Start the ISSU sync data:
    1. Stop service Device Manager, Schema Transformer and SVC Monitor, edit manager's manifest, and add devicemanager, schematransformer, and svcmonitor containers to the containers section.
      Note:

      Ensure that the config pod is restarted and the removed containers are not listed in the containers section.

    2. Select one master node to run the ISSU scripts and then label the master node (for example, node1).
    3. Collect data from the operator environment. For example:
    4. Prepare issu-configmap.
    5. Modify issu.env file according to your environment and create a configmap.
    6. Prepare jobs.

      Modify the yaml files if you need to customize your configurations.

      Note:

      Modify images to use already pulled images on K8s nodes. This is because, the ISSU jobs do not pull images and rely on images used to deploy the K8s cluster.

    7. Run a pair job to pair control nodes between the old and new clusters.

      Wait till the job is completed and log is validated.

    8. Pre-synchronize data to the new cluster and run the presync job.

      Wait till the job is completed and log is validated.

    9. Run the sync data between clusters.

      1. Run the sync job.

      2. Check that the sync job is running normally.

        Note:

        At this point switch to main instruction and follow RHOSP update/upgrade procedure.

    10. Return to the main update workflow and update the cluster.
  4. Upgrade the undercloud:

    1. For RHOSP 13 to RHOSP 16.2, follow Chapter 4—Preparing for the undercloud upgrade to Chapter 6—Upgrading director of Framework for Upgrades (13 to 16.2).

    2. For RHOSP 16.1 to RHOSP 16.2, follow Chapter 2—Updating the undercloud of Keeping Red Hat OpenStack Platform Updated.

  5. Upgrade the overcloud:

    1. For RHOSP 13 to RHOSP 16.2, follow Chapter 7—Initial steps for overcloud preparation through Step 17.3—OpenStack overcloud external-upgrade run of Framework for Upgrades (13 to 16.2).

    2. For RHOSP 16.1 to RHOSP 16.2, follow Step 3.1—Running the overcloud update preparation through Step 3.9—Performing online database updates of Keeping Red Hat OpenStack Platform Updated.

  6. Stop the ISSU sync job, finalize sync data, and unpair control nodes:
    1. Stop the ISSU sync job.
    2. Finalize the sync and run the sync job.
    3. Check that the sync job is running normally.
    4. Finalize the sync of ZK data and run the sync job.
    5. Check that the sync job is running normally.
    6. Run the unpair job to delete pairing of control nodes between the old and new clusters.

      Wait till the job is completed and log is validated.

    7. Start service Device Manager, Schema Transformer and SVC Monitor, edit manager's manifest, and add devicemanager, schematransformer, and svcmonitor containers to the containers section.
      Note:

      Ensure that the config pod is restarted and the removed containers are not listed in the containers section.

      Example of the edited config section:

    8. Remove the label from the ISSU node and issu-config configmap.
    9. Return to the main update workflow.
  7. Continue with the overcloud upgrade process:

    1. For RHOSP 13 to RHOSP 16.2, follow Step 17.4—OpenStack overcloud upgrade converge of Framework for Upgrades (13 to 16.2).

    2. For RHOSP 16.1 to RHOSP 16.2, follow Step 3.10—Finalizing the update of Keeping Red Hat OpenStack Platform Updated.

  8. Remove the old Contrail Control plane nodes.
  9. (Optional) Perform the post-upgrade tasks.

    1. For RHOSP 13 to RHOSP 16.2, follow Chapter 25—Performing post-upgrade actions of Framework for Upgrades (13 to 16.2).

    2. For RHOSP 16.1 to RHOSP 16.2, follow Chapter 4—Rebooting the overcloud through Step 4.3—Rebooting Compute nodes of Keeping Red Hat OpenStack Platform Updated.

Related Documentation