close

keyboard_arrow_left

Table of Contents Expand all

play_arrow Installing and Upgrading Contrail
- play_arrow Understanding Contrail
- play_arrow Supported Platforms and Server Requirements
  - Server Requirements and Supported Platforms
- play_arrow Contrail Command
- play_arrow Installing Contrail
  - Installing Contrail with OpenStack and Kolla Ansible
  - Adding a New Compute Node to Existing Contrail Cluster
- play_arrow Using Contrail with AppFormix
  - Contrail and AppFormix Deployment Requirements
  - Installing AppFormix and AppFormix Flows using Contrail Command
- play_arrow Using Contrail with Kubernetes
- play_arrow Using Contrail with Mesos
  - Understanding Contrail with Mesos Architecture
  - Installing Contrail with Mesos
- play_arrow Using VMware vCenter with Containerized Contrail
- play_arrow Using Contrail with Red Hat OpenStack
- play_arrow Using Contrail with Red Hat OpenShift
  - Installing a Standalone Red Hat OpenShift Container Platform 3.11 Cluster with Contrail Using Contrail OpenShift Deployer
  - Installing a Nested Red Hat OpenShift Container Platform 3.11 Cluster Using Contrail Ansible Deployer
- play_arrow Using Contrail with Juju Charms
- play_arrow Using Contrail and AppFormix with Kolla/Ocata OpenStack
- play_arrow Upgrading Contrail Software
- play_arrow Backup and Restore Contrail Software
  - How to Backup and Restore Contrail Databases in JSON Format
- play_arrow Post Installation Tasks

list Table of Contents

English

How to Upgrade From Contrail Networking Release 4.1.4 with RHOSP10 to Contrail Networking Release 1912.L1 or 1912.L2 with RHOSP13

date_range 28-Aug-23

arrow_backward

arrow_forward

The goal of this topic is to provide a combined procedure to upgrade Red Hat OpenStack Platform (RHOSP) from RHOSP 10 to RHOSP 13 by leveraging Red Hat Fast Forward Upgrade (FFU) procedure while simultaneously upgrading Contrail Networking from Release 4.1.4 to Release 1912.L1 or 1912.L2. The procedure leverages the In Service Software Upgrade (ISSU) procedure from Contrail to minimize the downtime.

The downtime will be reduced by not requiring extra server reboots in addition to the ones that the RHOSP FFU procedure already requires for Kernel/RHEL upgrades.

Refer to Red Hat OpenStack Platform 13 Fast Forward Upgrades for details on RHOSP 10 to RHOSP 13 FFU procedure of OpenStack Platform environment from one long life version to the next long life version.

When to Use This Procedure

We recommend engaging Juniper Networking professional services for assistance with performing this procedure.

This procedure is used when you are running Contrail Networking Release 4.1.4 in an environment using Red Hat Openstack Platform 10 (RHOSP 10) for orchestration and want to upgrade your environment to a Contrail Networking 19 release using RHOSP13. The procedure leverages the Red Hat Fast Forward Upgrade (FFU) process for the RHOSP upgrade and the In Service Software Upgrade (ISSU) procedure for the Contrail Networking upgrade.

The procedure in this document has been validated for the following Contrail Networking upgrade scenarios:

Table 1: Validated Upgrade Scenarios
Starting Contrail Networking Release	Target Upgraded Contrail Networking Release
4.1.4	1912.L1
4.1.4	1912.L2

If you want to use this procedure to upgrade your Contrail Networking release to other releases, you must engage Juniper Networks professional services. Contact your Juniper representative for additional information.

If you want to upgrade from an environment using Contrail Networking Release 3 and RHOSP10 to an environment running Contrail Networking Release 19 and RHOSP13, see How to Upgrade From Contrail Networking Release 3.x or 4.x with RHOSP10 to Contrail Networking Release 1907 with RHOSP13. You can also use this procedure to upgrade from a Contrail Networking Release 4 environment to Contrail Networking Release 1907.

If you are using RHOSP13 for orchestration and would like to upgrade to a Contrail Networking Release 20 environment without upgrading RHOSP, see Updating Contrail Networking using the Zero Impact Upgrade Process in an Environment using Red Hat Openstack

If you are using RHOSP13 for orchestration and would like to upgrade to a Contrail Networking Release 19 environment without upgrading RHOSP, see Upgrading Contrail Networking with Red Hat Openstack 13 using ISSU.

Before You Begin

Before you begin:

Obtain the ContrailImageTag from the README Access to Contrail Registry 19XX.
Enable FFU RedHat subscription for overcloud nodes as the upgrade involves a yum update which needs RPM repositories enabled. The subscription must include access to both OSP10 and OSP13 repositories (rhel-7-server-openstack-13-rpms). Additionally, the subscription must have access to the repo rhel-server-rhscl-7-rpms repository. ISSU node uses python27-python-pip from this repository.
Ensure ISSU nodes have internet access during the installation to install docker-compose via pip.

If the ISSU nodes do not have internet access, you must download required packages manually using a node with internet access after the ISSU node installation. You must then transfer packages from this node to the ISSU nodes and install them.
Per Red Hat Openstack support guidelines, do not change IP addresses during this upgrade.

Upgrading Contrail Networking Release 4.1.4 with RHOSP10 to Contrail Networking Release 1912.L1 or 1912.L2 with RHOSP13

To perform the upgrade:

Follow chapter 2 from step 2.1 through step 2.8 of Red Hat OpenStack Platform 13 Fast Forward Upgrades to prepare for an Openstack Platform Upgrade.

Note:
Do not reboot the compute nodes as mentioned in step 2.9.
Prepare to deploy Contrail In-Service Software Upgrade (ISSU) node.
1. Update Contrail TripleO Heat Templates (Stable/Newton branch) from https://github.com/Juniper/contrail-tripleo-heat-templates.
2. Update Contrail TripleO Puppet module to the latest version and prepare Swift Artifacts.
  mkdir -p ~/usr/share/openstack-puppet/modules/tripleo git clone -b stable/newton https://github.com/Juniper/contrail-tripleo-puppet usr/share/openstack-puppet/modules/tripleo tar czvf puppet-modules.tgz usr/ upload-swift-artifacts -c contrail-artifacts -f puppet-modules.tgz
3. Prepare docker registry with Contrail Networking Release 1912.L1 or 1912.L2 images. It can be undercloud or a separate node.
  In addition to Contrail Networking images, the required ansible deployer image is contrail-kolla-ansible-deployer which is available at https://hub.docker.com/r/opencontrailnightly/contrail-kolla-ansible-deployer/tags.
  
  You must use the same tag as of other Contrail Networking containers.
  
  To prepare images:
  
  contrail-tripleo-heat-templates/tools/contrail/import_contrail_container_docker.sh
  
  For details on building container, refer to https://github.com/Juniper/contrail-deployers-containers.
Deploy Contrail In-Service Software Upgrade (ISSU) node.
1. Prepare new bare-metal server node and create flavor contrail-controller-issu for ISSU node. The hardware requirements for ISSU node is the same as for the Contrail Controller Node.
2. Prepare parameters in the yaml file.
  tripleo-heat-templates/environments/contrail/contrail-issu-ffu.yaml: ContrailIssuSshKey - generate and put ssh keys (for ISSU there is SSH access required between ISSU and Contrail Controller nodes) Provide docker registry info: ContrailRegistryCertUrl, ContrailRegistry, ContrailRegistryInsecure, ContrailImageTag
3. Run deploy command with all parameters used for deployment and new environment file.
  '-e tripleo-heat-templates/environments/contrail/contrail-issu-ffu.yam
4. If you do not have internet access, after ISSU node installation, you must download required packages manually an a node with internet access, transfer packages on all the ISSU nodes and install them.
  To do so:
  1. Prepare required modules for docker-compose.
    
    Run the following commands on the a RHEL7 node which has internet access:
    yum install -y --enablerepo=rhel-server-rhscl-7-rpms python27-python-pip source scl_source enable python27 mkdir pip-pkg docker-compose-pkg pip download pip -d ./pip-pkg pip download pip -d ./docker-compose-pkg tar cvfz docker-compose.tar.gz docker-compose-pkg pip-pkg
  2. Upload the archive to all the ISSU nodes.
  3. Install packages on all the ISSU nodes.
    source scl_source enable python27 tar xvfz docker-compose.tar.gz pushd pip-pkg pip install * -f ./ --no-index popd pushd docker-compose-pkg pip install * -f ./ --no-index popd
Run In-Service Software Upgrade (ISSU).
1. Make SSH connection to ISSU node.
  
  Note:
  If you have 3 ISSU nodes deployed, you must perform SSH operations and run scripts on the same node for the entire procedure.
2. Deploy ISSU node.
  cd /etc/contrail/issu ./issu_node_deploy.sh
3. Check status of Contrail Networking service.
  sudo contrail-status
  
  The status must be active.
4. Pair ISSU node with the old cluster.
  ./issu_node_pair.sh
5. Check status of Contrail service on ISSU node.
  sudo contrail-status
  
  Note:
  All services must be active except config_devicemgr, config_schema and config_svcmonitor.
6. Run ISSU sync container.
  ./issu_node_sync.sh
7. Check ISSU container logs.
  sudo docker logs issu-run-sync Config Sync initiated... Config Sync done... Started runtime sync... Start Compute upgrade…
  sudo docker exec issu-run-sync cat /var/log/contrail/issu_contrail_run_sync.log 2019-02-21 17:03:56,769 SYS_DEBUG Control on node 192.168.206.115 has CID 427885c366a5 2019-02-21 17:03:56,875 SYS_INFO Signal sent to process. exit_code = 0, stdout = "[u'427885c366a5\n']", stderr="[]" 2019-02-21 17:03:56,878 SYS_INFO Start Compute upgrade..
8. Restart the contrail_control_control container on all ISSU nodes:
  sudo docker restart contrail_control_control
Upgrade vRouter on Compute nodes.
Perform these steps on all the Compute Nodes.
1. Select the Compute node for upgrade and migrate workload from it.
2. Make SSH connection to Compute node.
3. Deploy new vRouter
  cd /etc/contrail/issu/ ./issu_compute_deploy.sh
4. Verify the network-functions-vrouter-ffu-env-pre file.
  cat /etc/sysconfig/network-scripts/network-functions-vrouter-ffu-env-pre TYPE=kernel BIND_INT=ens3 CONTRAIL_VROUTER_AGENT_CONTAINER_NAME=contrail_vrouter_agent
  You required this file because os-net-config is outdated in Newton and doesn’t provide the required variables for Contrail Networking Release 19 software.
  
  For kernel mode and DPDK, the file is created automatically by the script mentioned in the step 5.d . Verify the file is created as per your deployment.
  DRIVER=uio_pci_generic CPU_LIST=0x1f BOND_MODE=4 BOND_POLICY=layer2+3 VLAN_ID=101
5. Reboot the node.
  sudo reboot
6. After node has rebooted, make SSH connection to the compute node and verify node status.
  modinfo vrouter (for kernel mode) sudo contrail-status
7. Repeat this step to upgrade the vRouter on each compute node.
Upgrade undercloud to OpenStack Plaform (OSP) 13.
Follow chapter 3 till chapter 5 of Red Hat OpenStack Platform 13 Fast Forward Upgrades to configure a container image source and prepare for the overcloud upgrade.

Copy the TripleO (OOO, OpenStack on OpenStack) Heat Templates and prepare united Contrail parameters for OSP 10 with new parameters for OSP 13.

Backup old TripleO Heat Templates.

content_copy zoom_out_map
mv tripleo-heat-templates tripleo-heat-templates-osp10
cp -r /usr/share/openstack-tripleo-heat-templates/ ~/tripleo-heat-templates
git clone -b stable/queens https://github.com/Juniper/contrail-tripleo-heat-templates
cp -r ~/contrail-tripleo-heat-templates/* ~/tripleo-heat-templates

Set united for OSP 10 and OSP 13 parameters.

content_copy zoom_out_map
tripleo-heat-templates/environments/contrail/contrail-services.yaml
tripleo-heat-templates/environments/contrail/contrail-net.yaml

Define NIC templates.

content_copy zoom_out_map
tripleo-heat-templates/network/config/contrail/compute-nic-config.yaml
tripleo-heat-templates/network/config/contrail/contrail-controller-nic-config.yaml
tripleo-heat-templates/network/config/contrail/controller-nic-config.yaml

Define role parameters for Computer Node, DPDK, TSN, etc.

content_copy zoom_out_map
ComputeParameters:
    ContrailSettings:
      VROUTER_GATEWAY: 192.168.206.2
      MAINTANENCE_MODE: true
      BGP_ASN: 64512
      BGP_AUTO_MESH: true

Remove contrail-artifacts, if any to avoid rewriting contrail Puppets.
```
swift delete contrail-artifacts
rm -f .tripleo/environments/deployment-artifacts.yaml 
```
Make SSH connection to ISSU node.
Stop the ISSU service and un-pair ISSU node with the old Contrail Control plane.
cd /etc/contrail/issu/ ./issu_node_sync_post.sh ./issu_node_pair.sh del

Upgrade overcloud to OpenStack Platform (OSP) 13.
Note:
Ensure the contrail-services.yaml file has an empty string for the ContrailVrouterHugepages1GB: parameter before the compute nodes are updated in this step.

parameter_defaults: … ContrailVrouterHugepages1GB: ””

The compute node updates occur in Step 6.2 of the Red Hat OpenStack Platform 13 Fast Forward Upgrades procedure.
Follow chapter 6 of Red Hat OpenStack Platform 13 Fast Forward Upgrades.
1. You can use this role file—tripleo-heat-templates/roles_data_contrail_ffu.yaml or update the role file with ISSU role and ISSU services.
2. Add the following command environment files:
  - tripleo-heat-templates/environments/contrail/contrail-plugins.yaml
  - tripleo-heat-templates/environments/contrail/contrail-services.yaml
  For example:
  openstack overcloud ffwd-upgrade prepare \ --templates tripleo-heat-templates/ \ --roles-file tripleo-heat-templates/roles_data_contrail_ffu.yaml \ -e tripleo-heat-templates/environments/contrail/contrail-services.yaml \ -e tripleo-heat-templates/environments/contrail/contrail-plugins.yaml \ -e tripleo-heat-templates/environments/contrail/contrail-issu-ffu.yaml
3. Perform the following steps after upgrading all the controller nodes as stated in step 6.2 of Red Hat OpenStack Platform 13 Fast Forward Upgrades:
  Note:
  If the contrail api and nodemgr statuses are Inactive with a failed to connect to keystone error after upgrading the controller nodes, perform the following steps:
  
  docker cp config_api_1:/entrypoint.sh ./ vi entrypoint.sh #!/bin/bash export KEYSTONE_AUTH_URL_VERSION=/v3 docker cp entrypoint.sh config_api_1:/entrypoint.sh docker restart config_api_1
  
  You may have to repeat these exact steps on the config node manager.
  1. Upgrade Contrail control plane nodes.
    
    openstack overcloud upgrade run --nodes ContrailController,ContrailAnalytics,ContrailAnalyticsDatabase --skip-tags validation
  2. Check status of Contrail Networking service.
    
    sudo contrail-status
    
    The status must be active.
  3. Make SSH connection to ISSU node.
  4. Pair ISSU node with upgraded Contrail control plane.
    
    cd /etc/contrail/issu ./issu_node_pair.sh add pair_with_new
  5. Check status of Contrail Networking service.
    
    sudo contrail-status
    
    On ISSU node, the status for all the services must be active.
    
    On Contrail control nodes, the status for all the services except config_device_manager, config_schema and config_svc_monitor must be active. The status for config_device_manager, config_schema and config_svc_monitor must be inactive.
  6. Sync data with new Contrail control plane.
    
    issu_config=issu_revert.conf ./issu_node_sync.sh
  7. Check ISSU Logs.
    
    sudo docker logs issu-run-sync sudo docker exec issu-run-sync cat /var/log/contrail/issu_contrail_run_sync.log
4. Follow Step 6.3 of Red Hat OpenStack Platform 13 Fast Forward Upgrades to upgrade compute nodes.
5. Perform the following steps after upgrading all the compute nodes as stated in step 6.4 of Red Hat OpenStack Platform 13 Fast Forward Upgrades:
  1. Check status of Contrail Networking service on compute nodes.
    
    sudo contrail-status
    
    The status must be active.
  2. Make SSH connection to ISSU node.
  3. Un-pair ISSU node with the old Contrail cluster.
    
    issu_config=issu_revert.conf ./issu_node_sync_post.sh ./issu_node_pair.sh del pair_with_new
  4. Check status of Contrail Networking service on control nodes.
    
    sudo contrail-status
    
    All services must be active.
Follow steps 6.5 till 6.10 of Red Hat OpenStack Platform 13 Fast Forward Upgrades to upgrade CEPH storage node, converged nodes, etc.
Finalize ISSU upgrade.
Remove ISSU node from the cluster.

set ContrailControllerIssuCount: 0

Run stack deploy command.
Follow chapter 7 of Red Hat OpenStack Platform 13 Fast Forward Upgrades to execute post upgrade steps.

arrow_backward PREVIOUS Upgrading Contrail Networking with Red Hat Openstack 13 using ISSU

NEXT arrow_forward How to Upgrade From Contrail Networking Release 3.x or 4.x with RHOSP10 to Contrail Networking Release 1907 with RHOSP13

Contrail Networking Installation and Upgrade Guide

ON THIS PAGE

How to Upgrade From Contrail Networking Release 4.1.4 with RHOSP10 to Contrail Networking Release 1912.L1 or 1912.L2 with RHOSP13

When to Use This Procedure

Before You Begin

Upgrading Contrail Networking Release 4.1.4 with RHOSP10 to Contrail Networking Release 1912.L1 or 1912.L2 with RHOSP13

Stay in touch

Helped me install or use the product
Accelerated my deployment

Discuss the product with a co-worker
Make a purchase inquiry

Contrail Networking Installation and Upgrade Guide

ON THIS PAGE

How to Upgrade From Contrail Networking Release 4.1.4 with RHOSP10 to Contrail Networking Release 1912.L1 or 1912.L2 with RHOSP13

When to Use This Procedure

Before You Begin

Upgrading Contrail Networking Release 4.1.4 with RHOSP10 to Contrail Networking Release 1912.L1 or 1912.L2 with RHOSP13

See Also

Related Documentation

Stay in touch