Expand All close

list Table of Contents

English

Using Helm Charts to Provision Multinode Contrail OpenStack Ocata with High Availability

Release: Contrail Networking 5.1

date_range 20-Nov-20

This is the installation procedure for using Helm charts to provision a multinode Contrail system with OpenStack Ocata and high availability.

Note

Contrail Command is not supported for Helm deployed Contrail clusters.

System Specifications

This procedure uses Juniper OpenStack Helm infrastructure and the OpenStack Helm repository to provision an OpenStack Ocata Contrail multinode deployment.

This procedure is tested with:

Operating system: Ubuntu 17.03.2 LTS
Kernel: 4.4.0-112-generic
Docker: 1.13.1-cs9
Helm: v2.7.2
Kubernetes: v1.9.3
OpenStack: Ocata

For a list of supported platforms, see Supported Platforms Contrail 5.1.

Preparing to Install

This section is the prerequisites needed to prepare your system before provisioning multinode Contrail with OpenStack Ocata and high availability.

Generate SSH key on master node and copy to all nodes, in below example three nodes with IP addresses 10.13.82.43, 10.13.82.44, and 10.13.82.45 are used.

content_copy zoom_out_map
(k8s-master)> ssh-keygen

(k8s-master)> ssh-copy-id -i ~/.ssh/id_rsa.pub 10.13.82.43
(k8s-master)> ssh-copy-id -i ~/.ssh/id_rsa.pub 10.13.82.44
(k8s-master)> ssh-copy-id -i ~/.ssh/id_rsa.pub 10.13.82.45

Make sure NTP is configured in all nodes and each node is synched to the time-server in your environment. In below example the NTP server IP is "10.84.5.100".

content_copy zoom_out_map
(k8s-all-nodes)> ntpq -p
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
*10.84.5.100     66.129.255.62    2 u   15   64  377   72.421  -22.686   2.628

Get the contrail-helm-deployer.
From Juniper Networks, download contrail-helm-deployer-5.1.0-0.38.tgz onto your provisioning host.
- scp contrail-helm-deployer-5.1.0-0.38.tgz to all nodes on your cluster.
- Untar contrail-helm-deployer-5.1.0-0.38.tgz on all nodes.
  tar -zxf contrail-helm-deployer-5.1.0-0.38.tgz -C /opt/

Export required variables.

content_copy zoom_out_map
(k8s-master)> cd /opt
(k8s-master)> export BASE_DIR=$(pwd)
(k8s-master)> export OSH_PATH=${BASE_DIR}/openstack-helm
(k8s-master)> export OSH_INFRA_PATH=${BASE_DIR}/openstack-helm-infra
(k8s-master)> export CHD_PATH=${BASE_DIR}/contrail-helm-deployer

Install necessary packages and deploy Kubernetes.

Note
If you want to install a different version of Kubernetes, CNI, or Calico, edit ${OSH_INFRA_PATH}/tools/gate/devel/local-vars.yaml to override the default values in ${OSH_INFRA_PATH}/tools/gate/playbooks/vars.yaml.
(k8s-master)> cd ${OSH_PATH} (k8s-master)> ./tools/deployment/developer/common/001-install-packages-opencontrail.sh

Create an inventory file on the master node for Ansible base provisioning. In the following output, 10.13.82.43/.44/.45 are the IP addresses of the nodes, and will use the SSK-key generated in step 1.

content_copy zoom_out_map
#!/bin/bash
(k8s-master)> set -xe
(k8s-master)> cat > /opt/openstack-helm-infra/tools/gate/devel/multinode-inventory.yaml <<EOF
all:
 children:
   primary:
     hosts:
       node_one:
         ansible_port: 22
         ansible_host: 10.13.82.43
         ansible_user: root
         ansible_ssh_private_key_file: /root/.ssh/id_rsa
         ansible_ssh_extra_args: -o StrictHostKeyChecking=no
   nodes:
     hosts:
       node_two:
         ansible_port: 22
         ansible_host: 10.13.82.44
         ansible_user: root
         ansible_ssh_private_key_file: /root/.ssh/id_rsa
         ansible_ssh_extra_args: -o StrictHostKeyChecking=no
       node_three:
         ansible_port: 22
         ansible_host: 10.13.82.45
         ansible_user: root
         ansible_ssh_private_key_file: /root/.ssh/id_rsa
         ansible_ssh_extra_args: -o StrictHostKeyChecking=no
EOF

Create an environment file on the master node for the cluster.

Note

By default. Kubernetes v1.9.3, Helm v2.7.2, and CNI v0.6.0 are installed. If you want to install a different version, edit the ${OSH_INFRA_PATH}/tools/gate/devel/multinode-vars.yaml file to override the values given in ${OSH_INFRA_PATH}/playbooks/vars.yaml.

Sample multinode-vars.yaml :

content_copy zoom_out_map
(k8s-master)> cat > /opt/openstack-helm-infra/tools/gate/devel/multinode-vars.yaml <<EOF
# version fields
version:
 kubernetes: v1.9.3
 helm: v2.7.2
 cni: v0.6.0

kubernetes:
 network:
   # enp0s8 is your control/data interface, to which kubernetes will bind to
   default_device: enp0s8
 cluster:
   cni: calico
   pod_subnet: 192.168.0.0/16
   domain: cluster.local
docker:
 # list of insecure_registries, from where you will be pulling container images
 insecure_registries:
   - "10.87.65.243:5000"
 # list of private secure docker registry auth info, from where you will be pulling container images
 #private_registries:
 #  - name: <docker-registry-name>
 #    username: username@abc.xyz
 #    email: username@abc.xyz
 #    password: password
 #    secret_name: contrail-image-secret
 #    namespace: openstack
EOF

Run playbooks on the master node.

content_copy zoom_out_map
(k8s-master)> set -xe
(k8s-master)> cd ${OSH_INFRA_PATH}
(k8s-master)> make dev-deploy setup-host multinode
(k8s-master)> make dev-deploy k8s multinode

Verify the kube-dns connection from all nodes. Use nslookup to verify that you are able to resolve Kubernetes cluster-specific names.

content_copy zoom_out_map
 (k8s-all-nodes)> nslookup
  > kubernetes.default.svc.cluster.local
  Server:         10.96.0.10
  Address:        10.96.0.10#53

  Non-authoritative answer:
  Name:   kubernetes.default.svc.cluster.local
  Address: 10.96.0.1

Installation of OpenStack Helm Charts

Use this procedure to install the OpenStack Helm charts.

Before installing the OpenStack Helm charts, review the default labels for the nodes.
The default nodes have the labels openstack-control-plane and openstack-compute-node.The default configuration creates OpenStack Helm (OSH) pods on all the nodes. Use the following commands to check the default OpenStack labels.
(k8s-master)> kubectl get nodes -o wide -l openstack-control-plane=enabled (k8s-master)> kubectl get nodes -o wide -l openstack-compute-node=enabled
If you need to restrict the creation of OSH pods on specific nodes, disable the OpenStack labels. The following example shows how to disable the openstack-compute-node label on the ubuntu-contrail-9 node.
(k8s-master)> kubectl label node ubuntu-contrail-9 --overwrite openstack-compute-node=disabled

Deploy OpenStack Helm charts.

content_copy zoom_out_map
(k8s-master)> set -xe
  (k8s-master)> cd ${OSH_PATH}

  (k8s-master)> ./tools/deployment/multinode/010-setup-client.sh
  (k8s-master)> ./tools/deployment/multinode/021-ingress-opencontrail.sh
  (k8s-master)> ./tools/deployment/multinode/030-ceph.sh
  (k8s-master)> ./tools/deployment/multinode/040-ceph-ns-activate.sh
  (k8s-master)> ./tools/deployment/multinode/050-mariadb.sh
  (k8s-master)> ./tools/deployment/multinode/060-rabbitmq.sh
  (k8s-master)> ./tools/deployment/multinode/070-memcached.sh
  (k8s-master)> ./tools/deployment/multinode/080-keystone.sh
  (k8s-master)> ./tools/deployment/multinode/090-ceph-radosgateway.sh
  (k8s-master)> ./tools/deployment/multinode/100-glance.sh
  (k8s-master)> ./tools/deployment/multinode/110-cinder.sh
  (k8s-master)> ./tools/deployment/multinode/131-libvirt-opencontrail.sh
  # Edit ${OSH_PATH}/tools/overrides/backends/opencontrail/nova.yaml and
  # ${OSH_PATH}/tools/overrides/backends/opencontrail/neutron.yaml
  # to make sure that you are pulling init container image from correct registry and tag
  (k8s-master)> ./tools/deployment/multinode/141-compute-kit-opencontrail.sh
  (k8s-master)> ./tools/deployment/developer/ceph/100-horizon.sh

Installation of Contrail Helm Charts

Use this procedure to install the Contrail Helm charts.

Label the Contrail pods. All Contrail pods are to be deployed in the namespace contrail, using the following labels:
- Controller components—config, control, analytics
- vRouter kernel—opencontrail.org/vrouter-kernel
- vRouter DPDK—opencontrail.org/vrouter-dpdk
The following example shows how to label ubuntu-contrail-11 as DPDK and label ubuntu-contrail-10as kernel vrouter.
(k8s-master)> kubectl label node ubuntu-contrail-11 opencontrail.org/vrouter-dpdk=enabled (k8s-master)> kubectl label node ubuntu-contrail-10 opencontrail.org/vrouter-kernel=enabled (k8s-master)> kubectl label nodes ubuntu-contrail-9 ubuntu-contrail-10 ubuntu-contrail-11 opencontrail.org/controller=enabled

Create Kubernetes ClusterRoleBinding for Contrail.

content_copy zoom_out_map
(k8s-master)> cd $CHD_PATH
(k8s-master)> kubectl replace -f ${CHD_PATH}/rbac/cluster-admin.yaml

Set up the Contrail Helm charts and set the configuration settings specific to your system in the values.yaml file for each of the charts.

content_copy zoom_out_map
 (k8s-master)> cd $CHD_PATH
 (k8s-master)> make

 # Please note in below example, 192.168.1.0/24 is "Control/Data" network
 # Export variables
 (k8s-master)> export CONTROLLER_NODES="192.168.1.43,192.168.1.44,192.168.1.45"
 (k8s-master)> export VROUTER_GATEWAY="192.168.1.1"
 (k8s-master)> export CONTROL_DATA_NET_LIST="192.168.1.0/24"
 (k8s-master)> export BGP_PORT="1179"

 # [Optional] By default, it will pull latest image from opencontrailnightly

 (k8s-master)> export CONTRAIL_REGISTRY="opencontrailnightly"
 (k8s-master)> export CONTRAIL_TAG="latest"

 # [Optional] only if you are pulling images from a private docker registry
 export CONTRAIL_REG_USERNAME="abc@abc.com"
 export CONTRAIL_REG_PASSWORD="password"

 tee /tmp/contrail-env-images.yaml << EOF
 global:
   contrail_env:
     CONTROLLER_NODES: ${CONTROLLER_NODES}
     CONTROL_NODES: ${CONTROL_NODES:-CONTROLLER_NODES}
     LOG_LEVEL: SYS_NOTICE
     CLOUD_ORCHESTRATOR: openstack
     AAA_MODE: cloud-admin
     VROUTER_GATEWAY: ${VROUTER_GATEWAY}
     BGP_PORT: ${BGP_PORT}
   contrail_env_vrouter_kernel:
     CONTROL_DATA_NET_LIST: ${CONTROL_DATA_NET_LIST}
     AGENT_MODE: nic
   contrail_env_vrouter_dpdk:
     AGENT_MODE: dpdk
   images:
     tags:
       kafka: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-external-kafka:${CONTRAIL_TAG:-latest}"
       cassandra: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-external-cassandra:${CONTRAIL_TAG:-latest}"
       redis: "redis:4.0.2"
       zookeeper: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-external-zookeeper:${CONTRAIL_TAG:-latest}"
       contrail_control: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-controller-control-control:${CONTRAIL_TAG:-latest}"
       control_dns: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-controller-control-dns:${CONTRAIL_TAG:-latest}"
       control_named: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-controller-control-named:${CONTRAIL_TAG:-latest}"
       config_api: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-controller-config-api:${CONTRAIL_TAG:-latest}"
       config_devicemgr: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-controller-config-devicemgr:${CONTRAIL_TAG:-latest}"
       config_schema_transformer: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-controller-config-schema:${CONTRAIL_TAG:-latest}"
       config_svcmonitor: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-controller-config-svcmonitor:${CONTRAIL_TAG:-latest}"
       webui_middleware: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-controller-webui-job:${CONTRAIL_TAG:-latest}"
       webui: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-controller-webui-web:${CONTRAIL_TAG:-latest}"
       analytics_api: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-analytics-api:${CONTRAIL_TAG:-latest}"
       contrail_collector: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-analytics-collector:${CONTRAIL_TAG:-latest}"
       analytics_alarm_gen: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-analytics-alarm-gen:${CONTRAIL_TAG:-latest}"
       analytics_query_engine: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-analytics-query-engine:${CONTRAIL_TAG:-latest}"
       analytics_snmp_collector: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-analytics-snmp-collector:${CONTRAIL_TAG:-latest}"
       contrail_topology: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-analytics-snmp-topology:${CONTRAIL_TAG:-latest}"
       build_driver_init: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-vrouter-kernel-build-init:${CONTRAIL_TAG:-latest}"
       vrouter_agent: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-vrouter-agent:${CONTRAIL_TAG:-latest}"
       vrouter_init_kernel: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-vrouter-kernel-init:${CONTRAIL_TAG:-latest}"
       vrouter_dpdk: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-vrouter-agent-dpdk:${CONTRAIL_TAG:-latest}"
       vrouter_init_dpdk: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-vrouter-kernel-init-dpdk:${CONTRAIL_TAG:-latest}"
       nodemgr: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-nodemgr:${CONTRAIL_TAG:-latest}"
       contrail_status: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-status:${CONTRAIL_TAG:-latest}"
       node_init: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-node-init:${CONTRAIL_TAG:-latest}"
       dep_check: quay.io/stackanetes/kubernetes-entrypoint:v0.2.1
EOF

Note

If any other environment variables need to be added, add them in the values.yaml file of the respective charts.

content_copy zoom_out_map
# [Optional] only if you are pulling contrail images from a private registry
tee /tmp/contrail-registry-auth.yaml << EOF
global:
  images:
    imageCredentials:
      registry: ${CONTRAIL_REGISTRY:-opencontrailnightly}
      username: ${CONTRAIL_REG_USERNAME}
      password: ${CONTRAIL_REG_PASSWORD}
EOF

# [Optional] only if you are pulling images from a private registry
export CONTRAIL_REGISTRY_ARG="--values=/tmp/contrail-registry-auth.yaml "

Use Helm install commands to deploy each of the Contrail Helm charts.

content_copy zoom_out_map
 (k8s-master)> helm install --name contrail-thirdparty ${CHD_PATH}/contrail-thirdparty \
  --namespace=contrail \
  --values=/tmp/contrail-env-images.yaml \
  ${CONTRAIL_REGISTRY_ARG}

  (k8s-master)> helm install --name contrail-controller ${CHD_PATH}/contrail-controller \
  --namespace=contrail \
  --values=/tmp/contrail-env-images.yaml \
  ${CONTRAIL_REGISTRY_ARG}

  (k8s-master)> helm install --name contrail-analytics ${CHD_PATH}/contrail-analytics \
  --namespace=contrail \
  --values=/tmp/contrail-env-images.yaml \
  ${CONTRAIL_REGISTRY_ARG}

  # Edit contrail-vrouter/values.yaml and make sure that global.images.tags.vrouter_init_kernel is right. Image tag name will be different depending upon your linux. Also set the global.node.host_os to ubuntu or centos depending on your system

  (k8s-master)> helm install --name contrail-vrouter ${CHD_PATH}/contrail-vrouter \
  --namespace=contrail \
  --values=/tmp/contrail-env-images.yaml \
  ${CONTRAIL_REGISTRY_ARG}

When the Contrail pods are up and running, deploy the OpenStack Heat chart.

content_copy zoom_out_map
# Edit ${OSH_PATH}/tools/overrides/backends/opencontrail/nova.yaml and
# ${OSH_PATH}/tools/overrides/backends/opencontrail/heat.yaml  
# to make sure that you are pulling the right opencontrail init container image
(k8s-master)> ./tools/deployment/multinode/151-heat-opencontrail.sh

When finished, run the compute kit test.
(k8s-master)> ./tools/deployment/multinode/143-compute-kit-opencontrail-test.sh

Basic Testing OpenStack Helm Contrail Cluster

Use the following commands to perform basic testing on the virtual network and the virtual machines in your OpenStack Helm Contrail cluster.

content_copy zoom_out_map
(k8s-master)> export OS_CLOUD=openstack_helm

(k8s-master)> openstack network create MGMT-VN
(k8s-master)> openstack subnet create --subnet-range 172.16.1.0/24 --network MGMT-VN MGMT-VN-subnet

(k8s-master)> openstack server create --flavor m1.tiny --image 'Cirros 0.3.5 64-bit' \
--nic net-id=MGMT-VN \
Test-01

(k8s-master)> openstack server create --flavor m1.tiny --image 'Cirros 0.3.5 64-bit' \
--nic net-id=MGMT-VN \
Test-02

Accessing the Contrail OpenStack Helm Cluster

Use the following topic to access the OpenStack and Contrail Web UI and prepare the OpenStack client for command-line interface (CLI):

Accessing a Contrail OpenStack Helm Cluster

ON THIS PAGE

Using Helm Charts to Provision Multinode Contrail OpenStack Ocata with High Availability

System Specifications

Preparing to Install

Installation of OpenStack Helm Charts

Installation of Contrail Helm Charts

Basic Testing OpenStack Helm Contrail Cluster

Accessing the Contrail OpenStack Helm Cluster

Related Documentation

Stay in touch

Helped me install or use the product
Accelerated my deployment

Discuss the product with a co-worker
Make a purchase inquiry