Installing Contrail Networking for Kubernetes using Helm

1. Download the file contrail-helm-deployer-release-tag.tgz onto your provisioning host. It contains the required two required Helm repositories: /opt/openstack-helm-infra (which contains code to deploy k8s) and /opt/contrail-helm-deployer.
2. Run the command scp contrail-helm-deployer-release-tag.tgz for all nodes in the cluster.
3. Untar contrail-helm-deployer-release-tag.tgz on all nodes:
   content_copy zoom_out_map

   tar -zxf contrail-helm-deployer-release-tag.tgz -C /opt/
   

4. Using any node in the cluster, export the following variables:
   content_copy zoom_out_map

   export BASE_DIR=/opt
   export OSH_INFRA_PATH=${BASE_DIR}/openstack-helm-infra
   export CHD_PATH=${BASE_DIR}/contrail-helm-deployer
   

5. In this step, all the required packages are installed and Kubernetes is deployed. If you want to install a different version of Kubernetes or CNI, edit the file ${OSH_INFRA_PATH}/tools/gate/devel/multinode-vars.yaml. Doing this overrides the default values in ${OSH_INFRA_PATH}/playbooks/vars.yaml. Following is an example multinode-vars.yaml file, with sample values indicated for the private_registries section:
   content_copy zoom_out_map

   version:
     kubernetes: v1.9.3
     helm: v2.7.2
     cni: v0.6.0
   docker:
     # list of insecure_registries, from where you will be pulling container images
     insecure_registries:
       - "10.87.65.243:5000"
     # list of private secure docker registry auth info, from where you will be pulling container images
     #private_registries:
     #  - name: docker-registry-name
     #    username: username@abc.xyz
     #    email: username@abc.xyz
     #    password: password
     #    secret_name: contrail-image-secret
     #    namespace: openstack
   kubernetes:
     network:
       default_device: ens3
     cluster:
       cni: calico
       pod_subnet: 192.168.0.0/16
       domain: cluster.local
   

6. Install the dependent packages using sudo apt-get.
   content_copy zoom_out_map

   sudo apt-get update
   sudo apt-get install --no-install-recommends -y ca-certificates make jq nmap curl uuid-runtime ipcalc linux-headers-$(uname -r)
   

7. Prepare the nodes definition in $OSH_INFRA_PATH/tools/gate/devel/multinode-inventory.yaml, similar to this example:
   content_copy zoom_out_map

   all:
     children:
       primary:
         hosts:
           controller1:
             ansible_port: 22
             ansible_host: 10.10.0.1
             ansible_user: root
             ansible_ssh_extra_args: -o StrictHostKeyChecking=no
             ansible_ssh_private_key_file: /path/to/ssh/key/file
       nodes:
         hosts:
           controller2:
             ansible_port: 22
             ansible_host: 10.10.0.2
             ansible_user: root
             ansible_ssh_extra_args: -o StrictHostKeyChecking=no
             ansible_ssh_private_key_file: /path/to/ssh/key/file
   

8. Deploy k8s to the nodes and use the kubectl get nodes command to verify the deployment is successful.
   content_copy zoom_out_map

   cd ${OSH_INFRA_PATH}
   make dev-deploy setup-host multinode
   make dev-deploy k8s multinode
   
   nslookup kubernetes.default.svc.cluster.local || /bin/true
   kubectl get nodes -o wide
   

9. Set the correct labels for the nodes.
   content_copy zoom_out_map

   kubectl label node controller1.localdomain --overwrite openstack-compute-node=disable
   kubectl label node controller1.localdomain opencontrail.org/controller=enabled
   kubectl label node controller2.localdomain --overwrite openstack-compute-node=disable
   kubectl label node controller2.localdomain opencontrail.org/controller=enabled
   

10. Deploy the OpenContrail charts.
    content_copy zoom_out_map

    cd $CHD_PATH
    	make
    # Change k8s rbac settings
    	kubectl replace -f ${CHD_PATH}/rbac/cluster-admin.yaml
    

11. Prepare the values for Contrail in /tmp/contrail.yml, similar to the following example.
    Note

    This example uses bash variables you should replace with exact values using any preferred means (sed, eval, cat, and so on). Similarly, replace the other variables with actual values where indicated, including IPDATA_SERVICE_HOST, METADATA_PROXY_SECRET, and keystone IP/VIP details.

    content_copy zoom_out_map

    global:
      images:
        tags:
          kafka: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-external-kafka:${CONTRAIL_TAG:-latest}"
          cassandra: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-external-cassandra:${CONTRAIL_TAG:-latest}"
          redis: "redis:4.0.2"
          zookeeper: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-external-zookeeper:${CONTRAIL_TAG:-latest}"
          contrail_control: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-controller-control-control:${CONTRAIL_TAG:-latest}"
          control_dns: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-controller-control-dns:${CONTRAIL_TAG:-latest}"
          control_named: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-controller-control-named:${CONTRAIL_TAG:-latest}"
          config_api: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-controller-config-api:${CONTRAIL_TAG:-latest}"
          config_devicemgr: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-controller-config-devicemgr:${CONTRAIL_TAG:-latest}"
          config_schema_transformer: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-controller-config-schema:${CONTRAIL_TAG:-latest}"
          config_svcmonitor: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-controller-config-svcmonitor:${CONTRAIL_TAG:-latest}"
          webui_middleware: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-controller-webui-job:${CONTRAIL_TAG:-latest}"
          webui: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-controller-webui-web:${CONTRAIL_TAG:-latest}"
          analytics_api: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-analytics-api:${CONTRAIL_TAG:-latest}"
          contrail_collector: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-analytics-collector:${CONTRAIL_TAG:-latest}"
          analytics_alarm_gen: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-analytics-alarm-gen:${CONTRAIL_TAG:-latest}"
          analytics_query_engine: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-analytics-query-engine:${CONTRAIL_TAG:-latest}"
          analytics_snmp_collector: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-analytics-snmp-collector:${CONTRAIL_TAG:-latest}"
          contrail_topology: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-analytics-topology:${CONTRAIL_TAG:-latest}"
          build_driver_init: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-vrouter-kernel-build-init:${CONTRAIL_TAG:-latest}"
          vrouter_agent: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-vrouter-agent:${CONTRAIL_TAG:-latest}"
          vrouter_init_kernel: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-vrouter-kernel-init:${CONTRAIL_TAG:-latest}"
          vrouter_dpdk: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-vrouter-agent-dpdk:${CONTRAIL_TAG:-latest}"
          vrouter_init_dpdk: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-vrouter-kernel-init-dpdk:${CONTRAIL_TAG:-latest}"
          nodemgr: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-nodemgr:${CONTRAIL_TAG:-latest}"
          contrail_status: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-status:${CONTRAIL_TAG:-latest}"
          node_init: "${CONTRAIL_REGISTRY:-opencontrailnightly}/contrail-node-init:${CONTRAIL_TAG:-latest}"
          dep_check: quay.io/stackanetes/kubernetes-entrypoint:v0.2.1
    
      contrail_env:
        CONTROLLER_NODES: 10.10.0.1,10.10.0.2
        LOG_LEVEL: SYS_DEBUG
        CLOUD_ORCHESTRATOR: openstack
        JVM_EXTRA_OPTS: "-Xms1g -Xmx2g"
        BGP_PORT: "1179"
        CONFIG_DATABASE_NODEMGR__DEFAULTS__minimum_diskGB: "2"
        DATABASE_NODEMGR__DEFAULTS__minimum_diskGB: "2"
        IPFABRIC_SERVICE_HOST: metadata IP of old OpenStack setup
        METADATA_PROXY_SECRET: metadata proxy secret of old OpenStack setup
    endpoints:
      keystone:
        auth:
          username: admin
          password: password
          project_name: admin
          user_domain_name: admin_domain
          project_domain_name: admin_domain
          region_name: RegionOne
        hosts:
          default: keystone IP/VIP
        path:
          default: /v3
        port:
          admin:
            default: 35357
          api:
            default: 5000
        scheme:
          default: http
        host_fqdn_override:
          default: keystone IP/VIP
        namespace: null
    

12. If you are using a private registry, add the username and password under the imageCredentials section as follows:
    content_copy zoom_out_map

    global:
      images:
        imageCredentials:
          registry: ${CONTRAIL_REGISTRY:-opencontrailnightly}
          username: ${CONTRAIL_REG_USERNAME}
          password: ${CONTRAIL_REG_PASSWORD}
    

13. Finally, deploy the Contrail charts:
    content_copy zoom_out_map

    helm install --name contrail-thirdparty ${CHD_PATH}/contrail-thirdparty --namespace=contrail --values=/tmp/contrail.yaml
    helm install --name contrail-analytics ${CHD_PATH}/contrail-analytics --namespace=contrail --values=/tmp/contrail.yaml
    helm install --name contrail-controller ${CHD_PATH}/contrail-controller --namespace=contrail --values=/tmp/contrail.yaml
    

After all containers are deployed, you can check cluster status using the contrail-status command. You can also use the Contrail web browser interface to view and verify the cluster status.