Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
 

Manage Multi-Cluster Contrail

SUMMARY Learn how to perform life cycle management tasks specific to a multi-cluster installation.

This section covers tasks that are specific to a multi-cluster installation. If you want to perform management tasks in a specific cluster (such as adding and removing nodes within a cluster and upgrading a cluster) within the multi-cluster installation, then see Manage Single Cluster Contrail.

Attach a Workload Cluster

Attach a Workload Cluster Running Kernel Mode Data Plane in Release 22.1

Use this procedure to create and attach a distributed workload cluster (running a kernel mode data plane) to the central cluster in release 22.1. If you want to run a DPDK data plane, then extrapolate from the DPDK examples in this document.

The manifests that you will use in this example procedure are multi-cluster/dist-kernel/distributed_cluster_deployer.yaml and multi-cluster/dist-kernel/deployer_ha_kernel.yaml. The procedure assumes that you've placed these manifests into a manifests directory.

  1. Prepare the distributed workload cluster as described in Before You Install.
  2. Create the distributed workload cluster.

    Create a fresh cluster. You can follow the procedure in Create a Kubernetes Cluster or use your own methods to create a cluster. The cluster should have the following characteristics:

    • Cluster has no CNI plug-in.
    • Disable Node Local DNS.
    • In a multi-cluster setup, you must configure different pod and service subnets on each cluster. These subnets must be unique within the entire multi-cluster.
    • In a multi-cluster setup, you must configure different node names for nodes in each cluster. Node names must be unique across the entire multi-cluster.
  3. Install Contrail components on the distributed workload cluster.
    1. On the workload cluster, create the contrail-deploy namespace.
    2. On the workload cluster, copy the kubeconfig from the central cluster. We'll call this central-cluster-kubeconfig here.
    3. On the workload cluster, create a Kubernetes secret from the central cluster kubeconfig and name the secret central-kubeconfig.
      Note:

      You must name the secret central-kubeconfig.

      Note:

      You must specify the absolute path to the central-cluster-kubeconfig file.

    4. On the workload cluster, apply the deployer manifest. The deployer provides life cycle management for the Contrail components.
      This manifest includes a reference to the central-kubeconfig secret you created in the previous substep.
    5. Check that the contrail-deployer has come up. This may take a few minutes.
  4. On the central cluster, attach the new workload cluster by creating a kubemanager for the new workload cluster.
    1. On the central cluster, copy the kubeconfig from the distributed workload cluster. We'll call this workload-cluster-kubeconfig here.
    2. On the central cluster, create a Kubernetes secret from the distributed workload cluster kubeconfig and choose a meaningful name for the secret (for example, workload-kubeconfig).
    3. Create the kubemanager manifest with the following content. Choose a meaningful name for the manifest (for example, kubemanager-cluster1.yaml).
      Table 1 explains the parameters that you need to set.
      Table 1: Kubemanager CRD
      Parameter Meaning Example
      name The name of the custom resource. kubemanager-cluster1
      image The repository where you pull images hub.juniper.net/cn2/contrail-k8s-kubemanager:22.1.0.93
      podV4Subnet The IPv4 pod subnet that you configured earlier for the distributed workload cluster.

      This subnet must be unique within the entire multi-cluster.

      10.234.64.0
      serviceV4Subnet The IPv4 service subnet that you configured earlier for the distributed workload cluster.

      This subnet must be unique within the entire multi-cluster.

      10.234.0.0/18
      podV6Subnet The IPv6 pod subnet that you configured earlier for the distributed workload cluster.

      This subnet must be unique within the entire multi-cluster.

      fd85:ee78:d8a6:8608::1:0000/112
      serviceV6Subnet The IPv6 service subnet that you configured earlier for the distributed workload cluster.

      This subnet must be unique within the entire multi-cluster.

      fd85:ee78:d8a6:8608::1000/116
      clusterName The name of the workload cluster. workload-cluster
      kubeconfigSecretName The name of the secret containing the workload cluster kubeconfig token. workload-kubeconfig
      enableNad True or false (to enable network address definition or not) true
      listenerPort The port that the Contrail controller listens on for communications with this workload cluster.

      Set the port for the first workload cluster to 19446 and increment by 1 for each subsequent workload cluster.

      19446
      constantRouteTargetNumber The route target for this workload cluster.

      Set the route target for the first workload cluster to 7699 and increment by 100 for each subsequent workload cluster.

      7699
    4. On the central cluster, apply the kubemanager manifest you just created.
    5. On the central cluster, verify that you can see the workload cluster's namespaces.

      The namespaces are in the following format: <kubemanager-name>-<workload-cluster-name>-<namespace>. For example:

  5. On the workload cluster, modify the deployer_ha_kernel.yaml manifest as necessary. The deployer_ha_kernel.yaml manifest that we provide is a sample that you may need to tailor to your setup.

    The provided deployer_ha_kernel.yaml file specifies different networks for the different cluster nodes, which is different from our example cluster. In our cluster, the cluster nodes all connect to the same data network. We'll need to modify deployer_ha_kernel.yaml to remove the extra networks and extra node configuration.

    1. Remove the extra networks.

      Edit deployer_ha_kernel.yaml to remove the following lines that define the contrail-network-config ConfigMap:

    2. Remove the following lines that define the contrail-vrouter-nodes-7 vRouter:
    3. Similarly, remove the contrail-vrouter-nodes-8 section completely
    4. Similarly, remove the contrail-vrouter-nodes-14 section completely
    5. Add the following contrail-vrouter-nodes section to the end:
  6. Finally, on the workload cluster, install the vRouter.

    After a few minutes, verify that all pods are up:

You've now created and attached a distributed workload cluster to the central cluster.

Attach a Workload Cluster Running Kernel Mode Data Plane in Release 22.2 or Higher

Use this procedure to create and attach a distributed workload cluster (running a kernel mode data plane) to the central cluster in release 22.2 or higher. If you want to run a DPDK data plane, then extrapolate from the DPDK examples in this document.

The manifests that you will use in this example procedure are multi-cluster/distributed_cluster_deployer_example.yaml and multi-cluster/distributed_cluster_vrouter_example.yaml. The procedure assumes that you've placed these manifests into a manifests directory.

  1. Prepare the distributed workload cluster as described in Before You Install.
  2. Create the distributed workload cluster.

    Create a fresh cluster. You can follow the procedure in Create a Kubernetes Cluster or use your own methods to create a cluster. The cluster should have the following characteristics:

    • Cluster has no CNI plug-in.
    • Disable Node Local DNS.
    • In a multi-cluster setup, you must configure different pod and service subnets on each cluster. These subnets must be unique within the entire multi-cluster.
    • In a multi-cluster setup, you must configure different node names for nodes in each cluster. Node names must be unique across the entire multi-cluster.
  3. Install Contrail components on the distributed workload cluster.
    1. On the workload cluster, copy the kubeconfig from the central cluster. We'll call this central-cluster-kubeconfig here.
    2. On the workload cluster, create a Kubernetes secret from the central cluster kubeconfig and name the secret central-kubeconfig.
      Note:

      You must name the secret central-kubeconfig.

      Note:

      You must specify the absolute path to the central-cluster-kubeconfig file.

    3. On the workload cluster, apply the deployer manifest. The deployer provides life cycle management for the Contrail components.
      This manifest includes a reference to the central-kubeconfig secret you created in the previous substep.
    4. Check that the contrail-deployer has come up. This may take a few minutes.
  4. On the central cluster, attach the new workload cluster by creating a kubemanager for the new workload cluster.
    1. On the central cluster, copy the kubeconfig from the distributed workload cluster. We'll call this workload-cluster-kubeconfig here.
    2. On the central cluster, create a Kubernetes secret from the distributed workload cluster kubeconfig and choose a meaningful name for the secret (for example, workload-kubeconfig).
    3. Create the kubemanager manifest with the following content. Choose a meaningful name for the manifest (for example, kubemanager-cluster1.yaml).
      Table 2 explains the parameters that you need to set.
      Table 2: Kubemanager CRD
      Parameter Meaning Example
      name The name of the custom resource. kubemanager-cluster1
      image The repository where you pull images

      hub.juniper.net/cn2/contrail-k8s-kubemanager:22.1.0.93

      enterprise-hub.juniper.net/contrail-container-prod/contrail-k8s-kubemanager:22.3.0.71

      podV4Subnet The IPv4 pod subnet that you configured earlier for the distributed workload cluster.

      This subnet must be unique within the entire multi-cluster.

      10.234.64.0
      serviceV4Subnet The IPv4 service subnet that you configured earlier for the distributed workload cluster.

      This subnet must be unique within the entire multi-cluster.

      10.234.0.0/18
      podV6Subnet The IPv6 pod subnet that you configured earlier for the distributed workload cluster.

      This subnet must be unique within the entire multi-cluster.

      fd85:ee78:d8a6:8608::1:0000/112
      serviceV6Subnet The IPv6 service subnet that you configured earlier for the distributed workload cluster.

      This subnet must be unique within the entire multi-cluster.

      fd85:ee78:d8a6:8608::1000/116
      clusterName The name of the workload cluster. workload-cluster
      kubeconfigSecretName The name of the secret containing the workload cluster kubeconfig token. workload-kubeconfig
      enableNad True or false (to enable network address definition or not) true
      listenerPort The port that the Contrail controller listens on for communications with this workload cluster.

      Set the port for the first workload cluster to 19446 and increment by 1 for each subsequent workload cluster.

      19446
      constantRouteTargetNumber The route target for this workload cluster.

      Set the route target for the first workload cluster to 7699 and increment by 100 for each subsequent workload cluster.

      7699
    4. On the central cluster, apply the kubemanager manifest you just created.
    5. On the central cluster, verify that you can see the workload cluster's namespaces.

      The namespaces are in the following format: <kubemanager-name>-<workload-cluster-name>-<namespace>. For example:

  5. Finally, on the workload cluster, install the vRouter.

    After a few minutes, verify that all pods are up:

You've now created and attached a distributed workload cluster to the central cluster.

Detach a Workload Cluster

Use this procedure to detach a distributed workload cluster from the central cluster.
  1. On the central cluster, delete the associated kubemanager.
  2. On the distributed workload cluster that you're deleting, delete the vRouters.
  3. On the central cluster, delete the namespaces of the distributed workload cluster.
    List the namespaces and delete all namespaces associated with the distributed workload cluster.
  4. On the central cluster, delete the secret associated with the workload cluster that you're detaching.
    where <secret-name> is the secret you created when you attached the workload cluster to the central cluster earlier. In our example, we called it workload-kubeconfig.

Uninstall Contrail

Use this procedure to uninstall Contrail from the central cluster and the workload clusters.
  1. Follow the steps in Detach a Workload Cluster to detach the workload cluster that you want to delete.
  2. Uninstall Contrail in the workload cluster.
    1. Follow the steps in Uninstall Contrail to uninstall Contrail in the workload cluster.

      Ignore any NotFound errors because those resources have already been deleted.

    2. On the workload cluster, verify that all resources related to default pod network namespace of the distributed workload cluster are gone, and verify that all resources related to the contrail namespace are gone.

    If all you want to do is to uninstall Contrail from the workload cluster, then you're done. If you want to also uninstall Contrail from the central cluster, then proceed to the next step.

  3. Uninstall Contrail in the central cluster.
    1. Follow the steps in Uninstall Contrail to uninstall Contrail in the central cluster.

      Ignore any NotFound errors because those resources have already been deleted.

    2. On the central cluster, verify that all resources related to the contrail, contrail-system, and contrail-deploy namespaces are deleted.