Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
 

Create a Kubernetes Cluster

Use this example procedure to create an upstream Kubernetes cluster.

Note:

This procedure relates to the installation of third party software that is outside of our control. We provide this example to you purely for convenience. Although we make every effort to ensure its accuracy, we cannot guarantee that this example is up-to-date at all times.

There are multiple ways you can create a cluster. In this example, we'll use kubespray (version 2.20).

To make the steps easier to follow, we'll use a separate installer machine to perform the installation and to run kubectl and other tools.

For more information on creating a cluster, see the official Kubernetes documentation (https://kubernetes.io/docs/home/).

Note:

The command line examples below don't show the directory paths. We leave it to you to apply these commands within your directory structure.

Before you start, make sure you've brought up the servers or VMs that you plan to use for the cluster nodes.

  1. Install a fresh OS on the installer machine, configuring the OS minimally for the following:
    • static IP address and mask (for example, 172.16.0.10/24 for our single cluster) and gateway
    • access to one or more DNS servers
    • SSH connectivity including root SSH access
    • NTP
    The installer machine used in our examples is a Ubuntu host attached to the cluster network.
  2. From your local computer, SSH into the installer machine as the root user.
  3. Install kubectl. In this example, we run kubectl on the installer machine. If you want to run kubectl on another machine (for example, your local computer), download and install kubectl on that machine instead.
    1. This downloads kubectl version 1.24.3:
    2. Make the file executable and move it to a directory in your path (for example, /usr/local/bin).
  4. Install the Python virtual environment for the Python version you're running. In this example, we're running Python 3.8.
  5. If you want to install Contrail Analytics, then install Helm 3.0 or later.
    For information on how to install Helm, see https://helm.sh/docs/intro/install/.
  6. Configure SSH password-less root access from the installer machine to the control plane and worker nodes. This allows ansible to log in to these nodes when you run the playbook later.
    1. Create an SSH key.

      In this example, we store the SSH key in its default location ~/.ssh/id_rsa.pub.

    2. Copy the key to the root user on the control plane and worker nodes. For example:
  7. Clone the kubespray directory. In this example, we're cloning release 2.20.
    For example:
  8. Install the required packages to run this version of kubespray. The required packages are listed in kubespray/requirements.txt. We'll install these packages inside a Python virtual environment.
    1. Set up the virtual environment.
      The virtual environment is indicated in the prompt by env.
    2. Install the required packages within the virtual environment.
    Note:

    Perform subsequent steps in this virtual environment until the cluster is set up. This virtual environment ensures you're running the correct version of ansible for this version of kubespray.
  9. Configure the pod and service subnets if desired.
    The default subnets used by kubespray are defined in the kubespray/roles/kubespray-defaults/defaults/main.yaml file. Look for the following parameters in that file and change accordingly.
    Note:

    If you're creating a multi-cluster CN2 setup, you must configure different pod and service subnets on each cluster. These subnets must be unique within the entire multi-cluster.

    If you're following the multi-cluster example in this document, then leave the subnets on the central cluster at their defaults and configure the subnets on the workload cluster as follows:

  10. If you're running DPDK in your cluster, then configure multus.
    Multus is required when running DPDK.
    1. Enable multus.
      In kubespray/roles/kubespray-defaults/defaults/main.yaml, enable multus:
    2. Set the multus version to 0.3.1, which is the version required for running DPDK. You set the version in two files.
      In kubespray/roles/network_plugin/multus/defaults/main.yml, configure the multus version:
      In kubespray/extra_playbooks/roles/network_plugin/multus/defaults/main.yml, configure the multus version:
  11. Create the inventory file for ansible to use. For example:

    The host names (k8s-cp0, k8s-worker0, k8s-worker1) that you specify in the file are automatically configured on the node when the override_system_hostname parameter is set to true.

    Note:

    If you're creating a multi-cluster CN2 setup, you must configure different node names for each node in the multi-cluster. Node names must be unique across the entire multi-cluster.
    Note:

    If you're running DPDK, set kube_network_plugin_multus: true.

    If you want to run with a different container runtime, change the container_manager value above.

    Ensure enable_nodelocaldns is set to false.

    If you want to run with a different number of control plane and worker nodes, adjust the inventory accordingly.

    Place the inventory file outside of the kubespray directory hierarchy.

  12. Check that ansible can SSH into the control plane and worker nodes based on the contents of the inventory.yaml file. In this example, the inventory.yaml file is in the ~/contrail directory.
  13. To create the cluster, run the playbook from the kubespray directory. Adjust the command below to reference the inventory.yaml within your directory structure.
    This step can take a an hour or more to complete, depending on the size of your cluster.
    Note:

    You can safely ignore network and CNI warnings and errors because you haven't configured a CNI yet.

  14. Optionally, deactivate the Python virtual environment. We've finished running ansible, so we no longer need the virtual environment.
  15. Copy the cluster's secure token to the default ~/.kube/config location. The kubeconfig must be at that default location for CN2 tools to work.
    You can find the secure token location from the inventory.yaml file. If you use the inventory file in this example, the token is in /tmp/mycluster.
    Note:

    If you have a kubeconfig that already holds tokens for existing clusters, then you'll need to merge rather than overwrite the ~/.kube/config file.

  16. Use standard kubectl commands to check on the health of the cluster.
    1. Show the status of the nodes.
      You can see that the nodes are not ready because there is no CNI plug-in. This is expected because you haven't installed CN2 yet.
    2. Show the status of the pods.
      All pods should have a STATUS of Running except for the DNS pods. The DNS pods do not come up because there is no networking. This is what we expect.
You have successfully installed a new cluster. You can now proceed to install CN2 on this cluster.