Install Multinode Cluster on Red Hat Enterprise Linux

We refer to this directory as pa-download-dir in this guide.

The Paragon Automation setup installation bundle consists of the following scripts and TAR files to install each of the component modules:

• davinci.tar.gz, which is the primary installer file.

• infra.tar, which installs the Kubernetes infrastructure components including Docker and Helm.

• ems.tar, which installs the base platform component.

• northstar.tar, which installs the Paragon Pathfinder and Paragon Planner components.

• healthbot.tar, which installs the Paragon Insights component.

• paragon_ui.tar, which installs the Paragon Automation UI component.

• addons.tar, which installs infrastructure components that are not part of the base Kubernetes installation. The infrastructure components include, IAM, Kafka, ZooKeeper, cert-manager, Ambassador, Postgres, Metrics, Kubernetes Dashboard, Open Distro for Elasticsearch, Fluentd, Reloader, ArangoDB, and Argo.

• rhel-84-airgap.tar.gz, which installs Paragon Automation using the air-gap method on nodes only where the base OS is Red Hat Enterprise Linux (RHEL).

• run script, which executes the installer image.

Now that you've downloaded the software, you're ready to install Paragon Automation.

config-dir is a user-defined directory on the control host that contains configuration information for a particular installation. The init command automatically creates the directory if it does not exist. Alternatively, you can create the directory before you execute the init command.

Ensure that you include the dot and slash (./) with the run command.

If you are using the same control host to manage multiple installations of Paragon Automation, you can differentiate between installations by using differently named configuration directories.

Copy the private key that you generated in Configure SSH client authentication to the user-defined config-dir directory. The installer allows the Docker container to access the config-dir directory. The SSH key must be available in the directory for the control host to connect to the cluster nodes.

Ensure that you include the dot (.) at the end of the copy command (cp).

1. Customize the inventory file using the inv command:

   The following table lists the configuration options that the inv command prompts you to enter.

   Table 1: inv Command Options

   inv Command Prompts	Description
   Kubernetes master nodes	Enter IP addresses of the Kubernetes primary nodes.
   Kubernetes worker nodes	Enter IP addresses of the Kubernetes worker nodes.
   Local storage nodes	Define the nodes that have disk space available for applications. The local storage nodes are prepopulated with the IP addresses of the primary and worker nodes. You can edit these addresses. Enter IP addresses of the nodes on which you want to run applications that require local storage. Services such as Postgres, ZooKeeper, and Kafka use local storage or disk space partitioned inside export/local-volumes. By default, worker nodes have local storage available. If you do not add primary nodes here, you can run only those applications that do not require local storage on the primary nodes. Note: Local storage is different from Ceph storage.
   External registry nodes	(Optional) Configure an existing external user registry.
   Kubernetes nodes' username (for example, root)	Configure the user account and authentication methods to authenticate the installer with the cluster nodes. The user account must be root or, in the case of non-root users, the account must have superuser (sudo) privileges.
   SSH private key file (optional)	If you chose ssh-key authentication, for the control host to authenticate with the nodes during the installation process, configure the directory (config-dir) where the ansible_ssh_private_key_file is located, and the id_rsa file, as "{{ config-dir }}/id_rsa".
   Kubernetes nodes' password (optional)	If you chose password authentication for the control host to authenticate with the nodes during the installation process, enter the authentication password directly. WARNING: The password is written in plain text. We do not recommend using this option for authentication.
   Kubernetes cluster name (optional)	Enter a name for your Kubernetes cluster.
   Write inventory file?	Click Yes to save the inventory information.

   For example:

2. Alternatively, you can customize the inventory file manually using a text editor.

   Edit the following groups in the inventory file.

   1. Add the IP addresses of the Kubernetes primary and worker nodes of the cluster.

      The master group identifies the primary nodes, and the node group identifies the worker nodes. You cannot have the same IP address in both master and node groups.

      To create a multi-primary node setup, list the addresses or hostnames of all the nodes that will be acting as primary nodes under the master group. Add the addresses or hostnames of the nodes that will be acting as worker nodes under the node group.

   2. Define the nodes that have disk space available for applications under the local_storage_nodes:children group.

   3. Configure the user account and authentication methods to authenticate the installer in the Ansible control host with the cluster nodes under the vars group.

   4. (Optional) Specify a name for your Kubernetes cluster in the kubernetes_cluster_name group.

The conf command runs an interactive installation wizard that enables you to choose the components you want to install and configure a basic Paragon Automation setup. The command populates the config.yml file with your input configuration. For advanced configuration, you must edit the config.yml file manually.

Enter the information as prompted by the wizard. Use the cursor keys to move the cursor, use the space key to select an option, and use the a or i key to toggle selecting or clearing all options. Press Enter to move to the next configuration option. You can skip configuration options by entering a period (.). You can reenter all your choices by exiting the wizard and restarting from the beginning. The installer allows you to exit the wizard after you save the choices that you already made or to restart from the beginning. You cannot go back and redo the choices that you already made in the current workflow without exiting and restarting the wizard altogether.

The following table lists the configuration options that the conf command prompts you to enter :

[edit groups northstar]
root@system# show protocols bgp group northstar
type internal;
family traffic-engineering {
    unicast;
}
export TE;
allow 10.xx.43.0/24;

[edit groups northstar]
root@system# show policy-options policy-statement TE
from family traffic-engineering;
then accept;

[edit protocols bgp group BGP-LS]
root@vmx101# show | display set 
set protocols bgp group BGP-LS family traffic-engineering unicast
set protocols bgp group BGP-LS peer-as 11
set protocols bgp group BGP-LS allow 10.x.43.1
set protocols bgp group BGP-LS allow 10.x.43.2
set protocols bgp group BGP-LS allow 10.x.43.3
set protocols bgp group BGP-LS export TE

[edit protocols bgp group BGP-LS]
root@vmx101# show | display set
set protocols bgp group BGP-LS family traffic-engineering unicast
set protocols bgp group BGP-LS peer-as 11
set protocols bgp group BGP-LS neighbor 10.xx.43.1
set protocols bgp group BGP-LS neighbor 10.xx.43.2
set protocols bgp group BGP-LS neighbor 10.xx.43.3
set protocols bgp group BGP-LS passive
set protocols bgp group BGP-LS export TE

set protocols rsvp interface interface.unit

set protocols isis interface interface.unit
set protocols isis traffic-engineering igp-topology
Or
set protocols ospf area area interface interface.unit
set protocols ospf traffic-engineering igp-topology

set protocols mpls interface interface.unit
set protocols mpls traffic-engineering database import igp-topology

The config.yml file consists of an essential section at the beginning of the file that corresponds to the configuration options that the installation wizard prompts you to enter. The file also has an extensive list of sections under the essential section that allows you to enter complex configuration values directly in the file.

You can configure the following options:

• Set the opendistro_es_admin_password password to log in to the Kibana application. You use Open Distro to consolidate and index application logs and the visualization tool Kibana to search logs using keywords and filters.

  By default, the username is preconfigured as admin in #opendistro_es_admin_user: admin and the install_opendistro_es option is set to true to replace the Elasticsearch version with Open Distro. Use admin as username and this password to log in to Kibana.

  By default, data is retained on the disks for seven days, before being purged, in a production deployment. You can edit the number of days to a smaller number in opendistro_es_retain if your disk size is low.

  If you do not configure the opendistro_es_admin_password password, the installer generates a random password. You can retrieve the password using the command:

  # kubectl -n kube-system get secret opendistro-es-account -o jsonpath={..password} | base64 -d

• Set the iam_skip_mail_verification configuration option to true for user management without SMTP by Identity and Access Management (IAM). By default, this option is set to false for user management with SMTP. You must configure SMTP in Paragon Automation so that you can notify Paragon Automation users when their account is created, activated, or locked, or when their account password is changed.

• Configure the callback_vip option with an IP address different from that of the virtual IP (VIP) address of the ingress controller. You can use an IP address from the MetalLB pool of VIP addresses. You configure this IP address to enable segregation of management and data traffic from the southbound and northbound interfaces. By default, callback_vip is assigned the same or one of the addresses of the ingress controller.

• If you want to use an interface other than the default interface for intercluster communication, set the kubernetes_system_interface variable. The current setting is "{{ ansible_default_ipv4.interface }}", which is the interface that the default route uses. The kubernetes_system_interface variable configures the Kubernetes API server and Calico.

  To view the default interface, run this command on a primary node:

  In this example, ens3 is default interface for this machine.

  If you want to use an interface different from the default one and the same interface can be used on all cluster nodes, configure kubernetes_system_interface in the config.yml file. For example:

  kubernetes_system_interface: ens4

  If you want to use an interface different from the default one but the interface is different on different nodes, you must remove kubernetes_system_interface from the config.yml file. Instead, configure the interface names in the inventory file. For example:

  Note that, calico_ip_autodetect is set to "interface={{ kubernetes_system_interface }}", and takes the same value as kubernetes_system_interface and does not need to be explicitly changed if the default interface is changed.

Save and exit the file after you finish editing it.

By default, Ambassador uses a locally generated certificate signed by the Kubernetes cluster-internal CA.

The installation time to install the configured cluster depends on the complexity of the cluster. A basic setup installation takes at least 45 minutes to complete.

The installer checks NTP synchronization at the beginning of installation. If clocks are out of sync, installation fails.

For multi-primary node deployments only, the installer checks the disk input/output operations per second (IOPS) at the beginning of installation. If the IOPS value is below 300, installation fails. To disable disk IOPS check, use the # ./run -c config-dir deploy -e ignore_iops_check=yes command and rerun the deployment.

If you are installing Paragon Automation on an existing Kubernetes cluster, the deploy command upgrades the currently deployed cluster to the latest Kubernetes version. The command also upgrades the Docker CE version, if required. If Docker EE is already installed on the nodes, the deploy command does not overwrite it with Docker CE. When upgrading the Kubernetes version or the Docker version, the command performs the upgrade sequentially on one node at a time. The command cordons off each node and removes it from scheduling. It performs upgrades, restarts Kubernetes on the node, and finally uncordons the node and brings it back into scheduling.

Use a text editor to configure the following recommended information for Paragon Insights in the limits.conf and sysctl.conf files. These values set the soft and hard memory limits for influx DB memory requirements. If you do not set these limits, you might see errors such as “out of memory” or “too many open files” because of the default system limits.

Repeat this step for all worker nodes.