Contrail Insights Installation for Containerized OpenStack (OpenStack Kolla, Red Hat OpenStack Platform 13)
On a containerized OpenStack setup, Contrail Insights monitorsOpenStack resources as well as the containers in which the OpenStack services are deployed. This topic provides the steps to deploy Contrail Insights when OpenStack services are running inside Docker containers, without a container orchestration engine. For a setup in which OpenStack is running in containers orchestrated by Kubernetes (OpenStack Helm), see Contrail Insights Installation for OpenStack Helm.
Architecture
Contrail Insights Agent monitors resource usage on OpenStack compute nodes.
Contrail Insights Platform offers REST APIs to configure the system.
Contrail Insights DataManager stores data from multiple Agents.
Contrail Insights Dashboard provides a Web-based user interface.
OpenStack Adapter discovers virtual machines (VMs) running on the OpenStack cluster.
Contrail Insights Agent discovers Docker containers on which OpenStack services are running.
Requirements
For each host, on which Contrail Insights Platform is installed, see Contrail Insights General Requirements for hardware and software requirements. For a list of Contrail Insights Agent supported platforms, see Contrail Insights Agent Requirements.
API access to OpenStack services. Contrail Insights reads information about the OpenStack cluster through the APIs exposed by the various OpenStack services. The user credentials provided during configuration must provide sufficient permission for read-only API calls. Further, Contrail Insights Platform must be able to open a connection to the host and port on which the API server runs.
- Note:
Upgrade notice: Starting with Contrail Insights 3.2.6, the requirement for a license file is removed. If you are installing a version earlier than 3.2.6, a license is required prior to installation.
You can obtain a license key from mailto:APPFORMIX-KEY-REQUEST@juniper.net. Provide the following information in your request:
Group name: Target customers or use: Cluster type: Containerized OpenStack Number of hosts: Number of instances:
Connectivity
One IP address for the Platform host.
Platform host must have IP connectivity to reach compute nodes.
Platform host requires Internet connectivity during installation.
Control plane services are distributed as Docker images that are retrieved from a registry hosted by Contrail Insights. Optionally, the Docker images may be downloaded and served from a local registry.
Dashboard client (in browser) must have IP connectivity to Platform host.
Workflow
The installation includes the following workflow:
Initial setup.
Prerequisites for Agent installation.
Configuring OpenStack.
Contrail Insights Installation.
Optional Configuration.
Initial Setup
To complete the initial setup:
Install Ansible and other required packages on the
appformix_controller
node:#Ubuntu apt-get update apt-get install python-pip python-dev build-essential libssl-dev libffi-dev pip install ansible==2.3.0.0 markupsafe httplib2
#RHEL/CentOS yum install epel-release #Enable EPEL repository In case the above command does not work, manually download and install the epel-release package with one of the below commands, depending on your system’s version. yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-6.noarch.rpm yum groupinstall 'Development Tools' #Install development tools yum install openssl-devel libffi libffi-devel #Dependencies yum install python-pip python-devel #Install Pip pip install ansible==2.3.0 #Install Ansible 2.3 pip install markupsafe httplib2 #Dependencies
Note:For RHEL, the following
iptables
rule is needed to access port 9000:iptables -t filter -A IN_public_allow -p tcp --dport 9000 -j ACCEPT
Install
libvirt
on the compute nodes using the following command:yum -y install libvirt
This installs the KVM (Kernel-based Virtual Machine) and associated packages for collecting data from virtual machines running on the compute nodes.
Install Python
virtualenv
on the vRouter compute nodes where Contrail Insights Agent runs:#Ubuntu apt-get install -y python-pip pip install virtualenv
#RHEL/CentOS yum install -y python-pip pip install virtualenv
Set up passwordless SSH between the Contrail Insights Platform node and the OpenStack cluster nodes. Run the following commands to generate and copy the SSH public keys to all the nodes:
ssh-keygen -t rsa ssh-copy-id -i ~/.ssh/id_rsa.pub <user>@<openstack-node-1> ... ssh-copy-id -i ~/.ssh/id_rsa.pub <user>@<appformix-platform-node>
Prerequisites for Agent Installation
Contrail Insights uses the libvirt
library to collect data from virtual machines running on an OpenStack
compute node. Run the following command on all OpenStack computes
to install the library:
On Ubuntu hosts: sudo apt-get install libvirt0:amd64 On Centos hosts: sudo yum install libvirt-client
Configure OpenStack
Contrail Insights reads information about all projects configured in your OpenStack cluster. The software requires administrator privilege. We recommend you create a new administrator account for Contrail Insights. If you do not create a new administrator account, then you must provide the username and password of an existing administrator account during the configuration of Contrail Insights.
To create an administrator account for Contrail Insights, perform the following steps in the OpenStack Horizon dashboard:
Create a user account and name it
appformix
.Select a new project for the user account.
Select role as
admin
.
Install Contrail Insights
To install Contrail Insights:
Download the Contrail Insights installation packages from software downloads to the Contrail Insights Platform node. Get the following files:
contrail-insights-<version>.tar.gz contrail-insights-dependencies-images-<version>.tar.gz contrail-insights-openstack-images-<version>.tar.gz contrail-insights-platform-images-<version>.tar.gz
If you are installing a version earlier than 3.2.6, copy the Contrail Insights license file to the Contrail Insights Platform node.
Unzip
contrail-insights-<version>.tar.gz
. This package contains all the Ansible playbooks required to install Contrail Insights.tar -xvzf contrail-insights-<version>.tar.gz cd contrail-insights-<version>/
Note:The remaining steps should be executed from within the
contrail-insights-<version>/
directory. Although the product name changed from "AppFormix" to "Contrail Insights," the UI and internal command paths continue to show AppFormix and will reflect the new name at a later date.Using
sample_inventory
as a template, create an inventory file for the installation. List the OpenStack compute nodes in thecompute
section and the Contrail Insights Platform node in theappformix_controller
section. List the node running the OpenStack controller services in theopenstack_controller
section.# List all Compute Nodes [compute] <openstack-compute-node1-ip-or-hostname> <openstack-compute-node2-ip-or-hostname> ... # appformix_controller host [appformix_controller] <appformix-controller-node-ip-or-hostname> [openstack_controller] <openstack-controller-node-ip-or-hostname>
Ensure that all of the hosts mentioned in the inventory file are reachable from the Contrail Insights Platform.
export ANSIBLE_HOST_KEY_CHECKING=False ansible -i inventory -m ping all
Create a directory called
group_vars
. Create a file namedall
inside this directory with configuration variables required by Contrail Insights.mkdir group_vars vi group_vars/all appformix_docker_images: - /path/to/contrail-insights-platform-images-<version>.tar.gz - /path/to/contrail-insights-dependencies-images-<version>.tar.gz - /path/to/contrail-insights-openstack-images-<version>.tar.gz - /path/to/contrail-insights-network_device-images-<version>.tar.gz appformix_dns_version: 2 openstack_platform_enabled: true appformix_docker_container_discovery: true openstack_deployment_mode: containerized_openstack
If you are installing a version earlier than 3.2.6, include the path to the Contrail Insights license file in
group_vars/all
:appformix_license: path/to/<contrail-insights-license-file>.sig
To enable network device monitoring in the cluster, include the following in the
group_vars/all
file:# For enabling prerequisites for package installation. appformix_install_snmp_dependencies: true appformix_install_jti_dependencies: true # For running the appformix-network-device-adapter network_device_discovery_enabled: true appformix_plugins: '{{ appformix_network_device_factory_plugins }}' # After 3.1, SNMP Traps can be enabled also so appformix_plugins can be specified as below: # appformix_plugins: '{{ appformix_network_device_factory_plugins }} + {{ appformix_snmp_trap_factory_plugins }}'
Source the
openrc
file from the OpenStack controller node (/etc/contrail/openstackrc
) to the Contrail Insights Platform node where the playbooks are going to be executed. The file should look like the following:$ cat openrc export OS_USERNAME=admin export OS_PASSWORD=<admin-password> export OS_AUTH_URL=http://172.16.80.2:5000/v2.0/ export OS_NO_CACHE=1 export OS_PROJECT_DOMAIN_NAME=Default export OS_USER_DOMAIN_NAME=Default export OS_PROJECT_NAME=admin export OS_IDENTITY_API_VERSION=3 export OS_IMAGE_API_VERSION=2 $ source openrc
Run Ansible with the created inventory file.
ansible-playbook -i inventory appformix_openstack.yml
The playbook should run to completion without any errors.
Log in to the Contrail Insights Dashboard:
http://<contrail-insights-platform-node-ip>:9000
In the Auth Service dialog box, two options are provided:
OpenStack Log in with OpenStack credentials.
AppFormix Log in with Contrail Insights credentials available at
/opt/appformix/etc/appformix_token.rst
on the Contrail Insights Platform node.
Optional Configuration
To install Contrail Insights certified plug-ins on the cluster,
include the following variables in the group_vars/all
file:
appformix_plugins: <list of certified plugins to be installed> appformix_openstack_log_plugins: <list of OpenStack log plugins to be installed>
Example:
appformix_plugins: - { plugin_info: 'certified_plugins/cassandra_node_usage.json' } - { plugin_info: 'certified_plugins/contrail_vrouter.json' } - { plugin_info: 'certified_plugins/zookeeper_usage.json' } - { plugin_info: 'certified_plugins/heavy_hitters.json' } appformix_openstack_log_plugins: - { plugin_info: 'certified_plugins/cinder_api_logparser.json', log_file_path: '/var/log/cinder/cinder-api.log' } - { plugin_info: 'certified_plugins/glance_logparser.json', log_file_path: '/var/log/glance/glance-api.log' } - { plugin_info: 'certified_plugins/keystone_logparser.json', log_file_path: '/var/log/apache2/keystone_access.log,/var/log/httpd/keystone_wsgi_admin_access.log,/var/log/keystone/keystone.log' }
For a list of all Contrail Insights certified plug-ins that
can be installed, look for the entries starting with plugin_info
in the file roles/appformix_defaults/defaults/main.yml
.
The OpenStack log parser plug-ins parse the API log files of
each OpenStack service to collect metrics about API calls and response
status codes. To install these plug-ins, add them to the variable appformix_openstack_log_plugins
in group_vars/all
, as shown in the example above. Each plug-in entry in this list
requires a parameter called log_file_path
to be specified. This parameter should be set to the complete path
to the service's API log file on the OpenStack Controller node(s).
Multiple comma-separated paths may be specified.
To identify the right log file to be specified in log_file_path
, look for entries like the following,
containing a client IP address, REST call type, and response status
code:
2019-04-02 06:50:13.103 3465 INFO nova.osapi_compute.wsgi.server [req-d07e953a-6921-4224-a056-afb6ff69adde 953ea56a96b944b3b170a299af9e87bd 10c9e8809feb4bd1b55955d9c2ed5aba - - -] 172.18.0.6 "GET /v2/10c9e8809feb4bd1b55955d9c2ed5aba/os-hypervisors/detail HTTP/1.1" status: 200 len: 1427 time: 0.0208740 2019-04-02 06:50:13.183 3465 INFO nova.osapi_compute.wsgi.server [req-34b2f686-9eb5-4112-b3fc-e0b37798a302 953ea56a96b944b3b170a299af9e87bd 10c9e8809feb4bd1b55955d9c2ed5aba - - -] 172.18.0.6 "GET /v2/10c9e8809feb4bd1b55955d9c2ed5aba/servers/detail?all_tenants=1&status=SHELVED_OFFLOADED HTTP/1.1" status: 200 len: 211 time: 0.0754580
Default locations for these files are listed in the variable appformix_openstack_log_factory_plugins
in roles/appformix_defaults/defaults/main.yml
.
In containerized OpenStack environments, log files are generated
inside the containers running the OpenStack services. However, they
have to be available on the OpenStack controller host for the Contrail
Insights plug-ins to be able to read them. The path specified in log_file_path
should be the location of the file on
the OpenStack Controller host.
In Contrail Insights version 3.0, all OpenStack log parser plug-ins
have to be specified in the variable appformix_openstack_log_plugins
. When upgrading from an older version to 3.0, make sure to move
all OpenStack log parser plug-ins defined under appformix_plugins
to appformix_openstack_log_plugins
. Also,
in Contrail Insights 3.0, all entries in this list have to be specified
with a log_file_path
value, as described
above.
Remove a Node from Contrail Insights
To remove a node from Contrail Insights:
Edit the inventory file and add
appformix_state=absent
to each node that you want to remove from Contrail Insights.# Example naming schemes are as below: # hostname ansible_ssh_user='username' ansible_sudo_pass='password' # List all Compute Nodes [compute] 172.16.70.5 appformix_state=absent 172.16.70.17
Run Ansible with the edited inventory file.
ansible-playbook -i inventory appformix_openstack.yml
This removes the node and all its resources from Contrail Insights.