ON THIS PAGE
Contrail Insights Installation and Configuration for OpenStack
Getting Started with OpenStack
Contrail Insights provides resource control and visibility for hosts and virtual machines in an OpenStack cluster. This topic explains how to install Contrail Insights for an OpenStack cluster. Contrail Insights Agent runs on a host to monitor resource consumption of the host itself and the virtual machines executing on that host. Figure 1 shows the Contrail Insights architecture with OpenStack.
Contrail Insights Agent monitors resource usage on the compute nodes.
Contrail Insights Platform offers REST APIs to configure the system.
Contrail Insights DataManager stores data from multiple Agents.
Contrail Insights Dashboard provides a Web-based user interface.
An adapter discovers platform-specific resources and configures the Contrail Insights Platform.
Adapters exist for OpenStack and Kubernetes.
Requirements
The following are the requirements for installing Contrail Insights for OpenStack.
Supported OpenStack versions: Icehouse, Juno, Kilo, Liberty, Mitaka, Newton, Ocata.
See Contrail Insights General Requirements for software and hardware requirements.
An administrator account for OpenStack.
API access to OpenStack services: Cinder, Glance, Heat, Keystone, Neutron, Nova, and Swift. Contrail Insights reads information from these services. The administrator account must provide sufficient permission for read-only API calls. Further, Contrail Insights Platform must be able to open connections to the host and port on which these services listen. Contrail Insights can be configured to use the
admin,internal, orpublicservice endpoints. SeeOS_ENDPOINT_TYPEin OpenStack environment variables in the section Installing Contrail Insights.- Note:
Upgrade notice: Starting with Contrail Insights 3.2.6, the requirement for a license file is removed. If you are installing a version earlier than 3.2.6, a license is required prior to installation.
You can obtain a license key from mailto:APPFORMIX-KEY-REQUEST@juniper.net. Provide the following information in your request:
Group name: Target customers or use: Cluster type: OpenStack Number of hosts: Number of instances:
Workflow in Three Steps
Installation consists of the following steps:
Configure OpenStack.
Create Ansible inventory.
Install Contrail Insights.
Configure OpenStack
To create an administrator account for Contrail Insights, perform the following steps in the OpenStack Horizon dashboard:
Create a user account and name it appformix.
Select a new project for the user account.
Select role as admin.
Create Ansible Inventory
Ansible is used to deploy the software to the compute node(s) and the Platform Host. An Ansible inventory file describes groups of hosts in your cluster. Define the inventory in a separate location than the release files, so that the inventory may be reused for an upgrade.
Contrail Insights requires two groups compute and appformix_controller. Each group lists the hosts in that group. Only the agent is installed on the compute hosts. The agent and the Contrail Insights Platform services are installed on the appformix_controller host.
Optionally, an openstack_controller group can be defined. The agent is installed on hosts in this group to monitor the hosts that execute OpenStack controller services. (New in v2.3.0)
Create a directory inventory (or
name of your choice) that contains a hosts file and a group_vars/all file. For example:
inventory/
hosts # inventory file
group_vars/
all # configuration variablesThe inventory/hosts file contains
the list of hosts in each group. For example:
[appformix_controller] appformix01 [compute] compute01 compute02 compute03 [openstack_controller] openstack_infra01 openstack_infra02
See Ansible inventory documentation.
Ansible Configuration Variables
The Contrail Insights software includes a number of Ansible
roles to perform the configuration of Contrail Insights settings.
Define the values of variables in the inventory/group_vars/all file, in order to be able to use the settings and inventory for
future upgrades.
In the inventory/group_vars/all file,
configure the following variables for installation of Contrail Insights
for OpenStack.
appformix_docker_images:
- /path/to/contrail-insights-platform-images-<version>.tar.gz
- /path/to/contrail-insights-dependencies-images<version>.tar.gz
- /path/to/contrail-insights-openstack-images-<version>.tar.gzRefer to Platform Dependencies for steps to install dependencies on a Platform Host that cannot fetch files from the Internet.
Configure an HTTP Proxy for Fetching Prerequisites
The Ansible playbook will fetch files from the Internet to install
prerequisites on the Platform Host. If the Platform Host requires
an HTTP proxy to access the Internet, configure the following variables
in inventory/group_vars/all:
http_proxy_url: 'http://proxy.example.com:1234'
prerequisites_env:
http_proxy: '{{ http_proxy_url }}'
https_proxy: '{{ http_proxy_url }}'The prerequisites_env is a dictionary
that defines environment variables that will be used when invoking
commands to install prerequisites. In the above example, the same
proxy URL (http://proxy.example.com:1234) is used for both the http_proxy and https_proxy environment variables because the single proxy can be used to access
HTTP and HTTPS URLs. As a convenience, the proxy URL is defined once
in the http_proxy_url variable. Adjust prerequisites_env as necessary for the proxy requirements
of your network.
Install Contrail Insights
To install Contrail Insights:
Install Ansible on the Contrail Insights Platform node. Ansible will install docker and docker-py on the platform.
#Ubuntu apt-get install python-pip python-dev #Installs Pip pip install ansible==2.3.0 #Installs Ansible 2.3 sudo apt-get install build-essential libssl-dev libffi-dev #Dependencies pip install markupsafe httplib2 requests #Dependencies #RHEL/CentOS yum install epel-release #Enable EPEL repository In case the above command does not work, manually download and install the epel-release package with one of the below commands, depending on your system’s version. yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-6.noarch.rpm yum groupinstall 'Development Tools' #Install development tools yum install openssl-devel libffi libffi-devel #Dependencies yum install python-pip python-devel #Install Pip pip install ansible==2.3.0 #Install Ansible 2.3 pip install markupsafe httplib2 requests #DependenciesNote:For RHEL, the following IPtables rule is needed to access port 9000.
iptables -t filter -A IN_public_allow -p tcp --dport 9000 -j ACCEPT
Install
libvirton the compute nodes using the following command:yum -y install libvirt
This installs the KVM (Kernel-based Virtual Machine) and associated packages for collecting data from virtual machines running on the compute nodes.
On the compute nodes where Contrail Insights Agent runs, verify that python
virtualenvis installed.#Ubuntu apt-get install -y python-pip pip install virtualenv #RHEL/CentOS yum install -y python-pip pip install virtualenvEnable passwordless login to facilitate Contrail Insights Platform node with Ansible to install agents on the nodes. Create a SSH public key on the node where Ansible playbooks are run, and then copy the key to the
appformix_controllernode.ssh-keygen -t rsa #Creates Keys ssh-copy-id -i ~/.ssh/id_rsa.pub <target_host> #Copies key from the node to other hosts
Use the
Sample_Inventoryfile as a template to create a host file.# Example naming schemes are as below: # hostname ansible_ssh_user='username' ansible_sudo_pass='password' # List all Compute Nodes [compute] 172.16.70.5 172.16.70.17 # appformix_controller host # # Host variables can be defined to control Contrail Insights configuration parameters # for particular host. For example, to specify the directory in which MongoDB # data is stored on hostname1 (the default is /opt/appformix/mongo/data): # # hostname1 appformix_mongo_data_dir=/var/lib/appformix/mongo # # For variables with same value for all appformix_controller hosts, set group # variables below. # [appformix_controller] 172.16.70.119 [openstack_controller] 172.16.70.120
Verify that all the hosts listed in the inventory file are reachable from the Contrail Insights Platform.
export ANSIBLE_HOST_KEY_CHECKING=False # Eliminates interactive experience prompting for Known_Hosts ansible -i inventory -m ping all # Pings all the hosts in the inventory file
At the top-level of the distribution, create a directory named
group_vars.mkdir group_vars
Download the Contrail Insights installation packages from software downloads to the Contrail Insights Platform node. Get the following files:
contrail-insights-<version>.tar.gz contrail-insights-dependencies-images-<version>.tar.gz contrail-insights-openstack-images-<version>.tar.gz contrail-insights-platform-images-<version>.tar.gz contrail-insights-network_device-images-<version>.tar.gz
If you are installing a version earlier than 3.2.6, copy the Contrail Insights license file to the Contrail Insights Platform node.
In
group_varsdirectory, create a file namedall. Add the following:openstack_platform_enabled: true appformix_manager_version: <version> appformix_docker_images: - /path/to/contrail-insights-platform-images-<version>.tar.gz - /path/to/contrail-insights-dependencies-images-<version>.tar.gz - /path/to/contrail-insights-openstack-images-<version>.tar.gzIf you are installing a version earlier than 3.2.6, include the path to the Contrail Insights license file in
group_vars/all:appformix_license: path/to/<contrail-insights-license-file>.sig
Contrail Insights must be configured to communicate with the OpenStack cluster. The Ansible playbooks use OpenStack environment variables to configure Contrail Insights with details of the OpenStack environment.
OS_AUTH_URL Keystone URL (e.g., https://host:5000/) OS_ENDPOINT_TYPE endpoint type to communicate with service (default: publicURL) OS_USERNAME admin account to be used by Contrail Insights OS_PASSWORD password for admin account OS_PROJECT_NAME admin project created for Contrail Insights account OS_PROJECT_DOMAIN_NAME domain for admin project OS_USER_DOMAIN_NAME domain for admin user OS_DOMAIN_NAME (optional) use domain-scoped token for admin account
-
Source the
openrcfile that contains the (step 10) environment variables and ensure the variables are in the environment of the shell from which the Ansible-playbooks are going to be executed. Then, install Contrail Insights by executing theappformix_openstack.ymlplaybook. Specify the path to theinventorydirectory that you created earlier. -
Open the Contrail Insights Dashboard in a Web browser. For example:
http://<contrail-insights-platform-node-ip>:9000
Select Skip Installation because the initial configuration was performed by Ansible using the OpenStack environment variables in step 10. Log in to Contrail Insights Dashboard using OpenStack Keystone credentials.
Additional Ansible Configuration
To set up additional Ansible configurations:
To install in a Keystone SSL-enabled cluster, include the following variables in the
group_vars/allfile:appformix_keystone_ssl_ca: '/path/to/keystone_ca.crt'
Contrail Insights Ansible will distribute this Keystone CA to all of the Contrail Insights Platform nodes and ask Contrail Insights components to talk to Keystone using this CA file with SSL enabled.
Note:Deprecation Notice: The
appformix_mongo_cache_size_gbparameter previously available starting in Contrail Insights 2.19.5 is now deprecated and no longer supported from Contrail Insights 3.2.0 and going forward. Starting with Contrail Insights version 3.2.0, Mongo will be configured to use a maximum of 40 percent of the available memory on the Contrail Insights Platform nodes.To enable network device monitoring in the cluster, include the following in the
group_vars/allfile:# For enabling pre-requisites for packdge installation. appformix_install_snmp_dependencies: true appformix_install_jti_dependencies: true # For running the appformix-network-device-adapter network_device_discovery_enabled: true appformix_plugins: '{{ appformix_network_device_factory_plugins }}' # After 3.1, SNMP Traps can be enabled also so appformix_plugins can be specified as below: # appformix_plugins: '{{ appformix_network_device_factory_plugins }} + {{ appformix_snmp_trap_factory_plugins }}'To install Contrail Insights certified plug-ins on the cluster, include the following variables in the
group_vars/allfile:appformix_plugins: <list of certified plugins to be installed> appformix_openstack_log_plugins: <list of OpenStack log plugins to be installed>
For example:
appformix_plugins: - { plugin_info: 'certified_plugins/cassandra_node_usage.json' } - { plugin_info: 'certified_plugins/contrail_vrouter.json' } - { plugin_info: 'certified_plugins/zookeeper_usage.json' } - { plugin_info: 'certified_plugins/heavy_hitters.json' } appformix_openstack_log_plugins: - { plugin_info: 'certified_plugins/cinder_api_logparser.json', log_file_path: '/var/log/cinder/cinder-api.log' } - { plugin_info: 'certified_plugins/glance_logparser.json', log_file_path: '/var/log/glance/glance-api.log' } - { plugin_info: 'certified_plugins/keystone_logparser.json', log_file_path: '/var/log/apache2/keystone_access.log,/var/log/httpd/keystone_wsgi_admin_access.log,/var/log/keystone/keystone.log' }For a list of all Contrail Insights certified plug-ins that can be installed, look for the entries starting with
plugin_infoin the fileroles/appformix_defaults/defaults/main.yml.The OpenStack log parser plug-ins parse the API log files of each OpenStack service to collect metrics about API calls and response status codes. To install these plug-ins, add them to the variable
appformix_openstack_log_pluginsingroup_vars/all, as shown above. Each plug-in entry in this list requires a parameter calledlog_file_pathto be specified. This parameter should be set to the complete path to the service's API log file on the OpenStack Controller node(s). Multiple comma-separated paths may be specified.To identify the right log file to be specified in
log_file_path, look for entries like the following, containing a client IP address, REST call type, and response status code:2019-04-02 06:50:13.103 3465 INFO nova.osapi_compute.wsgi.server [req-d07e953a-6921-4224-a056-afb6ff69adde 953ea56a96b944b3b170a299af9e87bd 10c9e8809feb4bd1b55955d9c2ed5aba - - -] 172.18.0.6 "GET /v2/10c9e8809feb4bd1b55955d9c2ed5aba/os-hypervisors/detail HTTP/1.1" status: 200 len: 1427 time: 0.0208740 2019-04-02 06:50:13.183 3465 INFO nova.osapi_compute.wsgi.server [req-34b2f686-9eb5-4112-b3fc-e0b37798a302 953ea56a96b944b3b170a299af9e87bd 10c9e8809feb4bd1b55955d9c2ed5aba - - -] 172.18.0.6 "GET /v2/10c9e8809feb4bd1b55955d9c2ed5aba/servers/detail?all_tenants=1&status=SHELVED_OFFLOADED HTTP/1.1" status: 200 len: 211 time: 0.0754580
Default locations for these files are listed in the variable
appformix_openstack_log_factory_pluginsinroles/appformix_defaults/defaults/main.yml.In Contrail Insights version 2.19.8, a timeout value can be configured for connecting to OpenStack services. The default value of this timeout is
10seconds and can be changed to a value between5and20seconds (both inclusive). To change the value, add the following variable togroup_vars/all:appformix_openstack_session_timeout: <number of seconds>
To modify the timeout value after the Contrail Insights Platform has been installed, add the variable to the
group_vars/allfile as described above and re-run the Contrail Insights installation playbook. Restart theappformix-openstack-adapterDocker container after the playbook has completed:docker restart appformix-openstack-adapter
Upgrade Notices
In Contrail Insights version 3.2.0, support for discovering
OpenStack Octavia Load Balancer services is added. Contrail Insights
only collects load balancer state information, such as provisioning_status and operating_status, as well as flavor information. To enable this service discovery,
provide Octavia service's endpoint as variable appformix_octavia_endpoint_url in the group_vars/all file. For example:
appformix_octavia_endpoint_url: http://10.1.1.1:9876
Chargeback costs can also be configured for the Octavia Load Balancer services. See Configure Load Balancer Costs.
Run Ansible with the created inventory file.
ansible-playbook -i inventory appformix_openstack.yml
In Contrail Insights version 3.0, the variable
appformix_openstack_factory_plugins is deprecated. All
OpenStack log parser plug-ins have to be specified in the variable
appformix_openstack_log_plugins. When upgrading from an
older version to version 3.0, make sure to move all OpenStack log parser
plug-ins defined in appformix_openstack_factory_plugins or
appformix_plugins to
appformix_openstack_log_plugins. Also, in Contrail Insights
version 3.0, all entries in this list have to be specified with a
log_file_path value, as described in step 3 above.
When upgrading from version 2.18.x to version 3.0, make the
following changes in the group_vars/all file:
In version 2.18.x:
appformix_openstack_factory_plugins:
- { plugin_info: 'certified_plugins/cinder_api_logparser.json', log_file_path: '/var/log/cinder/cinder-api.log'}
- { plugin_info: 'certified_plugins/glance_logparser.json',log_file_path: '/var/log/glance/api.log'}
- { plugin_info: 'certified_plugins/heavy_hitters.json' }
- { plugin_info: 'certified_plugins/keystone_logparser.json', log_file_path: '/var/log/keystone/keystone.log'}
- { plugin_info: 'certified_plugins/neutron_logparser.json', log_file_path: '/var/log/neutron/server.log'}
- { plugin_info: 'certified_plugins/nova_logparser.json', log_file_path: '/var/log/nova/nova-api.log'}
appformix_plugins: {{ appformix_openstack_factory_plugins }} + ...
In version 3.0.x:
appformix_plugins:
- { plugin_info: 'certified_plugins/heavy_hitters.json' }
appformix_openstack_log_plugins:
- { plugin_info: 'certified_plugins/cinder_api_logparser.json', log_file_path: '/var/log/cinder/cinder-api.log'}
- { plugin_info: 'certified_plugins/glance_logparser.json',log_file_path: '/var/log/glance/api.log'}
- { plugin_info: 'certified_plugins/keystone_logparser.json', log_file_path: '/var/log/keystone/keystone.log'}
- { plugin_info: 'certified_plugins/neutron_logparser.json', log_file_path: '/var/log/neutron/server.log'}
- { plugin_info: 'certified_plugins/nova_logparser.json', log_file_path: '/var/log/nova/nova-api.log'}
Source the openrc file from the OpenStack
controller node (/etc/contrail/openstackrc) to the Contrail Insights Platform node to authenticate the adapter
to access administrator privileges over the controller services. The
file should look like the following:
export OS_USERNAME=admin export OS_PASSWORD=<admin-password> export OS_AUTH_URL=http://172.16.80.2:5000/v2.0/ export OS_NO_CACHE=1 export OS_PROJECT_DOMAIN_NAME=Default export OS_USER_DOMAIN_NAME=Default export OS_PROJECT_NAME=admin export OS_IDENTITY_API_VERSION=3 export OS_IMAGE_API_VERSION=2
Run Ansible with the created inventory file.
ansible-playbook -i inventory appformix_openstack.yml
Remove a Node from Contrail Insights
To remove a node from Contrail Insights:
Edit the inventory file and add
appformix_state=absentto each node that you want to remove from Contrail Insights.# Example naming schemes are as below: # hostname ansible_ssh_user='username' ansible_sudo_pass='password' # List all Compute Nodes [compute] 172.16.70.5 appformix_state=absent 172.16.70.17
Run Ansible with the edited inventory file. This will remove the node and all its resources from Contrail Insights.
ansible-playbook -i inventory appformix_openstack.yml
Upgrade Contrail Insights for an OpenStack Cluster
Contrail Insights can be easily upgraded by running the appformix_openstack.yml playbook of the new release.
Follow the same procedure as the installation.
Uninstall Contrail Insights from an OpenStack Cluster
To uninstall Contrail Insights and destroy all data, execute the following command:
ansible-playbook -i <inventory_file> clean_appformix_openstack.yml