Installing Contrail with OpenStack and Kolla Ansible
We recommend using Contrail Command to add compute nodes to existing Contrail clusters in most Contrail Networking deployments. See Adding a New Compute Node to Existing Contrail Cluster Using Contrail Command.
The procedure in this document should only be performed by network administrators with significant YAML file expertise in environments not using the Contrail Command GUI.
The goal of this topic is to install Contrail Networking with OpenStack, using Kolla Ansible
playbook contrail-kolla-ansible.
Kolla is an OpenStack project which provides tools to build container images for OpenStack services. Kolla Ansible provides Ansible playbooks to deploy the Kolla images.
The contrail-kolla-ansible playbook
works in conjunction with contrail-ansible-deployer to install OpenStack and Contrail Networking containers.
Refer to Installing Contrail Cluster using Contrail Command and instances.yml to deploy a Contrail cluster using Contrail Command.
Follow the procedure
to deploy Kolla containers using contrail-kolla-ansible and Contrail Networking containers using contrail-ansible-deployer:
Set Up the Base Host
Update CentOS and kernel version. For a list of supported platforms, see https://www.juniper.net/documentation/en_US/release-independent/contrail/topics/reference/contrail-supported-platforms.pdf.
The vRouter has a dependency with the host kernel.
To set up the base host:
Download Ansible Deployer installer package from the Contrail Downloads page.
Install Ansible.
yum -y install epel-releaseyum -y install git ansible-2.7.10Install python-pip.
yum install -y python-pipRun the following commands.
yum -y remove PyYAML python-requestspip install PyYAML requestsUntar the tgz file.
- tar xvf contrail-ansible-deployer-19<xx>.<NN>.tgzThe instances.yaml is located at thecontrail-ansible-deployer/config/
Configure Contrail and Kolla parameters in the file instances.yaml, using the following guidelines:
The provider configuration (
provider_config) section refers to the cloud provider where the Contrail cluster will be hosted, and contains all parameters relevant to the provider. For bare metal servers, the provider isbms.The
kolla_globalssection refers to OpenStack services. For more information about all possiblekolla_globals, see https://github.com/Juniper/contrail-kolla-ansible/.../globals.yml.Additional Kolla configurations (
contrail-kolla-ansible) are possible ascontrail_additions. For more information about all possiblecontrail_additionsto Kolla, see https://github.com/Juniper/contrail-kolla-ansible/.../all.yml.The
contrail_configurationsection contains parameters for Contrail services.CONTAINER_REGISTRYspecifies the registry from which to pull Contrail containers. It can be set to your local Docker registry if you are building your own containers. If a registry is not specified, it will try to pull the containers from the Docker hub.If a custom registry is specified, also specify the same registry under
kolla_globalsascontrail_docker_registry.CONTRAIL_VERSION, if not specified, will default to the "latest" tag.For more information about all possible parameters for
contrail_configuration, see https://github.com/tungstenfabric/tf-container-builder/blob/master/containers/base/common.sh.You must specify the roles in the instances.yaml file. Otherwise, the installation procedure will fail.
If there are host-specific values per host, for example, if the names of the interfaces used for "network_interface" are different on the servers in your cluster, use the example configuration at Configuration Sample for Multi Node OpenStack HA and Contrail (multi interface).
Many of the parameters are automatically derived to sane defaults (how the first configuration works). You can explicitly specify variables to override the derived values if required. Review the code to see the derivation logic.
Example: instances.yaml
This example is a bare minimum configuration for a single node, single interface, all-in-one cluster.
provider_config: bms: ssh_pwd: <password> ssh_user: root ntpserver: <IP NTP server> domainsuffix: local instances: bms1: provider: bms ip: <IP BMS> roles: config_database: config: control: analytics_database: analytics: analytics_alarm: analytics_snmp: webui: vrouter: openstack: openstack_compute: contrail_configuration: RABBITMQ_NODE_PORT: 5673 AUTH_MODE: keystone KEYSTONE_AUTH_URL_VERSION: /v3 kolla_config: kolla_globals: enable_haproxy: no kolla_passwords: keystone_admin_password: <Keystone admin password>Example: instances.yaml
This example is a more elaborate configuration for a single node, single interface, all-in-one cluster.
provider_config: bms: ssh_pwd: <password> ssh_user: root ntpserver: <IP NTP server> domainsuffix: local instances: bms1: provider: bms ip: <IP BMS> roles: config_database: config: control: analytics_database: analytics: analytics_alarm: analytics_snmp: webui: vrouter: openstack: openstack_compute: global_configuration: CONTAINER_REGISTRY: <Registry FQDN/IP>:<Registry Port> REGISTRY_PRIVATE_INSECURE: True contrail_configuration: CONTRAIL_VERSION: latest CLOUD_ORCHESTRATOR: openstack VROUTER_GATEWAY: <IP gateway> RABBITMQ_NODE_PORT: 5673 PHYSICAL_INTERFACE: <interface name> AUTH_MODE: keystone KEYSTONE_AUTH_URL_VERSION: /v3 kolla_config: kolla_globals: kolla_internal_vip_address: <Internal VIP> contrail_api_interface_address: <Contrail API Addr> enable_haproxy: no kolla_passwords: keystone_admin_password: <Keystone Admin Password>Run the following commands from the contrail-ansible-deployer folder:
ansible-playbook -e orchestrator=openstack -i inventory/ playbooks/configure_instances.ymlansible-playbook -i inventory/ playbooks/install_openstack.ymlansible-playbook -e orchestrator=openstack -i inventory/ playbooks/install_contrail.yml
Open web browser and type https://contrail-server-ip:8143 to access Contrail Web UI.
The default login user name is admin. Use the same password which was entered in step 6
Multiple Interface Configuration Sample for Multinode OpenStack HA and Contrail
This is a configuration sample for a multiple interface, multiple node deployment of high availability OpenStack and Contrail Networking. Use this sample to configure parameters specific to your system.
For more information or for recent updates, refer to the github topic Configuration Sample for Multi Node OpenStack HA and Contrail (multi interface).
Configuration Sample—Multiple Interface
provider_config:
bms:
ssh_pwd: <Pwd>
ssh_user: root
ntpserver: <NTP Server>
domainsuffix: local
instances:
bms1:
provider: bms
ip: <BMS1 IP>
roles:
openstack:
bms2:
provider: bms
ip: <BMS2 IP>
roles:
openstack:
bms3:
provider: bms
ip: <BMS3 IP>
roles:
openstack:
bms4:
provider: bms
ip: <BMS4 IP>
roles:
config_database:
config:
control:
analytics_database:
analytics:
analytics_alarm:
analytics_snmp:
webui:
bms5:
provider: bms
ip: <BMS5 IP>
roles:
config_database:
config:
control:
analytics_database:
analytics:
analytics_alarm:
analytics_snmp:
webui:
bms6:
provider: bms
ip: <BMS6 IP>
roles:
config_database:
config:
control:
analytics_database:
analytics:
analytics_alarm:
analytics_snmp:
webui:
bms7:
provider: bms
ip: <BMS7 IP>
roles:
vrouter:
PHYSICAL_INTERFACE: <Interface name>
VROUTER_GATEWAY: <Gateway IP>
openstack_compute:
bms8:
provider: bms
ip: <BMS8 IP>
roles:
vrouter:
# Add following line for TSN Compute Node
TSN_EVPN_MODE: True
openstack_compute:
contrail_configuration:
CLOUD_ORCHESTRATOR: openstack
KEYSTONE_AUTH_URL_VERSION: /v3
IPFABRIC_SERVICE_HOST: <Service Host IP>
# Add following line for TSN Compute Node
TSN_NODES: <TSN NODE IP List>
# For EVPN VXLAN TSN
ENCAP_PRIORITY: "VXLAN,MPLSoUDP,MPLSoGRE"
PHYSICAL_INTERFACE: <Interface name>
kolla_config:
kolla_globals:
kolla_internal_vip_address: <Internal VIP>
kolla_external_vip_address: <External VIP>
contrail_api_interface_address: <Contrail API IP>
kolla_passwords:
keystone_admin_password: <Keystone Admin Password>
Single Interface Configuration Sample for Multinode OpenStack HA and Contrail
This is a configuration sample for a multiple node, single interface deployment of high availability OpenStack and Contrail Networking. Use this sample to configure parameters specific to your system.
For more information or for recent updates, refer to the github topic Configuration Sample for Multi Node OpenStack HA and Contrail (single interface).
Configuration Sample—Single Interface
provider_config:
bms:
ssh_pwd: <password>
ssh_user: root
ntpserver: xx.xx.x.xx
domainsuffix: local
instances:
centos1:
provider: bms
ip: ip-address
roles:
openstack:
centos2:
provider: bms
ip: ip-address
roles:
openstack:
centos3:
provider: bms
ip: ip-address
roles:
openstack:
centos4:
provider: bms
ip: ip-address
roles:
config_database:
config:
control:
analytics_database:
analytics:
analytics_alarm:
analytics_snmp:
webui:
centos5:
provider: bms
ip: ip-address
roles:
config_database:
config:
control:
analytics_database:
analytics:
analytics_alarm:
analytics_snmp:
webui:
centos6:
provider: bms
ip: ip-address
roles:
config_database:
config:
control:
analytics_database:
analytics:
analytics_alarm:
analytics_snmp:
webui:
centos7:
provider: bms
ip: ip-address
roles:
vrouter:
openstack_compute:
centos8:
provider: bms
ip: ip-address
roles:
vrouter:
openstack_compute:
contrail_configuration:
CONTRAIL_VERSION: <contrail_version>
CONTROLLER_NODES: ip-addresses separated by comma
CLOUD_ORCHESTRATOR: openstack
RABBITMQ_NODE_PORT: 5673
VROUTER_GATEWAY: gateway-ip-address
PHYSICAL_INTERFACE: eth1
IPFABRIC_SERVICE_IP: ip-address
KEYSTONE_AUTH_HOST: ip-address
KEYSTONE_AUTH_URL_VERSION: /v3
kolla_config:
kolla_globals:
kolla_internal_vip_address: ip-address
contrail_api_interface_address: ip-address
network_interface: "eth1"
enable_haproxy: "yes"
kolla_passwords:
keystone_admin_password: <password>Replace <contrail_version> with the
correct contrail_container_tag value for your Contrail
release. The respective contrail_container_tag values are
listed in README Access to Contrail Registry 19XX.
Frequently Asked Questions
This section presents some common error situations and gives guidance on how to resolve the error condition.
- Using Host-Specific Parameters
- Containers from Private Registry Not Accessible
- Error: Failed to insert vrouter kernel module
- Fatal Error When Vrouter Doesn’t Specify OpenStack
- Need for HAProxy and Virtual IP on a Single OpenStack Cluster
- Using the kolla_toolbox Container to Run OpenStack Commands
Using Host-Specific Parameters
You might have a situation where you need to specify host-specific parameters, for example, the interface names are different for the different servers in the cluster. In this case, you could specify the individual names under each role, and the more specific setting takes precedence.
For example, if there is no "network_interface" setting under the role "openstack" for example “bms1”, then it will take its setting from the global variable.
An extended example is available at: Configuration Sample for Multi Node OpenStack HA and Contrail.
Containers from Private Registry Not Accessible
You might have a situation in which containers that are pulled from a private registry named CONTAINER_REGISTRY are not accessible.
To resolve, check to ensure that REGISTRY_PRIVATE_INSECURE is set to True.
Error: Failed to insert vrouter kernel module
You might have a situation in which the vrouter module is not getting installed on the compute nodes, with the vrouter container in an error state and errors are shown in the Docker logs.
[srvr5] ~ # docker logs vrouter_vrouter-kernel-init_1 /bin/cp: cannot create regular file '/host/bin/vif': No such file or directory INFO: Load kernel module for kver=3.10.0 INFO: Modprobing vrouter /opt/contrail/vrouter-kernel-modules/3.10.0-957.11.6.el7.x86_64/vrouter.ko total used free shared buff/cache available Mem: 62G 999M 55G 9.1M 5.9G 60G Swap: 0B 0B 0B total used free shared buff/cache available Mem: 62G 741M 61G 9.1M 923M 61G Swap: 0B 0B 0B insmod: ERROR: could not insert module /opt/contrail/vrouter-kernel-modules/3.10.0-957.11.6.el7.x86_64/vrouter.ko: Unknown symbol in module ERROR: Failed to insert vrouter kernel moduleIn this release, the vrouter module requires the host kernel version to be 3.10.0-957.11.6.el7.x86_64. To get this kernel version, before running provision, install the kernel version on the target nodes.
yum -y install kernel-3.10.0-957.11.6.el7.x86_64 yum update reboot
Fatal Error When Vrouter Doesn’t Specify OpenStack
You might encounter a fatal error when vrouter needs to be provisioned without nova-compute.
2018-03-21 00:47:16,884 p=16999 u=root | TASK [iscsi : Ensuring config directories exist] ******************** 2018-03-21 00:47:16,959 p=16999 u=root | fatal: [ip-address]: FAILED! => {"msg": "The conditional check 'inventory_hostname in groups['compute'] or inventory_hostname in groups['storage']' failed. The error was: error while evaluating conditional (inventory_hostname in groups['compute'] or inventory_hostname in groups['storage']): Unable to look up a name or access an attribute in template string ({% if inventory_hostname in groups['compute'] or inventory_hostname in groups['storage'] %} True {% else %} False {% endif %}).\nMake sure your variable name does not contain invalid characters like '-': argument of type 'StrictUndefined' is not iterable\n\nThe error appears to have been in '/root/contrail-kolla- ansible/ansible/roles/iscsi/tasks/config.yml': line 2, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n---\n- name: Ensuring config directories exist\n ^ here\n"} 2018-03-21 00:47:16,961 p=16999 u=root | to retry, use: --limit @/root/contrail-ansible- deployer/playbooks/install_contrail.retryThere is a use case in which vrouter needs to be provisioned without being accompanied by nova-compute. Consequently, the "openstack_compute" is not automatically inferred when "vrouter" role is specified. To resolve this issue, the "openstack_compute" role needs to be explicitly stated along with "vrouter".
For more information about this use case, refer to the bug #1756133.
Need for HAProxy and Virtual IP on a Single OpenStack Cluster
By default, all OpenStack services listen on the IP interface
provided by the kolla_internal_vip_address/network_interface variables under the kolla_globals section
in config/instances.yaml. In most
cases this corresponds to the ctrl-data network, which means that
even Horizon will now run only on the ctrl-data network. The only
way Kolla provides access to Horizon on the management network is
by using HAProxy and keepalived. Enabling keepalived requires a virtual
IP for VRRP, and it cannot be the interface IP. There is no way to
enable HAProxy without enabling keepalived when using Kolla configuration
parameters. For this reason,you need to provide two virtual IP addresses:
one on management (kolla_external_vip_address) and one on ctrl-data-network (kolla_internal_vip_address). With this configuration, Horizon will be accessible on the management
network by means of the kolla_external_vip_address.
Using the kolla_toolbox Container to Run OpenStack Commands
The directory /etc/kolla/kolla-toolbox on the base host on which OpenStack containers are running is mounted
and accessible as /var/lib/kolla/config_files from inside the kolla_toolbox container.
If you need other files when executing OpenStack commands, for example
the command openstack image create needs
an image file, you can copy the relevant files into the /etc/kolla/kolla-toolbox directory of the base host
and use them inside the container.
The following example shows how to run OpenStack commands in this way:
# ON BASE HOST OF OPENSTACK CONTROL NODE
cd /etc/kolla/kolla-toolbox
wget http://download.cirros-cloud.net/0.4.0/cirros-0.4.0-x86_64-disk.img
docker exec -it kolla_toolbox bash
# NOW YOU ARE INSIDE THE KOLLA_TOOLBOX CONTAINER
(kolla-toolbox)[ansible@server1 /]$ source /var/lib/kolla/config_files/admin-openrc.sh
(kolla-toolbox)[ansible@server1 /]$ cd /var/lib/kolla/config_files
(kolla-toolbox)[ansible@server1 /var/lib/kolla/config_files]$ openstack image create cirros2 --disk-format qcow2 --public --container-format bare --file cirros-0.4.0-x86_64-disk.img
+------------------+------------------------------------------------------+
| Field | Value |
+------------------+------------------------------------------------------+
| checksum | 443b7623e27ecf03dc9e01ee93f67afe |
| container_format | bare |
| created_at | 2018-03-29T21:37:48Z |
| disk_format | qcow2 |
| file | /v2/images/e672b536-0796-47b3-83a6-df48a5d074be/file |
| id | e672b536-0796-47b3-83a6-df48a5d074be |
| min_disk | 0 |
| min_ram | 0 |
| name | cirros2 |
| owner | 371bdb766278484bbabf868cf7325d4c |
| protected | False |
| schema | /v2/schemas/image |
| size | 12716032 |
| status | active |
| tags | |
| updated_at | 2018-03-29T21:37:50Z |
| virtual_size | None |
| visibility | public |
+------------------+------------------------------------------------------+
(kolla-toolbox)[ansible@server1 /var/lib/kolla/config_files]$ openstack image list
+--------------------------------------+---------+--------+
| ID | Name | Status |
+--------------------------------------+---------+--------+
| e672b536-0796-47b3-83a6-df48a5d074be | cirros2 | active |
| 57e6620e-796a-40ee-ae6e-ea1daa253b6c | cirros2 | active |
+--------------------------------------+---------+--------+