Understanding Contrail with Mesos Architecture

Contrail Controller

Contrail controller performs all the decision making. It includes config management, analytics, UI and control plane components for network virtualization. For further details, refer to https://github.com/Juniper/contrail-controller.

Contrail controller exposes APIs for creating configuration and updating virtual network components. In Mesos, mesos manager updates the task (universal docker) information to the Contrail controller via API server. All Contrail controller components are microservice docker containers.

Mesos Manager

Mesos manager consists of two sub modules:

• VNC server.

• Interaction with CNI and mesos agent.

Mesos manager application runs inside a docker on every slave node.

Mesos manager creates two networking by default: mesos-default-pod-task network and ip-fabric network.

All the pods and tasks are created in the mesos-default-pod-task network.

ip-fabric network is created in the respective domains of mesos-default and project-default.

CNI receives the task information and posts it to the Mesos manager. On receiving the task information, the Mesos manager creates the contrail-vnc objects.

Configuration information for the Mesos manager is present in contrail-mesos.conf file. The file is located at /etc/contrail/contrail-mesos.conf in the mesos manager docker.

Sample of contrail-mesos.conf file:

[MESOS]
listen_ip_addr=127.0.0.1
listen_port=6991
pod_task_subnets=10.x.x.0/12
ip_fabric_subnets=10.x.x.0/12

[VNC]
vnc_endpoint_ip=127.0.0.1
vnc_endpoint_port=8082
admin_user=admin
admin_password=admin
admin_tenant=admin
rabbit_server=127.0.0.1
rabbit_port=5673
cassandra_server_list=127.0.0.1:9161

[DEFAULTS]
disc_server_ip=127.0.0.1
disc_server_port=5998
log_local=1
log_level=SYS_NOTICE
log_file=/var/log/contrail/contrail-mesos-manager.log

[SANDESH]
#sandesh_ssl_enable=False
#introspect_ssl_enable=False
#sandesh_keyfile=/etc/contrail/ssl/private/server-privkey.pem
#sandesh_certfile=/etc/contrail/ssl/certs/server.pem
#sandesh_ca_cert=/etc/contrail/ssl/certs/ca-cert.pem

You can add the network to pod or task through annotation. You can set the network using labels.

Sample task/pod input json file:

networks": [
    {
      "name": "contrail-cni-plugin",
      "mode": "container",
      "labels": {
        "networks": "default-domain:default:blue-network",
        "pod-subnets": "default-domain:default:blue-network"
      }
    }

Introspect for mesos-manager objects on the port 8109.

Contrail Container Network Interface (CNI)

The Container Network Interface (CNI) is located at /opt/mesosphere/active/cni/contrail-cni-plugin. It is a run to completion executable file.

The config file is located at /opt/mesosphere/etc/dcos/network/cni/contrail-cni-plugin.conf.

Sample contrail-cni-plugin.conf file:

{
    "cniVersion": "0.2.0",
    "contrail" : {
        "vrouter-ip"    : "slave-ip",
        "vrouter-port"  : 9091,
        "cluster-name"  : "slave-hostname",
        "config-dir"    : "/var/lib/contrail/ports/vm",
        "poll-timeout"  : 15,
        "poll-retries"  : 5,
        "log-file"      : "/var/log/contrail/cni/opencontrail.log",
        "log-level"     : "debug",
        "mesos-ip"      : "localhost",
        "mesos-port"    : "6991",
        "mode"          : "mesos"
    },

    "name": "contrail-cni-plugin",
    "type": "contrail-cni-plugin"
}

Mesos agent invokes Contrail CNI when custom/host network provider is mentioned as contrail-cni-plugin in the task description.