Example: Setting Up a VXLAN Layer 2 Gateway and OVSDB Connections in a VMware NSX Environment (Trunk Interfaces Supporting Untagged Packets)
In a physical network, a Juniper Networks device that supports Virtual Extensible LAN (VXLAN) can function as a hardware virtual tunnel endpoint (VTEP). In this role, the Juniper Networks device encapsulates Layer 2 Ethernet frames received from software applications that run directly on a physical server in VXLAN packets. The VXLAN packets are tunneled over a Layer 3 transport network. Upon receipt of the VXLAN packets, software VTEPs in the virtual network de-encapsulate the packets and forward the packets to virtual machines (VMs).
In this VXLAN environment, you can also include VMware NSX controllers and implement the Open vSwitch Database (OVSDB) management protocol on the Juniper Networks device that functions as a hardware VTEP. The Junos OS implementation of OVSDB provides a means through which VMware NSX controllers and Juniper Networks devices can exchange MAC addresses of entities in the physical and virtual networks. This exchange of MAC addresses enables the Juniper Networks device that functions as a hardware VTEP to forward traffic to software VTEPs in the virtual network and software VTEPs in the virtual network to forward traffic to the Junipet Networks device in the physical network.
This example explains how to configure a Juniper Networks device that supports VXLAN as a hardware VTEP. (The VTEP serves as a Layer 2 gateway.) This example also explains how to configure this device with an OVSDB connection to an NSX controller.
In this example, only one VXLAN is deployed. Given this scenario, the packets exchanged between an application running on a physical server and a VM in the VXLAN are untagged. As a result, the QFX Series switch dynamically configures a logical trunk interface for the connection between the physical server and the switch, as well as a native VLAN. The native VLAN enables the trunk interface to handle the untagged packets.
Requirements
This example includes the following hardware and software components:
A physical server on which software applications directly run.
A QFX10002 switch running Junos OS software 15.1X53-D30 or later.
On the QFX Series switch, physical interface ge-1/0/0 provides a connection to physical server 1.
A cluster of five NSX controllers. (In this example, you explicitly configure a connection with one NSX controller.)
NSX Manager.
A service node that handles the replication and forwarding of Layer 2 broadcast, unknown unicast, and multicast (BUM) traffic within the VXLAN used in this example.
A host that includes VMs managed by a hypervisor, which includes a software VTEP.
Before you begin:
Create an SSL private key and certificate, and install them in the /var/db/certs directory of the QFX Series switch. See Creating and Installing an SSL Key and Certificate on a Juniper Networks Device for Connection with SDN Controllers.
Using NSX Manager, specify the IP address of the service node.
For information about using NSX Manager, see the documentation that accompanies these VMware products.
Overview and Topology
Figure 1 shows a topology in which a software application running directly on physical server 1 in the physical network needs to communicate with virtual machine VM 1 in VXLAN 1 and vice versa.
To establish communication between the software application on physical server 1 and VM 1 in VXLAN 1, a connection with an NSX controller is explicitly configured on the management interface of the QFX Series switch by using the Junos OS CLI.
Also, some entities in the VXLAN-OVSDB topology must be configured in both NSX Manager and on the QFX Series switch. Table 1 provides a summary of the entities that must be configured and where they must be configured.
Entities |
What Must Be Configured in NSX Manager |
What Must Be Configured on a QFX Series Switch |
|---|---|---|
VXLAN 1 |
Logical switch for VXLAN 1 |
VXLAN 1 Note:
The QFX Series switch dynamically configures this VXLAN. |
Physical interface (ge-1/0/0) between physical server 1 and QFX Series switch |
A gateway service. For gateway service type, select VTEP L2 Gateway service. |
OVSDB management. Specify that interface ge-1/0/0 is managed by OVSDB. |
One logical interface (ge-1/0/0.0) associated with VXLAN 1 |
One logical switch port for VXLAN 1. For this port, specify VLAN number 0. Note:
A VLAN number of 0 indicates that the port must handle untagged packets. |
One logical interface (ge-1/0/0.0) for VXLAN 1. Note:
The QFX Series switch dynamically configures this logical interface. |
QFX Series switch (hardware VTEP 1) |
Gateway |
– |
In NSX Manager, a logical switch for VXLAN 1 is configured. In this configuration, a VXLAN network identifier (VNI) of 100 is specified. Also, the universally unique identifier (UUID) that NSX Manager assigns to the logical switch is 28805c1d-0122-495d-85df-19abd647d772. Based on this configuration, the QFX Series switch dynamically creates the following configuration for a Junos OS-equivalent VXLAN:
set vlans 28805c1d-0122-495d-85df-19abd647d772 vxlan vni 100
Based on the gateway service and logical switch port configuration (VLAN number 0) in NSX Manager, the QFX Series switch dynamically creates the following configuration for a Junos OS-equivalent interface:
set interfaces ge-1/0/0 flexible-vlan-tagging set interfaces ge-1/0/0 native-vlan-id 4094 set interfaces ge-1/0/0 encapsulation extended-vlan-bridge set interfaces ge-1/0/0 unit 0 vlan-id 4094 set vlans 28805c1d-0122-495d-85df-19abd647d772 interface ge-1/0/0.0
This configuration sets physical interface ge-1/0/0 as a trunk interface. It also configures a native VLAN with an ID of 4094. The configuration creates logical interface ge-1/0/0.0 and specifies that it is a member of the native VLAN. As a result, logical interface ge-1/0/0.0 handles incoming untagged packets.
The configuration also associates logical interface ge-1/0/0.0 with VXLAN 28805c1d-0122-495d-85df-19abd647d772.
Table 2 provides a summary of the VXLAN-OVSDB topology components that are configured on the QFX Series switch and the configuration settings for each component.
Topology
Component |
Setting |
|---|---|
NSX controller |
IP address: 10.94.184.1 |
OVSDB-managed physcal interface |
Interface name: ge-1/0/0 Native VLAN ID: 4094 |
Logical interface |
Note:
The QFX Series switch dynamically creates this logical interface configuration, which is based on the gateway service configuration and logical switch port configuration in NSX Manager. Therefore, no manual configuration is required. Interface name: ge-1/0/0.0 Interface type: trunk Member of native VLAN 4094 Associated with VXLAN 28805c1d-0122-495d-85df-19abd647d772 |
OVSDB-managed VXLAN |
Note:
The QFX Series switch dynamically creates this VXLAN configuration, which is based on the logical switch configuration in NSX Manager. Therefore, no manual configuration is required. For VXLAN 1: VXLAN name: 28805c1d-0122-495d-85df-19abd647d772 VNI: 100 |
OVSDB tracing operations |
Filename: /var/log/ovsdb File size: 10 MB Flag: All |
Hardware VTEP source identifier |
Source interface: loopback (lo0.0) Source IP address: 10.17.17.17/32 |
Handling of Layer 2 BUM traffic in VXLAN 28805c1d-0122-495d-85df-19abd647d772 |
Service node Note:
By default, one or more service nodes handle Layer 2 BUM traffic within a VXLAN; therefore, no manual configuration is required. |
Non-OVSDB and Non-VXLAN Configuration
CLI Quick Configuration
To quickly configure this example, copy the
following commands, paste them into a text file, remove any line breaks,
change any details necessary to match your configuration, copy and
paste the commands into the CLI at the [edit] hierarchy
level, and then enter commit from configuration mode.
set interfaces ge-1/0/9 unit 0 family inet address 10.40.40.1/24 set routing-options static route 10.19.19.19/32 next-hop 10.40.40.2 set routing-options router-id 10.17.17.17 set protocols ospf area 0.0.0.0 interface lo0.0 set protocols ospf area 0.0.0.0 interface ge-1/0/9.0
Procedure
Step-by-Step Procedure
To configure the Layer 3 network over which the packets exchanged between the physical server and VMs are tunneled:
Configure the Layer 3 interface.
[edit interfaces] user@switch# set ge-1/0/9 unit 0 family inet address 10.40.40.1/24
Set the routing options.
[edit routing-options] user@switch# set static route 10.19.19.19/32 next-hop 10.40.40.2 user@switch# set router-id 10.17.17.17
Configure the routing protocol.
[edit protocols] user@switch# set ospf area 0.0.0.0 interface lo0.0 user@switch# set ospf area 0.0.0.0 interface ge-1/0/9.0
OVSDB and VXLAN Configuration
CLI Quick Configuration
To quickly configure this example, copy the
following commands, paste them into a text file, remove any line breaks,
change any details necessary to match your configuration, copy and
paste the commands into the CLI at the [edit] hierarchy
level, and then enter commit from configuration mode.
set switch-options ovsdb-managed set protocols ovsdb controller 10.94.184.1 set protocols ovsdb interfaces ge-1/0/0 set protocols ovsdb traceoptions file ovsdb set protocols ovsdb traceoptions file size 10m set protocols ovsdb traceoptions flag all set interfaces lo0 unit 0 family inet address 10.17.17.17/32 primary set interfaces lo0 unit 0 family inet address 10.17.17.17/32 preferred set switch-options vtep-source-interface lo0.0
Procedure
Step-by-Step Procedure
To configure the QFX Series switch as a hardware VTEP with an OVSDB connection to an NSX controller:
Enable the QFX Series switch to dynamically configure OVSDB-managed VXLANs and associated interfaces.
[edit switch-options] user@switch# ovsdb-managed
Explicitly configure a connection with an NSX controller.
[edit protocols] user@switch# set ovsdb controller 10.94.184.1
Specify that the interface between hardware VTEP 1 and physical server 1 is managed by OVSDB.
[edit protocols] user@switch# set ovsdb interfaces ge-1/0/0
Set up OVSDB tracing operations.
[edit protocols] user@switch# set ovsdb traceoptions file ovsdb user@switch# set ovsdb traceoptions file size 10m user@switch# set ovsdb traceoptions flag all
Specify an IP address for the loopback interface. This IP address serves as the source IP address in the outer header of any VXLAN-encapsulated packet.
[edit interfaces] user@switch# set lo0 unit 0 family inet address 10.17.17.17/32 primary user@switch# set lo0 unit 0 family inet address 10.17.17.17/32 preferred
-
Set the loopback interface as the interface that identifies hardware VTEP 1.
[edit switch-options] user@switch# set vtep-source-interface lo0.0
In NSX Manager, configure a logical switch for VXLAN 1. See the VMware documentation that accompanies NSX Manager.
In NSX Manager, configure a gateway for the QFX Series switch, and configure a gateway service and logical switch port for the logical interface (ge-1/0/0.0). See VMware NSX Configuration for Juniper Networks Devices Functioning as Virtual Tunnel Endpoints.
Verification
Confirm that the configuration is working properly:
- Verifying the Logical Switch Configuration
- Verifying the MAC Address of VM 1
- Verifying the NSX Controller Connection
- Verifying the OVSDB-Managed Interface
Verifying the Logical Switch Configuration
Purpose
Verify that the configuration of the logical switch
with the UUID of 28805c1d-0122-495d-85df-19abd647d772 is present in
the OVSDB schema for physical devices and that the Flags field of
the show ovsdb logical switch output displays Created
by both.
Action
From operational mode, enter the show ovsdb logical-switch command.
user@switch> show ovsdb logical-switch Logical switch information: Logical Switch Name: 28805c1d-0122-495d-85df-19abd647d772 Flags: Created by both VNI: 100 Num of Remote MAC: 1 Num of Local MAC: 0
Meaning
The output verifies that the configuration for the
logical switch is present. The Created by both state indicates
that the logical switch was configured in NSX Manager, and that the
QFX Series switch dynamically created the corresponding VXLAN. In
this state, the logical switch and the VXLAN are operational.
If the state of the logical switch is something other than Created by both, see Troubleshooting a Nonoperational Logical Switch and Corresponding Junos OS OVSDB-Managed VXLAN.
Verifying the MAC Address of VM 1
Purpose
Verify that the MAC address of VM 1 is present in the OVSDB schema.
Action
From operational mode, enter the show ovsdb mac
remote command.
user@switch> show ovsdb mac remote Logical Switch Name: 28805c1d-0122-495d-85df-19abd647d772 Mac IP Encapsulation Vtep Address Address Address a8:59:5e:f6:38:90 0.0.0.0 Vxlan over Ipv4 10.17.17.17
Meaning
The output shows that the MAC address for VM 1 is present and is associated with the logical switch with the UUID of 28805c1d-0122-495d-85df-19abd647d772. Given that the MAC address is present, VM 1 is reachable through the QFX Series switch, which functions as a hardware VTEP.
Verifying the NSX Controller Connection
Purpose
Verify that the connection with the NSX controller is up.
Action
From operational mode, enter the show ovsdb controller command to verify that the controller connection state is up.
user@switch> show ovsdb controller VTEP controller information: Controller IP address: 10.94.184.1 Controller protocol: ssl Controller port: 6632 Controller connection: up Controller seconds-since-connect: 542325 Controller seconds-since-disconnect: 542346 Controller connection status: active
Meaning
The output shows that the connection state of the NSX
controller is up, in addition to other information about the controller.
The up state of the NSX controller indicates that OVSDB
is enabled on the QFX Series switch.
Verifying the OVSDB-Managed Interface
Purpose
Verify that interface ge-1/0/0.0 is managed by OVSDB.
Action
From operational mode, enter the show ovsdb interface command to verify that interface ge-1/0/0.0 is managed by OVSDB.
user@switch> show ovsdb interface Interface VLAN ID Bridge-domain ge-1/0/0 0 28805c1d-0122-495d-85df-19abd647d772
Meaning
The output shows that interface ge-1/0/0 is managed by OVSDB. It also indicates that the interface is associated with VXLAN 28805c1d-0122-495d-85df-19abd647d772, which has a VLAN ID of 0.