Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

 
 

Example: Setting Up a VXLAN Layer 2 Gateway and OVSDB Connections in a VMware NSX Environment (Trunk Interfaces Supporting Tagged Packets)

In a physical network, a Juniper Networks device that supports Virtual Extensible LAN (VXLAN) can function as a hardware virtual tunnel endpoint (VTEP). In this role, the Juniper Networks device encapsulates Layer 2 Ethernet frames received from software applications that run directly on a physical server in VXLAN packets. The VXLAN packets are tunneled over a Layer 3 transport network. Upon receipt of the VXLAN packets, software VTEPs in the virtual network de-encapsulate the packets and forward the packets to virtual machines (VMs).

In this VXLAN environment, you can also include VMware NSX controllers and implement the Open vSwitch Database (OVSDB) management protocol on the Juniper Networks device that functions as a hardware VTEP. The Junos OS implementation of OVSDB provides a means through which VMware NSX controllers and Juniper Networks devices can exchange MAC addresses of entities in the physical and virtual networks. This exchange of MAC addresses enables the Juniper Networks device that functions as a hardware VTEP to forward traffic to software VTEPs in the virtual network and software VTEPs in the virtual network to forward traffic to the Junipet Networks device in the physical network.

This example explains how to configure a Juniper Networks device that supports VXLAN as a hardware VTEP. (The VTEP serves as a Layer 2 gateway.) This example also explains how to configure this device with an OVSDB connection to an NSX controller.

Starting with Junos OS Release 14.1X53-D15 for QFX5100 switches, 15.1X53-D10 for QFX10002 switches, 15.1X53-D30 for QFX10008 switches, 15.1X53-D60 for QFX10016 switches, 15.1X53-D210 for QFX5110 and QFX5200 switches, and 18.1R1 for QFX5210 switches, the dynamic configuration of trunk interfaces and their association with an OVSDB-managed VXLAN is supported. In this example, an application running directly on a physical server needs to communicate with a VM in a VXLAN, while another application on the physical server needs to communicate with VMs in another VXLAN. Therefore, the packets exchanged between the applications running on the physical server and the respective VMs with which they must communicate are tagged. As a result, a trunk interface is used for the connection between the physical server and the Juniper Networks device.

Requirements

This example includes the following hardware and software components:

  • A physical server on which software applications directly run.

  • A Juniper Networks switch that supports VXLAN and OVSDB. This switch can be a QFX5100 switch running Junos OS Release 14.1X53-D15 or later.

  • On the Juniper Networks switch, physical interface ge-1/0/0 provides a connection to physical server 1.

  • A cluster of five NSX controllers. (In this example, you explicitly configure a connection with one NSX controller.)

  • NSX Manager.

  • A service node that handles the replication and forwarding of Layer 2 broadcast, unknown unicast, and multicast (BUM) traffic within the VXLANs.

  • Two hosts that include VMs. Each host is managed by a hypervisor, and each hypervisor includes a software VTEP.

Before you begin the configuration, you must perform the following tasks:

For information about using NSX Manager, see the documentation that accompanies NSX Manager.

Overview and Topology

Figure 1 shows a topology in which a software application running directly on physical server 1 in the physical network needs to communicate with virtual machine VM 1 in VXLAN 1 and vice versa, and another software application on physical server 1 needs to communicate with virtual machines VM 3 and VM 4 in VXLAN 2 and vice versa.

Figure 1: VXLAN/OVSDB Layer 2 Gateway TopologyVXLAN/OVSDB Layer 2 Gateway Topology

To establish communication between the software applications on physical server 1 and the VMs in VXLANs 1 and 2, some entities in the VXLAN-OVSDB topology must be configured in both NSX Manager and on the Juniper Networks switch. Table 1 provides a summary of the entities that must be configured and where they must be configured.

Note:

The term used for an entity configured in NSX Manager can differ from the term used for essentially the same entity configured on the Junos Network switch. To prevent confusion, Table 1 shows the NSX Manager and Junos OS entities side-by-side.

Table 1: NSX Manager and Junos OS Entities That Must Be Configured

Entities

What Must Be Configured In NSX Manager

What Must Be Configured on Juniper Networks Switch

VXLAN 1

VXLAN 2

Logical switch for VXLAN 1

Logical switch for VXLAN 2

VXLAN 1

VXLAN 2

Note:

The Juniper Networks switch dynamically configures these VXLANs.

Interface (ge-1/0/0) between physical server 1 and Juniper Networks switch

A gateway service. For gateway service type, select VTEP L2 gateway service.

OVSDB management. Specify that interface ge-1/0/0 is managed by OVSDB.

One logical interface associated with VXLAN 1

One logical interface associated with VXLAN 2

One logical switch port for VXLAN 1. For this port, specify VLAN number 10.

One logical switch port for VXLAN 2. For this port, specify VLAN number 20.

Note:

A VLAN number from 1 through 4000 indicates that the port is a trunk port.

One logical interface (ge-1/0/0.10) for VXLAN 1

One logical interface (ge-1/0/0.20) for VXLAN 2

Note:

The Juniper Networks switch dynamically configures these logical interfaces.

Juniper Networks switch (hardware VTEP 1)

Gateway

Based on the configuration of the entities in NSX Manager as described in Table 1, the Juniper Networks switch dynamically creates VXLANs 1 and 2 and their associated logical interfaces. Table 2 provides the relevant NSX Manager configuration and the resulting VXLANs and associated logical interfaces that the Juniper Networks switch dynamically configures.

Table 2: NSX Manager Configurations and Dynamic Configurations by Juniper Networks Switch

NSX Manager Configuration: Logical Switch and Logical Switch Port

VXLANs and Associated Logical Interfaces Dynamically Configured By Juniper Networks Switch

Logical switch configuration:

UUID: 28805c1d-0122-495d-85df-19abd647d772

VNI: 100

Logical switch port configuration:

VLAN ID: 10

For VXLAN 1:

set vlans 28805c1d-0122-495d-85df-19abd647d772 vxlan vni 100

For associated logical interface ge-1/0/0.10:

set interfaces ge-1/0/0 flexible-vlan-tagging
set interfaces ge-1/0/0 encapsulation extended-vlan-bridge
set interfaces ge-1/0/0 unit 10 vlan-id 10
set vlans 28805c1d-0122-495d-85df-19abd647d772 interfaces ge-1/0/0.10 

Logical switch configuration:

UUID: 9acc24b3-7b0a-4c2e-b572-3370c3e1acff

VNI: 200

Logical switch port configuration:

VLAN ID: 20

For VXLAN 2:

set vlans 9acc24b3-7b0a-4c2e-b572-3370c3e1acff vxlan vni 200

For associated logical interface ge-1/0/0.20:

set interfaces ge-1/0/0 flexible-vlan-tagging
set interfaces ge-1/0/0 encapsulation extended-vlan-bridge
set interfaces ge-1/0/0 unit 20 vlan-id 20
set vlans 9acc24b3-7b0a-4c2e-b572-3370c3e1acff interfaces ge-1/0/0.20 

For VXLANs 1 and 2, the Juniper Networks switch uses the UUIDs and VNI values that were provided for the corresponding logical switches.

In the logical switch port configurations in NSX Manager, VLAN ID values 10 and 20 and logical switch mappings are specified. As a result, the Juniper Networks switch creates logical interfaces ge-1/0/0.10 and ge-1/0/0.20, respectively. Both of these logical interfaces function as trunk interfaces. The Juniper Networks switch also maps the logical interfaces ge-1/0/0.10 and ge-1/0/0.20 to their respective VXLANs.

Based on the configurations generated by the Juniper Networks switch, the interface ge-1/0/0.10 accepts packets with a VLAN tag of 10 from VXLAN 1, and interface ge-1/0/0.20 accepts packets with a VLAN tag of 20 from VXLAN 2. On receiving packets from VXLAN 1, a VLAN tag of 100 is added to the packets, and a VLAN tag of 200 is added to packets from VXLAN 2. These tags are added to the respective packet streams to map the VLAN ID in a particular VXLAN to the corresponding VNI.

Table 3 provides a summary of the components that are configured on the Juniper Networks switch. Unless noted, all configurations are performed manually in the Junos OS CLI.

Topology

Table 3: Components for Two VXLAN Topologies Configured on a Juniper Networks Switch that Functions as a Hardware VTEP

Components

Settings

NSX controller

IP address: 10.94.184.1

OVSDB-managed interface

Interface name: ge-1/0/0

VXLAN 1 and associated logical interface

Note:

The Juniper Networks switch dynamically configures the VXLAN and associated logical interface, which are based on the logical switch and logical switch port configurations in NSX Manager. Therefore, no manual configuration is required.

VXLAN name: 28805c1d-0122-495d-85df-19abd647d772

VNI: 100

Logical interface name: ge-1/0/0.10

VLAN ID: 10

Interface type: trunk

VXLAN 2 and associated logical interface

Note:

The Juniper Networks switch dynamically configures the VXLAN and associated interface, which are based on the logical switch and logical switch port configurations in NSX Manager. Therefore, no manual configuration is required.

VXLAN name: VXLAN 9acc24b3-7b0a-4c2e-b572-3370c3e1acff

VNI: 200

Logical interface name: ge-1/0/0.20

VLAN ID: 20

Interface type: trunk

OVSDB tracing operations

Filename: /var/log/ovsdb

File size: 10 MB

Flag: All

Hardware VTEP source identifier

Source interface: loopback (lo0.0)

Source IP address: 10.17.17.17/32

Handling of Layer 2 BUM traffic within VXLAN 28805c1d-0122-495d-85df-19abd647d772 and within VXLAN 9acc24b3-7b0a-4c2e-b572-3370c3e1acff

Service node

Note:

By default, one or more service nodes handle Layer 2 BUM traffic in a VXLAN; therefore, no configuration is required.

Non-OVSDB and Non-VXLAN Configuration

CLI Quick Configuration

To quickly configure this example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.

Procedure

Step-by-Step Procedure

To configure the Layer 3 network over which the packets exchanged between physical server 1 and VM1 are tunneled:

  1. Configure the Layer 3 interface.

  2. Set the routing options.

  3. Configure the routing protocol.

OVSDB and VXLAN Configuration

CLI Quick Configuration

To quickly configure this example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your configuration, copy and paste the commands into the CLI at the [edit] hierarchy level, and then enter commit from configuration mode.

Procedure

Step-by-Step Procedure

To configure the Juniper Networks switch as hardware VTEP 1 and with an OVSDB connection to an NSX controller:

  1. Enable the Juniper Networks switch to dynamically configure OVSDB-managed VXLANs and associated interfaces.

  2. Explicitly configure a connection with an NSX controller.

  3. Specify that interface ge-1/0/0 is managed by OVSDB.

  4. Set up OVSDB tracing operations.

  5. Specify an IP address for the loopback interface. This IP address serves as the source IP address in the outer header of any VXLAN-encapsulated packets.

  6. Set the loopback interface as the interface that identifies hardware VTEP 1.

  7. In NSX Manager, configure a logical switch for VXLAN 1 and a logical switch for VXLAN 2. See the documentation that accompanies NSX Manager.

  8. In NSX Manager, configure a gateway for the Juniper Networks switch, a gateway service for OVSDB-managed interface ge-1/0/0, and a logical switch port for logical interface ge-1/0/0.10, which is associated with VXLAN 1, and a logical switch port for logical interface ge-1/0/0.20, which is associated with VXLAN 2.

    See VMware NSX Configuration for Juniper Networks Devices Functioning as Virtual Tunnel Endpoints.

Verification

Confirm that the configuration is working properly.

Verifying the Logical Switch Configuration

Purpose

Verify that the configuration of logical switches with the UUIDs of 28805c1d-0122-495d-85df-19abd647d772 and 9acc24b3-7b0a-4c2e-b572-3370c3e1acff are present in the OVSDB schema for physical devices and that the Flags field of the show ovsdb logical-switch output is Created by both.

Action

From operational mode, enter the show ovsdb logical-switch command.

Meaning

The output verifies that the configuration for the logical switches is present. The Created by both state indicates that the logical switches were configured in NSX Manager, and that the Juniper Networks switch dynamically configured the corresponding VXLANs. In this state, the logical switches and VXLANs are operational.

If the state of the logical switches is something other than Created by both, see Troubleshooting a Nonoperational Logical Switch and Corresponding Junos OS OVSDB-Managed VXLAN.

Verifying the MAC Addresses of VM 1, VM 3, and VM 4

Purpose

Verify that the MAC addresses of VM 1, VM 3, and VM 4 are present in the OVSDB schema.

Action

From operational mode, enter the show ovsdb mac remote command.

Meaning

The output shows that the MAC addresses for VM 1, VM 3, and VM 4 are present and are associated with their respective logical switches. Given that the MAC addresses are present, VM 1, VM 3, and VM 4 are reachable through the Juniper Networks switch, which functions as a hardware VTEP.

Verifying the NSX Controller Connection

Purpose

Verify that the connection with the NSX controller is up.

Action

From operational mode, enter the show ovsdb controller command to verify that the controller connection state is up.

Meaning

The output shows that the connection state of the NSX controller is up, in addition to other information about the controller. By virtue of this connection being up, OVSDB is enabled on the Juniper Networks switch.

Verifying the OVSDB-Managed Interface

Purpose

Verify that interface ge-1/0/0 is managed by OVSDB.

Action

From operational mode, enter the show ovsdb interface command, and verify that logical interfaces ge-1/0/0.10 and ge-1/0/0.20 are managed by OVSDB.

Meaning

The output shows that logical interfaces ge-1/0/0.10 and ge-1/0/0.20 are managed by OVSDB. It also indicates that interface ge-1/0/0.10 is associated with VXLAN 28805c1d-0122-495d-85df-19abd647d772 and interface ge-1/0/0.20 is associated with VXLAN 9acc24b3-7b0a-4c2e-b572-3370c3e1acff.

Change History Table

Feature support is determined by the platform and release you are using. Use Feature Explorer to determine if a feature is supported on your platform.

Release
Description
14.1X53-D15
Starting with Junos OS Release 14.1X53-D15 for QFX5100 switches, 15.1X53-D10 for QFX10002 switches, 15.1X53-D30 for QFX10008 switches, 15.1X53-D60 for QFX10016 switches, 15.1X53-D210 for QFX5110 and QFX5200 switches, and 18.1R1 for QFX5210 switches, the dynamic configuration of trunk interfaces and their association with an OVSDB-managed VXLAN is supported.