Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

Announcement: Try the Ask AI chatbot for answers to your technical questions about Juniper products and solutions.

close
external-header-nav
keyboard_arrow_up
close
keyboard_arrow_left
list Table of Contents
file_download PDF
{ "lLangCode": "en", "lName": "English", "lCountryCode": "us", "transcode": "en_US" }
English
keyboard_arrow_right

Examples: Manually Configuring VXLANs on QFX Series and EX4600 Switches

date_range 22-Nov-23

The following examples show use cases for manually configuring VXLANs on QFX5100, QFX5110, QFX5200, QFX5210, and EX4600 switches.

Example: Configuring a VXLAN Transit Switch

If a QFX Series or EX4600 switch acts as a transit switch for downstream devices acting as VTEPs, you do not need to configure any VXLAN information on the QFX Series or EX4600 switch. You do need to configure PIM on the switch so that it can form the multicast tree required so that the VTEPs can establish reachablity with each other.

Requirements

This example uses the following hardware and software components:

  • Two QFX5100 switches

  • Junos OS Release 14.1X53-D10

Overview

This example shows a simple use case in which QFX5100 switches are connected to downstream servers acting as VTEPs. The QFX5100 switches need to forward VXLAN packets between VM 1 on Server 1 and VM 2 on Server 2. Because this configuration allows Layer 2 connectivity between the VMs through the VXLAN tunnels, applications can VMotion between the VMs.

Topology

Figure 1 shows QFX 5100 switches configured to forward VXLAN packets for downstream VTEPs.

Figure 1: QFX5100 Acting as a VXLAN Transit SwitchQFX5100 Acting as a VXLAN Transit Switch

Configuring PIM on the Transit Switches

CLI Quick Configuration

To quickly configure this example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your network configuration, and then copy and paste the commands into the CLI at the [edit] hierarchy level.

content_copy zoom_out_map
set protocols pim interface all
set protocols pim rp static address ip-address

Procedure

Step-by-Step Procedure

If you are not using an SDN controller to create a VXLAN control plane, you must enable PIM on each switch so that the VTEP can use multicast groups to advertise its existence and to learn about other VTEPs. (Configuring PIM automatically enables IGMP.) You do not need to perform any VXLAN-specific configuration. Note that you also do not need to configure VLAN 1 on either switch.

  1. Enable PIM.

    content_copy zoom_out_map
    [edit]
    user@switch# set protocols pim interface all
  2. Configure the address of a PIM rendezvous point.

    content_copy zoom_out_map
    [edit]
    user@switch#  set protocols pim rp static address ip-address

Example: Configuring a VXLAN Layer 2 Gateway

If a QFX Series or EX4600 switch is connected to a downstream server that hosts a VM that needs Layer 2 connectivity with another VM that is reachable only through a Layer 3 network, you must do the following:

  • Configure the switch to act as a VTEP—that is, a Layer 2 gateway for downstream Layer 2 devices.

  • Configure PIM on the switch so that it can form the multicast tree required for reachability with other VTEPs and to allow BUM traffic to be forwarded between the VTEPs.

  • Enable a routing protocol, for example, OSPF, on the VTEP’s loopback interface and Layer 3 interfaces.

Requirements

This example uses the following hardware and software components:

  • Two QFX5100 switches

  • Junos OS Release 14.1X53-D10

Overview

This example shows a use case in which QFX5100 switches act as VTEPs that allow Layer 2 connectivity between VM 1 on Server 1 and VM 2 on Server 2 so that VMotion can occur between the VMs. The servers in this example can be in the same or different data centers—the only constraint is that there must be Layer 3 connectivity between the QFX5100 switches. This allows your network to be very agile in response to demand for server usage or changes in bandwidth requirements.

Note that because the same VLAN exists in both Layer 2 domains and both switches encapsulate the VLAN traffic into the same VXLAN, you do not need a gateway for the VXLAN traffic in the Layer 3 network. The Layer 3 VXLAN packets are routed normally and no de-encapsulation or re-encapsulation is required.

Topology

Figure 2 shows QFX5100 switches configured to act as VTEPs.

Figure 2: QFX5100 Acting as a VTEPQFX5100 Acting as a VTEP

Configuring the Switches

CLI Quick Configuration

To quickly configure the QFX5100-96S 1 in this example, copy the following commands, paste them into a text file, remove any line breaks, change any details necessary to match your network configuration, and then copy and paste the commands into the CLI at the [edit] hierarchy level.

content_copy zoom_out_map
set interfaces lo0 unit 0 family inet address 10.1.1.1
set switch-options vtep–source-interface lo0.0
set protocols pim interface lo0.0
set protocols ospf area 0.0.0.0 interface lo0.0
set protocols pim interface xe-0/0/0.0
set protocols ospf area 0.0.0.0 interface xe-0/0/0.0
set protocols pim rp static address 10.2.2.2
set vlans VLAN1 vlan-id 100 vxlan vni 100 multicast-group 233.252.0.2
set vlans VLAN1 vxlan encapsulate-inner-vlan
set vlans VLAN1 vxlan unreachable-vtep-aging-timer 600
set protocols l2-learning decapsulate-accept-inner-vlan
set interfaces xe-0/0/0 unit 0 family inet address 10.2.2.100/24
set interfaces xe-0/0/1 unit 0 family ethernet-switching interface-mode trunk
set interfaces xe-0/0/1 unit 0 family ethernet-switching vlan members all
set protocols ospf enable
set protocols ospf area 0.0.0.0 interface em0.0 disable
set protocols ospf area 0.0.0.0 interface all

The configuration for QFX5100-96S 2 is identical except for changes to a few of the addresses:

content_copy zoom_out_map
set interfaces lo0 unit 0 family inet address 10.1.1.2
set switch-options vtep–source-interface lo0.0
set protocols pim interface lo0.0
set protocols ospf area 0.0.0.0 interface lo0.0
set protocols pim interface xe-0/0/0.0
set protocols ospf area 0.0.0.0 interface xe-0/0/0.0
set protocols pim rp static address 10.2.2.2
set vlans VLAN1 vlan-id 100 vxlan vni 100 multicast-group 233.252.0.2
set vlans VLAN1 vxlan encapsulate-inner-vlan
set vlans VLAN1 vxlan unreachable-vtep-aging-timer 600
set protocols l2-learning decapsulate-accept-inner-vlan
set interfaces xe-0/0/0 unit 0 family inet address 10.2.2.200/24
set interfaces xe-0/0/1 unit 0 family ethernet-switching interface-mode trunk
set interfaces xe-0/0/1 unit 0 family ethernet-switching vlan members all
set protocols ospf enable
set protocols ospf area 0.0.0.0 interface em0.0 disable
set protocols ospf area 0.0.0.0 interface all
Note:

You must configure the same multicast group address for VLAN1 on both switches.

Procedure

Step-by-Step Procedure

Perform the following procedure on both switches to set up the example configuration.

  1. Create a reachable IPv4 address on the loopback interface.

    content_copy zoom_out_map
    [edit]
    user@switch# set interfaces lo0 unit 0 family inet address 10.1.1.1

    For switch QFX5100-96S 2, use address 10.1.1.2.

  2. Configure the loopback interface—and therefore, its associated address—to be used as the tunnel source address.

    content_copy zoom_out_map
    [edit]
    user@switch# set switch-options vtep-source-interface lo0.0
  3. Enable PIM on the loopback interface.

    content_copy zoom_out_map
    [edit]
    user@switch# set protocols pim interface lo0.0
  4. Add the loopback interface to OSPF area 0.0.0.0.

    content_copy zoom_out_map
    [edit]
    user@switch# set protocols ospf area 0.0.0.0 interface lo0.0
  5. Enable PIM on the interface that connects to the Layer 3 network.

    content_copy zoom_out_map
    [edit]
    user@switch# set protocols pim interface xe-0/0/0.0
  6. Add the interface that connects to the Layer 3 network to OSPF area 0.0.0.0.

    content_copy zoom_out_map
    [edit]
    user@switch# set protocols ospf area 0.0.0.0 interface xe-0/0/0.0
  7. Configure the address of a PIM rendezvous point.

    content_copy zoom_out_map
    [edit]
    user@switch#  set protocols pim rp static address 10.2.2.2
  8. Create a VLAN, map it to a VXLAN, and assign a multicast group address to the VXLAN. All members of a VXLAN must use the same multicast group address.

    content_copy zoom_out_map
    [edit]
    user@switch# set vlans VLAN1 vlan-id 100 vxlan vni 100 multicast-group 233.252.0.2

    In this example, the vlan-id and vni are both set to 100. This is done only for simplicity and clarity. You do not need to set the vlan-id and vni to the same value.

  9. (Optional) Configure the switch to retain the original VLAN tag (in the inner Ethernet packet) after VXLAN encapsulation. By default, the original tag is dropped when the packet is encapsulated.

    content_copy zoom_out_map
    [edit]
    user@switch# set vlans VLAN1 vxlan encapsulate-inner-vlan
  10. (Optional) Configure the system to age out the address for the remote VTEP (the other QFX5100 switch) if all the MAC addresses learned from that VTEP age out. The address for the remote VTEP expires the configured number of seconds after the last learned MAC address expires.

    content_copy zoom_out_map
    [edit]
    user@switch# set vlans VLAN1 vxlan unreachable-vtep-aging-timer 600

    (Optional) Configure the switch to de-encapsulate and accept original VLAN tags in VXLAN packets. By default, a preserved VLAN tag is dropped when the packet is de-encapsulated.

    content_copy zoom_out_map
    [edit]
    user@switch# set protocols l2-learning decapsulate-accept-inner-vlan
  11. Configure the interface that connects to the Layer 3 network.

    content_copy zoom_out_map
    [edit]
    user@switch# set interfaces xe-0/0/0 unit 0 family inet address 10.2.2.100/24 

    For switch QFX5100-96S 2, use address 10.2.2.200.

  12. Configure the server-facing interface to support multiple VLANs.

    content_copy zoom_out_map
    [edit]
    user@switch# set interfaces xe-0/0/1 unit 0 family ethernet-switching interface-mode trunk
    [edit]
    user@switch# set interfaces xe-0/0/1 unit 0 family ethernet-switching vlan members all
    Note:

    Because this example shows only one VLAN, this step is not required for the example. In a real-world configuration, however, it would be required in order to support multiple VMs connected to multiple VLANs. In this case you would also need to configure additional VLAN to VXLAN mappings.

  13. Enable OSPF on all interfaces that are mapped to area 0.0.0.0..

    content_copy zoom_out_map
    [edit]
    user@switch# set protocols ospf enable
    [edit]
    user@switch# set protocols ospf area 0.0.0.0 interface em0.0 disable
    [edit]
    user@switch# set protocols ospf area 0.0.0.0 interface all

Results

From configuration mode, confirm your configuration by entering the following commands on QFX5100-96S 1. If the output does not display the intended configuration, repeat the instructions in this example to correct the configuration.

content_copy zoom_out_map
user@switch# show switch-options
content_copy zoom_out_map
vtep-source-interface lo0.0;
content_copy zoom_out_map
user@switch# show vlans
content_copy zoom_out_map
VLAN1 {
    vlan-id 100;
    vxlan {
        vni 100;
        multicast-group 233.252.0.2;
        encapsulate-inner-vlan;
    }
}
content_copy zoom_out_map
user@switch# show interfaces
content_copy zoom_out_map
xe-0/0/0 {
    unit 0 {
        family inet {
            address 10.2.2.100/24;
        }
    }
}
xe-0/0/1 {
    unit 0 {
        family ethernet-switching {
            interface-mode trunk;
            vlan {
                members all;
            }
        }
    }
}
lo0 {
    unit 0 {
        family inet {
            address 10.1.1.1/32;
        }
    }
}
content_copy zoom_out_map
user@switch# show protocols pim
content_copy zoom_out_map
rp {
    static {
        address 10.2.2.2;
    }
}
interface xe-0/0/0.0

Verification

Confirm that the configuration is working properly.

Verifying VXLAN Reachability

Purpose

On QFX5100-96S 1, verify that there is connectivity with the remote VTEP (QFX5100-96S 2).

Action
content_copy zoom_out_map
user@switch> show ethernet-switching vxlan-tunnel-end-point remote
Logical System Name       Id  SVTEP-IP         IFL   L3-Idx
<default>                 0   10.1.1.2     lo0.0    0 
RVTEP-IP         IFL-Idx   NH-Id
10.1.1.2         559       1728
VNID             MC-Group-IP
100              233.252.0.2
Meaning

The VTEP on QFX5100-96S 2 is reachable because its IP address (the address assigned to the loopback interface) appears in the output. The output also shows that the VXLAN (VNI 100) and corresponding multicast group are configured correctly on the remote VTEP.

Verifying That the Local VTEP Is Configured Correctly

Purpose

On QFX5100-96S 1, verify that the tunnel endpoint is correct.

Action
content_copy zoom_out_map
user@switch> show ethernet-switching vxlan-tunnel-end-point source

Logical System Name       Id  SVTEP-IP         IFL   L3-Idx
<default>                 0   10.1.1.1     lo0.0    0 
L2-RTT                   Bridge Domain              VNID     MC-Group-IP
default-switch           VLAN1+100                  100      233.252.0.2
Meaning

The VTEP on QFX5100-96S 1 shows the correct tunnel source IP address (assigned to the loopback interface), VLAN, and multicast group for the VXLAN.

Verifying MAC Learning from the Remote VTEP

Purpose

On QFX5100-96S 1, verify that it is learning MAC addresses from the remote VTEP.

Action
content_copy zoom_out_map
user@switch> show ethernet-switching table

MAC flags (S - static MAC, D - dynamic MAC, L - locally learned, P - Persistent static
           SE - statistics enabled, NM - non configured MAC, R - remote PE MAC)
Ethernet switching table : 2 entries, 2 learned
Routing instance : default-switch
    Vlan                MAC                 MAC         Age    Logical
    name                address             flags              interface
    VLAN1                00:00:00:ff:ff:ff   D             -   vtep.12345
    VLAN1                00:10:94:00:00:02   D         -       xe-0/0/0.0
Meaning

The output shows the MAC addresses learned from the remote VTEP (in addition to those learned on the normal Layer 2 interfaces). It also shows the logical name of the remote VTEP interface (vtep.12345 in the above output).

Monitor the Remote Interface

Purpose

On QFX5100-96S 1, monitor traffic details for the remote VTEP interface.

Action
content_copy zoom_out_map
user@switch> show interface vtep.12345 detail

M    Flags: Up SNMP-Traps Encapsulation: ENET2
    VXLAN Endpoint Type: Remote, VXLAN Endpoint Address: 10.1.1.2, L2 Routing Instance: default-switch, L3 Routing Instance: default
    Traffic statistics:
     Input  bytes  :         228851738624
     Output bytes  :                    0
     Input  packets:            714162415
     Output packets:                    0
    Local statistics:
     Input  bytes  :                    0
     Output bytes  :                    0
     Input  packets:                    0
     Output packets:                    0
    Transit statistics:
     Input  bytes  :         228851738624                    0 bps
     Output bytes  :                    0                    0 bps
     Input  packets:            714162415                    0 pps
     Output packets:                    0                    0 pps
    Protocol eth-switch, MTU: 1600, Generation: 277, Route table: 5
Meaning

The output shows traffic details for the remote VTEP interface. To get this information, you must supply the logical name of the remote VTEP interface (vtep.12345 in the above output), which you can learn by using the show ethernet-switching table command.

Verifying OSPF Neighbor

Purpose

On QFX5100-96S 1, verify that the remote VTEP (QFX5100-96S 2) has established itself as an OSPF neighbor.

Action
content_copy zoom_out_map
user@switch> show ospf neighbor 

Address          Interface              State     ID               Pri  Dead
10.2.2.200       xe-0/0/0.0             Full      10.2.3.2          128    38
Meaning

The output shows that interface xe-0/0/0.0 on QFX5100-96S 2 is in the Full state, which means that QFX5100-96S 2 is a fully adjacent neighbor. The Full state also means that Layer 3 connectivity exists between QFX5100-96S 1 and QFX5100-96S 2.

external-footer-nav