close

keyboard_arrow_left

Table of Contents Expand all

play_arrow Overview
- play_arrow Storage Overview
play_arrow Transit Switch, FCoE, and FIP Snooping
- play_arrow Using FCoE on a Transit Switch
play_arrow Fibre Channel and FCoE-FC Gateways
- play_arrow Using Fibre Channel and FCoE-FC Gateways
play_arrow Data Center Bridging (DCBX, PFC)
- play_arrow Using Data Center Bridging (DCBX, PFC)
- play_arrow Learn About Technology
  - Learn About Data Center Bridging
  - Data Center Technology Overview Videos
play_arrow Configuration Statements and Operational Commands
- Junos CLI Reference Overview

list Table of Contents

English

Example: Configuring an FCoE LAG on a Redundant Server Node Group

date_range 23-Nov-23

arrow_backward

arrow_forward

This example shows how to configure a Fibre Channel over Ethernet (FCoE) link aggregation group (LAG) on a redundant server Node group (RSNG) to transport FCoE traffic and regular Ethernet traffic across the same link aggregation bundle. The FCoE servers have converged network adapters (CNAs) and communicate with the Fibre Channel (FC) storage area network (SAN). FCoE servers are usually connected to a switch that performs FIP snooping, such as an FCoE transit switch or an FCoE-FC gateway switch that performs FIP snooping. This example provides a common FCoE LAG configuration for an FCoE transit switch and an FCoE-FC gateway, and shows how to disable FIP snooping scaling on an FCoE untrusted FCoE-FC gateway fabric (fc-fabric).

Requirements

This example uses the following hardware and software components:

Two Juniper Networks QFabric System Node devices configured as an RSNG. The Node devices can be configured as FCoE transit switches or as FCoE-FC gateways. (A configuration with one Node device as an FCoE transit switch and the other Node device as an FCoE-FC gateway is possible providing that the transit switch and the FCoE-FC gateway use different FCoE VLANs.)
Junos OS Release 13.2X52-D10 or later for the QFX Series
One FCoE server with two CNA ports

Overview

Standard LAGs use a hashing algorithm to determine which physical link in the LAG is used for a transmission, so a series of communications between two devices might use different physical links in the LAG for different transmissions. However, FCoE traffic requires a point-to-point link (or a virtual point-to-point link) between the FCoE device and the Fibre Channel (FC) storage area network (SAN) switch.

An FCoE LAG solves this problem by ensuring that the same LAG link is used for communication between a given FCoE device and the QFabric system Node device, preserving point-to-point link emulation. At the same time, regular Ethernet traffic (traffic that is not FCoE traffic) on the LAG is distributed across member interfaces in the same way as on a standard LAG. FCoE traffic is treated properly in terms of maintaining a virtual point-to-point link with the FC SAN, and regular Ethernet traffic enjoys the usual LAG benefits of load balancing and link redundancy.

Note:

Configuring a LAG as an FCoE LAG does not provide link redundancy for FCoE traffic, and does not load balance FCoE traffic.

On FCoE-FC gateway untrusted Fibre Channel fabrics (fc-fabrics), if you configure an FCoE LAG, you must also disable enhanced FIP snooping scaling (scaling up to 2,500 sessions), which reduces the number of supported FIP snooping sessions to 376 sessions. On an FCoE-FC gateway, disabling enhanced FIP snooping scaling is global to the Node device. Trusted fc-fabrics on an FCoE-FC gateway support enhanced FIP snooping scaling.

This example shows you how to:

Configure the RSNG and its Node devices
Configure the FCoE LAG on the RSNG
Configure a dedicated VLAN for FCoE traffic (an FCoE VLAN) and a native VLAN for untagged FCoE initialization protocol (FIP) traffic
Enable VN2VF_Port FIP snooping on the FCoE VLAN
Disable FIP snooping scaling on an untrusted FCoE-FC gateway fabric

Note:

FCoE traffic requires lossless transport across the Ethernet network to comply with the requirements for transporting storage traffic. This example describes how to configure an FCoE LAG to provide redundancy for FCoE traffic. See Example: Configuring CoS PFC for FCoE Traffic for how to configure lossless transport for FCoE traffic.

Note:

On a Node device that is configured as an FCoE-FC gateway, you must create a Fibre Channel fabric, configure native FC interfaces, configure an FCoE VLAN interface (a Layer 3 RVI) for the FCoE VLAN (which includes the FCoE LAG as a member interface), and add the native FC interfaces and FCoE VLAN interface to the FC fabric. For an example of FCoE-FC gateway interface configuration, see Example: Setting Up Fibre Channel and FCoE VLAN Interfaces in an FCoE-FC Gateway Fabric.

Topology

Table 1 shows the configuration components for this example.

Table 1: Components of the FCoE LAG Configuration Example
Component	Settings
Hardware	Two QFabric system Node devices configured as an RSNG (the Node devices can be configured as FCoE transit switches or as FCoE-FC gateways; this example is valid for both modes): RSNG name—RSNG1 First Node device—Serial number ABCD1234, alias name row1-rack1 Second Node device—Serial number ABCD1235, alias name row1-rack2 Note: The alias names chosen for this example indicate the physical locations of the Node devices. You can use any aliasing system you want to make identifying Node devices easier, or you can use the default Node device names (the Node device serial numbers). One FCoE server with two CNA ports.
LAG configuration	RSNG device count—48 FCoE LAG name—RSNG1:ae20 FCoE LAG member interfaces—row1rack1:xe-0/0/20 and row1rack2:xe-0/0/20 FCoE LAG LACP—active FCoE LAG port mode—trunk MTU—2180 FCoE LAG VLAN memberships—FCoE VLAN (`fcoe-vlan1`) and native VLAN
FCoE VLAN	Name—`fcoe-vlan1` VLAN ID—`2000` Member interfaces—`RSNG1:ae20`
Native VLAN	Name—`native` VLAN ID—`1` Member interfaces—`RSNG1:ae20`
VN2VF_Port FIP snooping	Enabled on the FCoE VLAN (`fcoe-vlan1`)
FIP snooping scaling	Enabled for FCoE transit switch portion of the example. Disabled for the FCoE-FC gateway portion of the example (gateway FC fabric is FCoE untrusted).

Figure 1 shows the network topology for this example.

Figure 1: FCoE LAG Example Topology

Configuration

To configure an FCoE LAG between an FCoE server with two CNA ports and the two Node device members of an RSNG, perform these tasks:

CLI Quick Configuration
Configuring an FCoE LAG on an RSNG (FCoE Transit Switch or FCoE-FC Gateway)
Disabling Enhanced FIP Snooping Scaling on an FCoE-FC Gateway
Results

CLI Quick Configuration

In this example, the enhanced FIP snooping scaling is disabled (376 sessions) on the FCoE-FC gateway because the gateway fabric is an untrusted fc-fabric.

Most of the FCoE LAG configuration is common to both the FCoE transit switch and FCoE-FC gateway modes of operation. The CLI Quick Configuration shows the common configuration statements first, followed by the additional configuration statement that disables FIP snooping scaling on the FCoE-FC gateway. Disabling FIP snooping scaling on an FCoE-FC gateway is a global configuration that affects all of the fc-fabrics on the gateway. (On an FCoE transit switch, you can disable FIP snooping scaling on an individual FCoE VLAN without affecting other FCoE VLANs.)

Note:

This example does not include configuring the FC fabric, the native FC fabric ports, and the Layer 3 FCoE VLAN interface.

To quickly configure an FCoE LAG , copy the following commands, paste them in a text file, remove line breaks, change variables and details to match your network configuration, and then copy and paste the commands into the CLI at the [edit] hierarchy level.

Common configuration:

content_copy zoom_out_map
set fabric aliases node-device ABCD1234 row1-rack1
set fabric aliases node-device ABCD1235 row1-rack2
set fabric resources node-group RSNG1 node-device row1-rack1
set fabric resources node-group RSNG1 node-device row1-rack2
set chassis node-group RSNG1 aggregated-devices ethernet device-count 48
set interfaces RSNG1:ae20 unit 0 family ethernet-switching port-mode trunk vlan members fcoe-vlan1
set interfaces RSNG1:ae20 unit 0 family ethernet-switching native-vlan-id 1
set interfaces RSNG1:ae20 mtu 2180
set interfaces RSNG1:ae20 aggregated-ether-options fcoe-lag
set interfaces RSNG1:ae20 aggregated-ether-options lacp active
set interfaces row1-rack1:xe-0/0/20 ether-options 802.3ad RSNG1:ae20
set interfaces row1-rack2:xe-0/0/20 ether-options 802.3ad RSNG1:ae20
set vlans fcoe-vlan1 vlan-id 2000
set vlans native vlan-id 1
set vlans fcoe-vlan1 interface RSNG1:ae20
set ethernet-switching-options secure-access-port vlan fcoe-vlan1 examine-fip

Note:

If you want to configure an FCoE-FC gateway fabric as a trusted fabric so that you can leave enhanced FIP snooping scaling enabled on the gateway, add the following statement to the configuration, replacing the variable fc-fabric-name with the name of the FC fabric (if you do this, do not disable FIP snooping scaling as shown in the FCoE-FC Gateway Additional Configuration):

content_copy zoom_out_map
set fc-fabrics fc-fabric-name protocols fip fcoe-trusted

Additional configuration to disable enhanced FIP snooping scaling on an FCoE-FC gateway untrusted FC fabric:

content_copy zoom_out_map
set fc-options no-fip-snooping-scaling

Configuring an FCoE LAG on an RSNG (FCoE Transit Switch or FCoE-FC Gateway)

Step-by-Step Procedure

To configure the RSNG member Node devices, the FCoE LAG, the FCoE VLAN, and VN2VF_Port FIP snooping on an FCoE transit switch or an FCoE-FC gateway:

Define aliases for the two Node devices that will be in the RSNG (aliases are easier to remember and more descriptive than the Node device serial number). Name the Node device with serial number ABCD1234 as row1-rack1 and the Node device with the serial number ABCD1235 as row1-rack2:
```
admin@qfabric# set fabric aliases node-device ABCD1234 row1-rack1
admin@qfabric# set fabric aliases node-device ABCD1235 row1-rack2
```

Configure the Node device membership for row1-rack1 and row1-rack2 in the RSNG RSNG1:

content_copy zoom_out_map
admin@qfabric# set fabric resources node-group RSNG1 node-device row1-rack1
admin@qfabric# set fabric resources node-group RSNG1 node-device row1-rack2

Configure the number of LAG interfaces that RSNG RSNG1 can support. (Each Node device in the RSNG has 48 server-facing ports. If we used one port from each Node device to provide Node device redundancy for each LAG, we might need to support a maximum of 48 LAGs, so we set the device count to 48 LAGs.)
```
admin@qfabric# set chassis node-group RSNG1 aggregated-devices ethernet device-count 48
```
Configure the LAG interface (ae20) on RSNG1 and set the port mode to trunk mode. In the same statement, configure the LAG interface membership in the dedicated FCoE VLAN fcoe-vlan1:
```
admin@qfabric# set interfaces RSNG1:ae20 unit 0 family ethernet-switching port-mode trunk vlan members fcoe-vlan1
```

Configure the LAG interface membership in the native VLAN:

content_copy zoom_out_map
admin@qfabric# set interfaces RSNG1:ae20 unit 0 family ethernet-switching native-vlan-id 1

Configure the LAG interface with an MTU of 2180 to accommodate the size of the FCoE frame and headers.
```
admin@qfabric# set interfaces RSNG1:ae20 mtu 2180
```

Configure the LAG RSNG1:ae20 as an FCoE LAG:

content_copy zoom_out_map
admin@qfabric# set interfaces RSNG1:ae20 aggregated-ether-options fcoe-lag

Enable LACP on the FCoE LAG:

content_copy zoom_out_map
admin@qfabric# set interfaces RSNG1:ae20 aggregated-ether-options lacp active

Assign one Ethernet interface on each RSNG Node device to the FCoE LAG:

content_copy zoom_out_map
admin@qfabric# set interfaces row1-rack1:xe-0/0/20 ether-options 802.3ad RSNG1:ae20
admin@qfabric# set interfaces row1-rack2:xe-0/0/20 ether-options 802.3ad RSNG1:ae20

Configure a dedicated VLAN for FCoE traffic (an FCoE VLAN) named fcoe-vlan1 with the VLAN ID 2000:
```
admin@qfabric# set vlans fcoe-vlan1 vlan-id 2000
```
Configure a native VLAN with the VLAN ID 1 to carry untagged FIP traffic:
```
admin@qfabric# set vlans native vlan-id 1
```
Assign the FCoE LAG interface to the FCoE VLAN:
```
admin@qfabric# set vlans fcoe-vlan1 interface RSNG:ae20
```
Assign the FCoE LAG interface to the native VLAN:
```
admin@qfabric# set vlans native interface RSNG:ae20
```

Enable VN2VF_Port FIP snooping on the FCoE VLAN:

content_copy zoom_out_map
admin@qfabric# set ethernet-switching-options secure-access-port vlan fcoe-vlan1 examine-fip

Disabling Enhanced FIP Snooping Scaling on an FCoE-FC Gateway

Step-by-Step Procedure

To disable enhanced FIP snooping scaling on an FCoE-FC gateway:

Disable FIP snooping scaling on the gateway fabrics. Disabling FIP snooping scaling on an FCoE-FC gateway is global to the gateway, so every FC fabric on the gateway reverts to supporting 376 sessions (instead of 2,500 sessions as with FIP snooping scaling enabled).
```
admin@qfabric# set fc-options no-fip-snooping-scaling
```

Results

Display the results of the configuration. The results below show the configuration on an FCoE transit switch and have been edited to include only the components configured in the example:

content_copy zoom_out_map
admin@qfabric> show configuration
root@qfabric>fabric {
    resources {
        node-group RSNG1 {
            node-device row1-rack1;
            node-device row1-rack2;
        }
    }
    aliases {
        node-device ABCD1234 {
            row1-rack1;
        }
        node-device ABCD1235 {
            row1-rack2;
        }
    }
}
chassis {
    node-group RSNG1 {
        aggregated-devices {
            ethernet {
                device-count 48;
            }
        }
    }
}
interfaces {
    RSNG1:ae20 {
        aggregated-ether-options {
            fcoe-lag;
            lacp {
                active;
            }
        }
        unit 0 {
            family ethernet-switching {
                port-mode trunk;
                vlan {
                    members fcoe-vlan1;
                }
                native-vlan-id 1;
            }
        }
    }
    row1-rack1:xe-0/0/20 {
        ether-options {                 
            802.3ad RSNG1:ae20;
        }
    }
    row1-rack2:xe-0/0/20 {
        ether-options {
            802.3ad RSNG1:ae20;
        }
    }
}
ethernet-switching-options {
    secure-access-port {
        vlan fcoe-vlan1 {
            examine-fip;
        }
    }
}
vlans {
    fcoe-vlan1 {
        vlan-id 2000;
        interface {
            RSNG1:ae20.0;
        }
    }
    native {
        vlan-id 1;
        interface {
            RSNG1:ae20.0;
        }
    }
}

Verification

To verify the configuration of the QFabric system Node device resources, FCoE LAG, FCoE VLAN, native VLAN, and FIP snooping, perform these tasks:

Verifying the Node Device Aliases (Names)
Verifying the Node device Assignment to the Node Group
Verifying the Number of Aggregated Ethernet Logical Devices (LAG Interfaces) That the Node Group Can Support
Verifying the FCoE LAG Interface Configuration
Verifying the FCoE VLAN and Native VLAN Configuration
Verifying the FIP Snooping Configuration

Verifying the Node Device Aliases (Names)

Purpose
Action
Meaning

Purpose

Verify that the Node device alias names are configured.

Action

List the Node device inventory on the QFabric system using the show fabric administration inventory node-devices command:

content_copy zoom_out_map
admin@qfabric> show fabric administration inventory node-devices
root@qfabric>show fabric administration inventory node-devices   
Item                    Identifier              Connection      Model
Node device
  row1-rack1            ABCD1234                Connected       qfx3500 
  row1-rack2            ABCD1235                Connected       qfx3500 

Meaning

The show fabric administration inventory node-devices command lists the Node device names in the Node device column and lists the Node device serial numbers in the Identifier column. The Connection column shows if the Director device has detected the Node device, and the Model column lists QFX switch model type.

The command output shows that Node device ABCD1234 is configured with the name (alias) row1-rack1, and the Node device ABCD1235 is configured with the name row1-rack2.

Verifying the Node device Assignment to the Node Group

Purpose
Action
Meaning

Purpose

Verify that the redundant server Node group includes the two Node devices.

Action

Verify that the QFabric system Node group RSNG1 is configured with the correct Node devices using the show configuration fabric resources command:

content_copy zoom_out_map
admin@qfabric> show configuration fabric resources
root@qfabric> show configuration fabric resources 
node-group RSNG1 {
    node-device row1-rack1;
    node-device row1-rack2;
}

Meaning

The show configuration fabric resources command lists the Node groups and the Node devices in the Node groups. The command output shows that Node group RSNG1 consists of the Node devices row1-rack1 and row1-rack2.

Verifying the Number of Aggregated Ethernet Logical Devices (LAG Interfaces) That the Node Group Can Support

Purpose
Action
Meaning

Purpose

Verify the number of LAG interfaces that the redundant server node group supports.

Action

List the LAG interface device count using the show configuration chassis command:

content_copy zoom_out_map
admin@qfabric> show configuration chassis
node-group RSNG1 {
    aggregated-devices {
        ethernet {
            device-count 48;
        }
    }
}

Meaning

The show configuration chassis command displays the Ethernet device count (the number of LAG interfaces supported) as 48 devices.

Verifying the FCoE LAG Interface Configuration

Purpose
Action
Meaning

Purpose

Verify that the FCoE LAG interface, port mode, interface VLAN membership, and Node device interface membership in the FCoE LAG are correctly configured.

Action

List the FCoE LAG interface and Node device interface information using the show configuration interfaces command:

content_copy zoom_out_map
admin@qfabric> show configuration interfaces
RSNG1:ae20 {
    aggregated-ether-options {
        fcoe-lag;
        lacp {
            active;
        }
    }
    unit 0 {
        family ethernet-switching {
            port-mode trunk;
            vlan {
                members fcoe-vlan1;
            }
            native-vlan-id 1;
        }
    }
}
row1-rack1:xe-0/0/20 {
    ether-options {
        802.3ad RSNG1:ae20;
    }
}
row1-rack2:xe-0/0/20 {
    ether-options {
        802.3ad RSNG1:ae20;
    }
}

Meaning

The show configuration interfaces command lists both the LAG interfaces and the individual Node device interfaces, and their configuration.

The command output shows a lot of information about the interfaces:

The LAG interface name is RSNG1:ae20
fcoe-lag confirms the LAG is an FCoE LAG
lacp is configured in active mode
Port mode is trunk
The LAG has membership in the fcoe-vlan1 VLAN and in the native VLAN with the VLAN ID 1.
Interface row1-rack1:xe-0/0/20 is a member of FCoE LAG RSNG1:ae20
Interface row1-rack2:xe-0/0/20 is a member of FCoE LAG RSNG1:ae20

Verifying the FCoE VLAN and Native VLAN Configuration

Purpose
Action
Meaning

Purpose

Verify that the FCoE VLAN fcoe-vlan1 and the native VLAN native are configured with the correct VLAN tags (2000 and 1, respectively) and that the FCoE LAG interface RSNG1:ae20 is assigned to the VLANs.

Action

List the VLAN information using the show configuration vlans command:

content_copy zoom_out_map
admin@qfabric> show configuration vlans
fcoe-vlan1 {
    vlan-id 2000;
    interface {
        RSNG1:ae20.0;
    }
}
native {
    vlan-id 1;
    interface {
        RSNG1:ae20.0;
}

Meaning

The show configuration vlans command lists the configured VLANs, their VLAN IDs, and the interfaces assigned to the VLANs.

The command output shows that the FCoE VLAN fcoe-vlan1 is configured with the VLAN ID 2000 and is assigned to the FCoE LAG interface RSNG1:ae20.

The command output also shows that the native VLAN native is configured with the VLAN ID 1 and is assigned to the FCoE LAG interface RSNG1:ae20.

Purpose

Verify that VN2VF_Port FIP snooping is enabled on the FCoE VLAN (fcoe-vlan1).

Action

List the FIP snooping information using the show configuration ethernet-switching-options command:

content_copy zoom_out_map
admin@qfabric> show configuration ethernet-switching-options
secure-access-port {
    vlan fcoe-vlan1 {
        examine-fip;
    }
}

Meaning

The show configuration ethernet-switching-options command lists the security options configured on VLANs. The command output shows that on VLAN fcoe-vlan1, VN2VF_Port FIP snooping is enabled (examine-fip output).

arrow_backward PREVIOUS Configuring an FCoE LAG

NEXT arrow_forward Understanding OxID Hash Control for FCoE Traffic Load Balancing on QFabric Systems

Storage User Guide

ON THIS PAGE

Example: Configuring an FCoE LAG on a Redundant Server Node Group

Requirements

Overview

Topology

Configuration

CLI Quick Configuration

Configuring an FCoE LAG on an RSNG (FCoE Transit Switch or FCoE-FC Gateway)

Step-by-Step Procedure

Disabling Enhanced FIP Snooping Scaling on an FCoE-FC Gateway

Step-by-Step Procedure

Results

Verification

Verifying the Node Device Aliases (Names)

Purpose

Action

Meaning

Verifying the Node device Assignment to the Node Group

Purpose

Action

Meaning

Verifying the Number of Aggregated Ethernet Logical Devices (LAG Interfaces) That the Node Group Can Support

Purpose

Action

Meaning

Verifying the FCoE LAG Interface Configuration

Purpose

Action

Meaning

Verifying the FCoE VLAN and Native VLAN Configuration

Purpose

Action

Meaning

Verifying the FIP Snooping Configuration

Purpose

Action

Meaning

Stay in touch

Helped me install or use the product
Accelerated my deployment

Discuss the product with a co-worker
Make a purchase inquiry

Storage User Guide

ON THIS PAGE

Example: Configuring an FCoE LAG on a Redundant Server Node Group

Requirements

Overview

Topology

Configuration

CLI Quick Configuration

Configuring an FCoE LAG on an RSNG (FCoE Transit Switch or FCoE-FC Gateway)

Step-by-Step Procedure

Disabling Enhanced FIP Snooping Scaling on an FCoE-FC Gateway

Step-by-Step Procedure

Results

Verification

Verifying the Node Device Aliases (Names)

Purpose

Action

Meaning

Verifying the Node device Assignment to the Node Group

Purpose

Action

Meaning

Verifying the Number of Aggregated Ethernet Logical Devices (LAG Interfaces) That the Node Group Can Support

Purpose

Action

Meaning

Verifying the FCoE LAG Interface Configuration

Purpose

Action

Meaning

Verifying the FCoE VLAN and Native VLAN Configuration

Purpose

Action

Meaning

Verifying the FIP Snooping Configuration

Purpose

Action

Meaning

Related Documentation

Stay in touch