Help us improve your experience.

Let us know what you think.

Do you have time for a two-minute survey?

Navigation

Example: Configuring an FCoE Transit Switch

You can use an EX4500 CEE-enabled switch as a Fibre Channel over Ethernet (FCoE) transit switch, enabling it to transport both FCoE and Ethernet LAN traffic. Using the same switch to support both your storage network and traditional IP-based data communications reduces the costs of powering, cooling, provisioning, maintaining, and managing your network.

This example includes:

  • FIP snooping for security
  • Priority-based flow control (PFC) for lossless transport
  • The FCoE forwarding class for the DCBX application protocol type, length, value (TLV) exchange
  • A trusted port connecting to the FCoE forwarder (FCF)
  • Enlarged maximum transmission unit (MTU) size for handling FCoE traffic

This example shows how to configure an FCoE transit switch:

Requirements

This example uses the following hardware and software components:

  • One EX4500 switch (CEE-capable model)
  • Junos OS Release 12.1 or later for EX Series switches
  • One FCoE Node (ENode)
  • One FCoE forwarder (FCF)

Before you begin, be sure you have:

Overview and Topology

FCoE transmissions are vulnerable to address spoofing and man-in-the-middle attacks, because they are not actually sent through point-to-point links. This example describes how to configure the switch so that it provides security similar to that provided by traditional Fibre Channel (FC) networks. The switch is transparent to the ENode and the FCF, so the ENode and FCF communicate just as they would for a point-to-point link.

FIP snooping is disabled by default. You enable FIP snooping on a per-VLAN basis for VLANs that carry FCoE traffic. Ensure that a VLAN that carries FCoE traffic carries only FCoE traffic, because enabling FIP snooping denies access for all other Ethernet traffic.

This example shows how to configure FIP snooping on a VLAN of the EX4500 switch that is connected with one ENode, that is, a server equipped with converged network adapters (CNAs). The setup for this example includes the VLAN fcoe-vlan on the switch.

This example also shows how to configure PFC on the interfaces that are being used for FCoE traffic and how to configure an FCoE trusted port to handle traffic between the switch and the FCF gateway to the storage area network (SAN).

You must configure PFC properties for the interfaces that are carrying FCoE traffic, because flow control must be implemented on the link level for this type of traffic.

Note: Data Center Bridging Capability Exchange protocol (DCBX) is enabled by default on all 10-Gigabit Ethernet interfaces on EX4500 switches. DCBX automatically controls whether PFC is enabled or disabled on the interface. However, you must configure the PFC properties selecting the traffic class and queue. See Configuring Priority-Based Flow Control for an EX Series Switch (CLI Procedure).

You configure trunk interfaces that connect to the FCF as trusted interfaces. The switch must use the same FCoE MAC Address Prefix (FC-MAP) value that is being used by the FCF. Therefore, if the FCF is using a nondefault FC-MAP value, you must configure the FC-MAP value on the switch to match that value.

You must also enlarge the MTU size for all interfaces (both access and trunk) that are handling FCoE traffic to accommodate the maximum FC frame and Ethernet header sizes.

This example also includes configuring the fcoe forwarding class to be used for the FCoE traffic, so that it can take advantage of DCBX support for the Application Protocol TLV Exchange. See Understanding Data Center Bridging Capability Exchange Protocol for EX Series Switches for additional information.

Note: Configuring and applying PFC and a forwarding class fcoe on the DCBX interfaces automatically enables the DCBX FCoE application protocol exchange on those interfaces. Do not explicitly configure an FCoE application map, because doing that generates a commit error. See Understanding Data Center Bridging Capability Exchange Protocol for EX Series Switches for additional information.

Note: PFC is supported only on 10-Gigabit Ethernet interfaces.

Note: We recommend that you also:

  • Configure the PFC congestion notification profile for the same 802.1p code points that you are using for the fcoe forwarding class. We recommend code point 011, because this is the conventional IEEE 802.1p code point for FCoE traffic.
  • Configure at least 20 percent of the buffer for the queue that is using PFC.
  • Do not specify the exact option when configuring the buffer for the queue that is using PFC.
  • Configure the loss-priority statement to low for a traffic class that is using PFC.
  • Configure an appropriate percent of the buffer for any other forwarding classes (default forwarding classes and the user-defined forwarding classes) that you are using

The components of the topology for this example are shown in Table 1.

Table 1: Components of the FCoE Security Topology

PropertiesSettings

Switch hardware

One EX4500 CEE-enabled switch

VLAN name and ID

fcoe-vlan, tag 20

Forwarding class for FCoE traffic

fcoe, code point 011

Interfaces in fcoe-vlan

xe-0/0/1
xe-0/0/2
xe-0/0/3
xe-0/0/30

FCoE trusted port to the FCF

xe-0/0/30

PFC interfaces

xe-0/0/1
xe-0/0/2
xe-0/0/3
xe-0/0/30

CoS forwarding-class interface

xe-0/0/30

CoS scheduler-map interface

xe-0/0/30

Interfaces configured with MTU of 2500

xe-0/0/1

xe-0/0/2

xe-0/0/3

xe-0/0/30

In this example, the switch has already been configured as follows:

  • All access ports are untrusted, which is the default setting.
  • DCBX is enabled by default on all 10-Gigabit Ethernet interfaces.
  • The port connecting the switch to the FCF is configured as a trunk port.

Configuration

To configure an FCoE transit switch, perform these tasks:

CLI Quick Configuration

To quickly configure an FCoE transit switch, copy the following commands and paste them into the switch terminal window:

[edit]
set ethernet-switching-options secure-access-port vlan fcoe-vlan examine-fip fc-map 0x0EFC03
set ethernet-switching-options secure-access-port interface xe-0/0/30 fcoe-trusted
set interfaces xe-0/0/1 ether-options no-flow-control
set interfaces xe-0/0/2 ether-options no-flow-control
set interfaces xe-0/0/3 ether-options no-flow-control
set interfaces xe-0/0/30 ether-options no-flow-control
set class-of-service congestion-notification-profile cn-profile input ieee-802.1 code-point 011 pfc
set class-of-service interfaces xe-0/0/1 congestion-notification-profile cn-profile
set class-of-service interfaces xe-0/0/2 congestion-notification-profile cn-profile
set class-of-service interfaces xe-0/0/3 congestion-notification-profile cn-profile
set class-of-service interfaces xe-0/0/30 congestion-notification-profile cn-profile
set class-of-service classifiers ieee-802.1 pfc-class import default
set class-of-service classifiers ieee-802.1 pfc-class forwarding-class fcoe loss-priority low code-points 011
set class-of-service interfaces xe-0/0/1 unit 0 classifiers ieee-802.1 pfc-class
set class-of-service interfaces xe-0/0/2 unit 0 classifiers ieee-802.1 pfc-class
set class-of-service interfaces xe-0/0/3 unit 0 classifiers ieee-802.1 pfc-class
set class-of-service interfaces xe-0/0/30 unit 0 classifiers ieee-802.1 pfc-class
set class-of-service forwarding-classes class fcoe queue-num 3
set class-of-service schedulers pfc-sched buffer-size percent 25
set class-of-service schedulers default-sched buffer-size percent 17
set class-of-service scheduler-maps pfc-map forwarding-class fcoe scheduler pfc-sched
set class-of-service scheduler-maps pfc-map forwarding-class assured-forwarding scheduler default-sched
set class-of-service scheduler-maps pfc-map forwarding-class best-effort scheduler default-sched
set class-of-service scheduler-maps pfc-map forwarding-class network-control scheduler default-sched
set class-of-service scheduler-maps pfc-map forwarding-class expedited-forwarding scheduler default-sched
set class-of-service interfaces xe-0/0/30 scheduler-map pfc-map
set interfaces xe-0/0/1 mtu 2500
set interfaces xe-0/0/2 mtu 2500
set interfaces xe-0/0/3 mtu 2500
set interfaces xe-0/0/30 mtu 2500

Step-by-Step Procedure

To configure an FCoE transit switch:

  1. Enable FIP snooping on the VLAN and modify the FC-MAP value to match the FC-MAP value being used by the FCF:
    [edit ethernet-switching-options secure-access-port]
    user@switch# set vlan fcoe-vlan examine-fip fc-map 0x0EFC03
  2. Set the FCF-facing interface (xe-0/0/30) as FCoE-trusted:
    [edit ethernet-switching-options secure-access-port]
    user@switch# set interface xe-0/0/30 fcoe-trusted
  3. Configure a congestion notification profile, specifying the name of the profile and applying it to the traffic class that is indicated by the User Priority bits in the 802.1Q tagged frame of an incoming packet:

    Note: The ENode and the switch must use the same traffic class for the FCoE traffic. DCBX advertises the traffic class being used by the switch and detects the traffic class being used by the ENode. If there is a mismatch, the switch disables the PFC capability of the switch interface.

    [edit class-of-service]
    user@switch# set congestion-notification-profile cn-profile input ieee-802.1 code-point 011 pfc

    Note: The configuration of PFC includes two different ieee-802.1 configuration statements:

  4. Disable standard flow control on the interfaces that you want to use for the FCoE VLAN.

    Note: PFC and standard flow control cannot be enabled on the same interface, and you must use PFC for FCoE traffic.

    [edit interfaces]
    user@switch# set xe-0/0/1 ether-options no-flow-control
    user@switch# set xe-0/0/2 ether-options no-flow-control
    user@switch# set xe-0/0/3 ether-options no-flow-control
    user@switch# set xe-0/0/30 ether-options no-flow-control
  5. Bind the congestion notification profile to all interfaces of the FCoE VLAN:
    [edit class-of-service]
    user@switch# set interface xe-0/0/1 congestion-notification-profile cn-profile
    user@switch# set interface xe-0/0/2 congestion-notification-profile cn-profile
    user@switch# set interface xe-0/0/3 congestion-notification-profile cn-profile
    user@switch# set interface xe-0/0/30 congestion-notification-profile cn-profile
  6. Create a CoS classifier for the fcoe forwarding class:
    [edit class-of-service]
    user@switch# set forwarding-classes fcoe queue-num 3
  7. Configure this forwarding class (fcoe) to use a low loss priority value and to use the same code point that is used for PFC:

    Note: We recommend that you use code point 011, because this is the conventional IEEE 802.1p code point for FCoE traffic.

    [edit class-of-service]
    user@switch# set classifiers ieee-802.1 pfc-class forwarding-class fcoe loss-priority low code-points 011
  8. Bind the pfc-class classifier to all interfaces of the FCoE VLAN:
    [edit class-of-service]
    user@switch# set interfaces xe-0/0/1 unit 0 classifiers ieee-802.1 pfc-class
    user@switch# set interfaces xe-0/0/2 unit 0 classifiers ieee-802.1 pfc-class
    user@switch# set interfaces xe-0/0/3 unit 0 classifiers ieee-802.1 pfc-class
    user@switch# set interfaces xe-0/0/30 unit 0 classifiers ieee-802.1 pfc-class
  9. Assign forwarding-class fcoe to an egress queue:
    [edit class-of-service]
    user@switch# set forwarding-classes fcoe queue-num 3
  10. Set a scheduler for this queue, allocating at least 20 percent of the buffer to pfc-sched:
    [edit class-of-service]
    user@switch# set schedulers pfc-sched buffer-size percent 25
  11. Set a scheduler for the default queue, allocating 17 percent of the buffer to that queue:
    [edit class-of-service]
    uuser@switch# set schedulers default-sched buffer-size percent 17
  12. Configure a scheduler map (pfc-map) that associates the scheduler (pfc-sched) with the fcoe forwarding class and associates the default forwarding classes (assured-forwarding, best-effort and network-control) with the default schedule:
    [edit class-of-service]
    user@switch# set scheduler-maps pfc-map forwarding-class fcoe scheduler pfc-sched
    user@switch# set scheduler-maps pfc-map forwarding-class assured-forwarding schedulerdefault-sched
    user@switch# set scheduler-maps pfc-map forwarding-class best-effort scheduler default-sched
    user@switch# set scheduler-maps pfc-map forwarding-class network-control scheduler default-sched
    user@switch# set scheduler-maps pfc-map forwarding-class expedited-forwarding scheduler default-sched
  13. Assign the scheduler map (pfc-map) to the FCF-facing interface (xe-0/0/30):
    [edit class-of-service]
    user@switch# set interfaces xe-0/0/30 scheduler-map pfc-map
  14. Enlarge the MTU size to 2500 bytes for all the interfaces (both access and trunk) that are handling FCoE traffic:
    [edit interfaces]
    user@switch# set xe-0/0/1 mtu 2500
    user@switch# set xe-0/0/2 mtu 2500
    user@switch# set xe-0/0/3 mtu 2500
    user@switch# set xe-0/0/30 mtu 2500

Results

Display the results of the configuration:

[edit]
user@switch#show
interfaces {xe-0/0/1 {mtu 2500;ether-options {no-flow-control;}unit 0 {family ethernet-switching {vlan {members fcoe-vlan;}}}}xe-0/0/2 {mtu 2500;ether-options {no-flow-control;}unit 0 {family ethernet-switching {vlan {members fcoe-vlan;}}}}xe-0/0/3 {mtu 2500;ether-options {no-flow-control;}unit 0 {family ethernet-switching {vlan {members fcoe-vlan;}}}}xe-0/0/30 {mtu 2500;ether-options {no-flow-control;}unit 0 {family ethernet-switching {port-mode trunk;vlan {members fcoe-vlan;}}}}}class-of-service {classifiers {ieee-802.1 pfc-class {import default;forwarding-class fcoe {loss-priority low code-points 011;}forwarding-classes {class fcoe queue-num 3;}congestion-notification-profile {cn-profile {input {ieee-802.1 {code-point 011 {pfc;}}}}}interfaces {xe-0/0/1 {congestion-notification-profile cn-profile;unit 0 {classifiers {ieee-802.1 pfc-class;}}}xe-0/0/2 {congestion-notification-profile cn-profile;unit 0 {classifiers {ieee-802.1 pfc-class;}}xe-0/0/3 {congestion-notification-profile cn-profile;unit 0 {classifiers {ieee-802.1 pfc-class;}}}xe-0/0/30 {congestion-notification-profile cn-profile;scheduler-map pfc-map;unit 0 {classifiers {ieee-802.1 pfc-class;}}}scheduler-maps {pfc-map {forwarding-class fcoe scheduler pfc-sched;forwarding-class assured-forwarding scheduler default-sched;forwarding-class best-effort scheduler default-sched;forwarding-class network-control scheduler default-sched;forwarding-class expedited-forwarding scheduler default-sched;}}schedulers {pfc-sched {buffer-size percent 25;}default-sched {buffer-size percent 17;}}}}ethernet-switching-options {secure-access-port {interface xe-0/0/30.0 {fcoe-trusted;}vlan fcoe-vlan {examine-fip {fc-map 0x0EFC03;}}}}

Verification

Confirm that the configuration of the FCoE transit switch is working properly:

Verifying That FIP Snooping Is Working Correctly on the Switch

Purpose

Verify that FIP snooping is being implemented on the appropriate VLAN.

Action

Send some requests from ENodes to the switch.

Display the FIP snooping information :

user@switch> show fip snooping vlan detail fcoe-vlan
VLAN: fcoe-vlan,   FC-MAP: 0e:fc:03
  FCF Information
  FCF-MAC            : 30:10:94:01:00:00
  Active Sessions    : 2
  Configured FKA-ADV : 195
  Running FKA-ADV    : 73
    Enode Information
    Enode-MAC: 10:10:94:01:00:01,       Interface: xe-0/0/1
    Configured FKA-ADV : 195
    Running FKA-ADV    : 103
      Session Information
      VN-Port MAC: 0E:FC:03:01:0A:01,   FKA-ADV : 178
      VN-Port MAC: 0E:FC:03:01:0B:01,   FKA-ADV : 194
  FCF Information
  FCF-MAC            : 40:10:94:01:00:00
  Active Sessions    : 2
  Configured FKA-ADV : 258
  Running FKA-ADV    : 212
    Enode Information
    Enode-MAC: 20:10:94:01:00:02,       Interface: xe-0/0/0
    Configured FKA-ADV : 258
    Running FKA-ADV    : 242
      Session Information
      VN-Port MAC: 0E:FC:03:02:0C:02,   FKA-ADV : 254
      VN-Port MAC: 0E:FC:03:02:0D:02,   FKA-ADV : 269

Meaning

The output for this VLAN (fcoe-vlan) includes the FC MAP value that you configured. It shows the MAC addresses of the FCF and the ENode that are transmitting FCoE traffic through the switch.

Verifying That PFC is Enabled, That the FCoE Application Is Advertised, and That the Switch Interface and DCB Peer Are Using the Same 802.1p Code Points

Purpose

Verify that PFC is enabled on the local switch interface and on the peer interface, and that the local interface and the peer interface are using the same code point.

Action

Send some requests from ENodes to the switch.

Display the DCBX information advertised by the configured CoS forwarding class interface (xe-0/0/30) and detected by the switch:

user@switch> show dcbx neighbors interface xe-0/0/30
Interface : xe-0/0/30.0

    Protocol-State: in-sync
 
    Local-Advertisement:
        Operational version: 0
        sequence-number: 1, acknowledge-id: 1
 
    Peer-Advertisement:
        Operational version: 0
        sequence-number: 1, acknowledge-id: 1
 
    Feature: PFC, Protocol-State: in-sync

      Operational State: Enabled

      Local-Advertisement:
        Enable: Yes, Willing: No, Error: No
        Maximum Traffic Classes capable to support PFC: 6

        Code Point          Admin Mode
          000                 Disabled 
          001                 Disabled 
          010                 Disabled 
          011                 Disabled  
          100                 Disabled 
          011                 Enabled
          110                 Disabled 
          111                 Disabled 

 

      Peer-Advertisement:
        Enable: Yes, Willing: No, Error: No
        Maximum Traffic Classes capable to support PFC: 6
 

        Code Point          Admin Mode
          000                 Disabled 
          001                 Disabled 
          010                 Disabled 
          011                 Disabled 
          100                 Disabled 
          011                 Enabled 
          110                 Disabled 
          111                 Disabled

Feature: Application, Protocol-State: in-sync

      Local-Advertisement:

        Enable: Yes, Willing: No, Error: No

 

        Appl-Name    Ethernet-Type    Socket-Number    Priority-Map    Status

          FCoE         0x8906                            00001000       Enabled


      Peer-Advertisement:

        Enable: Yes, Willing: No, Error: No

 

        Appl-Name    Ethernet-Type    Socket-Number    Priority-Map    Status

          FCoE         0x8906                            00001000        Enabled

          

Meaning

PFC is a requirement for transmitting FCoE traffic and PFC works only when the local and peer devices are both enabled for PFC and are both using the same traffic class (code point) for transmitting the PFC traffic.

In the output for Feature: PFC, check the status of Local-Advertisement to verify that PFC is enabled. If DCBX detects a misconfiguration with the DCB peer, it disables the PFC capability. In this example, the PFC Operational State is enabled, because PFC is configured symmetrically on the switch and the DCB peer. Both devices are using code point 011 for forwarding the traffic.

If the results show that PFC is disabled, you van use the information provided by this command to reconfigure the congestion notification profile to match the code point being used for PFC by the peer device. See Configuring Priority-Based Flow Control for an EX Series Switch (CLI Procedure).

Appl-Name shows the default FCoE application. The FCoE application always indicates Ethernet-Type 0x8906. The Priority-Map for the FCoE application shows the 8-bit format of the code-point setting that was specified for the PFC congestion notification profile. In this case, the three bit code point is 3, 011. So the Priority-Map for the default FCoe application is 00001000.

The fcoe forwarding-class and PFC were configured; and the configuration of the application on the switch and on the DCB are synchronized. Therefore, the Status of the FCoE application is Enabled.

If the configuration of the FCoE application on the switch did not match the FCoE application of the DCB peer, the status of the application would appear as Disabled.

Published: 2012-12-07