ON THIS PAGE
Example: Setting Up Fibre Channel and FCoE VLAN Interfaces in an FCoE-FC Gateway Fabric
To transmit Fibre Channel (FC) traffic between FCoE devices and a storage area network (SAN) FC switch, you configure a local FC fabric on the gateway. The gateway FC fabric includes FCoE and native FC interfaces, and a VLAN to carry FCoE traffic from FCoE-capable devices. The gateway FC fabric creates the path between the FCoE devices and the SAN.
This example describes how to configure the interfaces, VLAN, and FC fabric to connect FCoE devices to the FC switch and route traffic between the VLAN and FC interfaces:
Requirements
This example uses the following hardware and software components:
A configured and provisioned Juniper Networks QFX3500 Switch to act as an FCoE-FC gateway
FCoE-capable devices in an Ethernet network equipped with converged network adapters (CNAs)
An FC switch to transmit and receive native FC traffic
FC storage devices in the SAN
Junos OS Release 11.1 or later for the QFX Series
Overview
No interfaces are configured for FC network connectivity by default. You need to configure the FC fabric and its interfaces explicitly. Each FC fabric consists of a combination of at least one FCoE VLAN interface between the FCoE-FC gateway and the FCoE devices, and one or more native FC interfaces between the FCoE-FC gateway and the FC switch.
An FCoE VLAN interface connects the FCoE-FC gateway to FCoE devices. FCoE traffic between the devices and the FCoE-FC gateway requires a dedicated VLAN used only for FCoE traffic. You cannot mix standard Ethernet traffic and FCoE traffic on the FCoE VLAN.
IGMP snooping is not supported on FCoE VLANs. IGMP snooping is enabled by default on all VLANs in all software versions before Junos OS Release 13.2. Disable IGMP snooping on FCoE VLANs if you are using software that is older than 13.2.
Storm control is not supported on Ethernet interfaces that belong to the FCoE VLAN. Ensure that storm control is disabled on all Ethernet interfaces that belong to the FCoE VLAN to prevent FCoE traffic from being dropped.
When FCoE frames enter the FCoE-FC gateway, the gateway:
Strips the Ethernet encapsulation from the FCoE frames.
Sends the resulting native FC frames to the FC switch through the gateway’s native FC interfaces.
Each FC interface and FCoE VLAN interface can belong to only one FC fabric. Different FC fabrics must use different native FC interfaces and different FCoE VLAN interfaces. Multiple FC fabrics on the FCoE-FC gateway can connect to the same FC switch, but they must use different FC interfaces and different FCoE VLAN interfaces.
The Ethernet interfaces that belong to the FCoE VLAN should be configured in tagged-access port mode and must include the native VLAN because FIP VLAN discovery and notification frames are exchanged as untagged packets. These Ethernet interfaces require a maximum transmission unit (MTU) size of at least 2180 bytes to accommodate the FC payload and FCoE encapsulation. (Sometimes the MTU is rounded up to 2500 bytes. If larger frames are expected on the interface, set the MTU size accordingly.)
This example shows a simple configuration to illustrate the basic steps for creating:
The FCoE-device-facing VLAN and its 10-Gigabit Ethernet interfaces
The VLAN interface
The FC-switch-facing native FC interfaces
One FC fabric on the FCoE-FC gateway
Configuring these elements results in traffic being routed between the VLAN and FC interfaces, thus connecting the FCoE devices to the FC switch through the FCoE-FC gateway.
A VLAN called blue
transports FCoE traffic between
FCoE devices and the FCoE-FC gateway using an FCoE VLAN interface
called vlan.100
. The FCoE-FC gateway’s vlan.100
interface presents an F_Port interface to the FCoE devices on the
VLAN. For each FCoE device ENode that logs in to the FCoE-FC gateway,
the gateway instantiates a virtual F_Port (VF_Port) interface. This
creates a virtual link between the ENode VN_Port and the FCoE-FC gateway.
The FCoE-FC gateway’s native FC interfaces transport FC traffic
between the gateway and the FC switch.
Configuring both the FCoE VLAN interface and the native FC interfaces as part of a gateway fabric associates them in the switch and makes the connection between the FCoE servers and the FC switch.
Topology
The topology for this example consists of one QFX3500 switch with FC-capable ports to connect to the FC switch and with Ethernet ports in tagged-access mode to connect to the FCoE devices. Table 1 and Figure 1 show the configuration components of this example.
Property |
Settings |
---|---|
Switch hardware |
QFX3500 switch in gateway mode |
FCoE VLAN name and tag ID |
IGMP snooping disabled on the FCoE VLAN. |
Interfaces in VLAN |
Interfaces: Note:
FCoE VLANs (any VLAN that carries FCoE traffic) support only Spanning Tree Protocol (STP) and link aggregation group (LAG) Layer 2 features. FCoE traffic cannot use a standard LAG because traffic might be hashed to different physical LAG links on different transmissions. This breaks the (virtual) point-to-point link that Fibre Channel traffic requires. If you configure a standard LAG interface for FCoE traffic, FCoE traffic might be rejected by the FC SAN. QFabric systems support a special LAG called an FCoE LAG, which enables you to transport FCoE traffic and regular Ethernet traffic across the same link aggregation bundle. An FCoE LAG ensures that FCoE traffic uses the same physical link in the LAG for requests and replies in order to preserve the virtual point-to-point link between the FCoE device converged network adapter (CNA) and the FC SAN switch across the QFabric system Node device. An FCoE LAG does not provide load balancing or link redundancy for FCoE traffic. However, regular Ethernet traffic receives the usual LAG benefits of load balancing and link redundancy in an FCoE LAG. |
FCoE VLAN interface |
|
Native Fibre Channel interfaces |
Interfaces: |
Fibre Channel fabric |
Fabric type: |
This configuration example creates a VLAN for FCoE traffic and routes its traffic to an FCoE VLAN interface that is part of the FC fabric. It also creates the FC interfaces needed to connect to the FC switch.
To set up FC interfaces and FCoE VLAN interfaces:
Configure a VLAN to use as a dedicated FCoE VLAN:
Configure the interfaces the FCoE VLAN uses as Ethernet switching interfaces in tagged-access port mode.
If storm control is enabled, disable it on the interfaces.
Configure the interfaces the FCoE VLAN uses with the native VLAN.
Configure the FCoE VLAN to use the desired Ethernet interfaces.
Disable IGMP snooping on the FCoE VLAN. (Before Junos OS Release 13.2, IGMP snooping was enabled by default on all VLANs, but is not supported on FCoE VLANs. Starting with Junos OS Release 13.2, IGMP snooping is enabled by default only on the default VLAN.)
Configure the FCoE VLAN interface.
Define the interface for the FCoE VLAN (associate the VLAN with the FCoE VLAN interface).
Configure the physical FC interfaces (either one or two 6-port blocks) that connect to the FC switch.
Configure the logical FC interfaces that connect to the FC switch.
Configure the FCoE-FC gateway fabric:
Configure the fabric ID.
Configure the fabric as a proxy fabric.
Add the FCoE VLAN interface and the native FC interfaces to the fabric.
To keep the example simple, the configuration steps show six Ethernet interfaces in the FCoE VLAN and six native FC interfaces in the FC fabric. Use the same configuration procedure to add more interfaces to the FCoE VLAN or to the FC fabric.
Configuration
Procedure
CLI Quick Configuration
To quickly configure FCoE and native FC interfaces on an FCoE-FC gateway and route traffic between the FCoE VLAN and FC interfaces, copy the following commands and paste them into the switch terminal window:
[edit] set vlans blue vlan-id 100 set vlans native vlan-id 1 set interfaces xe-0/0/6 unit 0 family ethernet-switching port-mode tagged-access vlan members blue set interfaces xe-0/0/7 unit 0 family ethernet-switching port-mode tagged-access vlan members blue set interfaces xe-0/0/8 unit 0 family ethernet-switching port-mode tagged-access vlan members blue set interfaces xe-0/0/9 unit 0 family ethernet-switching port-mode tagged-access vlan members blue set interfaces xe-0/0/10 unit 0 family ethernet-switching port-mode tagged-access vlan members blue set interfaces xe-0/0/11 unit 0 family ethernet-switching port-mode tagged-access vlan members blue set interfaces xe-0/0/6 unit 0 family ethernet-switching native-vlan-id 1 set interfaces xe-0/0/7 unit 0 family ethernet-switching native-vlan-id 1 set interfaces xe-0/0/8 unit 0 family ethernet-switching native-vlan-id 1 set interfaces xe-0/0/9 unit 0 family ethernet-switching native-vlan-id 1 set interfaces xe-0/0/10 unit 0 family ethernet-switching native-vlan-id 1 set interfaces xe-0/0/11 unit 0 family ethernet-switching native-vlan-id 1 set interfaces xe-0/0/6 mtu 2180 set interfaces xe-0/0/7 mtu 2180 set interfaces xe-0/0/8 mtu 2180 set interfaces xe-0/0/9 mtu 2180 set interfaces xe-0/0/10 mtu 2180 set interfaces xe-0/0/11 mtu 2180 set vlans blue interface xe-0/0/6.0 set vlans blue interface xe-0/0/7.0 set vlans blue interface xe-0/0/8.0 set vlans blue interface xe-0/0/9.0 set vlans blue interface xe-0/0/10.0 set vlans blue interface xe-0/0/11.0 set protocols igmp-snooping vlan blue disable set interfaces vlan unit 100 family fibre-channel port-mode f-port set vlans blue l3-interface vlan.100 set chassis fpc 0 pic 0 fibre-channel port-range 0 5 set interfaces fc-0/0/0 unit 0 family fibre-channel port-mode np-port set interfaces fc-0/0/1 unit 0 family fibre-channel port-mode np-port set interfaces fc-0/0/2 unit 0 family fibre-channel port-mode np-port set interfaces fc-0/0/3 unit 0 family fibre-channel port-mode np-port set interfaces fc-0/0/4 unit 0 family fibre-channel port-mode np-port set interfaces fc-0/0/5 unit 0 family fibre-channel port-mode np-port set interfaces fc-0/0/0 fibrechannel-options speed 4g set interfaces fc-0/0/1 fibrechannel-options speed 4g set interfaces fc-0/0/2 fibrechannel-options speed 4g set interfaces fc-0/0/3 fibrechannel-options speed 4g set interfaces fc-0/0/4 fibrechannel-options speed 4g set interfaces fc-0/0/5 fibrechannel-options speed 4g set fc-fabrics fcproxy1 fabric-id 1 set fc-fabrics fcproxy1 fabric-type proxy set fc-fabrics fcproxy1 interface vlan.100 set fc-fabrics fcproxy1 interface fc-0/0/0.0 set fc-fabrics fcproxy1 interface fc-0/0/1.0 set fc-fabrics fcproxy1 interface fc-0/0/2.0 set fc-fabrics fcproxy1 interface fc-0/0/3.0 set fc-fabrics fcproxy1 interface fc-0/0/4.0 set fc-fabrics fcproxy1 interface fc-0/0/5.0
Step-by-Step Procedure
Configure FCoE and FC interfaces in an FCoE-FC gateway FC fabric and set up traffic routing between the FCoE VLAN and FC interfaces:
Configure the VLAN for FCoE traffic:
[edit vlans] user@switch# set blue vlan-id 100
Configure the native VLAN:
[edit vlans] user@switch# set native vlan-id 1
Configure the Ethernet interfaces for the FCoE VLAN in tagged-access mode and as members of the FCoE VLAN (VLAN
blue
):[edit interfaces] user@switch# set xe-0/0/6 unit 0 family ethernet-switching port-mode tagged-access vlan members blue user@switch# set xe-0/0/7 unit 0 family ethernet-switching port-mode tagged-access vlan members blue user@switch# set xe-0/0/8 unit 0 family ethernet-switching port-mode tagged-access vlan members blue user@switch# set xe-0/0/9 unit 0 family ethernet-switching port-mode tagged-access vlan members blue user@switch# set xe-0/0/10 unit 0 family ethernet-switching port-mode tagged-access vlan members blue user@switch# set xe-0/0/11 unit 0 family ethernet-switching port-mode tagged-access vlan members blue
Configure the native VLAN on the Ethernet interfaces in the FCoE VLAN:
[edit interfaces] user@switch# set xe-0/0/6 unit 0 family ethernet-switching native-vlan-id 1 user@switch# set xe-0/0/7 unit 0 family ethernet-switching native-vlan-id 1 user@switch# set xe-0/0/8 unit 0 family ethernet-switching native-vlan-id 1 user@switch# set xe-0/0/9 unit 0 family ethernet-switching native-vlan-id 1 user@switch# set xe-0/0/10 unit 0 family ethernet-switching native-vlan-id 1 user@switch# set xe-0/0/11 unit 0 family ethernet-switching native-vlan-id 1
Set the MTU to 2180 for each Ethernet interface:
[edit interfaces] user@switch# set xe-0/0/6 mtu 2180 user@switch# set xe-0/0/7 mtu 2180 user@switch# set xe-0/0/8 mtu 2180 user@switch# set xe-0/0/9 mtu 2180 user@switch# set xe-0/0/10 mtu 2180 user@switch# set xe-0/0/11 mtu 2180
Assign the Ethernet interfaces to the FCoE VLAN:
[edit vlans blue interface] user@switch# set xe-0/0/6.0 user@switch# set xe-0/0/7.0 user@switch# set xe-0/0/8.0 user@switch# set xe-0/0/9.0 user@switch# set xe-0/0/10.0 user@switch# set xe-0/0/11.0
Disable IGMP snooping on the FCoE VLAN:
[edit protocols] user@switch# set igmp-snooping vlan blue disable
Configure the FCoE VLAN interface and port mode for the FCoE traffic:
[edit interfaces] user@switch# set vlan unit 100 family fibre-channel port-mode f-port
Define the FCoE VLAN interface as the interface for the FCoE VLAN:
[edit vlans] user@switch# set blue l3-interface vlan.100
Configure the physical FC interfaces the fabric uses to connect to the FC switch:
[edit chassis fpc 0 pic 0] user@switch# set fibre-channel port-range 0 5
Note:When you configure ports as FC ports, the port designation changes from
xe-n/n/n.n
format tofc-n/n/n.n
format to indicate that the interface is an FC interface. FC interfaces do not support 10-Gbps interface speed but instead conform to FC interface speeds of 2 Gbps, 4 Gbps, or 8 Gbps.Configure the native FC interfaces and port mode:
[edit interfaces] user@switch# set fc-0/0/0 unit 0 family fibre-channel port-mode np-port user@switch# set fc-0/0/1 unit 0 family fibre-channel port-mode np-port user@switch# set fc-0/0/2 unit 0 family fibre-channel port-mode np-port user@switch# set fc-0/0/3 unit 0 family fibre-channel port-mode np-port user@switch# set fc-0/0/4 unit 0 family fibre-channel port-mode np-port user@switch# set fc-0/0/5 unit 0 family fibre-channel port-mode np-port
Configure the native FC interface port speed:
[edit interfaces] user@switch# set fc-0/0/0 fibrechannel-options speed 4g user@switch# set fc-0/0/1 fibrechannel-options speed 4g user@switch# set fc-0/0/2 fibrechannel-options speed 4g user@switch# set fc-0/0/3 fibrechannel-options speed 4g user@switch# set fc-0/0/4 fibrechannel-options speed 4g user@switch# set fc-0/0/5 fibrechannel-options speed 4g
Configure the FC fabric name and unique ID:
[edit fc-fabrics] user@switch# set fcproxy1 fabric-id 1
Define the FC fabric as an FCoE-FC gateway:
[edit fc-fabrics] user@switch# set fcproxy1 fabric-type proxy
Assign the FCoE VLAN interface to the fabric:
[edit fc-fabrics] user@switch# set fcproxy1 interface vlan.100
Assign the native FC interfaces to the fabric:
[edit fc-fabrics] user@switch# set fcproxy1 interface fc-0/0/0.0 user@switch# set fcproxy1 interface fc-0/0/1.0 user@switch# set fcproxy1 interface fc-0/0/2.0 user@switch# set fcproxy1 interface fc-0/0/3.0 user@switch# set fcproxy1 interface fc-0/0/4.0 user@switch# set fcproxy1 interface fc-0/0/5.0
Results
Display the results of the configuration:
user@switch> show configuration fc-0/0/0 { fibrechannel-options { speed 4g; } unit 0 { family fibre-channel { port-mode np-port; } } } fc-0/0/1 { fibrechannel-options { speed 4g; } unit 0 { family fibre-channel { port-mode np-port; } } } fc-0/0/2 { fibrechannel-options { speed 4g; } unit 0 { family fibre-channel { port-mode np-port; } } } fc-0/0/3 { fibrechannel-options { speed 4g; } unit 0 { family fibre-channel { port-mode np-port; } } } fc-0/0/4 { fibrechannel-options { speed 4g; } unit 0 { family fibre-channel { port-mode np-port; } } } fc-0/0/5 { fibrechannel-options { speed 4g; } unit 0 { family fibre-channel { port-mode np-port; } } } xe-0/0/6 { mtu 2180; unit 0 { family ethernet-switching { port-mode tagged-access; vlan { members blue; } native-vlan-id 1; } } } xe-0/0/7 { mtu 2180; unit 0 { family ethernet-switching { port-mode tagged-access; vlan { members blue; } native-vlan-id 1; } } } xe-0/0/8 { mtu 2180; unit 0 { family ethernet-switching { port-mode tagged-access; vlan { members blue; } native-vlan-id 1; } } } xe-0/0/9 { mtu 2180; unit 0 { family ethernet-switching { port-mode tagged-access; vlan { members blue; } native-vlan-id 1; } } } xe-0/0/10 { mtu 2180; unit 0 { family ethernet-switching { port-mode tagged-access; vlan { members blue; } native-vlan-id 1; } } } xe-0/0/11 { mtu 2180; unit 0 { family ethernet-switching { port-mode tagged-access; vlan { members blue; } native-vlan-id 1; } } } vlan { unit 100 { family fibre-channel { port-mode f-port; } } } fc-fabrics { fcproxy1 { fabric-id 1 fabric-type proxy interface { vlan.100 fc-0/0/0.0; fc-0/0/1.0; fc-0/0/2.0; fc-0/0/3.0; fc-0/0/4.0; fc-0/0/5.0; } } } protocols { igmp-snooping { vlan blue { disable; } } } vlans { blue { vlan-id 100 interface { xe-0/0/6.0; xe-0/0/7.0; xe-0/0/8.0; xe-0/0/9.0; xe-0/0/10.0; xe-0/0/11.0; } l3-interface vlan.100 } native { vlan-id 1; } }
To quickly configure the interfaces, issue the load
merge terminal
command and then copy the hierarchy and paste
it into the switch terminal window.
Verification
To verify that the native FC interfaces and FCoE VLAN interface have been created, added to the FC fabric, and are operating properly, perform these tasks:
- Verifying That the Native FC Interfaces and the FCoE VLAN Interface Have Been Created
- Verifying That the FCoE VLAN Includes the Correct Ethernet Interfaces
- Verifying That the FC Fabric Includes the Correct Interfaces
- Verifying Native FC Interface Operation
- Verifying That IGMP Snooping Has Been Disabled on the FCoE VLAN
Verifying That the Native FC Interfaces and the FCoE VLAN Interface Have Been Created
Purpose
Verify that the six native FC interfaces and the FCoE VLAN interface have been created on the switch and are configured in the correct mode.
Action
List all of the FC interfaces configured on the switch
using the show fibre-channel interfaces
command:
user@switch> show fibre-channel interfaces Native Config Oper Interface Idx Type Fabric-id NPIV Mode Mode State fc-0/0/0.0 70 FC 1 YES NP NP up fc-0/0/1.0 71 FC 1 YES NP NP up fc-0/0/2.0 72 FC 1 YES NP NP up fc-0/0/3.0 73 FC 1 YES NP NP up fc-0/0/4.0 74 FC 1 YES NP NP up fc-0/0/5.0 75 FC 1 YES NP NP up vlan.100 67 FCOE 1 YES F F up
Meaning
The show fibre-channel interfaces
command
lists all native FC interfaces and FCoE VLAN interfaces configured
on the switch. The command output shows that the FC interfaces fc-0/0/0.0
, fc-0/0/1.0
, fc-0/0/2.0
, fc-0/0/3.0
, fc-0/0/4.0
, and fc-0/0/5.0
have been created and that those six interfaces:
Are native Fibre Channel interfaces (type
FC
).Belong to the FC fabric with a configured fabric ID of
1
.Are capable of N_Port ID virtualization (NPIV).
Have a configured mode and an operational mode of proxy N_Port (
NP
), which means that they should be connected to an FCF or an FC switch, not to an FCoE device, and that they carry native FC traffic.Show an operational state of
up
.
The command output also shows that the FCoE VLAN interface vlan.100
has been created and that interface:
Is an FCoE VLAN interface (type
FCOE)
.Belongs to the FC fabric with a configured fabric ID of
1
.Is capable of N_Port ID virtualization (NPIV).
Has a configured mode and an operational mode of F_Port (
F
), which means that its interfaces connect to FCoE devices and carry FCoE traffic.Shows an operational state of
up
.
Verifying That the FCoE VLAN Includes the Correct Ethernet Interfaces
Purpose
Verify that the FCoE VLAN blue
has been
created with the correct VLAN tag (100
) and with the correct
Ethernet interfaces.
Action
List all of the interfaces configured on the switch
in VLAN blue
using the show vlans
command:
user@switch> show vlans blue Name Tag Interfaces blue 100 xe-0/0/6.0, xe-0/0/7.0, xe-0/0/8.0, xe-0/0/9.0, xe-0/0/10.0 xe-0/0/11.0
Meaning
The show vlans blue
command lists the interfaces
that are members of the FCoE VLAN blue
. The command output
shows that the blue
VLAN has a tag ID of 100 and includes
the interfaces xe-0/0/6.0
, xe-0/0/7.0
, xe-0/0/8.0
, xe-0/0/9.0
, xe-0/0/10.0
,
and xe-0/0/11.0
.
Verifying That the FC Fabric Includes the Correct Interfaces
Purpose
Verify that the FC fabric configuration is configured on the switch with the correct native FC and FCoE VLAN interfaces.
Action
List all of the interfaces configured on FC fabrics
on the switch using the show fibre-channel fabric
command:
user@switch> show fibre-channel fabric Name Fabric-id Type Interfaces fcproxy1 1 PROXY fc-0/0/0.0 fc-0/0/1.0 fc-0/0/2.0 fc-0/0/3.0 fc-0/0/4.0 fc-0/0/5.0 vlan.100
Meaning
The show fibre-channel fabric
command lists
the interfaces that are members of each FC fabric. The command output
shows that the only fabric configured on the switch is named fcproxy1
, has a fabric-id of 1
, and is a proxy
fabric in an FCoE-FC gateway. The command output also shows that
the native FC interfaces fc-0/0/0.0
, fc-0/0/1.0
, fc-0/0/2.0
, fc-0/0/3.0
, fc-0/0/4.0
, and fc-0/0/5.0
, and the FCoE VLAN interface vlan.100
belong to fcproxy1
.
Verifying Native FC Interface Operation
Purpose
Verify that the native FC interfaces are online and display the number of FC sessions on each interface.
Action
List all of the native FC NP_Port interface states and
sessions by FC fabric using the show fibre-channel proxy np-port
command:
user@switch> show fibre-channel proxy np-port Fabric: fcproxy1, Fabric-id: 1 NP-Port State Sessions LB state LB weight fc-0/0/0.0 online 3 ON 4 fc-0/0/1.0 online 3 ON 4 fc-0/0/2.0 online 2 ON 4 fc-0/0/3.0 online 2 ON 4 fc-0/0/4.0 online 2 ON 4 fc-0/0/5.0 online 2 ON 4
Meaning
The show fibre-channel proxy np-port
command
lists the interfaces that are configured as native FC proxy N_Port
interfaces. The command output shows:
The fabric name is
fcproxy1
and its fabric ID is1
.The interfaces are
online
.The number of FC sessions (virtual links) running on each interface.
The load-balancing (LB) state is
ON
for all of the interfaces.The LB weight reflects the port speed of each interface, which is
4
Gbps.
Verifying That IGMP Snooping Has Been Disabled on the FCoE VLAN
Purpose
Verify that IGMP snooping is disabled on the FCoE VLAN.
Action
List the IGMP snooping protocol information for the
FCoE VLAN using the show configuration protocols igmp-snooping
vlan blue
command:
user@switch> show configuration protocols igmp-snooping vlan blue disable;
Meaning
The show configuration protocols igmp-snooping
vlan blue
command lists the IGMP snooping configuration for
the FCoE VLAN. The command output shows that IGMP snooping is disabled
on the FCoE VLAN.