The Cisco Nexus device can be deployed as a Fibre Channel Forwarder (FCF) as shown in the following figure.

The following rules are used to process FIP frames to avoid the FCF being used as a transit between an FCoE node (ENode) and another FCF. These rules also prevent login sessions between ENodes and FCFs in different fabrics.

• FIP solicitation and login frames received from the CNAs are processed by the FCF and are not forwarded.

• If an FCF receives solicitations and advertisements from other FCFs over an interface, the following occurs:

  • The frames are ignored and discarded if the FC-MAP value in the frame matches the value of the FCF (the FCF is in the same fabric).

  • The interface is placed in the "FCoE Isolated" state if the FC-MAP value in the FIP frame does not match that of the FCF (the FCF is in a different fabric).

CNAs cannot discover or log in to FCFs that are reachable only through a transit Cisco Nexus FCF. The Cisco Nexus device cannot perform the FCoE transit function between a CNA and another FCF due to hardware limitations.

Because the Cisco Nexus FCF cannot perform the transit FCoE function, you must design your network topology so that the active Spanning Tree Protocol (STP) path of FCoE VLANs is always over the directly connected links between the CNA and the FCF. Make sure that you configure the FCoE VLAN on the directly connected links only.

The Cisco Nexus device can be deployed as a Fibre Channel Forwarder (FCF) for remotely connected CNAs, but not as a FIP snooping bridge, as shown in the following figure.

The following rules are used to process FIP frames to avoid the FCF being used as a transit between an ENode and another FCF. These rules also prevent login sessions between ENodes and FCFs in different fabrics.

• FIP solicitation and login frames received from the CNAs are processed by the FCF and are not forwarded.

• If an FCF receives solicitations and advertisements from other FCFs over an interface, the following occurs:

  • The frames are ignored and discarded if the FC-MAP value in the frame matches the value of the FCF (the FCF is in the same fabric).

  • The interface is placed in the "FCoE Isolated" state if the FC-MAP value in the FIP frame does not match that of the FCF (the FCF is in a different fabric).

Because the Cisco Nexus FCF cannot perform the transit FCoE function, you must design your network topology so that the active STP path of FCoE VLANs is always over the directly connected links between the CNA and the FCF. Make sure that you configure the FCoE VLAN on the directly connected links only.

The following figure shows a best practices topology for an access network that is using directly connected CNAs with Cisco Nexus devices.

Follow these configuration best practices for the deployment topology in the preceding figure:

1. You must configure a unique dedicated VLAN at every converged access switch to carry traffic for each Virtual Fabric (VSAN) in the SAN (for example, VLAN 1002 for VSAN 1, VLAN 1003 for VSAN 2, and so on). If you enable Multiple Spanning Tree (MST), you must use a separate MST instance for FCoE VLANs.

2. You must configure the unified fabric (UF) links as trunk ports. Do not configure the FCoE VLAN as a native VLAN. You must configure all FCoE VLANs as members of the UF links to allow extensions for VF_Port trunking and VSAN management for the virtual Fibre Channel interfaces.

   Note	A unified wire carries both Ethernet and FCoE traffic.

3. You must configure the UF links as spanning-tree edge ports.

4. You must not configure the FCoE VLANs as members of Ethernet links that are not designated to carry FCoE traffic because you want to ensure that the scope of the STP for the FCoE VLANs is limited to UF links only.

5. If the converged access switches (in the same SAN fabric or in another) need to be connected to each other over Ethernet links for a LAN alternate path, you must explicitly configure such links to exclude all FCoE VLANs from membership. This action ensures that the scope of the STP for the FCoE VLANs is limited to UF links only.

6. You must use separate FCoE VLANs for FCoE in SAN-A and SAN-B.

The following figure shows a best practices topology for an access network using remotely connected CNAs with Cisco Nexus devices.

Follow these configuration best practices for the deployment topology in the preceding figure:

1. You must configure a unique dedicated VLAN at every converged access switch to carry traffic for each Virtual Fabric (VSAN) in the SAN (for example, VLAN 1002 for VSAN 1, VLAN 1003 for VSAN 2, and so on). If you enable MST, you must use a separate MST instance for FCoE VLANs.

2. You must configure the unified fabric (UF) links as trunk ports. Do not configure the FCoE VLAN as a native VLAN. You must configure all FCoE VLANs as members of the UF links to allow extensions for VF_Port trunking and VSAN management for the virtual Fibre Channel interfaces.

   Note	A unified fabric link carries both Ethernet and FCoE traffic.

3. You must configure the CNAs and the blade switches as spanning-tree edge ports.

4. A blade switch must connect to exactly one Cisco Nexus device converged access switch, preferably over an EtherChannel, to avoid disruption due to STP reconvergence on events such as provisioning new links or blade switches.

5. You must configure the Cisco Nexus device converged access switch with a better STP priority than the blade switches that are connected to it. This requirement allows you to create an island of FCoE VLANs where the converged access switch is the spanning-tree root and all the blade switches connected to it become downstream nodes.

6. Do not configure the FCoE VLANs as members of Ethernet links that are not designated to carry FCoE traffic because you want to ensure that the scope of the STP for the FCoE VLANs is limited to UF links only.

7. If the converged access switches and/or the blade switches need to be connected to each over Ethernet links for the purposes of LAN alternate pathing, you must explicitly configure such links to exclude all FCoE VLANs from membership. This action ensures the scope of the STP for FCoE VLANs is limited to UF links only.

8. You must use separate FCoE VLANs for FCoE in SAN-A and SAN-B.

There are four types of FCoE default policies: network QoS, output queuing, input queuing, and QoS. You can enable the FCoE default policies by enabling the FCoE feature using the feature-set fcoe command command. The default QoS ingress policy, default-fcoe-in-policy, is implicitly attached to all FC and SAN-port-channel interfaces to enable FC to FCoE traffic; this can be verified by using show interface {fc slot/port | san-port-channel <no>} all command. The default QoS policy uses CoS3 and Q1 for all FC and FCoE traffic.

To use a different queue or CoS value for FCoE traffic, create user-defined policies. The user-defined QoS ingress policy has to be created and attached explicitly to both FC and FCoE interfaces to enable traffic to use a different queue or CoS. User-defined QoS policies must be created and activated for system-wide QoS.

Note	The Ethernet or port-channel interface must be configured with MTU 9216 (or the maximum available MTU size) for supporting FCoE.

The following example demonstrates how to configure and activate user-defined QoS policies that use CoS3 and Q2 for all FC and FCoE traffic.

• Creating a user-defined network QOS policy:

  
  switch(config)# policy-map type network-qos fcoe_nq
  switch(config-pmap-nqos)# class type network-qos c-nq1
  switch(config-pmap-nqos-c)# mtu 1500
  switch(config-pmap-nqos-c)# class type network-qos c-nq2
  switch(config-pmap-nqos-c)# mtu 9216
  switch(config-pmap-nqos-c)# pause pfc-cos 3 
  switch(config-pmap-nqos-c)# class type network-qos c-nq3
  switch(config-pmap-nqos-c)# mtu 1500
  switch(config-pmap-nqos-c)# class type network-qos c-nq-default
  switch(config-pmap-nqos-c)# mtu 1500
  switch(config-pmap-nqos-c)# exit
  switch(config-pmap-nqos)# exit
  switch(config)#
  

• Creating a user-defined input queuing policy:

  switch(config)# policy-map type queuing fcoe-in-policy
  switch(config-pmap-que)# class type queuing c-in-q2
  switch(config-pmap-c-que)# bandwidth percent 50
  switch(config-pmap-c-que)# class type queuing c-in-q-default
  switch(config-pmap-c-que)# bandwidth percent 50
  switch(config-pmap-c-que)# exit
  switch(config-pmap-que)# exit
  switch(config)
  

• Creating a user-defined output queuing policy:

  switch(config)# policy-map type queuing fcoe-out-policy
  switch(config-pmap-que)# class type queuing c-out-q3
  switch(config-pmap-c-que)# priority level 1
  switch(config-pmap-c-que)# class type queuing c-out-q-default
  switch(config-pmap-c-que)# bandwidth remaining percent 50
  switch(config-pmap-c-que)# class type queuing c-out-q1
  switch(config-pmap-c-que)# bandwidth remaining percent 0
  switch(config-pmap-c-que)# class type queuing c-out-q2
  switch(config-pmap-c-que)# bandwidth remaining percent 50
  switch(config-pmap-c-que)# exit
  switch(config-pmap-que)# exit
  switch(config)#
  
  

• Creating a user-defined QoS input policy:

  
  switch(config)# class-map type qos match-any fcoe
  switch(config-cmap-qos)# match protocol fcoe
  switch(config-cmap-qos)# match cos 3
  switch(config-cmap-qos)# exit
  switch(config)#
  switch(config)# policy-map type qos fcoe_qos_policy
  switch(config-pmap-qos)# class fcoe
  switch(config-pmap-c-qos)# set cos 3
  switch(config-pmap-c-qos)# set qos-group 2
  switch(config-pmap-c-qos)# exit
  switch(config-pmap-qos)# exit
  switch(config)#
  

• Activating a user-defined system QoS policy:

  switch(config)# system qos 
  switch(config-sys-qos)# service-policy type queuing input fcoe-in-policy 
  switch(config-sys-qos)# service-policy type queuing output fcoe-out-policy 
  switch(config-sys-qos)# service-policy type network-qos fcoe_nq 
  switch(config-sys-qos)# exit 
  switch(config)#

• Applying a QoS input policy to an FC or FCoE interface:

  
  switch# conf 
  switch(config)# interface {fc <slot>/<port> | ethernet <slot>/<port> | san-port-channel <no> | port-channel <no>}
  switch(config-if)# service-policy type qos input fcoe_qos_policy

• Removing a QoS input policy from an FC or FCoE interface:

  
  switch# conf 
  switch(config)# interface {fc <slot>/<port> | ethernet <slot>/<port> | san-port-channel <no> | port-channel <no>}
  switch(config-if)# no service-policy type qos input fcoe_qos_policy

• Verifying a QoS input policy applied to an FC or FCoE interface:

  
  switch# show running-config interface {fc <slot>/<port> | interface <slot>/<port> | san-port-channel <no> | port-channel <no>} all

Note	When a user-defined QoS policy is used, the same QoS input policy must be applied to all FC and FCoE interfaces in the switch. Do not configure match protocol fcoe under more than one QoS class map, as FCoE traffic is supported only on a single CoS.

Traffic shaping is used to control access to available bandwidth and to regulate the flow of traffic in order to avoid congestion that can occur when the sent traffic exceeds the access speed. Because traffic shaping limits the rate of transmission of data, you may use this command only when necessary.

The following example demonstrates how to configure traffic shaper:

• The following command displays the default system level settings for all FC interfaces:

  
  switch(config)# show running-config all | i i rate
  hardware qos fc rate-shaper  
  switch(config)#
  

• The following example shows how to configure rate shaper. This command is applied on all FC interfaces:

  Note

  Rarely, you may see input discards on any of the 4G, 8G, 16G, or 32G interfaces. Use the command hardware qos fc rate-shaper [low], to configure the rate shape. Because this is a system level configuration, it will apply to all the FC ports and will reduce the rates for all FC ports. The default option of the command hardware qos fc rate-shaper is applicable to all FC interfaces.

  
  switch(config)# hardware qos fc rate-shaper low
  switch(config)#  
  switch(config)#end
  

The Cisco Nexus N9K-93180YC-FX, N9K-C9336C-FX2-E, and N9K-C93360YC-FX2 devices support vPCs. The vPCscan be configured to increase bandwidth and provide increased load-balancing to the Ethernet fabric. The following are example configurations to explain how to configure FCoE when using vPCs on the Cisco Nexus 9000 Series switches:

Note	FCoE VLANs should not be trunked across vPC peer-links.

Note	Only FC uplinks are supported on Cisco Nexus N9K switches (switchmode) that connects to core switches.

switchname: tme-switch-1
switchname: tme-switch-2
mgmt ip: 172.25.182.66
mgmt ip: 172.25.182.67

The configuration example includes the following hardware:

• Emulex CNA or CISCO CNA

• 2 Cisco Nexus 9000 switches running Cisco NX-OS Release 10.2(1)F or later releases.

The configuration example includes the following considerations and requirements:

• Generation 2 CNAs that support DCBX are required.

• Single host CNA port channel connection to a separate switch. FCoE interfaces will not be brought up if the port channel on a single switch contains more than one member port in a port channel or vPC.

• Cisco NX-OS Release 10.2(1)F or later releases.