Cisco Nexus 5000 Series NX-OS Software Configuration Guide

Licensing Requirements

On Cisco Nexus 5000 Series switches, FCoE capability is included in the Storage Protocol Services License.

Before using FCoE capabilities, ensure that:

• The correct license is installed (N5010SS or N5020SS).
• FCoE is activated by entering the feature fcoe command in configuration mode.

Converged Network Adapters

The following types of converged network adapters (CNAs) are available:

• Hardware adapter

– Works with the existing FC HBA driver and LAN NIC driver in the server.

– Server operating system view of the network is unchanged; the CNA presents a SAN interface and a LAN interface to the operating system.

• FCoE software stack

– Runs on existing 10-Gigabit Ethernet adapters.

FCoE offers a number of optional capabilities. The available capabilities and their configurable values are negotiated between the switch and the adapter. To accomplish this, the adapter and the switch exchange information using the Data Center Bridging Exchange Protocol (DCBX).

To reduce configuration errors and simplify administration. you can configure the switch to distribute the configuration data to all the connected adapters.

DCBX Capabilities

The DCBX capabilities supported by Cisco Nexus 5000 Series switches are described in the following topics:

• FCoE
• Priority Flow Control
• Logical Link Up/Down

FCoE

By default, each Ethernet interface attempts to enable FCoE capability by advertising the capability to the adapter.

If the FCoE negotiation fails, you can configure the switch to disable FCoE or to force-enable FCoE for this interface.

Priority Flow Control

The priority flow control (PFC) capability allows you to apply pause functionality to specific classes of traffic. PFC decides whether to apply pause based on the IEEE 802.1p CoS value. When the switch enables PFC, it configures the connected adapter to apply the pause functionality to packets with specific CoS values.

By default, the switch negotiates to enable the PFC capability. If the negotiation succeeds, PFC is enabled and link-level flow control remains disabled (regardless of its configuration settings).

If the PFC negotiation fails, you can either force PFC to be enabled on the interface or you can enable IEEE 802.x link-level flow control.

Link-level flow control can be enabled on the interface only if PFC is not enabled. For additional information about link-level flow control, see the “Link-Level Flow Control” section.

Logical Link Up/Down

On a native Fibre Channel link, some configuration actions (such as changing the VSAN) require you to reset the interface status. The switch achieves the reset by disabling the interface, and then immediately reenabling the interface.

If an Ethernet link is providing FCoE service, the physical link should not be reset because this action is disruptive to all traffic on the link.

The logical link up/down feature provides the ability to reset an individual virtual link. The switch sends a DCBX message to request the adapter to reset only the virtual Fibre Channel interface.

Note If the adapter does not support the logical link level up/down feature, the adapter resets the physical link. In this case, all traffic on the Ethernet interface is disrupted.

DCE Bridging Capability Exchange Protocol

DCBX protocol is an extension of the Link Layer Discovery Protocol (LLDP). DCBX end points exchange request and acknowledgment messages. For flexibility, parameters are coded in a type-length-value (TLV) format.

DCBX runs on the physical Ethernet link between the Cisco Nexus 5000 Series switch and the converged network adapter on the server. By default, DCBX is enabled on Ethernet interfaces. When an Ethernet interface is brought up, the switch automatically starts to communicate with the adapter.

During normal operation of FCoE between the switch and the adapter, the DCBX protocol provides link-error detection.

DCBX is also used to negotiate capabilities between the switch and the adapter and to send configuration values to the adapter.

Adapters connected to a Cisco Nexus 5000 Series switch are programmed to accept the configuration values sent by the switch, allowing the switch to distribute configuration values to all attached adapters. The capability reduces the possibility of configuration error and simplifies administration of the adapters.

The switch provides a manual configuration override for each parameter negotiated using DCBX. The override takes effect if the adapter does not support DCBX, or if the adapter does not support the specific capability requested by the switch.

DCBX Feature Negotiation

The switch and adapter exchange capability information and configuration values. The Cisco Nexus 5000 Series switches support the following capabilities:

• FCoE

If the adapter supports FCoE capability, the switch sends the IEEE 802.1p CoS value to be used with FCoE packets.

• Priority Flow Control (PFC)

If the adapter supports PFC, the switch sends the IEEE 802.1p CoS values to be enabled with PFC.

• Ethernet logical link up and down signal
• FCoE logical link up and down signal

The following rules determine whether the negotiation results in a capability being enabled:

• If a capability and its configuration values match between the switch and the adapter, the feature is enabled.
• If the capability matches, but the configuration values do not match:

– If the adapter is configured to accept the switch configuration value, the capability is enabled using the switch value.

– If the adapter is not configured to accept the switch configuration value, the capability remains disabled.

• If the adapter does not support the DCBX capability, the capability remains disabled.
• If the adapter does not implement DCBX, all capabilities remain disabled.

Note The Cisco Nexus 5000 Series switch provides CLI commands to manually override the results of the negotiation with the adapter. On a per-interface basis, you can force capabilities to be enabled or disabled. For additional information, see the “Configuring FCoE” section.

Ethernet Frame Formats

Ethernet frames sent by the switch to the adapter may include the IEEE 802.1Q tag. This tag includes a field for the CoS value used by PFC. The IEEE 802.1Q tag also includes a VLAN field, currently not used by the Cisco Nexus 5000 Series switch.

The switch will always accept tagged or untagged frames from the adapter.

If the FCoE capability is enabled between the switch and the adapter:

• All Ethernet frames sent to the adapter include IEEE 802.1Q tags.

– The VLAN ID field is always set to 0.

– The CoS field is set to the CoS value of the associated system class. For additional information, see the “System Classes” section.

• The switch accepts tagged or untagged frames from the adapter.

If the FCoE capability is disabled between the switch and the adapter:

• Ethernet frames sent to the adapter do not include IEEE 802.1Q tags.
• The switch accepts tagged or untagged frames from the adapter.

Enabling FCoE

You need to enable the FCoE capability after the FC_FEATURES_PKG is installed. To enable FCoE, perform this task:

The following example shows how to enable FCoE on the switch:

Enabling FCoE on Ethernet Interfaces

By default, 10-Gigabit Ethernet interfaces negotiate FCoE capability with the adapter. You can override the negotiation result by force-enabling the FCoE capability. To force-enable the FCoE capability, perform this task:

The following example shows how to force-enable FCoE for an Ethernet interface:

This example shows how to disable FCoE for an Ethernet interface:

The fcoe command can only be applied to a physical Ethernet interface.

Configuring Priority Flow Control

By default, the Ethernet interfaces negotiate PFC capability with the adapter. You can override the negotiation result by force-enabling the PFC capability. To force-enable the PFC capability, perform this task:

The following example shows how to force-enable PFC on an interface:

To disable PFC capability for this interface, perform this task:

Configuring IEEE 802.3x Link-Level Flow Control

By default, link-level flow control capability on Ethernet interfaces is disabled. Only enable the link-level flow control capability if PFC is disabled on the interface.

To configure link-level flow control, see the “Configuring IEEE 802.3x Link-Level Flow Control” section.

Configuring Global LLDP Commands

You can set global LLDP settings. These settings include the length of time before discarding LLDP information received from peers, the length of time to wait before performing LLDP initialization on any interface, and the rate at which LLDP packets are sent.

To configure LLDP settings, perform this task:

The following example shows how to set LLDP timer option to 15 seconds:

switch# configure terminal

switch(config)# lldp timer 15

To reset LLDP settings, perform this task:

Configuring Interface LLDP Commands

To configure the LLDP feature for a physical Ethernet interface, perform this task:

The following example shows how to set an interface to transmit LLDP packets:

The following example shows how to configure an interface to disable LLDP:

You can only apply this command to a physical Ethernet interface.