Licensing Requirements
For a complete explanation of Cisco NX-OS licensing recommendations and how to obtain and apply licenses, see the Cisco NX-OS Licensing Guide and the Cisco NX-OS Licensing Options Guide.
The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.
This chapter contains the following sections:
For a complete explanation of Cisco NX-OS licensing recommendations and how to obtain and apply licenses, see the Cisco NX-OS Licensing Guide and the Cisco NX-OS Licensing Options Guide.
This chapter provides an overview of SAN switching for Cisco Nexus 9000 devices. This chapter includes the following sections:
When you use expansion modules up to 8 Fibre Channel ports are available on the Cisco Nexus 5010 switch and up to 16 Fibre Channel ports are available on the Cisco Nexus 5020 switch.
Domain Parameters
The Fibre Channel domain (fcdomain) feature performs principal switch selection, domain ID distribution, FC ID allocation, and fabric reconfiguration functions as described in the FC-SW-2 standards. The domains are configured per VSAN . If you do not configure a domain ID, the local switch uses a random ID.
N Port Virtualization
Cisco NX-OS software supports industry-standard N port identifier virtualization (NPIV), which allows multiple N port fabric logins concurrently on a single physical Fibre Channel link. HBAs that support NPIV can help improve SAN security by enabling zoning and port security to be configured independently for each virtual machine (OS partition) on a host. In addition to being useful for server connections, NPIV is beneficial for connectivity between core and edge SAN switches.
N port virtualizer (NPV) is a complementary feature that reduces the number of Fibre Channel domain IDs in core-edge SANs. Cisco MDS 9000 family fabric switches operating in the NPV mode do not join a fabric; they only pass traffic between core switch links and end devices, which eliminates the domain IDs for these switches. NPIV is used by edge switches in the NPV mode to log in to multiple end devices that share a link to the core switch. This feature is available only for Cisco MDS Blade Switch Series, the Cisco MDS 9124 Multilayer Fabric Switch, and the Cisco MDS 9134 Multilayer Fabric Switch.
N port virtualizer (NPV) is a complementary feature that reduces the number of Fibre Channel domain IDs in core-edge SANs. Cisco Nexus 9000 series fabric switches operating in the NPV mode do not join a fabric; they only pass traffic between core switch links and end devices, which eliminates the domain IDs for these switches. NPIV is used by edge switches in the NPV mode to log in to multiple end devices that share a link to the core switch.
VSAN Trunking
Trunking, also known as VSAN trunking, enables interconnect ports to transmit and receive frames in more than one VSAN over the same physical link. Trunking is supported on E ports and F ports.
SAN Port Channels
PortChannels aggregate multiple physical ISLs into one logical link with higher bandwidth and port resiliency for Fibre Channel traffic. With this feature, up to 16 expansion ports (E-ports) or trunking E-ports (TE-ports) can be bundled into a PortChannel. ISL ports can reside on any switching module, and they do not need a designated primary port. If a port or a switching module fails, the PortChannel continues to function properly without requiring fabric reconfiguration.
Cisco NX-OS software uses a protocol to exchange PortChannel configuration information between adjacent switches to simplify PortChannel management, including misconfiguration detection and autocreation of PortChannels among compatible ISLs. In the autoconfigure mode, ISLs with compatible parameters automatically form channel groups; no manual intervention is required.
PortChannels load balance Fibre Channel traffic using a hash of source FC-ID and destination FC-ID, and optionally the exchange ID. Load balancing using PortChannels is performed over both Fibre Channel and FCIP links. Cisco NX-OS software also can be configured to load balance across multiple same-cost FSPF routes.
Virtual SANs
Virtual SANs (VSANs) partition a single physical SAN into multiple VSANs. VSANs allow the Cisco NX-OS software to logically divide a large physical fabric into separate, isolated environments to improve Fibre Channel SAN scalability, availability, manageability, and network security.
Each VSAN is a logically and functionally separate SAN with its own set of Fibre Channel fabric services. This partitioning of fabric services greatly reduces network instability by containing fabric reconfiguration and error conditions within an individual VSAN. The strict traffic segregation provided by VSANs can ensure that the control and data traffic of a specified VSAN are confined within the VSAN's own domain, which increases SAN security. VSANs can reduce costs by facilitating consolidation of isolated SAN islands into a common infrastructure without compromising availability.
You can create administrator roles that are limited in scope to certain VSANs. For example, you can set up a network administrator role to allow configuration of all platform-specific capabilities and other roles to allow configuration and management only within specific VSANs. This approach improves the manageability of large SANs and reduces disruptions due to human error by isolating the effect of a user action to a specific VSAN whose membership can be assigned based on switch ports or the worldwide name (WWN) of attached devices.
VSANs are supported across Fibre Channel over IP (FCIP) links between SANs, which extends VSANs to include devices at a remote location. The Cisco SAN switches also implement trunking for VSANs. Trunking allows Inter-Switch Links (ISLs) to carry traffic for multiple VSANs on the same physical link.
Zoning
Zoning provides access control for devices within a SAN. The Cisco NX-OS software supports the following types of zoning:
N port zoning-Defines zone members based on the end-device (host and storage) port.
WWN
Fibre Channel identifier (FC-ID)
Fx port zoning-Defines zone members based on the switch port.
WWN
WWN plus the interface index, or domain ID plus the interface index
Domain ID and port number (for Brocade interoperability)
iSCSI zoning-Defines zone members based on the host zone.
iSCSI name
IP address
LUN zoning-When combined with N port zoning, logical unit number (LUN) zoning helps ensure that LUNs are accessible only by specific hosts, providing a single point of control for managing heterogeneous storage-subsystem access.
Read-only zones-An attribute can be set to restrict I/O operations in any zone type to SCSI read-only commands. This feature is useful for sharing volumes across servers for backup, data warehousing, and so on.
Broadcast zones-An attribute can be set for any zone type to restrict broadcast frames to members of the specific zone.
To provide strict network security, zoning is always enforced per frame using access control lists (ACLs) that are applied at the ingress switch. All zoning polices are enforced in the hardware, and none of them cause performance degradation. Enhanced zoning session-management capabilities further enhance security by allowing only one user at a time to modify zones.
Device Alias Services
The software supports Device Alias Services (device alias) on per VSAN and fabric wide. Device alias distribution allows you to move host bus adapters (HBAs) between VSANs without manually reentering alias names.
Fibre Channel Routing
Fabric Shortest Path First (FSPF) is the protocol used by Fibre Channel fabrics. FSPF is enabled by default on all Fibre Channel switches. You do not need to configure any FSPF services except in configurations that require special consideration. FSPF automatically calculates the best path between any two switches in a fabric. Specifically, FSPF is used to perform these functions:
SCSI Targets
Small Computer System Interface (SCSI) targets include disks, tapes, and other storage devices. These targets do not register logical unit numbers (LUNs) with the name server. The SCSI LUN discovery feature is initiated on demand, through CLI or SNMP. This information is also synchronized with neighboring switches, if those switches belong to the Cisco Nexus device.
Advanced Fibre Channel Features
You can configure Fibre Channel protocol-related timer values for distributed services, error detection, and resource allocation.
You must uniquely associate the WWN to a single switch. The principal switch selection and the allocation of domain IDs rely on the WWN. Cisco Nexus devices support three network address authority (NAA) address formats.
Fibre Channel standards require that you allocate a unique FC ID to an N port that is attached to an F port in any switch. To conserve the number of FC IDs used, Cisco Nexus devices use a special allocation scheme.
FC-SP and DHCHAP
The Fibre Channel Security Protocol (FC-SP) provides switch-to-switch and hosts-to-switch authentication to overcome security challenges for enterprise-wide fabrics. The Diffie-Hellman Challenge Handshake Authentication Protocol (DHCHAP) is an FC-SP protocol that provides authentication between Cisco SAN switches and other devices. DHCHAP consists of the CHAP protocol combined with the Diffie-Hellman exchange.
With FC-SP, switches, storage devices, and hosts can prove their identity through a reliable and manageable authentication mechanism. With FC-SP, Fibre Channel traffic can be secured per frame to prevent snooping and hijacking even over untrusted links. A consistent set of policies and management actions are propagated through the fabric to provide a uniform level of security across the entire fabric.
Port Security
The port security feature prevents unauthorized access to a switch port by binding specific world-wide names (WWNs) that have access to one or more given switch ports.
When port security is enabled on a switch port, all devices connecting to that port must be in the port security database and must be listed in the database as bound to a given port. If both of these criteria are not met, the port will not achieve an operationally active state and the devices connected to the port will be denied access to the SAN.
Fabric Binding
Fabric binding ensures Inter-Switch Links (ISLs) are enabled only between specified switches in the fabric binding configuration, which prevents unauthorized switches from joining the fabric or disrupting the current fabric operations. This feature uses the Exchange Fabric Membership Data (EEMD) protocol to ensure that the list of authorized switches is identical in all of the switches in a fabric.
Fabric Configuration Servers
The Fabric Configuration Server (FCS) provides discovery of topology attributes and maintains a repository of configuration information of fabric elements. A management application is usually connected to the FCS on the switch through an N port. Multiple VSANs constitute a fabric, where one instance of the FCS is present per VSAN.
The following are the general following guidelines and limitations of SAN switching:
SAN switching is supported only on Cisco Nexus C93180YC-FX and C93360YC-FX2 switches. Beginning with Cisco NX-OS Release 10.2(2), SAN switching is also supported on Cisco N9K-C9336C-FX2-E platform switches.
VE-port or virtual expansion port (ISL) is supported from Cisco NX-OS Release 10.2(3)F.
Dynamic Port VSAN Membership (DPVM) not supported.
Fabric Extender (FEX) with switch mode is not supported
IP over Fibre Channel (IPFC) function is not supported.
Inter VSAN Routing(IVR) is not supported
XML and DME of CLIs are not supported.
OBFL (show logging onboard) feature support is limited to the error statistics.
Note |
For more information on OBFL, see: Cisco Nexus 9000 Series NX-OS Troubleshooting Guide, Release 9.3(x) |
Nexus 9000 only supports the IDLE fill pattern on 8 Gbps Fibre Channel interfaces. For Nexus 9000 FC interface to operate at 8 Gbps, peer device must be configured to use a matching IDLE fill pattern. Most server and target FC interfaces do not support this and thus cannot connect to Nexus 9000 at 8 Gbps. To interoperate with other Fibre Channel switches at 8 Gbps ensure the peer switch FC interface also uses a matching IDLE fill pattern. For Cisco MDS switches, configure using the switchport fill-pattern interface configuration command. To connect to a peer Nexus 9000 at 8 Gbps, use no fill pattern configuration, as both devices use matching IDLE fill patterns by default.
Beginning with Cisco NX-OS Release 10.2(2), the operating speed and member addition to san-po limitation on Cisco Nexus N9K-C9336C-FX2-E platform switch is as follows:
Speed change of fc-bo:
Default speed of fc-bo is 32G.
Speed change cannot be done on a single fc-bo interface level.
Speed change of fc-bo is done on a range of fc-bo interface level.
The range should contain full set of fc-bo corresponding to a front panel port.
Note |
For any partial range, speed configuration displays the ERR_01 error. |
The range should not contain any fc-bo which is a part of san-po.
Note |
If the range has any san-po member, speed configuration displays the ERR_02 error. |
The range can have fc-bo ports corresponding to multiple front panel ports.
Speed change of san-po:
Default speed of san-po is 32G.
Speed change of san-po is allowed only if its members include all fc-bo ports corresponding to a front panel port.
Note |
If san-po has partially set fc-bo ports corresponding to a front panel port, the speed change displays the ERR_03 error. |
Speed change of san-po can be done by providing a range of san-po interfaces.
Speed config in running config:
Speed config (not the default) will be displayed in the fc-bo interface range level; it will not be displayed under the individual fc-bo interface for the sh runn command.
Speed config (not the default) will be displayed in the show interface fc<int no> command.
Member addition to san-po (channel-group x):
The interface range should contain the full set of fc-bo corresponding to a front panel port.
Note |
Though the channel addition is successful, the WARN_01 warning message will be displayed for any partial range. |
The range can have fc-bo ports corresponding to multiple front panel ports.
ERR_01 : if-range contains partial set of fc1/18/1-4 fc-bo ports
ERR_02 : if-range contains fc1/21/1-4 ports; some are part sanpo
ERR_03 : san-port-channel21 does not contain full set of fc1/22/1-4 fc-bo ports
WARN_01 : Warning: if-range contains partial set of fc1/22/1-4 fc-bo ports
Beginning with Cisco NX-OS Release 10.2(3)F, virtual E port (VE port) connectivity between Fibre Channel Forwarders (FCFs) is supported on Cisco N9K-C93180YC-FX, N9K-C9336C-FX2-E, and N9K-C93360YC-FX2 platform switches.