A switch is in NPV mode after enabling NPV. NPV mode applies to an entire switch. All end devices connected to a switch that are in NPV mode must log in as an N port to use this feature (loop-attached devices are not supported). All links from the edge switches (in NPV mode) to the NPV core switches are established as NP ports (not E ports), which are used for typical inter-switch links. NPIV is used by the switches in NPV mode to log in to multiple end devices that share a link to the NPV core switch.

The following figure shows an interface-level view of an FC NPV configuration.

In FC NPV mode, the edge switch relays all traffic to the core switch and shares the domain ID of the core switch.

FC NPV is enabled by installing and enabling feature-set fcoe-npv. You cannot configure FC NPV mode on a per-interface basis. FC NPV mode applies to the entire switch.

• In Cisco Nexus 9000 Series switches, server interfaces can be FC or vFC interfaces.

• Server interfaces are F ports on the edge switch that connect to the servers. A server interface may support multiple end devices by enabling the N port identifier virtualization (NPIV) feature. NPIV provides a means to assign multiple FC IDs to a single N port, which allows the server to assign unique FC IDs to different applications.

  Note	To use NPIV, enable the NPIV feature and reinitialize the server interfaces that will support multiple devices.

• FC server interfaces should be in trunk mode off. Trunk mode on is not supported.

• vFC server interfaces should be in trunk mode on.

• Server interfaces are automatically distributed among the NP uplinks to the core switch. All of the end devices connected to a server interface are mapped to the same NP uplink.

• When you connect a 16G host adapter to a 32G SFP port on Cisco Nexus 93360YC-FX and/or 93360YC-FX2 switches, the link may not come up when the speed is configured as auto speed. Or sometimes, it defaults to 8G speed. Then, to use 16G speed, you must manually configure the port using the command switchport speed 16000.

• 8G speed is not supported for server and target interfaces.

• In Cisco Nexus 9000 Series switches, NP uplink interfaces can be native Fibre Channel interfaces, virtual fiber channel interfaces, SAN port channel interfaces, or virtual ethernet port-channel interfaces.

• All interfaces from the edge switch to the core switch are configured as proxy N ports (NP ports).

• An NP uplink is a connection from an NP port on the edge switch to an F port on the core switch. When an NP uplink is established, the edge switch sends a fabric login message (FLOGI) to the core switch then (if the FLOGI is successful) registers itself with the name server on the core switch. Subsequent FLOGIs from end devices connected to this NP uplink are forwarded as-is to the core switch. Subsequent FLOGIs from the same VSAN are forwarded as fdisc.

  Note	In the switch CLI configuration commands and output displays, NP uplinks are called External Interfaces.

• The default speed of NP links is set to auto.

• The features below must be enabled on the core switch:

  • feature npiv

  • feature fport-channel-trunk

• If the FC uplink speed is 8G, the fill pattern should be configured as IDLE on the core switch.

  Note

  Following is an example of configuring IDLE fill pattern on a Cisco MDS switch: Switch(config)# int fc2/3 Switch(config)# switchport fill-pattern IDLE speed 8000 Switch(config)# sh run int fc2/3 interface fc2/3 switchport speed 8000 switchport mode NP switchport fill-pattern IDLE speed 8000 no shutdown

Note

To enable trunking and for the FLOGI from NP uplink of Cisco Nexus 9000 Series switches to be successful on the core switch, both the core and the Cisco Nexus 9000 Series switch should be configured with the OUI of each other. Configure the OUI on the core and Nexus 9000 switch only if the OUI value is not registered by default on either of them. The OUI is found and configured as follows: N9K(config-if)# show wwn switch Switch WWN is 20:00:2c:d0:2d:50:ea:64 N9K(config-if)# On the core, we see the output below if the OUI (0x2cd02d) is already registered. MDS9710(config-if)# sh wwn oui | i 2cd02d 0x2cd02d Cisco Default MDS9710(config-if) # If the OUI is not registered with the core, configure it manually. MDS9710(config-if)# wwn oui 0x2cd02d Beginning with Cisco NX-OS Release 7.3(0)D1(1), the OUI is configurable on a Cisco MDS 9700 Series core switches. If the uplinks to core switch are FCoE enabled, then the FKA advertisement period is taken from configured value on the core switch. If the uplinks to core switch are FC enabled, FKA a period is taken from configured value on local NPV switch. Note The following example shows the FCoE uplink. Because the switch has an FCoE link, the value is taken from FCF: switch(config)# sh run fcoe_mgr | i i fka fcoe fka-adv-period 12 switch(config)# sh fcoe FCF details for interface vfc-po142 FCF-MAC is 54:7f:ee:ec:71:84 FC-MAP is 0e:fc:00 FCF Priority is 128 FKA Advertisement period for FCF is 8 seconds <<<<< The following example shows the FC uplink: switch(config)# sh run | i i fka fcoe fka-adv-period 10 switch(config)# sh fcoe FCF details for interface san-port-channel29 FCF-MAC is 2c:d0:2d:50:e4:29 FC-MAP is 0e:fc:00 FCF Priority is 129 FKA Advertisement period for FCF is 10 seconds

When an NP port becomes operational, the switch first logs itself in to the core switch by sending a FLOGI request (using the port WWN of the NP port).

After completing the FLOGI request, the switch registers itself with the fabric name server on the core switch (using the symbolic port name of the NP port and the IP address of the edge switch).

The following table identifies port and node names in the edge switch used in FC NPV mode.

NPV supports automatic selection of NP uplinks. When a server interface is brought up, the NP uplink interface with the minimum load is selected from the available NP uplinks in the same VSAN as the server interface.

When a new NP uplink interface becomes operational, the existing load is not redistributed automatically to include the newly available uplink. Server interfaces that become operational after the NP uplink can select the new NP uplink.

FC NPV supports traffic maps. A traffic map allows you to specify the NP uplinks that a server interface can use to connect to the core switches.

Note	When an FC NPV traffic map is configured for a server interface, the server interface must select only from the NP uplinks in its traffic map. If none of the specified NP uplinks are operational, the server remains in a non-operational state.

The FC NPV traffic map feature provides the following benefits:

• Facilitates traffic engineering by allowing configuration of a fixed set of NP uplinks for a specific server interface (or range of server interfaces).

• Ensures correct operation of the persistent FC ID feature; this is because a server interface will always connect to the same NP uplink (or one of a specified set of NP uplinks) after an interface reinitialization or switch reboot.

FC NPV supports disruptive load balancing of server logins. When disruptive load balancing is enabled, FC NPV redistributes the server interfaces across all available NP uplinks when a new NP uplink becomes operational. To move a server interface from one NP uplink to another NP uplink, FC NPV forces reinitialization of the server interface so that the server performs a new login to the core switch.

In Release 7.0(3)I7(2) and later software releases, FC NPV supports disruptive load balancing of server logins. When disruptive load balancing is enabled, FC NPV redistributes the server interfaces across all available NP uplinks when a new NP uplink becomes operational. To move a server interface from one NP uplink to another NP uplink, FC NPV forces reinitialization of the server interface so that the server performs a new login to the core switch.

Only server interfaces that are moved to a different uplink are reinitialized. A system message is generated for each server interface that is moved.

Note	Redistributing a server interface causes traffic disruption to the attached end devices. Adding a member to the existing port-channel does not trigger disruptive auto load-balance.

To avoid disruption of server traffic, you should enable this feature only after adding a new NP uplink, and then disable it again after the server interfaces have been redistributed.

If disruptive load balancing is not enabled, you can manually reinitialize some or all of the server interfaces to distribute server traffic to new NP uplink interfaces.

When deploying FC NPV traffic management, follow these guidelines:

• Use FC NPV traffic management only when automatic traffic engineering does not meet your network requirements.
• You do not need to configure traffic maps for all server interfaces. By default, FC NPV will use automatic traffic management.
• Server interfaces configured to use a set of NP uplink interfaces cannot use any other available NP uplink interfaces, even if none of the configured interfaces are available.
• When disruptive load balancing is enabled, a server interface may be moved from one NP uplink to another NP uplink. Moving between NP uplink interfaces requires FC NPV to relogin to the core switch, causing traffic disruption.
• To link a set of servers to a specific core switch, associate the server interfaces with a set of NP uplink interfaces that all connect to that core switch.
• Configure Persistent FC IDs on the core switch and use the Traffic Map feature to direct server interface traffic onto NP uplinks that all connect to the associated core switch.

When configuring FC NPV, note the following guidelines and limitations:

• Fibre Channel N-port Virtualization (NPV) can co-exist with VXLAN on different fabric uplinks but on same or different front panel ports on the Cisco Nexus 93180YC-FX, N9K-C9336C-FX2-E, and N9K-C93360YC-FX2 switches. VXLAN can only exist on the Ethernet front panel ports but not on the FC front panel ports.

• In-order data delivery is not required in FC NPV mode because the exchange between two end devices always takes the same uplink from the edge switch to the core. Upstream of the edge switch, core switches will enforce in-order delivery if configured.

• You can configure zoning for end devices that are connected to edge switches using all the available member types on the core switch. However, the preferred way of zoning servers connected to any switch in NPV mode is via PWWN, device-alias and fcalias. You must place multiple servers in the same zone only when using smart zoning. For more information about smart zoning on Cisco MDS switches, refer the chapter Configuring and Managing Zones in Cisco MDS 9000 Series Fabric Configuration Guide .

• Port tracking is not supported in FC NPV mode.

• Port security is supported on the core switch for devices logged in through the FC NPV switch. Port security is enabled on the core switch on a per-interface basis. To enable port security on the core switch for devices that log in through an FC NPV switch, you must adhere to the following requirements:

  • The internal FLOGI must be in the port security database; in this way, the port on the core switch will allow communications and links.

  • All the end device pWWNs must also be in the port security database.

• Edge switches can connect to multiple core switches. In other words, different NP ports can be connected to different core switches.

• If a server interface goes down and then returns to service, the interface is not guaranteed to be assigned to the same NP uplink.

• The server interface is only operational when its assigned NP uplink is operational.

• Both servers and targets can be connected to the switch when in FC NPV mode.

• Fibre Channel switching is not performed in the edge switch; all traffic is switched in the core switch.

• FC NPV supports NPIV-capable servers. This capability is called nested NPIV.

• Connecting two Cisco FC NPV switches together is not supported.

• Only F and NP ports are supported in FC NPV mode.

• Speed auto-negotiation is supported only for Cisco Nexus 93180YC-FXand N9k-C93360YC-FX2 switches. The default speed is set to auto.

• Speed auto-negotiation is not supported in N9K-C9336C-FX2-E switch and default speed is set to 32G.

• Nexus 9000 only supports the IDLE fill pattern on 8 Gbps Fibre Channel interfaces. For Nexus 9000 FC interface to operate at 8 Gbps, peer device must be configured to use a matching IDLE fill pattern. Most server and target FC interfaces do not support this and thus cannot connect to Nexus 9000 at 8 Gbps. To interoperate with other Fibre Channel switches at 8 Gbps ensure the peer switch FC interface also uses a matching IDLE fill pattern. For Cisco MDS switches, configure using the switchport fill-pattern interface configuration command. To connect to a peer Nexus 9000 at 8 Gbps, use no fill pattern configuration, as both devices use matching IDLE fill patterns by default.

• The default port-speed for all FC interfaces is auto for Cisco Nexus N9k-C93180YC-FX and N9k-C93360YC-FX2 switches.

• The default port-speed of all FC ports is 32G for Cisco Nexus N9K-C9336C-FX2-E switch.

• The receive B2B credit value is 64 in N9K-C93180YC-FX and 32 in N9K-C93360YC-FX2 and N9K-C9336C-FX2-E. This is not configurable.

• When a san-port channel is created, it is created in channel mode active by default; channel mode on is not supported for NPV switch.

• vFC flap may be required to bring up N Port vFC interfaces after changing the FCoE FC map.

• FC-NPV (up to 32G) and FCoE-NPV are supported on N9K-C93180YC-FX, N9K-C9336C-FX2-E, and N9k-C93360YC-FX2 switches both as NP uplink and F host port.

• Beginning with Cisco NX-OS Release 10.2(2)F, FC-NPV is supported on Cisco Nexus N9K-C9336C-FX2-E switch.

• Operating speed and member addition to san-po limitation on N9K-C9336C-FX2-E:

  1. Speed change of fc-breakout

     • Default speed is 32G

     • Speed change cannot be done on a single fc-breakout interface level

     • Speed change of fc-breakout is done on range of fc-breakout interface level

       • The range should contain full set of fc-breakout corresponds to a front panel port - for any partial range, speed config throws ERR_01 error.

       • The range should not contain any fc-breakout which is part of san-po - if range has any san-po member, speed config throws ERR_02 error.

       • The range can have fc-breakout ports corresponds to multiple front panel ports.

  2. Speed change of san-po

     • Default speed is 32G.

     • Speed change of san-po is allowed only if its members includes all fc-breakout ports corresponds to a front panel port - if san-po has partial set fc-breakout ports corresponds a front panel port, speed change throws ERR_03 error.

     • Speed change of san-po can be done by giving range of san-po interfaces.

  3. Speed config in running config

     • Speed config(not default one) will be displayed in fc-breakout interface range level; it will not be displayed under individual fc-breakout interface for the "sh runn" command. Speed config(not default one) will be displayed in the "show interface fc<int no>" command.

  4. Member addition to san-po (channel-group x)

     • The interface range should contain full set of fc-breakout corresponds to a front panel port - for any partial range, though the channel addition is successful, warningWARN_01 message will be thrown.

     • The range can have fc-breakout ports corresponds to multiple front panel ports.

• Error and Warning messages:

  • ERR_01:

    if-range contains partial set of fc1/18/1-4 fc-breakout ports

  • ERR_02:

    if-range contains fc1/21/1-4 ports; some are part sanpo

  • ERR_03:

    san-port-channel21 does not contain full set of fc1/22/1-4 fc-breakout ports

  • WARN_01:

    Warning: if-range contains partial set of fc1/22/1-4 fc-breakout ports

After you enable FC NPV, you should configure the NP uplink interfaces and the server interfaces.