Overview

Supported Platforms

Starting with Cisco NX-OS release 7.0(3)I7(1), use the Nexus Switch Platform Support Matrix to know from which Cisco NX-OS releases various Cisco Nexus 9000 and 3000 switches support a selected feature.

About Interfaces

Cisco NX-OS supports multiple configuration parameters for each of the interface types supported. Most of these parameters are covered in this guide but some are described in other documents.

The following table shows where to get further information on the parameters you can configure for an interface.

Table 1. Interface Parameters

Feature

Parameters

Further Information

Basic parameters

Description, duplex, error disable, flow control, MTU, beacon

“Configuring Basic Interface Parameters”

Layer 3

Medium, IPv4 and IPv6 addresses

“Configuring Layer 3 Interfaces”

Layer 3

Bandwidth, delay, IP routing, VRFs

Cisco Nexus 9000 Series NX-OS Unicast Routing Configuration Guide

Cisco Nexus 9000 Series NX-OS Multicast Routing Configuration Guide

Port Channels

Channel group, LACP

“Configuring Port Channels”

Security

EOU

Cisco Nexus 9000 Series NX-OS Security Configuration Guide

Ethernet Interfaces

  • Ethernet interfaces include routed ports.

  • For N9K-C9316D-GX, Ports 1-16 supports 400G/100G/40G, and 10G with QSA.

Cisco Nexus N9K-C9364C-GX and N9K-C93600CD-GX Guidelines and Limitations

Cisco Nexus N9K-C9364C-GX and N9K-C93600CD-GX switches have the following guidelines and limitations:

  • Consecutive groups of four interfaces (1-4, 5-8, 9-12, and so on, are referred to as a quad group). Attempting to use a mix of link speeds within a quad group is not supported. This applies to ports 1-24 of the N9K-C93600CD-GX and all ports of the N9K-C9364C-GX.

  • Only one speed will be up in a quad group at a time. The first link up in a quad group determines the speed of the quad group. Ports with any other speed will be down with the reason of "Link not connected".

  • The speed which remains functional when mixing within a quad group is not stored. When a mismatched speed transceiver is inserted into a quad group and brought up, all ports in the quad group will be reset. The first link which comes up after the reset determines the speed of the quad group. It is possible that pre-existing links may be shut down. You can remove the mismatched speed transceiver to recover from this state.

  • FC-FEC is not supported on the second lane of the 50Gx2 breakout port. The second breakout port will not come up when 50Gx2 breakout is configured, ensure to configure RS-FEC with 50Gx2 breakout.

For Cisco NX-OS Release 10.1(1), Port-group Support - GX provides the following support:

  • The following points are applicable for N9K-C93600CD-GX:

    • For ports 1-24, every four ports (1-4, 5-8, 9-12, and so on, referred to as a "quad") operate at the same speed.

    • All the ports in a quad operate in 10G with QSA, or 40G or 100G.

    • Mixed speed is not supported within the same quad.

    • With QSA, all ports in a quad can operate at 10G speed.

    • The speed of the quad is determined by the first Transceiver Type plugged in that quad and not by what Transceiver is plugged in first port of the quad.

      For example, if port 14 (belonging to quad containing ports 13-16) is a 10G Transceiver with QSA plugged in as the first in the quad, then all other ports in the quad should be of speed 10G.

      Similarly if Port 23 (belonging to quad 21-24) if plugged in with 40G Transceiver as the first port in the quad then all other ports should be plugged in with 40G Transceiver.

    • If a 100G Transceiver is plugged in port 24 while port 21 is already plugged-in with 40G Transceivers, then the 100G Interface will be in “XCVR speed mismatch” state and link will not come up.

    • In order for Port 24 with 100G Transceiver to be able to link up then all other non 100G transceivers in that quad needs to be removed and port 24 needs to be flapped.

    • This is same with the case when QSA+10G is inserted in a port where there is a non QSA+10G Transceiver plugged already in that quad.

    • When a Mismatch Transceiver is plugged into a quad, syslog will be generated as “Interface Ethernet1/X is down (Reason: Inserted transceiver speed mismatch with quad speed Y).

    • If Port 4 is plugged in with a 100G Transceiver followed by a 40G Transceiver in Port 1, then 100G will bring up the link but 40G transceiver will not link up and be in “XCVR mismatch speed”. If this configuration is copied as startup configuration and switch is reloaded, after the switch comes up, 100G transceiver will be up and 40G Transceiver will be in “XCVR mismatch speed” state.

    • The port number will not decide the speed of the quad, only the first plugged-in Transceiver will determine the speed even when all interfaces are in “Admin shut” state in the quad. Same state will be preserved when “copy running-config startup-config” is performed and the switch is reloaded. But when the switch is reloaded with “Reload ascii” option, after the switch comes up, first port (with a plugged in transceiver) in the quad will determine the port group. Other mismatching transceivers will be in “XCVR mismatch Speed”.

    • If 40G transceiver needs to be up then 100G Transceiver needs to be removed. All other ports in that quad can be empty or plugged in only with 40G transceiver for all the transceivers to link up in that particular quad.

    • Only exception to this is when a quad has 40G Transceivers and is plugged in with a Dual Rate Transceiver (40G/100G Capable) configured with 40G Speed then dual rate transceiver with 40G Speed will link up. The port number will not decide the speed of the quad, only the first plugged-in Transceiver will determine the speed of the quad even when all interfaces are in “Admin shut” state in the quad. Same state will be preserved when “copy running-config startup-config” is performed and switch is reloaded but when the switch is reloaded with “Reload ascii” option, after the switch comes up, first port (with a plugged in transceiver) in the quad will determine the speed of the quad, other mismatching transceivers will be in “XCVR mismatch Speed”.

    • It is highly recommended to use only Transceivers of same speed in a quad to avoid any disruption or indeterministic state. Same logic is extended for quad ports 25-26 and other quad ports 27-28.

  • For N9K-C9316D-GX: Ports 1-16 support 400G/100G/40G and 10G with QSA and there are no port group limitations.

  • Beginning with Cisco Nexus NX-OS Release 10.1(2) Auto negotiation is supported for Speed 40G and 100G on NX-OS N9K-C93600CD-GX, N9K-C9316D-GX and N9K-C9364C-GX in NRZ mode.

  • Mixed speed is not supported within the same quad.

  • With QSA all ports in a quad can operate at 10G speed.

  • Beginning with Cisco Nexus NX-OS Release 10.4(3)F, on N9K-C93600CD-GX and N9K-C9316D-GX, auto negotiation is not supported on 100g copper PAM4 links. You must configure speed 100000 on peer side to bring the link up.

  • Beginning with Cisco Nexus NX-OS Release 10.4(3)F, on N9K-C93600CD-GX, 100g PAM4 links is supported on ports 29-36 only.

Cisco Nexus N9K-X9400-16W Guidelines and Limitations

Cisco Nexus 9000 C9408 Chassis, N9K-X9400-16W (16x200G Line-Card Expansion Module (LEM)) port has the following guidelines and limitations:

  • Native port supports 100G, 40G, 10G on all ports.

  • Breakout ports support 4x10G, 4x25G with the following limitations:

    1. The 4x10G, 4x25G breakout ports are supported only on odd ports.

    2. When breakout x4 is configured on an odd port, the next corresponding even port is purged automatically.

  • Breakout ports support 2x50G with the following limitations:

    1. The 2x50G breakout is supported on odd and even ports.

    2. When the 2x50G breakout is configured on an odd/even port, the corresponding even/odd port is broken out 2x50G automatically.

  • 10G using QSA is supported on all ports with the following limitations:

    1. When a 10G transceiver is present on an odd/even port in the linked up state, it does not allow any other speed on the corresponding even/odd port. A warning/syslog will be printed for the mismatched XCVR and port status changes to speed mismatch state for the XCVR port that was inserted later. And port status will be indicated in the show interface brief and show interface status command outputs.

    2. When a 100G/40G transceiver is present on an odd/even port in linked up state, and a 10G transceiver is inserted in the corresponding even/odd port, a warning/syslog will be printed for the mismatched XCVR and port status changes to speed mismatch state for the XCVR port that was inserted later. And port status will be indicated in the show interface brief and show interface status command outputs.

    3. When odd port has 40G/100G and corresponding even port has 10G transceiver or vice versa, in admin shut status, no precedence is decided as long as the ports remain admin shut, whichever port is configured as no shutdown gets the first precedence.

    4. When odd port has 40G/100G and corresponding even port has 10G transceiver or vice versa, in admin shut status, if both ports are configured as no shutdown at the same time, then the port that was detected first by software gets precedence and the other gets xcvr mismatch state.

    5. When the state in step 4 is achieved, in this state, if switch is reloaded, during boot-up again the port which is detected first by software takes precedence and rest gets speed mismatch state.

Beginning with Cisco Nexus NX-OS Release 10.5(1), the following guidelines and limitations are applicable:

  • For ports 1-16, every pair of ports (1,2 | 3,4 | 5,6 | 7,8 | 9,10 | 11,12 | 13,14 | 15,16) is referred to as a quad group.

  • All the ports in a quad operate in 10G with QSA, or 40G or 100G or 200G

  • Mixed speed is not supported within the same quad with the following exceptions:

    • Mixed speed of 40G & 100G can be supported in quad

    • However, 100G-CR2 cannot be mixed with either 40G or other types of 100G optics in quad

  • Quad speed mismatch check runs on optics insertion and removal sequence

  • The first inserted transceiver in a quad group determines the speed of the quad group. Ports with any other unsupported speed is down with the reason of "XCVR speed mismatch. With unsupported mixed speeds, only one speed is up in a quad group at a time.

  • For a particular port to be up and functional, ensure to remove all the optics or cables from all the ports in that quad and plug in the optics or cables first in the port that needs to be up and then plug in the other optics or cables.

  • For a particular speed mismatch port to be up and functional, ensure to remove the optics or cables from all other ports in that quad, flap the needed port, and then plug in the other ports.

  • Save the config (copy running start-up) to have port states persistency.

  • When a Mismatch Transceiver is plugged into a quad, syslog is generated as Interface Ethernet1/X is down (Reason: Inserted transceiver speed mismatch with quad speed Y).

  • Port states may not be persistent on reload ascii. Port states depends on the order of interface detected sequence on reload ascii.

  • Ensure to only use the Transceivers of same speed in a quad to avoid any disruption or indeterministic state.

Access Ports

An access port carries traffic for one VLAN. This type of port is a Layer 2 interface only.

For more information on access ports, see the “Information About Access and Trunk Interfaces” section.

Routed Ports

A routed port is a physical port that can route IP traffic to another device. A routed port is a Layer 3 interface only.

For more information on routed ports, see the “Routed Interfaces” section.

Management Interface

You can use the management Ethernet interface to connect the device to a network for remote management using a Telnet client, the Simple Network Management Protocol (SNMP), or other management agents. The management port (mgmt0) is autosensing and operates in full-duplex mode at a speed of 10/100/1000 Mb/s.

For more information on the management interface, see the Cisco Nexus 9000 Series NX-OS Fundamentals Configuration Guide.

Port-Channel Interfaces

A port channel is a logical interface that is an aggregation of multiple physical interfaces. You can bundle up to 32 individual links (physical ports) into a port channel to improve bandwidth and redundancy. For more information about port-channel interfaces, see the “Configuring Port Channels” section.

Subinterfaces

You can create virtual subinterfaces using a parent interface configured as a Layer 3 interface. A parent interface can be either a physical port or a port-channel. A parent interface can be a physical port. Subinterfaces divide the parent interface into two or more virtual interfaces on which you can assign unique Layer 3 parameters such as IP addresses and dynamic routing protocols.

Loopback Interfaces

A loopback interface is a virtual interface with a single endpoint that is always up. Any packet that is transmitted over a virtual loopback interface is immediately received by that interface. Loopback interfaces emulate a physical interface. For more information about subinterfaces, see the “Loopback Interfaces” section.

Breakout Interfaces

Cisco NX-OS supports the breakout of a high bandwidth interface into one or more low bandwidth interfaces at the module level or at the per-port level.

Module Level Breakout

For module level breakout, the interface breakout command splits the high bandwidth 40G interface of a module into four 10G interfaces. The module is reloaded and the configuration for the interface is removed when the command is executed.

The following is an example of the command: 

switch# configure terminal
switch(config)# interface breakout module 1
Module will be reloaded. Are you sure you want to continue(yes/no)? yes

The no interface breakout module module_number command undoes the breakout configuration. It puts all interfaces of the module in 40G mode and deletes the configuration for the previous 10G interfaces.

Dynamic Breakout (Per-port Level Breakout)

For dynamic breakout (per-port level breakout), the interface breakout command splits high bandwidth 40G ports into four 10G breakout ports and 100G ports into four 25G breakout ports. The breakout ports are identified as Ethernet <slot>/<front-panel-port>/<breakout-port> . For example, per-port level breakout ports can be identified as Ethernet 1/2/1, Ethernet 1/2/2, Ethernet 1/2/3, and Ethernet 1/2/4.

When one or more 40G interfaces of the module are broken-out at the per-port level, the configuration for the interfaces are removed when the command is executed.


Note

A per-port level breakout does not require the module to be reloaded.

The following is an example of configuring a breakout port: 


switch(config)# interface breakout module 1 port 1 map 10g-4x
switch(config)#

The following is an example of configuring a multiple breakout ports: 


switch(config)# interface breakout module 1 port 1-4 map 10g-4x
switch(config)#

The following is an example of configuring 40G and 10G interfaces mixed together.


switch(config-if)# show int eth1/49 transceiver 
Ethernet1/49
transceiver is present
type is QSFP-40G-SR-BD
name is CISCO-AVAGO 
part number is AFBR-79EBPZ-CS2 
revision is 01

switch(config-if)# show int eth1/52 transceiver
Ethernet1/52
transceiver is present
type is QSFP-Cazadero
name is CISCO-DNI 
part number is CAZADERO-R 
revision is 03
nominal bitrate is 10000 MBit/sec per channel

switch(config-if)# show int eth1/53 transceiver 
Ethernet1/53
transceiver is present
type is QSFP-Cazadero
name is CISCO-DNI 
part number is CAZADERO-R 
revision is 03
nominal bitrate is 10000 MBit/sec per channel

switch(config)# interface breakout module 1 port 52-53 map 10g-4x 

switch(config-if)# show int br | i up
mgmt0 -- up 10.122.160.192 100 1500
Eth1/49 -- eth routed up none 40G(D) — << Running 40G
Eth1/50 -- eth routed up none 40G(D) --
Eth1/52/1 -- eth routed up none 10G(D) — << Broken out to 10G 
Eth1/53/1 -- eth routed up none 10G(D) -- << Broken out to 10G

You can undo the breakout ports with the no interface breakout command.

The following is an example to undo the breakout ports: 


switch(config)# no interface breakout module 1 port 1 map 10g-4x
switch(config)#

About the Lane Selector

The lane selector is a push button switch and 4 LEDs located on the Cisco Nexus switch (left side of front panel, labeled "LS"). The push button switch and LEDs are used to indicate the status of the ports. The lane selector is supported on Cisco Nexus Series 9000 series switches and the Cisco Nexus 3164 and 3232 switches.

By default, the LEDs indicate the link/activity status of a 1 x 40G configuration. When the ports are configured as 4 x 10G, you can access the link status of each individual 10G port with the lane selector.

By pressing the lane selector push button, the port LED shows the selected lane’s link/activity status. The 1st time the push button is pressed, the first LED displays the status of the first port. Pressing the push button a 2nd time displays the status of the second port, and so on. You can display the status of each of the four ports by pressing the push button in this manner.

For example, if port 60 is configured as 4 x 10G, pressing the lane selector push button once displays the link status of 60/1/1. Pressing the push button a second time displays the link status of 60/1/2.

When you press the push button after displaying the status of the last port, all four of the LEDs should extinguish to indicate that the lane selector has returned to display the status for the default 1 x 40G configuration.


Note

A 10G breakout port's LED blinks when the beacon feature has been configured for it.


Note

When a port is configured to be in 10G breakout mode and no lane is selected, the 40G port's LED illuminates as green even though only one of the 10G breakout ports is up.

Notes About Breakout Interfaces

Cisco Nexus 9516 switch does not support breakout on Modules 8 to 16.

Beginning with Cisco NX-OS Release 7.0(3)F2(1), the 36-port 100-Gigabit Ethernet QSFP28 line cards (N9K-X9636C-R) and 36-port 40-Gigabit Ethernet QSFP+ line cards (N9K-X9636Q-R) provide 4x10-Gigabit support.

Beginning with Cisco NX-OS Release 9.2(1), N9K-9636C-R, N9K-X9636Q-R and N9K-X9636C-RX line cards support breaking out 40G ports into 4x10 Gigabits.

Beginning with Cisco NX-OS Release 9.2(2), the following transceivers are supported on Cisco Nexus 9636C-R and 9636C-RX switches:

  • QSFP-4SFP25G-CU1M

  • QSFP-4SFP25G-CU2M

  • QSFP-4SFP25G-CU3M

  • QSFP-4SFP25G-CU5M

Beginning with Cisco NX-OS Release 9.2(2), SFP-25G-SR-S transceiver is supported on the Cisco Nexus 96136YC-R line card.

Beginning with Cisco NX-OS Release 9.2(2), N9K-X9636C-R and N9K-X9636C-RX line cards support breaking out 100G ports into 4x25 Gigabits. From Cisco NX-OS Release 9.3(3), the default FEC mode on N9K-X9636C-R and N9K-X9636C-RX is FC-FEC for 25Gx4 and 50Gx2. because RS-FEC is not supported. When you connect N9K-X9636C-RX to N9K-X9636C-R, you must configure FC-FEC on N9K-X9636C-RX. The N9K-C9636C-R does not support RS-FEC and the N9K-X96136YC-R line card does not support breakout.

The Cisco Nexus 9364C-GX switch is a 2-rack unit (RU) top-of-rack with fixed-port switch offering 64 100-Gigabit SFP28 ports, 2 management ports and 1 USB port. The Cisco Nexus 9364C-GX switch supports breakout feature and it supports all transceivers that are supported by Cisco Nexus 9000 series switches. Each QSFP28 port operates in 100G and 4x25G modes; 40G or 4x10G modes. It also supports Cisco QSA native 10G speed, and support SFP, SFP+ modules and one console port (RS-232).

The Cisco Nexus 93600CD-GX switch is a 1-rack unit (RU) fixed-port switch offering 28 40/100-Gigabit QSFP28 ports, 8 100/400-Gigabit QSFP-DD ports, 2 management ports, 1 console port (RS-232), and 1 USB port. The Cisco Nexus 93600CD-GX switch supports breakout feature.

Beginning with Cisco NX-OS Release 9.3(3), Cisco Nexus 9500 R-Series switches support breaking out 100G ports into 2x50 Gigabits.

The following table provides detailed information of the supported or not supported breakout modes. For more information, see Cisco Nexus Data Sheets:

Table 2. Breakout Modes Support Matrix

Switches

4x10G

4x25G

2x50G

N9K-X9636C-RX

Yes

Yes

Yes

N9K-X9636C-R

Yes

Yes

Yes

N9K-X9636Q-R

Yes

No

No

N9K-X96136YC-R

No

No

No

N9K-93108TC-EX

Yes

Yes

Yes

N9K-93180YC-EX

Yes

Yes

Yes

N9K-93108TC-FX

Yes

Yes

Yes

N9K-93180YC-FX

Yes

Yes

Yes

N9K-9348GC-FXP

Yes

Yes

Yes

N3K-C3636C-R

Yes

Yes

Yes

N3K-C36180YC-R

Yes

Yes

Yes

N9K-93108TC-FX3P

Yes

Yes

Yes

N9K-C9364D-GX2A

Yes

Yes

Yes

N9K-X9836DM-A

Yes

Yes

Yes

Note

100G ports into 2x50G breakout is only supported on QSFP-100G-PSM4 and QSFP-100G-AOC (1M-30M) transceivers.

Note

On Nexus 9500 R-Series switches with N9K-X9636C-R and N9K-X9636C-RX line cards, limited optics (QSFP-100G-PSM4-S, QSFP-100G-AOC, QSFP-100G-CU1M -CU3M) are supported with 2x50G and 4x25G breakout. For more information see Cisco Optics-to-Device Compatibility Matrix.

Caveats

  • As of Cisco NX-OS Release 7.0(3)I7(2), manual breakout of QSA ports is not supported.

    Manual breakout is supported on the following platforms because auto-breakout does not happen successfully on them—N9K-C93128TX, N9K-9332, N9K-C9396PX, N9K-C9396TX, N9K-C9372PX, N9K-C9372TX, N9K-C9332PQ, N9K-C93120TX, N9K-9432PQ, N9K-9536PQ, N9K-9636PQ, N9K-X9632PC-QSFP100, N9K-X9432C-S, N3K-C3132Q-V, N3K-C3164Q, N3K-C3132C, N3K-C3232C, N3K-C3264Q, N3K-C3264C, N3K-3064Q, N3K-3016, N3K-3172.

    You need to perform manual breakout using "interface breakout module <module number> port <port range> map <breakout mapping>" command.

  • When configuring a break-out port as a part of a port-channel, you need to re-apply the configuration to avoid the port-channel being created without members.

    When a break-out port is configured as a part of a port-channel, you need to re-apply the configuration to assure the effectiveness of the port-channel.

    When a break-out port is configured as a part of a port-channel, you need to apply the configuration twice (after write-erase/reload), to ensure the effectiveness of the port-channel. Refer to CSCuz47922.

  • When you upgrade a Cisco Nexus 9000 device to Cisco NX-OS Release 7.0(3)I7(2) or later releases, if a QSFP port is configured with the manual breakout command and is using a QSA, the configuration of the interface Ethernet 1/50/1 is no longer supported and will need to be removed. To restore the configuration, you must manually configure the Ethernet 1/50 on the device.

    This behaviour is not applicable to the following platforms—N9K-C93128TX, N9K-9332, N9K-C9396PX, N9K-C9396TX, N9K-C9372PX, N9K-C9372TX, N9K-C9332PQ, N9K-C93120TX, N9K-9432PQ, N9K-9536PQ, N9K-9636PQ, N9K-X9632PC-QSFP100, N9K-X9432C-S, N3K-C3132Q-V, N3K-C3164Q, N3K-C3132C, N3K-C3232C, N3K-C3264Q, N3K-C3264C, N3K-3064Q, N3K-3016, N3K-3172—because manual breakout is supported on these platforms.

  • Cisco Nexus 9000 Series switches have 40G ports. When you breakout one of the 40G ports into 4x10G ports with a QSFP breakout cable, not all of the subinterfaces can be added to a port channel. The following error message is displayed: 

    
switch# channel-group 99 mode active
command failed: port not compatible [Buffer boost]

    Note

    As a workaround, configure no buffer-boost on all of the subinterfaces. This enables the channel-group configuration to go through.

    Using the force keyword on the port-channel does not enable all the interfaces to be added to the port-channel even though the error message suggests using the keyword.

  • Cisco Nexus 9000 switches defaults to FC-FEC mode while the FC-FEC mode is turned off on N9K-X9636C-RX line cards, by default. To enable interoperability between two such devices, you may need to manually change the FEC settings to match with each other.

  • Beginning with Cisco NX-OS Release 7.0(3)I7(3) you see two additional options to configure FEC such as rs-cons16 and rs-ieee as per IEEE standards.

  • Beginning with Cisco NX-OS Release 7.0(3)I7(7) you can display the admin and oper status of FEC interface information with the show interface fec command.

    Example: 
    
switch# show interface fec
---------------------------------------------------------------------------------------------------
Name		Ifindex	Admin-fec 	Oper-fec    Status 	Speed 	Type
---------------------------------------------------------------------------------------------------
Eth1/1  	0x1a000000	auto		auto	connected     10G	 SFP-H10GB-AOC2M
Eth1/2  	0x1a000200   			Rs-fec    notconneced    auto	QSFP-100G-AOC3M
Eth1/3/1	0x38014000	auto		auto	disabled	auto	QSFP-H40G-AOC3M
Eth1/3/2	0x38015000	auto		auto	disabled	auto	QSFP-H40G-AOC3M
Eth1/3/3	0x38016000	auto		auto	disabled	auto	QSFP-H40G-AOC3M
Eth1/3/4	0x38017000	auto		auto	disabled	auto	QSFP-H40G-AOC3M

Note

Auto-FEC is not supported in Cisco NX-OS Release 7.0(3)I7(x)

When configuring a break-out port, ensure that the FEC is matching for the link to be up.
High Bandwidth Interfaces

The breakout of high bandwidth interfaces are supported only on:

The breakout of high bandwidth interfaces (module level or per-port level) are supported only on:

  • The X9636PQ, X9432PQ, X9536PQ, and X9732C-EX line cards on a Cisco Nexus 9500 Series switch.

  • The Cisco Nexus 9332PQ switch.

  • The Cisco Nexus 3164Q switch.

Cisco Nexus C92160YC Switch

For 7.0(3)I3(1) and later, the Cisco Nexus C92160YC switch provides two different modes of operation:

  • Mode 1: 48 * 10G/25G + 4 * 40G + 2 * 100G (Default configuration)

    • Hardware profile portmode 48x25G + 2x100G + 4x40G

    • Breakout is supported in 2 * 100G ports

  • Mode 2: 48 * 10G/25G + 4 * 100G

    • Hardware profile portmode 48x25G + 4x100G

    • Breakout is supported on 3 * 100G ports (Ports 50,51 & 52).

Use the show running-config | grep portmode command to display the current operation mode.

Example: 


switch(config-if-range)# show running-config | grep portmode

hardware profile portmode 48x25G+2x100G+4x40G

See the installation guide for the Cisco Nexus C92160YC switches for more information. ( Install and Upgrade Guides for Cisco Nexus 9000 Series Switches )

With the Cisco Nexus C92160YC switch , there are two breakout modes:

  • 40G to 4x10G breakout ports

    • Enables the breakout of 40G ports into 4 X 10G ports.

    • Use the interface breakout module 1 port x map 10g-4x command.

  • 100G to 4x25G breakout ports

    • Enables the breakout of 100G ports into 4 X 25G ports.

    • Use the interface breakout module 1 port x map 25g-4x command.

Cisco Nexus C9272Q Switch

For 7.0(3)I3(1) and later, the Cisco Nexus C9272Q Switch provides 72 40G ports. Ports 38 - 72 support breakout interfaces.

For 7.0(3)I3(1) and later, the Cisco Nexus C9272Q Switch provides 72 40G ports. Ports 37 - 71 support breakout interfaces.

To configure a breakout interface, use the interface breakout module 1 port x map 10g-4x command.

Example: 


switch(config)# interface breakout module 1 port 38 map 10g-4x
switch(config)# show interface ethernet 1/38 capabilities | grep -i break

      Breakout capable:      yes
Cisco Nexus C9332PQ Switch

For 7.0(3)I3(1) and later, the Cisco Nexus C9332PQ Switch provides 24 40G ports that support breakout mode and can be connected to four 10G NIF ports on a FEX. Ports 1 - 12 and 15 - 26 ports are supported. (Ports 13 and 14 ports are reserved and cannot be used for breakout mode.)


Note

All FEXs are supported.


Note

Only the Cisco Nexus 9332PQ switch provides interface breakout support for FEX fabric interfaces. (7.0(3)I3(1) and later)

Cisco Nexus 9000 C93180LC-EX Switch

For 7.0(3)I7(1) and later, Cisco Nexus 9000 C93180LC-EX switch provides three different modes of operation:

  • Mode 1: 28 x 40G + 4 x 40G/100G (Default configuration)

    • Hardware profile portmode 4x100g + 28x40g.

    • 10x4 breakout is supported on the top ports from 1 to 27 (ports 1,3,5, 7...27). If any of the top port is broken out, the corresponding bottom port becomes non-operational. For example, if port 1 is broken out port 2 becomes non-operational.

    • 1 Gigabit and 10 Gigabit QSA is supported on ports 29, 30, 31, and 32. However, QSAs on the top and bottom front panel ports must be of same speed.

    • Ports 29, 30, 31, and 32 support 10x4, 25x4, and 50x2 breakout.

  • Mode 2: 24 x 40G + 6 x 40G/100G

    • Hardware profile portmode 6x100g + 24x40g.

    • 10x4 breakout is supported on the top ports from 1 to 23 (ports 1,3,5, 7...23). If any of the top port is broken out the corresponding bottom port becomes non-operational.

    • Ports 25, 27, 29, 30, 31, and 32 support 10x4, 25x4, and 50x2 breakout.

    • 1 Gigabit and 10 Gigabit QSA is supported on ports 29, 30, 31, and 32. However, QSAs on the top and bottom front panel ports must be of same speed.

  • Mode 3: 18 x 40G/100G

    • Hardware profile portmode 18x100g.

    • 10x4, 25x4, and 50x2 breakout is supported on top ports from 1 to 27 (ports 1,3,5, 7...27) and on ports 29,30,31,32.

    • 1 Gigabit and 10 Gigabit QSA is supported on all the 18 ports.

Changing Mode 3 to any other mode or vice versa requires copy running-config startup-config command followed by reload command to take effect. However, moving between Modes 1 and 2 is dynamic and requires only copy running-config startup-config command.

Use the show running-config | grep portmode command to display the current operation mode.

Example: 


switch(config-if-range)# show running-config | grep portmode

hardware profile portmode 4x100G+28x40G

With the Cisco Nexus C93180LC-EX switch, there are three breakout modes:

  • 40G to 4x10G breakout ports

    • Enables the breakout of 40G ports into 4 X 10G ports.

    • Use the interface breakout module 1 port x map 10g-4x command.

  • 100G to 4x25G breakout ports

    • Enables the breakout of 100G ports into 4 X 25G ports.

    • Use the interface breakout module 1 port x map 25g-4x command.

  • 100G to 2x50G breakout ports

    • Enables the breakout of 100G ports into 2 X 50G ports.

    • Use the interface breakout module 1 port x map 50g-2x command.

Cisco Nexus 9000 C9364C-GX Switch

Cisco Nexus N9K-C9364C-GX breakout considerations:

  • For ports 1-64, 2 x 50G, 4 x 25G and 4 x 10G breakout is supported only on odd numbered ports.

  • When an odd numbered port in a quad is broken out, the even ports in that quad are removed and the other odd port in the same quad is broken out automatically to the same speed. For example, if port 1 or port 3 is broken out into 2 x 50, 4 x 25G or 4 x 10G, then the other odd port in that quad is automatically broken out to same speed and ports 2 and 4 in that quad are removed. When the above breakout configuration is removed, all ports in that quad revert to default.

  • QSFP28 (100G) transceivers support the 4 x 25G breakout feature. Beginning Cisco NX-OS Release 9.3(5), the 2 x 50G breakout feature is supported.

  • QSFP+ (40G) transceivers support the 4 x 10G breakout feature.

The Cisco Nexus 9364C-GX switch that has 64 100-Gigabit SFP28 ports, support the breakout feature. Each QSFP28 port operates in 100G or 4x25G modes; 40G or 4x10G modes. It also supports Cisco QSA native 10G speed with SFP+ modules. Of the ports 1-64, every four ports (1-4, 5-8, 9-12, and so on, referred to as a "quad") operates at a fixed speed. All the ports in a quad operate in 10G, or 40G or 100G. This switch does not support mixed speeds in a quad.

  • 100G to 2x50G breakout ports

    • Enables the breakout of 100G ports into 2 X 50G ports on all odd ports.

    • Use the interface breakout module 1 port x map 50g-2x command.

  • 100G to 4x25G breakout ports

    • Enables the breakout of 100G ports into 4 X 25G ports.

    • Use the interface breakout module 1 port x map 25g-4x command

  • 40G to 4x10G breakout ports

    • Enables the breakout of 40G ports into 4 X 10G ports.

    • Use the interface breakout module 1 port x map 10g-4x command.


Note

  • 25Gx4x and 10Gx4x breakout modes are only applicable on odd ports. If odd ports in a quad are broken out into 25Gx4x or 10Gx4x, the even ports in that quad disappear.

  • For example, in a quad, if the port-1 is broken out into 4x25G or 4x10G, then the port-3 automatically configures itself the breakout configuration implemented on port-1. Similarly, if the port-3 is broken out into 4x25G or 4x10G, then port-1 automatically configures itself the breakout configuration implemented on port-3.

Cisco Nexus 9000 C93600CD-GX Switch

Cisco Nexus N9K-C93600CD-GX breakout considerations:

  • Port s1-24 do not support the breakout feature.

  • Ports 25-28 support 4 x 25G and 4 x 10G. Port 25-26 should have the same speed/breakout configured and port 27-28 should have same speed/breakout configured. Mismatch of breakout speed on ports 25-26 or 27-28 is not supported.

  • When the port 25 is broken out, port 26 is automatically broken out to same speed as port 25 and vice versa. When the above breakout configuration is removed, ports 25 and 26 will be reverted to default. The same behavior applies for port 27 and 28.

  • Beginning Cisco NX-OS Release 9.3(5), 2 x 50G breakout is supported on ports 1-36.

  • 4x25 and 4x10 breakout is supported on odd ports between ports 1 through 24. The even ports will be purged within a quad (4 ports).

  • Within a quad, the breakout configuration and the speed must be same. The six quads are ports 1-4, 5-8, 9-12, 13-16, 17-20 and 21-24.

  • Ports 25-28 support 4 x 10G, 4 x 25G and 2 x 50G breakout features. These breakout feature are supported in port pairs - for example 25-26 and 27-28.

  • Ports 29-36 breakout consideration:

    • QSFPDD 400G Copper transceivers support only 4 x 100G, 2x 200G, and 4 x 50G breakout features.

    • QSFP-DD-400G-DR4 transceivers support only the 4 x 100G breakout feature.

    • QSFP-DD-400G-FR4 and QSFP-DD-400G-LR8 transceivers do not support the breakout feature.

    • QSFP28 (100G) transceivers support the 2 x 50G and 4 x 25G breakout features.

    • QSFP+ (40G) transceivers support the 4 x 10G breakout feature.

  • Port s1-24 do not support the breakout feature.

  • Ports 25-28 support 4 x 25G and 4 x 10G. Port 25-26 should have the same speed/breakout configured and port 27-28 should have same speed/breakout configured. Mismatch of breakout speed on ports 25-26 or 27-28 is not supported.

  • When the port 25 is broken out, port 26 is automatically broken out to same speed as port 25 and vice versa. When the above breakout configuration is removed, ports 25 and 26 will be reverted to default. The same behavior applies for port 27 and 28.

  • In Cisco Nexus N9K-C93600CD-GX, every 4 ports from 1 through 24 are referred to as a quad. The breakout configuration and the speed must be same within a quad. The breakout feature may not function as expected if there is a mismatch of speed or breakout configuration within a quad. The six quads are made of ports 1-4, 5-8, 9-12, 13-16, 17-20 and 21-24.

  • Beginning Cisco NX-OS Release 9.3(5), 2x50G breakout is supported on ports 1-36.

  • 4x25G and 4x10G breakout is supported only on odd ports, between ports 1 through 24. The even ports will be purged within a quad (4 ports).

  • When an odd-numbered port in a quad is broken out, the even ports in that quad are removed and the other odd ports within the quad is broken out automatically to the same speed. For example, if port 1 is broken out into 4x25G or 4x10G, then the other odd port in that quad is automatically broken out to same speed; and ports 2 and 4 in that quad are removed. When this breakout configuration is removed, all ports in that quad reverts to the default configuration.

  • 2x50G breakout is supported on all ports from 1 through 24. All ports in a quad are broken out automatically to same speed when one port in a quad is broken out to 2x50G. For example when Port 2 is broken out into 2x50G, ports 1,3, and 4 are automatically broken out into 2x50G.


    Note

    Only RS-FEC is supported on both lanes for 50G speed on ports 1 through 24

  • Beginning with Cisco NX-OS Release 9.3(3) ports 25-28 support 4x10G, 4x25G, and 2x50G breakout features. These breakout feature are supported in port pairs. - for example 25-26 and 27-28.


    Note

    Lane 2 of 2x50G should be configured with RS-FEC for link to be up.

  • Beginning with Cisco NX-OS Release 9.3(3), consider the following breakout configuration for ports 29-36:

    • QSFP-DD-400G-DR4 transceivers support only the 4 x 100G breakout feature.

    • QSFP-DD-400G-FR4 and QSFP-DD-400G-LR8 transceivers do not support the breakout feature.

    • QSFP28 (100G) transceivers support the 2 x 50G and 4 x 25G breakout features.

    • QSFP+ (40G) transceivers support the 4 x 10G breakout feature.


Note

  • Breakout modes are not supported on ports 1-24. On Ports 1-24, every four ports are referred as quad (ports 1-4, 5-8, 9-12 etc.). Ports in the same quad operate at a fixed speed, all four ports should either be 40G or 100G. With Cisco QSA with SFP+ all four ports can operate in 10G speed.

  • 2x50G, 4x25G, and 4x10G are the breakout modes supported on ports 25-28. Ports 25 and 26 must be of the same speed. Similarly, ports 27 and 28 must be of the same speed. For example, if a Cisco QSA with SFP+ is plugged into port 25, port 26 must also have QSA with SFP+. A transceiver of a different speed cannot be plugged into port 26. Similarly, if port 25 is broken out into a particular speed, port 26 is automatically broken out to the same speed as port 25. The same applies to ports 27 and 28 as well.

  • On ports 29-36, speed is not restricted. Each port can operate in 10G (Cisco QSA with SFP+), 40G, 100G, 400G, 4x10G, 2X50G, 4x100G, 2x200G, and 4x50G speeds.

  • For 50Gx2 breakup mode, on ports 25-29, the default FEC type is FC-FEC. The second breakout lane may not link up with FC-FEC. You must configure RS-FEC on the interface for the second lane to link up.

  • On ports 1-36, Cisco QSA with SFP+ in 10G speed is supported.

The following breakout modes are supported on Cisco Nexus 93600CD-GX switches:

  • 400G to 4x100G breakout ports

    • Enables the breakout of 400G ports into 4 X 100G ports.

    • Use the interface breakout module 1 port x map 100g-4x command.

  • 100G to 2x50G breakout ports

    • Enables the breakout of 100G ports into 2 X 50G ports.

    • Use the interface breakout module 1 port x map 50g-2x command.

  • 100G to 4x25G breakout ports

    • Enables the breakout of 100G ports into 4 X 25G ports.

    • Use the interface breakout module 1 port x map 25g-4x command

  • 40G to 4x10G breakout ports

    • Enables the breakout of 40G ports into 4 X 10G ports.

    • Use the interface breakout module 1 port x map 10g-4x command.

Cisco Nexus 9000 C9316D-GX Switch

Cisco Nexus N9K-C9316D-GX breakout considerations:

  • Port 1-16 breakout consideration:

    • QSFP-DD-400G-DR4 transceivers support only the 4 x 100G and 4x10G breakout feature.

    • QSFP-DD-400G-FR4 and QSFP-DD-400G-LR8 transceivers do not support the breakout feature.

    • QSFP28 (100G) transceivers support the 2 x 50G, 4 x 25G, and 4x10G breakout feature.


Note

For 50G x 2 breakup mode, on ports 25-29, the default FEC type is FC-FEC. The second breakout lane may not link up with FC-FEC. You must configure RS-FEC on the interface for the second lane to link up.

Disable USB Ports on Cisco NX-OS Switches

The USB ports on the Cisco NX-OS switches are enabled by default. You can disable or enable all the USB ports using the following commands:

To disable USB port(s), use the following command:

switch(config)# port usb disable

This command requires a reload. Use the Copy running-config startup-config command and reload the switch. 

switch(config)#
switch(config)# sh running-config | i usb
port usb disable
switch(config)#

To enable USB port(s) if previously disabled, use the following command:

switch(config)# no port usb disable

This command requires a reload. Use the Copy running-config startup-config command and reload the switch. 

switch(config)#
switch(config)# sh running-config | i usb
switch(config)#

Note

For changes to take effect on already inserted USB, it is recommended to use the Copy running-config startup-config command and reload the switch.

Virtual Device Contexts

Cisco NX-OS can segment operating system and hardware resources into virtual device contexts (VDCs) that emulate virtual devices. The Cisco Nexus 9000 Series switch does not support multiple VDCs. All switch resources are managed in the default VDC.

High Availability for Interfaces

Interfaces support stateful and stateless restarts. A stateful restart occurs on a supervisor switchover. After the switchover, Cisco NX-OS applies the runtime configuration.