Table of Contents

61_nxos_rn.fm

Cisco Nexus 9000 Series NX-OS Release Notes, Release 6.1(2)I2(1)

This document describes the features, caveats, and limitations for Cisco NX-OS Release 6.1(2)I2(1) software for use on the Cisco Nexus 9000 Series switches. Use this document in combination with documents listed in the Related Documentation.

shows the online change history for this document.

Table 1 Online History Change

Part Number

Revision

Date

Description

OL-31713-01

A0

March 17, 2014

Created the release notes for Release 6.1(2)I2(1).

B0

March 27, 2014

Updated the “Upgrade Instructions” section.

C0

August 29, 2014

Added the “Software Maintenance Upgrades” section.

D0

October 16, 2014

Updated hardware information for the Cisco Nexus 93128TX switch.

E0

February 19, 2015

- Added an NLB limitation in the “Limitations” section.

- Added the bug ID CSCuq03168 to “Open Caveats—Cisco NX-OS Release 6.1”

F0

February 23, 2015

Added new limitations in “Limitations”.

G0

May 4, 2015

Added a new line to explain Cisco ALE port limitation in the “Limitations” section.

Contents

This document includes the following sections:

- Introduction

- System Requirements

- New and Changed Information

- Installation Notes

- Upgrade Instructions

- Software Maintenance Upgrades

- Limitations

- Unsupported Features

- Caveats

- Related Documentation

- Obtaining Documentation and Submitting a Service Request

Introduction

Cisco NX-OS software is a data center-class operating system designed for performance, resiliency, scalability, manageability, and programmability at its foundation. The Cisco NX-OS software provides a robust and comprehensive feature set that meets the requirements of virtualization and automation in mission-critical data center environments. The modular design of the Cisco NX-OS operating system makes zero-impact operations a reality and enables exceptional operational flexibility.

The Cisco Nexus 9000 Series uses an enhanced version of Cisco NX-OS software with a single binary image that supports every switch in the series, which simplifies image management.

System Requirements

This section includes the following topics:

- Supported Device Hardware

- Supported Optics

Supported Device Hardware

lists the Cisco Nexus 9000 Series hardware that Cisco NX-OS Release 6.1(2)I2(1) supports.

Table 2 Cisco Nexus 9000 Series Hardware

Product ID

Hardware

Quantity

N9K-C9508

Cisco Nexus 9508 8-slot chassis

1

N9K-C9508-FM

Cisco Nexus 9500 Series fabric module

6

N9K-X9564PX

Cisco Nexus 9500 Series 48-port, 1-/10-Gbps SFP+ plus 4-port QSFP+ I/O module

Up to 4 in the Cisco Nexus 9504

N9K-X9564TX

Cisco Nexus 9500 Series 48-port, 1-/10-Gbps BASE-T plus 4-port QSFP+ I/O module

Up to 4 in the Cisco Nexus 9504

N9K-X9636PQ

Cisco Nexus 9500 36-port, 40 Gigabit Ethernet QSPF aggregation module

Up to 4 in the Cisco Nexus 9504

N9K-SC-A

Cisco Nexus 9500 Series System Controller Module

2

N9K-SUP-A

Cisco Nexus 9500 Series supervisor module

2

N9K-C9508-FAN

Cisco Nexus 9508 fan trays

3

N9K-PAC-3000W-B

Cisco Nexus 9500 Series 3000 W AC power supply

Up to 4 in the Cisco Nexus 9504

N9K-C9504

Cisco Nexus 9504 4-slot chassis

1

N9K-C9504-FM

Cisco Nexus 9504 fabric module

6

N9K-C9504-FAN

Cisco Nexus 9504 fan trays

3

N9K-C9396PX

Cisco Nexus 9300 switch with 48 1-/10-Gigabit Ethernet SFP+ and 12 40-Gigabit Ethernet QSPF switch

1

N9K-C93128TX

Cisco Nexus 9300 switch with 96 1-/10-Gigabit BASE-T ports and eight 40-Gigabit Ethernet QSPF ports (The 1-/10-Gigabit BASE-T ports also support a speed of 100 Megabits.)

1

N9K-M12PQ

Cisco Nexus 9300 uplink module, 12-port, 40 Gigabit Ethernet QSPF

1 (required)

N9K-PAC-650W1

Cisco Nexus 9300 650 W AC power supply, hot air out (red)

2 or less

Cisco Nexus 9300 650 W AC power supply, cold air in (blue)

2 or less

N9K-PAC-1200W2

Cisco Nexus 9300 1200 W AC power supply, hot air out (red)

2 or less

Cisco Nexus 9300 1200 W AC power supply, cold air in (blue)

2 or less

Cisco Nexus 9300 fan 1, hot air out (red)

3

Cisco Nexus 9300 fan 1, cold air in (blue)

3

Cisco Nexus 9300 fan 2, hot air out (red)

3

Cisco Nexus 9300 fan 2, cold air in (blue)

3

1.For use with the Cisco Nexus 9396 switch (N9K-C9396PX).

1.For use with the Cisco Nexus 9396 switch (N9K-C9396PX).

2.For use with the Cisco Nexus 93128 switch (N9K-C93128TX).

2.For use with the Cisco Nexus 93128 switch (N9K-C93128TX).

Supported Optics

.

Table 3 Transceivers and Cables

Product ID

Transceivers and Cables

QSFP-40G-SR4

40GBASE-SR4 QSFP+ transceiver module for MMF, 4-lanes, 850-nm wavelength, 12-fiber MPO/MTP connector

QSFP-40G-CSR4

40GBASE-CSR4 QSFP+ transceiver module for MMF, 4-lanes, 850-nm wavelength, 12-fiber MPO/MTP connector, 300-m reach with OM3 fiber

QSFP-40G-SR-BD

QSFP+ bidirectional transceiver module, duplex multimode fiber, LC duplex connector, 100-m reach with OM3 fiber

QSFP-40GE-LR4

40GBASE-LR4 QSFP+ 40G transceiver module for single mode fiber, 4 CWDM lanes in 1310-nm window muxed inside module, duplex LC connector, 10-km, 40G Ethernet rate only

QSFP-4x10G-AC7M

40GBASE-CR4 QSFP+ to four 10GBASE-CU SFP+ direct attach breakout cable assembly, 7 meter active

QSFP-4x10G-AC10M

40GBASE-CR4 QSFP+ to four 10GBASE-CU SFP+ direct attach breakout cable assembly, 10 meter active

QSFP-H40G-CU5M

40GBASE-CR4 QSFP+ direct-attach copper cable, 5 meter passive

QSFP-H40G-CU3M

40GBASE-CR4 QSFP+ direct-attach copper cable, 3 meter passive

QSFP-H40G-CU1M

40GBASE-CR4 QSFP+ direct-attach copper cable, 1 meter passive

QSFP-H40G-ACU7M

40GBASE-CR4 QSFP+ direct-attach copper cable, 7 meter active

QSFP-H40G-ACU10M

40GBASE-CR4 QSFP+ direct-attach copper cable, 10 meter active

SFP-10G-SR

10GBASE-SR SFP+ module

SFP-10G-LR

10GBASE-LR SFP+ module

SFP-H10GB-CU1M

10GBASE-CU SFP+ cable 1 meter

SFP-H10GB-CU3M

10GBASE-CU SFP+ cable 3 meter

SFP-H10GB-CU5M

10GBASE-CU SFP+ cable 5 meter

SFP-H10GB-ACU-7M

Active Twinax cable assembly, 7 meter

SFP-H10GB-ACU-10M

Active Twinax cable assembly, 10 meter

GLC-T

1000BASE-T SFP

GLC-SX-MM

GE SFP, LC connector SX transceiver

GLC-LH-SM

GE SFP, LC connector LX/LH transceiver

Note For the current release, if you are using the four 10G breakout cables with a Cisco Nexus 9000 Series switch, all ports on the I/O module must be set to breakout mode. A maximum of three l/O modules can be placed in breakout mode.

New and Changed Information

This section lists the new and changed features in Release 6.1(2)I2(1), and includes the following topics:

- New Hardware Features in Cisco NX-OS Release 6.1(2)I2(1)

- New Software Features in Cisco NX-OS Release 6.1(2)I2(1)

New Hardware Features in Cisco NX-OS Release 6.1(2)I2(1)

The Cisco NX-OS Release 6.1(2)I2(1) supports the following new hardware:

- The Cisco Nexus 9504 switch chassis (N9K-C9504) that supports:

– Fabric modules—up to six fabric modules (N9K-C9504-FM) behind the fan trays

– Supervisor modules—up to two supervisor modules (N9K-SUP-A)

– System controllers—up to two system controller modules (N9K-SC-A)

– 36-port 40-Gbps QSFP+ aggregation I/O module (N9K-X9636PQ)

– Fan trays—three (N9K-C9504-FAN)

– AC power supplies (up to four 3-kW AC power supplies (N9K-PAC-3000W-B)

- The Cisco Nexus 9396PX switch (N9K-C9396PX) is a two-rack-unit (RU) switch that supports 960 Gbps of bandwidth across 48 fixed 10-Gbps SFP+ ports and 12 40-Gbps QSFP+ ports. The 40-Gbps ports are provided on the uplink module.

- The Cisco Nexus 93128TX switch (N9K-C93128TX) is a 3RU switch that supports 1.28 Tbps across 96 fixed 1-/10-Gbps BASE-T ports and eight 40-Gbps QSFP uplink ports. The 1-/10-Gigabit BASE-T ports also support a speed of 100-Megabits. The 40-Gbps ports are the leftmost eight ports provided on the uplink module.

- The Cisco Nexus 9300 uplink module (N9K-M12PQ) can be serviced and replaced by the user and is the same for both Cisco Nexus 9300 switches.

- The Cisco Nexus 9500 Series 48-port, 1-/10-Gbps SFP+ plus 4-port QSFP+ I/O module (N9K-X9564PX) is designed for use with Cisco Nexus 2000 Series Fabric Extenders and for mixed 10 and 40 Gigabit Ethernet aggregation. It supports both direct-attach 10 Gigabit Ethernet copper cabling and optical transceivers and offers 4 QFSP+ ports that provide 40 Gigabit Ethernet server access, uplink, and downlink capacity. This module can be used in application centric infrastructure (ACI) leaf configurations.

- The Cisco Nexus 9500 Series 48-port 1-/10-Gbps BASE-T plus 4-port QSFP+ I/O module (N9K-X9564TX) is designed for end-of-row (EoR) and middle-of-row (MoR) environments. It supports 100 Megabit Ethernet, 1 Gigabit Ethernet, and 10-Gbps BASE-T copper cabling connectivity for server access, and offers 4 QFSP+ ports that provide 40 Gigabit Ethernet server access, uplink, and downlink capacity. This module can be used in ACI leaf configurations.

- Hardware rate limiters for catch-all exception traffic, Layer 3 glean packets, and Layer 3 multicast data packets.

New Software Features in Cisco NX-OS Release 6.1(2)I2(1)

The Cisco NX-OS Release 6.1(2)I2(1) supports the software features listed in this section.

- VLANs

- Switch Virtual Interface (SVI), interface VLANs

- VLAN Trunking Protocol (VTP) transparent mode

- VLAN counters

- Spanning Tree Protocol (STP) features:

– STP Rapid PVST+ (Per VLAN Spanning Tree +)

– STP Multiple Spanning Tree (MST)

– STP Extensions, including bridge protocol data unit (BPDU) guard, Bridge Assurance, loop guard, and root guard

- Layer 2 features:

– L2 storm control

– L2 switch port interfaces

– L2 portchannels

- vPCs, including the following vPC features:

– Peer switch

– Peer Gateway

– ARP Synchronization

– ND Synchronization

– Reload delay

– IGMP join sync

- Hot Standby Router Protocol (HSRP)

- Virtual Router Redundancy Protocol (VRRP)

- IGMP Snooping

- Port access control lists (PACL), VLAN access control lists (VACL)

- Virtual Extensible Local Area Network (VXLAN) with a virtual port channel (vPC)

- VXLAN gateway functionality on Cisco Nexus 9300 Series switches.

- VXLAN bridging functionality on Cisco Nexus 9300 Series switches.

- Enhanced object tracking

- Access ports and trunk ports

- VLAN as a SPAN source (for Rx only)

- Software maintenance upgrades (SMUs)

- Cisco Data Center Network Manager (DCNM)

Installation Notes

Upgrade Instructions

Note A software downgrade from Cisco NX-OS Release 6.1(2)I2(1) to Cisco NX-OS Release 6.1(2)I1(1) is not supported.

A software upgrade from Cisco NX-OS Release 6.1(2)I1(1) to Cisco NX-OS Release 6.1(2)I2(1) will be disruptive.

Upgrade Procedure

If you have a Cisco Nexus 9000 Series switch that is running Cisco NX-OS Release 6.1(2)I2(1) build 6.1(2)I2(0.206) or any earlier build of 6.1(2)I2(0.xxx), follow the special upgrade steps presented here.

Note If you have a Cisco Nexus 9508 switch that is running the n9000-dk9.6.1.2.I1.1.bin image from November 18, 2013, you do not need to enter the write erase command in Step 5. We do recommend upgrading EPLD images, but it is not mandatory.

To determine the date of the software image on the switch, enter the dir bootflash: command or the show version command which shows the date the image was compiled.

image to bootflash.

Step 2 Change the boot variables to the NX-OS image by entering the following commands:

command to set the startup boot variables to the NX-OS image.

Step 4 Copy the running-configuration file to a backup-configuration file to ensure that you load the running configuration after you make the upgrade.

command. The boot variables remain set.

command.

and wait for the auto copy operation to finish.

Step 7 Wait 2 minutes after the reload for all modules to come online before proceeding to the next step.

command to upgrade the BIOS. The chassis will reboot at the completion of this step. Do not attempt to reboot or power off the chassis during this operation. Wait 2 minutes after the reload for all the modules to come online before proceeding to the next step.

command to upgrade the EPLD. The chassis will reboot automatically.

command to upgrade the golden EPLD. The chassis will reboot automatically.

Step 11 Restore the configuration that you saved in Step 4 from the bootflash. If this is a fresh install or upgrade, you do not need to restore the previous running configuration.

Software Maintenance Upgrades

Note If you perform a software maintenance upgrade (SMU) and later upgrade your device to a new Cisco NX-OS software release, the new image will overwrite both the previous Cisco NX-OS release and the SMU package file.

Limitations

This section lists limitations related to Cisco NX-OS Release 6.1(2)I2(1).

- The uplink module should not be removed from a Cisco 9300 switch that is running Cisco NX-OS Release 6.1(2)I2(1). The ports on the uplink module should be used only for uplinks.

- The Cisco ALE port is not able to strip the dot1q tag for native VLANs, so packets will carry the dot1q tag for native VLANs.

- The N9K-M12PQ GEM module front panel ports do not support Auto negotiation with copper cables.

- GOLD Port loopback tests are not supported.

- The ASIC Memory-NS test is not applicable for N9K-X9636PQ line card and will be removed in future releases for the N9K-X9636PW line card. The test is also shown incorrectly for the N9K-X9636PQ line card. The test is applicable only for the N9K-X9564PX and N9K-X9564TX line cards.

- 802.1Q user-priority tagged frames (VLAN 0 with a 802.1p CoS value in the VLAN tag) are dropped on Cisco ALE access port on TOR switches.

- A Cisco ALE trunk port sends out tagged packets on the native VLAN of the port. Normally, untagged packets are sent on the native VLAN.

- On the Cisco Nexus 9300 Series switches with the N9K-X9636PQ and N9K-X9636TX line cards, there is no support for PFC, and no WRED and ECN support.

- The N9K-M12PQ GEM module front panel ports do not support Auto negotiation with copper cables.

- Cisco NX-OS Release 6.1(2)I2(2b) supports flooding for Microsoft Network Load Balancing (NLB) unicast mode on Cisco Nexus 9500 Series switches but not on Cisco Nexus 9300 Series switches. NLB is not supported in max-host system routing mode. NLB multicast mode is not supported on Cisco Nexus 9500 or 9300 Series switches.

Note To work around the situation of Unicast NLB limitation, we can statically hard code the ARP and MAC address pointing to the correct interface. Please refer to bug ID CSCuq03168 in detail in the “Open Caveats—Cisco NX-OS Release 6.1” section.

- When routed ACL is applied to multiple SVIs (switched virtual interfaces) in the egress direction, TCAM resources are not shared.

- When VACL (VLAN ACL) is applied to multiple VLANs, TCAM resources are not shared.

- N9K hardware does not support range checks (layer 4 operators) in egress TCAM. Because of this, ACL/QoS policies with layer 4 operations-based classification need to be expanded to multiple entries in the egress TCAM. Egress TCAM space planning should take this limitation into account.

- If the same QOS policy and ACL is applied on multiple interfaces, the label will be shared only when the qos-policy is applied with the no-stats option.

- Limitations for ALE uplink ports are listed at the following URL:

Unsupported Features

This section lists features that are not supported in the current release.

VXLAN Features

This section lists VXLAN features that are not supported.

- VXLAN routing is not supported.

The default Layer 3 gateway for VXLAN VLANs should be provisioned on a different device.

- Switch virtual interface (SVI) is not supported on VXLAN VLANs.

- VXLAN Layer 3 uplinks are not supported on a nondefault virtual routing and forwarding (VRF) instance.

- Switched Port Analyzer (SPAN) Tx for VXLAN traffic is not supported for the access to the network direction.

- RACLs are not supported on Layer 3 uplinks for VXLAN traffic. Egress VACLs cannot be used on decapsulated packets in the network-to-access direction on the inner payload. As a best practice, use PACLs/VACLs for the access-to-network direction.

- QoS classification is not supported for VXLAN traffic in the network-to-access direction.

- The QoS buffer-boost feature is not applicable for VXLAN traffic.

- Access control list (ACL) and quality of service (QoS) for VXLAN traffic in the network-to-access direction is not supported.

- There is no uplink SVI support. As a best practice, use the Layer 3 port-channel uplinks/equal cost multi path (ECMP) uplinks instead.

- There is no native VLAN support for VXLAN. All traffic on VXLAN Layer 2 trunks need to be tagged.

- Consistency checkers are not supported for VXLAN tables.

- Just one network virtualization edge (NVE) interface is allowed on the switch.

- Because the NVE (VXLAN) process is not restartable, patching support is not supported for VXLAN.

- Per-VNI statistics and per-VTEP statistics are not supported. Only aggregate statistics are available.

- vPC type-1 consistency checkers are not supported for VXLAN configurations.

- Dynamic re-IP of an NVE tunnel is not supported. Tunnels must be shut down prior to live IP address changes.

VXLAN Topology Restrictions

- A device cannot be a VXLAN gateway mode (vxlan-vlan flows) and a VXLAN bridging mode (vxlan-vxlan flows) for the same multicast groups, which are also called the bud-node topology. As a best practice, use the device as either a bridging device or a gateway device, but not both.

- Due to bud node restrictions, a VXLAN tunnel endpoint (VTEP) cannot reach the rendezvous point (RP) through another VTEP. Because of this limitation, there can be no direct Layer 3 links between two VTEPs, unless one of the VTEPs is the RP.

VXLAN ACL Limitations

The following ACL related features are not supported:

- Ingress router access control list (RACL) that is applied on an uplink Layer 3 interface that matches on the inner or outer payload in the network-to-access direction (decapsulated path)

- Egress RACL that is applied on an uplink Layer 3 interface that matches on the inner or outer payload in the access-to-network direction (encapsulate path)

- Egress VACL for decapsulated VXLAN traffic

We recommend that you use a port access control list (PACL)/VACL on the access side to filter out traffic entering the overlay network.

PVLANs

Private VLANs (PVLANs) are not supported.

DHCP

DHCP subnet broadcast is not supported.

Caveats

This section includes the following topic:

- Open Caveats—Cisco NX-OS Release 6.1

Open Caveats—Cisco NX-OS Release 6.1

lists the open caveats in the Cisco NX-OS Release 6.1(2)I2(1) release. Click the Bug ID to access the Bug Search tool and see additional information about the bug.

Table 4 Open Caveats in Cisco NX-OS Release 6.1(2)I2(1)

Bug ID

Description

The Link Pause feature is not supported on the Cisco Nexus 9500 Series devices.

CSCui89809

command was entered after applying a QoS policy,

DHCP relay to a subnet broadcast address does not work.

CSCuj57404

A DLB configuration should be rejected if PFC is also configured.

CSCul18670

command shows the incorrect license if a valid license is installed over an honor license.

CSCum22205

Priority tagged packets or untagged packets that are received by trunk-mode switch ports do not get encapsulated in VXLAN.

CSCum32811

Multicast packets that are received on Layer 3 to VXLAN groups should not be sent to the CPU.

CSCum36233

The MAC address for the ToR switch does not match IP packets with the mac packet-classify feature.

CSCum38846

Prevent OIF creation on receiving a (*,G) join from VTEP for VXLAN groups.

CSCun76810

Configuring NTP on a Cisco Nexus 9000 Series switch cause Onep to lose its connection.

CSCun00831

During the bootup of a peer switch, native-vlan mismatch syslog messages with CDP might appear.

CSCun06345

A VXLAN vPC inconsistency is not detected with mismatched VXLAN configurations.

CSCun14773

Tagged packets in a non-VXLAN VLAN are encapsulated if the trunk port in a native VLAN is a VXLAN VLAN.

CSCun26726

HSRP packet decoding fails with an assertion error.

CSCun30078

After adding a new member to an uplink port channel, an error displays.

CSCun34824

command, some VXLAN tunnels are not programmed correctly and traffic to those peers is silently dropped.

CSCun34856

All VLANs are suspended if one has a QoS policy but the TCAM is not configured.

CSCun54435

command does not clear the NS EOQ drop count.

CSCun69358

command does not count decap traffic under the out counters.

CSCun69596

Unicast traffic still goes after out after the VNI configuration is removed.

CSCun72335

Following a switch reload when there is a vPC with one leg down, the synchronized MAC addresses are not installed in the hardware.

CSCun72357

Traffic drops after a VLAN with a vPC leg down is deleted and recreated.

CSCun74838

Following a switch reload, an access port-channel member goes into internal failure when the port channel comes up.

CSCun75245

The ports of a vPC port-channel member are stuck in initializing state after a bootup and the port channel is administratively shut down.

CSCuq03168

Microsoft NLB traffic being routed into the destination VLAN is experiencing packet loss.

Related Documentation

The entire Cisco Nexus 9000 Series NX-OS documentation set is available at the following URL:

Configuration Guides

Cisco Nexus 9000 Series NX-OS Layer 2 Switching Configuration Guide

Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide

Other Software Documents

Cisco Nexus 9000 Series Software Upgrade and Downgrade Guide

Cisco Nexus 9000 Series NX-OS System Messages Reference

Hardware Documents

Release Notes

s

Documentation Feedback

To provide technical feedback on this document, or to report an error or omission, please send your comments to nexus9k-docfeedback@cisco.com. We appreciate your feedback.

Obtaining Documentation and Submitting a Service Request

, which also lists all new and revised Cisco technical documentation, at:

Open a service request online at:

as a Really Simple Syndication (RSS) feed and set content to be delivered directly to your desktop using a reader application. The RSS feeds are a free service and Cisco currently supports RSS version 2.0.

This product includes cryptographic software written by Eric Young (eay@cryptsoft.com). This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit. (http://www.openssl.org/). This product includes software written by Tim Hudson (tjh@cryptsoft.com).

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: www.cisco.com/go/trademarks . Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1110R)

Cisco Nexus 9000 Series NX-OS Release Notes, Release 6.1(2)I2(1)

© 2013 - 2014 Cisco Systems, Inc. All rights reserved.