Cisco Nexus 9000 Series NX-OS Release Notes, Release 9.3(3)
For more information, see Related Content.
April 25, 2024 |
Added CSCwh50989 to Open Issues. |
May 8, 2020 |
Updated VXLAN for Static Tunnels feature description in New Software Features. |
April 17, 2020 |
Added Licensing Support for Segment Routing v6 to New Software Features. |
April 14, 2020 |
Added Cisco Nexus 9332C to the Standard ISSU Support feature. |
April 10, 2020 |
Moved CSCvr58479 to Resolved Issues. |
March 24, 2020 |
Feature name “Unnumbered BGP with IPv6 Link-Local Peering (5549) for eBGP” updated to “BGP Interface Peering via IPv6 Link-Local for IPv4 and IPv6 Address Families” in New Software Features. |
March 4, 2020 |
Updated the Release Versioning Strategy section. |
March 2, 2020 |
Added N9K-C9316D-GX to Table 15. |
February 21, 2020 |
Update SVI and Sub-Interface Ingress/Egress Unicast Counters description in New Software Features. |
February 20, 2020 |
Update the NXA-PAC-750W-PE and NXA-PAC-750W-PI power supplies to include support for Cisco Nexus 93240YC-FX2, 9332C, and 9336C-FX2 switches. |
January 16, 2020 |
Removed Trigger-Based Event Log Auto-Collection from New Software Features. |
January 9, 2020 |
Added CSCvc95008 to General Known Issues. |
■ Cisco Network Insights for Data Center
Where the Z = 1 is always the first FCS release of a Major/Minor release.
Note: The Cisco Nexus 34180YC and 3464C platform switches are not supported in Cisco NX-OS Release 9.3(3).
Headline: Storm control counters do not work on Cisco Nexus 9500-R and Cisco Nexus 3600 platforms. Symptoms: On Cisco Nexus 9500-R and Nexus 3600 platform switches, storm control counters do not increment when the interface is flooded with broadcast traffic. This bug is applicable to the following hardware types: LCs: Workaround: None |
|
Headline: OC ACL: delete ipv4 ace with hop-limit configs fails Symptoms: When we have an ACE with hop-limit configs that were configured through the CLI and you try to delete it through NETCONF, it returns 'List delete failed' error, instead of deleting that entry from the configs. Workaround: Create ACE through netconf (or) Delete ACE through CLI |
|
Headline: CRC errors seen with 40/100G and 40G BiDi Symptoms: With the N9K-X97160YC-EX line card, using a dual rate 40/100G BiDi optics to connect to a peer which uses 40G BiDi optics can in some rare cases lead to CRC errors on the link. Workaround: None |
|
Symptoms: When SVI on a PTP master switch receives an IGMP membership report and creates IGMP entries, PTP High correction issues occur. %PTP-2-PTP_HIGH_CORR: Slave port Eth1/X High correction -125750482(nsec) This issue is applicable only to Cisco Nexus 9500-R platforms. Workaround: A or B A. - remove PIM config from PTP VLAN SVI B. - use other VLAN to PTP instead of VLAN used for multicast |
|
Headline: ACL Stats - Packet hitting an ACE entry with log option will be counted twice Symptoms: With ACL logging enabled on the ingress ACL, each packet hit is counted twice in TCAM stats. Once for the original packet and once for the SUP copy. Workaround: None |
|
Headline: VLAN counters should display L3 counters info Symptoms: The show vlan counters command doesn’t display L3 unicast counters. It currently displays the total of L2 and L3 packets. Workaround: NA |
|
Headline: EOR/FX:FT latency max out issue is observed Symptoms: When an EoR switch is configured as a spine switch and is acting as the PTP master for the leaf switches, the EoR and the egress leaf reports FT latency as 0XFFFF which causes hop-by-hop latency and the end-end FT latency is broken on Tetration. Workaround: If PTP is configured on the dedicated layer 2 port on ToR/EoR , the egress leaf reports a meaningful FT latency value. |
|
Headline: Generate error message for FHS policy when label is not available. Symptoms: FHS policy is not applied to the hardware and will not show in the configuration after a successful configuration attempt. Workaround: None |
|
Headline: Snmpbulkwalk/getnext periodic slowness seen in PFC-EXT-mibs. Symptoms: None Workaround: None |
|
Symptoms: MAC table is pointing to incorrect NH after multiple NVE flaps Workaround: Clear ip arp force-delete or clear ipv6 nd force-delete (if v6 host) and clear MAC address table |
|
Headline: Nexus 9300 - Block CLI to configure sub-interface with 40G uplink ports Symptoms: Sub-interfaces are not supported for 40G uplinks ports with 1st generation Cisco Nexus 9300 platform switches. Not all traffic is impacted depending on internal load-balancing of traffic between internal ASICs. Workaround: Do not use sub-interfaces for 40G uplinks ports. You can have sub-interfaces on 10G downstream ports. |
|
Headline: Interface BW not accounting unicast BW after SSO switchover Symptoms: After a switchover, the unicast fabric bandwidth does not get reserved on fabric links. Workaround: Flapping the fabric interface should fix it. |
|
Headline: TRM Multisite: traffic drop on BGW after restarting ngmvpn Symptoms: TRM traffic loss for 1-2 seconds. Workaround: None |
|
Headline: FCS error / link flap on N3K-C3636C-R with QSFP-100G-CU3M and QSFP-100G40G-BIDI in same BV Symptoms: 10 FCS error in a 12-hour period on the port with copper cable Workaround: Split cables across different port mappings |
|
Headline: 25g AOC/LR/SR Cable type is shown as unknown Symptoms: Cable type is shown as "unknown" for 25G SFP- LR, SR, and AOC cables. However, for copper 25G cables 'cable type' is displayed as CA-L, CA-N and CA-S accordingly. Workaround: There is no workaround for this problem. |
|
Headline: The host specific routes (IP-MAC) may stay in BGP after corresponding SVI removed Symptoms: Once SVI is down or removed, IP-MAC /32 route is stuck in BGP on remote VTEPs. Workaround: Entry can be cleared by removing member vni on nve interface. |
|
Headline: Sequence timeout seen at reload with VXLAN PBR -- SVI flap optimizations needed. Symptoms: The system might experience a sequence timeout that might cause the L2alredirect loopback test to fail, rpm verification to fail, or a DHCP snoop hardware programming failure. When hit on the vPC secondary, this situation might result in vPC VLANs getting suspended on primary. Workaround: Once the issue is hit, you can shut/no-shut the MCT link in vPC primary or operational primary to bring up the suspended VLANS. Also, to avoid getting into this situation, you may use GIR (graceful insertion and removal) to isolate the 9500, upgrade the chassis, and after all modules are up, insert the switch in to the network. |
|
Headline: Netstack core seen in non-destructive ISSU using FQDN for NTP Symptoms: FQDN for NTP server and ND ISSU on T2 ToR Workaround: Do not use FQDN for NTP Servers. Use IP Address. |
|
Headline: FC 48 ports entitlement tag is consumed even with 16 or less ports are configured Symptoms: FC 48 ports entitlement tag is consumed irrespective of the number of FC ports acquired. Workaround: None. This will not impact the functionality of ports. |
|
Headline: MPLS entries present after no feature-set mpls command Symptoms: After entering the "no feature-set mpls" command, the output of the "show for adjacency mpls stats" command is not empty. Workaround: Reload the box. |
|
Headline: N9K-9364C-GX : CRCs with 100G-CU1M on Ports 17,40 connected to Cisco Nexus 9232C Port 8 and Port 12. Symptoms: CRC seen on 100G 1M CU xcvr connected to peer with multiple reload Workarounds: Flap the links to recover the port from low SNR. |
|
Headline: Support 25G FEC rscons16 and rs-ieee on Cisco Nexus 9300-GX platform switches Symptoms: 9.3.3 software version does not support rscons16 and rs-ieee fec on 25G breakout links. Workarounds: No workarounds |
|
Headline: N9K-9364C-GX: CRCs on Port 24 connected to Port 32 with 40G-AOC Symptoms: When the system is idle and when peer send out chopped packets in FCS error. Workarounds: Reload the switch |
|
Headline: N9K-C93600CD-GX: Extra Flaps seen after Multiple reloads/flaps on different Optics on Gearbox Ports Symptoms: One Extra Flap seen after Multiple reloads/flaps on different Optics on ports 1-24 of N9K-C93600CD-GX. Similar extra flap seen after Multiple reloads/flaps on QSFP-100G-PSM4 and QSFP-100G-FR of N9K-9364C-GX. Workaround: None |
|
Headline: N3K-C3408-S: Delayed linkup observed with 100g CR4 on Portflap/OiR Symptoms: CR4 copper 100G cables sometimes, may experience higher link up times when connected between N3K-C3408-S and N9K Switches. Link up time varies depending upon different Peers. This happens only when auto-negotiation is enabled. The issue may or may not be seen across all the ports. Workarounds: Disable auto-negotiation and apply force speed. |
|
Headline: Traffic outage when switching 4x25G --> 2x50G --> 4x25G soft-breakout, control plane is fine Symptoms: PIXMC-SDB would be wrong Breakout third port LTL will be assigned 2nd port ifidx Workarounds: Reload module. |
|
Headline: Multicast traffic drop due to NF flag set Symptoms: On T2-EoR that is part of a vPC pair, ND-ISSU (modular) is done from 7.0.(3)I4(0) to 9.3(3) and there is a loss of multicast traffic towards receivers on L3 interfaces. That is, non vPC-SVI OIF. Workarounds: Use an intermediate ISSU from 7.0(3)I4(0) to 9.3(2) on both vPC peers, and then an ISSU from 9.3(2) to 9.3(3). |
|
Headline: SRv6: BGP VPNv4 - CNH with null IOD added to URIB upon removal of IPv6 address using intf range cmd Symptoms: Stale SRv6 VPNv4 routes will be seen in BGP and URIB databases after IPv6 address is removed on all of the core/spine facing interfaces using the interface range command. In SRv6-VXLAN Dual-GW environment, adding IPv6 address back to the interfaces might cause a Netstack crash Workarounds: Remove IPv6 address from one interface at a time |
|
Headline: N9K-C93600CD-GX: CRC errors seen 40G AOC-3M on Port Eth1/6 when connected N9K-C9236C Re-Timer Port Symptoms: Seeing CRC on N9K-C93600CD-GX port 6 when connected to N9K-C9236C retimer port using 40G AOC3M cable. Workarounds: Don't use 40G AOC on N9K-C93600CD-GX port6. |
|
Headline: IPIP/GRE pkts coming in non-default VRF gets decap by matching tunnel where transport vrf is default Symptoms: IPIP/GRE encapsulated pkts coming on interface in non-default VRF say VRF-X getting decapsulated and egress on interface in default VRF. Workarounds: Below are some work-arounds 1. Don't configure the GRE/IP-IP tunnel in default VRF 2. If user needs to configure a GRE/IPIP tunnel in default VRF and a matching pkt can ingress in non-default VRF configure the Tunnel with matching tunnel src and tunnel destination in respective non default VRF also. For example, when a ipip pkt with outer dest-ip 132.127.101.1 ingress on interface with vrf member vrf-cust-1 In the following example you should configure another Tunnel 102 in non-default VRF vrf-cust-1
interface Tunnel101 tunnel mode ipip decapsulate-any ip tunnel source 132.127.101.1 no shutdown
interface Tunnel102 vrf member vrf-cust-1 tunnel mode ipip decapsulate-any ip tunnel source 132.127.101.1 tunnel use-vrf vrf-cust-1 no shutdown |
|
Headline: N9K-C93600CD-GX Interop with N9K-C9236C is not supported in 9.3(3) Symptoms: N9K-C93600CD-GX connections to N9K-C9236C are not supported in the 9.3(3) release since it has not been fully qualified. Workarounds: Use a peer switch other than N9K-C9236C. |
|
Headline: Custom COPP causing transit traffic to be punted to the CPU on Nexus 9300-GX2 Symptoms: When custom-COPP policy contains ACL rules which match on Layer 4 destination or source port, transit traffic also hits the COPP and the packets are copied to CPU. This causes duplication of traffic as CPU also routes the copied packets to the destination. Workarounds: Custom COPP policy using src/dst match mitigates punt for transit traffic. |
Headline: NXOS DATACORRUPTION-DATAINCONSISTENCY error in PIM process Symptoms: The following error is observed under PIM process – Workaround: Not available. |
|
Headline: Physical interfaces - sub interface support range from 1-4096 Symptoms: In order to support a static breakout configuration, the sub-interface range is restricted to a number from 1-511. This enhancement request is to allow the user to configure any sub-interface number from 1-4094, but internally map the sub-interface to one of the values from 1-511, so that it makes migration of configurations from other platforms easier, and lets the user keep their dot1q VLAN and sub-interface number the same for configuration readability. Workaround: Use port-channel sub interfaces |
|
Headline: Need support for DOM on FEX HIF ports Symptoms: When entering the "show interface ethxxx/y/z transceiver detail"command on a Cisco Nexus 9000 Series switch (FEX HIF), we see that DOM is not supported. This defect is an enhancement to support DOM on the Cisco Nexus 9000 FEX HIFs. Workaround: None |
|
Headline: Increased CPU usage for nsusd process (25%) Symptoms: Increased CPU usage for nsusd process (25%) Workaround: None |
|
Headline: ARP Does Not Respond For VRRPv3 VIP After Module Reload "Destination address is not local" Symptoms: ARP for VRRPv3 VIP on Master is not being responded to due to "non-local address" Workaround: Remove VRRPv3 configuration from impacted interface and reapply Shutdown of the interface (Not main interface in case of subif's or SVI's, user should shutdown the subif or SVI) |
|
Headline: CRC errors occur on the neighbor devices when connects QSFP-100G-LR4-S on N3K-C36180YC-R Symptoms: On release 9.2.3, Oplink device will have CRC Error. Workarounds: Use other vendors: Finisar V02 |
|
Headline: TCAM resource usage increase causes TCAM resource exhausted: BFD Symptoms: Log message:2019 Jan 17 05:32:20.225742 switch %ACLQOS-SLOT1-2-ACLQOS_OOTR: TCAM resource exhausted: BFD Seeing in instance 0x1 100% utilization for BFD: Workaround: Reload |
|
Symptoms: N9K-C93180LC-EX might experience a crash in the "tahusd" process when physically inserting a third-party 10 Gbps SFP into a 40 Gbps or 100 Gbps QSFP breakout cable. Workaround: 1) Use a Cisco-branded SFP in the QSFP 2) If the "hardware profile port mode" configuration is also inappropriate, either: 2a) Remove / change the QSFP breakout cables to match the "hardware profile portmode" configuration seen via "show run | grep portmode" 2b) Change the "hardware profile portmode" configuration to match the number and type of QSFP breakout cables installed, and reload |
|
Headline: N9k do not age out Snooping entry against vPC Peer link port after receipt of GSQ Symptoms: Snooping table points to peer link on both the switches for multicast groups Workaround: None |
|
Headline: N9K NX-OS 9.2(3) SNMPd Crash / MTS Queue Congestion When Doing GETBULK on entPhysicalEntry Symptoms: A Cisco Nexus 9000 Series switch running Cisco NX-OS Release 9.2(3) might see crashes in the SNMPd process due to its MTS queue becoming congested. Workaround: Configure any NMS polling stations to stop doing GETBULK requests for entPhysicalEntry (1.3.6.1.2.1.47.1.1.1.1) |
|
Headline: TCP networking vulnerabilities in FreeBSD and Linux kernels(TCP_SACK) Symptoms: This bug has been filed to evaluate the product against the vulnerability released by Netflix on June 17th affecting FreeBSD and Linux kernels, identified by CVE IDs: Workaround: Not currently available. |
|
Headline: RMAC in L2RIB points to the wrong NH despite URIB has the correct information Symptoms: Router MAC used to route through L3VNI, points to the wrong next-hop in L2RIB, despite BGP having learned the correct route with the proper NH information. Depending on the topology, this might cause severe packet loss or total traffic blackhole. Workarounds: 1. Flap the L3 VLAN having the spurious RMAC. This will flush out the wrong entries from L2RIB. 2. If many such VLANs are affected, interface NVE can be flapped. |
|
Headline: Selective advertisement not working in multisite Symptoms: If we have a couple Layer 2 VNIs, and only some VNIs are extended to another site. If we enable L3vni extended other site and all the L2 VNI mac addresses are advertised to other site. Workaround: None |
|
Headline: Intermittent VNI in DOWN state due to vni-add-await-buffer Symptoms: VNI in down state due to vni-add-await-buffer. Workaround: Remove entry and recreate resolves the issue. |
|
Headline: "speed xxxx" under line console doesn't take effect Symptoms: "speed xxxx" under line console doesn't take effect Workaround: None |
|
Headline: MRIB process crash and switch in bootloop after upgrade from 9.2(2) to 9.3(1) Symptoms: Symptoms: + As soon as we apply feature PIM on Cisco NX-OS Release 9.3(1), device reloads with a core. Workaround: Issue was not found to occur when u4route-mem + u6route-mem is <= 1 GB. 750MB is recommended as a safe upper limit. |
|
Headline: GRE traffic with payload with wrong IP header is dropped Symptoms: GRE traffic with payload with wrong IP header is dropped Workaround: Downgrade to previous software version than 7.0(3)I7(6) |
|
Headline: IP forwarding broken when "hardware access-list tcam label ing-racl 9" config Symptoms: After setting "hardware access-list tcam label ing-racl 9" and rebooting the system, IP/ARP/ND forwarding is broken. Workaround: No workaround available other than not using this knob: hardware access-list tcam label ing-racl 9 unconfigure and reload: no hardware access-list tcam label ing-racl 9 |
|
Headline: DCHAL changes for QinQ, Selective QinQ and Multiple Provider VLAN Symptoms: QinQ , Selective QinQ changes porting to Irvine train from 9.2(2) release. Workarounds: Set the provider_bd flag for the provider_vlan in rwx_rwbdstatetable Example: switch(config)# slot 1 qu "debug hardware internal hom mod asic 0 slice 0 table tah_hom_rwx_rwbdstatetable 99 1 provider_bd=0x0000001" asic instance is 0 asic slice is 0 tbl name is tah_hom_rwx_rwbdstatetable start entry is 99 entry count is 1 field value is provider_bd=0x0000001 Block base address: 0x01800000 1st table entry address: 0x01e20318 switch(config)# |
|
Headline: Congested SPAN traffic, causing drops to normal forwarding traffic Symptoms: Congested SPAN traffic, causing drops to normal forwarding traffic. Workaround: Make sure no congestion for the SPAN Destination port. Forwarding ports, and the SPAN destination port has same capacity or more for SPAN port. |
|
Headline: N9k FX2 - MPLS transit forwarding affected through FX2 series switches Symptoms: Connectivity/forwarding is impacted when an MPLS labelled frame transits through a Cisco Nexus 9000 FX2 Series switches. Workaround: No workaround available. |
|
Headline: VXLAN: BUM traffic dropped on DCI/BL devices working as Bud node Symptoms: If the Cisco Nexus 9000 Series switches in a VXLAN multi-pod setup that were used to interconnect the DCI were previously configured as BUD nodes (transit box + VTEP with VNI configured), you might experience drops in BUM traffic. Note that the VNI configured previously must have been using the same mcast group as the one used for transit traffic. Workaround: - Reload the switch - Create the VNIs configuration for the VNIs present in the transit traffic (VLAN/VNI mapping and VNI config under NVE interface). Note that the same mcast group must be used. |
|
Headline: "show hardware capacity forwarding" does not have complete output in JSON Symptoms: "show hardware capacity forwarding" not completely JSONized Workaround: None |
|
Headline: ISIS Hellos not forwarded on N3K-C36180YC-R Symptoms: ISIS hellos are not being forwarded on 9500-R switches. Cu has ISIS on ASRs. Adjacency came up fine when customer replaced N3K with WS-C3850-12XS-E (same configuration). Workaround: None |
|
Headline: N9K-C92160YC-X // BGP - Some routes are forwarded via incorrect interface Symptoms: Several N9K-C92160YC-X switches running Cisco NX-OS Release 7.0(3)I7(4) code and placed in similar scenarios suffered the same hardware mis-programming. Workaround: Use LPM heavy mode |
|
Headline: Community deletion leads to Assertion 'tmp_com == del_com' failed. Symptoms: %BGP-3-ASSERT: bgp-[29078] ../routing-sw/routing/bgp/bgp_pcl_cache.c:662: Assertion 'tmp_com == del_com' failed. Workaround: If we are using: set comm-list comlist delete problem is not occurring Not applicable in some scenarios. |
|
Headline: After upgrading(disruptive) N9K to I7(6) control plane is stuck. Symptoms: After upgrade or reload of a Cisco Nexus 9500 platform switch with -S LC/FM might experience control plane traffic issues Workaround: None as additional reloads may retrigger issue. |
|
Headline: KIM Process MTS Buffers Stuck Symptoms: KIM Process MTS buffers stuckcopy r s may also fail due to deadlock with KIM process. Workaround: Reload the switch clears stuck MTS buffers. |
|
Symptoms: High bandwidth utilization on internal module ports with minimal traffic on front-facing ports Workaround: FM reload fixes the issue. We can reload FMs one by one so that traffic will not be impacted. |
|
Headline: The host specific routes (IP-MAC) may stay in BGP after corresponding SVI removed Symptoms: Once SVI is down or removed, IP-MAC /32 route is stuck in BGP on remote VTEPs. Workaround: Entry can be cleared by removing member vni on nve interface. |
|
Headline: N9K-C92160YC-X // Routing and forwarding issue -no FIB Hardware entry for prefix Symptoms: Several N9K-C92160YC-X switches running Cisco NX-OS Release 7.0(3)I7(4) code and placed in similar scenarios suffered the same hardware mis-programming (no entry in FIB Hardware). Workaround: Using `clear ip route vrf <vrf_name> <ip_prefix>` may help in correcting the issue. |
|
Headline: SNMP walk using OID 1.3.6.1.2.1.1 returns NULL [Expert Info (Note/Response): endOfMibView] Symptoms: N9508/N9504 Running 9.2.3 in vPC Workaround: Work around is to remove the community string and the mib view command and add them back. |
|
Headline: Netflow / destination command is broken in rollback/patch Symptoms: The destination subcommand in Netflow configuration may not be interpreted properly by the Nexus parser when entered. Workaround: None |
|
Headline: N9K-X9736C-FX // debounce time Unexpected Behaviour Symptoms: N9k // N9K-X9736C-FX // DWDM Interface Flap when DWDM link protection is triggered Workaround: N/A |
|
Headline: Dynamic NAT configuration on the N9k causes L2 forwarding issues. Symptoms: L2 traffic destined to a MAC not belonging to the Cisco Nexus 9000 Series switch is software switched (CPU punt) and dropped, which should not be the case. Workaround: disable NAT |
|
Headline: Unable to configure user defined MAC after a failure condition. Symptoms: User-defined MAC not able to be applied to an L3 interface following programming failure Workaround: Remove the MAC and re-add to the impacted interface. |
|
Headline: August CPU Side-Channel Information Disclosure Vulnerabilities -- issue with some hardware Symptoms: Few switches were seeing PCI error during bringup. Workaround: It’s not a workaround. There was some code which was creating PCI errors. That code was not needed for Spectre issue. |
|
Headline: Copy run start fails on Nexus 3500 switch due to service "confelem" failure Symptoms: On a Cisco Nexus 3500 platform switch, you might be unable to copy running-configuration to startup-configuration due to the confelem process failing to store its configuration. Workaround:None |
|
Headline: receive-only path overwrites BRIB path in ephemeral DME database Symptoms: When querying rest API, some BGP paths are missing Workaround: remove "always" keyword from "soft-reconfiguration inbound " |
|
Headline: Wrong output of 'show snapshots compare' command with multiple VRFs Symptoms: The output of `show snapshots compare snap_before_maintenance snap_after_maintenance` is incorrect then having multiple VRFs configured on the Nexus device. Workaround: None |
|
Headline: Unable to apply ACL to remote SNMP user Symptoms: Configuration of an ACL to a remote SNMP user via the global configuration command 'snmp-server user <snmp_user> use-ipv4acl <access_list>' fails, where snmp_user is a remote user (that is, a username not configured locally) and access_list is an ACL to define permissions. Workaround: Configure the user locally if an ACL is needed or downgrade to a 7.0(3)I7(x) release. |
|
Headline: Debounce is not working for N9K-C9364C using LR4 transceiver and link flaps Symptoms: For the N9K-C9364C switch, after configuring debounce to max value we do see link flap and time for link to come up is high + Issue is noticed for LR4 transceiver Workaround: Use SR4 optics |
|
Headline: Nexus 3500 BGP-3-ASSERT syslog in IPv4 Multicast AF with Ext. Communities Symptoms: A Cisco Nexus 3500 platform switch configured as a BGP speaker that receives a prefix in the IPv4 multicast address-family with an Extended Communities attribute might produce a "BGP-3-ASSERT" syslog. The specific syslog will vary depending upon the NX-OS software release that the device is running. Workaround: No workaround is known for this issue at this time. |
|
Headline: Storm control gets triggered even when threshold is not reached Symptoms: Storm control gets triggered when ESXi doing vmotion or reload even when threshold is not reached. Workaround: None |
|
Headline: vpcm process memory leak @ libnve.so and libvlan_mgr_mcec.so Symptoms: Command `show vpc consistency-parameters global` or `show vpc consistency-parameters vlans` on the vPC VTEP (VXLAN setup) might cause a slow memory leak in libnve.so library, which in the long term perspective can cause the vPC process to be unresponsive or crash. You may also experience this issue by running `show run`: Workaround: In the unlikely event of hitting this issue, please contact Cisco Support Ceter for further verification. Alternatively, you can consider chassis reload. |
|
Headline: vsh.bin process crash Symptoms: The vsh.bin process might crash when attempting to access the Cisco Nexus switch via SSH and the MTS payload of the authentication packets is corrupted. This will be reported in the log as follows: Workaround: None |
|
Headline: PTP Packets punted when feature ptp is enabled/disabled Symptoms: PTP packets punted to CPU Workaround:Configure feature ptp reload of the N9k box DONOT attempt to reload active FM which will not resolve this issue. |
|
Headline: An interface may forward disallowed VLAN traffic over a trunk Symptoms: Port forwards VLAN traffic which is removed from trunk port Workaround: Remove "lacp vpc-convergence" |
|
Headline: Security: service NTP SIGABRT due to heartbeat failure Symptoms: NTP process gets busy at times, while processing older version NTP packets and may miss sending heartbeat messages. The NTP process gets restarted and continues as before. This doesn't impact the time synchronization functionality of NTP in any manner. Workarounds: None |
|
Headline: Pre-check of ND ISSU failed on LACP but show lacp issu-impact print nothing Symptoms: show lacp issu-impact will show nothing even though there is ISSU impact because of LACP. Workarounds: Try Non disruptive ISSU and check for any LACP related failures. |
|
Headline: N9K: high CPU on ipfib and system lockup after frequent IPv6 update/flap/removal Symptoms: High number of add/delete events of a large number of IPv6 routes might cause high CPU due to an IPFIB process, inband instability (BFD flap, OSPF flap), MGMT instability (SSH, TACACS, slow responses to CLI and in some instances a crash of the IPFIB process. Workaround: - Reload Nexus switch to recover - Reduce IPv6 scope to number lower than 250k |
|
Headline: MALLOC_FAILED: mcastfwd [27776] m_copyin failed in mfwd_ip_main() Symptoms: 2019 Nov 18 22:12:11 N9300 mcastfwd[1983]: m_copyback: m_get() fails. 2019 Nov 18 22:12:10 N9300 %MCASTFWD-3-MALLOC_FAILED: mcastfwd [1983] m_copyin failed in mfwd_ip_main() 2019 Nov 18 22:12:11 N9300 mcastfwd[1983]: m_copyback: m_get() fails. 2019 Nov 18 22:12:20 N9300 %MCASTFWD-4-SYSLOG_SL_MSG_WARNING: MCASTFWD-3-MALLOC_FAILED: message repeated 1 time in last 377 sec Workaround: Restart mcastfwd process. |
|
Headline: Using GRE, inner DSCP value is not copied to the outer DSCP on N9K. Symptoms: Using GRE, inner DSCP value is not copied to the outer DSCP on N9K. Workaround: NA |
|
Headline: Multicast Storm-control not working for N9K. Symptoms: Storm-control not working properly for multicast traffic. Workaround: no ip igmp snooping |
|
Headline: N9K-C9396 // OID Return Wrong Values Symptoms: On, N9K-C9396 while queuing for a DOM values via SNMP walk, sometimes the OID returns as "No Such Instance" randomly and reads fine after some time. Workaround: Depends on the number of ports on the setup. The rate of the error can be reduced by matching the SNMP query frequency with the DOM read back timer callback frequency. Configure SNMP query at a rate matching the DOM read back timer: (PC_FCOT_POLL_TIME/2) / (num_ports ) ; Where: PC_FCOT_POLL_TIME = 10 minutes num_ports = total number of Physical ports supported by the switch PORT_POLL_DDM_INCREMENT = 4 For example, On an N9K-C9396, which is a 48 port switch SNMP poll interval can be set to not less than and multiple of: (10x60/2) / (48) seconds = 6.25 seconds per port NOTE: This workaround applies only if the SNMP query is done for all ports linearly. Not guaranteed to work for random port queries. |
|
Headline: PBR routing failure after micro-flap on egress interface. Symptoms: PBR routed traffic may stop getting forwarded after experiencing a micro-flap on the egress interface. Workarounds: + Disable debounce timer to produce a complete flap of the interface, avoiding any ASIC mis-programming + If the traffic is getting black-holed due to a micro-flap, flapping the egress interface should re-program the ASIC correctly |
|
Headline: QinVNI BUM traffic to vPC peer is dropped Symptoms: QinVNI BUM traffic is received on a leaf, and decapped successfully and flooded. However, the PIP tunnel copy is dropped by the vPC peer. Workarounds: There is no work around for this packet drop. |
|
Headline: Egress QoS policing not working when applied on the egress VTEP Symptoms: Egress QoS policing not working when applied on the egress VTEP Using traffic generator, 1 Gbps of traffic is generated. During the working scenario the traffic is rate limited as per the policy. During the non-working scenario, traffic is not rate limited. Workarounds: Issue is resolved when the service policy is reconfigured. |
|
Headline: Cisco Nexus N9K-X9788TC-FX continuously aging out MAC addresses Symptoms: Partial MAC address will be aged out and deleted with each hit of aging time and is quickly relearned. Workarounds: None |
On Cisco Nexus 9300-EX, 9348GC-FXP, 93108TC-FX, 93180YC-FX, 9336C-FX2, 93216TC-FX2, 93360YC-FX2, 93240YC-FX2, 92348GC-X, C93108TC-EX-24, C93108TC-FX-24, C93180YC-EX-24, C93180YC-FX-24, 9316D-GX, 9364C-GX, and 93600CD-GX switches, when 802.1q EtherType has changed on an interface, the EtherType of all interfaces on the same slice will be changed to the configured value. This change is not persistent after a reload of the switch and will revert to the EtherType value of the last port on the slice. |
|
CoPP violations can be seen under class-map copp-system-p-class-l2-default and access-group copp-system-p-acl-mac-undesirable in an MVPN setup on a PE device. This can cause an impact to MVPN control plane functionality for packets such as MSDP and PIM register messages, in case of a large number of MVPN PE devices and MDT groups. You can create a custom CoPP policy with an increased "cir" value until no CoPP violation is seen for that class. |
|
Per-VRF Configuration of MDT MTU size is not supported on MVPN PE devices on N9K-X9636C-R/RX, N3K-C36180YC-R, N3K-C3636C-R platforms. While, Tunnel MTU size is not configurable interface MTU for the core facing interface can be configured to control port-level MTU. MDT tunnel is capable of carrying up to jumbo MTU size of 9192 (excluding tunnel header), provided interface MTU for the core-facing interface also supports jumbo MTU. |
|
CMIS standards prescribe delays at each state as mentioned by the QSFP-DD firmware on those optics. If you are using those optics with delays, you will see a higher link-up time. |
|
CMIS standards prescribe delays at each state as mentioned by the QSFP-DD firmware on those optics. If you are using those optics with delays, you will see a higher link-up time. |
|
The Cisco Nexus 9300-GX ASIC does not support FC-FEC on the second lane of 50x2 breakout port. This is due to an ASIC limitation. The second link cannot come up when 50x2 breakout is done. Workaround: You must configure RS-FEC with 50x2 breakout. |
|
Multicast routes used by Data MDT are not deleted immediately on MVPN PE (where Encapsulation takes place) after all the customer (VRF) traffic stops which use the same Data MDT. They may stay up for 15 minutes and then get deleted. |
|
When large files, for example NX-OS, images are copied to USB, the following message is printed: As long as these messages correspond to a copy operation to USB, this message can be ignored. |
|
</nf:source> <============nf: is extra <namespace> : extra characters are seen with XML and JSON from NX-API. |
|
PTP is not supported on the 96136YC-R line card or for line cards on the Cisco Nexus 9504 switch. |
|
The following features are not supported on the Cisco Nexus 3464C and 9364C switches. |
|
■ The following feature is not supported on the Cisco Nexus 9332C: |
|
Only the following switches support QSFP+ with the QSFP to SFP/SFP+ adapter (40 Gb to 10 Gb): |
|
The Cisco Nexus 9300 platforms support for the QSFP+ breakout has the following limitations: ■ For the Cisco Nexus 9332PQ switch, all ports except 13-14 and 27-32 can support breakout. |
|
The following switches support the breakout cable (40 Gb ports to 4x10-Gb ports): ■ N9K-C93180LC-EX—last four ports are breakout capable (10x4, 24x4, 50x2) |
|
Limitations for ALE (Application Link Engine) uplink ports are listed at the following location: Limitations for ALE 40G Uplink Ports on Cisco Nexus 9000 Series Switches |
|
|
On Cisco Nexus N2K-C2348TQ HIFs fail to utilize redundant Port-Channel links, to NIF, during link failover events. |
Table 1 Cisco Nexus 9500 Switches
Table 2 Cisco Nexus 9500 Cloud Scale Line Cards
Table 3 Cisco Nexus 9500 R-Series Line Cards
Table 4 Cisco Nexus 9500 Classic Line Cards
Table 5 Cisco Nexus 9500 Cloud Scale Fabric Modules
Table 6 Cisco Nexus 9500 R-Series Fabric Modules
Table 7 Cisco Nexus 9500 Fabric Modules
Table 8 Cisco Nexus 9500 Fabric Module Blanks with Power Connector
Table 9 Cisco Nexus 9500 Supervisor Modules
Table 10 Cisco Nexus 9500 System Controller
Table 11 Cisco Nexus 9500 Fans and Fan Trays
Table 12 Cisco Nexus 9500 Power Supplies
Table 13 Cisco Nexus 9200 and 9300 Fans and Fan Trays
Table 14 Cisco Nexus 9200 and 9300 Power Supplies
Table 15 Cisco Nexus 9200 and 9300 Switches
Table 16 Cisco Nexus 9000 Series Uplink Modules
Table 1 Cisco Nexus 9500 Switches
Table 2 Cisco Nexus 9500 Cloud Scale Line Cards
Product ID |
Description |
Maximum Quantity |
||
Cisco Nexus 9504 |
Cisco Nexus 9508 |
Cisco Nexus 9516 |
||
N9K-X9736C-FX |
Cisco Nexus 9500 36-port 40/100 Gigabit Ethernet QSFP28 line card |
4 |
8 |
16 |
N9K-X9732C-FX |
Cisco Nexus 9500 32-port 40/100 Gigabit Ethernet QSFP28 line card |
4 |
8 |
16 |
N9K-X9732C-EX |
Cisco Nexus 9500 32-port 40/100 Gigabit Ethernet QSFP28 line card |
4 |
8 |
16 |
N9K-X9736C-EX |
Cisco Nexus 9500 36-port 40/100 Gigabit Ethernet QSFP28 line card |
4 |
8 |
16 |
N9K-X9788TC-FX |
Cisco Nexus 9500 48-port 1/10-G BASE-T Ethernet and 4-port 40/100 Gigabit Ethernet QSFP28 line card |
4 |
8 |
16 |
N9K-X97160YC-EX |
Cisco Nexus 9500 48-port 10/25-Gigabit Ethernet SFP28 and 4-port 40/100 Gigabit Ethernet QSFP28 line card |
4 |
8 |
16 |
Table 3 Cisco Nexus 9500 R-Series Line Cards
Cisco Nexus 9500 36-port 40/100 Gigabit Ethernet QSFP28 line card |
|||
Cisco Nexus 9500 36-port 40/100 Gigabit Ethernet QSFP28 line card |
|||
Table 4 Cisco Nexus 9500 Classic Line Cards
Product ID |
Description |
Maximum Quantity |
||
Cisco Nexus 9504 |
Cisco Nexus 9508 |
Cisco Nexus 9516 |
||
N9K-X9408C-CFP2 |
Line card with 8 100 Gigabit CFP2 ports |
4 |
8 |
16 |
N9K-X9432C-S |
Cisco Nexus 9500 32-port 40/100 Gigabit Ethernet QSFP28 line card |
4 |
8 |
N/A |
N9K-X9432PQ |
Cisco Nexus 9500 32-port 40 Gigabit Ethernet QSFP+ line card |
4 |
8 |
16 |
N9K-X9636PQ |
Cisco Nexus 9500 36-port 40 Gigabit Ethernet QSFP+ line card |
4 |
8 |
N/A |
N9K-X9464PX |
Cisco Nexus 9500 48 1/10-Gigabit SFP+ and 4-port 40-Gigabit Ethernet QSFP+ line card |
4 |
8 |
16 |
N9K-X9464TX |
Cisco Nexus 9500 48 port 1/10-Gigabit BASE-T Ethernet and 4-port 40-Gigabit Ethernet QSFP+ line card |
4 |
8 |
16 |
N9K-X9464TX2 |
Cisco Nexus 9500 48 port 1/10-Gigabit BASE-T Ethernet and 4-port 40-Gigabit Ethernet QSFP+ line card |
4 |
8 |
16 |
N9K-X9536PQ |
Cisco Nexus 9500 36-port 40 Gigabit Ethernet QSFP+ line card |
4 |
8 |
16 |
N9K-X9564PX |
Cisco Nexus 9500 48 1/10-Gigabit SFP+ and 4 port 40-Gigabit Ethernet QSFP+ line card |
4 |
8 |
16 |
N9K-X9564TX |
Cisco Nexus 9500 48 port 1/10-Gigabit BASE-T Ethernet and 4 port 40-Gigabit Ethernet QSFP+ line card |
4 |
8 |
16 |
Table 5 Cisco Nexus 9500 Cloud Scale Fabric Modules
Table 6 Cisco Nexus 9500 R-Series Fabric Modules
Table 7 Cisco Nexus 9500 Fabric Modules
Table 8 Cisco Nexus 9500 Fabric Module Blanks with Power Connector
Cisco Nexus 9508 Fabric blank with Fan Tray Power Connector module |
|||
Cisco Nexus 9516 Fabric blank with Fan Tray Power Connector module |
Table 9 Cisco Nexus 9500 Supervisor Modules
Table 10 Cisco Nexus 9500 System Controller
Table 11 Cisco Nexus 9500 Fans and Fan Trays
Table 12 Cisco Nexus 9500 Power Supplies
Table 13 Cisco Nexus 9200 and 9300 Fans and Fan Trays
Fan 1 module with port-side intake airflow (burgundy coloring) |
||||
Fan 2 module with port-side intake airflow (burgundy coloring) |
||||
Fan 3 module with port-side intake airflow (burgundy coloring) |
||||
Fan module with port-side intake airflow (burgundy coloring) |
||||
|
||||
Fan module with port-side intake airflow (burgundy coloring) |
|
|||
Fan module with port-side intake airflow (burgundy coloring) |
92160YC-X |
|||
92160YC-X |
||||
4 |
|
|||
6 |
9316D-GX |
93600CD-GX |
||
Fan module with port-side intake airflow (burgundy coloring) |
4 |
|
||
6 |
93600CD-GX |
|||
Fan module with port-side exhaust airflow (burgundy coloring) |
1For specific fan speeds see the Overview section of the Hardware Installation Guide.
Table 14 Cisco Nexus 9200 and 9300 Power Supplies
1 Compatible with Cisco NX-OS Release 9.3(3) and later.
Table 15 Cisco Nexus 9200 and 9300 Switches
Table 16 Cisco Nexus 9000 Series Uplink Modules
An enhanced version of the Cisco Nexus N9K-M6PQ uplink module. |
|
Cisco Nexus 9300 uplink module with 12 40-Gigabit Ethernet QSPF+ ports. |
To determine which transceivers and cables are supported by this switch, see the Transceiver Module (TMG) Compatibility Matrix.
To see the transceiver specifications and installation information, see Install and Upgrade Guides.
Straight-through FEX module support has been added for the following switches:
For more information, see the Cisco Nexus 9000 Series Switch FEX Support page.
■ Active-Active FEX and straight-through FEX are not supported on the Cisco Nexus 92348GC switch.
For more information, see the Cisco Network Insights documentation.
To perform a software upgrade or downgrade, follow the instructions in the Cisco Nexus 9000 Series NX-OS Software Upgrade and Downgrade Guide, Release 9.3(x).
For information about an In Service Software Upgrade (ISSU), see the Cisco NX-OS ISSU Support application.
■ Cisco Nexus 9316D-GX, 9364C-GX and 93600D-GX Switches
■ Cisco Nexus 9200, 9300-EX, and 9300-FX Platform Switches
■ Cisco Nexus 9500 Platform N9K-X9408PC-CFP2 Line Card and 9300 Platform Switches
The following features are not supported for the Cisco Nexus 9316D-GX, 9364C-GX, and 93600D-GX switches:
■ 50x2 Breakout - Cisco Nexus 9364C-GX
■ 802.1x with VXLAN
■ Asymmetric PFC
■ Autonegotiation on all ports
■ DCI Handoff (VXLAN to SR MPLS and MPLS to VXLAN)
■ ERSPAN destination on Cisco Nexus 9200 and 9300-EX platform switches
■ ERSPAN Termination
■ EVPN Multisite
■ FC-FEC for Cisco Nexus 9316D-GX and 93600CD-GX is not supported on the second lane of 50x2 breakout port.
■ FEX
■ Flex Link
■ FTE
■ IP Load Sharing New Options
■ IPv6 FHS
■ IPv6 Flow Label Hashing
■ IPv6 MLD Snooping
■ IPv6 Underlay, TRM + Multi-Site
■ IPSG
■ ISSU for Cisco Nexus 9316D-GX, 9364C-GX and 93600CD-GX
■ ITD
■ ITD with NAT
■ ITD with VXLAN
■ MPLS Strip
■ MPLS/SR feature of L2 EVPN
■ MTU Truncation
■ Multi Auth with COA
■ Multicast NLB
■ Multicast over GRE
■ Multiple VRF support on Tunnel Decap
■ Multi-Site Scale (25 sites), 6 Border Gateways per site
■ NAT
■ PACL Redirect / Multicast (Tap Agg)
■ PBR
■ PFC WD
■ PIM on SVI
■ PMN
■ PTP/TTAG
■ PVLAN
■ QinVNI
■ RTP Monitoring
■ Selective QinQ with multiple provider for VLANs
■ Selective QinVNI with multiple provider for VLANs
■ SR QoS
■ SSX
■ SVI Unnumbered
■ TRM
■ TRM + Multi-Site
■ VMCT
■ VRF Aware FT
■ VXLAN F&L
■ VXLAN – PBR
■ VXLAN Static Route
The following features are not supported for the Cisco Nexus 9200 platform switches and the Cisco Nexus 93108TC-EX and 93180YC-EX switches:
■ Cisco Nexus 9272PQ and Cisco Nexus 92160YC platforms do not support the PXE boot of the Cisco NX-OS image from the loader.
■ ACL filters to span sub-interface traffic on the parent interface
■ Egress QoS policer is supported on the Cisco Nexus 9300-EX and 9300-FX platform switches. It is not supported on the Cisco Nexus 9200 platform switch. The only policer action supported is drop. Remark action is not supported on egress policer.
■ FEX (supported for Cisco Nexus 9300-EX platform switches but not for Cisco Nexus 9200 platform switches.)
■ GRE v4 payload over v6 tunnels
■ IP-in-IP on Cisco Nexus 92160 switch
■ ISSU enhanced is not supported on the Cisco Nexus 9300-FX, and Cisco Nexus 9300-FX2 platform switches and the Cisco Nexus 9364C switch.
■ Maximum Transmission Unit (MTU) checks for packets received with an MPLS header
■ NetFlow is not supported on Cisco Nexus 9200 platform switches.
■ Packet-based statistics for traffic storm control (only byte-based statistics are supported)
■ PVLANs (supported on Cisco Nexus 9300 and 9300-EX platform switches but not on Cisco Nexus 9200 platform switches)
■ Q-in-VNI is not supported on Cisco Nexus 9200 platform switches. Beginning with Cisco NX-OS Release 7.0(3)I5(1), Q-in-VNI is supported on Cisco Nexus 9300-EX platform switches.
■ Q-in-Q for VXLAN is not supported on Cisco Nexus 9200 and 9300-EX platform switches
■ Q-in-VNI is not supported on Cisco Nexus 9200 platform switches (supported on Cisco Nexus 9300-EX platform switches)
■ Resilient hashing for port-channel
■ Rx SPAN for multicast if the SPAN source and destination are on the same slice and no forwarding interface is on the slice
■ SVI uplinks with Q-in-VNI are not supported with Cisco Nexus 9300-EX platform switches
■ Traffic storm control for copy-to-CPU packets
■ Traffic storm control with unknown multicast traffic
■ Tx SPAN for multicast, unknown multicast, and broadcast traffic
■ VACL redirects for TAP aggregation
The following features are not supported for the Cisco Nexus 9500 platform N9K-X9408PC-CFP2 line card and Cisco Nexus 9300 platform switches with generic expansion modules (N9K-M4PC-CFP2):
■ FEX (this applies to the N9K-X9408PC-CFP2 and –EX switches, not all Cisco Nexus 9300 platform switches)
■ MCT (Multichassis EtherChannel Trunk)
■ PTP (Precision Time Protocol)
■ PVLAN (supported on Cisco Nexus 9300 platform switches)
■ Shaping support on 100g port is limited
■ SPAN destination/ERSPAN destination IP
The following features are not supported for the N9K-X96136YC-R line card:
■ PTP and gPTP are not supported.
The following feature is not supported for the N9K-X9736C-FX line card:
■ Ports 29-36 do not support 1 Gbps speed.
The following features are not supported for Cisco Nexus 9500 cloud scale (EX/FX) line cards:
■ IPv6 support for policy-based routing
■ SPAN port-channel destinations
See the Cisco Nexus 9000 Series Switches page for the documentation.
The Cisco Nexus 3000 and 9000 Series NX-API REST SDK User Guide and API Reference is available at the following location: Cisco Nexus NX-API Reference
The Cisco NX-OS Supported MIBs URL:
ftp://ftp.cisco.com/pub/mibs/supportlists/nexus9000/Nexus9000MIBSupportList.html
Cisco Nexus 9000 Series Software Upgrade and Downgrade Guide: Cisco Nexus 9000 Series NX-OS Software Upgrade and Downgrade Guide, Release 9.3(x)
The Cisco Nexus 9000 Series FPGA/EPLD Upgrade Release Notes, Release 9.3(3) is available at the following location:
Cisco Nexus 9000 Series FPGA/EPLD Upgrade Release Notes, Release 9.3(3)
For more information, see the Cisco NX-OS Licensing Guide.
Cisco Nexus 9000 Series NX-OS FC-NPV and FCoE-NPV Configuration Guide, Release 9.3(x)
■ This is a new configuration guide, as of the Cisco NX-OS Release 9.3(3), which is a consolidation of the Cisco Nexus 9000 Series NX-OS FC NPV and Cisco Nexus 9000 FCoE NPV Configuration Guide, Release 9.3(x).
Cisco Nexus 9000 Series NX-OS SAN Switching Configuration Guide, Release 9.3(x)
■ This is a new configuration guide as of the Cisco NX-OS Release 9.3(3).
Cisco Nexus 9000 Series NX-OS SRv6 Configuration Guide, Release 9.3(x).
■ This is a new configuration guide as of the Cisco NX-OS Release 9.3(3).
To provide technical feedback on this document, or to report an error or omission, please send your comments to nexus9k-docfeedback@cisco.com. We appreciate your feedback.
Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1110R)