Cisco Nexus 3600 NX-OS Verified Scalability Guide

First Published: May 9, 2024

Introduction

This document describes the Cisco NX-OS configuration limits for Cisco Nexus 3600 Series switches.

The values provided in this guide should not be interpreted as theoretical system limits for Cisco Nexus 3600 platform hardware or Cisco NX-OS software. These limits refer to values that have been validated by Cisco. They can increase over time as more testing and validation is done.

Verified Scalability Limits (Unidimensional)

The tables in this section list the verified scalability limits for Cisco NX-OS Release 10.3(5)M. These limits are validated with a unidimensional configuration. The values provided in these tables focus on the scalability of one particular feature at a time.

Each number is the absolute maximum currently supported by this Cisco NX-OS release for the corresponding feature. If the hardware is capable of a higher scale, future software releases might increase this verified maximum limit. Results might differ from the values listed here when trying to achieve maximum scalability with multiple features enabled.

Table 1. Intelligent Traffic Director Verified Scalability Limits (Unidimensional)

Feature

Supported Platforms

Verified Limit

Nodes per device group

N3K-C36180YC-R, N3K-C3636C-R

16

Buckets per ITD service

N3K-C36180YC-R, N3K-C3636C-R

64
Table 2. Interfaces Verified Scalability Limits (Unidimensional)

Feature

Verified Limit

DHCP servers/relay IPs per switch

5 (IPv4) + 5 (IPv6)1

MAC address - table limit per port

2,000 - (Nexus 3636C-R and 36180YC-R switches)

MAC address - table system, VLAN limit

2,000 - (Nexus 3636C-R and 36180YC-R switches)

Port channel member links

32

SVIs

3967

vPCs

48

1 This limit has not been tested
Table 3. Label Switching Verified Scalability Limits (Unidimensional)

Feature

Verified Limit

LDP sessions

200

Forwarding Equivalence Classes (FECs)

1,000

Equal-cost multipaths (ECMPs)

8

FECs ECMPs

4,000

IAS option B labels

450,000

Layer 3 VPN routes

100,000

ECMPs

2,000

Table 4. Layer 2 Switching Verified Scalability Limits (Unidimensional)

Feature

Verified Limit

MAC addresses (default template)

196,000

MAC addresses (L2-scale template)

384,000

MST instances

64

MST virtual ports

218,185

RPVST virtual ports

13,750

VLANs

3,967

VLANs in RPVST mode

250

Private VLANs (PVLANs)

Primary VLANs

Nexus N3K-C36180YC-R and N3K-C3636C-R switches

7

Secondary VLANs

Nexus N3K-C36180YC-R and N3K-C3636C-R switches

7

Ports in Community host mode

Nexus N3K-C36180YC-R and N3K-C3636C-R switches

40

Ports in isolated host mode

Nexus N3K-C36180YC-R and N3K-C3636C-R switches

40

Ports in isolated trunk host mode

Nexus N3K-C36180YC-R and N3K-C3636C-R switches

40

Ports in promiscuous mode

Nexus N3K-C36180YC-R and N3K-C3636C-R switches

40

Ports in promiscuous trunk mode

Nexus N3K-C36180YC-R and N3K-C3636C-R switches

40

PVLANs allowed on a PVLAN port

Nexus N3K-C36180YC-R and N3K-C3636C-R switches

7

Maximum number of VLANs (private VLAN’s, native VLANS and normal VLANS) configurable on a PVLAN ports per ASIC

Nexus N3K-C36180YC-R and N3K-C3636C-R switches

4096
Table 5. Programmability Verified Scalability Limits (Unidimensional)

Feature

Supported Platforms

Verified Limits

NX-API

See Guidelines and Limitations for NX-API limitations.

Maximum Number of concurrent VSH session

Nexus 3600 switches and line cards

20 concurrent VSH sessions

Number of worker processes in Nginx

Nexus 3600 switches and line cards

4 worker processes

Number of VSH sessions per worker process

Nexus 3600 switches and line cards

A maximum of 5 persistent VSH sessions are supported for each worker process

Maximum response size supported in output

Nexus 3600 switches and line cards

10 MB

Maximum number of concurrent session supported for chunk mode. See Configuring the Message Format and Command Type to know more about chunk mode

Nexus 3600 switches and line cards

2

Maximum size of response supported in chunk mode

Nexus 3600 switches and line cards

After 10.3(1) release, the maximum size supported in chunk mode is the same as the amount of space available in volatile.

Table 6. Layer 3 Multicast Verified Scalability Limits (Unidimensional)

Feature

Verified Limit

IPv4 multicast routes

32,000 (Layer 3)

Outgoing interfaces (OIFs)

16 OIFs for 32K mroutes or 287 OIFs for 1000 mroutes

PIM neighbors

500

SVI

50-60

IGMP snooping groups

8000

MVPN- unidimensional

Multicast VRFs

100 (N3K-C3636C-R and N3K-C36180YC-R)

Default MDT groups

100 (N3K-C3636C-R and N3K-C36180YC-R)

MVPN Peers (PIM neighbors) per device

100 (N3K-C3636C-R and N3K-C36180YC-R)
Table 7. Security Verified Scalability Limits (Unidimensional)

Feature

Verified Limit

IPv4 ingress access control entries (ACEs)

RACL-2000, PACL-1024 (without TCAM Carving)

IPv6 ingress access control entries (ACEs)

RACL-1000, PACL-1024 (without TCAM Carving)

ACL

12,000 (with TCAM Carving)

Egress ACLs

20,000 (N3K-C3636C-R and N3K-C36180YC-R)

RACLs

4,000 (N3K-C3636C-R and N3K-C36180YC-R)

System ACLs

4,000 TCAM entries in internal TCAM and 64,000 TCAM entries in external TCAM (N3K-C3636C-R and N3K-C36180YC-R)
Table 8. System Management Verified Scalability Limits (Unidimensional)

Feature

Verified Limit

SPAN and ERSPAN

Configurable SPAN or ERSPAN sessions

32

Active SPAN or ERSPAN sessions

32

Active localized SPAN or ERSPAN session per line card

32 sessions across ports on single line card

Active localized SPAN or ERSPAN session (Rx and Tx, Rx, or Tx)

32 sessions, 128 sources and 1 destination

Destination interfaces per SPAN session

1

Source VLANs per SPAN or ERSPAN

6

Table 9. Layer 3 Unicast Routing Verified Scalability Limits (Unidimensional) - For Default system routing template

Feature

Verified Limit

BFD sessions (echo mode)

288

100 MHBFD sessions (N3K-C3636C-R and N3K-C36180YC-R)

BGP neighbors

256

HSRP groups

498

IPv4 ARP

75,000

IPv4 host routes

750,000

IPv6 host routes

62,000

IPv6 ND

32,000

IPv4 unicast routes (LPM)

192,000

IPv6 unicast routes (LPM)

62,000

OSPFv2 neighbors

1,000

OSPFv3 neighbors

1,000

OSPF/OSPFv3 LSA/LSDB size

250,000

OSPF/OSPFv3 areas

15

VRFs

3,967

VRRP

VRRP groups per interface or I/O module

15

Table 10. Layer 3 Unicast Routing Verified Scalability Limits (Unidimensional) - For Internet-peering system routing template

Feature

Verified Limit

Routes (internet-peering mode)

852000

IPv4 routes (internet-peering mode)

781000

IPv6 routes (internet-peering mode)

71000
Table 11. HSRP Verified Scalability Limits (Unidimensional)

Feature

Verified Limit

Groups with default timers (3s/10s) and multiple group optimization. [There are 2 primary, one for IPv4 and the other for IPv6, and 7926 secondary]

7,928

Groups with aggressive timers (1s/3s) and multiple groups optimization. [There are 2 primary, one for IPv4 and the other for IPv6, and 7926 secondary]2

7,928

Groups per interface or I/0 module

Maximum 16 (Because 16 is the unique virtual MAC address limit)

2 If the user has Multi-protocol configuration, user should configure appropriate COPP policies so as to avoid any control plane traffic drops.
Table 12. VXLAN Verified Scalability Limits (Unidimensional)

Feature

Verified Limit 3

IGMP snooping over VXLAN

VXLAN VLANs

1,000

VTEP peers

256

Underlay multicast groups

128
VXLAN Flood and Learn

Virtual network identifiers (VNIs) or VXLAN-mapped VLANs

Not applicable

Virtual network identifiers (VNIs) or VXLAN-mapped VLANs

Not applicable

Underlay multicast groups.

Not applicable

Overlay MAC addresses

Not applicable

Remote VXLAN tunnel endpoints (VTEPs)

Not applicable

Ingress replication peers

Not applicable

Ingress replication Layer 2 VNIs

Not applicable

MAC addresses for ingress replication

Not applicable

Port VLAN translations under an interface

Not applicable

Port VLAN translations in a switch

Not applicable

Static MAC addresses pointing to a remote VTEP

Not applicable

VXLAN VLAN logical port VP count

Not applicable

VXLAN VLANs per FEX port (host interface)

Not applicable

Layer 2 routed VNIs for vPC-centralized gateway

Not applicable

IGMP groups

Not applicable
VXLAN BGP eVPN

Layer 2 VNIs

2,000

Xconnect VLANs

Not applicable

SVI with Distributed Anycast Gateway; Layer 2 VNI extended

2,000

Layer 3 VNIs / VRFs

900

Underlay multicast groups

128

VTEPs

256

MAC addresses

90,000

IPv4 host routes

350,000

IPv6 host routes

48,000

Overlay IPv4 LPM routes

180,000

Overlay IPv6 LPM routes

48,000

VXLAN VLAN logical port VP count

Not applicable

VXLAN VLANs per FEX port (host interface)

Not applicable

IGMP groups

8192
VXLAN BGP eVPN Ingress Replication

Layer 2 VNIs

Not applicable

Xconnect VLANs

Not applicable

SVI with Distributed Anycast Gateway; Layer 2 VNI extended

Not applicable

Layer 3 VNIs / VRFs

Not applicable

VTEPs

Not applicable

MAC addresses

Not applicable

IPv4 host routes

Not applicable

IPv6 host routes

Not applicable

Overlay IPv4 LPM routes

Not applicable

Overlay IPv6 LPM routes

Not applicable

VXLAN VLAN logical port VP count

Not applicable

VXLAN VLANs per FEX port (host interface)

Not applicable

IGMP groups

Not applicable
3 For Cisco Nexus 3636C-R and Cisco Nexus 36180YC-R switches

Verified Scalability Limits (Multidimensional)

The tables in this section list the verified scalability limits for Cisco NX-OS Release 10.3(5)M. These limits are validated with a multidimensional configuration. The values provided in these tables focus on the scalability of one particular feature at a time.

Each number is the absolute maximum currently supported by this Cisco NX-OS release for the corresponding feature. If the hardware is capable of a higher scale, future software releases might increase this verified maximum limit. Results might differ from the values listed here when trying to achieve maximum scalability with multiple features enabled.


Attention
These numbers are not the maximum verified values if each feature is viewed in isolation. For these numbers, see the "Verified Scalability Limits" section.
Table 13. MSDC Profile Verified Scalability Limits (Multidimensional)

Feature

Verified Limit

Number of 100G ports

6

36 (N3K-C3636C-R)

vPC port channels

10

ISIS IPv4 /32 unicast routes

1,291

ISIS IPv6 /128 unicast routes

1,291

Multicast IPv4 SSM

10,000

VRF IPv4/IPv6

100

PIM neighbors

100

IGMP snooping database entries

240

VRRP v4 and v6

1,000 vlans

Multicast SSM

10,000

HSRP v4 and v6

1,000 vlans

SVI

100 (N3K-C3636C-R)

Sub-interfaces

100 (N3K-C3636C-R)

MAC

1000 (N3K-C3636C-R)

BGP IPv4/IPv6 VLSM routes

1000 (N3K-C3636C-R)

BGP IPv4/IPv6 Unicast routes

10,000 (N3K-C3636C-R)

ECMP

16-way Upstream (N3K-C3636C-R)

SPAN sessions

1 local SPAN session (N3K-C3636C-R)

Table 14. MPLS Verified Scalability Limits (Multidimensional)

Feature

Verified Limit

MPLS Layer 3 VPN

3,715

VPE

3,715

PE nodes

Nil

PE routes

Nil

ACL (IPv4)

1,100

ACL (IPv6)

440

HSRP and IPv6 VIP

3.715 each for v4 and v6

vPC uRPF

Nil

Strict uRPF

Yes

VRF

3,715

SVI

3,715

Layer 3 VPN routes IP ECMP

<500

MPLS LSR ECMP

<500

VPN IPv4 routes

65,000

VPN IPv6 routes

25,000

EBGP neighbors

Nil

Table 15. Layer 2/Layer 3 TOR Boundary Verified Scalability Limits (Multidimensional)

Feature

Verified Limit

ECMP

16-way (Upstream)

vPC port channels

44

OSPFv2 neighbors

16

OSPFv3 neighbors

16

OSPF IPv4 /32 unicast routes

45,000

OSPF IPv4 VLSM unicast routes

1,000

OSPF IPv6 /128 unicast routes

25,000

OSPF IPv6 VLSM unicast routes

1,000

BFD sessions

230

100 MHBFD sessions (N3K-C3636C-R and N3K-C36180YC-R)

VLAN

1,250

SVI

1,000

1250 (N3K-C3636C-R)

Sub-interfaces

250 per interface and 500 across the system (N3K-C3636C-R)

VRRP IPv4 groups

1,000 VRRS / 8 VRRPv3

VRRP IPv6 groups

1,000 VRRS / 8 VRRPv3

PIM neighbors

230

IPv4 (*,G) multicast routes

300

IPv4 (S,G) multicast routes

2,320

IGMP snooping database entries

6,300

Sflow enabled interfaces

63

45 (N3K-C3636C-R)

UDLD enabled interfaces

65

48 (N3K-C3636C-R)

SPAN sessions

1 local SPAN session

MVR VLANs

250

MVR receiver ports

10

MVR multicast groups

1,000

MAC

20,000 (N3K-C3636C-R)

Q-in-Q tunnel ports

26

RSTP VLANS (tunneled over L2PT)

3,960

Table 16. Layer 2/Layer 3 Spine Boundary (for N3K-C3636C-R) Verified Scalability Limits (Multidimensional)

Feature

Verified Limit

Number of 100G ports

36

Number of 10G ports

36 x 4 (Breakout)

ECMP

16-way (Upstream)

vPC port channels

40

OSPFv2 neighbors

100

OSPFv3 neighbors

100

OSPF IPv4 /32 unicast routes

45,000

OSPF IPv4 VLSM unicast routes

1,000

OSPF IPv6 /128 unicast routes

25,000

OSPF IPv6 VLSM unicast routes

1,000

BFD sessions

280

100 MHBFD sessions (N3K-C3636C-R and N3K-C36180YC-R)

VLAN

3,967

SVI

3,967

Sub-interfaces

250 per interface and 511 across system

VRRP IPv4 groups

1,996 VRRS / 4 VRRPv3

VRRP IPv6 groups

1,996 VRRS / 4 VRRPv3

HSRP IPv4

1,743 Secondary Groups / 7 Primary Groups

HSRP IPv6

1,743 Secondary Groups / 7 Primary Groups

PIM neighbors

230

IPv4 (*,G) multicast routes

2,000

IPv4 (S,G) multicast routes

30,000

IGMP snooping database entries

6,300

sFlow enabled interfaces

45

UDLD enabled interfaces

48

SPAN sessions

1 local SPAN session

MAC

50,000

Table 17. Segment Routing Verified Scalability Limits (Multidimensional)

Feature

Verified Limit

LACP

26

LACP members

1 or 4

eBGP IPv6 neighbors

25

eBGP IPv4 LU neighbors

24

IPv4 (LU) routes

1,537

IPv4 (LU) paths

6,987

IPv6 routes

1,486

IPv6 paths

6,915

SR ECMP (max)

18

MPLS HW entries

6,868

Table 18. Segment Routing (for N3K-C3636C-R) Verified Scalability Limits (Multidimensional)

Feature

Verified Limit

VLAN

100

SVI

100

MAC entries

10,000

ARP entries

70

HSRPv4, HSRPv6 VIPs

100, 100

LACP

3

LACP members

4

eBGP IPv6 neighbors

2

eBGP IPv4 neighbors

2

IPv4 (LU) routes

6,848

IPv4 (LU) paths

8,187

IPv6 routes

6,640

IPv6 paths

7,975

SR ECMP

2

MPLS HW entries

2,682

Table 19. VXLAN Profile Verified Scalability Limits (Multidimensional)

Feature

Verified Limit

Number of ports

16

ECMP

8-way (Upstream)

BGP neighbors

2

BGP EVPN Layer 2 VPN host routes

60,000

BGP IPv4 VLSM unicast routes or ospf

10,000

BGP IPv6 VLSM unicast routes or ospf

2,000

BFD sessions

10

PIM neighbors

10

IPv4 (*,G) multicast routes (co-existing)

4,000

IPv4 (S,G) multicast routes (co-existing)

2,000

Layer 3 VNI

100

Layer 2 VNI

400

Local VTEP

1

Remote VTEPs

205

VLAN

400

SVI

100

MAC

80,000

vPC hosts

1