Verified Scalability Limits

This chapter describes the Cisco NX-OS configuration limits for the Cisco Nexus 9000 Series switches.

Introduction

The values provided in this guide should not be interpreted as theoretical system limits for Cisco Nexus 9000 Series hardware or Cisco NX-OS software. These limits refer to values that have been validated by Cisco. They can increase over time as more testing and validation is done.

Verified Scalability Limits

The tables in this section list the unidimensional verified scalability limits for Cisco NX-OS Release 6.1(2)I2(3). These limits are validated with a unidimensional configuration. The values provided in these tables focus on the scalability of one particular feature at a time.

Each number is the absolute maximum currently supported by this Cisco NX-OS release for the corresponding feature. If the hardware is capable of a higher scale, future software releases might increase this verified maximum limit. Results might differ from the values listed here when trying to achieve maximum scalability with multiple features enabled.

Table 1. Cisco Nexus 2000 Series Fabric Extenders (FEX) Verified Scalability Limits (Unidimensional)

Feature

9500 Series Verified Limit

9300 Series Verified Limit

Number of Fabric Extenders and total number of Fabric Extender server interfaces

Not applicable

6 and 288

Number of VLANs per Fabric Extender

Not applicable

2000 (across all Fabric Extenders)

Number of VLANs per Fabric Extender server interface

Not applicable

75


Note

The Cisco Nexus 2000 Series Fabric Extender is supported only with the Cisco Nexus 9396 chassis.

Table 2. Interfaces Verified Scalability Limits (Unidimensional)

Feature

9500 Series Verified Limit

9300 Series Verified Limit

Port channel links

32

8

SVIs

490

250

vPCs

275

100 (280 with Fabric Extenders)

Table 3. Layer 2 Switching Verified Scalability Limits (Unidimensional)

Feature

9500 Series Verified Limit

9300 Series Verified Limit

MST instances

64

64

MST virtual ports

85,000

48,000

RPVST virtual ports

22,000

12,000

VLANs

4000

3900

VLANs in RPVST mode

500

500


Note

The number of supported VLANs per vPC should be within the MST or RPVST virtual port count specified in this table, depending on the topology.

Table 4. Multicast Routing Verified Scalability Limits (Unidimensional)

Feature

9500 Series Verified Limit

9300 Series Verified Limit

IPv4 multicast routes

32,000

8000

Outgoing interfaces (OIFs)

40

40


Note

The IPv4 multicast routes and the IPv4/IPv6 host routes share the same hardware table. Limits are provided for both the default line card mode and the max host line card mode.


Note
High availability (graceful restart and stateful switchover) is not supported when unicast or multicast aggressive timers are configured at any scale.
Table 5. Security Verified Scalability Limits (Unidimensional)

Feature

9500 Series Verified Limit

9300 Series Verified Limit

IPv4 ingress ACLs

3072 (per network forwarding engine)

3072 (per network forwarding engine)

IPv4 egress ACLs

768 (per network forwarding engine)

768 (per network forwarding engine)

IPv6 ingress ACLs

1536 (per network forwarding engine)

1536 (per network forwarding engine)

IPv6 egress ACLs

256 (per network forwarding engine)

256 (per network forwarding engine)

Table 6. System Management Verified Scalability Limits (Unidimensional)

Feature

9500 Series Verified Limit

9300 Series Verified Limit

Configurable SPAN or ERSPAN sessions

32

4

Active SPAN or ERSPAN sessions1

4 to 32, based on the number of line cards and the session configuration

4

Active localized SPAN or ERSPAN sessions per line card2

4

4

Source interfaces per SPAN or ERSPAN session (Rx and Tx, Rx, or Tx)

48

48

Destination interfaces per SPAN session

1 (physical interface)

1 (physical interface)

Source VLANs per SPAN or ERSPAN session

32

32

1 A single forwarding engine instance supports four SPAN or ERSPAN sessions. For Cisco Nexus 9300 Series switches, if the first three sessions have bidirectional sources, the fourth session has hardware resources only for Rx sources. This limitation might also apply to Cisco Nexus 9500 Series switches, depending on the SPAN or ERSPAN source's forwarding engine instance mappings.
2 The number of SPAN or ERSPAN sessions per line card reduces to two if the same interface is configured as the bidirectional source in more than one session.
Table 7. Unicast Routing Verified Scalability Limits (Unidimensional)

Feature

9500 Series Verified Limit

9300 Series Verified Limit

BGP neighbors

1000

150

HSRP groups per interface or I/O module

490

250

IPv4 ARP

60,000

45,000

IPv4 host routes

88,000

60,000 (max host mode)

90,000

IPv6 host routes

20,000

30,000 (max host mode)

40,000

IPv6 ND

30,000

20,000

IPv4 unicast routes (LPM)

128,000

16,000 (max host mode)

12,000

IPv6 unicast routes (LPM)

20,000

4000 (max host mode)

6000 (5000 routes < /64, 1000 routes > /64)

MAC addresses

90,000

90,000

OSPFv2 neighbors

1000

200

OSPFv3 neighbors

300

200

VRRP groups per interface or I/O module

250

250


Note

The IPv4 and IPv6 unicast routes share the same hardware table. Limits are provided for both the default line card mode and the max host line card mode.


Note

The IPv4/IPv6 host routes and the IPv4 multicast routes share the same hardware table. Limits are provided for both the default line card mode and the max host line card mode.


Note
High availability (graceful restart and stateful switchover) is not supported when unicast or multicast aggressive timers are configured at any scale.

Guidelines and Limitations for OSPF Verified Scalability Limits

  • To achieve the highest scale, we recommend that you use a single OSPF instance instead of multiple instances.
  • Each OSPFv2 and OSPFv3 scale value might vary when combined with other parameters.
  • The graceful restart timeout value might need to be increased in multi-dimensional scenarios.
Table 8. VXLAN Verified Scalability Limits (Unidimensional)

Feature

9500 Series Verified Limit

9300 Series Verified Limit

Virtual network identifiers (VNIs) or VXLAN-mapped VLANs

Not applicable

640

Overlay multicast groups

Not applicable

128

Overlay MAC addresses

Not applicable

64,000

Remote VXLAN tunnel endpoints (VTEPs)

Not applicable

256

Deployment Case Studies

This section provides sample topologies for some common deployments. For each topology, the scalability numbers are the limits with all of the listed features enabled at the same time.


Attention

These numbers are not the maximum verified values if each feature is viewed in isolation. For these numbers, see Verified Scalability Limits.

Verified Scalability Limits for a Layer 2/Layer 3 Aggregation Topology (Max-Host Routing Mode)

This Layer 2/Layer 3 aggregation topology consists of Cisco Nexus 9508 switches as virtual port channel (vPC) aggregation pairs. These aggregation nodes are fully loaded with N9K-X9564TX, N9K-X9564PX, and N9K-X9636PQ line cards. The N9K-X9636PQ line cards are used in normal mode and breakout mode. Cisco Nexus 9396PX and 93128TX switches are used as top-of-rack units with Cisco Nexus 3000 Series switches to achieve the desired vPC scale.

The Cisco Nexus 9508 switch is also used as a core Layer 3 node that connects to a pair of vPC aggregation nodes. The focus of the topology is to test IPv4 ARP, IPv6 neighbor discovery (ND), and Layer 2 scalability and other routing, switching, and Layer 4 through Layer 7 features for management and operations. All Layer 3 interfaces are configured for dual stack, and the traffic is dual stack for all VLANs.

In the following table, the Verified Limit column lists the verified scaling capabilities with all listed features enabled at the same time. The scale numbers listed here exceed those used by most customers in their topologies. These numbers are not the maximum verified values if each feature is viewed in isolation.

Table 9. Verified Scalability Limits for a Layer 2/Layer 3 Aggregation Topology (Max-Host Routing Mode)

Feature

9500 Series Verified Limit (Max-Host Routing Mode)

9300 Series Verified Limit

Fully loaded chassis

6 N9636PQ line cards + 1 N9564TX line card + 1 N9564PX line card + 6 fabric modules + 2 system controllers + 2 supervisors

Not applicable

Physical interfaces enabled

300

Not applicable

Multicast S,G routes

12,000

Not applicable

Multicast *,G routes

3400

Not applicable

IPv4 unicast routes (LPM)

2400

2200

IPv6 unicast routes (LPM)

2200

2200

IPv4 ARP

60,000

5000

IPv6 ND

30,000

5000

MAC addresses

90,000

72,000

VLANs

500 (RPVST)

3900 (MST)

vPCs*

275

100

OSPFv2 neighbors

16

200

OSPFv3 neighbors

16

200

BGP (IPv4) neighbors

64 (eBGP)

150 (iBGP)

BGP (IPv6) neighbors

64 (eBGP)

150 (iBGP)

SVIs

490

250

MST instances

Not applicable

64

HSRP VLANs (IPv4/IPv6)

490

250

Virtual ports

3000 (RPVST)

24,000 (MST)

Port channel links

32

8

* The number of VLANs per vPC supported should be within the MST or RPVST virtual port count specified in this table, depending on the topology.