Traffic queuing is the ordering of packets and applies to both input and output of data. Device modules can support multiple queues, which you can use to control the sequencing of packets in different traffic classes. You can also set weighted random early detection (WRED) and taildrop thresholds. The device drops packets only when the configured thresholds are exceeded.

Traffic scheduling is the methodical output of packets at a desired frequency to accomplish a consistent flow of traffic. You can apply traffic scheduling to different traffic classes to weight the traffic by priority.

The queuing and scheduling processes allow you to control the bandwidth that is allocated to the traffic classes so that you achieve the desired trade-off between throughput and latency for your network.

You can use the following methods to proactively avoid traffic congestion on the device:

• Apply WRED to TCP or non-TCP traffic.

• Apply tail drop to TCP or non-TCP traffic.

For egress packets, you can choose one of the following congestion management methods:

• Specify a bandwidth that allocates a minimum data rate to a queue.

• Impose a minimum and maximum data rate on a class of traffic so that excess packets are retained in a queue to shape the output rate.

• Allocate all data for a class of traffic to a priority queue. The device distributes the remaining bandwidth among the other queues.

For information about configuring congestion management, see the Configuring WRED on Egress Queues section.

ECN is an extension to WRED that marks packets instead of dropping them when the average queue length exceeds a specific threshold value. When configured with the WRED ECN feature, routers and end hosts use this marking as a signal that the network is congested to slow down sending packets.

Note	The ECN feature is not supported on the Cisco Nexus 9508 switch (NX-OS 7.0(3)F3(3)).

Note	Enabling WRED and ECN on a class on a network-qos policy implies that WRED and ECN is enabled for all ports in the system.

Note	On extended output queues (EOQ), the approximate fair-drop (AFD) feature for bandwidth management is always enabled. The WRED configuration is ignored on EOQs. The configuration for EOQs is based on the system queuing policy and not on the per port policy.

Traffic shaping allows you to control the traffic going out of an interface in order to match its flow to the speed of the remote target interface and to ensure that the traffic conforms to policies contracted for it. You can shape traffic that adheres to a particular profile to meet downstream requirements. Traffic shaping eliminates bottlenecks in topologies with data-rate mismatches.

Traffic shaping regulates and smooths out the packet flow by imposing a maximum traffic rate for each port’s egress queue. Packets that exceed the threshold are placed in the queue and are transmitted later. Traffic shaping is similar to traffic policing, but the packets are not dropped. Because packets are buffered, traffic shaping minimizes packet loss (based on the queue length), which provides better traffic behavior for TCP traffic.

Using traffic shaping, you can control access to available bandwidth, ensure that traffic conforms to the policies established for it, and regulate the flow of traffic to avoid congestion that can occur when the egress traffic exceeds the access speed of its remote, target interface. For example, you can control access to the bandwidth when policy dictates that the rate of a given interface should not, on average, exceed a certain rate even though the access rate exceeds the speed.

Queue length thresholds are configured using the WRED configuration.

Note

Traffic shaping is not supported on ALE enabled device 40G front panel ports. When traffic shaping is configured for the system level, the setting is ignored and no error message is displayed. When traffic shaping commands are configured for the port level, the setting is rejected and an error message is displayed.

Queuing and scheduling have the following prerequisites:

• You must be familiar with using modular QoS CLI.

• You are logged on to the device.

Queuing and scheduling have the following configuration guidelines and limitations:

Note	For scale information, see the release-specific Cisco Nexus 9000 Series NX-OS Verified Scalability Guide.

• The minimum egress shaper granularity is 200 Mbps per queue for Cisco Nexus 9300-GX2/HX platform switches and line cards.

• show commands with the internal keyword are not supported.

• PVLANs do not provide support for PVLAN QoS.

• The device supports a system-level queuing policy, so all ports in the system are impacted when you configure the queuing policy.

• A type queuing policy can be attached to the system or to individual interfaces for input or output traffic.

• Changes are disruptive. The traffic passing through ports of the specified port type experience a brief period of traffic loss. All ports of the specified type are affected.

• Performance can be impacted. If one or more ports of the specified type do not have a queuing policy applied that defines the behavior for the new queue, the traffic mapping to that queue can experience performance degradation.

• When there is a link flap on a port with active traffic, it results in a packet/traffic loss flowing through other ports on the same or different slices. To avoid the flow discards, make sure you reduce the queue limit from the default value to a lower value and apply it at the system level.

• Traffic shaping can increase the latency of packets due to queuing because it falls back to store-and-forward mode when packets are queued.

• Traffic shaping is not supported on the Cisco Nexus 9300 ALE 40G ports. For more information on ALE 40G uplink ports, see the Limitations for ALE 40G Uplink Ports on the Cisco Nexus 9000 Series Switches.

• When configuring priority for one class map queue (SPQ), configure the priority for QoS group 3. When configuring priority for more than one class map queue, configure the priority on the higher numbered QoS groups. In addition, the QoS groups must be next to each other. For example, if you want to have two SPQs, you have to configure the priority on QoS group 3 and on QoS group 2.

• About queue limits for 100G enabled devices (such as the Cisco Nexus 9300 platform switch with the N9K-M4PC-CFP2 GEM):

  • The maximum dynamic queue-limit alpha value can be greater that 8. However 8 is the maximum alpha value supported. If you configure the alpha value to a value greater than 8, it is overridden and set to the maximum.

    No message is issued when the alpha value is overridden.

  • The static queue-limit has a maximum of 20,000 cells. Any value specified greater than the maximum 20,000 cell limit is overridden by the 20,000 cell limit.

    No message is issued when the cell limit is overridden.

• 100G enabled devices (such as the Cisco Nexus 9300 Series switch with the N9K-M4PC-CFP2 GEM), the WRED threshold has a maximum of 20,000 cells. Any value specified greater than the maximum 20,000 cell limit is overridden by the 20,000 cell limit.

  No message is issued when the cell limit is overridden.

• FEX support for:

  • System input (ingress) level queuing for HIF to NIF traffic.

  • System output (egress) level queuing for NIF to HIF traffic and HIF to HIF traffic.

• Egress queuing feature works only for base ports and not for FEX ports.

• When the switch supported system queuing policy is configured, the FEX uses the default policy.

• The FEX QoS system level queuing policy does not support WRED, queue-limit, shaping, or policing features.

• The FEX QoS system level queuing policy does not support multiple priority levels.

• Assigning a high alpha value on a Cisco Nexus 9200 platform switch uses more than the expected 50% of the available buffer space.

  Assigning a lower alpha value (7 or less) assures the usage of the expected 50% of the available buffer space.

• For Cisco Nexus 9200 platform switches, when a static limit is configured on a queue, both the static limit and the dynamic limit are calculated using the dynamic threshold (alpha value).

• Maximum queue occupancy for Leaf Spine Engine (LSE) enabled switches are limited to 64K cells (~13MB).

• For the following Cisco Nexus series switches and line cards, the lowest value that the egress shaper can manage, per queue, is 100 Mbps:

  • Cisco Nexus 9200 platform switches

  • Cisco Nexus 9300-EX/FX/FX2/GX platform switches

  • Cisco Nexus 9700-EX/FX line cards

• Beginning with Cisco NX-OS Release 10.1(2), Scheduling is supported on the N9K-X9624D-R2 and N9K-C9508-FM-R2 platform switches.

• For R2, though different priority levels can be set through CLI, only priority level 1 is supported in queuing policy.

• Beginning with Cisco NX-OS Release 10.3(1)F, Queuing statistics is supported on the Cisco Nexus 9808 platform switches. The queue depth counter per queue is not supported but additional queuing counters on VOQ tail drops are supported.

• Beginning with Cisco NX-OS Release 10.4(1)F, Queuing statistics is supported on the Cisco Nexus 9804 platform switches. The queue depth counter per queue is not supported but additional queuing counters on VOQ tail drops are supported.

• Cisco Nexus 9804 platform switches have the following limitations for Queuing and scheduling support:

  • Eight queues - SPAN and CPU Queues that are overloaded with eight user queues are supported.

  • SP, DWRR, WRED, and ECN are supported. However, the shaper and DWRR accuracy will have 5% variance

  • Maximum shaper and static limit are supported.

  • Micro-Burst Monitoring is not supported.

  • Link Level Flow Control is not supported.

  • Priority Flow Control is not supported.

  • Dynamic queue-limit is not supported.

  • Multicast Queuing statistics is not supported.

• Cisco Nexus 9808 switches support only the eight queue configuration in Queuing and Scheduling policies. Fewer queues can be configured but are not supported.

• Cisco Nexus 9804 switches support only the eight queue configuration in Queuing and Scheduling policies. Fewer queues can be configured but are not supported.

• The queue-limit configuration is applicable only in ingress queuing policy on Cisco Nexus 9500 switches with 9600-R/RX line cards.

• The bandwidth percent configuration is applicable only in egress queuing policy on Cisco Nexus 9500 switches with 9600-R/RX line cards.

• For Cisco Nexus 9300-EX Series switches, a queue is allocated a minimum bandwidth of 5%, regardless of whether a lower bandwidth is configured. In contrast, for Cisco Nexus 9300-FX Series and subsequent series switches, the minimum bandwidth allocated to a queue is 1%.

• If granted buffer is not carved out using a custom input queuing policy for a specified group, only global shared buffers are used.

• From Cisco NX-OS Release 10.4(1)F, Queuing and Scheduling policies are supported on Cisco Nexus C9348GCFX3.

• From Cisco NX-OS Release 10.4(1)F, below limitations apply on Cisco Nexus and C9348GC-FX3PH switch:

  • Queuing and Scheduling policies are supported on switch except for the ports 41-48.

  • Configuring WRED on egress queues is not supported on the switch.

• From Cisco NX-OS Release 10.4(2)F, Queuing and Scheduling policies are supported on Cisco Nexus C93108TC-FX3 switch.

Buffer-Boost

The buffer-boost feature enables the line card to use extra buffers. This capability is enabled by default on line cards such as the Cisco Nexus 9564PX.

• The command to enable the buffer-boost feature is buffer-boost .

• The command to disable the buffer-boost feature is no buffer-boost .

Generally, we recommends not to disable the buffer-boost feature. However, disabling the buffer-boost is necessary when there is a need to port channel two different member ports, from Cisco Nexus 9636PQ based line cards and Cisco Nexus 9564PX based line cards. However, we do not recommend to port channel such a configuration between ACI-capable leaf line cards and NX-OS line cards.

Note	Line cards like the Cisco Nexus 9636PQ and similar, do not offer the buffer-boost feature.

Queuing and scheduling are configured by creating policy maps of type queuing that you apply to an egress interface. You cannot modify system-defined class maps, which are used in policy maps to define the classes of traffic to which you want to apply policies.

System-defined class maps match based on QoS groups that can be customized using a type qos policy. By default, there is no type QoS policy and all traffic matches to qos-group 0. One consequence is that all traffic will hit the system-defined default-class of type network-qos and type queuing (assigns 100% bandwidth to qos-group 0). Since system-defined classes of type queuing and type network-qos are predefined to match based on distinct qos-groups and cannot be modified, the way to ensure that traffic hits a given type queuing/network-qos class is to configure a type qos policy that sets the corresponding qos-group for that traffic. For traffic classified into a system-defined class map matching on a qos-group other than 0, create a type QoS policy that sets the QoS groups. Once the traffic has been mapped, it will be subject to the default type network-qos and type queuing policies that operate on the non-default qos-group X (X !=0). You may need to further customize those type queuing and type network-qos policies in order to ensure the desired actions (e.g. re-allocate some bandwidth). For more information on setting the qos-group, see “Example of set qos-groups” in the Using Modular QoS CLI chapter.

For information about configuring policy maps and class maps, see the Using Modular QoS CLI chapter.

You can configure the congestion-avoidance features, which include tail drop and WRED, in any queue.

You can configure one of the egress congestion management features, such as priority, traffic shaping, and bandwidth in output queues.

Note	WRED is not supported on ALE enabled device front panel 40G uplink ports. When WRED is configured for the system level, the setting is ignored and no error message is displayed. When WRED is configured at the port level, the setting is rejected and an error message displays.

The system-defined policy map, default-out-policy, is attached to all ports to which you do not apply a queuing policy map. The default policy maps cannot be configured.

You can configure only one of the following congestion management methods in a policy map:

• Allocate a minimum data rate to a queue by using the bandwidth and bandwidth remaining commands.

• Allocate all data for a class of traffic to a priority queue by using the priority command. You can use the bandwidth remaining command to distribute remaining traffic among the nonpriority queues. By default, the system evenly distributes the remaining bandwidth among the nonpriority queues.

• Allocate a minimum and maximum data rate to a queue by using the shape command.

In addition to the congestion management feature that you choose, you can configure one of the following queue features in each class of a policy map:

• Tail drop thresholds based on the queue size and the queue limit usage. For more information, see Configuring Tail Drop on Egress Queues.

• WRED for preferential packet drops. For more information, see the Configuring WRED on Egress Queues section.

  Note	WRED is not supported on the Cisco Nexus 9508 switch (NX-OS 7.0(3)F3(3).

Use the following commands to verify the queuing and scheduling configuration:

The QoS buffer provides support per port/queue and shared space. You can control the QoS buffer that is shared by all flows by disabling or restricting reservations.

The hardware qos min-buffer command is used to control the QoS shared buffer.

The show hardware qos min-buffer command is used to display the current buffer configuration.

Beginning with NX-OS 7.0(3)I7(4), dynamic buffer sharing (egress buffering) across slices is configured with the hardware qos dynamic-buffer-sharing command. Following the command, you must reload the switch to enable the dynamic buffering.

Buffer sharing is enabled by dynamic bank allocation (1 bank = 4k cells, 1 cell = 416 bytes) and controlled by a global controller (eCPU) that manages the banks being distributed among slices. Dynamic buffer sharing provides six reserved banks (10MB) for each slice and twelve banks for sharing across slices (20MB).

Note	Dynamic Buffer Sharing is supported only on Nexus 9300-FX2 platform switches, see Nexus Switch Platform Support Matrix

The Cisco Nexus 9000 Series device has a 12-MB buffer memory that divides into a dedicated per port and dynamic shared memory. Each front-panel port has four unicast queues and four multicast queues in egress. In the scenario of burst or congestion, each egress port consumes buffers from the dynamic shared memory.

You can display the real-time and peak status of the shared buffer per port. All counters are displayed in terms of the number of cells. Each cell is 208 bytes in size. You can also display the global level buffer consumption in terms of consumption and available number of cells.

Note	Monitoring the shared buffer on ALE enabled devices is not supported for the port level.

Note	In the examples shown in this section, the port numbers are Broadcom ASIC ports.

switch# clear counters buffers
Max Cell Usage has been reset successfully

This example shows how to set a buffer utilization threshold for a specific module:

switch(config)# hardware profile buffer info port-threshold module 1 threshold 10
Port threshold changed successfully

Note	The buffer threshold feature is not enabled for ports if they have a no-drop class configured (PFC).

Note	The configured threshold buffer count is checked every 5 seconds against all the buffers used by that port across all the queues of that port.

Note	You can configure the threshold percentage configuration for all modules or for a specific module, which is applied to all ports. The default threshold value is 90% of the switch cell count of shared pool SP-0. This configuration applies to both Ethernet (front panel) and internal (HG) ports.

Note	The buffer threshold feature is not supported for ACI capable device ports.

switch# show interface hardware-mappings 
Legends:
       SMod  - Source Mod. 0 is N/A
       Unit  - Unit on which port resides. N/A for port channels
       HPort - Hardware Port Number or Hardware Trunk Id:
       FPort - Fabric facing port number. 255 means N/A
       NPort - Front panel port number
       VPort - Virtual Port Number. -1 means N/A

--------------------------------------------------------------------
Name       Ifindex  Smod Unit HPort FPort NPort VPort
--------------------------------------------------------------------
Eth2/1     1a080000 4    0    13    255   0     -1   
Eth2/2     1a080200 4    0    14    255   1     -1   
Eth2/3     1a080400 4    0    15    255   2     -1   
Eth2/4     1a080600 4    0    16    255   3     -1   
Eth2/5     1a080800 4    0    17    255   4     -1   
Eth2/6     1a080a00 4    0    18    255   5     -1   
Eth2/7     1a080c00 4    0    19    255   6     -1   
Eth2/8     1a080e00 4    0    20    255   7     -1   
Eth2/9     1a081000 4    0    21    255   8     -1   
Eth2/10    1a081200 4    0    22    255   9     -1   
Eth2/11    1a081400 4    0    23    255   10    -1   
Eth2/12    1a081600 4    0    24    255   11    -1   
Eth2/13    1a081800 4    0    25    255   12    -1   
Eth2/14    1a081a00 4    0    26    255   13    -1   
Eth2/15    1a081c00 4    0    27    255   14    -1   
Eth2/16    1a081e00 4    0    28    255   15    -1   
Eth2/17    1a082000 4    0    29    255   16    -1   
Eth2/18    1a082200 4    0    30    255   17    -1   
Eth2/19    1a082400 4    0    31    255   18    -1   
Eth2/20    1a082600 4    0    32    255   19    -1   
Eth2/21    1a082800 4    0    33    255   20    -1   
Eth2/22    1a082a00 4    0    34    255   21    -1   
Eth2/23    1a082c00 4    0    35    255   22    -1   
Eth2/24    1a082e00 4    0    36    255   23    -1 

In this section, you can find examples of configuring queuing and scheduling.

Note	The default system classes type queuing match based on qos-group (by default all traffic matches to qos-group 0, and this default queue gets 100% bandwidth). Create a type QoS policy that first sets the qos-group in order to drive the correct matching for the type queuing classes and policies.