Cisco Dynamic Ingress Rate Limiting Solution Overview
Available Languages
Bias-Free Language
The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.
The bridge to All-Flash arrays
Cisco® Dynamic Ingress Rate Limiting (DIRL) is an innovative technology to bridge the performance gaps between the newer ultra-fast All-Flash NVMe storage Arrays (AFAs) and slower application servers. DIRL prevents the spreading of congestion caused by performance issues or slow-drain conditions in a Storage Area Network (SAN). DIRL enables the slower and the faster application servers to coexist in the same fabric without affecting each other, and this innovation allows them to drive the AFAs to their full potential. The best part is that DIRL is not dependent on the end-devices, doesn’t require any additional license, and is available on Cisco MDS 9000 Series Multilayer Switches after a software-only upgrade.
What’s causing congestion in SAN?
All-Flash storage Arrays (AFAs) powered by NVMe and NVMe-over-Fabric technologies can drive millions of Input-and-Output operations Per Second (IOPS), with response times of a few microseconds, and throughput in hundreds of gigabytes per second. But applications in your data centers may not benefit from the full potential of the high-performing AFAs because of congestion issues. Sometimes the servers connected at slower speeds (4GFC, 8GFC, or 16GFC) cannot ingest the data from the faster speeds (32GFC) of the AFAs. At other times the network capacity is available, but the servers are overloaded and unable to ingest the fast response from the AFAs. In both cases, the servers build back-pressure within the SAN resulting in frame buffering and depletion of buffers on switch ports. Fibre Channel being a no-drop fabric doesn’t drop the frames, but it slows down the traffic on the Inter-Switch Links (ISLs) and the storage ports. This fabric-wide congestion affects other unrelated ports and devices as well.
● End-device independent – Upgrading of end-devices is not needed.
● Adaptive – DIRL dynamically adjusts as per the traffic profile of the host.
● Affordable – No additional license is needed.
● Easy adoption – DIRL is available on the Cisco MDS 9000 Series Multilayer Switches after a software-only upgrade.
● No side effects – The rate-limiting is applied only to the congested host. Other non-congested hosts and storage ports are not impacted.
● Topology independent – DIRL works in edge-core, edge-core-edge, or collapsed core (single switch fabric) topologies.
The recommendation with AFAs is to upgrade the end-to-end connectivity to 32GFC or 64GFC and to upgrade to high-capacity servers. Although a total upgrade is the best solution, it’s an expensive approach, and upgrading large environments is a multi-year project. While enterprises are deploying AFAs, they need a solution today—Cisco’s Dynamic Ingress Rate Limiting (DIRL) is the answer.
In block-storage networks, the storage arrays don’t send data by themselves, but the servers ask for data by initiating an I/O. The congestion in a SAN spreads because a server asks for more than it can ingest. The Cisco DIRL feature detects any symptoms of egress congestion on the switch ports. Then, it limits ingress data to prevent congestion in the egress direction. DIRL dynamically adapts the ingress traffic rate until the egress congestion goes away. By limiting ingress frames, DIRL also slows down the data-requesting frames (read I/O command) to the AFA. When the congested servers are allowed to ask for less, they will get less, and won’t cause any congestion. DIRL doesn’t drop any frames; rather, it uses the B2B credit pacing mechanism of the Cisco Fibre Channel ASICs to control the flow of frames. Because of this capability, slower and faster devices can coexist in the same fabric. Together they can push the efficiency of the AFAs to their maximum potential without negatively affecting each other.
Refer to the Cisco MDS 9000 Series Multilayer Switches configuration guides for more details.