Wireshark Use to Identify Bursty Traffic on Catalyst Switches

Introduction

This document describes how to identify burst traffic on the switchports of Cisco Catalyst switches.

Prerequisites

Requirements

There are no specific requirements for this document.

Components Used

The information in this document is based on the Cisco Catalyst Switch Series.

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command before executing the command.

Background Information

Traffic bursts can cause output drops even when the interface output rate is significantly lower than the maximum interface capacity. By default, the output rates in the show interface command are averaged over five minutes, which is not adequate to capture any short-lived bursts. It is best to average them over 30 seconds. In this case, you can use Wireshark in order to capture egress traffic with the Switched Port Analyzer (SPAN), which is analyzed in order to identify the bursts.

Troubleshooting Methodology

1. Identify an interface that has incremental output drops. For example, you notice output drops on a 100Mb link while the average utilization of the link is only 55Mb. Here is the output of the command:
   

   Switch#show int fa1/1 | i duplex|output drops|rate
     Full-duplex, 100Mb/s, media type is 10/100BaseTX
     Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 5756 
     5 minute input rate 55343353 bits/sec, 9677 packets/sec
     5 minute output rate 55456293 bits/sec, 9878 packets/sec

2. Configure SPAN on the switch in order to capture transmitted (TX) traffic. In order to capture this traffic, connect a PC that runs Wireshark and capture packets at the SPAN destination port.
   

   Switch#config t
   Switch(conf)#monitor session 1 source interface fa1/1 tx
   Switch(conf)#monitor session 1 destination interface fa1/2

3. Open the captured file in Wireshark and plot an IO graph like this one.
   

   

4. At the default scale, it appears that there is no bursty traffic. However, one second is a very large interval when you consider the rate at which buffering and packet switching happens. In a period of one second, a100 Mb/s link can accomodate 100 Mb of traffic across the interface in a neatly-shaped profile with a minimum need to buffer any packet.
   

   

   
   However, if a major portion of this traffic attempts to leave the interface in a fraction of a second, the switch needs to extensively buffer packets and drop them when the buffers are full. If you make the scales more granular, you see a more accurate picture of the actual traffic profile. Change the Y axis to bits/tick because interfaces show output rates in bits/sec.
   
   Link speed is 100 Mb/s
                  = 100,000,000 bits/s
                  = 100,000 bits/0.001 s
   
   Recalculate the scales on the X and Y axes. Change the tick interval to X Axis=0.001 sec and the scale to Y axis=00,000 (bits/tick).
   

   

5. Scroll through the graph in order to identify bursts. In this example, you can see that there is a burst of traffic that exceeded 100,000 bits on a 0.001 second scale. This confirms that traffic is bursty at the subsecond level and is expected to get dropped by the switch when the buffers are full in order to accommodate these bursts.
6. Click on the traffic spike on the graph in order to view that packet in the Wireshark capture. The capture analysis is a useful way to discover what traffic constitutes the burst.