Troubleshoot Hardware for Catalyst 4000 Series Switches
Available Languages
Contents
Introduction
This document provides troubleshooting procedures on how to diagnose hardware problems on Catalyst 4000 family switches. The Catalyst 4000 family includes the 4003 and 4006 modular chassis and the 2948G, 2980G, and 4912G fixed models. The naming conventions for the Catalyst 4000 and Catalyst 2900 can be very confusing. Refer to Understanding Catalyst 2900 and Catalyst 4000 Naming Conventions for more information on how to help clarify these issues.
The goal is to help Cisco customers identify and fix some basic hardware issues, or to perform more extensive troubleshooting before you contact Cisco Technical Support. An orderly troubleshooting process with the collection of specific diagnostics ensures that information necessary to the resolution of the problem is not lost. If you refine the scope of the problem, this saves valuable time in the search for a solution.
Prerequisites
Requirements
Cisco recommends that you have knowledge of these topics:
-
Catalyst 4000 Command Reference
Components Used
This document is not restricted to specific software and hardware versions.
Conventions
Refer to Cisco Technical Tips Conventions for more information on document conventions.
Preparation for Troubleshooting Hardware on Catalyst Switches
Many hardware problems encountered during field installations or during normal operation can be prevented by a thorough product overview ahead of time. For those customers not already familiar with general system and power requirements, proper installation procedure, switch management and software considerations for these switches, Cisco recommends that you read documents in Cisco Catalyst 4000 Series Switches Troubleshooting TechNotes.
This document covers this important information:
-
Which supervisor is supported in which chassis?
-
How do I back up my configuration?
-
Which software version is General Deployment (GD) for the Catalyst 4000 Family?
This document assumes familiarity with the Catalyst 4000 Command Reference. You should also have a prior understanding of switching fundamentals, or have read How LAN Switches Work. Additional online documentation is referenced throughout this document in order to assist in troubleshooting.
Online Troubleshooting Tools
Cisco has a variety of troubleshooting tools and resources in order to help you interpret switch output, determine hardware software compatibility, track bugs, and search field notices. These tools and resources are referenced throughout this document:
-
Output Interpreter (registered customers only) —Paste in the output of a command and get the interpretation with relevant errors, warnings, and status information.
-
Bug Toolkit (registered customers only) —Search for bugs.
-
Troubleshooting Assistant—This provides step-by-step instructions to many common network issues.
Catalyst 4000 Family Troubleshooting Procedures
This section discusses troubleshooting procedures, symptoms, show commands, and diagnostics for the Catalyst 4000 family. This section assumes you have read the companion guide to this document, as described in the Introduction of this document, and that you understand your switch and its capabilities.
Note: If the switch is connected to the network, do not reset or reseat modules as a first troubleshooting step! In addition to the downtime that users experience, the internal buffer, which logs system messages are erased and potentially useful information in regards to hardware or software errors are lost. If the switch is offline, you have more freedom to monitor LED status, pull cables, reseat modules, or reset the switch as necessary. Troubleshooting LED status is discussed in more detail later in this document.
Hidden Commands
Some commands presented in this document are known as hidden, which means that they cannot be parsed with a "?", and you cannot Tab in order to complete. When a hidden command is suggested in this document, simply gather the output and send it to the TAC engineer, if you open a case. It is possible that this output is useful in solving your case. These commands are undocumented, and therefore the TAC engineer is not required to explain the output to the customer.
If you want to troubleshoot any problem, this requires a method or set of procedures which, if followed correctly, produces a solution. Begin by understanding general problem solving for LAN networks.
General Problem Solving Model
If you want to troubleshoot any problem, this requires a method or set of procedures which, if followed correctly, produces a solution. Begin by understanding general problem solving for LAN networks. Hardware failures in LAN networks are characterized by certain symptoms. These symptoms can be general such as the inability to Telnet between switches, more specific such as link flapping, or perhaps the switch is resetting itself. Each symptom can be traced to one or more causes if you use specific troubleshooting techniques. A systematic approach works best. Define the specific symptoms, identify all potential problems that could be causing the symptoms, and then eliminate each potential problem, from most likely to least likely, until the symptoms disappear.
General Problem Solving Flow Chart
This diagram outlines the steps that detail the problem-solving process:
Complete these steps:
-
Define the problem.
It is important to first identify the problem being experienced. This allows you to identify what kinds of causes can result in these symptoms. In order to help determine the problem, ask yourself these questions:
-
What is the primary symptom?
-
Is the problem specific to this switch or does it affect other switches on the network as well?
-
Is this a problem with one or more ports on a specific module? What type of ports: 10/100, Multimode Fiber (MMF), Singlemode Fiber (SMF), GigabitEthernet, and so forth?
-
What device is connected to the switch ports that experiences the problem?
-
When did this problem first occur and has it occurred more than once?
-
What happened at the time the problem was first noticed? Is there anything unique about traffic conditions at that time of day? For example, was this a peak time for traffic?
-
Did you run any particular commands at the time or make any configuration changes?
-
-
Gather the facts.
Gather diagnostics and show commands output from the switch to isolate the scope of the problem. If physical access to the equipment is possible, locate and list any modules with red or yellow LEDs, disconnected cables, or loose connections.
-
Consider the possible causes.
Consider possible problems based on the information you gathered. With certain data, you are able, for example, to eliminate hardware as a problem, so that you can focus on software problems. At every opportunity, try to narrow the number of potential problems so that you can create an effective plan of action.
-
Create and implement an action plan.
Create an action plan based on the potential problems. Focus on only one potential problem at a time. If you alter more than one variable simultaneously, you can solve the problem, but the identification of the specific change that eliminated the symptom becomes far more difficult and does not help you solve the same problem if it occurs in the future.
-
Observe the results.
Be sure to gather and analyze the results each time a variable is changed to determine if the problem has been fixed.
-
Repeat the process.
Repeat testing for possible causes until the problem is resolved.
Common Problems
As described in the Problem Solving Model, the first step in resolving a problem is to identify the symptom. Refer to Catalyst Troubleshooting Tips for more information on some common problems associated with all Catalyst switches that can be resolved.
Most hardware problems with LAN networks fall into these categories and each category has various symptoms related to it:
-
Connectivity Problems
-
System/Supervisor/Module Problems
-
Supervisor Crashes
Connectivity Problems
These problems can occur when communication with the supervisor, module, or hosts connected to the module is intermittent or has been lost.
System/Supervisor/Module Problems
These problems can occur when system status LEDs indicate a problem, the supervisor or modules are not recognized or show faulty, or when users experience poor performance.
Supervisor Crashes
These problems can occur when the switch has reset, continually resets, or is down completely.
Symptom Description
This section discusses symptoms, troubleshooting procedures, and commands for Catalyst 4000 family switches. This section assumes you are able to identify your switch chassis, supervisor engine, modules, and feature cards, and that you understand the system specifications, cabling, power, and software requirements as described for Cisco Catalyst 4500 Series Switches Install and Upgrade Guides.
If you have not determined what your primary symptom is, see the General Problem Solving Model section of this document and apply the steps to your problem.
Connectivity Problems and Steps to Resolve Them
This section covers common connectivity issues that the customer can encounter with the Catalyst 4000.
These commands are supported by the Output Interpreter tool for CatOS and can be used to assist in troubleshooting switch port problems:
-
show version
-
show module
-
show system
-
show port
-
show mac
-
show counters
-
show cdp neighbors detail
If you have the output of the supported commands from your Cisco device, you can use the Output Interpreter (registered customers only) to display potential issues and fixes. In order to use Output Interpreter (registered customers only) , you must be a registered user, be logged in, and have JavaScript enabled.
Can not console/Telnet into the supervisor
Both of these problems are covered in the Catalyst Troubleshooting Tips document that is mentioned earlier.
-
Not able to console
-
Verify that the power switch is in the ON position (|) and the system OK LED is ON.
-
Connect the cable directly to the console port and not through a patch panel.
-
Verify that the correct cabling and hardware is used to connect to your particular supervisor engine. Refer to the Connecting a Terminal to the Console Port on Catalyst Switches document for more information.
-
-
Not able to Telnet
-
Complete the steps in thedetailed procedure described in Catalyst Troubleshooting Tips. If it is determined that the sc0 management interface is not configured or not configured correctly, refer to Configuring an IP Address on Catalyst Switches for more information.
-
Attempt to Telnet from a PC directly connected to the switch in the same VLAN as the sc0 interface in order to eliminate any routing issues.
-
Gain console access to the switch and make sure the supervisor is not in boot> or rommon>. If the switch is in one of these modes, you need to complete the steps in the recovery procedures. Refer to Recovering Catalyst 4000 and Catalyst 5000 Switches from Corrupted or Missing Software, or an Upgrade Failure, or from ROMMON Mode for more information on recovery.
-
Receiving the "Failed to allocate session block" error message
If you receive the Failed to allocate session block error message while you access the switch on the Telnet, the problem occurs because the switch cannot allocate the required memory for the Telnet application. The available free memory is low because of some process that uses more memory or because of a memory leakage in the switch.
In order to avoid the error, issue the show proc mem command and verify the process that uses more memory in the switch. In order to resolve the problem, either add more memory to the system or disable some features in order to free some of the existing memory.
If there is memory leak in the switch, reset the switch in order to release all the process in the memory. If the error message still appears even after you reboot, upgrade the software version of the switch.
Can not connect to a remote host, router, or another switch
Complete these steps:
-
Verify that the port LED status is green. If the link LED is solid orange, it has been disabled by the software. If it is blinking orange after supervisor bootup and module initialization, this is a hardware failure. If there is no link LED, check and swap the cables. Verify operation of the end device and NIC.
Refer to Troubleshooting Cisco Catalyst Switches to NIC Compatibility Issues for more information on NIC troubleshooting.
-
What type of media is involved? Fiber? Gigabit Interface Converter (GBIC)? Gigabit Ethernet? 10/100 BaseTX? If this a physical layer issue, refer to the Physical Layer Troubleshooting section of Troubleshooting Switch Port Problems for more information.
-
Issue the show port <mod/port> command in order to verify that the status is connected, which means that the port is operational. If any other status is displayed, see the Port status shows not connected, faulty, disabled, inactive, or errdisable section for troubleshooting steps.
If the end device is a Cisco router or switch, and Cisco Discovery Protocol (CDP) is enabled, issue the show cdp neighbor detail command in order to identify the device, remote interface type, and remote IP address.
Note: A status of connected does not mean that the ports are free of errors. If there are errors on the ports, proceed to the Seeing errors on the ports section of this document.
-
Swap the cables. Move the cable to a different port. Eliminate patch panels. Patch panels are a common source of connectivity failures, so attempt to connect directly to the end device. Verify the operation of the end device.
-
Capture the output of the show config , show module , and show test 0 commands.
-
Issue the show module command in order to verify that the status is ok for that module and not disabled or faulty.
-
If the status is disabled, issue the set module enable <mod> command.
-
If the status is faulty, establish a console connection to capture bootup Power On Self Test (POST) diagnostics and any system error messages. Issue the reset <mod> command in order to reset the module. Issue the show test 0 command in order to determine if this module passed all of it's diagnostic tests on bootup.
-
Remove the module and inspect for bent pins. Reseat the module, firmly press down the ejector levers, and tighten the captive installation screws. If the output of the show module command status is still faulty, try the module in another slot. Slot 2 accepts line cards or a supervisor engine. If necessary, power off/on the switch. If the status is still faulty, the module has failed.
-
-
Issue the show test 0 command in order to verify that the port has passed its last diagnostic test on bootup. If F is indicted for that port, proceed as in step a.
-
-
Verify whether this device is on the same or a different VLAN. Remember that this is a Layer 2 (L2) device and a router is required to route between VLANs.
-
If you connect to another switch, ask yourself these questions:
-
What type of port is this? A trunk port?
-
If it is a trunk port, what trunk encapsulations does it support?
-
Is the port capable of EtherChannel?
Issue the show port capabilities command for a quick look at port capabilities. Refer to LAN Technical Tips for more information on how to troubleshoot issues with trunking or EtherChannel.
-
Port status shows not connected, faulty, disabled, inactive, or errdisable
Possible Port Status
| Status | Description and Work Around |
|---|---|
connected |
Port is operational and connected to end device. A status of connected does not mean the ports are error-free. If there are errors on the ports, proceed to the Seeing errors on the ports section of this document. |
notconnect |
Nothing is connected to the port. Check or swap cables. Verify the operation of the end device. |
faulty |
Possible hardware failure. Issue the show test command in order to verify. If F displays for a port, proceed as in step 5 of the Can not connect to a remote host on the switch section of this document. |
disabled |
Manually disabled. Issue the set port enable <mod/port> command in order to enable the port. If the port status does not change to enable, issue the show module command in order to determine if the module is disabled. |
inactive |
Port belongs to a VLAN that does not exist. Issue the set vlan <vlan> command in order to add a VLAN. |
errdisable |
Port had been shut down due to errors. Refer to the Recovering From errDisable Port State on the CatOS Platforms document for more information. |
Seeing errors on the ports
Complaints of poor performance by users can sometimes translate to errors on switch ports. Output from the port error counters command help you troubleshoot connectivity problems.
-
Verify port status and troubleshoot accordingly. Refer to the Port status shows not connected, faulty, disabled, inactive, or errdisable section of this document.
-
Capture the output of the show port <mod/port> , show mac <mod/port> , and show counters <mod/port> commands.
These are common causes for data link errors on ports:
-
speed/duplex misconfiguration
-
network congestion
-
NICs or drivers
Refer to Troubleshooting Cisco Catalyst Switches to NIC Compatibility Issues for more information.
-
cabling
-
bad port
The show port <mod/port> command can show Late-Coll, Align-Err, FCS-Err, Xmit-Err, and Rcv-Err errors. Refer to the the Show Port for CatOS and Show Interfaces for Cisco IOS section of Troubleshooting Switch Port Problems for more informaiton on these errors and possible causes.
The show mac <mod/port> command shows the number of unicast, multicast, and broadcast frames transmitted. Issue this command in order to verify if frames are received and transmitted.
In-Discards show frames that do not need to be switched. This is normal if the port was connected to a hub and two devices exchanged data. Lrn-Discards indicate that Content Addressable Memory (CAM) entries are being discarded. In-Lost counter displays the sum of all error packets received on the port. The Out-Lost counter indicates egress port buffer overflows. Refer to the Show Mac for CatOS and Show Interfaces Counters for Cisco IOS section of Troubleshooting Switch Port Problems for more information on these errors and possible causes.
The show counters <mod/port> command is useful in particular for troubleshooting port problems.
For example, this counter results if you issue the command:
5 badTxCRC = 0
If badTxCRC were incrementing, this can be bad hardware corrupting packets. Capture the output of the show counters <mod/port> command and open a case with the Cisco Technical Support.
-
-
Issue the clear counters command in order to reset the output of the show port <mod/port>, show mac <mod/port>, and show counters <mod/port> commands. View the command outputs several times in order to see if errors are incrementing.
If you have not been able to track down any reason for intermittent connectivity loss on the switch in the previous steps mentioned, capture the output of the show nvramenv 1 command, as well as the other commands in the previous steps, and open a case with the Cisco Technical Support.
-
Refer to these documents for more information on how to troubleshoot the other causes of port errors:
Experiencing poor performance
Poor performance is often perceived to be a hardware problem, when in fact it can be attributed most often to connectivity problems. See the Seeing errors on the ports section for troubleshooting steps.
Getting continuous %PAGP-5 left/joined bridge messages
Complete these steps:
-
Capture the show port <mod/port>, show mac <mod/port>, and show spantree summary command output.
System messages similar to these messages are informational, although if the errors continue to repeat, the link can be flapping.
2002 Jan 19 14:59:05 %PAGP-5-PORTFROMSTP:Port 2/11 left bridge port 2/11 2002 Jan 19 14:59:23 %PAGP-5-PORTTOSTP:Port 2/11 joined bridge port 2/11
-
If these messages occur repeatedly on certain ports, refer to these document for possible causes:
-
If you also see errors on the port in the show port <mod/port> and show mac<mod/port> command output, see the Seeing errors on the ports section for troubleshooting steps.
-
Issue the show spantree summary command in order to verify how many ports are in each VLAN, if any ports on the switch are blocking, and which VLANs are being blocked. Since Spanning-Tree Protocol (STP) loops can cause link flaps or actually bring down a switch or network, with the appearance of a hardware failure, this is vital information to capture, whether troubleshooting hardware or software. Refer to LAN Technical Tips for more information on how to troubleshoot STP.
Can not autonegotiate or speed/duplex mismatch
Complete these steps:
-
Make sure you have speed and duplex configured identically on both sides of the link. Catalyst 4000 switchports are set to auto by default. When both sides of a 100 BaseTX link autonegotiate correctly, the show port <mod/port> command output is as follows:
Duplex Speed ------- ------- a-full a-100
Hardcode both sides. Remember when hardcoding the port, the port speed must be set first and then the duplex setting must be set. Issue the show port <mod/port> command. The switch output is as follows:
Duplex Speed ------- ------- full 100
Note: Even though the switch has been hard coded, the connecting device must still be hardcoded to eliminate problems.
-
If there is an autonegotiation problem caused by a speed/duplex mismatch or NIC incompatibility, errors show up on the ports. Refer to these documents for more information:
System/Supervisor/Module Problems and Steps to Resolve Them
System, supervisor, and module problems occur when either system status LEDs indicate a problem, the supervisor or modules are not recognized or show faulty, or when users are experiencing poor performance.
The following commands are supported by the Output Interpreter and can be used to assist in troubleshooting system, supervisor, and module problems: show version, show module, or show system.
If you have the output of the supported commands from your Cisco device, you can use the Output Interpreter (registered customers only) in order to display potential issues and fixes. In order to use Output Interpreter (registered customers only) , you must be a registered user, be logged in, and have JavaScript enabled.
Having problems upgrading software
Complete these steps:
-
Most customer problems that have to do with software upgrades are the result of not understanding the copy tftp procedure, the boot process, or the Flash system for the supervisor.
Refer to Working with System Software Images for more information, specifically, on the copy tftp procedure for your supervisor.
Refer to Using the Flash File System for more information on the Flash file system for your supervisor.
Refer to Recovering Catalyst Switches Running CatOS from Booting Failures for more information on rommon recovery information.
Capture the show version, show flash, or dir bootflash command output, which depends on the type of supervisor you have. Verify that you have enough DRAM and Flash for the image to which you attempt to upgrade, and then perform the copy tftp procedure.
-
Set the boot environment variable and the config-register. Refer to Modifying the Switch Boot Configuration for more information on these settings.
Cat4000-c> (enable) set boot ? auto-config Set auto config file config-register Set configuration register sync Set sync parameters system Set BOOT environment variable
Cisco recommends that you set the boot environment variable and config-register in this way:
-
Verify the image that you want to boot, currently installed in Flash. Issue the dir bootflash: command.
Cat4000-c> (enable) dir bootflash: -#- -length- -----date/time------ name 1 4106492 Aug 17 2001 16:22:52 cat4000.6-3-1.bin 2 3554592 Nov 28 2001 10:38:33 cat4000.5-5-11.bin 3 4199168 Dec 07 2001 10:30:01 cat4000-k9.6-3-3.bin 4 3651336 DEC 11 2001 12:26:20 cat4000.5-5-8.bin 216540 bytes available (15512100 bytes used)
-
Set the boot environment variable for the image in Flash from which you want to boot.
Cat4000-c> (enable) set boot system flash bootflash:cat4000.6-3-1.bin BOOT variable = bootflash:cat4000.6-3-1.bin,1;
-
Set the config-register to boot from Flash.
Cat4000-c> (enable) set boot config-register 0x2102 Configuration register is 0x2102 ignore-config: disabled auto-config: non-recurring console baud: 9600 boot: image specified by the boot system commands
-
-
If you end in rommon or boot mode during the upgrade, refer to Recovering Catalyst Switches Running CatOS from Booting Failures for more information.
-
Use the Bug Toolkit to track down bugs, or refer to Release Notes for Catalyst 4000 Family Software Release 5.x for Caveats.
Supervisor is not online or is stuck in boot or rommon
The most common causes for a Catalyst 4000 family supervisor not to be recognized is when it is stuck in boot or rommon mode due to a missing or corrupt image. In these modes, you are not able to Telnet to the supervisor and must have a console session open.
-
If the supervisor is stuck in either boot or rommon mode, complete the troubleshooting steps in Recovering Catalyst Switches Running CatOS from Booting Failures.
-
If the supervisor is not in either boot or rommon mode but is still not online, complete the troubleshooting steps for the Supervisor Engine in the System component LEDs are orange/red section of this document.
System component LEDs are orange/red or supervisor not online
Complete these steps:
-
If you observe orange or red LEDs on startup, wait until the system boots up completely before concluding that there is a problem. The system status LED on the supervisor will stay orange until bootup is complete, then turn green if bootup is successful. One cause of an orange sys-status LED is a fan failure.
Next, the supervisor initializes the switching modules, which operate differently depending on the module; some flash on and off, and others stay orange until initialization is complete. At this point, the link (port) LEDs turns off altogether until a signal is detected.
-
Understand the Catalyst 4000 family components and what the LEDs tell you. As a starting place, refer to Troubleshooting the Installation for more information:
Look at the front panel LEDs for your supervisor. Refer to these documents for more information:
Look at the front panel LEDs for your switching module. Refer to Catalyst 4500 E-Series Module Installation Note for more information:
-
Capture the show version, show system, show module, and show test 0 command output.
Power supply—includes the power supplies and power supply fans. The PS1, PS2, and PS3, for the Catalyst 4006, status LEDs should be green. If one or both are red, this can indicate a power supply failure.
-
When you issue the show system command, determine if PS1 or PS2 status is faulty.
Note: The Catalyst 4006 requires two power supplies installed to operate the switch and the third is for redundancy. Refer to Module Overview for more information.
-
Inspect the power supplies. Make sure there is power applied to both units. If a redundant power supply is installed but has no power, the show system command output shows that the power supply status and sys-status is faulty.
-
Reseat the power supply. Try a different circuit or swap power cords. If the status is still red, or the show system command output shows faulty, this is a power supply failure. Refer to Removal and Replacement Procedures for more information.
Fan assembly—Whenever system power is on, the system fan assembly should operate. You should be able to hear the fan assembly to determine if it operates.
-
Inspect the fan assembly and power supplies to verify if power is being applied to the system.
-
Issue the show system command to determine if the fan-status is faulty.
-
Reseat the fan assembly and tighten the captive installation screws. If necessary, reset the switch. If the show system command output still shows faulty, this is a fan failure. Refer to Removal and Replacement Procedures for more information.
Supervisor engine—The supervisor engine contains the system operating software. Check the supervisor engine if you have trouble with the system software. The status LED on the supervisor engine indicates whether the supervisor engine has passed all diagnostic tests. Have a console session open and determine whether the supervisor is in boot or rommon mode. If this is the case, see the Supervisor is not online or stuck in rommon section for troubleshooting steps.
-
Issue the show system command in order to determine if the sys-status is faulty. Issue the show test 0 command in order to determine if the supervisor has passed all diagnostic tests as of the last bootup of the switch. Note any F for fail results.
-
Inspect the fan assembly and power supplies for any problems.
-
Have a console session open and capture bootup POST diagnostics and system error messages. Reset the switch and issue the show test 0 command in order to determine if the diagnostic test on bootup has been passed.
-
Remove the supervisor and inspect for bent pins. Reseat the supervisor, firmly press down the ejector levers, and tighten the captive installation screws. Wait for the supervisor to initialize. If the show system command sys-status is still faulty, the supervisor has failed.
Switching modules—The status LEDs on each switching module indicate whether the switching module has been initialized correctly. The supervisor engine must be operating properly before the switching module initializes. If a switching module is improperly installed in the switch, it does not function.
-
If a link (port) LED is solid orange or is blinking orange after the supervisor bootup and module initialization, see the Can not connect to a remote host, router, or another switch section.
-
Capture the show version and show module command output. Determine if the software version you are running supports this module. Refer to the Module Overview section of Catalyst 4500 E-Series Module Installation Note for more information.
-
Determine if the status is disable. This indicates that the module was administratively disabled. The status LED is orange in this case. Issue the set module enable <mod> command.
-
View the output of the show module command in order to determine if status is faulty for that module. View the output of the show test 0 command in order to determine if this module passed all its diagnostic tests as of the last bootup of the switch. Note any F for fail results.
-
Have a console session open and capture bootup POST diagnostics and any system error messages. Issue the reset <mod> command in order to reset the module. Issue the show test 0 command in order to determine if this module has passed all of its diagnostic tests on bootup. Note any F for fail results.
-
Remove the module and inspect for bent pins. Reseat the module, firmly press down the ejector levers, and tighten the captive installation screws. If the show module status is still faulty, try the module in another slot. If necessary, power off/on the switch. If status is still faulty, the module has failed.
-
Switching module is not recognized
The most common cause for a switching module or a line card to not be recognized is due to the wrong version of software.
-
Determine that this is a problem with just one module and not all modules. If all modules are affected, complete the steps in the System component LEDs are orange/red or supervisor not online section. Capture the output the show version , show module , and show test 0 commands.
-
Issue the show version command in order to check the model number of the module you have problems with and the software version you use. Determine the total DRAM and total Flash. Refer to the Module Overview section of Catalyst 4500 E-Series Module Installation Note in order to determine if the hardware is compatible with the software.
If an upgrade is required, always check the release notes first. Refer to the to Catalyst 4500 Family Release Notes and choose the version to which you need to upgrade. For example, choose the Release Notes for Catalyst 4000 Family Software Release 5.x and search on any information in regards to your hardware.
Refer to Managing Software Images and Working with Configuration Files on Catalyst Switches for more information.
-
If the supervisor is not stuck in boot or rommon and you have determined that the module is supported by the current version of software, complete the steps for troubleshooting the Switching Module in the System component LEDs are orange/red or supervisor not online section.
Module status is showing faulty or not ok
Complete these steps:
-
Capture the show module and show test 0 command output.
-
For any status other than ok in the output of these two commands, complete the troubleshooting steps for the Switching Module in the System component LEDs are orange/red or supervisor not online section.
Experiencing poor performance
Poor performance is often perceived to be a hardware issue, but this is usually not the case. When customers describe to Cisco Technical Support that users on a particular switch experience slow performance, this often turns out to be related to connectivity problems, software misconfiguration, or problems elsewhere on the network.
-
Identify whether performance issues occur for users connected to all switching modules, one module in particular, or just users on one or more ports. Capture the show module and show test 0 command output. Make sure that the supervisor and modules have an ok status. If there is a faulty status, complete the troubleshooting steps for the Switching Module in the System component LEDs are orange/red or supervisor not online section.
-
Capture the show port <mod/port> , show Mac<mod/port> , and show counters <mod/port> command output. If you see incrementing errors on port counters, troubleshoot this performance issue as a connectivity issue. See the Seeing errors on the ports section for troubleshooting steps.
-
Capture the show config and show logging buffer 1023 command output. The show config command shows only the non-default configuration changes. Ideally, every time you make a change, you should have backed-up the configuration to use as a comparison. Issue the show config command in order to possibly associate a configuration change with the behavior you experience.
If you see any system messages other than informational messages that can indicate a hardware or some other problem, issue the show logging buffer 1023 command in order to capture these messages. This command displays the last 1023 system messages with timestamps, by default. Also, refer to Messages and Recovery Procedures well as Common CatOS Error Messages on Catalyst 4000 Series Switches in order to see if you can rule out any harmless system messages from those that can indicate a problem.
-
Many performance related problems are related to network traffic conditions. Capture the show system command output in order to see if this is a network traffic problem.
The show system command can be used to check the current backplane utilization, which is typically less than ten percent. If you believe that you are having performance related issues on a particular switch, look at the Peak field, which is the peak backplane utilization on the switch since it was last booted, and note the timestamp indicated by Peak-Time. Keep in mind that spikes in traffic percentage on the backplane can be a STP loop or broadcast storm. Refer to Spanning Tree Protocol Problems and Related Design Considerations for more information.
-
Capture the show proc cpu command output. This command helps identify a process that can cause high CPU utilization on the supervisor. This is an excerpt of show proc cpu command output:
Cat4000-c> (enable) show proc cpu CPU utilization for five seconds: 11.62% one minute: 12.00% five minutes: 12.00% PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process --- ----------- ---------- -------- ------- ------- ------- --- --------------- 1 20176816 0 0 88.38% 88.00% 88.00% -2 Kernel and IdleWhen you view the output of this command, remember that the CPU utilization is the first thing shown. Do not confuse the Kernel and Idle amount as CPU utilization. The Kernel and Idle is the percentage of CPU that was idle for that time frame. Therefore, in the past five minutes, only 11.62 percent of the CPU was utilized, which is within typical boundaries.
Refer to Understanding CPU Utilization on Catalyst 4000, 2948G, 2980G, and 4912G Switches for more information and a complete understanding of how the CPU is utilized on Catalyst 4000, 2948G, 2980G, and 4912G switches.
Complete these steps in order to get a baseline of your switch and help identify which process can cause a problem:
-
Issue the show proc cpu command during a time of normal activity for your network. Save the results.
-
Run this command again if you experience any performance related issues.
-
Compare the two outputs. Is there a process you can identify that is unusually high in comparison?
-
Run the command multiple times. Is there a significant increase or decrease in CPU utilization or spikes? Or, does CPU utilization remain consistently high?
The answer is most likely not a hardware problem, but points elsewhere.
-
-
One performance related issue that results from misconfiguration is when the inband channel, which is used for any control traffic terminating on the switch such as ping, Telnet, VLAN Trunk Protocol (VTP), STP, CDP, and so forth, is not put in a separate VLAN from user data.
It is always recommended to keep the management or sc0 interface of the switch in a separate VLAN from the user data. Otherwise, any broadcast or multicast storm can flood the inband channel to the Network Management Processor (NMP), which needs to be free to handle the protocols just mentioned.
If you have not been able to track down any reason for performance issues on the switch in the previous steps mentioned, capture the output of these commands, as well as the other commands in the previous steps, and open a case with the Cisco Technical Support:
-
show nvramenv 1 (hidden)
-
show interposition 1 (hidden)
These are hidden commands, which means they cannot be parsed with a "?" and you cannot Tab to complete. Type the command out in its entirety. It is possible that this output is not useful in the resolution your case. These commands are undocumented, and therefore the TAC engineer is not required to explain the output to the customer.
-
-
Although fairly rare, memory leaks do occur and can cause what seem naturally to be poor performance and other symptoms. If you have not been able to track down any reason for performance issues on the switch in the previous steps mentioned, capture the output of the show mbuf total (hidden) command, as well as the other commands in the previous steps, and open a case with the Cisco Technical Support.
There are two things to consider when you look at the output of this command in order to help determine if you have a memory leak issue:
-
Look at the output and if the free mbufs or clusters values decrease but never increase, this can indicate a possible memory leak.
-
Look at the output, and if the lowest free memory has ever approached zero or was at zero, this indicates the switch either runs low on or has ran out of memory.
Both of these issues indicate a memory issue that obviously affects the protocols/processes that require this memory.
Cat4000-c> (enable) show mbuf total mbufs 9280 clusters 3660 free mbufs 9256 clfree 3659 lowest free mbufs 9235 lowest clfree 3638
These are hidden commands, which means they cannot be parsed with a "?" and you cannot Tab to complete. Type the command out in its entirety. It is possible that this output is not useful in the resolution your case. These commands are undocumented, and therefore the TAC engineer is not required to explain the output to the customer.
-
Getting system error messages
As mentioned in the Introduction of this document, Cisco has a suite of online diagnostic tools to help you determine hw/sw compatibility, interpret output, and decode errors.
-
System messages have timestamps by default, which can help in isolating a timeframe for your problem. by Issue the show time command in order to make sure your system clock is set correctly. Also, verify that your connecting devices are set so that the logs match.
-
Capture the output of any system messages with the show logging buffer 1023 command. Many system messages are informational in nature while others can indicate a problem. Refer to these documents for more information:
Supervisor Crashes and Steps to Resolve Them
Supervisor crashes occur when the switch has reset, continually resets, or is down completely.
These commands are supported by the Output Interpreter and can be used to assist in troubleshooting supervisor crashes: show version or show system.
If you have the output of the supported commands from your Cisco device, you can use Output Interpreter (registered customers only) in order to display potential issues and fixes. In order to use Output Interpreter (registered customers only) , you must be a registered user, be logged in, and have JavaScript enabled.
Getting system error messages
System error messages can be useful if you experience a switch reset. See the Getting system error messages section for more information.
Switch has reset or is continually resetting
If the switch has reset or crashed due to a reason related to hardware or software, it is important to capture the output of certain show commands as quickly as possible.
-
Capture the show log, show version, show test 0,and show logging buffer 1023 command output.
The show log command output has a number of important indications of problems that can be related to a crash.
-
It keeps track of the last ten system resets with timestamps that show when the reboot occurred. This is a snapshot of the Reboot History output:
Reboot History: Jan 23 2002 11:14:16 0, Jan 22 2002 14:57:21 0 DEC 24 2001 13:56:38 0, DEC 24 2001 13:52:30 0 DEC 11 2001 12:31:59 0, DEC 07 2001 13:26:48 0 DEC 07 2001 10:42:19 0, DEC 07 2001 10:36:16 0 Nov 28 2001 11:03:10 0, Oct 26 2001 16:04:26 0The Reboot History only indicates that the switch was reset. It can have been reset manually by the user or due to a crash. But, the most recent manual reset of the switch is recorded further down in the output.
Last software reset by user: Jan 23 2002 11:14:16 0
Notice that the timestamp of the last manual reset, 1/23/2002,11:13:13, matches the most recent entry in the Reboot History.
-
It shows if there have been any exceptions. Exceptions are CPU dumps that occur immediately after a crash. For example:
MCP Exceptions/Hang: 0
In this case, there were no exceptions recorded. If there were an exception, it includes a timestamp that can be matched with the Reboot History, and also includes a HEX dump or stack, which can be decoded by a TAC engineer in order to determine whether this was a software forced exception or due to hardware.
The show version command provides software version information to use for a bug search. For example, if you identify an exception in the show log command output, use the Bug Toolkit in order to search for bugs on the Catalyst 4000 and the exception. Also, the show version command gives you a quick snapshot of how long the switch has been up. For example:
Uptime is 28 days, 11 hours, 42 minutes
The show test 0 command output indicates an F status on the supervisor or module if any of the diagnostics failed. An improperly seated module can cause the switch to crash. If the supervisor or module shows failed, proceed with the troubleshooting steps in the System component LEDs are orange/red or supervisor not online section of this document.
The show logging buffer 1023 command displays all system messages, which includes possible error messages that can relate to the crash. See the Getting system error messages section for troubleshooting suggestions.
-
-
Issue the show commands and troubleshooting procedures in the preceding steps first. If these steps fail, capture the show tech-support command output. This command displays output for all these commands continuously, which means the output continues to scroll until complete or until the display is ended with the Ctrl + C keystrokes:
sh version, sh flash, sh microcode, sh system, sh module, sh port, sh Mac, sh trunk, sh vlan, sh vtp domain, sh spantree active, sh spantree summary, sh test, sh arp, sh ip route, sh cdp neighbor detail, sh netstat stats, show memory buffers, show out-of-band stats, sh inband stats, show cam static, sh cam count dynamic, sh cam system, sh config, sh log, sh proc, sh proc mem, sh proc CPU, ps, Ps -c
Often, the output from all these commands is not necessary to resolve a specific problem, so TAC engineers cannot ask for it. But, it is beneficial to have this output should other show commands or troubleshooting steps fail to resolve the problem.
-
Should all the previous troubleshooting steps fail to diagnose the problem, capture these hidden commands, as well as the other commands in the previous steps, and open a case with the Cisco Technical Support:
-
ps-c (capture multiple times)
-
show mbuf all (hidden)
-
show nvramenv 1 (hidden)
-
show interposition 1 (hidden)
These are hidden commands, which means they cannot be parsed with a "?" and you cannot Tab to complete. Type the command out in its entirety. This output can or cannot be useful in the resolution of your case. These commands are undocumented, and therefore the TAC engineer is not required to explain the output to the customer.
-
Misleading Problems
There are many misleading problems that are thought to be caused by faulty hardware. This section lists a few issues that are often confused as a hardware failure.
-
One common customer issue is for the system LED to show faulty when additional power supplies are added, but not plugged in. When this happens, both the ps#-status and sys-status shows faulty. This is because the switch senses an additional power supply is installed but is not active. Since this can also mean that the additional power supply has actually failed, an onsite inspection is required.
-
A common misconception when you view output of the show proc cpu command is that the Kernel and Idle percentage is interpreted to be the CPU utilization for that time period. The Kernel and Idle is the percentage of CPU that was idle for that time frame.
show Command Descriptions
These table breaks down what show commands are used to help troubleshoot the different symptom types.
| Connectivity Problems | System/Supervisor/Module Problems | Supervisor Resets/Crashes |
|---|---|---|
| show version show config show module show system show port capabilities show port <mod/port> show Mac<mod/port> show counters <mod/port> clear counters show cdp neighbors detail show spantree summary | show version show module show flash show config show test 0 show system show time show logging buffer 1023 show proc CPU or Ps -c show mbuf all (hidden) show nvramenv 1 (hidden) show interposition 1 (hidden) | show log show logging buffer 1023 show version show test 0 show system show tech support ps-c (multiple times) (hidden) show mbuf all (hidden) show nvramenv 1 (hidden) show interposition 1 (hidden) |
Capture these show commands that depends on your symptom(s).
Notice that many of the commands in each previous symptom category overlap. This is because the same symptom can occur in different levels of severity; one can cause a performance issue and the other can cause a crash.
Notice also that some of the commands seem meant more for software troubleshooting or configuration problems. For instance, the show spantree summary command shows which VLANs run STP, how many ports are in each VLAN, if any ports on the switch are blocking, and for which VLANs that they are blocking. Since STP loops can actually bring down a switch or network that gives the appearance of a hardware failure, this is vital information to capture, whether troubleshooting hardware or software.
show version
This command verifies the version of software you are running. This command also has information about the size of Flash and DRAM. This is useful information should you need to upgrade. If an upgrade is required, always check the release notes first. Refer to the to Catalyst 4500 Family Release Notes and choose the version you need to upgrade. For example, choose the Release Notes for Catalyst 4000 Family Software Release 5.x and search on any information in regards to your hardware.
Refer to Table 2-82: show version Command Output Fields for more information.
Cat4000-c> (enable) show version
WS-C4006 Software, Version NmpSW: 6.3(1)
Copyright (c) 1995-2001 by Cisco Systems, Inc.
NMP S/W compiled on Jul 24 2001, 12:55:29
GSP S/W compiled on Jul 24 2001, 10:36:29
System Bootstrap Version: 5.4(1)
Hardware Version: 2.0 Model: WS-C4006 Serial #: JAB04380209
Mod Port Model Serial # Versions
--- ---- ---------- -------------------- ---------------------------------
1 2 WS-X4013 JAB04380209 Hw : 2.0
Gsp: 6.3(1.0)
Nmp: 6.3(1)
2 34 WS-X4232-L3 JAB045004AA Hw : 1.5
3 24 WS-X4424-GB-RJ45 JAB0514071N Hw : 0.7
5 6 WS-X4306 JAB02400048 Hw : 0.2
DRAM FLASH NVRAM
Module Total Used Free Total Used Free Total Used Free
------ ------- ------- ------- ------- ------- ------- ----- ----- -----
1 65536K 33235K 32301K 16384K 16173K 211K 480K 180K 300K
Uptime is 28 days, 11 hours, 42 minutes
show module
This command displays information about the modules installed in the switch. In particular, note the status of the module. If the status is faulty, this can be a hardware failure.
Cat4000-c> (enable) show module
Mod Slot Ports Module-Type Model Sub Status
--- ---- ----- ------------------------- ------------------- --- --------
1 1 2 1000BaseX Supervisor WS-X4013 no OK
2 2 34 Router Switch Card WS-X4232-L3 no OK
3 3 24 10/100/1000 Ethernet WS-X4424-GB-RJ45 no disable
5 5 6 1000BaseX Ethernet WS-X4306 no OK
Mod Module-Name Serial-Num
--- -------------------- --------------------
1 JAB04380209
2 JAB045004AA
3 JAB0514071N
5 JAB02400048
Mod MAC-Address(es) Hw Fw SW
--- -------------------------------------- ------ ---------- -----------------
1 00-02-b9-83-ac-00 to 00-02-b9-83-af-ff 2.0 5.4(1) 6.3(1)
2 00-02-16-f6-64-5c to 00-02-16-f6-64-7d 1.5 12.0(7)W5( 12.0(14)W5(20)
3 00-30-85-0e-2c-18 to 00-30-85-0e-2c-2f 0.7
5 00-10-7b-f6-9c-e4 to 00-10-7b-f6-9c-e9 0.2
Cat4000-c> (enable)
Refer to Table 2-35: show module Command Output Fields for more information.
show flash
This command displays the the contents of the Flash file system. Flash file systems differ between Catalyst supervisors. Some supervisors use the show flash command to display the contents, while others use the dir bootflash: command. When you copy an image to the SupIIIG, for example, you use the download command and the Flash is completely erased in the process of installing the image. With other sups, you can use the copy tftp flash command in order to add one or more images.
Many problems, both hardware and software related, can be avoided if you understand the Flash system for your supervisor.
Refer to the show flash or dir bootflash: command for more information.
Cat4000-c> sh flash -#- ED --type-- --crc--- -seek-- nlen -length- -----date/time------ name 1 .. ffffffff 4e88958b 42a97c 17 4106492 Aug 17 2001 16:22:52 cat4000.6-3n 2 .. ffffffff b965ace8 78e71c 18 3554592 Nov 28 2001 10:38:33 cat4000.5-5n 3 .. ffffffff 70a608c8 b8fa9c 20 4199168 DEC 07 2001 10:30:01 cat4000-k9.n 4 .. ffffffff e873ea40 f0b224 17 3651336 DEC 11 2001 12:26:20 cat4000.5-5n 216540 bytes available (15512100 bytes used) Cat4000-c>
show config
This command displays the non-default system configuration. This is useful to capture every time you make a configuration change as a way to possibly associate changes to hardware or software problems. Notice there is is a timestamp for each output. Compare the output to the show config all command output, which shows the entire system configuration and can be quite lengthy. Refer to the show config command for more information.
Cat4000-c> (enable) show config This command shows non-default configurations only. Use 'show config all' to show both default and non-default configurations. ............. .................. .................... .. begin ! # ***** NON-DEFAULT CONFIGURATION ***** ! ! #time: Tue Jan 22 2002, 11:20:05 ! #version 6.3(1) ! ! #system web interface version(s) ! #test ! #system set system name Cat4000-c ! #frame distribution method set port channel all distribution Mac both ! #vtp set vtp domain blah ! #ip set interface sc0 1 172.16.84.200/255.255.255.0 172.16.84.255 set interface sl0 down set interface me1 1.1.1.1 255.255.255.0 1.1.1.255 set ip route 0.0.0.0/0.0.0.0 172.16.84.1 ! #syslog set logging level cops 2 default ! #set boot command set boot config-register 0x2102 clear boot system all ! #mls set mls nde disable ! #port channel set port channel 1/1-2 100 ! #module 1 : 2-port 1000BaseX Supervisor set udld enable 1/1 set port channel 1/1-2 mode desirable silent ! #module 2 : 34-port Router Switch Card ! #module 3 : 24-port 10/100/1000 Ethernet set vlan 150 3/9 ! #module 4 empty ! #module 5 : 6-port 1000BaseX Ethernet ! #module 6 empty ! #cam set cam permanent 01-00-5e-01-01-01 1/1 1 end Cat4000-c> (enable)
show test 0
This command displays the results of diagnostic tests for the supervisor and all modules. It is very important to understand that the show test command only displays the results of diagnostics on the last bootup of the switch or a reset of the supervisor or modules. If the diagnostics for one module are required, issue the show test <mod #> command for this information.
If you are running 5.4.1 or later, check the status of the diaglevel by issuing the show test diaglevel command. A complete status test of the Encoded Address Recognition Logic (EARL), port loopback/bundle/inline rewrite, and DRAM/NVRAM/External cache is recommended. This test takes about one minute versus 30 seconds for a test level of minimal. But, it is more thorough. Results are output with a . for pass or F for fail, which indicates a hardware failure.
Display and/or change the diaglevel as follows:
Cat4000-c> (enable) show test diaglevel Diagnostic mode at next reset : minimal Cat4000-c> (enable) set test diaglevel ? complete Complete diagnostics minimal Minimal diagnostics bypass Bypass diagnostics Diagnostic level set to complete. Cat4000-c> (enable) show test diaglevel Diagnostic mode at next reset : complete
Refer to the show test command for more information.
Cat4000-c> (enable) show test 0
Diagnostic mode at next reset: complete
System Diagnostic Status : (. = Pass, F = Fail, N = N/A)
Module 1 : 2-port 1000BaseX Supervisor
Status: (. = Pass, F = Fail, U = Unknown)
Module 2 : 34-port Router Switch Card
Status: (. = Pass, F = Fail, U = Unknown)
Eeprom: .
CX1000 Regs:
Ports 3-11 : . Ports 12-19 : . Ports 20-27 : .
Ports 28-34 : .
CX1000 Sram:
Ports 3-11 : . Ports 12-19 : . Ports 20-27 : .
Ports 28-34 : .
10/100Base-TX Loopback Status:
Ports 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
-----------------------------------------------------------------------
. . . . . . . . . . . . . . . . . . . . . . . .
27 28 29 30 31 32 33 34
-----------------------
. . . . . . . .
1000Base-X Loopback Status:
Ports 1 2
-----
. .
Router CPU board Status:
Module 3 : 24-port 10/100/1000 Ethernet
Status: (. = Pass, F = Fail, U = Unknown)
Eeprom: .
Lemans Regs:
Ports 1-4 : . Ports 5-8 : . Ports 9-12 : .
Ports 13-16 : . Ports 17-20 : . Ports 21-24 : .
Lemans SRAM:
Ports 1-4 : . Ports 5-8 : . Ports 9-12 : .
Ports 13-16 : . Ports 17-20 : . Ports 21-24 : .
10/100/1000Base-TX Loopback Status:
Ports 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
-----------------------------------------------------------------------
. . . . . . . . . . . . . . . . . . . . . . . .
Module 5 : 6-port 1000BaseX Ethernet
Status: (. = Pass, F = Fail, U = Unknown)
Eeprom: .
Alpheratz: .
1000BaseX Loopback Status:
Ports 1 2 3 4 5 6
-----------------------
. . . . . .
Cat4000-c> (enable)
show system
This command displays system information. The status fields relate to the various LEDs on the system components. Take note of the uptime or how long the switch has been up and running. This would be useful information to know in the event of a switch crash. Refer to the show system command for more information.
Cat4000-c> (enable) show system PS1-Status PS2-Status PS3-Status PEM Installed PEM Powered ---------- ---------- ---------- ------------- ----------- OK OK none no no Fan-Status Temp-Alarm sys-status Uptime d,h:m:s Logout ---------- ---------- ---------- -------------- --------- OK off OK 28,15:10:39 20 min PS1-Type PS2-Type PS3-Type ------------ ------------ ------------ WS-C4008 WS-C4008 none Modem Baud Traffic Peak Peak-Time ------- ----- ------- ---- ------------------------- disable 9600 0% 0% Fri Jan 11 2002, 13:37:07 Power Capacity of the Chassis: 2 supplies System Name System Location System Contact CC ------------------------ ------------------------ ------------------------ --- Cat4000-c
show time
This command displays the day of the week/month/year and the time in a 24-hour format. This verifies the operation of the system clock, but also is a reminder that system log messages carry a timestamp. Make sure to set the time accurately or sync the switch to Network Time Protocol (NTP).
Cat4000-c> (enable) show time Wed Jan 23 2002, 10:41:22 Cat4000-c> (enable)
Refer to the show time command for more information.
show logging buffer 1023
This command displays system messages from the internal buffer. The show logging buffer command only gives you the last 20 system messages, while if you add the 1023 keyword, this gives you the last 1023 messages. Many of these messages are strictly informational. Others can contain clues as to the nature of the problem, whether it is a hardware problem, switch crash, or software problem. When you compare the logs on several pieces of equipment, verify that the time stamps are correct and issue the show time command.
For example, these types of messages are informational:
2002 Jan 06 16:07:04 %DTP-5-TRUNKPORTON:Port 2/23 has become dot1q trunk 2002 Jan 06 16:07:08 %PAGP-5-PORTTOSTP:Port 2/21 joined bridge port 2/21-24
A message like this one indicates a hw/sw incompatibility:
Module 6 is not supported (46)
A message like this one can indicate a hardware failure:
EARL-3-LTL: Failure to set LTL for module [DEC]
Refer to Messages and Recovery Procedures for a listing of system messages. Use the Bug Toolkit and other resources described under the Prerequisites section in this document. Also, refer to Common CatOS Error Messages on Catalyst 4000 Series Switches for more information.
Refer to the show logging buffer 1023 command for more information:
Cat4000-c> sh logging buffer 1023 2002 Jan 23 11:14:23 %SYS-5-MOD_OK:Module 1 is online 2002 Jan 23 11:14:32 %SYS-5-MOD_OK:Module 5 is online 2002 Jan 23 11:14:35 %SYS-5-MOD_OK:Module 3 is online 2002 Jan 23 11:14:54 %PAGP-5-PORTTOSTP:Port 3/9 joined bridge port 3/9 2002 Jan 23 11:15:14 %SYS-5-MOD_OK:Module 2 is online 2002 Jan 23 11:15:23 %PAGP-5-PORTFROMSTP:Port 3/9 left bridge port 3/9 2002 Jan 23 11:15:30 %PAGP-5-PORTTOSTP:Port 2/1 joined bridge port 2/1 2002 Jan 23 11:15:30 %PAGP-5-PORTTOSTP:Port 2/2 joined bridge port 2/2 2002 Jan 23 11:15:41 %PAGP-5-PORTTOSTP:Port 3/9 joined bridge port 3/9 2002 Jan 23 11:17:19 %PAGP-5-PORTFROMSTP:Port 3/9 left bridge port 3/9 2002 Jan 23 11:17:37 %PAGP-5-PORTTOSTP:Port 3/9 joined bridge port 3/9 Cat4000-c>
show proc cpu
This command displays information about CPU usage. Issue the ps-c command in order to format this information differently.
Refer to these documents for more information on how the CPU is utilized on Catalyst 4000, 2948G, 2980G, and 4912G switches
Cat4000-c> (enable) show proc cpu
CPU utilization for five seconds: 11.62%
one minute: 12.00%
five minutes: 12.00%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
--- ----------- ---------- -------- ------- ------- ------- --- ---------------
1 20176816 0 0 88.38% 88.00% 88.00% -2 Kernel and Idle
2 8 131 1000 0.00% 0.00% 0.00% -2 Flash MIB Updat
3 97245 176675 40000 0.25% 0.00% 0.00% -2 SynConfig
4 33358 34879 2000 0.96% 0.00% 0.00% -2 Statuspoll
5 6254 87069 1000 0.00% 0.00% 0.00% -2 PwrDevMsgUpd
6 376 5258 1000 0.00% 0.00% 0.00% -2 StatusPoll 5s
8 5 2 5000 0.00% 0.00% 0.00% -2 SecurityRx
9 106 1092 1000 0.00% 0.00% 0.00% -2 SWPoll64bCnt
10 1713 26229 1000 0.00% 0.00% 0.00% -2 Earl
11 172 2613 1000 0.00% 0.00% 0.00% -2 ProtocolFilter
12 0 1 0 0.00% 0.00% 0.00% -2 telnetd
13 0 1 0 0.00% 0.00% 0.00% -2 llcSSTPFlood
14 441829 9511273 1000 1.47% 1.00% 1.00% -2 gsgScpAggregati
15 347 444 1000 0.00% 0.00% 0.00% -2 cdpd
16 58134 26267 5000 0.57% 0.00% 0.00% -2 cdpdtimer
17 29751 26913 9000 0.96% 0.00% 0.00% -2 SptTimer
18 1 1 1000 0.00% 0.00% 0.00% -2 SptBpduRx
19 40610 26227 3000 0.28% 0.00% 0.00% -2 SptBpduTx
20 2230 26227 1000 0.16% 0.00% 0.00% -2 VtpTimer
21 0 1 0 0.00% 0.00% 0.00% -2 RMON AlarmTimer
22 22352 257353 9000 0.28% 0.00% 0.00% -2 ProtocolTimer
23 2024 2305 2000 0.00% 0.00% 0.00% -2 DTP_Rx
24 649 1200 16000 0.00% 0.00% 0.00% -2 EthChnlRx
25 901 1745 2000 0.00% 0.00% 0.00% -2 EthChnlConfig
26 15943 260008 1000 0.28% 0.00% 0.00% -2 sptHelper
27 0 1 0 0.00% 0.00% 0.00% -2 sptTraps
28 154 2629 1000 0.00% 0.00% 0.00% -2 ciscoRmonTimer
29 167 2629 1000 0.00% 0.00% 0.00% -2 ciscoUsrHistory
30 1 1 1000 0.00% 0.00% 0.00% -2 rmonMediaIndep
31 0 1 0 0.00% 0.00% 0.00% -2 SnmpTraps
32 0 1 0 0.00% 0.00% 0.00% -2 Acct Send Bkg
34 0 1 0 0.00% 0.00% 0.00% -2 l2t_server
36 164 504 1000 0.00% 0.00% 0.00% -2 SysLogTask
37 8188 26039 1000 0.80% 0.00% 0.00% -2 pinggateA
38 43007 876770 1000 0.44% 0.00% 0.00% -2 Authenticator_S
39 0 1 0 0.00% 0.00% 0.00% -2 dot1x_rx
40 3423 57501 1000 0.32% 0.00% 0.00% -2 Backend_Rx
41 39173 577158 1000 0.09% 0.00% 0.00% -2 Backend_SM
143 642792 9511281 34000 2.28% 2.00% 2.00% 0 Console
144 199 1 199000 0.00% 0.00% 0.00% -2 snmpdm
145 1 2 1000 0.00% 0.00% 0.00% -2 VtpRx
193 591423 783586 10730 2.26% 2.27% 2.22% 0 Packet forwardi
194 353123 359502 6164 1.33% 1.35% 1.36% 0 Switching overh
195 727712 633244 57354 2.83% 2.85% 2.77% 0 Admin overhead
Cat4000-c> (enable)
show port capabilities
This command displays the capabilities of the modules and ports in a switch. Think of this command as a quick way to display hardware/software features without the need to search the release notes. This command can answer question, such as what trunk encapsulation types are supported and can the ports etherchannel. Refer to Table 2-49: show port capabilities Command Output Fields for more information.
Cat4000-c> (enable) show port capabilities 2/1 Model WS-X4232-L3 Port 2/1 Type No Connector Speed 1000 Duplex full Trunk encap type 802.1Q Trunk mode on,off Channel 2/1-2 Flow control no Security yes Dot1x yes Membership static,dynamic Fast start yes QOS scheduling rx-(none),tx-(2q1t) CoS rewrite no ToS rewrite no Rewrite no UDLD yes Inline power no AuxiliaryVlan no SPAN source Link debounce timer yes Cat4000-c> (enable)
show port <mod/port>
This command displays port status and counters. If the status is anything other than connected, see the troubleshooting steps in the Port status shows not connected, faulty, disabled, inactive or errdisable section of this document . If the port counters show incrementing errors, see the troubleshooting steps in the Seeing errors on ports section.
Refer to the show port command for more information.
Cat4000-c> (enable) show port 3/9
Port Name Status Vlan Level Duplex Speed Type
----- ------------------ ---------- ---------- ------ ------ ----- ------------
3/9 connected 1 normal a-full a-100 10/100/1000
Port AuxiliaryVlan AuxVlan-Status InlinePowered PowerAllocated
Admin Oper Detected mWatt mA @51V
----- ------------- -------------- ----- ------ -------- ----- --------
3/9 none none - - - - -
Port Security Violation Shutdown-Time Age-Time Max-Addr Trap IfIndex
----- -------- --------- ------------- -------- -------- -------- -------
3/9 disabled shutdown 0 0 1 disabled 64
Port Num-Addr Secure-Src-Addr Age-Left Last-Src-Addr Shutdown/Time-Left
----- -------- ----------------- -------- ----------------- ------------------
3/9 0 - - - - -
Port Send FlowControl Receive FlowControl RxPause TxPause Unsupported
admin oper admin oper opcodes
----- -------- -------- -------- -------- ------- ------- -----------
3/9 on disagree desired off 0 0 0
Port Status Channel Admin Ch
Mode Group Id
----- ---------- -------------------- ----- -----
3/9 connected auto silent 40 0
Port Align-Err FCS-Err Xmit-Err Rcv-Err UnderSize
----- ---------- ---------- ---------- ---------- ---------
3/9 - 0 0 0 0
Port Single-Col Multi-Coll Late-Coll Excess-Col Carri-Sen Runts Giants
----- ---------- ---------- ---------- ---------- --------- --------- ---------
3/9 0 0 0 0 0 0 0
Last-Time-Cleared
--------------------------
Tue Jan 22 2002, 14:57:21
show mac <mod/port>
This command displays the MAC counters, and is useful in the determination of whether counters are incrementing as expected. This command shows the total unicast, multicast, and broadcast frames received on a port. The In-lost counter on the Catalyst 4000 reflects the sum of all error packets received on the port. This is different then the behavior of the In-Lost counter on the Catalyst 5000 switches; which reflects the sum of all receive buffer failures. The out-Lost counter on both the Catalyst 4000 and 5000, reflect outgoing frames that were lost before forwarded due to insufficient buffer space. This is commonly caused if you oversubscribe the interface.
See the Seeing errors on the ports section of this document for troubleshooting steps. Refer to the show mac command for more information.
Cat4000-c> (enable) show mac 2/1 Port Rcv-Unicast Rcv-Multicast Rcv-Broadcast -------- -------------------- -------------------- -------------------- 2/1 6 446 0 Port Xmit-Unicast Xmit-Multicast Xmit-Broadcast -------- -------------------- -------------------- -------------------- 2/1 6 16041 26236 Port Rcv-Octet Xmit-Octet -------- -------------------- -------------------- 2/1 149408 2901773 MAC Dely-Exced MTU-Exced In-Discard Lrn-Discrd In-Lost Out-Lost -------- ---------- ---------- ---------- ---------- ---------- ---------- 2/1 0 0 0 0 0 0 Last-Time-Cleared -------------------------- Tue Jan 22 2002, 14:57:21
show counters <mod/port>
This command displays hardware counters for the port and will vary depending on the type of port. See the Seeing errors on the ports section of this document for troubleshooting steps. Refer to the show counters command for more information.
Cat4000-c> (enable) show counters 2/1 2 rxUnicastPacketCount = 6 3 txUnicastPacketCount = 6 4 rxMulticastPacketCount = 447 5 txMulticastPacketCount = 16078 6 rxBroadcastPacketCount = 0 7 txBroadcastPacketCount = 26296 8 rxByteCount = 149742 9 txByteCount = 2908424 10 pkts64 = 40611 11 pkts65to127 = 890 12 pkts128to255 = 441 13 pkts256to511 = 891 14 pkts512to1023 = 0 15 pkts1024to1522 = 0 16 rxNoPacketBufferCount = 0 17 rxCRCAlignErrorPacketCount = 0 18 rxUndersizedPacketCount = 0 19 rxOversizedPacketCount = 0 20 rxFragmentPacketCount = 0 21 rxJabberPacketCount = 0 22 pauseControlFramesRx = 0 23 pauseControlFramesTx = 0 24 unsupportedOpcodesRx = 0 25 txQueueNotAvailable = 0 26 totalCollisionCount = 0 27 lateCollisionCount = 0 28 singleCollisionFrames = 0 29 multipleCollisionFrames = 0 30 excessiveCollisionFrames = 0 31 deferredTransmissions = 0 32 carrierSenseErrors = 0 33 falseCarrierDuringIdle = 0 34 symbolErrorDuringCarrier = 0 35 sequenceErrorDuringCarrier = 0
clear counters
This command is used to reset the show port, show mac, and show counter statistics. It is useful for the determination of errors that continue to increment or have been resolved.
Refer to the clear counters command for more information.
show cdp neighbors detail
This command shows details about remote Cisco devices using CDP. This is one quick way to get the IP address and interface of a Cisco device on any given switchport. Refer to the show cdp neighbors detail commands for more information.
Cat4000-c> (enable) show cdp neighbors detail Port (Our Port): 2/1 Device-ID: 8-4006-L3 Device Addresses: IP Address: 127.0.0.3 Holdtime: 170 sec Capabilities: ROUTER Version: Cisco Internetwork Operating System Software IOS (tm) L3 Switch/Router Software (CAT4232-IN-M), Version 12.0(14)W5(20) RE Copyright (c) 1986-2001 by cisco Systems, Inc. Compiled Thu 01-Mar-01 18:18 by integ Platform: cisco Cat4232L3 Port-ID (Port on Neighbors's Device): GigabitEthernet3 VTP Management Domain: unknown Native VLAN: unknown Duplex: unknown System Name: unknown System Object ID: unknown Management Addresses: unknown Physical Location: unknown ___________________________________________________________________________ Port (Our Port): 2/2 Device-ID: 8-4006-L3 Device Addresses: IP Address: 127.0.0.3 Holdtime: 170 sec Capabilities: ROUTER Version: Cisco Internetwork Operating System Software IOS (TM) L3 Switch/Router Software (CAT4232-IN-M), Version 12.0(14)W5(20) RE Copyright (c) 1986-2001 by cisco Systems, Inc. Compiled Thu 01-Mar-01 18:18 by integ Platform: cisco Cat4232L3 Port-ID (Port on Neighbors's Device): GigabitEthernet4 VTP Management Domain: unknown Native VLAN: unknown Duplex: unknown System Name: unknown System Object ID: unknown Management Addresses: unknown Physical Location: unknown Cat4000-c> (enable)
show spantree summary
This command provides a summary of STP information useful in troubleshooting link flaps and other network issues masquerading as hardware issues. Refer to the show spantree summary and the show spantree commands for more information.
Cat4000-c> (enable) show spantree summary
MAC address reduction: disabled
Root switch for vlans: 1.
BPDU skewing detection disabled for the bridge
BPDU skewed for vlans: none.
Portfast bpdu-guard disabled for bridge.
Portfast bpdu-filter disabled for bridge.
Uplinkfast disabled for bridge.
Backbonefast disabled for bridge.
Summary of connected spanning tree ports by vlan
VLAN Blocking Listening Learning Forwarding STP Active
----- -------- --------- -------- ---------- ----------
1 0 0 0 3 3
Blocking Listening Learning Forwarding STP Active
----- -------- --------- -------- ---------- ----------
Total 0 0 0 3 3
Cat4000-c> (enable)
show log
This command displays the error log for the system or a specific module. If there has been a switch reset or crash, the stack information necessary to determine the cause of the switch crash is displayed here. Refer to the show log command for more information.
Cat4000-c> show log
Network Management Processor (ACTIVE NMP) Log:
Reset count: 15
Reboot History: Jan 23 2002 11:14:16 0, Jan 22 2002 14:57:21 0
DEC 24 2001 13:56:38 0, DEC 24 2001 13:52:30 0
DEC 11 2001 12:31:59 0, DEC 07 2001 13:26:48 0
DEC 07 2001 10:42:19 0, DEC 07 2001 10:36:16 0
Nov 28 2001 11:03:10 0, Oct 26 2001 16:04:26 0
Bootrom Checksum Failures: 0 UART Failures: 0
Flash Checksum Failures: 0 Flash Program Failures: 0
Power Supply 1 Failures: 0 Power Supply 2 Failures: 0
DRAM Failures: 0
Exceptions: 0
Loaded NMP version: 6.3(1)
Reload same NMP version count: 2
Last software reset by user: 1/23/2002,11:13:13
MCP Exceptions/Hang: 0
Heap Memory Log:
Corrupted Block = none
NVRAM log:
01. 11/28/2001,11:03:11: convert_post_SAC_CiscoMIB:Nvram block 0 unconvertible:)
02. 11/28/2001,11:03:11: convert_post_SAC_CiscoMIB:Nvram block 1 unconvertible:)
03. 11/28/2001,11:03:11: convert_post_SAC_CiscoMIB:Nvram block 2 unconvertible:)
04. 11/28/2001,11:03:11: check_block_and_log:Block 3 has been deallocated: (0x1)
05. 11/28/2001,11:03:11: convert_post_SAC_CiscoMIB:Nvram block 5 unconvertible:)
06. 11/28/2001,11:03:11: check_block_and_log:Block 35 has been deallocated: (0x)
07. 11/28/2001,11:03:11: convert_post_SAC_CiscoMIB:Nvram block 44 unconvertible)
08. 11/28/2001,11:03:11: convert_post_SAC_CiscoMIB:Nvram block 62 unconvertible)
09. 11/28/2001,11:03:14: supVersion:Nmp version 5.5(11)
10. 12/7/2001,10:36:16: convert_post_SAC_CiscoMIB:Block 0 converted from versio5
11. 12/7/2001,10:36:20: supVersion:Nmp version 6.3(3)
12. 12/11/2001,12:32:00: convert_post_SAC_CiscoMIB:Nvram block 0 unconvertible:)
13. 12/11/2001,12:32:00: convert_post_SAC_CiscoMIB:Nvram block 1 unconvertible:)
14. 12/11/2001,12:32:00: convert_post_SAC_CiscoMIB:Nvram block 2 unconvertible:)
15. 12/11/2001,12:32:00: convert_post_SAC_CiscoMIB:Nvram block 5 unconvertible:)
16. 12/11/2001,12:32:00: check_block_and_log:Block 35 has been deallocated: (0x)
17. 12/11/2001,12:32:00: convert_post_SAC_CiscoMIB:Nvram block 44 unconvertible)
18. 12/11/2001,12:32:00: convert_post_SAC_CiscoMIB:Nvram block 62 unconvertible)
19. 12/11/2001,12:32:04: supVersion:Nmp version 5.5(8)
20. 12/24/2001,13:56:38: convert_post_SAC_CiscoMIB:Block 0 converted from versi5
21. 12/24/2001,13:56:42: supVersion:Nmp version 6.3(1)
Module 2 Log:
Reset Count: 16
Reset History: Wed Jan 23 2002, 11:15:13
Tue Jan 22 2002, 14:58:18
Tue Jan 15 2002, 17:03:35
Tue DEC 11 2001, 12:32:58
Module 3 Log:
Reset Count: 12
Reset History: Wed Jan 23 2002, 11:14:34
Tue Jan 22 2002, 14:57:39
Mon DEC 24 2001, 13:56:53
Fri DEC 7 2001, 13:27:07
Module 5 Log:
Reset Count: 15
Reset History: Wed Jan 23 2002, 11:14:31
Tue Jan 22 2002, 14:57:36
Mon DEC 24 2001, 13:56:51
Mon DEC 24 2001, 13:52:43
show tech-support
This command displays this as continuous output:
show version, sh flash, sh microcode, sh system, sh module, sh port, sh mac, sh trunk, sh vlan, sh vtp domain, sh spantree active, sh spantree summary, sh test, sh arp, sh ip route, sh cdp neighbor detail, sh netstst ststs, show memory buffers, show out-of-band stats, sh inband stats, show cam static, sh cam count dynamic, sh cam system, sh config, sh log, sh proc, sh proc mem, sh proc cpu, ps, ps -c
Refer to the show tech-support command for more information.
Related Information
Revision History
| Revision | Publish Date | Comments |
|---|---|---|
1.0 |
02-Dec-2013 |
Initial Release |
FeedbackContact Cisco
- Open a Support Case
- (Requires a Cisco Service Contract)