- System Boot Verification
- Problem Solving to the System Component Level
- Using LEDs to Identify Startup Problems
- Troubleshooting the Power Supply
- Troubleshooting the Fan Assembly
- Troubleshooting Backplane Modules
- Troubleshooting Switching Modules
- Troubleshooting Supervisor Engines
- Contacting the Cisco Technical Assistance Center
Troubleshooting
If your system has problems during start up or operation, use the information in this chapter to help isolate the cause. Problems with the initial startup are often caused by a switching module that has become dislodged from the backplane or a power cord that is disconnected from the power supply.
Although temperature conditions above the maximum acceptable level rarely occur at initial startup, some environmental monitoring functions are included in this chapter because they also monitor power supply output voltages.
Information about troubleshooting software features and configuration problems is not discussed in this chapter.
The information is presented in these sections:
•
Problem Solving to the System Component Level
•Using LEDs to Identify Startup Problems
•Troubleshooting the Power Supply
•Troubleshooting the Fan Assembly
•Troubleshooting Backplane Modules
•Troubleshooting Switching Modules
•Troubleshooting Supervisor Engines
•Contacting the Cisco Technical Assistance Center
Note This chapter covers only the chassis component hardware aspects of troubleshooting. For configuration issues, refer to the software configuration guide or command reference for your switch.
System Boot Verification
When the initial system boot is complete, verify the following:
•That the power supplies are supplying power to the system
The LEDs should be green. Use the show environment Cisco IOS command to view power supply activity.
•That the system fan assembly is operating
Listen for fan activity. The Fan tray LED should be green during operation.
•That the system software boots successfully
Hook up a terminal and view the startup banner. Use an RJ-45-to-RJ-45 rollover cable to connect the console port to a PC with terminal emulation software set for 9600 baud, 8 data bits, no parity, and 1 stop bit. Watch for any messages after startup.
•That the supervisor engine and all switching modules are installed properly in their slots, and that each initialized without problems
If all of these conditions are met and the hardware installation is complete, refer to the software configuration guide and command reference publications for your switch so that you can troubleshoot the software.
If any of these conditions is not met, use the procedures in this chapter to isolate and, if possible, resolve the problem.
Problem Solving to the System Component Level
The key to success when troubleshooting the system is to isolate the problem to a specific system component. Your first step is to compare what the system is doing to what it should be doing. Because a startup problem can usually be caused by a single component, it is more efficient to isolate the problem to a subsystem rather than troubleshoot each separate component in the system.
The switch consists of the following subsystems:
•Power supply—Includes the power supplies and power supply fans. If you suspect problems with the power supplies, see the "Troubleshooting the Power Supply" section.
•Fan assembly—The system fan assembly should be operating whenever system power is on. You should be able to hear the fan assembly when the system is operating. If you determine that the fan assembly is not operating, see the "Troubleshooting the Fan Assembly" section.
•Supervisor engine—The supervisor engine contains the operating system software, so check the supervisor engine if you have trouble with the system software. Status LEDs on the supervisor engine indicate whether the supervisor engine has initialized correctly. If there is a problem with the supervisor engine, try reseating the supervisor engine in the chassis and restarting the switch. For more information about LED meanings, refer to the "LEDs" section. If the supervisor engine does not boot correctly after reseating and restarting, contact Cisco TAC. See the "Contacting the Cisco Technical Assistance Center" section.
•Switching modules—The LEDs labeled STATUS on each switching module indicate whether the switching module has been initialized correctly. The supervisor engine must be operating properly before the switching module will initialize. If a switching module is improperly installed in the switch, it will not function, so if the supervisor engine is working but a switching module is not, try reseating the module. For more informations, see the "Troubleshooting Switching Modules" section. If you determine that the switching module is not operating, contact Cisco TAC as described in the "Contacting the Cisco Technical Assistance Center" section.
Using LEDs to Identify Startup Problems
All system states in the startup sequence are indicated using LEDs. By checking the LEDs, you can determine when and where the system failed in the startup sequence. If you have problems after the switch is on, refer to the configuration procedures in the software configuration guide for your switch.
After you connect the power cords to your Catalyst 4500 series switch, follow these steps to determine whether your system is operating properly:
Step 1 Check the power supply LEDs:
•The LED labeled GOOD should turn green when power is applied to the supply. The LED should remain on during normal system operation.
•If the LED labeled GOOD does not light, or if the LED labeled FAIL lights, see the "Troubleshooting the Power Supply" section.
Note If a power supply is installed and not connected to a power source, power supply LEDs will indicate a failure.
Step 2 Listen for the system fan assembly. If you do not immediately hear it begin to operate, see the "Troubleshooting the Fan Assembly" section.
Step 3 Check that the LEDs on the supervisor engine light as follows:
•The LED labeled STATUS flashes orange once and stays orange during diagnostic boot tests.
–It turns green when the module is operational (online).
–If the system software is unable to start up, this LED stays orange.
–If the LED labeled STATUS on the supervisor engine front panel is red or orange, connect a console to the management port and use the show environment command to check for possible problems. For more information, see the "Troubleshooting Supervisor Engines" section.
•The Ethernet management port LED turns green when the module is operational (online) and a link is established with another network device. If no signal is detected, the LED labeled LINK turns off.
Step 4 Verify that the LEDs labeled STATUS on each switching module are green when the supervisor engine completes initialization.
This LED indicates that the supervisor engine and switching modules are receiving power, have been recognized by the supervisor engine, and contain a valid Flash code version. However, this LED does not indicate the state of the individual interfaces on the switching modules. If an LED labeled STATUS is red or orange, try reseating the switching module or supervisor engine and restarting the switch. If the problem persists, contact Cisco TAC. See the "Contacting the Cisco Technical Assistance Center" section.
Step 5 If the boot information and system banner are not displayed, verify that the terminal is set for 9600 baud, 8 data bits, no parity, and 1 stop bit and connected properly to the console port.
Troubleshooting the Power Supply
To help isolate a power subsystem problem, follow these steps:
Step 1 Check whether the power supply LED labeled GOOD is on or the LED labeled FAIL is on. (on the DC multi-input power supply, the LEDs are labeled INPUT 1, 2, or 3 or OUTPUT FAIL.)
Step 2 If the LED labeled GOOD is off or if the LED labeled FAIL is on, take the following steps:
•Ensure that the power supply is flush with the back of the chassis.
•Unplug the power cord, loosen and reinstall the power supply, tighten the captive installation screws, and then plug in the power cord.
Step 3 If the LED labeled GOOD remains off, there might be a problem with the AC source or the power cable. Connect the power cord to another power source if one is available.
Step 4 If the LED labeled GOOD fails to light after you connect the power supply to a new power source, replace the power cord.
Note If this unit has more than one power cord, repeat Step 1 through Step 4 for each power input.
Step 5 If the LED labeled GOOD still fails to light when the switch is connected to a different power source with a new power cord, the power supply is probably faulty. Replace the power supply.
Step 6 If the LED labeled FAN OK fails to light when the switch is connected to a power source with a known good power cord, there is a malfunction in the fan that cools the power supply. Replace the power supply.
Step 7 If a second power supply is available, install it in the second power supply bay.
Step 8 Check that the LED labeled GOOD is on for the additional power supply. Check that the LED labeled FAIL is off.
Step 9 If the LEDs are not on, repeat the previous procedure to troubleshoot the second power supply.
If you are unable to resolve the problem, or if you determine that either a power supply or backplane connector is faulty, contact the Cisco Technical Assistance Center (TAC) for instructions.
Troubleshooting Power Problems using Cisco IOS
Check for system messages related to the power supply, and refer to the system message guide for your software release. You may need to add a power supply or upgrade to a larger one for your current configuration or verify that the switches are set properly. Connect a terminal to the console port, and look for any of the following system messages:
C4K_CHASSIS-2-INLINEPOWEROFF
C4K_CHASSIS-2-INSUFFICIENTPOWERDETECTED
C4K_CHASSIS-2-INSUFFICIENTPOWERSHUTDOWN
C4K_CHASSIS-3-INSUFFICIENTPOWER
C4K_CHASSIS-3-INSUFFICIENTPOWERSUPPLIESDETECTED
C4K_CHASSIS-3-MIXINPOWERDETECTED
C4K_IOSMODPORTMAN-3-UNKNOWNPOWERSUPPLY
C4K_IOSMODPORTMAN-4-INLINEPOWEROVERMAX
C4K_IOSMODPORTMAN-4-INLINEPOWERSUPPLYBAD
C4K_IOSMODPORTMAN-4-POWERSUPPLYBAD
C4K_IOSMODPORTMAN-4-POWERSUPPLYFANBAD
C4K_SUPERVISOR-3-POWERSUPPLYSTATUSREADFAILED
C4K_SUPERVISOR-3-POWERSUPPLYSEEPROMREADFAILED
C4K_SUPERVISOR-3-POWERSUPPLYSEEPROMINVALID
C4K_SUPERVISOR-4-INLINEVOLTAGEOUTOFRANGE
You may also use the show environment status powersupply, show module all, and show power commands to monitor PS status, load, and activity.
The 1400 W DC multi-input supply can not be used with other power supply types, but other power supplies in this product line work with the other types during an upgrade. If you mix power supplies in a Catalyst 4500 series chassis, the switch detects the type of power supply in power supply bay 1 (PS1) and ignores the power supply in power supply bay 2 (PS2) while issuing system messages and showing the power supply in bay 2 as in the err-disable state in the output of the show power command. When the power supply in bay 1 is removed, the switch recognizes the power supply in bay 2, and you may then put a new matching power supply in bay 1. Both supplies should now resume normal function.
Troubleshooting the Fan Assembly
Note All fans must be operating or a failure will occur.
Environmental problems may initially appear to be problems with the fan tray. To help isolate a fan assembly problem, follow these steps:
Step 1 Check the status LED on the fan tray.
•If the LED is off and the rest of the system is functioning, the fan tray is not getting power or is not seated correctly on the backplane.
•If the LED is green, the fans are operating normally. There may be conditions impairing fan performance, but they are minimal in impact.
•If the LED is red, one or more fans have failed.
Step 2 Connect a terminal and determine the fan tray status shown by the show environment status CLI command.
Step 3 Look for system messages reporting a temperature problem or problem with the fans. Individual messages may suggest different solutions. Connect a terminal to the console port, and look for any of the following system messages:
C4K_CHASSIS-2-INSUFFICIENTFANSDETECTED
C4K_CHASSIS-2-INSUFFICIENTFANSSHUTDOWN
C4K_IOSMODPORTMAN-4-CRITICALTEMP
C4K_IOSMODPORTMAN-4-FANTRAYBAD
C4K_IOSMODPORTMAN-4-FANTRAYPARTIALFAILURE
C4K_IOSMODPORTMAN-4-FANTRAYREMOVED
C4K_SUPERVISOR-3-FANTRAYREADFAILED
C4K_SUPERVISOR-3-FANTRAYSEEPROMREADFAILED
C4K_SUPERVISOR-3-FANTRAYSEEPROMINVALID
C4K_IOSMODPORTMAN-4-TEMPHIGH
C4K_IOSMODPORTMAN-4-TEMPUNDERCRITICAL
C4K_CHASSIS-2-OVERHEATINGSHUTDOWN
Step 4 Determine whether the airflow is restricted or if the ambient temperature in the room is too warm.
Step 5 Determine whether the power supply is functioning properly. See the "Troubleshooting the Power Supply" section.
Step 6 Verify that the fan assembly is properly seated in the backplane by loosening the captive installation screws, removing the fan assembly, and reinstalling it.
Step 7 Restart the system.
Step 8 Verify that all fans are operating. You should hear the fans at system start.
Step 9 If the system is still detecting a fan assembly failure, contact the Cisco TAC for assistance.
Troubleshooting Backplane Modules
The Cisco Catalyst 4500 Series redundancy scheme uses removable redundancy modules (also called mux-buffers), on the passive backplane to switch traffic to the active supervisor engine. There is one redundancy module per switching module. Redundancy modules and a redundant clock ship standard with every Cisco Catalyst 4507R and 4510R chassis. Spare redundancy modules (C4K-MUX=) and a clock module (C4K-CLOCK=) are available for serviceability.
The following conditions indicate that you may need to replace the redundancy modules and clock module:
•The switch powers down and stays down for a few minutes to a few days for no clear reason.
•The output-fail LED on the power supplies are red and no power is delivered to the chassis. The other LEDs on the power supply are green.
•The Status LEDs on the switching modules and the supervisor engine are flashing green.
•CPU Utilization LEDs are flashing green or off.
If you observe these conditions, contact the Cisco TAC for assistance in ordering replacement redundancy modules and a clock.
Troubleshooting Switching Modules
Each switching module has one LED labeled STATUS that provides information about the module and one numbered LED labeled LINK for each port on the module. Figure 5-1 shows the Gigabit Ethernet port and status LEDs. Figure 5-2 shows the 10/100BASE-T port LEDs. Table 5-1 describes the switching module LEDs and their expected behavior.
Figure 5-1 Gigabit Ethernet Port and Status LEDs
Figure 5-2 10/100BASE-T Port LEDs
|
|
|
|
|---|---|---|
STATUS |
Indicates the results of a series of self-tests and diagnostic tests performed by the switch. |
|
Green |
All the tests pass. |
|
Red |
A test other than an individual port test failed. |
|
Orange |
System boot, self-test diagnostics running, or the module is disabled. |
|
LINK1 |
Indicates the status of the port. |
|
Green |
The port is operational (a signal is detected). |
|
Orange |
The link has been disabled by software. |
|
Flashing orange |
The link has been disabled due to a hardware failure. |
|
Off |
No signal is detected. |
|
Port Status2 |
Indicates individual port status. |
|
Green |
The port is operational (a signal is detected). |
|
Orange |
The link has been disabled by software. |
|
Flashing orange |
The link has been disabled due to a hardware failure. |
|
Off |
No signal is detected. |
1 Used on the WS-X4232-L3 Ethernet routing module. 2 LEDs labeled 1 through the number of ports on the switching module are the individual port link LEDs. |
Troubleshooting Switching Modules Using Cisco IOS
Connect a terminal to the console port, and look for any of the following system messages:
C4K_CHASSIS-3-LINECARDMUXBUFFERTOSUPALIGNMENTWRONG
C4K_CHASSIS-3-LINECARDNOTVALIDINSLOT
C4K_CHASSIS-3-MODULENOTSUPPORTHALF
C4K_IOSINTF-5-STALEPHYPORT
C4K_IOSMODPORTMAN-4-INLINEPOWEROVERMAX
If you observe any of these messages, follow the suggestion for that message.
Some problems can be solved by resetting the switching module. Use the hw-module module <n> reset command to reset a switching module, or remove and re-insert the switching module, which resets, restarts, and power cycles the switching module. The show module and show diagnostics online module commands can also provide information useful in solving problems with ports on individual modules.
Not all software versions support all switching modules. If you are having trouble with a module, refer to the software release notes to be sure that it is supported by your software.
Troubleshooting Supervisor Engines
This section only addresses problems with hardware. Problems with features or configuration are not covered here. Refer for your software configuration guide and release notes for information on configuring features or identifying known problems.
Table 5-2 describes the supervisor engine LEDs. Check the LEDs on your supervisor and compare them to the described LED behaviors.
Troubleshooting Supervisor Engines Using Cisco IOS
Connect a terminal to the console port, and look for any of the following system messages:
C4K_CHASSIS-3-LINECARDMUXBUFFERTOSUPALIGNMENTWRONG
C4K_SUPERVISOR-3-MUXBUFFERREADSUPERVISORSELECTIONFAILED
C4K_CHASSIS-3-TEMPERATURESENSORREADFAILED
C4K_HW-3-X2IDENTIFICATIONFAILURE
C4K_HW-3-X2OUIREGREADFAILURE
C4K_HWACLMAN-4-CLASSIFCAMPARITYERROR
C4K_HWACLMAN-4-CLASSIFCAMREPLYPARITYERROR
C4K_HWACLMAN-4-CLASSIFCAMREQPARITYERROR
C4K_HWNETFLOWMAN-3-NETFLOWSTOPPED
C4K_HWNETFLOWMAN-4-FATALERRORINTERRUPTSEEN
C4K_HWNETFLOWMAN-4-NONFATALPARITYERRORINTERRUPTSEEN
C4K_IOSMODPORTMAN-4-NFLABSENT
C4K_IOSMODPORTMAN-4-NFLIDPROMINVALID
C4K_IOSMODPORTMAN-4-NFLMISMATCH
C4K_REDUNDANCY-2-HANDSHAKE_TIMEOUT
C4K_REDUNDANCY-2-POSTFAIL_RESET
C4K_REDUNDANCY-2-INCOMPATIBLE_SUPERVISORS
C4K_REDUNDANCY-2-IOS_VERSION_CHECK_FAIL
C4K_REDUNDANCY-2-IOS_VERSION_INCOMPATIBLE
C4K_REDUNDANCY-2-NON_SYMMETRICAL_REDUNDANT_SYSTEM
C4K_REDUNDANCY-2-POSTFAIL
C4K_REDUNDANCY-2-POSTFAIL_RESET
C4K_REDUNDANCY-4-CONFIGSYNCFAIL
C4K_SUPERVISOR-2-SUPERVISORSEEPROMINVALID
C4K_SUPERVISOR-3-RETIMERDISABLEFAILED
C4K_SUPERVISOR-3-RETIMERINITFAILED
C4K_SUPERVISOR-3-SEEPROMREADFAILED
C4K_SUPERVISOR-4-INLINEVOLTAGEOUTOFRANGE
C4K_SUPERVISOR-7-SEEPROMWRITEFAILED
C4K_SWITCHMANAGER-3-SSOACTIVEPORTACKTIMEOUT
C4K_SYSMAN-2-POWERONSELFTESTFAIL
These system messages indicate a problem with the supervisor engine. Some problems will prevent a console connection and will not allow you to use messages in diagnosing a problem. If you are unable to establish a terminal connection and the STATUS LED is red, contact Cisco TAC immediately to order a replacement.
Problems with redundant supervisor systems are often due to mismatched active and standby supervisor engines. Redundancy requires that both supervisor engines be the same model and running the same Cisco IOS release. If one supervisor has a NetFlow service card, the other must as well.
Some problems with supervisor engines are due to backplane connections that are not fully seated. If removing and reinserting the supervisor engine and then restarting the switch does not solve the problem, call Cisco TAC and replace the supervisor engine.
Some problems can be solved by resetting the supervisor engine. Use the hw-module module <n> reset power-cycle command to reset a switching module, or remove and re-insert the switching module, which resets, restarts and power cycles the switching module. Pressing the reset button on the supervisor engine causes the software to reload, but does not power cycle the supervisor engine.
Note When you power-cycle or remove a supervisor engine in a redundant system the other supervisor engine becomes the active supervisor and the ports retain connectivity. In a non-redundant system, all of the switching modules lose connectivity until the supervisor engine is reinserted and completely restarted.
The show diagnostics power-on command may provide useful information for some supervisor engine problems.
Not all software versions support all supervisor engines. If you are having trouble with a supervisor engine, refer to the software release notes to be sure that it is supported by your software.
Contacting the Cisco Technical Assistance Center
If you are unable to solve a startup problem after using the troubleshooting suggestions in this chapter, contact a Cisco TAC representative for assistance and further instructions.
Before you call, have the following information ready to help the Cisco TAC assist you as quickly as possible:
•Date you received the switch
•Chassis serial number (refer to Figure 5-3 to Figure 5-6 for serial number locations)
•Type of software and release number
•Maintenance agreement or warranty information
•Brief description of the problem
•Brief explanation of the steps you have already taken to isolate and resolve the problem
See the "Obtaining Documentation and Submitting a Service Request" section for more information about contacting the TAC.
Serial Numbers
When you contact Cisco Technical Assistance, have the serial number of your switch available. Refer to Figure 5-3 to 5-6 to locate the serial number on your switch. You may also get the serial number from the output of a show version command.
Figure 5-3 Catalyst 4503 Serial Number Location
Figure 5-4 Catalyst 4506 Serial Number Location
Figure 5-5 Catalyst 4507R Serial Number Location
Figure 5-6 Catalyst 4510R Serial Number Location
Feedback