Troubleshooting Catalyst 6500/6000 Series Switches Running CatOS on the Supervisor Engine and Cisco IOS on the MSFC

Introduction

This document provides information on how to troubleshoot common issues on the Cisco Catalyst 6500/6000 Series Switches that run Catalyst OS (CatOS) system software (CatOS on the Supervisor Engine and Cisco IOS^® Software on the Multilayer Switch Feature Card [MSFC]). This document does not cover how to troubleshoot for Catalyst 6500/6000 Series Switches that run Cisco IOS system software. The goal of this document is to help you identify and fix some common issues as well as perform more extensive troubleshooting before you contact Cisco Technical Support. If you follow an orderly troubleshooting process and collect specific diagnostics, information that is necessary for resolution of the problem is not lost. If you refine the scope of the problem, you save valuable time as you work to locate a solution.

Refer to Troubleshooting Hardware and Related Issues on the MSFC and MSFC2 in order to further troubleshoot the MSFC or MSFC2.

Prerequisites

Requirements

Review the field notices for the Catalyst 6500/6000 products before you start to review this document for known issues with software or hardware. Refer to Field Notices for Cisco Catalyst 6500 Series Switches.

This document discusses show commands, problem symptoms, and troubleshooting procedures for the Catalyst 6500/6000 Series Switches. Cisco recommends that you know the basic architecture information and have a basic understanding of various Catalyst 6500/6000 Series Switch models. Refer to these documents:

• White Papers: Cisco Catalyst 6500 Series Switches

• Product Overview: Catalyst 6000 and 6500 Series

Components Used

This document is not restricted to specific software and hardware versions.

Conventions

Refer to Cisco Technical Tips Conventions for more information on document conventions.

Diagnostic Commands

show version 1	show module	show port	show test
show log	show logging buffer	show config	show mac
show counters	show system	show environment	show errordetection
show boot	dir (switch)	dir (ROMmon2)	show system sanity 3

¹ Refer to the Catalyst 6000 Family Switch and ROM Monitor Commands for more information on show commands.

² ROMmon = ROM monitor.

³ See the Sanity Check for Configuration Issues and System Health section of this document for information on the show sanity check command.

Sanity Check for Configuration Issues and System Health

The show system sanity command runs a set of predetermined checks on the configuration with a possible combination of certain system states in order to compile a list of warning conditions. The checks are designed to look for anything that seems out of place. The checks are intended to help you maintain the desired and correct system configuration and functionality. This command is supported in CatOS version 8.3x or later.

This is a list of the checks that the command runs and the action that the system takes when the condition is found:

• Checks whether the default gateways are reachable.

  If the gateways are reachable, the system stops pinging.

• If a port autonegotiates to half duplex, the system flags it.

• Trunking checks:

  • If a trunk port has the mode set to "on," the system flags it.

  • If a port is trunking and mode is auto, the system flags it.

  • If a trunk port is not trunking and the mode is desirable, the system flags it.

  • If a trunk port negotiates to half duplex, the system flags it.

• Channeling checks:

  • If a port has channeling mode set to "on", the system flags it.

  • If a port is not channeling and the mode is set to desirable, the system flags it.

  • If a VLAN has a spanning-tree root of 32 K, which means that the root is not set, the system flags it.

• Spanning-tree VLAN checks:

  • If a VLAN has a max age on the spanning-tree root that is different than the default, the system flags it.

  • If a VLAN has a fwd delay on the spanning-tree root that is different than the default, the system flags it.

  • If a VLAN has a fwd delay on the bridge that is different than the default, the system flags it.

  • If a VLAN has a fwd delay on the bridge that is different than the default, the system flags it.

  • If a VLAN has a hello time on the bridge that is different than the default, the system flags it.

• Spanning-tree port checks:

  • If a port has a port cost that is different than the default, the system flags it.

  • If a port has a port priority that is different than the default, the system flags it.

• UniDirectional Link Detection (UDLD) checks:

  • If a port has UDLD disabled, the system flags it.

  • If a port has UDLD shut down, the system flags it.

  • If a port has a UDLD undetermined state, the system flags it.

• Assorted port checks:

  • If a port has receive flow control disabled, the system flags it.

  • If a trunk port has PortFast enabled, the system flags it.

  • If an inline-power port has either of the denied, faulty, other, or off states, the system flags it in either of these conditions:

    • If a port has a native VLAN mismatch

    • If a port has a duplex mismatch

• Bootstring and configuration register checks:

  • The configuration register on the primary Supervisor Engine (and on the secondary Supervisor Engine, if present) must be one of these values:

    • 0x2

    • 0x102

    • 0x2102

  • The system verifies the bootstring on the primary Supervisor Engine (and on the secondary Supervisor Engine, if present).

    The system displays a message if the bootstring is empty.

  • The system verifies that every file is specified in the bootstring.

    The system displays a message if the file is absent or shows up with a wrong checksum. If only "device:"  is specified as a file name, the system verifies that the first file is on the device.

• Assorted checks:

  • The system displays a message if Internet Group Management Protocol (IGMP) snooping is disabled.

  • The system displays a message if any of the values of the Simple Network Management Protocol (SNMP) community access strings (ro, rw, rw-all) is the same as the default.

  • The system displays a message if any of the modules is in a state other than "Ok".

  • The system displays a message that lists all the tests that failed in the show test all command.

    A failed test displays as an "F".

  • The system displays a message if *fast is not configured anywhere on the switch.

  • The system displays a message if there is enough room for the crashinfo file on the bootflash:.

  • The system displays a message if multicast routing is enabled globally but is not applied to all interfaces.

  • The system displays a message if IGMP snooping is disabled and Router-Port Group Management Protocol (RGMP) is enabled.

Here is an example output:

Note: The actual output can vary, which depends on the software version.

Console> (enable)show system sanity

Status of the default gateway is:
172.20.52.1 is alive

Please check your confreg value : 0x10f.

Invalid boot image slot0:cat6000-sup2k8.8-3-0-133-BOC.bin specified in the bootstring.
Please check your boot string.
Invalid boot image bootflash:cat6000-sup2k8.7-5-0-98.bin specified in the boot string.
Please check your boot string.
None of the images specified in the boot string are valid.
Please specify at least one valid image in the boot string to ensure the switch
is in a bootable state.

The value for Community-Access on read-only operations for SNMP is the same as default.
Please verify that this is the best value from a security point of view.

The value for Community-Access on read-write operations for SNMP is the same as
default.
Please verify that this is the best value from a security point of view.

The value for Community-Access on read-write-all operations for SNMP is the same
as default.
Please verify that this is the best value from a security point of view.

UDLD has been disabled globally - port-level UDLD sanity checks are being bypassed.

The following ports have receive flowControl disabled:
3/1,3/48

The following vlans have max age on the spanning tree root different from the default:
1-6,10,20,50,100,152,200,300,400,500,521-522,524,570,776,850,917,999

The following vlans have forward delay on the spanning tree root different from the 
default:
1-6,10,20,50,100,152,200,300,400,500,521-522,524,570,776,850,917,999

The following vlans have hello time on the spanning tree root different from the default:
2-6,10,20,50,100,152,200,300,400,500,521-522,524,570,776,850,917,999

Please check the status of the following modules:2

Module 8 failed the following tests :
Port LoopBack Test

Console> (enable)

Port/Connectivity Problems

Catalyst Switch to NIC Card Compatibility Issues

If your switch port is connected to a workstation/server with a network interface card (NIC), and if you find network issues such as slow performance on the workstation/server, intermittent connectivity problems, or Catalyst switch issues that deal with physical connectivity and data link errors, refer to Troubleshooting Cisco Catalyst Switches to NIC Compatibility Issues. This document provides information on how to troubleshoot common physical and data link layer issues between the switch port and NIC card, as well as the known issues with some vendor NICs and the workaround.

Port Shown in errDisable State by show port Command

A port can be in the errDisable state for many reasons. This list provides some of the error conditions:

• Duplex mismatch

• Port-channel misconfiguration

• Bridge Protocol Data Unit (BPDU) Guard violation

• UDLD condition

• Broadcast suppression

• Address Resolution Protocol (ARP) inspection

• Crossbar fallback

When a port is error-disabled (errDisable), it is effectively shut down and no traffic is sent or received on that port. The port LED is set to the color amber. When you issue the show port command, the port status shows errDisable. This is an example of what a port in the errDisable state looks like from the command-line interface (CLI) of the switch:

Console> (enable)show port 11/1
Port  Name               Status     Vlan       Level  Duplex Speed Type
----- ------------------ ---------- ---------- ------ ------ ----- ------------
11/1                     errDisable 1          normal   auto  auto 10/100BaseTX


!--- Output is suppressed.

In order to recover from errDisable state, disable and reenable the port. Issue the set port disable mod/port command and the set port enable mod/port command.

Refer to Recovering From errDisable Port State on the CatOS Platforms. This document explains why ports go into the errDisable state and provides further troubleshooting steps to avoid the condition.

Workstation Unable to Log In to Network During Startup/Unable to Obtain DHCP Address

When a client machine is powered up or rebooted, you may observe one of the symptoms here. The problem may be due to initial connectivity delay that the switch introduced.

• A Microsoft networking client displays No Domain Controllers Available.

• DHCP reports No DHCP Servers Available.

• A Novell Internetwork Packet Exchange (IPX) networking workstation does not have the Novell Login screen upon bootup.

• An AppleTalk networking client displays Access to your AppleTalk network has been interrupted. To re-establish your connection, open and close the AppleTalk control panel. It is also possible that the AppleTalk client chooser application either does not display a zone list or displays an incomplete zone list.

• IBM network stations might have one of these messages:

  • NSB83619--Address resolution failed

  • NSB83589--Failed to boot after 1 attempt

  • NSB70519--Failed to connect to a server

The switch performs a status check on various features such as Spanning Tree Protocol (STP), EtherChannel negotiation, trunking negotiation, or link speed/duplex negotiation. It can take 30 to 34 seconds for the switch to put the port in forwarding mode. The workstation may fail to log in or obtain the IP address through DHCP. Refer to Using PortFast and Other Commands to Fix Workstation Startup Connectivity Delays. The document explains how to reduce the initial connectivity delays and to resolve these issues.

1000BASE-T GBIC Is Not Recognized or Does Not Work

If you have a 1000BASE-T Gigabit Interface Converter (GBIC) and it is not recognized or does not work, refer to Cisco Gigabit Ethernet Transceiver Modules Compatibility Matrix in order to verify software support for the GBIC.

If you run the appropriate level of software but the link still does not work, refer to Connector and Cable Specifications for cable specifications.

Troubleshoot Errors in Switch Port

The symptoms for a nonworking port or a partially working port are innumerable. They fall into two categories:

• Loss of network connectivity

  For example, you cannot reach your server, you cannot ping, and so on.

• Slow performance

  For example, the performance is slower than normal, the network is not as fast as it could be, and so on.

These symptoms are caused by physical layer problems, misconfiguration, traffic overload, and so on. Refer to Troubleshooting Switch Port and Interface Problems. The document explains the various switch port problems and troubleshooting steps that are necessary to narrow down the problem.

Troubleshoot IP MLS on a Supervisor Engine I

If you have connectivity issues between workstations in different VLANs, you might need to troubleshoot the Multilayer Switching (MLS) feature on the Catalyst 6500 Supervisor Engine I-based system in order to ensure that proper entries are available in the hardware forwarding tables. Refer to Configuring and Troubleshooting IP MLS on Catalyst 6500/6000 Switches with an MSFC.

If you are unable to resolve the issue after you troubleshoot with the help of this document, contact Cisco Technical Support for further assistance.

Troubleshoot Unicast IP Routing Cisco Express Forwarding on Supervisor Engine II

If you have connectivity issues between workstations in different VLANs, you might need to troubleshoot the Cisco Express Forwarding (CEF) feature on the Catalyst 6500 Supervisor Engine II-based systems in order to ensure that proper entries are available in the hardware forwarding tables. Refer to Troubleshoot Unicast IP Routing Involving CEF on Catalyst 6500/6000 Series Switches with a Supervisor Engine 2 and Running CatOS System Software.

If you are unable to resolve the issue after you troubleshoot with use of this document, contact Cisco Technical Support for further assistance.

Port SPAN Fails

When you attempt to enable a Switched Port Analyzer (SPAN) port, it continues to fail and becomes inactive.

Switch> (enable)show span
----------------------------------------------------------
Destination     : Port 4/15
Admin Source    : Port 4/1
Oper Source     : None
Direction       : transmit/receive
Incoming Packets: enabled
Learning        : enabled
Multicast       : enabled
Filter          : -
Status          : inactive

When you use the external Shutdown button on any Service (SVC) modules instead of the command line, the newly created SPAN sessions do not work. After you enter the set span command and press Enter, the command line hangs for approximately a minute and then returns with an inactive SPAN session. No packets are forwarded to the SPAN destination.

In order to overcome this issue:

• Enter the set module power down mod_# command to shut down the module.

• Enter the set span disable all command to clear the existing SPAN session.

• Reconfigure the SPAN session.

Refer to Cisco bug ID CSCee07746 (registered customers only) , which tracks this problem.

LED Troubleshooting for Red/Orange SYSTEM and STATUS LEDs

This section provides information on how to troubleshoot why the SYSTEM or STATUS LEDs on a Catalyst 6500/6000 Supervisor Engine appear red or orange. With this information, you can understand the conditions that cause a nongreen LED status. The show commands are issued to check for and resolve each condition.

LED	Color	Possible Reasons
STATUS	Red	diagnostic test failed overtemperature condition has occurred
STATUS	Amber	switch in ROMmon
SYSTEM	Red	any voltage failure simultaneous temperature and fan failure 100% power supply failure (2 out of 2 or 1 out of 1) EEPROM failure SCP communication failure 1 failure of redundant clock
SYSTEM	Amber	temperature alarm fan failure or partial power supply failure (1 out of 2)

¹ SCP = Serial Control Protocol.

STATUS LED - Red

The STATUS LED on the Supervisor Engine appears red when one of these conditions is detected:

• A diagnostic test fails. The module is not operational because a fault is detected during the power on or bootup diagnostic tests.

• An overtemperature condition occurs. A major threshold has been exceeded during environmental monitoring.

Condition #1 - A Diagnostic Test Fails

The STATUS LED flashes orange once and stays orange during diagnostic boot tests. The LED turns green when the module is operational (online). The module is not operational because a fault is detected during the power on or bootup diagnostic tests. Issue the show test command in order to view the results of diagnostic tests. This example shows the error display for a Supervisor Engine in slot 2:

Console>show test 2 
Module 2 : 2-port 1000BaseX Supervisor 
Network Management Processor (NMP) Status: (. = Pass, F = Fail, U = Unknown) 
  ROM:  .   Flash-EEPROM: .   Ser-EEPROM: .   NVRAM: F   EOBC Comm: . 

Line Card Status for Module 2 : PASS 

Port Status : 
  Ports 1  2 
  ----------- 
        .  .

Line Card Diag Status for Module 2  (. = Pass, F = Fail, N = N/A)

Module 2 
  Cafe II Status : 
        NewLearnTest:             . 
        IndexLearnTest:           . 
        DontForwardTest:          . 
        DontLearnTest:            . 
        ConditionalLearnTest:     . 
        BadBpduTest:              . 
        TrapTest:                 . 
 Loopback Status [Reported by Module 2] : 
  Ports 1  2 
  ----------- 
        .  . 
Channel Status : 
  Ports 1  2 
  ----------- 
        .  .
Console>

The output of the show test command reports that NVRAM failed ( NVRAM:F ). See the Supervisor Engine Is Faulty Due to NVRAM Failure or show version Command Gives Checksum Failures section of this document for additional information to troubleshoot faulty NVRAM. If other components or tests have failed, contact Cisco Technical Support for further troubleshooting.

Condition #2 - An Overtemperature Condition Occurs

When environmental monitoring detects an overtemperature condition, it reports one of the syslog messages that this list displays and an SNMP major alarm, if enabled. These syslog messages are temperature-related:

• SYS-0-MOD_TEMPMAJORFAIL: Module [dec] major temperature threshold exceeded

• SYS-0-MOD_TEMPMAJORRECOVER: Module [dec] major temperature threshold recovered

• SYS-0-SUP_TEMPMAJORFAIL: Supervisor in slot [dec] [chars] major temperature threshold exceeded

• SYS-0-SUP_TEMPMAJORRECOVER: Supervisor in slot [dec] [chars] major temperature threshold recovered

• SYS-2-MOD_TEMPMINORFAIL: Module [dec] minor temperature threshold exceeded

• SYS-2-MOD_TEMPOK: Module [dec] temperature OK

• SYS-2-MOD_TEMPSHUTDOWN: Module [dec] shutdown due to high temperature reading

For more information and the recommended action, refer to the Messages and Recovery Procedures section of the Catalyst 6500/6000 documentation.

SYSTEM LED - Red

The SYSTEM LED turns green to indicate an OK status for all chassis environmental monitors. If one or more environmental monitors reports a problem, the SYSTEM LED appears orange or red. The chassisMajorAlarm changes to on status in response to one of these conditions:

• Any voltage failure

• Simultaneous temperature and fan failure

• 100 percent power supply failure (2 out of 2 or 1 out of 1)

• EEPROM failure

• SCP communication failure

• Failure of redundant clock

Condition #1 - Any Voltage Failure

The WS-C6K-VTT= Voltage Termination (VTT) module terminates signals on the Catalyst switching bus. Refer to the Cisco Clock Module (WS-C6K-CL and WS-C6513-CL) and VTT Module (WS-C6K-VTT) Installation Note for more information. If a redundant Supervisor Engine is not installed, and there is a VTT module minor or major overtemperature condition, the system shuts down. There are three voltage regulators (VTT) in the backplane. If one VTT module fails, it is a minor alarm. If two fail, there is a major problem and the corrective action shuts down the system.

Issue the show test command or the show environment command in order to display the status of the VTTs, as this example shows:

Console>show test 
        
Environmental Status (. = Pass, F = Fail, U = Unknown,      N = Not Present) 
  PS1:.     PS2:N          PS1 Fan:.     PS2 Fan:N 
  Chassis-Ser-EEPROM:.          Fan:. 
  Clock(A/B):A              Clock A:.     Clock B:. 
  VTT1:.    VTT2:F         VTT3:.

The example shows a failure with clock module A and VTT module 2.

Console>show environment    
    Environmental Status (. = Pass,      F = Fail, U = Unknown, N = Not Present) 
           PS1:.     PS2:N          PS1 Fan:.     PS2 Fan:N 
           Chassis-Ser-EEPROM:.          Fan:. 
           Clock(A/B):B              Clock A:F     Clock B:. 
           VTT1:.    VTT2:F         VTT3:.

Condition #2 - Simultaneous Temperature and Fan Failure

Issue the show system command in order to determine whether an active temperature alarm occurs and you have a fan failure. A fan failure can cause a temperature alarm. In this sample output, the Temp-Alarm and Fan-Status fields are in boldface:

Console (enable)show system    
   PS1-Status PS2-Status 
   ---------- ---------- 
   ok         none    
Fan-Status Temp-Alarm Sys-Status Uptime      d,h:m:s Logout 
     ---------- ---------- ---------- -------------- ---------      
     ok       off             ok         14,08:03:03         20 min 
PS1-Type     PS2-Type      
     ------------ ------------ 
     WS-CAC-1300W none

Condition #3 - 100 Percent Power Supply Failure (2 out of 2 or 1 out of 1)

Issue the show system command in order to determine if you have a 100 percent power supply failure. In the sample output here, the PS1-Status and PS2-Status fields are in boldface. The system in this output has only one power supply. The status displays as faulty. A replacement of the power supply can be necessary. Contact Cisco Technical Support for further assistance.

cat6k> (enable)show system
PS1-Status PS2-Status
---------- ----------
faulty 		none

Fan-Status Temp-Alarm Sys-Status Uptime d,h:m:s Logout
---------- ---------- ---------- -------------- ---------
ok 		off 	faulty 	   409,16:15:08 20 min

PS1-Type 	PS2-Type
-------------------- --------------------
WS-CAC-1300W WS-CAC-1300W

Modem 	Baud Traffic  Peak Peak-Time
------- ----- ------- ---- -------------------------
disable 9600 	0% 	10% Fri Nov 22 2002, 15:24:46

Condition #4 - EEPROM Failure

Issue the show test 1 command in order to view the status of diagnostic tests on the Flash EEPROM and serial number EEPROM. If the Supervisor Engine is not able to read the EEPROM of a specific module, the module is marked faulty (F). It does not come up. The module can have been seated improperly. Reseat the module to see if the issue is resolved. If this does not resolve the issue, power cycle the switch. If this does not resolve the issue, a replacement of the module can be necessary. Contact Cisco Technical Support for further assistance.

This example shows that the Ser-EEPROM has failed on the Supervisor Engine in slot 1.

Console (enable)show test 1   
Diagnostic mode: minimal   (mode at next      reset: minimal) 
       
Module 1 : 2-port 1000BaseX Supervisor      
     Network Management Processor (NMP) Status: (. = Pass,      F = Fail, U = Unknown) 
       ROM:  .   Flash-EEPROM: .        Ser-EEPROM: F   NVRAM: .   EOBC Comm: .

Condition #5 - SCP Communication Failure

SCP defines the protocol that is used to exchange messages between the Supervisor Engine CPU and the MSFC and other line cards through the Ethernet-out-of-Band Channel (EOBC) on the Catalyst 6500.

If there is a failure in this communication, these error messages can appear in the console or syslog. This table provides a brief description of the message and the recommended action:

Message	Explanation	Recommended Action
SYS-2-MOD_NOSCPRESPONSE: Module [dec] not responding over SCP	A module has failed to respond to the Supervisor Engine requests over the out-of-band communication channel. This error can occur when the out-of-band channel is very busy. [dec] is the number of the module that does not respond.	If the problem persists, contact Cisco Technical Support with the output of the show log, show tech-support, and show logging buffer -1023 commands.
SYS-2-MOD_SCPERROR2: Module [dec] local SCP error detected... resetting	This message indicates that there is a problem with the Supervisor Engine serial channel that is used for out-of-band communication with other modules in the system. [dec] is the failing Supervisor Engine number.	The system attempts to recover with a reset of the failing Supervisor Engine. If the problem persists, contact Cisco Technical Support with the output of the show log, show tech-support, and show logging buffer -1023 commands.

These are a few other error messages seen during the SCP communication failure:

Supervisor sends a SCP ping once in 2 seconds to every line card. If no response is received after 3 pings (6 seconds), it is counted as the first failure. After 25 such successive failures (that is, after 150 seconds of not receiving a response from the line card), the supervisor power cycles that line card. After every 30 seconds, this error message is seen on the switch:

%CPU_MONITOR-SP-6-NOT_HEARD: CPU_MONITOR messages have not been heard for 61
   seconds [2/0]
%CPU_MONITOR-SP-6-NOT_HEARD: CPU_MONITOR messages have not been heard for 151 
   seconds [2/0]

After 150 seconds, the module gets power cycled with these syslogs:

%CPU_MONITOR-SP-3-TIMED_OUT: CPU_MONITOR messages have failed, resetting module 
   [2/0]
%OIR-SP-3-PWRCYCLE: Card in module 1, is being power-cycled off (Module not 
   responding to Keep Alive polling)
%OIR-SP-3-PWRCYCLE: Card in module 2, is being power-cycled off (Heartbeat 
   Messages Not Received From Module)

Condition #6 - Failure of Redundant Clock

Refer to the Cisco Clock Module (WS-C6K-CL and WS-C6513-CL) and VTT Module (WS-C6K-VTT) Installation Note for information on the clock module (WS-C6K-CL=).

On systems that run the Catalyst software, issue the show environment command. The example here shows how to check the status of the VTT modules and clock modules on a Catalyst 6000 series switch that runs the Catalyst software. It shows that Clock A has failed and that you need to replace it:

Console>show environment 
    Environmental Status (. = Pass,    F = Fail, U = Unknown, N = Not Present) 
         PS1:.     PS2:N        PS1 Fan:.     PS2 Fan:N 
         Chassis-Ser-EEPROM:.        Fan:. 
         Clock(A/B):B            Clock A:F     Clock B:. 
         VTT1:.    VTT2:.       VTT3:. 
       Console>

STATUS LED - Orange

If the system software is unable to start up, the STATUS LED stays orange and the switch goes in ROMmon mode. Refer to Recovering Catalyst Switches Running CatOS from Booting Failures in order to recover the switch from ROMmon mode.

SYSTEM LED - Orange

An orange SYSTEM LED indicates that a minor chassis alarm occurs because of one of these conditions:

• Temperature alarm

• Fan failure or partial power supply failure (1 out of 2)

Condition #1 - Temperature Alarm

Issue the show system command in order to determine if an active temperature alarm occurs. In this sample output, the Temp-Alarm field is in boldface:

Console (enable)show system    
   PS1-Status PS2-Status 
   ---------- ---------- 
   ok         none    
Fan-Status Temp-Alarm Sys-Status Uptime d,h:m:s      Logout 
     ---------- ---------- ---------- -------------- ---------      
     ok       off             ok         14,08:03:03         20 min 
PS1-Type     PS2-Type      
     ------------ ------------ 
     WS-CAC-1300W none

Conditions #2 - Fan Failure or Partial Power Supply Failure

Issue the show test command, but do not specify a slot number. This action enables you to view the status of system hardware components such as the power supply and power supply fan. The show test command indicates that one of the power supplies failed a diagnostic test, which the F indicates.

Console (enable)show test  

Diagnostic mode: minimal   (mode at next   reset: minimal) 
Environmental Status (. = Pass, F = Fail, U =   Unknown, N = Not Present) 
  
  PS1: .     PS2: N     PS1 Fan:  .     PS2 Fan: N 
  Chassis-Ser-EEPROM: .     Fan:  . 
  Clock(A/B):  A         Clock A:  .     Clock B: . 
  VTT1: .    VTT2: .    VTT3:  .

Supervisor Engine Module Problems

Supervisor Engine Module Is Not Recognized/Does Not Come On Line

If you have a Supervisor Engine module installed in the Catalyst 6500/6000 Switch chassis and the module is not recognized or does not come on line, follow these recommendations:

• Connect a terminal to the console port of the switch and verify that the switch is in ROMmon mode. The switch can go into ROMmon mode for many reasons, such as incorrect boot variables or a corrupted software image. Refer to Recovering Catalyst Switches Running CatOS from Booting Failures in order to recover the switch from ROMmon mode.

• If you do not get the console prompt or ROMmon prompt, the other reason can be that the Supervisor Engine module is improperly seated in the slot. Power off the switch, remove the module, and inspect for bent pins on the slot connector on the backplane. Use a flashlight when you inspect the connector pins on the chassis backplane, if necessary. You can only install the Supervisor Engines in slot 1 and 2. Try to reseat the module again. Check that the screws on both sides are tightened. Confirm that the Supervisor Engine module is inserted tightly into the chassis. Switch on the chassis and watch the status. Even if the Supervisor Engine has a bad backplane connection, as long as it receives power during bootup, it logs startup logs.

• If your switch still does not work, create a service request with Cisco Technical Support for more help. The switch can be getting stuck during bootup. Collect all the logging from the console during startup until the moment at which the Supervisor Engine gets stuck and create a service request with Cisco Technical Support.

Supervisor Engine Has Reset on Its Own

If you suspect that the switch has reset by itself, issue the show version command in order to verify the switch uptime, which is the time since the last reset. Issue the show log command in order to look at the reboot history, as this example shows. View this command output in order to see if there are any exceptions recorded:

sup2a> (enable)show version
WS-C6506 Software, Version NmpSW: 6.3(10)


!--- Output is suppressed.


Uptime is 7 days, 4 hours, 27 minutes
sup2a> (enable)show log

Network Management Processor (ACTIVE NMP) Log:
  Reset count:   1
  Re-boot History:   Jan 06 2003 10:35:56 0

  Bootrom Checksum Failures:      0   UART Failures:                  0
  Flash Checksum Failures:        0   Flash Program Failures:         0
  Power Supply 1 Failures:        0   Power Supply 2 Failures:        0
  Swapped to CLKA:                0   Swapped to CLKB:                0
  Swapped to Processor 1:         0   Swapped to Processor 2:         0
  DRAM Failures:                  0

  Exceptions:                     0

  Loaded NMP version:            6.3(10)
  Software version:              slot0:cat6000-sup2.6-3-10.bin
  Reload same NMP version count: 1

  Last software reset by user: 1/6/2003,10:35:35

  EOBC Exceptions/Hang:            0

Heap Memory Log:
Corrupted Block = none

This show log command output displays no software exceptions. The last reboot of the switch is Jan 06 2003. The reboot time matches in the Last software reset field.

The show log command output here shows an exception that was recorded at the time of the last reboot. If your switch shows such a software exception, create a Cisco Technical Support service request with the output of the show tech command output and the show log command output. A Technical Support engineer can identify the reason for the crash.

esc-cat5500-b (enable)show log

Network Management Processor (STANDBY NMP) Log:
Reset count: 38
Re-boot History: Oct 14 2001 05:48:53 0, Jul 30 2001 06:51:38 0
Jul 28 2001 20:31:40 0, May 16 2001 21:15:39 0
May 02 2001 01:02:53 0, Apr 26 2001 21:42:24 0
Apr 07 2001 05:23:42 0, Mar 25 2001 02:48:03 0
Jan 05 2001 00:21:39 0, Jan 04 2001 4:54:52 0
Bootrom Checksum Failures: 0 UART Failures: 0
Flash Checksum Failures: 0 Flash Program Failures: 0
Power Supply 1 Failures: 4 Power Supply 2 Failures: 0
Swapped to CLKA: 0 Swapped to CLKB: 0
Swapped to Processor 1: 3 Swapped to Processor 2: 0
DRAM Failures: 0

Exceptions: 1

Loaded NMP version: 5.5(7)
Reload same NMP version count: 3

Last software reset by user: 7/28/2001,20:30:38
Last Exception occurred on Oct 14 2001 05:47:29 ...
Software version = 5.5(7)
Error Msg:
PID = 86 telnet87
EPC: 80269C44


!--- Output is suppressed.

If your switch shows such a software exception, capture the show log command output and the show module command output. Use the Output Interpreter (registered customers only) tool in order to decode the software crash. If an issue is identified, you can upgrade to the software version that has the fix for the issue.

Check the power source for the switch to make sure that it did not fail. If you use an uninterruptible power supply (UPS), make sure that it works properly.

If there is no log or the Output Interpreter (registered customers only) tool cannot identify the issue, create a Cisco Technical Support service request with the output of the show tech command and the show log command. A Technical Support engineer can identify the reason for the crash.

Supervisor Engine Is at ROMmon Prompt

The switch can go into ROMmon mode for any of these reasons:

• The boot variables are not set correctly to boot the switch from a valid software image.

• The configuration register is not set correctly.

• The software image on the bootflash: is missing or corrupted, or there is a software upgrade failure.

Refer to Recovering Catalyst Switches Running CatOS from Booting Failures in order to recover the switch from the ROMmon prompt.

Supervisor Engine Is Faulty Due to NVRAM Failure or show version Command Gives Checksum Failures

If the Supervisor Engine shows the faulty status due to NVRAM component failure, as the show test command output in this section shows, and the show version command gives checksum failure as the command output shows, the problem can be Cisco bug ID CSCdx87646 (registered customers only) . The caveat is resolved in CatOS software release 7.5(1) or later.

Console>show test 2 


Module 2 : 2-port 1000BaseX Supervisor 
Network Management Processor (NMP) Status: (. = Pass, F = Fail, U = Unknown) 
  ROM:  .   Flash-EEPROM: .   Ser-EEPROM: .   NVRAM: F   EOBC Comm: . 

Line Card Status for Module 2 : PASS 

Port Status : 
  Ports 1  2 
  ----------- 
        .  .

Line Card Diag Status for Module 2  (. = Pass, F = Fail, N = N/A)

Module 2 
  Cafe II Status : 
        NewLearnTest:             . 
        IndexLearnTest:           . 
        DontForwardTest:          . 
        DontLearnTest:            . 
        ConditionalLearnTest:     . 
        BadBpduTest:              . 
        TrapTest:                 . 
 Loopback Status [Reported by Module 2] : 
  Ports 1  2 
  ----------- 
        .  . 
Channel Status : 
  Ports 1  2 
  ----------- 
        .  .
Console>

Console>show version

WS-C6509 Software, Version NmpSW: 5.5(4b) 
Copyright (c) 1995-2000 by Cisco Systems 
NMP S/W compiled on Nov 26 2000, 12:28:35 
System Bootstrap Version: 5.3(1) 
Hardware Version: 2.0 Model: WS-C6509 Serial #: SCA0508004S  


Mod Port Model               Serial #    Versions 
--- ---- ------------------- ----------- -------------------------------------- 
1   2    WS-X6K-SUP1A-2GE    SAD050806TA Hw : 7.0 
                                         Fw : 5.3(1) 
                                         Fw1: 5.4(2) 
                                         Sw : 5.5(4b) 
                                         Sw1: 5.5(4b) 
         WS-F6K-PFC          SAD05090CR3 Hw : 1.1 


!--- Output is suppressed.

       DRAM                    FLASH                   NVRAM 
Module Total   Used    Free    Total   Used    Free    Total Used  Free 
------ ------- ------- ------- ------- ------- ------- ----- ----- ----- 
1       65408K  38048K  27360K  16384K   6966K   9418K  512K  283K  229K 


EXTBAN checksum failed. 
VTP checksum failed.

!--- or

Global checksum failed.

Uptime is 8 days, 9 hours, 3 minutes

Console>

The global checksum error means that the next time the box is reloaded, the NVRAM most likely is lost due to a CRC checksum that failed while it read the configuration. This is not usually a hardware error, but the switch corrects itself. This does not have any impact to an operational switch unless configuration changes are made while the switch is in this condition. But most of the time, a reset resolves the checksum failure as it is re-calculated.

Complete these steps in order to recover the switch from this error state:

1. Backup the configuration of the switch. Refer to Uploading Configuration Files to a TFTP Server for more information on backing the configuration.

2. Reset the Supervisor module by issuing the reset supervisor_module_# command.

3. Once the switch boots up, issue the show version and show test commands in order to verify if the output is normal.

4. Verify the configuration existing on the switch, and restore from the backup if required.

Switch Backplane Traffic Registers 100 Percent

A Catalyst 6500 with a Supervisor Engine 2 can show 100 percent traffic in the show system command output and the show traffic command output. This switch output shows the switch backplane traffic information:

6k-2a> (enable)show system
PS1-Status PS2-Status
---------- ----------
ok           ok

Fan-Status Temp-Alarm Sys-Status Uptime d,h:m:s Logout
---------- ---------- ---------- -------------- ---------
ok         off        ok         20,05:37:25      none

PS1-Type              PS2-Type
-------------------- --------------------
WS-CAC-1300W          WS-CAC-1300W

Modem Baud Backplane-Traffic Peak Peak-Time
------- ----- ----------------- ---- -------------------------
disable 9600 100% 100% Fri Sep 26 2003, 08:33:18

PS1 Capacity: 1153.32 Watts (27.46 Amps @42V)
PS2 Capacity: 1153.32 Watts (27.46 Amps @42V)
PS Configuration : PS1 and PS2 in Redundant Configuration.

The value in this output represents the current backplane utilization based on the values 0 percent through 100 percent. Under normal operating conditions, this value is low. The Supervisor Engine is affected by a known, cosmetic issue, Cisco bug ID CSCdx54751 (registered customers only) , if all these items are true:

• The value displays as 100 percent.

• The utilization LEDs on the Supervisor Engine are not at 100 percent.

• The output of the show top command is also below 100 percent.

  Note: The show top command provides port utilization snapshots.

This issue is cosmetic and does not indicate a problem that affects service. The fix to this issue is to download the traffic meter update image that is located in the software center. In order to reprogram the electrically programmable logic device (EPLD) for the proper output, download the epld-sup2-trafficmeter-swupdate.hZ image and follow the instructions in the README.epld_update document.

In order to find the image and document with instructions, refer to the Software Download - Catalyst 6500/6000 CatOS System Software (registered customers only) images.

Out-Discards Increment on Supervisor Engine 32 Ports

Out-discards increment every 5 seconds on the WS-SUP32-GE-3B, even after you disconnect the port.

A software bug in Cisco IOS Software releases that are earlier than 8.4(4) causes the issue. Refer to Cisco bug ID CSCei40623 (registered customers only) for more information.

Multicast MAC Addresses [3333.0000.0000.000x] appear in the MAC Address Table of Supervisor Engine 720

In Supervisor Engine 720 with Cisco IOS Software Release 12.2(14)SX and later, explicit host tracking is enabled for IGMPv3 hosts. The switch software automatically programs the hardware for these multicast MAC addresses [3333.0000.0000.000x]. Refer to RFC 2464 for more information on these multicast MAC addresses.

Note: There is a discussion available in Cisco Support Community. Refer to Static 3333.0000.000d in CAM table for more information about the discussion.

Sup720Switch#show mac-address-table
Legend: * - primary entry
        age - seconds since last seen
        n/a - not available

  vlan   mac address     type    learn     age              ports
------+----------------+--------+-----+----------+--------------------------
*  500  3333.0000.000d    static  Yes          -   Gi1/1,Gi1/2,Gi1/3,Gi1/4
                                                   Gi1/5,Gi1/6,Fa2/1,Fa2/4
                                                   Fa2/5,Fa2/6,Fa2/7,Fa2/14
                                                   Fa2/15,Fa2/16,Fa2/17,Fa2/19
                                                   Fa2/22,Fa2/23,Fa2/24
                                                   Router,Switch
*  501  3333.0000.000d    static  Yes          -   Gi1/1,Gi1/2,Gi1/3,Gi1/4
                                                   Gi1/5,Gi1/6,Fa2/1,Fa2/4
                                                   Fa2/5,Fa2/6,Fa2/7,Fa2/14
                                                   Fa2/15,Fa2/16,Fa2/17,Fa2/19
                                                   Fa2/22,Fa2/23,Fa2/24
                                                   Router,Switch
*  611  3333.0000.0001    static  Yes          -   Switch
*  610  3333.0000.0001    static  Yes          -   Switch

!--- Output is suppressed.

If IGMP is not used in the network and if it is needed in order to eliminate the multicast MAC address entries for the MAC address table, issue these commands under the VLAN interface configuration:

no ip igmp snooping explicit-tracking
no ipv6 mld snooping explicit-tracking
no ipv6 mld snooping

Note: The IGMP commands are automatically enabled when the Cisco IOS software is upgraded. These IGMP commands must be disabled again.

What is Explicit Host Tracking of IGMPv3 Hosts?

Explicit host tracking is supported only with IGMPv3 hosts.

When you enable explicit host tracking and the Catalyst 6500 Series Switch works in proxy-reporting mode, the router might not be able to track all the hosts that are behind a VLAN interface. In proxy-reporting mode, the Catalyst 6500 Series Switch forwards only the first report for a channel to the router and suppresses all other reports for the same channel.

With IGMPv3 proxy reporting, the Catalyst 6500 Series Switch does proxy reporting for unsolicited reports and reports that are received in the general query interval.

Proxy reporting is turned on by default. When you disable proxy reporting, the Catalyst 6500 Series Switch works in transparent mode and updates the IGMP snooping database as it receives reports and forwards this information to the upstream router. The router can then explicitly track all reporting hosts.

When you disable explicit tracking, it disables fast-leave processing and proxy reporting.

IGMPv3 supports explicit host tracking of membership information on any port. The explicit host-tracking database is used for fast-leave processing for IGMPv3 hosts, proxy reporting, and statistics collection. When you enable explicit host tracking on a VLAN, the IGMP snooping software processes the IGMPv3 report that it receives from a host and builds an explicit host-tracking database that contains this information:

• The port that is connected to the host

• The channels that are reported by the host

• The filter mode for each group that is reported by the host

• The list of sources for each group that is reported by the hosts

• The router filter mode of each group

• The list of hosts that request the source for each group

Supervisor Engine Crashes due to DBE Bus Error Exception (data store or load)

A Bus Error Exception happens when the CPU receives erroneous data from memory. The two types of bus errors are instruction bus error and data bus error.

An instruction bus error occurs when the CPU is unable to fetch an instruction. A uncorrupted stack shows which instruction access failed. An instruction bus error is rare and is usually a software bug.

A data bus error occurs when a data load or store instruction results in either a parity error or memory access timeout.

If the CPU attempts to access a memory location where no hardware is present, the system controller forces a data bus error exception to the CPU. Data bus error exceptions can be caused by a software bug or hardware problems.

If the crash occurs for the first time then it might be because of a transient parity error on the memory. If the switch crashes again, upgrade the CatOS software to the latest version so that the root cause of the crash can be identified by the mistral register dump enhancement. The mistral can signal a DBE to the CPU when the system crashes.

Line Card/Module Problems

Line Card Is Not Recognized/Does Not Come Online

In some cases, you receive a new line card and, when you insert the line card in the Catalyst 6500/6000 Switch chassis, you experience one of these occurrences:

• The show module command on the switch does not recognize the line card in the inserted slot.

• The line card LED status is not green.

Use these recommendations in order to troubleshoot this problem:

• The line card fails to come up if the switch chassis or Supervisor Engine does not support it. Refer to the Cisco Catalyst 6500 Series Switches Release Notes for details about the line cards that are supported by the switch chassis and Supervisor Engine in your switch. Also, make sure that the line card is installed in the proper slot because the slot allocation for line cards varies with the chassis.

• The CatOS version that runs on the Supervisor Engine module may not support the new line card that you inserted in the chassis. Issue the show version command in order to check the CatOS version that runs on your switch. Verify the hardware/software compatibility in order to find out the minimum CatOS version that is required to support the given line card. Use the Software Support for Hardware feature of the Software Advisor (registered customers only) tool. Refer to the Catalyst 6500 Series Release Notes in order to find which hardware is supported by the CatOS that runs on your switch.

• The module can fail to come up if there is not sufficient power in the chassis. Issue the show module command in order to look for the status of the module. If the command output indicates power-deny, the issue is most likely not a hardware issue but a power budget issue. Issue the show environment power command in order to check the redundancy mode of the power supply. If you use 1 + 1 redundancy, you have these two choices:

  • Install two higher-wattage power supplies if you still want 1+1 redundancy.

  • Change the mode of power redundancy to combined. This means that the available power is now the sum of the two power supplies installed in the system. If you lose one of the power supplies, however, some of the modules may be shut down. This occurs because one of the power supplies is not able to supply power on its own.

  Refer to Power Management for Catalyst 6000 Series Switches for more information about the power management issues with the Catalyst 6500/6000.

• The other common cause is improper seating of line card in the slot. Power off the switch and remove the module. Inspect for bent pins on the slot connector on the backplane. Use a flashlight when you inspect the connector pins on the chassis backplane, if necessary. Try to reseat it again. Be sure that the screws on both sides are tightened. Confirm that the line card is inserted tightly into the chassis. Power on the chassis and observe the status. In some cases, a badly-seated card can cause symptoms that appear to be a hardware failure. A badly-seated card can cause traffic corruption on the backplane. This can result in various problems in the Catalyst chassis. For example, if one module corrupts traffic on the Catalyst backplane, the self test can fail for both itself and other modules. Reseat all the cards in order to resolve this and allow the self tests to pass.

• If your line card still does not work, create a service request with Cisco Technical Support.

Troubleshoot WS-X6348 Module Port Connectivity for Catalyst 6500/6000 That Uses CatOS

If you have connectivity issues with the hosts connecting on the WS-X6348 module or other 10/100 modules, refer to Troubleshooting WS-X6348 Module Port Connectivity for Catalyst 6500/6000 Using CatOS. The document provides detailed troubleshooting steps.

MSFC/MSFC 2 Is Not Recognized in Supervisor Engine show module Command

The MSFC/MSFC 2 can disappear from the show module command output. This occurs if the device fails to boot properly for one of these reasons:

• A Cisco IOS Software image is corrupted.

• A bootflash is misseated.

• The MSFC or MSFC2 drops to ROMmon for some reason.

This output is a sample:

Cat6500 (enable) show module
Mod Slot Ports Module-Type Model Sub Status
--- ---- ----- ------------------------- ------------------- --- --------
1 1 2 1000BaseX Supervisor WS-X6K-SUP1A-2GE yes ok
Mod Module-Name Serial-Num
--- -------------------- -----------
1 SAD040200B3
Cat6500 (enable) session 15
Module 15 is not installed.

This is a sample command output from the show module command when the MSFC is in the other state:

Cat6500 (enable) show module
Mod Slot Ports Module-Type Model Sub Status
--- ---- ----- ------------------------- ------------------- --- --------
1 1 2 1000BaseX Supervisor WS-X6K-SUP1A-2GE yes ok
15 1 1 Multilayer Switch Feature WS-F6K-MSFC no other
Cat6500 (enable) session 15
Trying Router-15... 

!--- The session is not created. !--- Hold down Ctrl and press C to escape.

Refer to Troubleshooting Hardware and Related Issues on the MSFC and MSFC2 for a procedure to recover the MSFC.

Password Recovery on Supervisor Engine/MSFC

If you need to recover a lost password on a Supervisor Engine, refer to Password Recovery for Catalyst 1200, 1400, 2901, 2902, 2926T/F, 2926GS/L, 2948G, 2980G, 4000, 5000, 5500, 6000, 6500 Running CatOS for a step-by-step procedure.

Refer to Password Recovery Procedure for the Catalyst 6000 MSFC for a procedure to recover a password on the MSFC.

Troubleshoot WS-X6348 Module Silent Reloads

This section defines a "Silent Reload" and explains how to identify it. A Silent Reload occurs when a line card experiences a CPU exception and resets. Usually, you see a "module up" in the logging buffer but no indication that the line card went down. Look at the show log command output in order to see uptime for this module.

For example, this output is from the show logging buffer -1023 command:

2003 Apr 27 01:07:14 GST +04:00 %SYS-5-MOD_OK:Module 4 is online 
2003 Apr 27 01:07:14 GST +04:00 %SYS-3-MOD_PORTINTFINSYNC:Port Interface in sync for 
Module 4
2003 Apr 27 01:07:30 GST +04:00 %PAGP-5-PORTTOSTP:Port 4/6 joined bridge port 4/6
2003 Apr 27 01:07:30 GST +04:00 %PAGP-5-PORTTOSTP:Port 4/7 joined bridge port 4/7
2003 Apr 27 01:07:30 GST +04:00 %PAGP-5-PORTTOSTP:Port 4/8 joined bridge port 4/8
2003 Apr 27 01:07:30 GST +04:00 %PAGP-5-PORTTOSTP:Port 4/9 joined bridge port 4/9
2003 Apr 27 01:07:30 GST +04:00 %PAGP-5-PORTTOSTP:Port 4/21 joined bridge port 4/21
2003 Apr 27 01:07:30 GST +04:00 %PAGP-5-PORTTOSTP:Port 4/22 joined bridge port 4/22
2003 Apr 27 01:07:30 GST +04:00 %PAGP-5-PORTTOSTP:Port 4/26 joined bridge port 4/26
2003 Apr 27 01:07:30 GST +04:00 %PAGP-5-PORTTOSTP:Port 4/27 joined bridge port 4/27
2003 Apr 27 01:07:30 GST +04:00 %PAGP-5-PORTTOSTP:Port 4/41 joined bridge port 4/41
2003 Apr 27 01:07:30 GST +04:00 %PAGP-5-PORTTOSTP:Port 4/42 joined bridge port 4/42
2003 Apr 27 01:07:30 GST +04:00 %PAGP-5-PORTTOSTP:Port 4/1 joined bridge port 4/1
2003 Apr 27 01:07:30 GST +04:00 %PAGP-5-PORTTOSTP:Port 4/3 joined bridge port 4/3
2003 Apr 28 06:30:12 GST +04:00 %SYS-5-MOD_OK:Module 4 is online 
2003 Apr 28 06:30:12 GST +04:00 %SYS-3-MOD_PORTINTFINSYNC:Port Interface in sync for 
Module 4
2003 Apr 28 06:30:27 GST +04:00 %PAGP-5-PORTTOSTP:Port 4/6 joined bridge port 4/6
2003 Apr 28 06:30:27 GST +04:00 %PAGP-5-PORTTOSTP:Port 4/7 joined bridge port 4/7
2003 Apr 28 06:30:27 GST +04:00 %PAGP-5-PORTTOSTP:Port 4/8 joined bridge port 4/8
2003 Apr 28 06:30:28 GST +04:00 %PAGP-5-PORTTOSTP:Port 4/9 joined bridge port 4/9
2003 Apr 28 06:30:28 GST +04:00 %PAGP-5-PORTTOSTP:Port 4/21 joined bridge port 4/21
2003 Apr 28 06:30:28 GST +04:00 %PAGP-5-PORTTOSTP:Port 4/22 joined bridge port 4/22
2003 Apr 28 06:30:28 GST +04:00 %PAGP-5-PORTTOSTP:Port 4/26 joined bridge port 4/26
2003 Apr 28 06:30:28 GST +04:00 %PAGP-5-PORTTOSTP:Port 4/27 joined bridge port 4/27
2003 Apr 28 06:30:28 GST +04:00 %PAGP-5-PORTTOSTP:Port 4/41 joined bridge port 4/41
2003 Apr 28 06:30:28 GST +04:00 %PAGP-5-PORTTOSTP:Port 4/42 joined bridge port 4/42
2003 Apr 28 06:30:28 GST +04:00 %PAGP-5-PORTTOSTP:Port 4/1 joined bridge port 4/1
2003 Apr 28 06:30:28 GST +04:00 %PAGP-5-PORTTOSTP:Port 4/3 joined bridge port 4/3
2003 Apr 29 05:48:54 GST +04:00 %SYS-5-MOD_OK:Module 4 is online 
2003 Apr 29 05:48:54 GST +04:00 %SYS-3-MOD_PORTINTFINSYNC:Port Interface in sync for 
Module 4

This output is from the show log command:

>show log

Module 3 Log:
  Reset Count:   12
  Reset History: Wed Nov 14 2001, 07:22:24 

!--- This was last reset when the switch reloaded.

                 Tue Nov 13 2001, 14:55:21
                 Tue Nov 13 2001, 05:56:41
                 Mon Nov 12 2001, 11:13:19  

Module 4 Log:
  Reset Count:   19
  Reset History: Fri Apr 4 2003, 04:03:45 

!--- This module continuously resets.

                 Wed Apr 2 2003, 18:59:01
                 Tue Apr 1 2003, 14:46:42
                 Tue Apr 1 2003, 13:01:57
             
Module 15 Log:
  Reset Count:   14
  Reset History: Wed Nov 14 2001, 07:22:23 

!--- This was last reset when the switch reloaded.

                 Tue Nov 13 2001, 14:55:19
                 Tue Nov 13 2001, 05:56:39
                 Mon Nov 12 2001, 11:13:18

Symptoms

You might see this error message in the show logging buffer command output:

2003 Apr 29 05:48:54 GST +04:00 %SYS-5-MOD_OK:Module 4 is online 

For a WS-X6348 line card, if you see only this message and no other messages that indicate why the line card reset, this indicates that the line card has experienced a Silent Reload. If the conditions that this section describes are met, the cause is Cisco bug ID CSCeb35612 (registered customers only) . A replacement of the affected module does not correct the problem.

Corrective Action

This bug has been fixed in these CatOS releases and later releases:

• 6.4(6)

• 7.6(3)

• 8.1(2)

• 8.2(1)

Upgrade the Catalyst switch code to these versions or later versions of CatOS in order to solve this issue.

Troubleshoot WS-X6101-OC12 Module Resets on Catalyst 6500/6000 That Uses CatOS

If the module reboots and there is no SCP ping response, the Supervisor Engine displays these messages:

%SYS-5-MOD_NOSCPPINGRESPONSE:Module 5 not responding... resetting module
%SYS-5-MOD_NOSCPPINGRESPONSE:Module 5 not responding... resetting module
%SYS-5-MOD_NOSCPPINGRESPONSE:Module 5 not responding... resetting module

Note:  Module 5 indicates that the ATM module is on slot 5.

If you have module reset issues or connectivity issues on the WS-X6101-OC12, refer to Troubleshooting EMANATE Subagent Error Message on WS-X6101-OC12 ATM Module.

Troubleshoot Based on Error Messages

Refer to Common CatOS Error Messages on Catalyst 6500/6000 Series Switches for information about the error messages on the console or syslog, and recovery procedures.

If you see an error message that is not in one of the common error messages that Common CatOS Error Messages on Catalyst 6500/6000 Series Switches shows, refer to these resources:

• Messages and Recovery Procedures Product Documentation

• Error Message Decoder (registered customers only)

Related Information

• Troubleshooting Hardware and Related Issues on the MSFC and MSFC2
• Best Practices for Catalyst 4500/4000, 5500/5000, and 6500/6000 Series Switches Running CatOS Configuration and Management
• Switches Product Support
• LAN Switching Technology Support
• Catalyst 6000 Product Support Pages
• Technical Support & Documentation - Cisco Systems