Cisco AS5800 Access Server Hardware Installation Guide - Troubleshooting the Cisco AS5800 [Cisco AS5800 Series Universal Gateways]

Table Of Contents

Hardware Troubleshooting

Problem Solving with Subsystems

Problem Solving with Subsystems—Cisco 7206 Router

Identifying Startup Problems

Starting Up the Cisco 7206 Router Shelf

Troubleshooting the Router Installation

Starting Up the Cisco 5814 Dial Shelf

Troubleshooting the Power Subsystems

Router Shelf Power Subsystem

Dial Shelf Power Subsystem

Troubleshooting the Cooling Subsystems

Router Shelf Cooling Subsystem

Dial Shelf Cooling Subsystem

Troubleshooting the Processor Subsystems

Troubleshooting the Router Shelf Processor Subsystem

Troubleshooting the I/O Controller Card

Troubleshooting the Network Processor Card

Troubleshooting the Port Adapters

Troubleshooting the Dial Shelf Interconnect Port Adapter

Troubleshooting the Dial Shelf Processor Subsystem

Troubleshooting the Dial Shelf Controller Card

Troubleshooting the Dial Shelf Cards

Troubleshooting Network Interfaces

Hardware Troubleshooting

Your Cisco AS5800 was thoroughly tested before leaving the factory. If you encounter problems initializing system startup, use the information in this chapter to help isolate the cause.

This chapter contains the following sections:

•Problem Solving with Subsystems

•Identifying Startup Problems

•Troubleshooting the Power Subsystems

•Troubleshooting the Cooling Subsystems

•Troubleshooting the Processor Subsystems

•Troubleshooting Network Interfaces

The procedures in this chapter assume that you are troubleshooting the initial hardware configuration and system startup. If you have removed or replaced components or changed any default settings, the recommendations in this chapter might not apply. Review the safety warnings listed in the publication Cisco AS5800 Universal Access Server Regulatory Compliance and Safety Information before using the troubleshooting procedures in this chapter.

If you are unable to resolve the problem, contact a customer service representative for assistance and further instructions. Provide the representative with the following information:

•Date you received the system

•Chassis serial number

•Type of software and release number

•Brief description of the problem

•Brief explanation of the steps you have taken to isolate and resolve the problem

•Maintenance agreement or warranty information

Problem Solving with Subsystems

The key to problem solving is isolating the problem to a specific subsystem. The first step in solving startup problems is to compare what the system is doing to what it should be doing. Try to isolate the problem to one of the following three router shelf or dial shelf subsystems:

•Power

•Cooling

•Processor

Start by checking the router shelf power components. If the problem does not seem to originate there, check the dial shelf power components.

Router shelf power components include the following:

•AC-input or DC-input power supplies

•Cisco 7206 router shelf midplane

•External-power cable connection

Dial shelf power components include the following:

•Power-entry modules (PEMs)

•Filter module

•AC-input power shelf

•Cisco 5814 dial shelf backplane

•External-power cable connection

•External monitor system

Next, check the router shelf cooling components. If the problem does not seem to originate there, check the dial shelf cooling components. Router shelf and dial shelf fans should all be operating whenever system power is on.

•Router shelf cooling components include the fan tray.

•Dial shelf cooling components include the blower assembly.

Last, check the router shelf processor cards. If the problem does not seem to originate there, check the dial shelf processor cards.

Most processor cards contain LEDs that indicate the status of the card. Observing these LEDs can help you isolate the problem to a particular card. (See Chapter 4, "Powering On the Cisco AS5800 and Observing Initial Startup Conditions," for LED descriptions and troubleshooting tips.)

The system memory and management functions reside on the router shelf I/O controller card and the network processor card. The enabled LED on each port adapter lights when the port adapter is initialized. A port adapter that is partially plugged into the midplane can cause the system to hang and crash.

The router shelf processor subsystem includes the following:

•I/O controller card

•Network processor card

•Dial shelf interconnect port adapter

•Other port adapters

Dial shelf processor cards include the following:

•Dial shelf controller card

•Modem cards

•Trunk cards

The following sections will help you isolate a problem within one of these subsystems and direct you to the appropriate troubleshooting section.

Problem Solving with Subsystems—Cisco 7206 Router

The key to solving problems with the system is isolating the problem to a specific subsystem. The first step in solving startup problems is to compare what the system is doing to what it should be doing. Because a startup problem is usually caused by a single component, it is more efficient to isolate the problem to a subsystem rather than to troubleshoot each component in the system. For these troubleshooting procedures, consider the following subsystems:

•Power subsystem—This subsystem comprises the power supplies, the external power cable, and the midplane.

•Cooling subsystem—The chassis fan tray is the single component in this subsystem. The fans should be operating whenever system power is on. Replace the fan tray if you determine that a fan is not functioning properly.

•Processor subsystem—This subsystem includes the I/O controller, network processing engine, and all port adapters. The system memory and management functions reside on the I/O controller and the network processing engine. The enabled LED on each port adapter indicates whether the port adapter is initialized. A port adapter that is partially installed in the midplane can cause the system to hang and crash.

You can quickly verify that your new router shelf is properly installed and operative by observing the router LEDs as described in the "Starting the Cisco 7206" section on page 4-8.

Figure 5-1 Troubleshooting Strategy for Startup Problems

Identifying Startup Problems

Hardware startup problems are commonly traced to cabling problems or incorrectly installed power supplies or cards. In rare cases, problems are caused by part failures.

When you start up a Cisco AS5800 for the first time, you should observe the startup sequence, described in the following sections. This chapter describes the normal startup sequence for the router shelf and the dial shelf and the steps to take if the system does not perform that sequence as expected.

In most cases, LEDs indicate system states in the startup sequence. By checking these LEDs, you can determine when and where the system failed. Use the following descriptions to isolate the problem to a subsystem, then proceed to the appropriate sections to try to resolve the problem.

Note   Although an overtemperature condition is unlikely at initial startup, the environmental monitoring functions are included in this chapter because these functions also monitor internal voltages.

Starting Up the Cisco 7206 Router Shelf

When you first power ON the router shelf power supply, the following should occur:

•The power supply LEDs (OK) should light immediately when you set the power supply switch to the ON ( | ) position and remain lit during normal system operation.

If the power OK LEDs do not light, proceed to the section "Troubleshooting the Power Subsystems."

•You should immediately hear the fans operating.

If the fans do not operate, proceed to the section "Troubleshooting the Cooling Subsystems."

•The I/O controller card LEDs should light in the following sequence. If they do not light as described, proceed to the section "Troubleshooting the Processor Subsystems."

•The I/O power OK LED lights immediately and indicates that the I/O controller card is receiving DC power from the router midplane. This LED lights during a successful router boot and remains on during normal operation of the router.

•The enabled LED lights after the I/O power OK LED and indicates that the network processor card and the I/O controller card are enabled for operation by the system; however, that the enabled LED lights does not mean that the Fast Ethernet port on the I/O controller card is functional or enabled. This LED lights during a successful router boot and remains lit during normal router operation.

•The FE enabled LED lights after the enabled LED and indicates that the Fast Ethernet port on the I/O controller card is initialized and enabled for operation by the system. This LED lights during a successful router boot and remains lit during normal router operation.

•The FE link LED lights only when the Fast Ethernet port on the I/O controller card is receiving a carrier signal from the network. (This LED remains off during normal router operation unless there is an incoming carrier signal and does not indicate startup problems.)

•The slot 1 and slot 2 LEDs light after the enabled LED and indicate which PCMCIA slot is in use.

During a successful router boot, these LEDs light and remain on if a Flash memory card is present in the slot. These LEDs also blink when either slot is being accessed by the system. These LEDs do not indicate startup problems.

•The port-adapter-enabled LEDs each light when the network processor card completes its initialization of the port adapter for operation.

The enabled LED indicates that the port adapter is receiving power and has been recognized by the network processor card; it does not indicate the state of the individual port adapter interfaces.

If an enabled LED does not light, proceed to the section "Troubleshooting the Port Adapters," later in this chapter.

•The dial shelf interconnect port adapter link status LED lights.

The link status LED indicates an active connection to the dial shelf and lights when the interconnect port adapter is receiving a carrier signal from the dial shelf controller card. This LED should light and remain on when the access server is operating.

If the interconnect port adapter link LED does not light, proceed to the section, "Troubleshooting the Dial Shelf Interconnect Port Adapter."

•When all router shelf LEDs light to indicate that the system has booted successfully, the initial system banner should be displayed on the router shelf console screen. If it is not displayed, verify that the terminal is set correctly and is properly connected to the I/O controller card console port as described in the "Connecting to the Router Shelf Console and Auxiliary Ports" section.

Troubleshooting the Router Installation

Your Cisco 7206 routers went through extensive testing before leaving the factory. However, if you encounter problems starting the routers, use the information in the chapter "Troubleshooting the Installation" in the Cisco 7206 Installation and Configuration Guide (Document Number DOC-7206-ICG=) to help isolate the cause of the problems. Be sure to review the safety warnings listed in the publication Regulatory Compliance and Safety Information for the Cisco 7200 Series Routers (Part Number 78-3419-xx) that accompanied your Cisco 7206 before using troubleshooting procedures.

If you are unable to easily solve the problem, contact a customer service representative for assistance and further instructions. Be prepared to provide the representative with the following information:

•Date you received the router

•Chassis serial number

•Type of software and release number

•Brief description of the problem you are having

•Brief explanation of the steps you have taken to isolate and resolve the problem

•Maintenance agreement or warranty information

shows the general troubleshooting strategy for Cisco 7206 routers. Refer to this chart, as necessary, to isolate problems to a specific subsystem; then resolve the problem if possible.

Starting Up the Cisco 5814 Dial Shelf

When you first power ON the dial shelf, the following should occur:

•Both PEM power LEDs should light immediately and remain on during normal system operation. If you are using the AC-input power shelf, both AC-input power supply LEDs should light.

If the green power LEDs do not light, proceed to the section "Troubleshooting the Power Subsystem."

•You should hear the fans operating in the blower assembly.

If the fans do not operate, proceed to the section "Troubleshooting the Cooling Subsystems."

•The LEDs on the dial shelf controller card, modem cards, and trunk cards should light as follows:

•Dial shelf controller card—Powers ON first, followed by remaining logic cards. The PWR and MBus LEDs should light. The remaining four alarm LEDs should all briefly flash on and then turn off.

If the MBus LED remains off but the power LED is on, there might be a problem with the MBus power supply on the card. Replace the card.

•Trunk card—At initial power ON, all LEDs light while the system runs a series of diagnostics. After the system passes initial diagnostics, all LEDs go off. The power, host CPU, and framer CPU LEDs then light to indicate that the trunk card is operating properly.

•Modem card—During normal operation, all five LEDs light at system power up. When the modem card CPU software image starts running, it shuts off the modem LED. The power, host CPU, and modem LEDs then light to indicate that the modem card has passed diagnostics and is operating properly.

If the dial shelf processor component LEDs do not light as described, proceed to the section "Troubleshooting the Processor Subsystems."

Troubleshooting the Power Subsystems

The Cisco AS5800 is designed to minimize problems in the power subsystem. The power subsystem includes the router shelf power supplies, the dial shelf PEMs and filter module, and the AC-input power shelf (if used).

The access server comes with two DC power supplies in both the router shelf and the dial shelf, which allow you to replace power supplies while the system is operating. However, if you are using an AC-configured system and you discover a faulty PEM, you must power off the system before performing a replacement.

Caution   If your system requires the AC-input power shelf for AC-to-DC conversion, you should schedule a time for system maintenance and replace the PEM at that time. Refer to the document AC-Input Power Shelf and Power Supply Installation and Replacement (Part Number 78-4659-xx) for installation and replacement instructions.

The failure of a single power supply in the router shelf, the dial shelf, or the AC-input power shelf, if used, will not stop the access server from operating; however, to maintain power redundancy, both power supplies in each unit must be receiving power.

Router Shelf Power Subsystem

In the router shelf, check the following to help isolate a problem with the power subsystem:

•Is the power OK LED on?

•If yes, the power source is good and the power supply is functional.

•If no, verify that the power cable is connected at both ends.

If the power OK LED remains off and the power switch is set correctly, suspect the power source or the power cable. Power OFF the system, connect the power cable to another power source, if available, then power ON the system. If the LED lights, the problem is the first power source.

•If the power OK LED does not light after you connect the power supply to a new power source, replace the power cord, then power ON the system. If the AC power (or input power) LED lights, return the first power cable for replacement.

•If the LED still does not light when the power supply is connected to a different power source with a new power cable, the power supply is probably faulty. If a second power supply is available, install it in the other power supply bay and contact a service representative for further instructions.

•Is the power OK LED on for the second (redundant) power supply?

•If yes, proceed to the section "Troubleshooting the Cooling Subsystems."

•If no, repeat each of the above procedures for the second power supply.

If you are unable to resolve the problem or you determine that either a power supply or chassis connector is faulty, contact a service representative for instructions.

Dial Shelf Power Subsystem

In the dial shelf, check the following to help isolate a problem with the power subsystem:

•Is the first PEM power OK LED on?

•If yes, the power source is good and the PEM is functional. Verify that the second PEM power LED is on for power redundancy.

•If no, verify that the power cable is connected at both ends and check that you have wired the DC-input connections correctly.

If the DC-input connection is wired correctly, the miswire LED remains off.

If the miswire LED is on, the two DC conductors entering the PEM terminal block are reversed. Power OFF the power at your power source and reverse the connections.

•If neither the power nor the miswire LED is on, check the voltage at the input terminal block. If the voltage reading falls between -40 VDC and -60 VDC, replace the PEM.

•If you are using the AC-input power shelf, are both power OK LEDs on?

•If yes, the power source is good and both power supplies are functional.

•If no, verify that the associated AC power cord is plugged in and the correct AC voltage (200 to 240 VAC) is present at the source.

•Verify that the fault LED is off.

If a power supply fault LED lights, the power shelf has detected an internal fault; the power supply is defective. You need to replace the power supply.

•Verify that the overtemperature LED is off.

If a power supply overtemperature LED lights, verify that the ambient temperature is below 120°F (50°C) and the air intake is not blocked. If the condition persists, replace the power supply.

•Verify that the current limit LED is off.

If a power supply current limit LED lights, verify that the DC bus terminals on the AC-input power shelf rear are not short-circuited and a supported dial shelf configuration is being used.

Note   The dial shelf power subsystem supports a maximum configuration of 10 modem cards.

Troubleshooting the Cooling Subsystems

The Cisco AS5800 is designed to minimize problems in the cooling subsystem. The access server can operate in temperatures of up to 120°F (50°C) for a maximum of 72 hours. The access server also contains rendundant fans in both the router shelf fan tray and the dial shelf blower assembly, which allow the system to continue operating despite a single fan failure. Both the router shelf fan tray and the dial shelf blower assembly can be removed and replaced while the system is operating, provided the procedure does not exceed 1 min.

Router Shelf Cooling Subsystem

In the router shelf, check the following to help isolate a problem with the cooling subsystem:

•When you start up the system, do the router shelf fans operate?

To determine whether the router shelf fans are operating, listen for them. In noisy environments, place your hand on the left side of the router shelf (when viewing the router shelf from the front) to feel for air being forced out the vents.

•If yes, power to the fans is good.

•If no, there is a problem with the router shelf fans or there is a problem with the power to the fan tray.

If you determine that the power supply is functioning normally but a fan is faulty, you can replace the fan tray without powering off the access server. Contact a customer service representative if you need assistance.

For problems with the system power, refer to the section "Troubleshooting the Power Subsystem" in this chapter.

Dial Shelf Cooling Subsystem

In the dial shelf, check the following to help isolate a problem with the cooling system:

•Do the blower assembly fans operate?

To determine whether the dial shelf fans are operating, listen for them. In noisy environments, place your hand under the exhaust vents on the back of the dial shelf blower assembly to feel for air being forced out the vents.

•If no, there is a problem with the dial shelf fans or there is a problem with the power to the blower assembly.

If you determine that the power supply is functioning normally but a fan is faulty, you can replace the blower assembly without powering off the access server. Refer to the blower assembly removal instructions in ","and the reinsertion instructions in "." For problems with the system power, refer to the section "Troubleshooting the Power Subsystem" in this chapter.

•Is the green power LED on the blower assembly front panel on?

If no, verify that the blower assembly is fully installed in the dial shelf, the connector is firmly connected to the backplane, and the captive screws are tightened adequately.

•Is the yellow fault LED on the blower assembly front panel on?

If yes, the newly installed blower assembly might be faulty or the chassis connector might be damaged. Shut off system power, remove the blower assembly from the dial shelf, and check the connectors. If the connectors are in good condition, reinstall the blower assembly in the dial shelf and power ON the system. If the fault LED is still on, assume that the blower assembly is faulty. Install another blower assembly and return the faulty blower assembly to the factory.

•The following error message, if displayed on the router shelf console, indicates that the system has detected an overtemperature condition or out-of-tolerance power somewhere in the system.
Queued messages:
%ENVM-1-SHUTDOWN: Environmental Monitor initiated shutdown
If an environmental shutdown results from an out-of-tolerance power condition, the router shelf power OK LED remains off and the system shuts down. (Refer to the section "Troubleshooting the Power Subsystems" in this chapter.) Although an overtemperature condition is unlikely at initial startup, ensure that heated exhaust air from other equipment is not entering the router's inlet vent and that there is sufficient clearance around the sides of the chassis to allow cooling air to flow. Refer to the "Preventive Site Configuration: Maintaining Normal Operation" section" for preventive site configurations.

The above message could also indicate a faulty component or temperature sensor. Before the system shuts down, use the show environment or show environment table command to display the internal chassis environment. Refer to the Cisco 7206 Installation and Configuration Guide for a description of show commands.

Troubleshooting the Processor Subsystems

The processor subsystem consists of the I/O controller card, network processor card, and all port adapters in the router shelf, and the dial shelf controller card, modem cards, and trunk cards in the dial shelf. The following sections contain specific troubleshooting information for each of these components.

The router shelf I/O controller card and network processor card are required system components. The system cannot operate unless the I/O controller card and network processor card are installed properly; however, the system can operate without any port adapters installed as long as none are in partial contact with the midplane pins. A port adapter that is partially connected to the midplane causes the router shelf to crash or to hang. Therefore, first ensure that the I/O controller card and the network processor card are installed properly and that the system software has initialized successfully. Then, if necessary, you can troubleshoot individual router shelf port adapters, dial shelf controller cards, modem cards, or trunk cards.

Troubleshooting the Router Shelf Processor Subsystem

The following sections describe troubleshooting procedures for the router shelf I/O controller card, the network processor card, and the port adapters.

Troubleshooting the I/O Controller Card

These procedures assume that the I/O controller card, network processor card, and router shelf itself are in the original factory configuration and that you have not made changes to your configuration file.

If the I/O controller card LEDs do not light as expected (refer to the section "Identifying Startup Problems" in this chapter), check the following items to help isolate the problem:

•Is the I/O power OK LED on?

•If yes, the router has booted successfully.

•If this LED remains OFF at system startup, either there is a problem with the power supply (it is damaged or not connected to the router midplane), or the network processor card or I/O controller card is not connected to the router midplane.

•Do all I/O controller card LEDs remain off when the system powers ON?

•If the LEDs remain off, first refer to the sections "Troubleshooting the Power Subsystems" and "Troubleshooting the Cooling Subsystems" in this chapter to ensure that both the fans and the power supply are functioning properly.

•If the power supply and fans appear operational but none of the I/O controller card LEDs are on, ensure that an improperly connected I/O controller card or port adapter has not hung the system. Tighten all captive installation screws, and then restart the system.

•With the power supply powered OFF, reseat the I/O controller card in its slot and restart the router.

•Is the I/O controller card enabled LED on?

•If yes, the system software has initialized successfully and the system is operational.

•If the enabled LED remains off, the system detected a processor hardware failure. (This LED should be on in normal operation.) Proceed to the section "Troubleshooting the Network Processor Card" to continue troubleshooting.

•Is the FE ENABLED LED on?

•If yes, the Fast Ethernet port on the I/O controller is initialized and enabled for operation.

•If the FE enabled LED remains off at system startup, there is probably a problem with the Fast Ethernet port on the I/O controller.

Troubleshooting the Network Processor Card

These procedures assume that the I/O controller card, network processor card, and router shelf are in the original factory configuration and that you have not made changes to your configuration file.

•Is the I/O controller card enabled LED on? (This LED should light in normal operation.)

•If the I/O controller card enabled LED does not light as expected and you have checked the I/O controller card, make sure that the power to the router is OFF, reseat the network processor card in its slot, and restart the router.

•If the enabled LED remains off, the system detected a processor hardware failure. Ensure that a partially seated port adapter is not causing the system to hang. Proceed to the following section, "Troubleshooting the Port Adapters."

Troubleshooting the Port Adapters

Check the following to help isolate a problem with the port adapters:

•Are all port adapter enabled LEDs on?

If yes, the system is operational.

•Are any port adapter enabled LEDs off?

•If the enabled LED on an individual port adapter is off, suspect that the port adapter has pulled away from the midplane. Reseat the port adapter in its slot and verify that the port adapter levers are in the locked position. (You do not have to power OFF system power when removing or replacing port adapters.) After the system reinitializes the interfaces, the enabled LED on the port adapter should light.

•If the enabled LED remains off, suspect a faulty port adapter. Replace the port adapter with a new one.

If the new port adapter enable LED fails to light, there may be a problem with the I/O controller card or the network processor card. Contact a service representative for instructions.

Troubleshooting the Dial Shelf Interconnect Port Adapter

Check the following to help isolate a problem with the interconnect port adapter:

•Is the dial shelf interconnect port adapter enabled LED on?

•If yes, the system is operational.

•If no, verify that the interconnect port adapter is seated properly and that the port adapter levers are in the locked position.

If the enabled LED remains off, reseat the port adapter and verify that the port adapter levers are in the locked position. Use the reload command to reload the system software.

•If the enabled LED remains off, see the section "Troubleshooting the Power Subsystems" to check your power connections.

•If power is supplied to the router shelf but the enabled LED remains off, the system has detected a hardware failure. You need to replace the interconnect port adapter.

•Is the link LED on?

The link LED indicates an active connection to the dial shelf. This LED lights when the dial shelf interconnect port adapter is receiving a carrier signal from the dial shelf.

•If the link LED remains off, check the dial shelf interconnect cable connection and tighten the jack screws at both ends of the cable.

•If the connection is good but the link LED remains off, proceed to the section "Troubleshooting the Dial Shelf Controller Card."

Troubleshooting the Dial Shelf Processor Subsystem

The following sections describe troubleshooting procedures for the DSC card, modem cards, VoIP card, and trunk cards.

Troubleshooting the Dial Shelf Controller Card

Check the following to help isolate a problem with the dial shelf controller card:

•Are the two power LEDs on?

Verify that the power LED and MBus LED light after the dial shelf controller card has been completely inserted into the dial shelf and the system is powered on.

•If both the MBus and power LEDs are on, the card should boot normally. During the boot sequence, the four alarm LEDs should momentarily flash and then turn off. In addition, the two four-character alphanumeric displays show status messages.

After the boot sequence completes, the alphanumeric display should read:
MSTR
If the boot sequence does not finish, contact a service representative for assistance.

•If either the power or MBus LED remains off, try removing and reinserting the card. If the problem persists, contact your service representative. You might need to replace the card.

If the problem persists with a new card installed, remove the dial shelf controller card from the dial shelf slot and examine the backplane for bent connector pins.

To inspect the backplane pins, first power OFF the system to avoid hazards caused by high voltages present on the backplane connectors. Next, remove cards in neighboring slots to allow an unimpeded view of the backplane connectors. Then, using a flashlight, verify that the backplane connectors are in good condition. If you discover bent pins, you need a new backplane. The backplane is an FRU. Contact your service representative to order a new backplane.

•You can also use the show command to diagnose a problem with the dial shelf controller card. Enter the following command:
router> enable
enter password <password>
router# show diag <type {shelf | slot}>
Ctrl-Z
Troubleshooting the Dial Shelf Cards

Troubleshooting information for CT1/ET1 or CT3 trunk cards, HMM or DMM modem cards, and the VoIP card is available in the Cisco AS5800 Universal Access Server Dial Shelf Card Guide that shipped with your system.

Troubleshooting Network Interfaces

For information about isolating problems with the network connections to your access server, refer to the publication Troubleshooting Internetworking Systems, which is available on the Cisco Documentation CD-ROM that shipped with your Cisco AS5800. For more information, see the "Related Documentation" section.

	Cisco AS5800 Access Server Hardware Installation Guide
	Troubleshooting the Cisco AS5800
Cisco AS5800 Access Server Hardware Installation Guide About This Guide Product Overview Preparing To Install Installing the Cisco AS5800 Access Server Powering On the Access Server Troubleshooting the Cisco AS5800 AS5800 Specifications Industry Standard Wiring Plans Cisco 5814 Dial Shelf Packaging Replacement Instructions	Download this chapter Troubleshooting the Cisco AS5800 Download the complete book PDF file of the entire guide (PDF - 6 MB) Feedback Table Of Contents Hardware Troubleshooting Problem Solving with Subsystems Problem Solving with Subsystems—Cisco 7206 Router Identifying Startup Problems Starting Up the Cisco 7206 Router Shelf Troubleshooting the Router Installation Starting Up the Cisco 5814 Dial Shelf Troubleshooting the Power Subsystems Router Shelf Power Subsystem Dial Shelf Power Subsystem Troubleshooting the Cooling Subsystems Router Shelf Cooling Subsystem Dial Shelf Cooling Subsystem Troubleshooting the Processor Subsystems Troubleshooting the Router Shelf Processor Subsystem Troubleshooting the I/O Controller Card Troubleshooting the Network Processor Card Troubleshooting the Port Adapters Troubleshooting the Dial Shelf Interconnect Port Adapter Troubleshooting the Dial Shelf Processor Subsystem Troubleshooting the Dial Shelf Controller Card Troubleshooting the Dial Shelf Cards Troubleshooting Network Interfaces Hardware Troubleshooting Your Cisco AS5800 was thoroughly tested before leaving the factory. If you encounter problems initializing system startup, use the information in this chapter to help isolate the cause. This chapter contains the following sections: •Problem Solving with Subsystems •Identifying Startup Problems •Troubleshooting the Power Subsystems •Troubleshooting the Cooling Subsystems •Troubleshooting the Processor Subsystems •Troubleshooting Network Interfaces The procedures in this chapter assume that you are troubleshooting the initial hardware configuration and system startup. If you have removed or replaced components or changed any default settings, the recommendations in this chapter might not apply. Review the safety warnings listed in the publication Cisco AS5800 Universal Access Server Regulatory Compliance and Safety Information before using the troubleshooting procedures in this chapter. If you are unable to resolve the problem, contact a customer service representative for assistance and further instructions. Provide the representative with the following information: •Date you received the system •Chassis serial number •Type of software and release number •Brief description of the problem •Brief explanation of the steps you have taken to isolate and resolve the problem •Maintenance agreement or warranty information Problem Solving with Subsystems The key to problem solving is isolating the problem to a specific subsystem. The first step in solving startup problems is to compare what the system is doing to what it should be doing. Try to isolate the problem to one of the following three router shelf or dial shelf subsystems: •Power •Cooling •Processor Start by checking the router shelf power components. If the problem does not seem to originate there, check the dial shelf power components. Router shelf power components include the following: •AC-input or DC-input power supplies •Cisco 7206 router shelf midplane •External-power cable connection Dial shelf power components include the following: •Power-entry modules (PEMs) •Filter module •AC-input power shelf •Cisco 5814 dial shelf backplane •External-power cable connection •External monitor system Next, check the router shelf cooling components. If the problem does not seem to originate there, check the dial shelf cooling components. Router shelf and dial shelf fans should all be operating whenever system power is on. •Router shelf cooling components include the fan tray. •Dial shelf cooling components include the blower assembly. Last, check the router shelf processor cards. If the problem does not seem to originate there, check the dial shelf processor cards. Most processor cards contain LEDs that indicate the status of the card. Observing these LEDs can help you isolate the problem to a particular card. (See Chapter 4, "Powering On the Cisco AS5800 and Observing Initial Startup Conditions," for LED descriptions and troubleshooting tips.) The system memory and management functions reside on the router shelf I/O controller card and the network processor card. The enabled LED on each port adapter lights when the port adapter is initialized. A port adapter that is partially plugged into the midplane can cause the system to hang and crash. The router shelf processor subsystem includes the following: •I/O controller card •Network processor card •Dial shelf interconnect port adapter •Other port adapters Dial shelf processor cards include the following: •Dial shelf controller card •Modem cards •Trunk cards The following sections will help you isolate a problem within one of these subsystems and direct you to the appropriate troubleshooting section. Problem Solving with Subsystems—Cisco 7206 Router The key to solving problems with the system is isolating the problem to a specific subsystem. The first step in solving startup problems is to compare what the system is doing to what it should be doing. Because a startup problem is usually caused by a single component, it is more efficient to isolate the problem to a subsystem rather than to troubleshoot each component in the system. For these troubleshooting procedures, consider the following subsystems: •Power subsystem—This subsystem comprises the power supplies, the external power cable, and the midplane. •Cooling subsystem—The chassis fan tray is the single component in this subsystem. The fans should be operating whenever system power is on. Replace the fan tray if you determine that a fan is not functioning properly. •Processor subsystem—This subsystem includes the I/O controller, network processing engine, and all port adapters. The system memory and management functions reside on the I/O controller and the network processing engine. The enabled LED on each port adapter indicates whether the port adapter is initialized. A port adapter that is partially installed in the midplane can cause the system to hang and crash. You can quickly verify that your new router shelf is properly installed and operative by observing the router LEDs as described in the "Starting the Cisco 7206" section on page 4-8. Figure 5-1 Troubleshooting Strategy for Startup Problems Identifying Startup Problems Hardware startup problems are commonly traced to cabling problems or incorrectly installed power supplies or cards. In rare cases, problems are caused by part failures. When you start up a Cisco AS5800 for the first time, you should observe the startup sequence, described in the following sections. This chapter describes the normal startup sequence for the router shelf and the dial shelf and the steps to take if the system does not perform that sequence as expected. In most cases, LEDs indicate system states in the startup sequence. By checking these LEDs, you can determine when and where the system failed. Use the following descriptions to isolate the problem to a subsystem, then proceed to the appropriate sections to try to resolve the problem. Note Although an overtemperature condition is unlikely at initial startup, the environmental monitoring functions are included in this chapter because these functions also monitor internal voltages. Starting Up the Cisco 7206 Router Shelf When you first power ON the router shelf power supply, the following should occur: •The power supply LEDs (OK) should light immediately when you set the power supply switch to the ON ( \| ) position and remain lit during normal system operation. If the power OK LEDs do not light, proceed to the section "Troubleshooting the Power Subsystems." •You should immediately hear the fans operating. If the fans do not operate, proceed to the section "Troubleshooting the Cooling Subsystems." •The I/O controller card LEDs should light in the following sequence. If they do not light as described, proceed to the section "Troubleshooting the Processor Subsystems." •The I/O power OK LED lights immediately and indicates that the I/O controller card is receiving DC power from the router midplane. This LED lights during a successful router boot and remains on during normal operation of the router. •The enabled LED lights after the I/O power OK LED and indicates that the network processor card and the I/O controller card are enabled for operation by the system; however, that the enabled LED lights does not mean that the Fast Ethernet port on the I/O controller card is functional or enabled. This LED lights during a successful router boot and remains lit during normal router operation. •The FE enabled LED lights after the enabled LED and indicates that the Fast Ethernet port on the I/O controller card is initialized and enabled for operation by the system. This LED lights during a successful router boot and remains lit during normal router operation. •The FE link LED lights only when the Fast Ethernet port on the I/O controller card is receiving a carrier signal from the network. (This LED remains off during normal router operation unless there is an incoming carrier signal and does not indicate startup problems.) •The slot 1 and slot 2 LEDs light after the enabled LED and indicate which PCMCIA slot is in use. During a successful router boot, these LEDs light and remain on if a Flash memory card is present in the slot. These LEDs also blink when either slot is being accessed by the system. These LEDs do not indicate startup problems. •The port-adapter-enabled LEDs each light when the network processor card completes its initialization of the port adapter for operation. The enabled LED indicates that the port adapter is receiving power and has been recognized by the network processor card; it does not indicate the state of the individual port adapter interfaces. If an enabled LED does not light, proceed to the section "Troubleshooting the Port Adapters," later in this chapter. •The dial shelf interconnect port adapter link status LED lights. The link status LED indicates an active connection to the dial shelf and lights when the interconnect port adapter is receiving a carrier signal from the dial shelf controller card. This LED should light and remain on when the access server is operating. If the interconnect port adapter link LED does not light, proceed to the section, "Troubleshooting the Dial Shelf Interconnect Port Adapter." •When all router shelf LEDs light to indicate that the system has booted successfully, the initial system banner should be displayed on the router shelf console screen. If it is not displayed, verify that the terminal is set correctly and is properly connected to the I/O controller card console port as described in the "Connecting to the Router Shelf Console and Auxiliary Ports" section. Troubleshooting the Router Installation Your Cisco 7206 routers went through extensive testing before leaving the factory. However, if you encounter problems starting the routers, use the information in the chapter "Troubleshooting the Installation" in the Cisco 7206 Installation and Configuration Guide (Document Number DOC-7206-ICG=) to help isolate the cause of the problems. Be sure to review the safety warnings listed in the publication Regulatory Compliance and Safety Information for the Cisco 7200 Series Routers (Part Number 78-3419-xx) that accompanied your Cisco 7206 before using troubleshooting procedures. If you are unable to easily solve the problem, contact a customer service representative for assistance and further instructions. Be prepared to provide the representative with the following information: •Date you received the router •Chassis serial number •Type of software and release number •Brief description of the problem you are having •Brief explanation of the steps you have taken to isolate and resolve the problem •Maintenance agreement or warranty information shows the general troubleshooting strategy for Cisco 7206 routers. Refer to this chart, as necessary, to isolate problems to a specific subsystem; then resolve the problem if possible. Starting Up the Cisco 5814 Dial Shelf When you first power ON the dial shelf, the following should occur: •Both PEM power LEDs should light immediately and remain on during normal system operation. If you are using the AC-input power shelf, both AC-input power supply LEDs should light. If the green power LEDs do not light, proceed to the section "Troubleshooting the Power Subsystem." •You should hear the fans operating in the blower assembly. If the fans do not operate, proceed to the section "Troubleshooting the Cooling Subsystems." •The LEDs on the dial shelf controller card, modem cards, and trunk cards should light as follows: •Dial shelf controller card—Powers ON first, followed by remaining logic cards. The PWR and MBus LEDs should light. The remaining four alarm LEDs should all briefly flash on and then turn off. If the MBus LED remains off but the power LED is on, there might be a problem with the MBus power supply on the card. Replace the card. •Trunk card—At initial power ON, all LEDs light while the system runs a series of diagnostics. After the system passes initial diagnostics, all LEDs go off. The power, host CPU, and framer CPU LEDs then light to indicate that the trunk card is operating properly. •Modem card—During normal operation, all five LEDs light at system power up. When the modem card CPU software image starts running, it shuts off the modem LED. The power, host CPU, and modem LEDs then light to indicate that the modem card has passed diagnostics and is operating properly. If the dial shelf processor component LEDs do not light as described, proceed to the section "Troubleshooting the Processor Subsystems." Troubleshooting the Power Subsystems The Cisco AS5800 is designed to minimize problems in the power subsystem. The power subsystem includes the router shelf power supplies, the dial shelf PEMs and filter module, and the AC-input power shelf (if used). The access server comes with two DC power supplies in both the router shelf and the dial shelf, which allow you to replace power supplies while the system is operating. However, if you are using an AC-configured system and you discover a faulty PEM, you must power off the system before performing a replacement. Caution If your system requires the AC-input power shelf for AC-to-DC conversion, you should schedule a time for system maintenance and replace the PEM at that time. Refer to the document AC-Input Power Shelf and Power Supply Installation and Replacement (Part Number 78-4659-xx) for installation and replacement instructions. The failure of a single power supply in the router shelf, the dial shelf, or the AC-input power shelf, if used, will not stop the access server from operating; however, to maintain power redundancy, both power supplies in each unit must be receiving power. Router Shelf Power Subsystem In the router shelf, check the following to help isolate a problem with the power subsystem: •Is the power OK LED on? •If yes, the power source is good and the power supply is functional. •If no, verify that the power cable is connected at both ends. If the power OK LED remains off and the power switch is set correctly, suspect the power source or the power cable. Power OFF the system, connect the power cable to another power source, if available, then power ON the system. If the LED lights, the problem is the first power source. •If the power OK LED does not light after you connect the power supply to a new power source, replace the power cord, then power ON the system. If the AC power (or input power) LED lights, return the first power cable for replacement. •If the LED still does not light when the power supply is connected to a different power source with a new power cable, the power supply is probably faulty. If a second power supply is available, install it in the other power supply bay and contact a service representative for further instructions. •Is the power OK LED on for the second (redundant) power supply? •If yes, proceed to the section "Troubleshooting the Cooling Subsystems." •If no, repeat each of the above procedures for the second power supply. If you are unable to resolve the problem or you determine that either a power supply or chassis connector is faulty, contact a service representative for instructions. Dial Shelf Power Subsystem In the dial shelf, check the following to help isolate a problem with the power subsystem: •Is the first PEM power OK LED on? •If yes, the power source is good and the PEM is functional. Verify that the second PEM power LED is on for power redundancy. •If no, verify that the power cable is connected at both ends and check that you have wired the DC-input connections correctly. If the DC-input connection is wired correctly, the miswire LED remains off. If the miswire LED is on, the two DC conductors entering the PEM terminal block are reversed. Power OFF the power at your power source and reverse the connections. •If neither the power nor the miswire LED is on, check the voltage at the input terminal block. If the voltage reading falls between -40 VDC and -60 VDC, replace the PEM. •If you are using the AC-input power shelf, are both power OK LEDs on? •If yes, the power source is good and both power supplies are functional. •If no, verify that the associated AC power cord is plugged in and the correct AC voltage (200 to 240 VAC) is present at the source. •Verify that the fault LED is off. If a power supply fault LED lights, the power shelf has detected an internal fault; the power supply is defective. You need to replace the power supply. •Verify that the overtemperature LED is off. If a power supply overtemperature LED lights, verify that the ambient temperature is below 120°F (50°C) and the air intake is not blocked. If the condition persists, replace the power supply. •Verify that the current limit LED is off. If a power supply current limit LED lights, verify that the DC bus terminals on the AC-input power shelf rear are not short-circuited and a supported dial shelf configuration is being used. Note The dial shelf power subsystem supports a maximum configuration of 10 modem cards. Troubleshooting the Cooling Subsystems The Cisco AS5800 is designed to minimize problems in the cooling subsystem. The access server can operate in temperatures of up to 120°F (50°C) for a maximum of 72 hours. The access server also contains rendundant fans in both the router shelf fan tray and the dial shelf blower assembly, which allow the system to continue operating despite a single fan failure. Both the router shelf fan tray and the dial shelf blower assembly can be removed and replaced while the system is operating, provided the procedure does not exceed 1 min. Router Shelf Cooling Subsystem In the router shelf, check the following to help isolate a problem with the cooling subsystem: •When you start up the system, do the router shelf fans operate? To determine whether the router shelf fans are operating, listen for them. In noisy environments, place your hand on the left side of the router shelf (when viewing the router shelf from the front) to feel for air being forced out the vents. •If yes, power to the fans is good. •If no, there is a problem with the router shelf fans or there is a problem with the power to the fan tray. If you determine that the power supply is functioning normally but a fan is faulty, you can replace the fan tray without powering off the access server. Contact a customer service representative if you need assistance. For problems with the system power, refer to the section "Troubleshooting the Power Subsystem" in this chapter. Dial Shelf Cooling Subsystem In the dial shelf, check the following to help isolate a problem with the cooling system: •Do the blower assembly fans operate? To determine whether the dial shelf fans are operating, listen for them. In noisy environments, place your hand under the exhaust vents on the back of the dial shelf blower assembly to feel for air being forced out the vents. •If no, there is a problem with the dial shelf fans or there is a problem with the power to the blower assembly. If you determine that the power supply is functioning normally but a fan is faulty, you can replace the blower assembly without powering off the access server. Refer to the blower assembly removal instructions in ","and the reinsertion instructions in "." For problems with the system power, refer to the section "Troubleshooting the Power Subsystem" in this chapter. •Is the green power LED on the blower assembly front panel on? If no, verify that the blower assembly is fully installed in the dial shelf, the connector is firmly connected to the backplane, and the captive screws are tightened adequately. •Is the yellow fault LED on the blower assembly front panel on? If yes, the newly installed blower assembly might be faulty or the chassis connector might be damaged. Shut off system power, remove the blower assembly from the dial shelf, and check the connectors. If the connectors are in good condition, reinstall the blower assembly in the dial shelf and power ON the system. If the fault LED is still on, assume that the blower assembly is faulty. Install another blower assembly and return the faulty blower assembly to the factory. •The following error message, if displayed on the router shelf console, indicates that the system has detected an overtemperature condition or out-of-tolerance power somewhere in the system. Queued messages: %ENVM-1-SHUTDOWN: Environmental Monitor initiated shutdown If an environmental shutdown results from an out-of-tolerance power condition, the router shelf power OK LED remains off and the system shuts down. (Refer to the section "Troubleshooting the Power Subsystems" in this chapter.) Although an overtemperature condition is unlikely at initial startup, ensure that heated exhaust air from other equipment is not entering the router's inlet vent and that there is sufficient clearance around the sides of the chassis to allow cooling air to flow. Refer to the "Preventive Site Configuration: Maintaining Normal Operation" section" for preventive site configurations. The above message could also indicate a faulty component or temperature sensor. Before the system shuts down, use the show environment or show environment table command to display the internal chassis environment. Refer to the Cisco 7206 Installation and Configuration Guide for a description of show commands. Troubleshooting the Processor Subsystems The processor subsystem consists of the I/O controller card, network processor card, and all port adapters in the router shelf, and the dial shelf controller card, modem cards, and trunk cards in the dial shelf. The following sections contain specific troubleshooting information for each of these components. The router shelf I/O controller card and network processor card are required system components. The system cannot operate unless the I/O controller card and network processor card are installed properly; however, the system can operate without any port adapters installed as long as none are in partial contact with the midplane pins. A port adapter that is partially connected to the midplane causes the router shelf to crash or to hang. Therefore, first ensure that the I/O controller card and the network processor card are installed properly and that the system software has initialized successfully. Then, if necessary, you can troubleshoot individual router shelf port adapters, dial shelf controller cards, modem cards, or trunk cards. Troubleshooting the Router Shelf Processor Subsystem The following sections describe troubleshooting procedures for the router shelf I/O controller card, the network processor card, and the port adapters. Troubleshooting the I/O Controller Card These procedures assume that the I/O controller card, network processor card, and router shelf itself are in the original factory configuration and that you have not made changes to your configuration file. If the I/O controller card LEDs do not light as expected (refer to the section "Identifying Startup Problems" in this chapter), check the following items to help isolate the problem: •Is the I/O power OK LED on? •If yes, the router has booted successfully. •If this LED remains OFF at system startup, either there is a problem with the power supply (it is damaged or not connected to the router midplane), or the network processor card or I/O controller card is not connected to the router midplane. •Do all I/O controller card LEDs remain off when the system powers ON? •If the LEDs remain off, first refer to the sections "Troubleshooting the Power Subsystems" and "Troubleshooting the Cooling Subsystems" in this chapter to ensure that both the fans and the power supply are functioning properly. •If the power supply and fans appear operational but none of the I/O controller card LEDs are on, ensure that an improperly connected I/O controller card or port adapter has not hung the system. Tighten all captive installation screws, and then restart the system. •With the power supply powered OFF, reseat the I/O controller card in its slot and restart the router. •Is the I/O controller card enabled LED on? •If yes, the system software has initialized successfully and the system is operational. •If the enabled LED remains off, the system detected a processor hardware failure. (This LED should be on in normal operation.) Proceed to the section "Troubleshooting the Network Processor Card" to continue troubleshooting. •Is the FE ENABLED LED on? •If yes, the Fast Ethernet port on the I/O controller is initialized and enabled for operation. •If the FE enabled LED remains off at system startup, there is probably a problem with the Fast Ethernet port on the I/O controller. Troubleshooting the Network Processor Card These procedures assume that the I/O controller card, network processor card, and router shelf are in the original factory configuration and that you have not made changes to your configuration file. •Is the I/O controller card enabled LED on? (This LED should light in normal operation.) •If the I/O controller card enabled LED does not light as expected and you have checked the I/O controller card, make sure that the power to the router is OFF, reseat the network processor card in its slot, and restart the router. •If the enabled LED remains off, the system detected a processor hardware failure. Ensure that a partially seated port adapter is not causing the system to hang. Proceed to the following section, "Troubleshooting the Port Adapters." Troubleshooting the Port Adapters Check the following to help isolate a problem with the port adapters: •Are all port adapter enabled LEDs on? If yes, the system is operational. •Are any port adapter enabled LEDs off? •If the enabled LED on an individual port adapter is off, suspect that the port adapter has pulled away from the midplane. Reseat the port adapter in its slot and verify that the port adapter levers are in the locked position. (You do not have to power OFF system power when removing or replacing port adapters.) After the system reinitializes the interfaces, the enabled LED on the port adapter should light. •If the enabled LED remains off, suspect a faulty port adapter. Replace the port adapter with a new one. If the new port adapter enable LED fails to light, there may be a problem with the I/O controller card or the network processor card. Contact a service representative for instructions. Troubleshooting the Dial Shelf Interconnect Port Adapter Check the following to help isolate a problem with the interconnect port adapter: •Is the dial shelf interconnect port adapter enabled LED on? •If yes, the system is operational. •If no, verify that the interconnect port adapter is seated properly and that the port adapter levers are in the locked position. If the enabled LED remains off, reseat the port adapter and verify that the port adapter levers are in the locked position. Use the reload command to reload the system software. •If the enabled LED remains off, see the section "Troubleshooting the Power Subsystems" to check your power connections. •If power is supplied to the router shelf but the enabled LED remains off, the system has detected a hardware failure. You need to replace the interconnect port adapter. •Is the link LED on? The link LED indicates an active connection to the dial shelf. This LED lights when the dial shelf interconnect port adapter is receiving a carrier signal from the dial shelf. •If the link LED remains off, check the dial shelf interconnect cable connection and tighten the jack screws at both ends of the cable. •If the connection is good but the link LED remains off, proceed to the section "Troubleshooting the Dial Shelf Controller Card." Troubleshooting the Dial Shelf Processor Subsystem The following sections describe troubleshooting procedures for the DSC card, modem cards, VoIP card, and trunk cards. Troubleshooting the Dial Shelf Controller Card Check the following to help isolate a problem with the dial shelf controller card: •Are the two power LEDs on? Verify that the power LED and MBus LED light after the dial shelf controller card has been completely inserted into the dial shelf and the system is powered on. •If both the MBus and power LEDs are on, the card should boot normally. During the boot sequence, the four alarm LEDs should momentarily flash and then turn off. In addition, the two four-character alphanumeric displays show status messages. After the boot sequence completes, the alphanumeric display should read: MSTR If the boot sequence does not finish, contact a service representative for assistance. •If either the power or MBus LED remains off, try removing and reinserting the card. If the problem persists, contact your service representative. You might need to replace the card. If the problem persists with a new card installed, remove the dial shelf controller card from the dial shelf slot and examine the backplane for bent connector pins. To inspect the backplane pins, first power OFF the system to avoid hazards caused by high voltages present on the backplane connectors. Next, remove cards in neighboring slots to allow an unimpeded view of the backplane connectors. Then, using a flashlight, verify that the backplane connectors are in good condition. If you discover bent pins, you need a new backplane. The backplane is an FRU. Contact your service representative to order a new backplane. •You can also use the show command to diagnose a problem with the dial shelf controller card. Enter the following command: router> enable enter password <password> router# show diag <type {shelf \| slot}> Ctrl-Z Troubleshooting the Dial Shelf Cards Troubleshooting information for CT1/ET1 or CT3 trunk cards, HMM or DMM modem cards, and the VoIP card is available in the Cisco AS5800 Universal Access Server Dial Shelf Card Guide that shipped with your system. Troubleshooting Network Interfaces For information about isolating problems with the network connections to your access server, refer to the publication Troubleshooting Internetworking Systems, which is available on the Cisco Documentation CD-ROM that shipped with your Cisco AS5800. For more information, see the "Related Documentation" section.
	Terms & Conditions \| Privacy Statement \| Cookie Policy \| Trademarks