High Availability and Redundancy in Catalyst 6500 Series Switches

Introduction

The Cisco Catalyst 6500/6000 series switches have an intelligent power management system that grants or denies power to various system components on the basis of power availability in the system. This document discusses the total power that is available with the currently shipped power supplies and the amount that is drawn from each line card. If you follow these guidelines, you prevent an oversubscription of the power budget, which can cause the modules to power down and other unexpected results. This document helps you understand the power management system in the Catalyst 6500/6000 series switches.

Prerequisites

Requirements

There are no specific requirements for this document.

Components Used

The information in this document is based on the Catalyst 6500/6000 series switches.

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.

Conventions

Refer to Cisco Technical Tips Conventions for more information on document conventions.

Power Redundancy

Catalyst 6500/6000 series modules have different power requirements. Certain switch configurations can require more power than a single power supply can provide, which depends on the wattage of the power supply. Although the power management feature allows you to power all the installed modules with two power supplies, redundancy is not supported in this configuration.

With redundancy enabled, if you power up the system with two power supplies of unequal wattage, both power supplies come on line with a corresponding syslog message. The message indicates that the lower-wattage power supply will be disabled. If the active power supply fails, the lower-wattage power supply that was disabled comes on line. If necessary, certain modules can be powered down in order to accommodate the lower-wattage power supply. For more information on the effects of power supply configuration changes, see the Effects of Power Supply Configuration Changes table in this section.

Note: The Catalyst 6500/6000 series switches allow you to mix AC-input and DC-input power supplies in the same chassis.

Effects of Power Supply Configuration Changes

Configuration Change	Effect
Redundant to nonredundant	System log and syslog messages are generated. The system power is increased to the combined power capability of both supplies. The modules that are marked as power-deny in the Status field of the show module command output are brought up, if there is sufficient power.
Nonredundant to redundant	System log and syslog messages are generated. The system power is the power capability of the larger-wattage supply. If there is not enough power for all previously powered-up modules, some modules are powered down and marked as power-deny in the Status field of the show module command output.
Insertion of equal-wattage power supply with redundancy enabled	System log and syslog messages are generated. The system power equals the power capability of one supply. There is no change in the module status because the power capability is unchanged.
Insertion of equal-wattage power supply with redundancy disabled	System log and syslog messages are generated. The system power is the combined power capability of both supplies. The modules that are marked as power-deny in the Status field of the show module command output are brought up, if there is sufficient power.
Insertion of higher-wattage power supply with redundancy enabled	System log and syslog messages are generated. The system disables the lower-wattage power supply. The higher-wattage supply powers the system.
Insertion of lower-wattage power supply with redundancy enabled	System log and syslog messages are generated. The system disables the lower-wattage power supply. The higher-wattage supply powers the system.
Insertion of higher- or lower-wattage power supply with redundancy disabled	System log and syslog messages are generated. The system power is increased to the combined power capability of both supplies. The modules that are marked as power-deny in the Status field of the show module command output are brought up, if there is sufficient power.
Removal of power supply with redundancy enabled	System log and syslog messages are generated. If the power supplies are of equal wattage, there is no change in the module status because the power capability is unchanged. If the power supplies are of unequal wattage and the lower-wattage supply is removed, there is no change in the module status. If the power supplies are of unequal wattage and the higher-wattage supply is removed, and if there is not enough power for all previously powered-up modules, some modules are powered down. The powered-down modules are marked as power-deny in the Status field of the show module command output.
Removal of power supply with redundancy disabled	System log and syslog messages are generated. The system power is decreased to the power capability of one supply. If there is not enough power for all previously powered-up modules, some modules are powered down and marked as power-deny in the Status field of the show module command output.
Bootup of system with power supplies that have different wattage installed and have redundancy enabled	System log and syslog messages are generated. The lower-wattage supply is disabled.
Bootup of system with power supplies that have equal or different wattage installed and have redundancy disabled	System log and syslog messages are generated. The system power equals the combined power capability of both supplies. The system powers up as many modules as the combined capacity allows.

In systems that are equipped with two power supplies, if one power supply fails and the other power supply cannot fully power all of the installed modules, system power management shuts down devices in this order:

1. Power over Ethernet (PoE) devices

   The system powers down PoE devices in descending order, starting with the highest numbered port on the module in the highest numbered slot.

2. Modules

   If additional power conservation is needed, the system powers down modules in descending order, starting with the highest numbered slot. Slots that contain Supervisor Engines or switch fabric modules are bypassed and not powered down.

This shut down order is fixed and cannot be changed.

For details on supported power supply configurations for each chassis, refer to Product Overview (Catalyst 6500 Series Installation Guide).

For details on redundant and nonredundant power configurations, refer to the Enabling or Disabling Power Redundancy section of Administering the Switch.

Power Supplies

Power Supply Rating	AC Input Model Product Number	DC Input Model Product Number
950 W	PWR-950-AC	PWR-950-DC
1000 W	WS-CAC-1000W	-
1300 W	WS-CAC-1300W	WS-CDC-1300W
1400 W	PWR-1400-AC	-
2500 W	WS-CAC-2500W	WS-CDC-2500W
2700 W	PWR-2700-AC/4	PWR-2700-DC/4
3000 W	WS-CAC-3000W	-
4000 W	WS-CAC-4000W-US1 WS-CAC-4000W-INT	PWR-4000-DC
6000 W	WS-CAC-6000W	-
8000 W	WS-CAC-8700W-E	-

Refer to Preparing for Installation for more information on circuit type and power cord requirements.

Refer to the Table 1-11 Power Supply Front Panel LEDs section of Product Overview (Catalyst 6500 Series Installation Guide) for more information on power supply LEDs.

The chassis does draw some power for fans and (some) bus termination, but this power is already factored in the power budget. Notice that the 1300 W power supply can supply 27.46 A. This is the value of the 42 volt (V) supply to which reference is made.

Here is an example:

27.46A * 42V = 1153W + 146W (for the chassis) = 1300W

This applies to all power supplies. The number that is published for the power supply is only for the power that the Supervisor Engines/line cards will consume.

There is no Catalyst 6500/6000 power supply with output that is rated at 1800 W. The 1800 W number relates to the 1300 W AC power supply. The 1800 W number appears on the front panel silkscreen (of some versions of this unit) and indicates the input power rating (or maximum system power dissipation) of the unit.

Note: The silkscreen marking on the unit has confused many people. Cisco has discontinued this type of marking on power supplies.

There is no issue with reliability when you run the power supply to the 27.46 A maximum limit that the power management software sets for the 1300 W power supply. This 27.46 A maximum is 75 to 80 percent of the theoretical maximum capacity of the supply at a 40-degrees Celsius (C) ambient. This derating is typical of power supplies and is there to make sure that there is plenty of margin. This increases the long-term reliability of the supply. Also, all the power consumption values for each of the cards are generated for a worst-case traffic configuration (about 100 percent). For example, power consumption for a Gigabit module includes all Gigabit Interface Converters (GBICs) that are installed. The actual usage is less during typical operation.

Supervisor Engines/Line Cards

Some initial-production Supervisor Engine 1 units were programmed to 4.30 A. The 5.2(1) software overrides the Supervisor Engine EEPROM (SEEPROM) value and uses a default of 3.00 A. The 5.2(2) software overrides the SEEPROM value and uses a default of 1.70 A.

Because a standby Supervisor Engine card is always immediately powered on upon insertion, enough power must be reserved in the redundant Supervisor Engine slot in order to accommodate a Supervisor Engine even if no card is installed in the slot. There are four cases when you do and do not have a redundant Supervisor Engine:

• No card in slot 2—1.7 A are allocated for the possible insertion of a Supervisor Engine.

  Note: If the primary Supervisor Engine has a Multilayer Switch Feature Card (MSFC)/Policy Feature Card (PFC), 3.30 A are reserved.

• Supervisor Engine in slot 2—The reserved 1.7 A are allocated.

  Note: If the Supervisor Engine has an MSFC/PFC, 3.30 A are reserved.

• A line card with less than 1.7 A in slot 2—The 1.7 A Supervisor Engine number is allocated.

  Note: There is no card that is currently available that consumes less than 1.7 A.

  Note: If the Supervisor Engine has an MSFC/PFC, 3.30 A are reserved.

• A line card with more than 1.7 A in slot 2—The actual card value from the SEEPROM is allocated.

  Note: If the Supervisor Engine has an MSFC/PFC, 3.30 A are reserved.

Some initial-production WS-X6408-GBIC units were incorrectly programmed to 1.5 A.

Refer to the Table 14-2 Module Power Requirements section of Administering the Switch for more information on module power requirements.

Use of the CLI to Power Modules Up or Down

You can issue one of these commands in order to power down a properly working module from the command-line interface (CLI):

• Catalyst OS (CatOS)— set module power down module_number

• Cisco IOS® Software— no power enable module slot

The module is marked as power-down in the Status field of the show module command output. In order to verify if adequate power is available in the system in order to turn the power on for a module that was previously powered down, issue one of these commands:

• CatOS— set module power up module_number

• Cisco IOS Software— power enable module slot

If there is not enough power available, the module status changes from power-down to power-deny.

show Commands

• show environment (CatOS)—This command provides the diagnostic result of switch components such as the power supply, clock, and fan.

  Cat6kCatOS show environment       
  Environmental Status (. = Pass, F = Fail, U = Unknown, N = Not Present)
  PS1: .     PS2: N     PS1 Fan: .     PS2 Fan: N
  Chassis-Ser-EEPROM: .     Fan: .
  Clock(A/B): A         Clock A: .     Clock B: .
  VTT1: .    VTT2: .    VTT3: .

• show environment status (Cisco IOS Software)—This command is similar to the show environment command in CatOS.

  Cat6kIOS#show environment status
  backplane: 
    operating clock count: 2
    operating VTT count: 3
  fan-tray: 
    fantray fan operation sensor: OK
  VTT 1: 
    VTT 1 OK: OK
    VTT 1 outlet temperature: 32C
  VTT 2: 
    VTT 2 OK: OK
    VTT 2 outlet temperature: 34C
  VTT 3: 
    VTT 3 OK: OK
    VTT 3 outlet temperature: 36C
  clock 1: 
    clock 1 OK: OK, clock 1 clock-inuse: in-use
  clock 2: 
    clock 2 OK: OK, clock 2 clock-inuse: not-in-use
  power-supply 1: 
    power-supply 1 fan-fail: OK
    power-supply 1 power-output-fail: OK
  module 1: 
    module 1 power-output-fail: OK
    module 1 outlet temperature: 30C
    module 1 device-2 temperature: 35C
    RP 1 outlet temperature: 36C
    RP 1 inlet temperature: 37C
    EARL 1 outlet temperature: 29C
    EARL 1 inlet temperature: 30C
  module 3: 
    module 3 power-output-fail: OK
    module 3 outlet temperature: 31C
    module 3 inlet temperature: 27C
  module 5: 
    module 5 power-output-fail: OK
    module 5 outlet temperature: 42C
    module 5 inlet temperature: 29C
    EARL 5 outlet temperature: 40C
    EARL 5 inlet temperature: 32C
  module 6: 
    module 6 power-output-fail: OK
    module 6 outlet temperature: 44C
    module 6 inlet temperature: 36C

• show environment power (CatOS)—This command provides details about the system power condition and the available power.

  Cat6kCatOS show environment power
  PS1 Capacity: 1153.32 Watts (27.46 Amps @42V)
  PS2 Capacity: none
  PS Configuration : PS1 and PS2 in Redundant Configuration.
  Total Power Available: 1153.32 Watts (27.46 Amps @42V)
  Total Power Available for Line Card Usage: 1153.32 Watts (27.46 Amps @42V)
  Total Power Drawn From the System: 377.58 Watts ( 8.99 Amps @42V)
  Remaining Power in the System: 775.74 Watts (18.47 Amps @42V)
  Default Inline Power allocation per port: 7.00 Watts (0.16 Amps @42V)
  
  Slot power Requirement/Usage :
  
  Slot Card Type           PowerRequested PowerAllocated CardStatus
  Watts   A @42V Watts   A @42V
  ---- ------------------- ------- ------ ------- ------ ----------
  1    WS-X6K-SUP1A-2GE    138.60   3.30  138.60   3.30  ok
  2                          0.00   0.00  138.60   3.30  none
  6    WS-X6348-RJ-45      100.38   2.39  100.38   2.39  OK

  Note: This example show environment power command output uses a Supervisor Engine 1 with PFC and MSFC.

• show power (Cisco IOS Software)—This command is similar to the show environment power command in CatOS.

  Redundancy is disabled:

  Cat6kIOS#show power
  system power redundancy mode = combined
  system power total = 55.500A
  system power used = 22.690A
  system power available = 32.810A
  FRU-type       #    current   admin state oper
  power-supply   1    55.500A   on          on                            
  module         1    4.300A    on          on                            
  module         2    4.300A    on          on                            
  module         3    5.500A    on          on                            
  module         4    5.500A    on          on                            
  module         5    3.090A    on          on                            
  module         6    5.400A    off         off (admin request)

  Redundancy is enabled:

  C6500-1> show power
  system power redundancy mode = redundant
  system power total =     1153.32 Watts (27.46 Amps @ 42V)
  system power used =       674.52 Watts (16.06 Amps @ 42V)
  system power available =  478.80 Watts (11.40 Amps @ 42V)
                          Power-Capacity PS-Fan Output Oper
  PS   Type               Watts   A @42V Status Status State
  ---- ------------------ ------- ------ ------ ------ -----
  1    WS-CAC-1300W       1153.32 27.46  OK     OK     on
  2    WS-CAC-1300W       1153.32 27.46  OK     OK     on
  

  Redundancy is enabled, but one of the power ssupplies does not work:

  C6500-2# show power
  system power redundancy mode = redundant
  system power redundancy operationally = non-redundant
  system power total =     3795.12 Watts (90.36 Amps @ 42V)
  system power used =      1786.68 Watts (42.54 Amps @ 42V)
  system power available = 2008.44 Watts (47.82 Amps @ 42V)
                          Power-Capacity PS-Fan Output Oper
  PS   Type               Watts   A @42V Status Status State
  ---- ------------------ ------- ------ ------ ------ -----
  1    WS-CAC-4000W-US    3795.12 90.36  OK     OK     on
  2    WS-CAC-4000W-US    3795.12 90.36  -      -      off

Related Information

• Removal and Replacement Procedures
• Catalyst 6000 and 6500 Series Multilayer Switch Module Installation and Configuration Note
• LAN Product Support Pages
• LAN Switching Support Page
• Technical Support & Documentation - Cisco Systems