Implementation Options

This chapter contains the following sections:

Module I/O Description

The following table provides details on the I/O signals.

Table 1. I/O Signals

IO Name

Description

Direction

I/O Standard

Notes

CLK_156M25_*

156.25MHz reference clock

Out

LDVS

If unused terminate with 100 ohm resistor

I2C_*_SDA

I2C data

In/Out

3.3V Open-drain

Pullup on Cisco card

I2C_*_SCL

I2C clock

In/Out

3.3V Open-drain

Pullup on Cisco card

I2C_MUX_RST_L

I2C mux reset

Out

3.3V

Resets the I2C mux

P3_3V_RTC

Backup power for real time clock

In

3.3V

—

CNS_RJ45_TXD_L

RS-232 console

Out

RS-232 Compliant

RS-232 console port

CNS_RJ45_RTS_L

RS-232 console

Out

RS-232 Compliant

RS-232 console port

CNS_RJ45_RXD_L

RS-232 console

In

RS-232 Compliant

RS-232 console port

CNS_RJ45_CTS_L

RS-232 console

In

RS-232 Compliant

RS-232 console port

USB_CONSOLE_[P/N]

USB console

Bi

USB 2.0 Compliant

USB 2.0 console port

LED_SH_DIN

Shift chain data

Out

3.3V

LED_SH_CLK

Shift chain clock

Out

3.3V

LED_SH_EN_L

Shift chain enable low

Out

3.3V

Pullup on Cisco card

LED_SH_LATCH

Shift chain latch

Out

3.3V

PUSH_BUTTON_L

Push button

In

3.3V

Pullup on Cisco card

TE1_[1-10]_TXDATA_[P/N]

XFI 10G tx data

Out

LVDS

—

TE1_[1-10]_RXDATA_[P/N]

XFI 10G rx data

In

LVDS

—

SFP_TE1_[1-10]_PRES_L

SFP present low

In

3.3V

Signal indicating the presence of the SFP

USB1_[1-2]_FAULT_L

USB power fault detected

In

3.3V

Pullup on Cisco card

USB1_[1-2]_PWR_EN

Turn on the USB 5V power

Out

3.3V

Pullup on Cisco card

USB1_[1-2]_D_[P/N]

USB for SD or USB Type A

Bi

USB 2.0 Compliant

—

RESET_HOLD_OFF_L

Keeps the Cisco card in reset

In

3.3V

This signal holds the Cisco card in reset until the integrator card is ready.

DC[A-B]_PWR_GOOD

DC power is good or either A or B supply

In

3.3V

High indicates the DC input is good. Pullup on Cisco card.

DYING_GASP_L

Indicates the power supplies are starting to fail

In

3.3V

The supplies must hold up the power until the dying gasp messages can be sent out.

TEMP_SENSOR_[P/N]

Transistor temperature sensor on the Cisco card

Out

N/A

These signals can be used for the integrator to monitor the temperature of the Cisco card.

MGMT_PHY_RXDATA

SGMII signal for management PHY

Out

LVDS

—

MGMT_PHY_TXDATA

SGMII signal for management PHY

In

LVDS

—

MGMT_PHY_RST_L

Management phy reset

Out

3.3V

Pull low if no management phy is populated

MDIO_MGMT_PHY

Management phy MDIO signal

Bi

3.3V

MDIO bus, MDIO_MGMT_PHY needs a pullup

MDC_MGMT_PHY

Management phy MDC signal

Out

3.3V

—

Note
In the above table, LVDS stands for Low-Voltage Differential Signaling. For further information see the TIA/EIA-644 technical standard.

Block Diagrams

The system integrator can find block diagrams that represents how the ESS board connects into their system located here:

ESS9300 Block Diagram

LED Definitions

LED functionality is provided by a dedicated controller for driving an LED shift chain for driving the LEDs on the integrator board. You can select any combination of LEDs to implement. You are not required to implement all of the LEDs but must implement the shift chain up to the last LED needed.

LED

Color

Description

System

Off

Flashing Green

Solid Green

Flashing Yellow

Yellow

System is not powered on.

Power on tests in progress.

System is operating normally.

System is receiving power but is not functioning properly.

System fault detected.

DC-A/B

Off

Solid Green

Solid Red

Power is not present on the circuit, or the system is not powered up.

Power is present on the associated circuit.

Power is not present on the associated circuit, and the switch is configured for dual-input power.

Alarm Out

Off

Solid Green

Flashing Red

Solid Red

Alarm Out is not configured.

Alarm Out is configured, no alarm detected.

Switch has detected a major alarm.

Switch has detected a minor alarm.

Alarm In

Off

Solid Green

Flashing Red

Solid Red

Alarm In is not configured.

Alarm In is configured, no alarm detected.

Switch has detected a major alarm.

Switch has detected a minor alarm.

Under Temperature

Red

The system is under temperature and is warming up.

Port

Off

Solid Green

Flashing Green

Alternating Green/Yellow

Solid Yellow

No link or the port was administratively shut down.

Link is present, no activity.

Link is healthy, with activity.

Link faulty or an error.

Port is disabled.

Console

Off

Solid Green

Flashing Green

USB cable or Blue-tooth dongle not connected.

USB console is active.

Blue-tooth dongle is active.

Zeroize

Off

Flashing Green

Solid Yellow

Solid Green

Normal operation.

Zeroization procedure has been initiated.

Zeroization procedure has completed; switch is about to reboot.

Zeroization procedure has completed.

Board LED Register Bits

The following table provides a listing of the Board LED register bits for the system integrator.

Table 3. LED Shift Chain

GPIO Position

Carrier

0 (First bit out of the Cisco card)

TE1/12 Green

1

TE1/12 Yellow

2

TE1/11 Green

3

TE1/11 Yellow

4

TE1/10 Green

5

TE1/10 Yellow

6

TE1/9 Green

7

TE1/9 Yellow

8

TE1/8 Green

9

TE1/8 Yellow

10

TE1/7 Green

11

TE1/7 Yellow

12

TE1/6 Green

13

TE1/6 Yellow

14

TE1/5 Green

15

TE1/5 Yellow

16

TE1/4 Green

17

TE1/4 Yellow

18

TE1/3 Green

19

TE1/3 Yellow

20

TE1/2 Green

21

TE1/2 Yellow

22

TE1/1 Green

23

TE1/1 Yellow

24

USB console Green

25

Under temperature Red

26

Alarm IN 2 Green

27

Alarm IN 2 Red

28

Alarm IN 1 Green

29

Alarm IN 1 Red

30

Alarm Out Green

31

Alarm Out Red

32

DC-B Green

33

DC-B Red

34

DC-A Green

35

DC-A Red

36

ZEROIZE Green

37

ZEROIZE Yellow

38

SYSTEM Green

39 (Last bit out of shift chain)

SYSTEM Yellow

Module Voltage Test Points

The following figure shows voltage test points on the board with descriptions in the following table.


Note
Red lines are Positive, Black lines are Ground.
Figure 1. Voltage Test Points

Test Point

Location

Location Color

Voltage

1

C311

Yellow

0.60V

2

C370

Orange

0.80V

3

C30_M1

Blue

0.85V

4

C373

Purple

0.90V

5

Surface trace

Pink

1.20V

6

Surface trace

Light Yellow

1.20V (VDDH)

7

Surface trace

Light Blue

1.80V

8

Surface trace

Dark Green

2.50V

9

Surface trace

Brown

3.30V

10

C16_M1

Red

5.0V

Mechanical and Environmental Testing

The tests listed in the following tables were successfully executed on the Cisco ESS9300 using Cisco passive cooling design. These tests used a representative enclosure that conforms to the mounting and thermal mechanisms. Because this type of testing is highly dependent on factors such as the test enclosure design, the thermal solution, the front panel connectors, and the mounting, the following test results should only be used as a reference.

Table 4. Temperature

High and Low Temperature Cycle Stress

(Operational)

High Temperature: 74°C (165°F)

Low Temperature: -40°C (-40°F)

Reference: MIL-STD-810F, Method 501.4, Procedure II and Method 502.4, Procedure II; SAE J1455 (Rev AUG94), Section 4.1.3

Thermal Shock

(Non-Operational)

High Temperature: 85°C (185 °F)

Low Temperature: -40°C (-40 °F)

Cycle: 2 hours high temperature, 2 hours low temperature

Test Period: 2 hour pre-soak at low temperature, followed by 5 cycles

Repetition: 5 test periods

Reference: MIL-STD-810F, Method 503.4; SAE J1455 (Rev AUG94), Section 4.1.3.2

High Temperature Component Thermal Test

(Operational)

Method: Thermocouples on all critical/hot components at board level. Bring temperature of top center surface of thermal plate to 85°C (185 °F) and allow it to stabilize. Ensure that all components are within manufacturer thermal specifications.

Table 5. Altitude

Low Pressure/Altitude

(Operational)

Altitude: 4.6km (15,000ft)

Equivalent Absolute Pressure: 57.2 kPa (8.3 lbf/in2)

Temperature: -40°C (-40°F) to 74°C (165°F)

Altitude Ramp Rate: 10m/s (max)

Temperature Ramp Rate: 1.5°C (min) to 4.5°C (max)

Reference: MIL-STD 810F, Method 500.4, Procedure II; SAE J1455 (Rev AUG94), Section 4.1.3.1

Operational Altitude

Altitude: 12.2km (40,000ft)

Equivalent Absolute Pressure: 18.6kPa (2.7lbf/in2)

Temperature: -40C (-40F) to 25C (77F)

Low Pressure/Altitude

(Non-Operational)

Altitude: 12.2km (40,000 ft)

Equivalent Absolute Pressure: 18.6kPa (2.7lbf/in2)

Temperature: -40°C (-40°F) to 85°C (185°F)

Altitude Ramp Rate: 10m/s (max)

Temperature Ramp Rate: 1.5°C (min) to 4.5°C (max)

Reference: MIL-STD-810F, Method 500.4; SAE J1455 (Rev AUG94), Section 4.1.3.1
Table 6. Humidity

Temperature & Humidity Cycle Stress

(Non-Operational; Energized)

Humidity: 95% +/- 5% RH

Pressure: 103.4 kPa (15 lbf in2)

Temperature: -40°C (-40°F) to 65°C (149°F)

Cycle: One, 24 hour cycle

Reference: SAE J1455 (Rev AUG94), Section 4.2.3

Active Temperature/Humidity

10 Day Soak

(Non-Operational; Energized)

Temperature: -40°C (-40°F) to 65 °C (149 °F)

Humidity: 95% +/- 5% RH

Cycle: Ramp from 25°C to 0°C over 75 minute period, dwell at 0°C for 240 minutes, ramp to 65°C over 120 minute period, dwell at 65°C for 240 minutes (95% +/- 5% RH), ramp to 25°C over 45 minute period, dwell at 25°C for 120 minutes (50% +/- 5% RH)

Repetition: 20 total cycles (10 days total)

Reference: MIL-STD-810F, Method 507.4; SAE J1211 (Rev NOV78), Section 4.2.2; SAE J1455 (Rev AUG94), Section 4.2.3
Table 7. Vibration and Shock

Random Vibration

(Operational)

Acceleration: 1.04g rms vertical, 0.204g rms transverse, 0.740g rms longitudinal

Duration: 2 hours per axis

Test orientation: 3 axes

Reference: MIL-STD-810F, Method 514.5, Category 4

Crash Hazard Shock

(Non-Operational)

Acceleration: 75G

Duration: 8-13ms

Test orientation: 3 axes (positive and negative)

Number of shocks: 2 shocks in each direction, 12 shocks total

Reference: MIL-STD-810F, Method 516.5, Procedure V

Functional Shock

(Operational)

Acceleration: 40G

Duration: 15-23ms

Test orientation: All 6 faces, in 3 perpendicular axes

Reference: MIL-STD-810F, Method 516.5, Procedure I

Bench handling shock (tip)

(Operational)

Test orientation: All four edges of each face to form 10° angle with bench top

Reference: MIL-STD-810F, Method 516.5, Procedure VI

Overtemperature Detection

The board has a temperature sensor mounted on the edge of the board that should be thermally attached to the Customer Designed Conduction Plate. When the temperature sensor detects a temperature exceeding the threshold of 203°F (96°C), the overtemperature LED will illuminate.

The digital temperature sensor measures the temperature of the Customer Designed Conduction Plate, not the local ambient temperature. The product datasheet states the board will operate as long as the conduction plate is in the range of -40C to +85C. The alarms are set accordingly, and the high temperature alarm thresholds are set as follows:

  • Minor alarm at +80C the Customer Designed Conduction Plate temperature is close to the rated thermal limit of the unit, and will notify the user. The components are still within the specification, so there is no degradation to the long term reliability of the system.
  • Major alarm at +90C the Customer Designed Conduction Plate temperature is over the rated thermal limit of the unit, and will notify the user. This will impact the long term reliability of the system.
  • Critical alarm at +96C the Customer Designed Conduction Plate temperature is way over the rated thermal limit of the unit, and will notify the user. This will impact the long term reliability of the system. For the Critical Alarm threshold to be reached, it means that the ambient temperature of the system will be exceeded. Hardware failure is immanent, and the failure time will depend upon your installation. Depending on the severity at this point, the failure may be temporary or permanent.

Caution

IOS will never shut down a device because the temperature exceeds the specification. Cisco does not guarantee the functionality, nor the long term reliability of a device operating beyond Cisco specifications, but lets the device continue operating until some piece of hardware physically shuts down. Operating outside of the temperature specifications will void the product warranty.

The status of the temperature sensors can be reported from the Cisco ESS-9300 IOS CLI: 


Switch# show environment all
ALARM CONTACT 1
   Status:      not asserted
   Description: external alarm contact 1
   Severity:    minor
   Trigger:     closed
ALARM CONTACT 2
   Status:      not asserted
   Description: external alarm contact 2
   Severity:    minor
   Trigger:     closed
Supervisor Temperature Value: 51 C
Temperature State: GREEN
System Temperature thresholds
-------------------------------
Minor Threshold    : 80 C (Yellow)
Major Threshold    : 90 C (Red)
Critical Threshold : 96 C 
Shutdown Threshold : 105 C 
Pwr Supply         Type     Status
------------------------------------
POWER SUPPLY-A      DC      OK
POWER SUPPLY-B      DC      OK

Thermal Design Considerations

The following sections outline the methods for dealing with thermal issues and the mounting options involving the Customer Designed Conduction Plate.

As the ESS9300 is intended for use in extreme environments, industrial temperature rated components are used.

As a general rule, the thermal plate of the card needs to make contact with an adequate thermal mass to draw heat away from the card. This can be done in a number of ways.

The important note is that the Customer Designed Conduction Plate temperature, as measured at the center of the top surface of the conduction plate, must not exceed 85°C. As long as this requirement is satisfied, all of the card's components will be within a safe operating temperature range on the high temperature side.


Note
The area in the following figure noted by the red square is the 45A power supply and needs some additional cooling.
Figure 2. Thermally Significant Components of Cisco ESS-9300

Note
Cisco uses the following TIMs at each REFDES:

The U1, U9, U12, U13, U14, U15, U8, and other items in the figure above use the Chomerics GEL30. U1 uses a Fujipoly 32x32, 150Xr-PE thermal pad.

Samtec has 3D models, footprints, and schematic symbols for their connectors here:

https://www.samtec.com/connectors/high-speed-board-to-board/high-density-arrays/searay

RefDes

Thermal Design Power (in W)

Allowable junction temp (in °C)

Allowable case temp (in °C)

Package Type

Theta Jc (in °C/W)

Theta Jb (in °C/W)

U1

33

115

—

FCTEBGA1155

—

-

U9, U12, U13, U14, U15

0.2 Each

—

95

FBGA96

3.0

—

U8

1

—

—

FBGA153

—

—

U37

—

—

—

MSOP8

—

—

Note
Q2 is a NPN transistor that can be used by the system integrator to read the temperature of the cooling plate for their use.

Validating a Thermal Solution

To validate a thermal solution, monitor the thermal sensor of the Cisco ESS 9300 cards in a thermal chamber set to the desired maximum ambient operating temperature and with traffic running.

The temperature sensor should make contact with the Customer Designed Conduction Plate using thermal interface material. The temperature of the sensors should be less than 90.5C. The show environment all command can be executed from the IOS prompt to monitor the thermal sensor temperatures. 


Switch# show environment all
ALARM CONTACT 1
   Status:      not asserted
   Description: external alarm contact 1
   Severity:    minor

Product Specifications

The following tables list the product specifications for the Cisco ESS 9300.

Table 8. Interface Support

Item

Description

ESS-9300

10 ports of 1 or 10 GE fiber (XFI)

Table 9. Memory

Item

Description

DRAM

4GB

SPI Flash

16MB

eMMC Flash

7 GB, 2.5 GB user accessible

Table 10. Hardware Specifications

Item

Description

Input voltages

+5Vdc (+/- 5%) and +3.3Vdc (+/- 3%)

Total Power

Thermal Power = 35W

Max Power = 43W

Mass

88 grams (3.10 ounces)


Note
For Environmental Specifications, please see the Testing section for complete specifications.

Power Requirements

The board requires +5 VDC and +3.3 VDC to operate. Board Electrical Power Consumption lists the DC power requirements.

The ESS-9300 can display a POWER GOOD status for two Power Inputs via the DC-A-GOOD and DC-B-GOOD signals. If these signals are not used, connect DC-A-GOOD to 3.3 V and DC-B-GOOD to ground through a 1k resistor.


Note
There is no specific voltage sequence requirement for the 5V and 3.3V power inputs. They can ramp up in any order.

Dying Gasp

If the switch is configured, and the feature is enabled, in the case of a temporary power outage, the switch will send a Dying Gasp packet. If the power recovers, the switch will continue to operate normally. See more about Dying Gasp in the ESS9300 Software Configuration Guide .

Board Electrical Power Consumption

Table 11. Power Requirements

Voltage Rail

Tolerance

Typical Current (A)

Maximum Current (A)

5V

+/- 3%

N/A

7.0A

3.3V

+/- 3%

N/A

2.0A

P3_3V RTC

+10% / -60%

0.4uA

0.7uA

SD Support

There is one Cisco SD card that has been tested and is recommended, the SD-IE-4GB. If the end user or system integrator chooses to use a 3rd party device, it may work for their application and to their satisfaction. However, the end user or system integrator is solely responsible for testing and ensuring proper operation.

The following message displays when a different SD card is installed: 


WARNING: Non-IT SD flash detected. Use of this card during normal operation can impact and
severely degrade performance of the system. Please use supported SD flash cards only.

You can find Cisco’s policy on Third Party Components here:

https://www.cisco.com/c/en/us/products/warranties/warranty-doc-c99-740959.html#_Toc3320258

SFP Support

The following table lists the specific SFP transceivers and their characteristics.


Note
LRM optics are not supported since the SFP is direct driven from the Cisco ASIC.

Supported SFP and SFP+ Modules

Table 12. Supported Modules

Part Number

Specification

SFP Type

Max Distance

Cable Type

Temp Range

DOM Support

GLC-SX-MM-RGD=

1000BASE-SX

GE

550m

MMF

IND

Yes

GLC-LX-SM-RGD=

1000BASE-LX/LH

GE

550m/10km

MMF/SMF

IND

Yes

GLC-SX-MMD=

1000BASE-SX

GE

550m

MMF

EXT

Yes

GLC-LH-SMD=

1000BASE-LX/LH

GE

550m/10km

MMF/SMF

EXT

Yes

GLC-BX-D=

1000BASE-BX10

GE

10km

SMF

COM

Yes

GLC-BX-U=

1000BASE-BX10

GE

10km

SMF

COM

Yes

GLC-ZX-SM-RGD

1000BASE-ZX

GE

Approx. 70km

SMF

IND

Yes

GLC-EX-SMD=

1000BASE-EX

GE

40km

SMF

EXT

Yes

SFP-GE-S=

1000BASE-SX

GE

550m

MMF

EXT

Yes

GLC-SX-MM=

1000BASE-SX

GE

550m

MMF

COM

No

GLC-T-RGD=

1000BASE-T

GE

100m

Copper

IND

N/A

GLC-LH-SM=

1000BASE-LX/LH

GE

550m/10km

MMF/SMF

COM

No

GLC-TE=

1000BASE-T

GE

100m

Copper

EXT

N/A

GLC-T=

1000BASE-T

GE

100m

Copper

COM

N/A

CWDM-SFP-xxxx= (8 freq)

CWDM 1000BASE-X

GE

—

SMF

COM

Yes

DWDM-SFP-xxxx= (40 freq)

DWDM 1000BASE-X

GE

—

SMF

COM

Yes

SFP-10G-BXD-I=

10GBASE-BX10

10GE

10km

SMF

IND

Yes

SFP-10G-BXU-I=

10GBASE-BX10

10GE

10km

SMF

IND

Yes

SFP-10G-SR-X=

10GBASE-SR

10GE

400m

MMF

EXT

Yes

SFP-10G-LR-X=

10GBASE-LR

10GE

10km

SMF

EXT

Yes

SFP-10G-SR=

10GBASE-SR

10GE

400m

MMF

COM

Yes

SFP-10G-LR=

10GBASE-LR

10GE

10km

SMF

COM

Yes

SFP-H10GB-CUxM=

10G Passive Twinax

10GE

1m/3m/5m

Twinax

COM

N/A

SFP-H10GB-ACUxM=

10G Active Twinax

10GE

7m/10m

Twinax

COM

N/A

SFP-10G-T-X  *

10GBASE-T

10GE

Up to 30 meters

Cat6A/Cat7

EXT

NA