CE-1000-4 Ethernet Operation

Note The terms "Unidirectional Path Switched Ring" and "UPSR" may appear in Cisco literature. These terms do not refer to using Cisco ONS 15xxx products in a unidirectional path switched ring configuration. Rather, these terms, as well as "Path Protected Mesh Network" and "PPMN," refer generally to Cisco's path protection feature, which may be used in any topological network configuration. Cisco does not recommend using its path protection feature in any particular topological network configuration.

This chapter describes the operation of the CE-1000-4 (Carrier Ethernet) card supported on the
Cisco ONS 15454 and Cisco ONS 15454 SDH. A CE-1000-4 card installed in an ONS 15454 SONET is restricted to SONET operation, and a CE-1000-4 card installed in an ONS 15454 SDH is restricted to SDH operation.

Use Cisco Transport Controller (CTC) or Transaction Language One (TL1) to provision the CE-1000-4 card. Cisco IOS is not supported on the CE-1000-4 card.

For Ethernet card specifications, refer to the Cisco ONS 15454 Reference Manual or the Cisco ONS 15454 SDH Reference Manual. For step-by-step Ethernet card circuit configuration procedures, refer to the Cisco ONS 15454 Procedure Guide or the Cisco ONS 15454 SDH Procedure Guide. Refer to the Cisco ONS SONET TL1 Command Guide or the Cisco ONS SDH Command Guide for TL1 provisioning commands.

Chapter topics include:

•CE-1000-4 Overview

•CE-1000-4 Ethernet Features

•CE-1000-4 SONET/SDH Circuits and Features

CE-1000-4 Overview

The CE-1000-4 is a Layer 1 mapper card with four Gigabit Ethernet ports. It maps each port to a unique SONET/SDH circuit in a point-to-point configuration. Figure 25-1 illustrates a sample CE-1000-4 application. In this example, data traffic from the Gigabit Ethernet port of a switch travels across the point-to-point circuit to the Gigabit Ethernet port of another switch.

Figure 25-1 CE-1000-4 Point-to-Point Circuit

The CE-1000-4 cards allow you to provision and manage an Ethernet private line service like a traditional SONET/SDH line. The CE-1000-4 card provides carrier-grade Ethernet private line services and high-availability transport.

The CE-1000-4 card carries any Layer 3 protocol that can be encapsulated and transported over Ethernet, such as IP or IPX. The Ethernet frame from the data network is transmitted into the gigabit interface converter (GBIC) on a CE-1000-4 card. The CE-1000-4 card transparently maps Ethernet frames into the SONET/SDH payload using packet-over-SONET/SDH (POS) encapsulation. The POS circuit with encapsulated Ethernet is then multiplexed onto an optical card like any other SONET synchronous transport signal (STS) or SDH synchronous transport mode (STM). When the payload reaches the destination node, the process is reversed and the data is transmitted from the GBIC in the destination CE-1000-4 card onto the Ethernet of the data network. The POS process is covered in detail in Chapter 20 "POS on ONS Ethernet Cards."

The CE-1000-4 card supports ITU-T G.707 and Telcordia GR-253 based standards. It allows an errorless soft reset. An exception to the errorless soft reset occurs when there is a provisioning change during the reset, or if the firmware is replaced during the software upgrade process. In these cases, the reset is equivalent to a hard reset. To perform a soft reset on a CE-1000-4 card using CTC, refer to the Cisco ONS 15454 Procedure Guide or the Cisco ONS 15454 SDH Procedure Guide.

CE-1000-4 Ethernet Features

The CE-1000-4 card has four front-end Ethernet ports which use standard GBIC connectors for Gigabit Ethernet. Ethernet Ports 1 through 4 each map to a POS port with a corresponding number. These Ethernet ports can be daisy chained.

At the Ethernet port level, a user can configure several characteristics:

•Port name

•Administrative state

•Automatic in-service (AINS) soak time

•Flow control

•Flow control watermark levels

•Auto negotiation

The CE-1000-4 card forwards valid Ethernet frames unmodified over the SONET/SDH network. Information in the headers is not affected by the encapsulation and transport. For example, IEEE 802.1Q information will travel through the process unaffected.

The CE-1000-4 supports Jumbo frames up to a total maximum of 10004 bytes, including Ethernet cyclic redundancy check (CRC), by default. In CTC you can also configure a total maximum frame size of 1548 bytes, including Ethernet CRC.

Note Many Ethernet attributes are also available through the network element (NE) defaults feature. For more information on NE defaults, refer to the "Network Element Defaults" appendix in the Cisco ONS 15454 Reference Manual or the Cisco ONS 15454 SDH Reference Manual.

Autonegotiation and Frame Buffering

On the CE-1000-4 card, Ethernet link autonegotiation is on by default. You can also enable and disable autonegotiation under the card-level Provisioning tab of CTC.

The CE-1000-4 supports field-programmable gate array (FPGA) buffering to reduce data traffic congestion. FPGA buffering supports SONET/SDH oversubscription. When the buffer nears capacity, the CE-1000-4 card uses IEEE 802.3x flow control to transmit a pause frame to the attached Ethernet device. Flow control and autonegotiation frames are local to the Gigabit Ethernet interfaces and the attached Ethernet devices. These frames do not continue through the POS ports.

Flow Control

The CE-1000-4 supports IEEE 802.3x flow control and allows you to enable symmetric flow control, enable asymmetric flow control, or to disable flow control. The configuration is done in CTC at the port level.

By default the CE-1000-4 card uses symmetric flow control and only proposes symmetric flow control when autonegotiating flow control with attached Ethernet devices. Symmetric flow control allows the CE-1000-4 cards to respond to pause frames sent from external devices and to send pause frames to external devices.

The pause frame instructs the source to stop sending packets for a specific period of time. The sending station waits the requested amount of time before sending more data. Figure 25-2 illustrates pause frames being sent and received by CE-1000-4 cards and attached switches.

Figure 25-2 Flow Control

This flow-control mechanism matches the sending and receiving device throughput to that of the bandwidth of the STS circuit. For example, a router might transmit to the Gigabit Ethernet port on the CE-1000-4 card. This particular data rate might occasionally exceed 51.84 Mbps, but the SONET circuit assigned to the CE-1000-4 port might be only STS-1 (51.84 Mbps). In this example, the CE-1000-4 card sends out a pause frame and requests that the router delay its transmission for a certain period of time. With flow control and a substantial per-port buffering capability, a private line service provisioned at less than full line rate capacity (STS-1) is efficient because frame loss can be controlled to a large extent.

Asymmetric enables the CE-1000-4 to receive flow control pauses, but not generate flow control pauses. This mode supports a link partner that cannot receive flow control pauses but can send flow control pauses. The CE-1000-4 does not have a mode where it would send flow control pauses but not be able to receive flow control pauses.

In pass-through mode, transmit flow control frames are not generated by the Ethernet port interfaces, and received flow control frames pass through transparently. Pass-through mode supports end-to-end flow control between clients using Ethernet over SONET/SDH transport.

Flow Control Threshold Provisioning

The CE-1000-4 card has flow control threshold provisioning, which allows a user to select one of three watermark (buffer size) settings: default, low latency, or custom. Default is the best setting for general use. Low latency is good for sub-rate applications, such as voice-over-IP (VoIP) over an STS-1. For attached devices with insufficient buffering, best effort traffic, or long access line lengths, set a higher latency.

The flow control high setting is the watermark for sending the Pause On frame to the attached Ethernet device; this frame signals the device to temporarily stop transmitting. The flow control low setting is the watermark for sending the Pause Off frame, which signals the device to resume transmitting. The default watermark setting values are 485 for the high threshold and 25 for the low threshold. Low latency watermark setting values are10 for the high threshold and 5 for the low threshold.

The custom setting allows you to specify the buffer size of Flow Ctrl Lo and Flow Ctrl Hi thresholds. The range is 1 to 511 units, where 1 unit is equal to 192 bytes. Make sure that the value of Flow Ctrl Lo is lesser than Flow Ctrl Hi with a difference of at least 160 units between the two values to ensure packets are not dropped.

Ethernet Link Integrity Support

The CE-1000-4 card supports end-to-end Ethernet link integrity (Figure 25-3). This capability is integral to providing an Ethernet private line service and correct operation of Layer 2 and Layer 3 protocols on the attached Ethernet devices. Link Integrity is implemented so that the Ethernet over SONET/SDH connection behaves more like an Ethernet cable from the viewpoint of the attached Ethernet devices.

End-to-end Ethernet link integrity means that if any part of the end-to-end path fails, the entire path fails. It disables the Ethernet port transmitter on the CE-1000-4 card when the remote Ethernet port does not have a receive signal or when the SONET/SDH near end of a far-end failure is detected. The failure of the entire path is ensured by turning off the transmit pair at each end of the path. The attached Ethernet devices recognize the disabled transmit pair as a loss of carrier and consequently an inactive link or link fail. The transport fail alarm is also raised when the port transmitter is disabled. Link integrity will support a double fault, which is when both Ethernet ports do not receive a signal.

Figure 25-3 End-to-End Ethernet Link Integrity Support

Some network devices can be configured to ignore a loss of carrier condition. If a device configured to ignore a loss of carrier condition attaches to a CE-1000-4 card at one end, alternative techniques (such as use of Layer 2 or Layer 3 keep-alive messages) are required to route traffic around failures. The response time of such alternate techniques is typically much longer than techniques that use link state as indications of an error condition.

Administrative and Service States with Soak Time for Ethernet and SONET/SDH Ports

The CE-1000-4 card supports the administrative and service states for the Ethernet ports and the SONET/SDH circuit. For more information about card and circuit service states, refer to the "Administrative and Service States" appendix in the Cisco ONS 15454 Reference Manual or the Cisco ONS 15454 SDH Reference Manual.

Ethernet ports can be set to the In-Service, Automatic In-Service (IS,AINS) administrative state. IS,AINS initially puts the port in the Out-of-Service and Autonomous, Automatic In-Service (OOS-AU,AINS) state. In this service state, alarm reporting is suppressed, but traffic is carried and loopbacks are allowed. After the soak time passes, the port changes to In-Service and Normal (IS-NR).

The default soak time is eight hours and zero minutes. The user can also configure the AINS soak time under the Provisioning tab > Ether Ports tab or under the Provisioning tab > POS Ports tab. The user can view the AINS soak time and the time remaining until IS under the Maintenance tab > AINS Soak tabs.

Raised fault conditions, whether their alarms are reported or not, can be retrieved on the CTC Conditions tab or by using the TL1 RTRV-COND command.Two Ethernet port alarms/conditions, CARLOSS and TPTFAIL, can prevent the port from going into service. This occurs even though alarms are suppressed when a CE-1000-4 circuit is provisioned with the Ethernet ports set to the IS,AINS state, because the CE-1000-4 link integrity function is active and ensures that the links at both ends are not enabled until all SONET/SDH and Ethernet errors along the path are cleared. If the link integrity function keeps the end-to-end path down, both ports will have at least one of the two conditions needed to suppress the AINS-to-IS transition. Therefore, the ports will remain in the AINS state with alarms suppressed.

ESM also applies to the SONET/SDH circuits of the CE-1000-4 card. If the SONET/SDH circuit is set up in IS,AINS state and the Ethernet error occurs before the circuit transitions to IS, then link integrity will also prevent the circuit transition to the IS state until the Ethernet port errors are cleared at both ends. The service state will be OOS-AU,AINS as long as the administrative state is IS,AINS. When there are no Ethernet or SONET errors, link integrity enables the Ethernet port at each end. Simultaneously, the AINS countdown begins as normal. If no additional conditions occur during the time period, each port transitions to the IS-NR state. During the AINS countdown, the soak time remaining is available in CTC and TL1. The AINS soaking logic restarts from the beginning if a condition appears again during the soak period.

A SONET/SDH circuit provisioned in the IS,AINS state remains in the initial Out-of-Service (OOS) state until the Ethernet ports on each end of the circuit transition to the IS-NR state. The SONET/SDH circuit transports Ethernet traffic and counts statistics when link integrity turns on the Ethernet port, regardless of whether this AINS-to-IS transition is complete.

RMON and SNMP Support

The CE-1000-4 card features remote monitoring (RMON) and simple network management protocol (SNMP) that allows network operators to monitor the health of the network with a network management system (NMS). The CE-1000-4 uses ONG RMON. ONG RMON contains the statistics, history, alarms, and events MIB groups from the standard RMON MIB. A user can access RMON threshold provisioning through TL1 or CTC. For RMON threshold provisioning with CTC, refer to the Cisco ONS 15454 Procedure Guide or the Cisco ONS 15454 SDH Procedure Guide.

Statistics and Counters

The CE-1000-4 has a full range of Ethernet and POS statistics information under the Performance > Ether Ports tabs or the Performance > POS Ports tabs.

CE-1000-4 SONET/SDH Circuits and Features

The CE-1000-4 card has four POS ports, numbered one through four, which can be managed with CTC or TL1. Each POS port is statistically mapped to a matching Ethernet port. The CE-1000-4 card provides a total bandwidth of STS-48c in any compatible slot within an ONS 15454 or a total bandwidth of STM-16 in any compatible slot within an ONS 15454 SDH.

At the POS port level, you can configure several characteristics:

•Port name

•Administrative state

•Automatic in-service (AINS) soak time

•Framing type

•Encapsulation CRC

Note Encapsulation CRC can only be turned on and off (CRC or no CRC), when the framing type is configured for GFP. When the framing type is set to HDLC, CRC is always on.

Click the card-level Provisioning > POS Ports tabs to configure the Administrative State, Framing Type, and Encapsulation Type. Click the card-level Performance > POS Ports tab to view the statistics, utilization, and history for the POS ports.

For specific circuit sizes and compatible card slots for the CE-1000-4 card, refer to the "Ethernet Cards" chapter in the Cisco ONS 15454 Reference Manual or the Cisco ONS 15454 SDH Reference Manual.

CE-1000-4 VCAT Characteristics

The CE-1000-4 card supports the software link capacity adjustment scheme (SW-LCAS). This makes the CE-1000-4 card compatible with the ONS 15454 SONET/SDH ML-Series cards, which also supports SW-LCAS. The CE-1000-4 card does not support standard LCAS, which is hardware-based. The CE-1000-4 also operates with no SW-LCAS enabled. In this mode, it is compatible with the ONS 15454 SONET/SDH's G-Series card, CE-100T-8 card, and ML-Series card, when the ML-Series card is not configured with SW-LCAS. For more information on Ethernet card compatibility, see Chapter 20 "POS on ONS Ethernet Cards."

To enable end-to-end connectivity in a VCAT circuit that traverses through a third-party network, you must create a server trail between the ports. For more details, refer to the "Create Circuits and VT Tunnels" chapter in the Cisco ONS 15454 Procedure Guide.

The CE-1000-4 card supports flexible VCAT groups (VCGs) and fixed (pure or non-flexible) VCGs. Flexible VCG corresponds to SW-LCAS, fixed VCG corresponds to no LCAS. With flexible VCGs, the CE-1000-4 can perform these operations:

•Add or remove members from groups

•Put members into or out of service, which also adds/removes them from the group

•Add or remove cross-connect circuits from VCGs

•Automatically remove errored members from the group

Adding or removing members from the VCG is service-affecting. Adding or removing cross-connect circuits is not service-affecting, if the asociated members are not in the group

The CE-1000-4 card also supports fixed (pure or non-flexible) VCGs. With non-flexible VCGs, the CE-1000-4 is more limited an can only perform these operations:

•Put members into or out of service

•Add or remove cross-connect circuits associated with members

With non-flexible VCGs, the limitations of the CE-1000-4 include:

•Cannot add or remove members from groups

•Cannot automatically remove errored members from the group

The CE-1000-4 card allows independent routing and protection preferences for each member of a VCAT circuit. The user can also control the amount of VCAT circuit capacity that is fully protected, unprotected, or uses Protection Channel Access (PCA) (when PCA is available). Alarms are supported on a per-member as well as per virtual concatenation group (VCG) basis.

The CE-1000-4 card supports VCAT common fiber routing and VCAT split fiber (diverse) routing. Common fiber routing is compatible with two-fiber bidirectional line switched ring (BLSR) protection schemes and APS. It does not support path protection and four-fiber BLSR protection schemes. Split fiber routing supports all protection types: Path Protection, two-fiber BLSR, four-fiber BLSR, and linear switching (1+1).

With VCAT split fiber routing, each member can be routed independently through the SONET/SDH network instead of having to follow the same path as required by CCAT and VCAT common fiber routing. This allows a more efficient use of network bandwidth, but the different path lengths and different delays encountered may cause slightly different arrival times for the individual members of the VCG. The VCAT differential delay is this relative arrival time measurement between members of a VCG. The maximum tolerable VCAT split fiber routing differential delay for the CE-1000-4 card is approximately 120 milliseconds. A loss of alignment alarm is generated if the maximum differential delay supported is exceeded.

The differential delay compensation function is automatically enabled when the user chooses split fiber routing during the CTC circuit configuration process. CCAT and VCAT common fiber routing do not enable or need differential delay support.

Caution Protection switches of less than 60ms are not guaranteed with the differential delay compensation function enabled. The compensation time is added to the switching time.

Note For TL-1, EXPBUFFERS parameter must be set to ON in the ENT-VCG to enable support for split fiber routing.

CE-1000-4 POS Encapsulation, Framing, and CRC

The CE-1000-4 card uses Cisco EoS LEX (LEX). LEX is the primary encapsulation of ONS Ethernet cards. In this proprietary HDLC-based encapsulation, the protocol field is set to the values specified in Internet Engineering Task Force (IETF) Request For Comments (RFC) 1841.

The user can provision framing on the CE-1000-4 as either the default frame-mapped generic framing procedure framing (GFP-F) or high-level data link control (HDLC) framing.

With GFP-F framing, the user can also configure a 32-bit CRC (default) or no CRC (none). When LEX is used over GFP-F it is standard Mapped Ethernet over GFP-F according to ITU-T G.7041.

HDLC framing provides a set 32-bit CRC. On CTC go to CE card view and click the Provisioning >pos ports tab, to see the various parameters that can be configured on the POS ports, see "Displaying ML-Series POS Ports Provisioning Information on CTC" section. Various parameters like, admin state, service state, framing type, CRC , MTU and soak time for a port can be configured here

For more details about the interoperability of ONS Ethernet cards, including information on encapsulation, framing, and CRC, see the "POS on ONS Ethernet Cards" chapter.

CE-1000-4 Loopback, J1 Path Trace, and SONET/SDH Alarms

The CE-1000-4 card supports terminal and facility loopbacks. It also reports SONET/SDH alarms and transmits and monitors the J1 Path Trace byte in the same manner as OC-N cards. Support for path termination functions include:

•H1 and H2 concatenation indication

•Bit interleaved parity 3 (BIP-3) generation

•G1 path status indication

•C2 path signal label (read only)

•Path level alarms and conditions, including loss of pointer (LOP), unequipped, payload mismatch, alarm indication signal (AIS) detection, and remote defect indication (RDI)

•J1 path trace for high-order circuit paths

•Extended signal label for the low-order paths