Setup Multi-chassis Configuration

Multiple Network Convergence System (NCS) 6008 single chassis can be connected using NCS 6000 fabric card chassis to form a multi-chassis system. This provides high scale of interfaces with single admin and control plane.

The multi-chassis system consists of two types of chassis:

  • Line card chassis (LCC)

  • Fabric card chassis (FCC)

Multi-chassis system is built with one or more LCC connected to FCCs, with a maximum hardware configuration of 16 LCC and a set of 4 FCC. The LCC has 8 line card slots and 6 fabric card slots, each fabric slot representing a logical fabric plane. The system provides 8 Tbps of full-duplex network bandwidth through eight line cards. The FCC hosts the agnostic fabric cards (FC). Each FCC can have 12 FC slots. The fabric cards used in a single chassis setup is different from the fabric cards used in the multi-chassis system. All the LCCs are connected to the FCC, and traffic flows from one LCC to another LCC through the FCC.

To form a multi-chassis system, two types of connectivity must be established:
  • Control ethernet connectivity: The racks are interconnected to form a network. The shelf controller (SC) cards and route processor (RP) cards have two control ethernet ports - port 0 and port 1. These ports are connected to two switches that are in-turn connected using 40G trunk ports.
  • Fabric connectivity: The FCC and LCC are connected to form a fabric plane.

After the hardware components are installed, and connectivity is established between the racks, you can setup a multi-chassis system.

The workflow for setting up a multi-chassis system is represented in this flow chart:

Note
This workflow image has clickable links that are enabled only in the HTML format (and not in the PDF format) of this document.
Figure 1. Multi-chassis Configuration Interactive Workflow


Assign Rack Number to Chassis

In a multi-chassis system, each chassis must have a unique rack number. This rack number is used to identify a chassis in the system.

Complete this task to identify the chassis and its connections and to establish a multi-chassis network:

Before you begin

  • Install the hardware components of the multi-chassis system. In Line card chassis (LCC), line cards, fabric cards, and route processors are installed, and in Fabric card chassis (FCC), fabric cards (FC), shelf-controller switch cards (SC-SW) and shelf controllers (SC) are installed.
  • Connect the chassis in the multi-chassis system using control ethernet cables. The control ethernet network provides inter-connectivity between chassis, and a two-way management ethernet path from System Admin and XR VM console. For more information see Cabling the Ethernet Control Plane Network section in the Cisco Network Convergence System 6000 Fabric Card Chassis Hardware Installation Guide.
  • The fabric cards on the LCC and FCC are connected using fabric cables. This is used for data traffic. For more information see Cabling the Fabric section in the Cisco Network Convergence System 6000 Fabric Card Chassis Hardware Installation Guide.

SUMMARY STEPS

  1. admin
  2. show chassis
  3. config
  4. chassis serial <chassis serial number>
  5. rack <rack number>
  6. Repeat step 3 to 5 for all the chassis in the multi-chassis system.
  7. commit

DETAILED STEPS

  Command or Action Purpose
Step 1

admin

Example:


RP/0/RP0/CPU0:router# admin

Enters System Admin EXEC mode.

Step 2

show chassis

Example:

The following example shows a 2 + 2 (2 LCC and 2 FCC) configuration: 
sysadmin-vm:0_RP0#show chassis
Wed Jun  4  17:18:22.498 UTC
Serial Num    Rack Num    Rack Type   Rack State  Data Plane  Ctrl Plane
--------------------------------------------------------------------------
FMP12020050   0           LCC         UP          CONN        CONN
FMP12260408   F1          FCC         UP          CONN        CONN
FMP16320213   1           LCC         UP          CONN        CONN
FMP17260334   F0          FCC         UP          CONN        CONN
The following example shows a 4 + 2 (4 LCC and 2 FCC)configuration: 
sysadmin-vm:0_RP0#show chassis
Wed Jun  4  17:18:22.498 UTC
Serial Num    Rack Num    Rack Type   Rack State  Data Plane  Ctrl Plane
--------------------------------------------------------------------------
FLM17326A2K   0           LCC         OPERATIONAL  CONN        CONN
FLM17326A2J   1           LCC         OPERATIONAL  CONN        CONN
FLM17386E28   2           LCC         OPERATIONAL  CONN        CONN 
FLM17075XWM   3           LCC         OPERATIONAL  CONN        CONN         
FMP17210228   F0          FCC         OPERATIONAL  CONN        CONN      
FMP17380420   F1          FCC         OPERATIONAL  CONN        CONN
Shows the chassis details including the chassis serial number.

Identify the serial numbers for which rack numbers are to be assigned.

Step 3

config

Example:

sysadmin-vm:0_RP0#config
Enters the configuration mode.
Step 4

chassis serial <chassis serial number>

Example:

sysadmin-vm:0_RP0(config)# chassis serial FMP12020050
Enters the configuration mode for the chassis serial number.
Step 5

rack <rack number>

Example:

sysadmin-vm:0_RP0(config-serial-FMP12020050)# rack 0
Type a rack number to associate the rack number to the chassis.
Step 6

Repeat step 3 to 5 for all the chassis in the multi-chassis system.

Step 7

commit

What to do next

Perform preliminary checks to verify that the chassis is configured correctly:

  • After configuring the chassis serial number, verify that a unique rack number is assigned for each chassis serial number using the show running-config chassis serial command in System Admin EXEC mode.

    The following example shows a 2+2 configuration: 
    sysadmin-vm:0_RP0# show running-config chassis serial
chassis serial FMP12020050
 rack 0
!
chassis serial FMP12260408
 rack F1
!
chassis serial FMP16320213
 rack 1
!
chassis serial FMP17260334
 rack F0
    The following example shows a 4+2 configuration: 
    sysadmin-vm:0_RP0# show running-config chassis serial
chassis serial FLM17326A2K
 rack 0
!
chassis serial FLM17326A2J
 rack 1
!
chassis serial FLM17386E28
 rack 2
!
chassis serial FLM17075XWM
 rack 3
!
chassis serial FMP17210228
 rack F0
!
chassis serial FMP17380420
 rack F1

  • After the rack numbers are assigned to the chassis serial number, verify that all the cards are operational, and XR VM is running on all the cards. For details, see Verify SDR Information.

  • If the XR VM is not running, no output is shown for that location in the result. In this case, verify the state of secure domain router (SDR) on the node using the show sdr command in System Admin EXEC mode. For details, see Verify SDR Information.

  • After the system has booted, all available interfaces must be discovered by the system. View the number of discovered interfaces using the show interfaces summary command. For details, see Verify Interface Status

After verifying that the cards in LCC are operational, configure the fabric cards in FCC.

Associate Fabric Card in Fabric Card Chassis to Fabric Plane

The fabric cards in the fabric card chassis (FCC) must be associated to a fabric plane in the multi-chassis system. The system fabric is divided into six fabric planes that are used to evenly distribute traffic across the fabric. The location of each fabric card in the fabric plane is identified by an instance number. The instance number starts from 0 and are sequential.

For information on multi-chassis configurations and cabling, see Cabling a Multi-Chassis Configuration in the Cisco Network Convergence System 6000 Fabric Card Chassis Hardware Installation Guide.

SUMMARY STEPS

  1. admin
  2. config
  3. controller fabric plane <plane number>
  4. instance <instance-number>
  5. location <fabric-chassis-number/fabric-card-number>
  6. Repeat step 3 to step 5 to associate all fabric cards to an instance in the fabric plane.

DETAILED STEPS

  Command or Action Purpose
Step 1

admin

Example:


RP/0/RP0/CPU0:router# admin

Enters System Admin EXEC mode.

Step 2

config

Example:

sysadmin-vm:0_RP0#config
Enter System Admin Config mode.
Step 3

controller fabric plane <plane number>

Example:

sysadmin-vm:0_RP0(config)# controller fabric plane 4
Enter configuration mode for the specified fabric plane.
Step 4

instance <instance-number>

Example:

sysadmin-vm:0_RP0(config-plane-4)# instance 0
Specify the instance number to which the fabric card is to be mapped.
Step 5

location <fabric-chassis-number/fabric-card-number>

Example:

sysadmin-vm:0_RP0(config-instance-0)# location F0/FC8
Associate the location of the fabric card in FCC with an instance in a fabric plane.
Step 6

Repeat step 3 to step 5 to associate all fabric cards to an instance in the fabric plane.

What to do next

Perform preliminary checks to verify that the fabric card chassis is configured correctly:

  • After associating the fabric card in FCC to a fabric plane, verify the mapping of fabric plane to an instance. The following is an example of a topology with two instances: 
                                 
sysadmin-vm:0_RP0# show running-config controller fabric

controller fabric plane 0
  instance 0
    location F0/FC0
  instance 1
    location F1/FC0
  !
!
controller fabric plane 1
   instance 0
      location F0/FC1
   instance 1
      location F1/FC1
   !
!
controller fabric plane 2
  instance 0
    location F0/FC2
  instance 1
    location F1/FC2
  !
!
controller fabric plane 3
   instance 0
      location F0/FC3
   instance 1
      location F1/FC3
   !
!
controller fabric plane 4
  instance 0
    location F0/FC4
  instance 1
    location F1/FC4
  !
!
controller fabric plane 5
   instance 0
      location F0/FC5
   instance 1
      location F1/FC5
   !
!
  • After all the fabric cards are mapped to an instance in the plane, verify that all the fabric planes are up. For details, see Verify Fabric Plane.
  • Verify the switch summary. For details, see Verify Control Ethernet Connections

The multi-chassis system is setup.