Overview
This chapter provides an overview of the Cisco Nexus 2000 Series Fabric Extender and includes the following sections:
About the Cisco Nexus 2000 Series Fabric Extender
The Cisco Nexus 2000 Series Fabric Extender is a new category of data center products that provides a highly scalable and flexible server networking solution. It works in conjunction with the Cisco Nexus 5000 Series switches to provide high-density, low-cost connectivity for server aggregation. Scaling across 1-Gigabit Ethernet, 10-Gigabit Ethernet, unified fabric, rack, and blade server environments, the Fabric Extender is designed to simplify data center architecture and operations.
The Fabric Extender integrates with its parent switch, the Cisco Nexus 5000 Series switch, to allow automatic provisioning and configuration taken from the settings on the parent switch. This integration allows large numbers of servers and hosts to be supported using the same feature set as the parent switch, including security and quality of service (QoS) configuration parameters, with a single management domain as shown in Figure 1-1. The Fabric Extender and its parent switch enable a large multi-path, loop-free, active-active data center topology without the use of Spanning Tree Protocol (STP).
Figure 1-1 Single Management Domain
The Cisco Nexus 2148T Fabric Extender forwards all traffic to its parent Cisco Nexus 5000 Series switch over 10-Gigabit Ethernet fabric uplinks, allowing all traffic to be inspected by policies established on the Cisco Nexus 5000 Series switch.
No software is included with the Cisco Nexus 2148T. Software is automatically downloaded and upgraded from its parent switch. For details, see the “Fabric Extender Image Management” section.
Fabric Extender Terminology
Some terms used in this document are as follows:
- Fabric interface—A 10-Gigabit Ethernet uplink port designated for connection from the Fabric Extender to its parent switch. A fabric interface cannot be used for any other purpose. It must be directly connected to the parent switch.
Note A fabric interface includes the corresponding interface on the parent switch. This interface is enabled when you enter the switchport mode fex-fabric command.
- EtherChannel fabric interface—An EtherChannel uplink connection from the Fabric Extender to its parent switch. This connection consists of fabric interfaces bundled into a single logical channel.
- Host interface—An Ethernet host interface for connection to a server or host system. These ports are 1-Gigabit Ethernet interfaces.
Note Do not connect a bridge or switch to a host interface. These interfaces are designed to provide end host or server connectivity.
Oversubscription
In a switching environment, oversubscription is the practice of connecting multiple devices to the same interface to optimize port usage. An interface can support a connection that runs at its maximum speed but because most interfaces are not run their maximum speed, you can take advantage of unused bandwidth by sharing ports. In the case of the Cisco Nexus 2000 Series Fabric Extender, oversubscription, which is a function of the available fabric interfaces to active host interfaces, provides cost-effective scalability and flexiblity for Ethernet environments.
The Cisco Nexus 2148T Fabric Extender has four 10-Gigabit Ethernet fabric interfaces and 48 1-Gigabit BaseT Ethernet host interfaces. With this system, you can have any number configurations. For example, you can configure:
Management Model
The Cisco Nexus 2000 Series Fabric Extender is managed by its parent switch over the fabric interfaces through a zero-touch configuration model. The Fabric Extender is discovered by the switch by detecting the fabric interfaces of the Fabric Extender.
After discovery, if the Fabric Extender has been correctly associated with the parent switch, the following steps are performed:
1.
The switch checks the software image compatibility and upgrades the Fabric Extender if necessary.
2. The switch and Fabric Extender establish in-band IP connectivity with each other. The switch assigns the Fabric Extender an IP address in the range of loopback addresses (127.0.0.0/8), to avoid conflicts with IP addresses that may be in use on the network.
3. The switch pushes the configuration data to the Fabric Extender. The Fabric Extender does not store any configuration locally.
4. The Fabric Extender updates the switch with its operational status. All Fabric Extender information is displayed using the switch commands for monitoring and troubleshooting.
Note In Cisco NX-OS Release 4.0(1a)N2(1), a Cisco Nexus 2000 Series Fabric Extender can only be managed by one parent switch.
Forwarding Model
The Cisco Nexus 2000 Series Fabric Extender does not perform any local switching. All traffic is sent to the parent switch that provides central forwarding and policy enforcement, including host-to-host communications between two systems connected to the same Fabric Extender as shown in Figure 1-2.
The forwarding model facilitates feature consistency between the Fabric Extender and its parent switch.
Note The Fabric Extender provides end-host connectivity into the network fabric. As a result, Bridge Protocol Data Unit (BPDU) Guard is enabled on all its host interfaces. If you connect a bridge or switch to a host interface, that interface is placed in an error-disabled state when a BPDU is received.
You cannot disable BPDU Guard on the host interfaces of the Fabric Extender.
The Fabric Extender supports egress multicast replication from the network to the host. Packets sent from the parent switch for multicast addresses attached to the Fabric Extender are replicated by the Fabric Extender ASICs and then sent to corresponding hosts.
Connection Model
Two methods (the static pinning fabric interface connection and the EtherChannel fabric interface connection) allow the traffic from an end host to the parent switch to be distributed when going through the Cisco Nexus 2000 Series Fabric Extender. This section describes the following topics:
Static Pinning Fabric Interface Connection
To provide a deterministic relationship between the host interfaces and the parent switch, you can configure the Fabric Extender to use individual fabric interface connections. This configuration connects the 10-Gigabit Ethernet fabric interfaces as shown in Figure 1-3. You can use any number of fabric interfaces up to the maximum available on the model of the Fabric Extender.
Figure 1-3 Static Pinning Fabric Interface Connections
When the Fabric Extender is brought up, its host interfaces are distributed equally among the available fabric interfaces. As a result, the bandwidth that is dedicated to each end host towards the parent switch is never changed by the switch but instead is always specified by you.
Note If a fabric interface fails, all its associated host interfaces are brought down and remain down until the fabric interface is restored.
You must use the pinning max-links command to create a number of pinned fabric interface connections so that the parent switch can determine a distribution of host interfaces. The host interfaces are divided by the number of the max-links and distributed accordingly. The default value is max-links 1.
The pinning order of the host interfaces is initially determined by the order in which the fabric interfaces were configured. When the parent switch is restarted, the configured fabric interfaces are pinned to the host interfaces in an ascending order by the port number of the fabric interface.
To guarantee a deterministic and sticky association across a reboot, you can manually redistribute the pinning. See the “Redistributing the Links” section.
Note The redistribution of the host interfaces will always be in an ascending order by the port number of the fabric interface.
EtherChannel Fabric Interface Connection
To provide load balancing between the host interfaces and the parent switch, you can configure the Fabric Extender to use an EtherChannel fabric interface connection. This connection bundles 10-Gigabit Ethernet fabric interfaces into a single logical channel as shown in Figure 1-4.
Figure 1-4 EtherChannel Fabric Interface Connection
When you configure the Fabric Extender to use an EtherChannel fabric interface connection to its parent switch, the switch load balances the traffic from the hosts that are connected to the host interface ports by using the following load-balancing criteria to select the link:
- For a Layer 2 frame, the switch uses the source and destination MAC addresses.
- For a Layer 3 frame, the switch uses the source and destination MAC addresses and the source and destination IP addresses.
Note A fabric interface that fails in the EtherChannel will not trigger a change to the host interfaces. Traffic is automatically redistributed across the remaining links in the EtherChannel fabric interface.
Port Numbering Convention
The following port numbering convention is used for the Fabric Extender:
interface ethernet chassis / slot / port
- chassis is configured by the administrator. A Fabric Extender must be directly connected to its parent switch via individual fabric interfaces or an EtherChannel fabric interface. You configure a chassis ID on a physical Ethernet interface or EtherChannel on the switch to identify the Fabric Extender discovered via those interfaces.
The chassis ID ranges from 100 to 199.
Note The chassis ID is required only to access a host interface on the Fabric Extender. A value of less than 100 indicates a slot on the parent switch.
Fabric Extender Image Management
No software ships with the Cisco Nexus 2000 Series Fabric Extender. The Fabric Extender image is bundled into the system image of the parent switch. The image is automatically verified and, if required, updated during the association process between the switch and the Fabric Extender.
When you enter the install all command, it upgrade the software on the parent Cisco Nexus 5000 Series switch and also upgrades the software on any attached Fabric Extender. To minimize downtime as much as possible, the Fabric Extender remains online while the installation process loads its new software image. Once the software image has successfully loaded, the parent switch and the Fabric Extender both automatically reboot. This process is required to maintain version compatibility between the switch and the Fabric Extender.
Fabric Extender Hardware
The Cisco Nexus 2000 Series Fabric Extender architecture allows hardware configurations with various host interface counts and speeds.
This section describes the hardware of the Fabric Extender and includes the following topics:
Feedback