- Information About FabricPath Switching
- Configuring FabricPath Switching
- Installing the FabricPath Feature Set
- Uninstalling the FabricPath Feature Set
- Enabling the FabricPath Feature Set
- Disabling the FabricPath Feature Set
- Creating a FabricPath Topology
- Changing a FabricPath Switch ID (Optional)
- Configuring the FabricPath Timers (Optional)
- Disabling FabricPath Graceful Merges (Optional)
- Forcing the Links to Come Up (Optional)
- Verifying the FabricPath Switching Configuration
- Monitoring FabricPath Switching Statistics
- Configuration Example for FabricPath Switching
- Default Settings for FabricPath Switching
- Feature History for Configuring FabricPath Switching
Configuring FabricPath Switching
This chapter describes how to configure FabricPath switching on the Cisco NX-OS 5500 Series switches.
This chapter includes the following sections:
- Information About FabricPath Switching
- Configuring FabricPath Switching
- Verifying the FabricPath Switching Configuration
- Monitoring FabricPath Switching Statistics
- Configuration Example for FabricPath Switching
- Default Settings for FabricPath Switching
- Feature History for Configuring FabricPath Switching
Note For information about prerequisites, guidelines and limitations, and licensing requirements for FabricPath, see Chapter1, “Overview”
Information About FabricPath Switching
FabricPath switching allows multipath networking at the Layer 2 level. The FabricPath network still delivers packets on a best-effort basis (which is similar to the Classical Ethernet [CE] network), but the FabricPath network can use multiple paths for Layer 2 traffic. In a FabricPath network, you do not need to run the Spanning Tree Protocol (STP). Instead, you can use FabricPath across data centers, some of which have only Layer 2 connectivity, with no need for Layer 3 connectivity and IP configurations.
The FabricPath encapsulation facilitates MAC mobility and server virtualization, which means that you can physically move the Layer 2 node but retain the same MAC address and VLAN association for the virtual machine. FabricPath also allows LAN extensions across data centers at Layer 2, which is useful in disaster recovery operations, as well as clustering applications such as databases. FabricPath is very useful in high-performance, low-latency computing.
With FabricPath, you use the Layer 2 Intermediate System-to-Intermediate System (IS-IS) protocol for a single control plane that functions for unicast, broadcast, and multicast packets. There is no need to run the Spanning Tree Protocol (STP); it is a strictly a Layer 2 domain. This FabricPath Layer 2 IS-IS is a separate process from Layer 3 IS-IS.
Cisco supports the conversation-based MAC learning schema. Conversational learning can only be applied to FabricPath VLANs. Using FabricPath and conversational MAC address learning, the switch has to learn far fewer MAC addresses, which results in smaller, more manageable MAC tables.
This section includes the following topics:
- FabricPath Encapsulation
- Default IS-IS Behavior with FabricPath
- Conversational MAC Address Learning
- Switching Using FabricPath
- Conflict Resolution and Optional FabricPath Tunings
FabricPath Encapsulation
FabricPath Headers
When a frame enters the FabricPath network, the software encapsulates the Layer 2 frame with a new FabricPath header. The switch ID that the software assigns to each FabricPath switch as it enters the FabricPath network is used as the MAC outer destination address (ODA) and MAC outer source address (OSA) in the FabricPath header. Figure 2-1 shows the FabricPath header encapsulation of the classical Ethernet (CE) frame.
Figure 2-1 FabricPath Frame Encapsulation
The software applies the encapsulation on the ingressing edge port of the FabricPath network and decapsulates the frame on the egressing edge port of the FabricPath network; all the ports within the FabricPath network are FabricPath ports that use only the hierarchical MAC address (see Chapter 3, “Configuring FabricPath Interfaces”, for more information on configuring FabricPath interfaces). This feature greatly reduces the size of the MAC tables in the core of the FabricPath network.
The software automatically assigns each switch in the FabricPath network with a unique switch ID. Optionally, you can configure the switch ID for the FabricPath switch.
The outer source address (OSA) is the FabricPath switch ID of the switch where the frame ingresses the FabricPath network, and the outer destination address (ODA) is the FabricPath switch ID of the switch where the frame egresses the FabricPath network. When the frame egresses the FabricPath network, the FabricPath switch strips the FabricPath header, and the original CE frame continues on the CE network. The FabricPath network uses only the OSA and ODA, with the Layer 2 IS-IS protocol transmitting the topology information. Both the FabricPath ODA and OSA are in a standard MAC format (xxxx.xxxx.xxxx).
The FabricPath hierarchical MAC address carries the reserved EtherType 0x8903.
When the frame is originally encapsulated, the software sets the Time to Live (TTL) to 32; on each hop through the FabricPath network, each switch decrements the TTL by 1. If the TTL reaches 0, that frame is discarded. This feature prevents the continuation of any loops that might form in the network.
Forwarding Tags (FTags)
The Forwarding Tag (FTag) in the FabricPath header specifies which path the packet traverses throughout the FabricPath network. The switch uses the FTag-specified paths for multidestination packets that enter the FabricPath network. The FTag, which is a fixed route that the switch learns from the topology, is a 10-bit field with the values from 1 to 1024 (see “Configuring FabricPath Forwarding,” for more information).
The software assigns the FTag to the FabricPath header on the edge port as the frame ingresses the FabricPath network. All subsequent FabricPath switches in that FabricPath network recognize the FTag. Each FTag is unique within the FabricPath topology.
Default IS-IS Behavior with FabricPath
The interfaces in a FabricPath network run only the FabricPath Layer 2 IS-IS protocol; you do not need to run STP in the FabricPath network because FabricPath Layer 2 IS-IS discovers topology information dynamically.
Note Cisco Nexus 5500 Series switches supports two FabricPath topologies, the default or base topology (topology 0), and another optional topology (for example, topology 1).
FabricPath Layer 2 IS-IS is a dynamic link-state routing protocol that detects changes in the network topology and calculates loop-free paths to other nodes in the network. Each FabricPath switch maintains a link-state database (LSDB) that describes the state of the network; each switch updates the status of the links that are adjacent to the switch. The FabricPath switch sends advertisements and updates to the LSDB through all the existing adjacencies. FabricPath Layer 2 IS-IS protocol packets do not conflict with standard Layer 2 IS-IS packets because the FabricPath packets go to a different Layer 2 destination MAC address than that used by standard IS-IS for IPv4 and IPv6 address families.
The software sends hello packets on the FabricPath core ports to form adjacencies. After the software forms IS-IS adjacencies, the FabricPath unicast traffic uses the equal cost multipath (ECMP) feature of Layer 2 IS-IS to forward traffic. ECMP provides up to 16 paths for unicast traffic.
Within the FabricPath network, you use a single control plane protocol, Layer 2 IS-IS, for all unicast, multicast, and broadcast traffic. To use the basic FabricPath functionality, you do not need to configure Layer 2 IS-IS. The control plane Layer 2 IS-IS comes up and runs automatically when you enable FabricPath on the switch.
The loop-free Layer 2 IS-IS protocol builds two trees for the topology. One tree carries unknown unicast and broadcast traffic, and the second tree carries multicast traffic. The software load balances the multicast traffic across the two trees (see “Configuring FabricPath Forwarding,” for more information about trees and topology).
FabricPath Layer 2 IS-IS is based on the standard IS-IS protocol with the following extensions for the FabricPath environment:
- FabricPath has a single IS-IS area with no hierarchical Layer 1 and Layer 2 routing as prescribed within the IS-IS standard. All switches within the FabricPath network are in a single Layer 1 area.
- The switch uses a MAC address that is different from the MAC address used for Layer 3 IS-IS instances.
- The software adds a new sub-TLV that carries switch ID information, which is not in standard IS-IS. This feature allows Layer 2 information to be exchanged through the existing IS-IS protocol implementation.
- Each switch computes its shortest path to every other switch in the network using the shortest-path first (SPF) algorithm. This path is used for forwarding unicast FabricPath frames. FabricPath Layer 2 IS-IS uses the standard IS-IS functionality to populate up to 16 routes for a given destination switch. The software uses multiple equal-cost available parallel links that provide ECMP.
- FabricPath IS-IS introduces certain modifications to the standard IS-IS in order to support the construction of broadcast and multicast trees (identified by the FTags). Specifically, using FabricPath, the software constructs two loop-free trees for forwarding multidestination traffic.
Once the adjacency is established among the switches in the FabricPath network, the software sends update information to all neighbors.
By default, you can run Layer 2 IS-IS with FabricPath with the default configuration, However, you can fine-tune some of the Layer 2 IS-IS parameters (see “Configuring Advanced FabricPath Features,” for information on configuring optional IS-IS parameters).
FabricPath IS-IS also helps to ensure that each switch ID in steady-state is unique within the FabricPath network. If FabricPath networks merge, switch IDs might collide. If the IDs are all dynamically assigned, FabricPath IS-IS ensures that this conflict is resolved without affecting any FabricPath traffic in either network.
Conversational MAC Address Learning
In traditional MAC address learning, each switch learns the MAC address of every other host in the network. When you configure a VLAN for conversational learning, the associated switches learn only those MAC addresses that are actively in conversation with the hosts connected to the edge interfaces of those switches. Not all switches need to learn all the MAC addresses, which greatly reduces the size of the MAC address tables.
You can optimize the MAC learning process. Conversational MAC learning is configured per VLAN. All FabricPath VLANs use conversational learning if a switch virtual interface (SVI) is not enabled on the switch. (See “Configuring FabricPath Forwarding,” for more information on CE and FabricPath VLANs.)
Conversational MAC address learning results in MAC address tables that are much smaller for FabricPath.
The MAC address learning modes are the traditional learning and conversational learning. These VLAN modes have the following MAC learning modes:
- FabricPath VLANs—Conversational MAC learning is enabled unless you enable an SVI.
- CE VLANs—Only traditional learning.
With conversational MAC learning, the switch learns the source MAC address of an ingressing frame only if the destination MAC address is already learned and present in the MAC address table. It requires a three-way handshake for the MAC address to be learned. Unknown MAC address are forwarded, or flooded throughout the network.
Switching Using FabricPath
The FabricPath hierarchical MAC address scheme and conversational learning result in much smaller, conversational learning MAC tables within the FabricPath network. Within the FabricPath network, the software uses Layer 2 IS-IS to transmit topology information. The switches on the edge of the network, which use conversational MAC address learning, do not have to learn all the MAC addresses in the network (see Figure 2-2).
Figure 2-2 FabricPath Ports Use Only the FabricPath Header to Switch Frames
MAC mobility is expedited using the FabricPath hierarchical MAC addresses. That is, when you want to move a host and keep its same MAC address and VLANs, only the switches at the edge of the FabricPath network track this change. Within the FabricPath network, the FabricPath switches update their tables with only the outer MAC addresses (ODA and OSA) that have changed from the FabricPath encapsulation.
The switch on the edge of the FabricPath network encapsulates the original frame inside the FabricPath header. Once the frame reaches the last, or directly connected, FabricPath switch, the egress interface strips the FabricPath header and forwards the frame as a normal CE frame.
The ports at the edge of a FabricPath network can use conversational learning to learn only those MAC addresses that the specified edge port is having a bidirectional conversation with. Every edge switch does not have to learn the MAC address of every other edge switch; they only need to learn the MAC addresses of the speakers.
As the frame traverses the FabricPath network, all the switches work only with the FabricPath header. The FabricPath switches work only with the ODAs and OSAs; they do not need to learn the MAC address for any of the CE hosts or other switches attached to the network. The hierarchical MAC addressing provided by the FabricPath headers results in much smaller MAC tables in the FabricPath network. The switches in the FabricPath network only need to know how to forward frames to another FabricPath switch so they can forward traffic without requiring large MAC address lookup tables in the core of the network.
The switches in the FabricPath network decrement the TTL in the FabricPath header by 1 at each hop. When the TTL reaches 0, the packet is dropped. This process prevents the continuation of any loops that might form in the network.
Conflict Resolution and Optional FabricPath Tunings
After you enable FabricPath in all switches, the software automatically assigns a random switch ID to each FabricPath switch. The switch ID is a 12-bit value that is dynamically assigned to every switch in the FabricPath network, with each switch being a unique value in that FabricPath network. Optionally, you can configure a specific switch ID. If any of the switch IDs in the FabricPath network are not unique, the software provides automatic conflict resolution.
The software chooses a random value for the switch ID and sets this value as tentative during a period when the software waits to hear if this value is already in use. If this value is being used by another switch in the network, the software begins a conflict resolution process. The switch with the lower switch ID keeps the specified value and the other switch gets a new value for its switch ID.
If a single switch joins an existing FabricPath network, that switch changes its switch ID value rather than any of the existing switches in the network changing values. If the specified value is not in use by another switch or after the conflict is resolved, the switch ID is marked as confirmed.
Graceful migration provides that there is no traffic disruption if a conflict arises in the resources, such as two switches that temporarily have the same switch ID.
The FabricPath interfaces come up, but they are not operational until the switch checks for FabricPath conflicts and resolves those conflicts.
The FabricPath resources timers have default values, but you can also change the timer values. You can tune the switch to wait longer or shorter periods to check the conflicts.
Some of the important processes of the FabricPath network are as follows:
- Achieves a conflict-free allocation of switch IDs and FTags.
- Provides graceful resource migration during network merges or partition healing.
- Supports static switch IDs.
- Provides fast convergence during link bringup or network merges.
FabricPath uses the Layer 2 IS-IS protocol to transport the database to all switches in the network. The information is distributed among the FabricPath network switches using an IS-IS TLV. Each switch sends its version of the database that contains information about all the switches.
The software allocates the FabricPath values, guarantees their uniqueness within the FabricPath network, and deletes the value from the database once that resource is no longer needed.
When you manually configure static switch IDs for the switch, the automatic conflict resolution process does not work and the network does not come up. You will see syslog messages about the conflict and must manually change one or more switch IDs of the switches in the network.
Configuring FabricPath Switching
After you enable FabricPath switching on each switch, the encapsulation, default IS-IS, and learning occur automatically.
Note You must install the FabricPath feature set before you enable FabricPath on the switch. See the “Enabling the FabricPath Feature Set” section for complete information on installing and enabling the FabricPath feature set.
Instead of using the default values, you can optionally configure the following FabricPath values that the software uses for conflict resolution and other tunings:
- Switch ID for the switch that is used globally in the FabricPath network
- Timers
- Graceful merge of FabricPath networks
(Enabled by default. You might experience traffic drops if the feature is disabled.) - A one-time forcing of the links to come up
This section includes the following topics:
- Installing the FabricPath Feature Set
- Uninstalling the FabricPath Feature Set
- Enabling the FabricPath Feature Set
- Disabling the FabricPath Feature Set
- Creating a FabricPath Topology
- Configuring the FabricPath Timers (Optional)
- Disabling FabricPath Graceful Merges (Optional)
- Forcing the Links to Come Up (Optional)
Installing the FabricPath Feature Set
You must install the FabricPath feature set on all switches on which you want to configure FabricPath.
BEFORE YOU BEGIN
Ensure that you have installed an Enhanced Layer 2 license on all switches on which you want to install FabricPath.
SUMMARY STEPS
2. install feature-set fabricpath
DETAILED STEPS
|
|
|
---|---|---|
(Optional) Displays which feature sets are enabled on the switch. |
||
(Optional) Copies the running configuration to the startup configuration. |
This example shows how to install the FabricPath feature set on a switch:
Uninstalling the FabricPath Feature Set
BEFORE YOU BEGIN
Ensure that you have disabled the FabricPath feature set on all switches from which you want to uninstall FabricPath. See the “Disabling the FabricPath Feature Set” section for more information.
SUMMARY STEPS
2. no install feature-set fabricpath
DETAILED STEPS
|
|
|
---|---|---|
(Optional) Displays which feature sets are enabled on the switch. |
||
(Optional) Copies the running configuration to the startup configuration. |
This example shows how to uninstall the FabricPath feature set from a switch:
Enabling the FabricPath Feature Set
You must enable the FabricPath feature set on all switches on which you want to run FabricPath. You cannot access the commands that you use to configure the feature until the feature set is installed and enabled.
BEFORE YOU BEGIN
Ensure that you have installed the FabricPath feature on the switch.
SUMMARY STEPS
DETAILED STEPS
|
|
|
---|---|---|
(Optional) Displays which feature sets are enabled on the switch. |
||
(Optional) Copies the running configuration to the startup configuration. |
This example shows how to enable the FabricPath feature set on a switch:
Disabling the FabricPath Feature Set
When you disable the FabricPath feature set, the switch clears all FabricPath configurations from the switch and you can no longer see any of the CLI commands that you need to configure FabricPath.
Note If your FabricPath configuration is large (multiple megabytes in size), disabling the FabricPath feature set may take some time to complete.
SUMMARY STEPS
DETAILED STEPS
|
|
|
---|---|---|
(Optional) Displays which feature sets are enabled on the switch. |
||
(Optional) Copies the running configuration to the startup configuration. |
This example shows how to disable the FabricPath feature:
Creating a FabricPath Topology
You must install the FabricPath feature set on all switches on which you want to configure FabricPath.
SUMMARY STEPS
DETAILED STEPS
This example shows how to configure the FabricPath topology:
Changing a FabricPath Switch ID (Optional)
By default, FabricPath assigns a unique switch ID to a switch after you enable FabricPath on the switch. However, you can manually change the switch ID if you prefer.
Note You do not lose any traffic during switch ID changes.
SUMMARY STEPS
DETAILED STEPS
|
|
|
---|---|---|
Specifies the switch ID. The range is from 1 to 4094. There is no default value. |
||
(Optional) Copies the running configuration to the startup configuration. |
This example shows how to manually change the FabricPath switch ID on a switch to 25:
Configuring the FabricPath Timers (Optional)
You can configure the following FabricPath timers:
- allocate-delay—Configures the delay for a new switch ID to be propagated throughout the network before that value becomes available and permanent.
- linkup-delay—Configures the link bringup delay to detect conflicts in the switch ID. If the software does find a conflict, the software takes some time to resolve the conflict and bring FabricPath to an operational state.
- transition-delay—Configures the delay for propagating a transitioned value in the network; during this period, all old and new switch ID values exist in the network. This situation occurs only while the link comes up and the software checks to see if the network has two identical switch IDs.
SUMMARY STEPS
2. fabricpath timers { allocate-delay seconds | linkup-delay seconds | transition-delay seconds }
DETAILED STEPS
This example shows how to configure the allocation-delay for a FabricPath timer to 600 seconds:
Disabling FabricPath Graceful Merges (Optional)
By default, graceful merges are enabled. If desired, you can disable this aspect of the FabricPath feature. However, if you disable graceful merges, you must do so on all FabricPath switches.
Note You may experience traffic drops if you disable this feature.
SUMMARY STEPS
2. [no] fabricpath graceful-merge disable
DETAILED STEPS
This example shows how to disable graceful merges on a FabricPath switch:
Forcing the Links to Come Up (Optional)
Note We do NOT recommend that you use the fabricpath force link-bringup command to force the network links to connect.
As a one-time event, you can force the FabricPath network links to connect if they are not coming up because of switch ID conflicts or other problems in the network.
You must repeat this configuration on every switch in the FabricPath network.
DETAILED STEPS
|
|
|
---|---|---|
Forces the FabricPath network links to come up as a one-time event. Note This command is not saved when you enter the copy running-config startup-config command. |
This example shows how to force the FabricPath network links to come up as a one-time event:
Verifying the FabricPath Switching Configuration
To display FabricPath switching information, perform one of the following tasks:
See “Configuring Advanced FabricPath Features,” for more commands that display the FabricPath switching functionality.
Monitoring FabricPath Switching Statistics
Use the following commands to display or clear the FabricPath switching statistics:
Configuration Example for FabricPath Switching
Step 1 Install FabricPath on all the switches.
Step 2 Enable FabricPath on all the switches.
Step 3 (Optional) Manually configure a switch ID for the FabricPath switch.
Step 4 Save the configuration.
Default Settings for FabricPath Switching
Table 2-1 lists the default settings for FabricPath switching parameters.
|
|
---|---|
Feature History for Configuring FabricPath Switching
Table 2-2 lists the release history for these features.
|
|
|
---|---|---|