A virtual port channel+ (vPC+) is an extension to virtual port channels (vPCs) that run CE only. A vPC+ domain allows a classical Ethernet (CE) vPC domain and a Cisco FabricPath cloud to interoperate and also provides a First Hop Routing Protocol (FHRP) active-active capability at the FabricPath to Layer 3 boundary. A vPC+ domain enables Cisco Nexus 7000 Series enabled with FabricPath devices to form a single vPC+, which is a unique virtual switch to the rest of the FabricPath network. For more detailed information on vPC+ see the Cisco Nexus 7000 Series NX-OS FabricPath Configuration Guide.

Note	You cannot configure a vPC+ domain and a vPC domain in the same VDC.

You can use only Layer 2 port channels in the vPC. A vPC domain is associated to a single Virtual Device Context (VDC), so all vPC interfaces belonging to a given vPC domain must be defined in the same VDC. You must have a separate vPC peer link and peer-keepalive link infrastructure for each VDC deployed. Consolidating a vPC pair (two vPC peer devices of the same domain) in two VDCs of the same physical device is not supported. The vPC peer link must use at least 10-Gigabit Ethernet ports for both ends of the link or the link will not form.

You configure the port channels by using one of the following:

• No protocol

• Link Aggregation Control Protocol (LACP)

When you configure the port channels in a vPC—including the vPC peer link channel—without using LACP, the F-series line cards can have 16 active links and M-series line cards can have 8 active links in a single port channel. When you configure the port channels in a vPC—including the vPC peer link channels—using LACP, F-series card each device can have eight active links and eight standby links in a single port channel. (See the “vPC Interactions with Other Features” section for more information on using LACP and vPCs.)

You can use the lacp graceful-convergence command to configure port channel Link Aggregation Control Protocol (LACP) graceful convergence. You can use this command only on a port-channel interface that is in an administratively down state. You cannot configure (or disable) LACP graceful convergence on a port channel that is in an administratively up state.

You can use the lacp suspend-individual command to enable LACP port suspension on a port channel. LACP sets a port to the suspended state if it does not receive an LACP bridge protocol data unit (BPDU) from the peer ports in a port channel. This can cause some servers to fail to boot up as they require LACP to logically bring up the port.

Note	You must enable the vPC feature before you can configure or run the vPC functionality.

From Cisco NX-OS Release 4.2, the system automatically takes a checkpoint prior to disabling the feature, and you can roll back to this checkpoint. See the Cisco Nexus 7000 Series NX-OS System Management Configuration Guide for information about rollbacks and checkpoints.

After you enable the vPC functionality, you create the peer-keepalive link, which sends heartbeat messages between the two vPC peer devices.

You can create a vPC peer link by configuring a port channel on one Cisco Nexus 7000 Series chassis by using two or more 10-Gigabit Ethernet ports in dedicated port mode. To ensure that you have the correct hardware to enable and run a vPC from Cisco NX-OS Release 4.1(5), enter the show hardware feature-capability command. If you see an X across from the vPC in your command output, your hardware cannot enable the vPC feature.

We recommend that you configure the vPC peer link Layer 2 port channels as trunks. On another Cisco Nexus 7000 Series chassis, you configure another port channel again using two or more 10-Gigabit Ethernet ports in the dedicated port mode. Connecting these two port channels creates a vPC peer link in which the two linked Cisco Nexus devices appear as one device to a third device. The third device, or downstream device, can be a switch, server, or any other networking device that uses a regular port channel to connect to the vPC. If you are not using the correct module, the system displays an error message.

Note	We recommend that you configure the vPC peer links on dedicated ports of different modules to reduce the possibility of a failure. For the best resiliency scenario, use at least two modules.

From Cisco NX-OS Release 4.2, if you must configure all the vPC peer links and core-facing interfaces on a single module, you should configure a track object that is associated with the Layer 3 link to the core and on all the links on the vPC peer link on both vPC peer devices. Once you configure this feature and if the primary vPC peer device fails, the system automatically suspends all the vPC links on the primary vPC peer device. This action forces all the vPC traffic to the secondary vPC peer device until the system stabilizes.

You can create a track object and apply that object to all links on the primary vPC peer device that connect to the core and to the vPC peer link. See the Cisco Nexus 7000 Series NX-OS Unicast Routing Configuration Guide for information about the track interface command.

The vPC domain includes both vPC peer devices, the vPC peer-keepalive link, the vPC peer link, and all of the port channels in the vPC domain connected to the downstream device. You can have only one vPC domain ID on each device.

In this version, you can connect each downstream device to a single vPC domain ID using a single port channel.

Note	Always attach all vPC devices using port channels to both vPC peer devices.

The terminology used in vPCs is as follows:

• vPC—The combined port channel between the vPC peer devices and the downstream device.

• vPC peer device—One of a pair of devices that are connected with the special port channel known as the vPC peer link.

• vPC peer link—The link used to synchronize states between the vPC peer devices. Both ends must be on 10-Gigabit Ethernet interfaces.

• vPC member port—An interface that belongs to a vPC.

• Host vPC port—A Fabric Extender host interfaces that belongs to a vPC.

• vPC domain—This domain includes both vPC peer devices, the vPC peer-keepalive link, and all of the port channels in the vPC connected to the downstream devices. It is also associated to the configuration mode that you must use to assign vPC global parameters.

• vPC peer-keepalive link—The peer-keepalive link monitors the vitality of a vPC peer Cisco Nexus 7000 Series device. The peer-keepalive link sends configurable, periodic keepalive messages between vPC peer devices.

We recommend that you associate a peer-keepalive link to a separate virtual routing and forwarding (VRF) instance that is mapped to a Layer 3 interface in each vPC peer device. If you do not configure a separate VRF, the system uses the management VRF by default. However, if you use the management interfaces for the peer-keepalive link, you must put a management switch connected to both the active and standby management ports on each vPC peer device (see the figure below).

No data or synchronization traffic moves over the vPC peer-keepalive link; the only traffic on this link is a message that indicates that the originating switch is operating and running a vPC.

• vPC member port—Interfaces that belong to the vPCs.

• Dual-active— Both vPC peers act as primary. This situation occurs when the peer-keepalive and peer-link go down when both the peers are still active. In this case, the secondary vPC assumes that the primary vPC is inactive and acts as the primary vPC.

• Recovery—When the peer-keepalive and the peer-link come up, one switch becomes the secondary vPC. On the switch that becomes the secondary vPC, the vPC links go down and come back up.

A vPC peer link is the link that is used to synchronize the states between the vPC peer devices. Both ends of the link must be on 10-Gigabit Ethernet interfaces.

• Keeps both vPC peer switches synchronized for control plane information (such as the vPC state, consistency parameters, and MAC addresses).

• Forwards data packets to the vPC peer switch, when the local vPC is down.

• A single vPC domain between two VDCs on the same physical Cisco Nexus 7000 device is not supported.

Note	You must configure the peer-keepalive link before you configure the vPC peer link or the peer link does not come up. (See the “Peer-Keepalive Link and Messages” section for information about the vPC peer-keepalive link and messages.)

Note

Starting from Cisco NX-OS Release 8.0(1) you cannot configure vPC peer-link on a port-channel with non-default MTU configuration. The following error message is displayed if you try to configure: ERROR: Cannot configure peer-link since mtu is non-default To configure peer-link, remove the non-default MTU configuration and re apply the vpc peer-link command. By default packets of all sizes are allowed in peer-link.

You can configure a vPC peer link to configure two devices as vPCs peers. You must use the module in order to configure a vPC peer link.

We recommend that you use the dedicated port mode when you configure a vPC peer link. For information about the dedicated port mode, see “Configuring Basic Interface Parameters.”

The Cisco NX-OS software uses the peer-keepalive link between the vPC peers to transmit periodic, configurable keepalive messages. You must have Layer 3 connectivity between the peer devices to transmit these messages; the system cannot bring up the vPC peer link unless the peer-keepalive link is already up and running.

We recommend that you associate the vPC peer-keepalive link to a separate VRF mapped to a Layer 3 interface in each vPC peer device. If you do not configure a separate VRF, the system uses the management VRF and management ports by default. Do not use the peer link itself to send and receive vPC peer-keepalive messages. For more information about configuring VRFs, see the Cisco Nexus 7000 Series NX-OS Unicast Routing Configuration Guide.

If one of the vPC peer devices fails, the vPC peer device on the other side of the vPC peer link senses the failure by not receiving any peer-keepalive messages. You can configure a hold-timeout and a timeout value simultaneously.

Hold-timeout value—The hold-timeout value range is between 3 to 10 seconds, with a default value of 3 seconds. This timer starts when the vPC peer link goes down. The purpose of the hold-timeout period is to prevent false-positive cases.

If you configure a hold-timeout value that is lower than the timeout value, then the vPC system ignores vPC peer-keepalive messages for the hold-timeout period and considers messages for the reminder of the timeout period. If no keepalive message is received for this period, the vPC secondary device takes over the role of the primary device. For example, if the hold-timeout value is 3 seconds and the timeout value is 5 seconds, for the first 3 seconds vPC keepalive messages are ignored (such as, when accommodating a supervisor failure for a few seconds after peer link failure) and keepalive messages are considered for the remaining timeout period of 2 seconds. After this period, the vPC secondary device takes over as the primary device, in case there is no keep alive message.

Timeout value—The timeout value range is between 3 to 20 seconds, with a default value of 5 seconds. This timer starts at the end of the hold-timeout interval. If you configure a timeout value that is lower than or equal to the hold-timeout value, then the timeout duration is initiated after the hold-timeout period. For example, if the timeout value is 3 seconds and the hold-timeout value is 5 seconds, the timeout period starts after 5 seconds

Note	Ensure that both the source and destination IP addresses used for the peer-keepalive messages are unique in your network and these IP addresses are reachable from the VRF associated with the vPC peer-keepalive link.

Use the CLI to configure the interfaces you are using the vPC peer-keepalive messages as trusted ports. Leave the precedence at the default (6) or configure it higher.

(config)# class-map type qos match-all trust-map
(config-cmap-qos)# match cos 4-7

(config)# policy-map type qos ingresspolicy
(config-pmap-qos)# class trust-map

(config)# interface Ethernet8/11
(config-if)# service-policy type qos input ingresspolicy

See the Cisco Nexus 7000 Series NX-OS Quality of Service Configuration Guide for complete information about configuring trusted ports and precedence.

From Cisco NX-OS Release 4.2(1), you can configure vPC peer devices to act as the gateway even for packets that are destined to the vPC peer device’s MAC address.

Use the peer-gateway command to configure this feature.

Note	From Cisco NX-OS Release 6.2(2), you can use the mode auto command to automatically enable this feature. See the “Enabling Certain vPC Commands Automatically” section for more information about using this command.

Some network-attached storage (NAS) devices or load balancers might have features that help to optimize the performances of particular applications. These features enable the device to avoid a routing-table lookup when responding to a request that originated from a host that is not locally attached to the same subnet. Such devices might reply to traffic using the MAC address of the sender Cisco Nexus 7000 Series and Cisco Nexus 7700 Series devices rather than the common HSRP gateway. This behavior is noncompliant with some basic Ethernet RFC standards. Packets that reach a vPC device for the nonlocal router MAC address are sent across the peer link and could be dropped by the built in vPC loop avoidance mechanism if the final destination is behind another vPC.

The vPC peer-gateway capability allows a vPC switch to act as the active gateway for packets that are addressed to the router MAC address of the vPC peer. This feature enables local forwarding of packets without the need to cross the vPC peer link. In this scenario, the feature optimizes use of the peer link and avoids potential traffic loss.

Configuring the peer-gateway feature must be done on both primary and secondary vPC peers and is nondisruptive to the operations of the device or to the vPC traffic. The vPC peer-gateway feature can be configured globally under the vPC domain submode.

When you enable this feature, Cisco NX-OS automatically disables IP redirects on all interface VLANs mapped over a vPC VLAN to avoid generation of IP redirect messages for packets switched through the peer gateway router.

Note

From Cisco NX-OS Release 5.1(3) and above, when a VLAN interface is used for Layer 3 backup routing on the vPC peer devices and an F1 line card is used as the peer link, the VLAN must be excluded from the peer-gateway feature, if enabled, by running the peer-gateway exclude-vlan vlan-number command. For more information about backup routes, see the “Configuring Layer 3 Backup Routes on a vPC Peer Link” section.

Packets that arrive at the peer-gateway vPC device have their Time to Live (TTL) decremented, so that packets carrying a TTL of 1 might get dropped in transit due to TTL expiration. You should take this situation into account when the peer-gateway feature is enabled and particular network protocols that source packets with a TTL of 1 operate on a vPC VLAN.

Dynamic Routing over vPC

Dynamic Routing over vPC feature is supported on F2E, F3, and M3 series modules (for IPv4 and IPv6 Unicast traffic). From Cisco NX-OS Release 8.4(1), the dynamic routing over vPC feature is supported on F4 Series modules.

This feature enables L3 routing protocols such as OPSF to form adjacency with the two vPC peer chassis. The equal routing cost matrices must be configured on applicable interface on each of the vPC peers, failure to do so can result in blocking the traffic. Asymmetric routing feature has to be implemented to address this issue and to configure Dynamic Routing over vPC. Additionally, when Dynamic Routing over vPC is enabled a warning log message is printed.

This section describes the Layer 3 over vPC for F2E, F3 and M3 Modules feature and how to configure it. Starting from Cisco NX-OS Release 7.2(0)D1(1), Layer 3 over vPC is available on F2E and F3 Series modules. Using this feature, a Layer 3 device can form peering adjacency between both the vPC peers in a vPC complex. vPC peers must have identical VLANs. The TTL of the traffic sent over a peer link does not decrement. The peer-gateway feature should be enabled on all I/O modules before configuring the Layer 3 over vPC feature. The peer-gateway feature allows the vPC peer (SVI-X) (refer the figure below) to forward packets on behalf of other peer (SVI-Y). This feature saves bandwidth by avoiding traffic over the peer link. You can set up peer adjacency between Layer 3 device and vPC peer without separate Layer 3 links. Both bridged and routed traffic can flow over the same link.

Routing adjacency between Layer 3 device and vPC peer is formed without a non-vPC VLAN. Adjacency is formed on the vPC VLAN. Routing adjacency between a Layer 3 device and a vPC peer is formed without Layer 3 inter-switch links between the vPC peers. Adjacency is formed on the vPC peer-link. There is faster convergence when a link or device fails for all traffic. vPC loop avoidance mechanism is available for all traffic.

The following are figures illustrates the Layer 3 over VPC Support in Cisco NX-OS Release 7.2(0)D1(1):

The FEX is connected to Nexus in straight-through topology. The router peers with both Nexus boxes over satellite ports. Layer 3 over vPC in FEX Active-Active mode vPC is not supported.

Peering with vPC peers over vPC+ interfaces is unsupported.

You can use the vPC domain ID to identify the vPC peer links and the ports that are connected to the vPC downstream devices.

The vPC domain is also a configuration mode that you use to configure the keepalive messages and other vPC peer link parameters rather than accept the default values. See the “Configuring vPCs” section for more information about configuring these parameters.

To create a vPC domain, you must first create a vPC domain ID on each vPC peer device using a number from 1 to 1000. You can have only one vPC domain per VDC.

You must explicitly configure the port channel that you want to act as the peer link on each device. You associate the port channel that you made a peer link on each device with the same vPC domain ID to form a single vPC domain. Within this domain, the system provides a loop-free topology and Layer 2 multipathing.

You can only configure these port channels and vPC peer links statically. All ports in the vPC on each of the vPC peer devices must be in the same VDC. You can configure the port channels and vPC peer links either using LACP or no protocol. We recommend that you use LACP with the interfaces in active mode to configure port channels in each vPC, which ensures an optimized, graceful recovery in a port-channel failover scenario and provides configuration checks against configuration mismatches among the port channels themselves.

The vPC peer devices use the vPC domain ID that you configure to automatically assign a unique vPC system MAC address. Each vPC domain has a unique MAC address that is used as a unique identifier for the specific vPC-related operations, although the devices use the vPC system MAC addresses only for link-scope operations, such as LACP. We recommend that you create each vPC domain within the contiguous Layer 2 network with a unique domain ID. You can also configure a specific MAC address for the vPC domain, rather than having the Cisco NX-OS software assign the address.

See the “CFSoE” section for more information about displaying the vPC MAC table. After you create a vPC domain, the Cisco NX-OS software creates a system priority for the vPC domain. You can also configure a specific system priority for the vPC domain.

Note	When manually configuring the system priority, you must ensure that you assign the same priority value on both vPC peer devices. If the vPC peer devices have different system priority values, vPC does not come up.

The figure below shows a basic configuration in which the Cisco Nexus 7000 Series device ports are directly connected to another switch or host and are configured as part of a port channel that becomes part of a vPC.

In the figure, vPC 20 is configured on port channel 20, which has Eth1/10 on the first device and Eth2/1 on the second as member ports.

From Cisco NX-OS Release 5.2(1), you can configure a vPC from the peer devices through Fabric Extenders (FEXs), as shown in the figure below.

In the figure, each FEX is single-homed (straight-through FEX topology) with a Cisco Nexus 7000 Series device. The host interfaces on this FEX are configured as port channels and those port channels are configured as vPCs. Eth100/1/1 and Eth102/1/5 are configured as members of PO200, and PO200 is configured for vPC 200.

In both topologies, port channels P020 and P0200 must be configured identically on the peer switches and configuration synchronization is used to synchronize the configurations of the vPC switches. See Cisco Nexus 2000 Series Fabric Extender Software Configuration Guide for Cisco Nexus 7000 Series Switches, Release 7.x for more information about configuring FEX ports.

Many configuration and operational parameters must be identical on all interfaces in the vPC. We recommend that you configure the Layer 2 port channels that you use for the vPC peer link in trunk mode.

After you enable the vPC feature and configure the peer link on both vPC peer devices, Cisco Fabric Services (CFS) messages provide a copy of the configuration on the local vPC peer device configuration to the remote vPC peer device. The system then determines whether any of the crucial configuration parameters differ on the two devices. (See the “CFSoE” section for more information about CFS.)

Note	Enter the show vpc consistency-parameters command to display the configured values on all interfaces in the vPC. The displayed configurations are only those configurations that would limit the vPC peer link and vPC from coming up.

The compatibility check process for vPCs differs from the compatibility check for regular port channels.

Once you have created the vPC domain ID and the vPC peer link, you create port channels to attach the downstream device to each vPC peer device. That is, you create one port channel to the downstream device from the primary vPC peer device and you create another port channel to the downstream device from the secondary peer device.

Note	We recommend that you configure the ports on the downstream devices that connect to a host or a network device that is not functioning as a switch or a bridge as STP edge ports. See the Cisco Nexus 7000 Series NX-OS Layer 2 Switching Configuration Guide for more information about STP port types.

On each vPC peer device, you assign a vPC number to the port channel that connects to the downstream device. You will experience minimal traffic disruption when you are creating vPCs. To simplify the configuration, you can assign the vPC ID number to every port channel to be the same as the port channel itself (that is, vPC ID 10 for port channel 10).

Note	The vPC number that you assign to the port channel that connects to the downstream device from the vPC peer device must be identical on both vPC peer devices.

The vPC Shutdown feature enables a user to isolate a switch from a vPC complex before it is debugged, reloaded, or even removed physically, so that the vPC traffic passing through the peer vPC switch in the vPC complex is not affected.

When the user executes the shutdown command, the MCEC module (MCECM) stops sending out-of-band (OOB) keep-alive messages and also brings down all the vPC ports, SVIs, and the peer-link. On detection of the peer-link going down and the non-availability of the keep-alive messages, the peer vPC switch takes over as the primary peer. As the keep-alive messages are not received, the peer vPC switch does not bring up the vPC peer-link even after a flap. The isolated vPC switch keeps all the vPCs down as the peer-link is down. The vPC orphan port suspends configured orphan ports.

When the user executes the no form of this command, the switch is brought back into the vPC complex with minimal disruption of the network traffic. Executing the no form of this command, starts the keepalives, brings up the peer links, and consecutively brings up all the vPCs.

When executed on the primary switch, the shutdown command dual-active status is established.

Orphan ports lose connectivity when the vPC shutdown command is executed.

Cisco NX-OS services saves the shutdown command in the persistent storage service (PSS). The command is restored when the switch reloads. The shutdown command is saved as vPC configuration. The shutdown command executed again along with the vPC configuration, if it has been copied to the startup configuration. The shutdown command is restored when the switch reloads

It is possible that the vPC operating version of an isolated vPC peer switch that comes up after debugging or after an ISSU, is different from that the peer switch. When the no shutdown command is applied, the vPC peer-link comes up with both the switches having as their versions the lower of the two versions.

The STP synchronizes the port states to the vPC peer causing the new primary vPC peer to take over from the current state, when the role switchover happens. If the MCECM take more than 6 seconds to detect the role change and notify the STP, then the STP bridge protocol data units (BPDUs) that are sent on the vPC are timed out. To avoid this, it is recommended to configure STP peer switch feature so that both vPC switches send BPDUs over the vPC ports.

If you configure the shutdown command on the switch to which a dual-homed FEX is connected in a vPC, the FEX goes offline on that switch. An ISSU of the isolated switch does not update the software image on the FEX. You cannot use the vPC shutdown command to perform ISSU by isolating and upgrading each switch for FEX Active-Active.

Consider the following FEX Active-Active scenario where peers Peer 1 and Peer 2 are involved:

• The inactive peer, that is Peer 2, is offline because of reasons such as the VPC shutdown command

• An ISSU has been performed on the active peer, that is Peer 1, for upgrading from one software image version to a higher version

All line cards and the remote line cards, including FEX Active-Active, upgrade to higher version of the software image. This happens because the FEX Active-Active is offline on the inactive peer.

Consecutively, when the inactive peer becomes online due to the VPC no shutdown command, this peer will still run the lower version of the software image. In such as case, the status of FEX Active-Active toggles between AA version mismatch and Offline in this peer. This is because both the peers run different versions of the software image. To avoid this situation, the user should not bring up the Peer 2, or execute the VPC shutdown command on it, until the Peer 2 is also upgraded to higher version software image.

When you execute the shutdown command, the Multichassis EtherChannel Module (MCECM) stops the keep-alive messages and brings down the peer-link. If the vPC peer switch does not receive keep-alive messages in 5 seconds, it assumes the primary role.

Note	You must attach a downstream device using a port channel to both vPC peer devices.

To connect to the downstream device, you create a port channel to the downstream device from the primary vPC peer device and you create another port channel to the downstream device from the secondary peer device. On each vPC peer device, you assign a vPC number to the port channel that connects to the downstream device. You will experience minimal traffic disruption when you are creating vPCs.

Note	We recommend that you configure the vPC peer links on dedicated ports of different modules to reduce the possibility of a failure. For the best resiliency scenario, use at least two modules.

From Cisco NX-OS Release 4.2, if you must configure all the vPC peer links and core-facing interfaces on a single module, you should configure, using the command-line interface, a track object and a track list that is associated with the Layer 3 link to the core and on all vPC peer links on both vPC peer devices. You use this configuration to avoid dropping traffic if that particular module goes down because when all the tracked objects on the track list go down, the system does the following:

• Stops the vPC primary peer device sending peer-keepalive messages, which forces the vPC secondary peer device to take over.

• Brings down all the downstream vPCs on that vPC peer device, which forces all the traffic to be rerouted in the access switch toward the other vPC peer device.

Once you configure this feature and if the module fails, the system automatically suspends all the vPC links on the primary vPC peer device and stops the peer-keepalive messages. This action forces the vPC secondary device to take over the primary role and all the vPC traffic to go to this new vPC primary device until the system stabilizes.

You should create a track list that contains all the links to the core and all the vPC peer links as its object. Enable tracking for the specified vPC domain for this track list. Apply this same configuration to the other vPC peer device. See the Cisco Nexus 7000 Series NX-OS Unicast Routing Configuration Guide for information about configuring object tracking and track lists.

See the Cisco Nexus 7000 Series NX-OS Unicast Routing Configuration Guide for information about configuring object tracking.

Note	This example uses Boolean OR in the track list and forces all traffic to the vPC peer device only for a complete module failure. Note that the Boolean AND operation is not supported with vPC object tracking.

A vPC deployment with a single Cisco Nexus 7000 Series M132XP-12 module or M108XP-12 module, where the L3 core uplinks and vPC peer-link interfaces are localized on the same module, is vulnerable to access layer isolation if the 10-Gbps module fails on the primary vPC (vPC member ports are defined on both 1-Gbps line cards and on 10-Gbps line card).

To configure a track list to switch over a vPC to the remote peer when all related interfaces on a single module fail, follow these steps:

1. Configure track objects on an interface (Layer 3 to core) and on a port channel (vPC peer link).

   
   switch(config-if)# track 35 interface ethernet 8/35 line-protocol
   switch(config-track)# track 23 interface ethernet 8/33 line-protocol
   switch(config)# track 55 interface port-channel 100 line-protocol

2. Create a track list that contains all the interfaces in the track list using the Boolean OR to trigger when all objects fail.

   
   switch(config)# track 44 list boolean OR
   switch(config-track)# object 23
   switch(config-track)# object 35
   switch(config-track)# object 55
   switch(config-track)# end

3. Add this track object to the vPC domain:

   
   switch(config)# vpc domain 1
   switch(config-vpc-domain)# track 44

4. Display the track object:

   
   switch# show vpc brief
   Legend:
          (*) - local vPC is down, forwarding via vPC peer-link
   vPC domain id                         : 1
   Peer status                           : peer adjacency formed ok
   vPC keep-alive status                 : peer is alive
   Configuration consistency status      : success
   vPC role                              : secondary
   Number of vPCs configured             : 52
   Track object                          : 44
   vPC Peer-link status
   ---------------------------------------------------------------------
   id    Port   Status  Active vlans
   --    ----   ------  --------------------------------------------------
   1     Po100   up      1-5,140
   vPC status
   ----------------------------------------------------------------------
   id    Port  Status Consistency Reason                     Active vlans
   --    ---- ------   ----------- -------------------------- ------------
   1      Po1   up      success    success                    1-5,140

switch# show track brief
Track Type      Instance           Parameter     State    Last
Change
23  Interface   Ethernet8/33        Line Protocol UP    00:03:05
35  Interface   Ethernet8/35        Line Protocol UP    00:03:15
44  List ----- Boolean
or   UP 00:01:19
55   Interface   port-channel100    Line Protocol UP    00:00:34

LACP uses the system MAC address of the vPC domain to form the LACP Aggregation Group (LAG) ID for the vPC.

You can use LACP on all the vPC port channels, including those channels from the downstream device. We recommend that you configure LACP with active mode on the interfaces on each port channel on the vPC peer devices. This configuration allows you to more easily detect compatibility between devices, unidirectional links, and multihop connection, and provides dynamic reaction to run-time changes and link failures.

With M Series modules and LACP, a vPC peer link supports 16 LACP interfaces: 8 active links and 8 hot standby links. You can configure 16 LACP links on the downstream vPC channel: 8 active links and 8 hot standby links. If you configure the port channels without using LACP, you can have only 8 links in each channel. With F-Series line cards, a vPC peer link and downstream vPC channels support up to 16 active LACP links. You can have 16 links in each channel even if the port channels are not configured using LACP.

We recommend that you manually configure the system priority on the vPC peer link devices to ensure that the vPC peer link devices have a higher LACP priority than the downstream connected devices. A lower numerical value system priority means a higher LACP priority.

Note	When manually configuring the system priority, you must ensure that you assign the same priority value on both vPC peer devices. If the vPC peer devices have different system priority values, vPC does not come up.

Although vPCs provide a loop-free Layer 2 topology, STP is still required to provide a fail-safe mechanism to protect against any incorrect or defective cabling or possible misconfiguration. When you first bring up a vPC, STP reconverges. STP treats the vPC peer link as a special link and always includes the vPC peer link in the STP active topology.

When the port-channel is designated as the vPC peer link, the spanning-tree port type network command is added and so the port-channel becomes the bridge assurance port. We recommend that you do not enable any of the STP enhancement features on vPC peer links. If the STP enhancements are already configured, they do not cause any problems for the vPC peer links.

When you are running both MST and Rapid PVST+, ensure that the PVST simulation feature is correctly configured.

See the Cisco Nexus 7000 Series NX-OS Layer 2 Switching Configuration Guide for information about STP enhancement features and PVST simulation.

Note	Bridge Assurance is enabled automatically on vPC peer-link during creation of link. It is recommended to retain Bridge assurance on the peer-link. Bridge Assurance on VPC is not supported.

You must configure a list of parameters to be identical on the vPC peer devices on both sides of the vPC peer link. See the “Compatibility Parameters for vPC Interfaces” section for information about these required matched settings.

STP is distributed; that is, the protocol continues running on both vPC peer devices. However, the configuration on the vPC peer device elected as the primary device controls the STP process for the vPC interfaces on the secondary vPC peer device.

The primary vPC device synchronizes the STP state on the vPC secondary peer device using Cisco Fabric Services over Ethernet (CFSoE). See the “CFSoE” section for information about CFSoE.

The STP process for vPC also relies on the periodic keepalive messages to determine when one of the connected devices on the peer link fails. See the “Peer-Keepalive Link and Messages” section for information about these messages.

The vPC manager performs a proposal/handshake agreement between the vPC peer devices that set the primary and secondary devices and coordinates the two devices for STP. The primary vPC peer device then controls the STP protocol on both the primary and secondary devices. We recommend that you configure the primary vPC peer device as the STP primary root device and configure the secondary VPC device to be the STP secondary root device.

If the primary vPC peer device fails over to the secondary vPC peer device, there is no change in the STP topology.

The BPDUs uses the MAC address set for the vPC for the STP bridge ID in the designated bridge ID field. The vPC primary device sends these BPDUs on the vPC interfaces.

You must configure both ends of vPC peer link with the identical STP configuration for the following parameters:

• STP global settings:

  • STP mode

  • STP region configuration for MST

  • Enable/disable state per VLAN

  • Bridge Assurance setting

  • Port type setting

  • Loop Guard settings

• STP interface settings:

  • Port type setting

  • Loop Guard

  • Root Guard

Note	If any of these parameters are misconfigured, the Cisco NX-OS software suspends all interfaces in the vPC. Check the syslog and enter the show vpc brief command to see if the vPC interfaces are suspended.

Ensure that the following STP interface configurations are identical on both sides of the vPC peer links or you may see unpredictable behavior in the traffic flow:

• BPDU Filter

• BPDU Guard

• Cost

• Link type

• Priority

• VLANs (PVRST+)

Note	Display the configuration on both sides of the vPC peer link to ensure that the settings are identical.

You can use the show spanning-tree command to display information about the vPC when that feature is enabled. See the Cisco Nexus 7000 Series NX-OS Layer 2 Switching Configuration Guide for an example.

We recommend that you configure the ports on the downstream devices as STP edge ports. You should configure all host ports connected to a switch as STP edge ports. See the Cisco Nexus 7000 Series NX-OS Layer 2 Switching Configuration Guide for more information about STP port types.

Note	If you bridge two VLANs on a Nexus 7000 peer-switch, with an Adaptive Security Appliance (ASA) in a transparent mode, the switch puts one of the VLAN in a STP dispute. To avoid this, disable peer-switch or STP on the ports.

The vPC peer switch feature is enabled on Cisco NX-OS Release 5.0(2) to address performance concerns around STP convergence. This feature allows a pair of Cisco Nexus 7000 Series devices to appear as a single STP root in the Layer 2 topology. This feature eliminates the need to pin the STP root to the vPC primary switch and improves vPC convergence if the vPC primary switch fails.

To avoid loops, the vPC peer link is excluded from the STP computation. In vPC peer switch mode, STP BPDUs are sent from both the vPC peer devices to avoid issues related to STP BPDU timeout on the downstream switches, which can cause traffic disruption.

This feature can be used with the vPC topology (non-hybrid), in which all the devices belong to the vPC topology.

Note

The Peer-switch feature on networks that use vPC and STP-based redundancy is not supported. If the vPC peer-link fails in a hybrid peer-switch configuration, you can lose traffic. In this scenario, the vPC peers use the same STP root ID as well as the same bridge ID. The access switch traffic is split in two with half traffic going to the first vPC peer and the other half traffic to the second vPC peer. With peer link failure, there is no impact to the north/south traffic but the east/west traffic is lost.

See the Cisco Nexus 7000 Series NX-OS Layer 2 Switching Configuration Guide for information about STP enhancement features and Rapid PVST+.

From Cisco NX-OS Release 6.0, Cisco NX-OS provides a way to control two peers to be partially designated forwarders when both vPC paths are up. When this control is enabled, each peer can be the designated forwarder for multi-destination southbound packets for a disjoint set of RBHs/FTAGs (depending on the hardware). The designated forwarder is negotiated on a per-vPC basis. This control is enabled with the fabricpath multicast load-balance command which is configured under vPC domain mode, for example:

switch(config)# vpc domain 1
switch(config-vpc-domain)# fabricpath multicast load-balance

From Cisco NX-OS Release 6.2(2), this feature is automatically enabled when the mode auto command is used. See the “Enabling Certain vPC Features Automatically” section for more information about using this command.

Note	Only an F2-series module supports multicast load balancing. On an F1-series module, the configuration is supported, but load balancing does not occur.

Note	The fabricpath multicast load-balance command is required for configuring vPC+ with FEX ports.

See the Cisco Nexus 7000 Series NX-OS FabricPath Configuration Guide for more detailed information on enabling designated forwarders on vPCs.

A feature was added in the Cisco NX-OS Release 4.2(6) to address table synchronization across vPC peers using the reliable transport mechanism of the Cisco Fabric Service over Ethernet (CFSoE) protocol. You must enable the ip arp synchronize and ipv6 nd synchronize commands to support faster convergence of address tables between the vPC peers. This convergence overcomes the delay that occurs in ARP table restoration for IPv4 or ND table restoration for IPv6 when the peer link port channel flaps or when a vPC peer comes back online.

Note	From Cisco NX-OS Release 6.2(2), you can use the mode auto command to automatically enable this feature. See the “Enabling Certain vPC Commands Automatically” section for information about using this command.

Note	The Cisco NX-OS software for the Nexus 7000 Series devices does not support Product Independent Multicast (PIM), Source-Specific Multicast(SSM) or Bidirectional (BIDR) on a vPC. The Cisco NX-OS software fully supports PIM Any Source Multicast (ASM) on a vPC.

A PIM adjacency between an Switched Virtual Interface (SVI) on a vPC VLAN (a VLAN that is carried on a vPC Peer-Link) and a downstream device is not supported; this configuration can result in dropped multicast packets. If a PIM neighbor relationship is required with a downstream device, a physical Layer 3 interface must be used on the Nexus switches instead of a vPC SVI.

For SVIs on vPC Vlans, only one PIM adjacency is supported - which is with the vPC Peer Switch. PIM adjacencies over the VPC Peer-Link with devices other than the VPC Peer Switch for the vPC-SVI are NOT supported.

The software keeps the multicast forwarding state synchronized on both of the vPC peer devices. The IGMP snooping process on a vPC peer device shares the learned group information with the other vPC peer device through the vPC peer link; the multicast states are always synchronized on both vPC peer devices. The PIM process in vPC mode ensures that only one of the vPC peer devices forwards the multicast traffic to the receivers.

Each vPC peer is a Layer 2 or Layer 3 device. Multicast traffic flows from only one of the vPC peer devices. You might see duplicate packets in the following scenarios:

• Orphan hosts

• When the source and receivers are in the Layer 2 vPC cloud in different VLANs with multicast routing enabled and a vPC member link goes down.

You might see negligible traffic loss in the following scenarios:

• When you reload the vPC peer device that is forwarding the traffic.

• When you restart PIM on the vPC peer device that is forwarding the traffic.

Overall multicast convergence times are scale and vPC role change / PIM restart duration dependent.

Ensure that you dual-attach all Layer 3 devices to both vPC peer devices. If one vPC peer device goes down, the other vPC peer device continues to forward all multicast traffic normally.

See the Cisco Nexus 7000 Series NX-OS Interfaces Command Reference for information about commands that display information on a vPC and multicast.

The following outlines vPC PIM and vPC IGMP/IGMP snooping:

• vPC PIM—The PIM process in vPC mode ensures that only one vPC peer device forwards multicast traffic. The PIM process in vPC mode synchronizes the source state with both vPC peer devices and elects which vPC peer device forwards the traffic.

• vPC IGMP/IGMP snooping—The IGMP process in vPC mode synchronizes the designated router (DR) information on both vPC peer devices. Dual DRs are available for IGMP when you are in vPC mode. Dual DRs are not available when you are not in vPC mode, because both vPC peer devices maintain the multicast group information between the peers.

Note

A PIM neighbor relationship between a vPC VLAN (a VLAN that is carried on a vPC peer link) and a downstream vPC-attached Layer 3 device is not supported, which can result in dropped multicast packets. If a PIM neighbor relationship is required with a downstream Layer 3 device, a physical Layer 3 interface must be used instead of a vPC interface.

You should enable or disable IGMP snooping identically on both vPC peer devices, and all the feature configurations should be identical. IGMP snooping is on by default.

Note	The following commands are not supported in vPC mode: ip pim spt-threshold infinity ip pim use-shared-tree-only See the Cisco Nexus 7000 Series NX-OS Multicast Routing Configuration Guide for more information about multicasting.

By default, a multicast router sends PIM joins upstream only if it has interested receivers. These interested receivers can either be IGMP hosts (they communicate through IGMP reports) or other multicast routers (they communicate through PIM joins).

VPC Device1:
------------
(*,G)
oif1 (igmp)

VPC Device2:
------------
(*,G)
oif1 (igmp)

VPC Device1 (say this is Forwarder for Source 'S'):
------------
(*,G)
  oif1 (igmp)

(S,G)
  oif1 (mrib)

VPC Device2:
------------
(*,G)
  oif1 (igmp)

(S,G)
NULL

In the case of a failure (for example, a Layer 3 Reverse Path Forwarding(RPF) link on the forwarder becomes inoperational or the forwarder gets reloaded), if the current nonforwarder ends up becoming the forwarder, it has to start sending PIM joins for (S,G) toward the source to pull the traffic. Depending upon the number of hops to reach the source, this operation might take some time (PIM is a hop-by-hop protocol).

To eliminate this issue and get better convergence, use the ip pim pre-build-spt command. This command enables PIM send joins even if the multicast route has 0 OIFs. In a vPC device, the nonforwarder sends PIM (S,G) joins upstream toward the source. The downside is that the link bandwidth upstream from the nonforwarder gets used for the traffic that is ultimately dropped by it. The benefits that result with better convergence far outweigh the link bandwidth usage. Therefore, we recommend that you use this command if you use vPCs.

Case 1: One OIF
===============
VPC Device1 (say this is PIM DR on oif1):
----------------------------------------
(*,G)
  oif1 (igmp)

VPC Device2:
------------
(*,G) will not be created.

VPC Device1 (say this is PIM DR on oif1):
----------------------------------------
(*,G)
  oif1 (igmp)
(S,G)
  oif1 (mrib)

VPC Device2:
------------
(*, G) will not be created.
(S, G) will not be created.

Case 2: Two OIFs
================
VPC Device1 (say this is PIM DR on oif1):
----------------------------------------
(*,G)
  oif1 (igmp)

VPC Device2 (say this is PIM DR on oif2):
----------------------------------------
(*,G)
  oif2 (igmp)

VPC Device1 (say this is PIM DR on oif1):
----------------------------------------
(*,G)
  oif1 (igmp)

(S,G)
  oif1 (mrib)

VPC Device1 (say this is PIM DR on oif2):
----------------------------------------
(*,G)
  oif2 (igmp)

(S,G)
  oif2 (mrib)

In the case of a vPC peer link with F2 modules, you do not need to enter the ip pim pre-build-spt command because PIM sends (S,G) joins upstream because associated routes have a non-NULL oiflist.

Note	Do not enter the ip pim pre-build-spt command if the vPC feature is enabled in F2 mode.

The First Hop Routing Protocols (FHRPs) interoperate with vPCs. The Hot Standby Routing Protocol (HSRP), Gateway Load Balancing Protocol (GLBP), and Virtual Router Redundancy Protocol (VRRP) all interoperate with vPCs. We recommend that you dual-attach all Layer 3 devices to both vPC peer devices.

The primary FHRP device responds to ARP requests, even though the secondary vPC device forwards the data traffic.

To simplify initial configuration verification and vPC/HSRP troubleshooting, you can configure the primary vPC peer device with the FHRP active router highest priority.

In addition, you can use the priority command in the if-hsrp configuration mode to configure failover thresholds for when a group state enabled on a vPC peer link is in standby or in listen state. You can configure lower and upper thresholds to prevent the interface from going up and down.

VRRP acts similarly to HSRP when running on vPC peer devices. You should configure VRRP the same way that you configure HSRP. For GLBP, the forwarders on both vPC peer devices forward traffic.

When the primary vPC peer device fails over to the secondary vPC peer device, the FHRP traffic continues to flow seamlessly.

We recommend that you configure routing adjacency between the two vPC peer devices to act as a backup routing path. If one vPC peer device loses Layer 3 uplinks, the vPC can redirect the routed traffic to the other vPC peer device and leverage its active Layer 3 uplinks.

You can configure the inter-switch link for a backup routing path in the following ways:

• Create a Layer 3 link between the two vPC peer devices.

• Use the non-VPC VLAN trunk with a dedicated VLAN interface.

• Use a vPC peer link with a dedicated VLAN interface.

We do not recommend that you configure the burnt-in MAC address option (use-bia) for HSRP or manually configure virtual MAC addresses for any FHRP protocol in a vPC environment because these configurations can adversely affect vPC load balancing. The HSRP use-bia option is not supported on vPCs. When you are configuring custom MAC addresses, you must configure the same MAC address on both vPC peer devices.

From Cisco NX-OS Release 4.2(1), you can use the delay restore command to configure a restore timer that delays the vPC coming back up until after the peer adjacency forms and the VLAN interfaces are back up. This feature enables you to avoid packet drops when the routing tables might not be converged before the vPC is once again passing traffic. Use the delay restore command to configure this feature.

To delay the VLAN interfaces on the restored vPC peer device from coming up, use the interfaces-vlan option to the delay restore command.

See the Cisco Nexus 7000 Series NX-OS Unicast Routing Configuration Guide for more information about FHRPs and routing.

The Cisco Fabric Services over Ethernet (FSoE) is a reliable state transport mechanism that is used to synchronize the actions of the vPC peer devices. Cisco FSoE carries messages and packets for many features linked with vPC, such as STP and IGMP. Information is carried in Cisco Fabric Service or Cisco FSoE protocol data units (PDUs).

When you enable the vPC feature, the device automatically enables Cisco FSoE, and you do not have to configure anything. Cisco FSoE distributions for vPCs do not need the capabilities to distribute over IP or the FS regions. You do not need to configure anything for the Cisco FSoE feature to work correctly on vPCs.

The Cisco FSoE transport is local to each VDC.

You can use the show mac address-table command to display the MAC addresses that Cisco FSoE synchronizes for the vPC peer link.

Note	Do not enter the no cfs eth distribute or the no cfs distribute command. You must enable Cisco FSoE for vPC functionality. If you do enter either of these commands with vPC enabled, the system displays an error message.

When you enter the show cfs application command, the output displays “Physical-eth,” which shows the applications that are using Cisco FSoE.

Cisco Fabric Service also transports data over TCP/IP. See the Cisco Nexus 7000 Series NX-OS System Management Configuration Guide for more information about Cisco Fabric Service over IP.

Note	The software does not support Cisco Fabric Service regions.

When a device that is not vPC-capable connects to each peer, the connected ports are known as orphan ports because they are not members of a vPC. The device’s link to one peer will be active (forwarding) and the other link will be standby (blocking) due to STP.

If a peer link failure or restoration occurs, an orphan port’s connectivity might be bound to the vPC failure or restoration process. For example, if a device’s active orphan port connects to the secondary vPC peer, the device loses any connections through the primary peer if a peer link failure occurs and the vPC ports are suspended by the secondary peer. If the secondary peer were to also suspend the active orphan port, the device’s standby port becomes active, provides a connection to the primary peer, and restores connectivity. From Cisco NX-OS Release 5.2(1), you can configure in the CLI that specific orphan ports are suspended by the secondary peer when it suspends its vPC ports and are restored when the vPC is restored.

The Fibre Channel over Ethernet (FCoE) over Physical Port Virtual Port Channels (vPCs) feature extends the shared model for physical Ethernet interfaces to vPC interfaces.

Each Ethernet interface that forms a vPC leg is shared between the storage virtual device context (VDC) and the Ethernet VDC. The shared Ethernet interface carries both FCoE and LAN traffic. Mutually exclusive FCoE and LAN VLANs are allocated to carry the traffic on the vPC leg; FCoE traffic is carried by the FCoE VLAN and LAN traffic is carried by the LAN VLAN.

Certain configuration and network parameters must be consistent across peer switches in order for physical port vDCs to work. If an inconsistency impacting the network (Type 1) is detected, the secondary vPC leg (the physical link between the access switch and the host) is brought down. With FCoE over physical port vPC, vPC legs carry both FCoE and LAN traffic so that the FCoE and LAN link are both brought down. The shutdown LAN feature enables you to shut down or bring up only the LAN VLANs on an Ethernet interface.

In a data center outage, both of the Cisco Nexus 7000 Series devices that include a vPC get reloaded. Occasionally only one peer can be restored. With no functioning peer-keepalive or peer link, the vPC cannot function normally, but depending on your Cisco NX-OS release, a method might be available to allow vPC services to use only the local ports of the functional peer.

During an In-Service Software Upgrade (ISSU), the software reload process on the first vPC device locks its vPC peer device by using CFS messaging over the vPC communications channel. Only one device at a time is upgraded. When the first device completes its upgrade, it unlocks its peer device. The second device then performs the upgrade process, locking the first device as it does so. During the upgrade, the two vPC devices temporarily run different releases of Cisco NX-OS, however the system functions correctly because of its backward compatibility support.

See the Cisco Nexus 7000 Series NX-OS High Availability and Redundancy Guide for complete information about high-availability features.

The hitless vPC role change feature can be used in the following scenarios:

• Role change request—When you want to change the roles of the peer devices in a vPC domain.

• Primary switch reload—When the devices comes up after a reload and roles are defined, you can use the hitless vPC role change feature to restore the roles. For example, after a reload if the primary device takes the role of operational secondary and the secondary device takes the role of primary operational, you can change the vPC peer roles to their original defined roles using the vpc role preempt command.

  Note	Always check the existing device role priority before configuring the vpc role preempt command. Configure no port-channel limit under the vpc domain command before configuring the vpc role preempt command.

• Dual-active recovery—In a dual-active recovery scenario, the vPC primary switch continues to be (operational) primary, but the vPC secondary switch becomes the targeted primary switch and keeps its vPC member ports up. You can use the vPC hitless feature and restore the device roles. After the Dual-active recovery, if one side is operational primary and the other side operational secondary, then you can use the the vpc role preempt command to restore the device roles to be primary and secondary.

Configuration synchronization benefits are as follows:

• Provides a mechanism to synchronize configuration from one switch to another switch.

• Merges configurations when connectivity is established between peers.

• Provides mutual exclusion for commands.

• Supports existing session and port profile functionality.

• Provides minimal user intervention.

• Minimizes the possibility of user error.

The following types of commands are enabled for configuration synchronization:

Note	The show vpc config-sync cli syntax command lists all the commands that are enabled for configuration synchronization. You cannot choose which commands are synchronized. For more information, see the Cisco Nexus 7000 Series NX-OS Interfaces Command Reference.

• Type-1 configurations:

  • Global configurations

  • vPC member port-channel configurations

• vPC configurations.

Note	The configurations can be given on either of the vPC peer switches.

To upgrade from an existing line card module to a new line card module for vPC using the reload method, perform the following tasks:

1. Install Cisco NX-OS image on vPC peers

2. Install line card module using the reload method

Before you plan to upgrade a line card module, refer the Cisco Nexus 7000 Series NX-OS Release Notes document, to see the supported Cisco NX-OS release version for a line card module.