A virtual port channel (vPC) allows links that are physically connected to two Cisco Nexus devices to appear as a single port channel by a third device (see the following figure). The third device can be a switch, server, or any other networking device. A vPC can provide multipathing, which allows you to create redundancy by enabling multiple parallel paths between nodes and load balancing traffic where alternative paths exist.

You configure the EtherChannels by using one of the following:

• No protocol

• Link Aggregation Control Protocol (LACP)

When you configure the EtherChannels in a vPC—including the vPC peer link channel—each switch can have up to 32 active links in a single EtherChannel.

Note	You must enable the vPC feature before you can configure or run the vPC functionality.

To enable the vPC functionality, you must create a peer-keepalive link and a peer-link under the vPC domain for the two vPC peer switches to provide the vPC functionality.

To create a vPC peer link you configure an EtherChannel on one Cisco Nexus device by using two or more Ethernet ports. On the other switch, you configure another EtherChannel again using two or more Ethernet ports. Connecting these two EtherChannels together creates a vPC peer link.

Note	We recommend that you configure the vPC peer-link EtherChannels as trunks.

The vPC domain includes both vPC peer devices, the vPC peer-keepalive link, the vPC peer link, and all of the EtherChannels in the vPC domain connected to the downstream device. You can have only one vPC domain ID on each vPC peer device.

Note	Always attach all vPC devices using EtherChannels to both vPC peer devices.

A vPC provides the following benefits:

• Allows a single device to use an EtherChannel across two upstream devices

• Eliminates Spanning Tree Protocol (STP) blocked ports

• Provides a loop-free topology

• Uses all available uplink bandwidth

• Provides fast convergence if either the link or a switch fails

• Provides link-level resiliency

• Assures high availability

The terminology used in vPCs is as follows:

• vPC—combined EtherChannel between the vPC peer devices and the downstream device.

• vPC peer device—One of a pair of devices that are connected with the special EtherChannel known as the vPC peer link.

• vPC peer link—link used to synchronize states between the vPC peer devices.

• vPC member port—Interfaces that belong to the vPCs.

• vPC domain—domain that includes both vPC peer devices, the vPC peer-keepalive link, and all of the port channels in the vPC connected to the downstream devices. It is also associated to the configuration mode that you must use to assign vPC global parameters. The vPC domain ID must be the same on both switches.

• vPC peer-keepalive link—The peer-keepalive link monitors the vitality of a vPC peer Cisco Nexus device. The peer-keepalive link sends configurable, periodic keepalive messages between vPC peer devices.

  No data or synchronization traffic moves over the vPC peer-keepalive link; the only traffic on this link is a message that indicates that the originating switch is operating and running vPCs.

To create a vPC domain, you must first create a vPC domain ID on each vPC peer switch using a number from 1 to 1000. This ID must be the same on a set of vPC peer devices.

You can configure the EtherChannels and vPC peer links by using LACP or no protocol. When possible, we recommend that you use LACP on the peer-link, because LACP provides configuration checks against a configuration mismatch on the EtherChannel.

The vPC peer switches use the vPC domain ID that you configure to automatically assign a unique vPC system MAC address. Each vPC domain has a unique MAC address that is used as a unique identifier for the specific vPC-related operations, although the switches use the vPC system MAC addresses only for link-scope operations, such as LACP. We recommend that you create each vPC domain within the contiguous network with a unique domain ID. You can also configure a specific MAC address for the vPC domain, rather than having the Cisco NX-OS software assign the address.

The vPC peer switches use the vPC domain ID that you configure to automatically assign a unique vPC system MAC address. The switches use the vPC system MAC addresses only for link-scope operations, such as LACP or BPDUs. You can also configure a specific MAC address for the vPC domain.

We recommend that you configure the same VPC domain ID on both peers and, the domain ID should be unique in the network. For example, if there are two different VPCs (one in access and one in aggregation) then each vPC should have a unique domain ID.

After you create a vPC domain, the Cisco NX-OS software automatically creates a system priority for the vPC domain. You can also manually configure a specific system priority for the vPC domain.

Note	If you manually configure the system priority, you must ensure that you assign the same priority value on both vPC peer switches. If the vPC peer switches have different system priority values, the vPC will not come up.

The Cisco NX-OS software uses a peer-keepalive link between the vPC peers to transmit periodic, configurable keepalive messages. You must have Layer 3 connectivity between the peer switches to transmit these messages; the system cannot bring up the vPC peer link unless a peer-keepalive link is already up and running.

If one of the vPC peer switches fails, the vPC peer switch on the other side of the vPC peer link senses the failure when it does not receive any peer-keepalive messages. The default interval time for the vPC peer-keepalive message is 1 second. You can configure the interval between 400 milliseconds and 10 seconds. You can also configure a timeout value with a range of 3 to 20 seconds; the default timeout value is 5 seconds. The peer-keepalive status is checked only when the peer-link goes down.

The vPC peer-keepalive can be carried either in the management or default VRF on the Cisco Nexus device. When you configure the switches to use the management VRF, the source and destination for the keepalive messages are the mgmt 0 interface IP addresses. When you configure the switches to use the default VRF, an SVI must be created to act as the source and destination addresses for the vPC peer-keepalive messages. Ensure that both the source and destination IP addresses used for the peer-keepalive messages are unique in your network and these IP addresses are reachable from the VRF associated with the vPC peer-keepalive link.

Note	We recommend that you configure the vPC peer-keepalive link on the Cisco Nexus device to run in the management VRF using the mgmt 0 interfaces. If you configure the default VRF, ensure that the vPC peer link is not used to carry the vPC peer-keepalive messages.

Many configuration and operational parameters must be identical on all interfaces in the vPC. After you enable the vPC feature and configure the peer link on both vPC peer switches, Cisco Fabric Services (CFS) messages provide a copy of the configuration on the local vPC peer switch configuration to the remote vPC peer switch. The system then determines whether any of the crucial configuration parameters differ on the two switches.

Enter the show vpc consistency-parameters command to display the configured values on all interfaces in the vPC. The displayed configurations are only those configurations that would limit the vPC peer link and vPC from coming up.

The compatibility check process for vPCs differs from the compatibility check for regular EtherChannels.

New Type 2 Consistency Check on the vPC Port-Channels

A new type 2 consistency check has been added to validate the switchport mac learn settings on the vPC port-channels. The CLI show vpc consistency-check vPC <vpc no.> has been enhanced to display the local and peer values of the switchport mac-learn configuration. Because it is a type 2 check, vPC is operationally up even if there is a mismatch between the local and the peer values, but the mismatch can be displayed from the CLI output.

switch# sh vpc consistency-parameters vpc 1112
 
    Legend:
        Type 1 : vPC will be suspended in case of mismatch
 
Name                                 Type            Local Value                 Peer Value
-------------                        ----            ----------------------     -----------------------
Shut Lan                             1               No                         No
STP Port Type                        1               Default                     Default
STP Port Guard                       1               None                        None
STP MST Simulate PVST                1               Default                     Default
nve configuration                    1               nve                        nve
lag-id                               1               [(fa0,                      [(fa0,
                                                     0-23-4-ee-be-64, 8458,     0-23-4-ee-be-64, 8458,
                                                     0, 0), (8000,              0, 0), (8000,
                                                     f4-4e-5-84-5e-3c, 457,     f4-4e-5-84-5e-3c, 457,
                                                     0, 0)]                     0, 0)]
mode                                 1               active                      active
Speed                                1               10 Gb/s                    10 Gb/s
Duplex                               1               full                        full
Port Mode                            1               trunk                       trunk
Native Vlan                          1               1                          1
MTU                                  1               1500                        1500
Admin port mode                      1
Switchport MAC Learn                 2               Enable                     Disable> Newly added consistency parameter
vPC card type                        1               Empty                      Empty
Allowed VLANs                        -               311-400                     311-400
Local suspended VLANs                -               -                      

The configuration parameters in this section must be configured identically on both switches at either end of the vPC peer link.

Note

You must ensure that all interfaces in the vPC have the identical operational and configuration parameters listed in this section. Enter the show vpc consistency-parameters command to display the configured values on all interfaces in the vPC. The displayed configurations are only those configurations that would limit the vPC peer link and vPC from coming up.

The switch automatically checks for compatibility of these parameters on the vPC interfaces. The per-interface parameters must be consistent per interface, and the global parameters must be consistent globally.

• Port-channel mode: on, off, or active
• Link speed per channel
• Duplex mode per channel
• Trunk mode per channel:
  • Native VLAN
  • VLANs allowed on trunk
  • Tagging of native VLAN traffic
• Spanning Tree Protocol (STP) mode
• STP region configuration for Multiple Spanning Tree (MST)
• Enable or disable state per VLAN
• STP global settings:
  • Bridge Assurance setting
  • Port type setting—We recommend that you set all vPC interfaces as normal ports
  • Loop Guard settings
• STP interface settings:
  • Port type setting
  • Loop Guard
  • Root Guard

If any of these parameters are not enabled or defined on either switch, the vPC consistency check ignores those parameters.

Note	To ensure that none of the vPC interfaces are in the suspend mode, enter the show vpc brief and show vpc consistency-parameters commands and check the syslog messages.

When any of the following parameters are not configured identically on both vPC peer switches, a misconfiguration might cause undesirable behavior in the traffic flow:

• MAC aging timers
• Static MAC entries
• VLAN interface—Each switch on the end of the vPC peer link must have a VLAN interface configured for the same VLAN on both ends and they must be in the same administrative and operational mode. Those VLANs configured on only one switch of the peer link do not pass traffic using the vPC or peer link. You must create all VLANs on both the primary and secondary vPC switches, or the VLAN will be suspended.
• All ACL configurations and parameters
• Quality of service (QoS) configuration and parameters—Local parameters; global parameters must be identical
• STP interface settings:
  • BPDU Filter
  • BPDU Guard
  • Cost
  • Link type
  • Priority
  • VLANs (Rapid PVST+)

To ensure that all the configuration parameters are compatible, we recommend that you display the configurations for each vPC peer switch once you configure the vPC.

A vPC peer link is the link that is used to synchronize the states between the vPC peer devices.

Note	You must configure the peer-keepalive link before you configure the vPC peer link or the peer link will not come up.

Once you have created the vPC domain ID and the vPC peer link, you can create EtherChannels to attach the downstream switch to each vPC peer switch. That is, you create one single EtherChannel on the downstream switch with half of the ports to the primary vPC peer switch and the other half of the ports to the secondary peer switch.

On each vPC peer switch, you assign the same vPC number to the EtherChannel that connects to the downstream switch. You will experience minimal traffic disruption when you are creating vPCs. To simplify the configuration, you can assign the vPC ID number for each EtherChannel to be the same as the EtherChannel itself (that is, vPC ID 10 for EtherChannel 10).

Note	The vPC number that you assign to the EtherChannel that connects to the downstream switch from the vPC peer switch must be identical on both vPC peer switches.

The Link Aggregation Control Protocol (LACP) uses the system MAC address of the vPC domain to form the LACP Aggregation Group (LAG) ID for the vPC.

You can use LACP on all the vPC EtherChannels, including those channels from the downstream switch. We recommend that you configure LACP with active mode on the interfaces on each EtherChannel on the vPC peer switches. This configuration allows you to more easily detect compatibility between switches, unidirectional links, and multihop connections, and provides dynamic reaction to run-time changes and link failures.

The vPC peer link supports 16 EtherChannel interfaces.

Note	When you manually configure the system priority, you must ensure that you assign the same priority value on both vPC peer switches. If the vPC peer switches have different system priority values, vPC does not come up.

When you first bring up the vPC functionality, STP reconverges. STP treats the vPC peer link as a special link and always includes the vPC peer link in the STP active topology.

We recommend that you set all the vPC peer link interfaces to the STP network port type so that Bridge Assurance is automatically enabled on all vPC peer links. We also recommend that you do not enable any of the STP enhancement features on VPC peer links.

You must configure a list of parameters to be identical on the vPC peer switches on both sides of the vPC peer link.

STP is distributed; that is, the protocol continues running on both vPC peer switches. However, the configuration on the vPC peer switch elected as the primary switch controls the STP process for the vPC interfaces on the secondary vPC peer switch.

The primary vPC switch synchronizes the STP state on the vPC secondary peer switch using Cisco Fabric Services over Ethernet (CFSoE).

The vPC manager performs a proposal/handshake agreement between the vPC peer switches that sets the primary and secondary switches and coordinates the two switches for STP. The primary vPC peer switch then controls the STP protocol for vPC interfaces on both the primary and secondary switches.

The Bridge Protocol Data Units (BPDUs) use the MAC address set for the vPC for the STP bridge ID in the designated bridge ID field. The vPC primary switch sends these BPDUs on the vPC interfaces.

Note	Display the configuration on both sides of the vPC peer link to ensure that the settings are identical. Use the show spanning-tree command to display information about the vPC.

The Cisco Fabric Services over Ethernet (CFSoE) is a reliable state transport mechanism that you can use to synchronize the actions of the vPC peer devices. CFSoE carries messages and packets for many features linked with vPC, such as STP and IGMP. Information is carried in CFS/CFSoE protocol data units (PDUs).

When you enable the vPC feature, the device automatically enables CFSoE, and you do not have to configure anything. CFSoE distributions for vPCs do not need the capabilities to distribute over IP or the CFS regions. You do not need to configure anything for the CFSoE feature to work correctly on vPCs.

You can use the show mac address-table command to display the MAC addresses that CFSoE synchronizes for the vPC peer link.

Note	Do not enter the no cfs eth distribute or the no cfs distribute command. CFSoE must be enabled for vPC functionality. If you do enter either of these commands when vPC is enabled, the system displays an error message.

When you enter the show cfs application command, the output displays "Physical-eth," which shows the applications that are using CFSoE.

vPCs have the following configuration guidelines and limitations:

• vPC is not supported between different types of Cisco Nexus 3000 Series switches.

• VPC peers should have same reserved VLANs for VXLAN. Different reserved VLANs on the peers may lead to undesired behavior with VXLAN.

• The output of the sh vpc brief CLI command displays two additional fields, Delay-restore status and Delay-restore SVI status.

• You must enable the vPC feature before you can configure vPC peer-link and vPC interfaces.

• You must configure the peer-keepalive link before the system can form the vPC peer link.

• The vPC peer-link needs to be formed using a minimum of two 10-Gigabit Ethernet interfaces.

• We recommend that you configure the same vPC domain ID on both peers and the domain ID should be unique in the network. For example, if there are two different vPCs (one in access and one in aggregation) then each vPC should have a unique domain ID.

• Only port channels can be in vPCs. A vPC can be configured on a normal port channel (switch-to-switch vPC topology) and on a port channel host interface (host interface vPC topology).

• You must configure both vPC peer switches; the configuration is not automatically synchronized between the vPC peer devices.

• Check that the necessary configuration parameters are compatible on both sides of the vPC peer link.

• You might experience minimal traffic disruption while configuring vPCs.

• You should configure all port channels in the vPC using LACP with the interfaces in active mode.

• You might experience traffic disruption when the first member of a vPC is brought up.

• OSPF over vPC and BFD with OSPF are supported on Cisco Nexus 3000 Series switches.

  SVI limitation: When a BFD session is over SVI using virtual port-channel(vPC) peer-link, the BFD echo function is not supported. You must disable the BFD echo function for all sessions over SVI between vPC peer nodes using no bfd echo at the SVI configuration level.

• When a Layer 3 link is used for peer-keepalive instead of the mgmt interface, and the CPU queues are congested with control plane traffic, vPC peer-keepalive packets could be dropped. The CPU traffic includes routing protocol, ARP, Glean, and IPMC miss packets. When the peer-keepalive interface is a Layer 3 link instead of a mgmt interface, the vPC peer-keepalive packets are sent to the CPU on a low-priority queue.

  If a Layer 3 link is used for vPC peer-keepalives, configure the following ACL to prioritize the vPC peer-keepalive:

  ip access-list copp-system-acl-routingproto2
  30 permit udp any any eq 3200

  Here, 3200 is the default UDP port for keepalive packets. This ACL must match the configured UDP port in case the default port is changed.