The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.
This document describes the private VLAN (PVLAN) support for the Cisco Unified Computing System (UCS) in 2.2(2c) release and later.
Caution: There is a change in behavior starting with UCS firmware version 3.1(3a) as described in the Behavior Change with UCS Version 3.1(3) and Later section.
Cisco recommends that you have knowledge of these topics:
This document is not restricted to specific software and hardware versions.
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.
A private VLAN is a VLAN configured for L2 isolation from other ports within the same private VLAN. Ports that belong to a PVLAN are associated with a common set of support VLANs, which are used in order to create the PVLAN structure.
There are three types of PVLAN ports:
Refer to RFC 5517, Cisco Systems' Private VLANs: Scalable Security in a Multi-Client Environment in order to understand the theory, operation, and concepts of PVLANs.
With Nexus 1000v or VMware DVS
Note: This example uses VLAN 1750 as the primary, 1785 as isolated and 1786 as community VLAN.
2. Create Isolated and Community VLANs accordingly as shown in the images. None of these has to be a Native VLAN.
3. Virtual Network Interface Card (vNIC) on service-profile carries regular VLANs as well as PVLANs, as seen in the image.
4. Uplink port-channel on UCS carries regular VLANs as well as PVLANs:
interface port-channel1 description U: Uplink switchport mode trunk pinning border switchport trunk allowed vlan 1,121,221,321,1750,1785-1786 speed 10000 F240-01-09-UCS4-A(nxos)# F240-01-09-UCS4-A(nxos)# show vlan private-vlan Primary Secondary Type Ports ------- --------- --------------- ------------------------------------------- 1750 1785 isolated 1750 1786 community
feature private-vlan vlan 1750 private-vlan primary private-vlan association 1785-1786 vlan 1785 private-vlan isolated vlan 1786 private-vlan community interface Vlan1750 ip address 10.10.175.252/24 private-vlan mapping 1785-1786 no shutdown interface port-channel114 Description To UCS switchport mode trunk switchport trunk allowed vlan 1,121,154,169,221,269,321,369,1750,1785-1786 spanning-tree port type edge spanning-tree bpduguard enable spanning-tree bpdufilter enable vpc 114 <=== if there is a 5k pair in vPC configuration only then add this line to both N5k
Prior to UCS version 3.1(3), you could have a VM in community VLAN communicate with a VM in Primary VLAN on VMware DVS where the Primary VLAN VM resides inside the UCS. This behavior was incorrect as the primary VM must always be northbound or outside of the UCS. This behavior is documented via defect ID CSCvh87378 .
From UCS version 2.2(2) onwards, due to a defect in the code, community VLAN was able to communicate with primary VLAN that was present behind the FI. But Isolated could never communicate with the primary behind the FI. Both (isolated and community) VMs are still able to communicate with the primary outside the FI.
From 3.1(3) onwards, this defect allows community to communicate with primary behind the FI, was rectified and thus community VMs won't be able to communicate with a VM in primary VLAN which resides within UCS.
In order to resolve this situation, the primary VM would either need to be moved (northbound) outside of UCS. If that’s not an option, then the primary VM would need to be moved into another VLAN that is a regular VLAN and not a private VLAN.
For example, prior to firmware 3.1(3), a VM in community VLAN 1786 could communicate to a VM in primary VLAN 1750 which resides within UCS, however, this communication would break in firmware 3.1(3) and later, as shown in the image.
NOTE:
-----------
CSCvh87378 has been addressed in 3.2(3l) and 4.0.4e & higher so we can have Primary Vlan behind UCS. However please note that isolated vlan inside UCS won't be able to talk to primary vlan inside UCS. Only community vlan & primary vlan can talk to each other when both are behind UCS.
Note: In this example, 4900 is L3 interface to outside network. If your topology for L3 is different, then kindly make changes accordingly
On the 4900 switch, take these steps, and set up the promiscuous port. The PVLAN ends at the promiscuous port.
Switch(config-if)#switchport mode trunk
switchport private-vlan mapping 1785-1786
switchport mode private-vlan promiscuous
On the upstream router, create a subinterface for the VLAN 1750 only. At this level, the requirements depend upon the network configuration you use:
interface GigabitEthernet0/1.1 encapsulation dot1Q 1750 IP address10.10.175.254/24
There is currently no verification procedure available for this configuration.
This section provides information you can use in order to troubleshoot your configuration.
This procedure describes how to test the configuration for VMware DVS with the use of PVLAN.
1. Run pings to other systems configured in the port-group as well as the router or other device at the promiscuous port. Pings to the device past the promiscuous port must work, while those to other devices in the isolated VLAN must fail as shown in the images.
Check the MAC address tables in order to see where your MAC is being learned. On all switches, the MAC must be in the isolated VLAN except on the switch with the promiscuous port. On the promiscuous switch, the MAC must be in the primary VLAN.
2. UCS as shown in the image.
3. Check on upstream n5k for same MAC, output similar to earlier output must be present on n5k and as shown in the image.
UCS configuration (which includes service-profile vNIC configuration) stays the same as per the example with VMware DVS.
feature private-vlan vlan 1750 private-vlan primary private-vlan association 1785-1786 vlan 1785 private-vlan isolated vlan 1786 private-vlan community same uplink port-profile is being used for regular vlans & pvlans. In this example vlan 121 & 221 are regular vlans but you can change them accordingly port-profile type ethernet pvlan-uplink-no-prom switchport mode trunk mtu 9000 switchport trunk allowed vlan 121,221,1750,1785-1786 channel-group auto mode on mac-pinning system vlan 121 no shutdown state enabled vmware port-group port-profile type vethernet pvlan_1785 switchport mode private-vlan host switchport private-vlan host-association 1750 1785 switchport access vlan 1785 no shutdown state enabled vmware port-group port-profile type vethernet pvlan_1786 switchport mode private-vlan host switchport access vlan 1786 switchport private-vlan host-association 1750 1786 no shutdown state enabled vmware port-group
This procedure describes how to test the configuration.
1. Run pings to other systems configured in the port-group as well as the router or other device at the promiscuous port. Pings to the device past the promiscuous port must work, while those to other devices in the isolated VLAN must fail, as shown in previous section and in the images.
2. On the N1K, the VMs are listed on the primary VLAN; this occurs because you are in PVLAN host ports that are associated to the PVLAN as shown in the image.
Check the MAC address tables in order to see where your MAC is being learned. On all switches, the MAC must be in the isolated VLAN except on the switch with the promiscuous port. On the promiscuous switch the MAC must be in the primary VLAN.
3. On the UCS system, you must learn all MACs in their respective private VLANs as shown in the image.
4. Check on upstream n5k for same MAC, output similar to earlier output must be present on n5k as shown in the image.
Since PVLAN ends at the promiscuous port, in this configuration, you contain PVLAN traffic to the N1K with only the primary VLAN used upstream. So UCS and upstream devices aren't aware of any PVLANs.
This procedure describes how to add the primary VLAN to the vNIC. There is no need for PVLAN configuration because you only need the primary VLAN.
Note: This example uses 1750 as the primary, 1785 as isolated and 1786 as community VLAN as also shown in the image.
These procedures describe how to configure the upstream devices. In this case, the upstream switches only need trunk ports, and they only need to trunk VLAN 1750 because it is the only VLAN the upstream switches see.
On the Nexus 5K, run these commands, and check uplink configuration:
Nexus5000-5(config-vlan)# vlan 1750
This procedure describes how to configure the N1K:
feature private-vlan vlan 1750 private-vlan primary private-vlan association 1785-1786 vlan 1785 private-vlan isolated vlan 1786 private-vlan community same uplink port-profile is being used for regular vlans & pvlans. In this example vlan 121 & 221 are regular vlans but you can change them accordingly port-profile type ethernet pvlan-uplink switchport mode private-vlan trunk promiscuous switchport trunk allowed vlan 121,221,1750 switchport private-vlan trunk allowed vlan 121,221,1750 <== Only need to allow Primary VLAN switchport private-vlan mapping trunk 1750 1785-1786 <=== PVLANs must be mapped at this stage mtu 9000 channel-group auto mode on mac-pinning no shutdown system vlan 121 state enabled vmware port-group port-profile type vethernet pvlan_1785 switchport mode private-vlan host switchport private-vlan host-association 1750 1785 switchport access vlan 1785 no shutdown state enabled vmware port-group port-profile type vethernet pvlan_1786 switchport mode private-vlan host switchport access vlan 1786 switchport private-vlan host-association 1750 1786 no shutdown state enabled vmware port-group
PVLAN must terminate at promiscuous port on uplink port-profile for n1k and UCS and subsequently all upstream device must see those VMs in primary VLANs. Here is the snapshot from upstream N5k and UCS.
Few things to remember:
private-vlan mapping trunk command does not decide or override the trunk configuration of a port.