Guidelines and Limitations for VXLAN
VXLAN has the following guidelines and limitations:
ACL Direction |
ACL Type |
VTEP Type |
Port Type |
Flow Direction |
Traffic Type |
Supported |
---|---|---|---|---|---|---|
Ingress |
PACL |
Ingress VTEP |
L2 port |
Access to Network [GROUP:encap direction] |
Native L2 traffic [GROUP:inner] |
YES |
VACL |
Ingress VTEP |
VLAN |
Access to Network [GROUP:encap direction] |
Native L2 traffic [GROUP:inner] |
YES |
|
Ingress |
RACL |
Ingress VTEP |
Tenant L3 SVI |
Access to Network [GROUP:encap direction] |
Native L3 traffic [GROUP:inner] |
YES |
Egress |
RACL |
Ingress VTEP |
Uplink L3/L3-PO/SVI |
Access to Network [GROUP:encap direction] |
VXLAN encap [GROUP:outer] |
NO |
Ingress |
RACL |
Egress VTEP |
Uplink L3/L3-PO/SVI |
Network to Access [GROUP:decap direction] |
VXLAN encap [GROUP:outer] |
NO |
Egress |
PACL |
Egress VTEP |
L2 port |
Network to Access [GROUP:decap direction] |
Native L2 traffic [GROUP:inner] |
NO |
VACL |
Egress VTEP |
VLAN |
Network to Access [GROUP:decap direction] |
Native L2 traffic [GROUP:inner] |
NO |
|
Egress |
RACL |
Egress VTEP |
Tenant L3 SVI |
Network to Access [GROUP:decap direction] |
Post-decap L3 traffic [GROUP:inner] |
YES |
-
For scale environments, the VLAN IDs related to the VRF and Layer-3 VNI (L3VNI) must be reserved with the system vlan nve-overlay id command.
-
NLB in the unicast, multicast, and IGMP multicast modes is not supported on Cisco Nexus 9000 switch VXLAN VTEPs. The work-around is to move the NLB cluster behind the intermediary device (which supports NLB in the respective mode) and inject the cluster IP address as an external prefix into the VXLAN fabric.
-
Support added for MultiAuth Change of Authorization (CoA). For more information, see the Cisco Nexus 9000 Series NX-OS Security Configuration Guide, Release 9.3(x).
-
The lacp vpc-convergence command can be configured in VXLAN and non-VXLAN environments that have vPC port channels to hosts that support LACP.
-
PIM BiDir for VXLAN underlay with and without vPC is supported.
The following features are not supported when PIM BiDir for VXLAN underlay is configured:
-
Flood and Learn VXLAN
-
Tenant Routed Multicast (TRM)
-
VXLAN EVPN Multi-Site
-
VXLAN EVPN Multihoming
-
vPC attached VTEPs
For redundant RPs, use Phantom RP.
For transitioning from PIM ASM to PIM BiDir or from PIM BiDir to PIM ASM underlay, we recommend that you use the following example procedure:
no ip pim rp-address 192.0.2.100 group-list 230.1.1.0/8 clear ip mroute * clear ip mroute date-created * clear ip pim route * clear ip igmp groups * clear ip igmp snooping groups * vlan all
Wait for all tables to clean up.
ip pim rp-address 192.0.2.100 group-list 230.1.1.0/8 bidir
-
-
When entering the no feature pim command, NVE ownership on the route is not removed so the route stays and traffic continues to flow. Aging is done by PIM. PIM does not age out entries having a VXLAN encap flag.
-
Fibre Channel over Ethernet (FCoE) N-port Virtualization (NPV) can coexist with VXLAN on different fabric uplinks but on the same or different front-panel ports on Cisco Nexus 93180YC-EX and 93180YC-FX switches.
Fibre Channel N-port Virtualization (NPV) can coexist with VXLAN on different fabric uplinks but on the same or different front-panel ports on Cisco Nexus 93180YC-FX switches. VXLAN can exist only on the Ethernet front-panel ports and not on the FC front-panel ports.
-
VXLAN is supported on the Cisco Nexus 9348GC-FXP switch.
-
VXLAN is not supported on the Cisco Nexus 92348GC switch.
-
When SVI is enabled on a VTEP (flood and learn, or EVPN), make sure that ARP-ETHER TCAM is carved using the hardware access-list tcam region arp-ether 256 command. This requirement does not apply to Cisco Nexus 9200, 9300-EX, 9300-FX/FX2/FX3, and 9300-GX platform switches and Cisco 9500 Series switches with 9700-EX line cards.
-
For information regarding the load-share keyword usage for PBR with VXLAN, see the Guidelines and Limitations for Policy-Based Routing section of the Cisco Nexus 9000 Series NX-OS Unicast Routing Configuration Guide, Release 9.3(x).
-
Beginning with Cisco NX-OS Release 9.3(3), ARP suppression is supported for Cisco Nexus 9300-GX platform switches.
-
Beginning with Cisco NX-OS Release 9.3(5), ARP suppression is supported with reflective relay for Cisco Nexus 9364C, 9300-EX, 9300-FX/FX2/FXP, and 9300-GX platform switches. For information on reflective relay, see the Cisco Nexus 9000 Series NX-OS Layer 2 Switching Configuration Guide.
-
Beginning with Cisco NX-OS Release 9.3(5), the subinterfaces on VXLAN uplinks has the ability to carry non-VXLAN L3 IP traffic for Cisco Nexus 9332C, 9364C, 9300-EX, 9300-FX/FX2/FXP, and 9300-GX platform switches and Cisco Nexus 9500 platform switches with -EX/FX line cards. This feature is supported for VXLAN flood and learn and VXLAN EVPN, VXLAN EVPN Multi-Site, and DCI.
-
Beginning with Cisco NX-OS Release 9.3(6), VXLAN flood and learn mode is supported for Cisco Nexus 9300-GX platform switches.
-
Beginning with Cisco NX-OS Release 10.1(1), VXLAN flood and learn mode is supported for N9K-C9316D-GX, N9K-C93600CD-GX, and N9K-C9364C-GX TOR switches.
-
For the Cisco Nexus 9504 and 9508 switches with -R line cards, VXLAN Layer 2 Gateway is supported on the 9636C-RX line card. VXLAN and MPLS cannot be enabled on the Cisco Nexus 9508 switch at the same time.
-
For the Cisco Nexus 9504 and 9508 switches with -R line cards, if VXLAN is enabled, the Layer 2 Gateway cannot be enabled when there is any line card other than the 9636C-RX.
-
For the Cisco Nexus 9504 and 9508 switches with -R line cards, PIM/ASM is supported in the underlay ports. PIM/Bidir is not supported. For more information, see the Cisco Nexus 9000 Series NX_OS Multicast Routing Configuration Guide, Release 9.3(x).
-
For the Cisco Nexus 9504 and 9508 switches with -R line cards, IPv6 hosts routing in the overlay is supported.
-
For the Cisco Nexus 9504 and 9508 switches with -R line cards, ARP suppression is supported.
-
For the Cisco Nexus 9504 and 9508 switches with -R line cards, VXLAN with ingress replication is not supported.
-
Beginning with Cisco NX-OS Release 10.1(1), ITD and ePBR over VXLAN feature is supported on N9K-X9716D-GX TOR and N9K-C93180YC-FX3S platform switches.
-
Beginning with Cisco NX-OS Release 10.1(1), PBR over VXLAN feature is supported on N9K-C9316D-GX, N9K-C93600CD-GX, and N9K-C9364C-GX TOR switches.
-
The load-share keyword has been added to the Configuring a Route Policy procedure for the PBR over VXLAN feature.
For more information, see the Cisco Nexus 9000 Series NX_OS Unicast Routing Configuration Guide, Release 9.x.
-
The lacp vpc-convergence command is added for better convergence of Layer 2 EVPN VXLAN:
interface port-channel10 switchport switchport mode trunk switchport trunk allowed vlan 1001-1200 spanning-tree port type edge trunk spanning-tree bpdufilter enable lacp vpc-convergence vpc 10 interface Ethernet1/34 <- The port-channel member-port is configured with LACP-active mode (for example, no changes are done at the member-port level.) switchport switchport mode trunk switchport trunk allowed vlan 1001-1200 channel-group 10 mode active no shutdown
-
Port-VLAN with VXLAN is supported on Cisco Nexus 9300-EX and 9500 Series switches with 9700-EX line cards with the following exceptions:
-
Only Layer 2 (no routing) is supported with port-VLAN with VXLAN on these switches.
-
No inner VLAN mapping is supported.
-
-
The system nve ipmc CLI command is not applicable to the Cisco 9200 and 9300-EX platform switches and Cisco 9500 platform switches with 9700-EX line cards.
-
Bind NVE to a loopback address that is separate from other loopback addresses that are required by Layer 3 protocols. A best practice is to use a dedicated loopback address for VXLAN. This best practice should be applied not only for the vPC VXLAN deployment, but for all VXLAN deployments.
-
To remove configurations from an NVE interface, we recommend manually removing each configuration rather than using the default interface nve command.
-
show commands with the internal keyword are not supported.
-
FEX ports do not support IGMP snooping on VXLAN VLANs.
-
VXLAN is supported for the Cisco Nexus 93108TC-EX and 93180YC-EX switches and for Cisco Nexus 9500 Series switches with the X9732C-EX line card.
-
DHCP snooping (Dynamic Host Configuration Protocol snooping) is not supported on VXLAN VLANs.
-
RACLs are not supported on Layer 3 uplinks for VXLAN traffic. Egress VACLs support is not available for de-capsulated packets in the network to access direction on the inner payload.
As a best practice, use PACLs/VACLs for the access to the network direction.
-
The QoS buffer-boost feature is not applicable for VXLAN traffic.
-
The following limitations apply to releases prior to Cisco NX-OS Release 9.3(5):
-
VTEPs do not support VXLAN-encapsulated traffic over subinterfaces, regardless of VRF participation or IEEE 802.1Q encapsulation.
-
VTEPs do not support VXLAN-encapsulated traffic over parent interfaces if subinterfaces are configured, regardless of VRF participation.
-
Mixing subinterfaces for VXLAN and non-VXLAN VLANs is not supported.
-
-
Beginning with Cisco NX-OS Release 10.1(1), VXLAN-encapsulated traffic over Parent Interface that Carries Subinterfaces is supported on Cisco Nexus 9300-FX3 platform switches.
-
Beginning with Cisco NX-OS Release 9.3(5), VTEPs support VXLAN-encapsulated traffic over parent interfaces if subinterfaces are configured. This feature is supported for VXLAN flood and learn, VXLAN EVPN, VXLAN EVPN Multi-Site, and DCI. As shown in the following configuration example, VXLAN traffic is forwarded on the parent interface (eth1/1) in the default VRF, and L3 IP (non-VXLAN) traffic is forwarded on subinterfaces (eth1/1.10) in the tenant VRF.
interface ethernet 1/1 description VXLAN carrying interface no switchport ip address 10.1.1.1/30 interface ethernet 1/1.10 description NO VXLAN no switchport vrf member Tenant10 encapsulation dot1q 10 ip address 10.10.1.1/30
-
Tenant VRF (VRF with VNI on it) cannot be used on an SVI that has no VNI binding into it (underlay infra VRF).
-
Point-to-multipoint Layer 3 and SVI uplinks are not supported.
-
SVI and subinterfaces as uplinks are not supported.
-
A FEX HIF (FEX host interface port) is supported for a VLAN that is extended with VXLAN.
-
In an ingress replication vPC setup, Layer 3 connectivity is needed between vPC peer devices.
-
Rollback is not supported on VXLAN VLANs that are configured with the port VLAN mapping feature.
-
The VXLAN UDP port number is used for VXLAN encapsulation. For Cisco Nexus NX-OS, the UDP port number is 4789. It complies with IETF standards and is not configurable.
-
VXLAN is supported on Cisco Nexus 9500 platform switches with the following line cards:
-
9500-R
-
9700-EX
-
9700-FX
-
-
Cisco Nexus 9300 Series switches with 100G uplinks only support VXLAN switching/bridging.
Cisco Nexus 9200, Cisco Nexus 9300-EX, and Cisco Nexus 9300-FX, and Cisco Nexus 9300-FX2 platform switches do not have this restriction.
Note
For VXLAN routing support, a 40G uplink module is required.
-
MDP is not supported for VXLAN configurations.
-
Consistency checkers are not supported for VXLAN tables.
-
ARP suppression is supported for a VNI only if the VTEP hosts the First-Hop Gateway (Distributed Anycast Gateway) for this VNI. The VTEP and SVI for this VLAN must be properly configured for the Distributed Anycast Gateway operation (for example, global anycast gateway MAC address configured and anycast gateway with the virtual IP address on the SVI).
-
ARP suppression is a per-L2VNI fabric-wide setting in the VXLAN fabric. Enable or disable this feature consistently across all VTEPs in the fabric. Inconsistent ARP suppression configuration across VTEPs is not supported.
-
The VXLAN network identifier (VNID) 16777215 is reserved and should not be configured explicitly.
-
VXLAN supports In-Service Software Upgrades (ISSUs). However, VXLAN ISSU is not supported for Cisco Nexus 9300-GX platform switches.
-
VXLAN does not support coexistence with the GRE tunnel feature or the MPLS (static or segment-routing) feature.
-
VTEP connected to FEX host interface ports is not supported.
-
If multiple VTEPs use the same multicast group address for underlay multicast but have different VNIs, the VTEPs should have at least one VNI in common. Doing so ensures that NVE peer discovery occurs and underlay multicast traffic is forwarded correctly. For example, leafs L1 and L4 could have VNI 10 and leafs L2 and L3 could have VNI 20, and both VNIs could share the same group address. When leaf L1 sends traffic to leaf L4, the traffic could pass through leaf L2 or L3. Because NVE peer L1 is not learned on leaf L2 or L3, the traffic is dropped. Therefore, VTEPs that share a group address need to have at least one VNI in common so that peer learning occurs and traffic is not dropped. This requirement applies to VXLAN bud-node topologies.
-
VXLAN does not support coexistence with MVR and MPLS for Cisco Nexus 9504 and 9508 with -R line cards.
-
Resilient hashing (port-channel load-balancing resiliency) and VXLAN configurations are not compatible with VTEPs using ALE uplink ports.
Note
Resilient hashing is disabled by default.
-
For Cisco Nexus 9504 and 9508 switches with -R line cards, the L3VNI's VLAN must be added on the vPC peer-link trunk's allowed VLAN list.
-
Native VLANs for VXLAN are not supported. All traffic on VXLAN Layer 2 trunks needs to be tagged. This limitation applies to Cisco Nexus 9300 and 9500 switches with 95xx line cards. This limitation does not apply to Cisco Nexus 9200, 9300-EX, 9300-FX, and 9500 platform switches with -EX or -FX line cards.
-
To refresh the frozen duplicate host during fabric forwarding, use only "fabric forwarding dup-host-recovery-timer " command and do not use "fabric forwarding dup-host-unfreeze-timer " command, as it is deprecated.
-
For traceroute through a VXLAN fabric when using L3VNI, the following scenario is the expected behavior:
If L3VNI is associated with a VRF and an SVI, the associated SVI does not have an L3 address that is configured but instead has the "ip forward" configuration command. Due to this interface setup it cannot respond back to the traceroute with its own SVI address. Instead, when a traceroute involving the L3VNI is run through the fabric, the IP address reported will be the lowest IP address of an SVI that belongs to the corresponding tenant VRF.
-
Routing protocol adjacencies using Anycast Gateway SVIs is not supported.