Configuring Seamless Integration of EVPN (TRM) with MVPN

This chapter contains the following sections:

About Seamless Integration of EVPN (TRM) with MVPN (Draft Rosen)

Seamless integration of EVPN (TRM) with MVPN (draft rosen) enables packets to be handed off between a VXLAN network (TRM or TRM Multi-Site) and an MVPN network. To support this feature, VXLAN TRM and MVPN must be supported on a Cisco Nexus device node, the handoff node.

The handoff node is the PE for the MVPN network and the VTEP for the VXLAN network. It connects to the VXLAN, MVPN, and IP multicast networks, as shown in the following figure.

Figure 1. VXLAN - MVPN Handoff Network

Sources and receivers can be in any of the three networks (VXLAN, MVPN, or IP multicast).

All multicast traffic (that is, the tenant traffic from the VXLAN, MVPN, or multicast network) is routed from one domain to another domain. The handoff node acts as the central node. It performs the necessary packet forwarding, encapsulation, and decapsulation to send the traffic to the respective receivers.

Supported RP Positions

The rendezvous point (RP) for the customer (overlay) network can be in any of the three networks (VXLAN, MVPN, or IP multicast).

Table 1. Supported RP Locations

RP Locations

Description

RP in IP network

  • The RP can be connected only to the MVPN PE and not to the handoff nodes.

  • The RP can be connected only to the VXLAN handoff nodes.

  • The RP can be connected to both the MVPN PE and VXLAN.

RP internal to VXLAN fabric

All VTEPs are RPs inside the VXLAN fabric. All MVPN PEs use the RP configured on the VXLAN fabric.

RP on VXLAN MVPN handoff node

The RP is the VXLAN MVPN handoff node.

RP in MVPN network

The RP is external to the VXLAN network. It's configured on one of the nodes in the MPLS cloud, other than the handoff node.

RP Everywhere (PIM Anycast RP or MSDP-based Anycast RP)

The Anycast RP can be configured on the VXLAN leaf. The RP set can be configured on the handoff node or any MVPN PE.

Guidelines and Limitations for Seamless Integration of EVPN (TRM) with MVPN

This feature has the following guidelines and limitations:

  • Only Cisco Nexus 9504 and 9508 platform switches with the N9K-X9636C-RX line card support seamless integration of EVPN (TRM) with MVPN. Other -R Series line cards can't function as the handoff node.

  • The handoff node can have local (directly connected) multicast sources or receivers for the customer network.

  • Any existing underlay properties, such as ASM/SSM for MVPN or ASM for TRM, are supported on the handoff node.

  • The handoff node supports PIM SSM and ASM for the overlay.

  • Inter-AS option A is supported on the handoff node toward the IP multicast network.

  • If the number of MDT source loopback IP addresses and NVE loopback IP addresses exceeds the maximum limit, traffic drops might occur.

  • The following functionality isn't supported for seamless integration of EVPN (TRM) with MVPN:

    • vPC on the handoff node

    • VXLAN ingress replication

    • SVIs and subinterfaces as core-facing interfaces for MVPN

    • Inter-AS options B and C on MVPN nodes

    • PIM SSM as a VXLAN underlay

    • Bidirectional PIM as an underlay or overlay

    • ECMP with a mix of MPLS and IP paths

  • Any existing limitations for VXLAN, TRM, and MVPN also apply to seamless integration of EVPN (TRM) with MVPN.

Configuring the Handoff Node for Seamless Integration of EVPN (TRM) with MVPN

This section documents the configurations that are required on the handoff node. Configurations for other nodes (such as VXLAN leafs and spines, MVPN PE, and RS/RR) are the same as in previous releases.

PIM/IGMP Configuration for the Handoff Node

Follow these guidelines when configuring PIM/IGMP for the handoff node:

  • Make sure that the Rendezvous Point (RP) is different for TRM and the MVPN underlay, as shown in the following example.

    ip pim rp-address 90.1.1.100 group-list 225.0.0.0/8 --- TRM Underlay
    ip pim rp-address 91.1.1.100 group-list 233.0.0.0/8 --- MVPN Underlay
    
    
  • Use a common RP for overlay multicast traffic.

  • The RP can be in static, PIM Anycast, or PIM MSDP mode. The following example shows the RP configuration inside the VRF:

    vrf context vrfVxLAN5001  
      vni 5001                
      ip pim rp-address 111.1.1.1 group-list 226.0.0.0/8
      ip pim rp-address 112.2.1.1 group-list 227.0.0.0/8
    
    
  • Enable IGMP snooping for VXLAN traffic using the ip igmp snooping vxlan command.

  • Enable PIM sparse mode on all source interfaces and interfaces required to carry PIM traffic.

BGP Configuration for the Handoff Node

Follow these guidelines when configuring BGP for the handoff node:

  • Add all VXLAN leafs as L2EVPN and TRM neighbors; include the redundant handoff node. If a route reflector is used, add only RR as a neighbor.

  • Add all MVPN PEs as VPN neighbors. In MDT mode, add the MVPN PEs as MDT neighbors.

  • Import configuration to advertise unicast routes from L2EVPN neighbors to VPN neighbors and vice versa.

  • The BGP source identifier can be different or the same as the source interfaces used for the VTEP identifier (configured under the NVE interface)/MVPN PE identifier.

    feature bgp
    address-family ipv4 mdt
    address-family ipv4 mvpn
    
    neighbor 2.1.1.1
      address-family ipv4 mvpn
        send-community extended
      address-family l2vpn evpn
        send-community extended
        import vpn unicast reoriginate
    
    neighbor 30.30.30.30
      address-family vpnv4 unicast
        send-community
        send-community extended
        next-hop-self
        import l2vpn evpn reoriginate
      address-family ipv4 mdt
        send-community extended
        no next-hop-third-party
    
    
  • Never use Inter-AS option B between MVPN peers. Instead, configure the no allocate-label option-b command under the VPNv4 unicast address family.

    address-family vpnv4 unicast
        no allocate-label option-b
    
    
  • Set maximum paths should be set in EBGP mode.

    address-family l2vpn evpn
        maximum-paths 8
    vrf vrfVxLAN5001
        address-family ipv4 unicast
            maximum-paths 8
    
    
  • If handoff nodes are deployed in dual mode, use the route-map command to avoid advertising prefixes associated with orphan hosts under the VPN address family.

    ip prefix-list ROUTES_CONNECTED_NON_LOCAL seq 2 premit 15.14.0.15/32
    
    route-map ROUTES_CONNECTED_NON_LOCAL deny
        match ip address prefix-list ROUTES_CONNECTED_NON_LOCAL
    
    neighbor 8.8.8.8
        remote-as 100
        update-source loopback1
        address-family vpnv4 unicast
          send-community
          send-community extended
          route-map ROUTES_CONNECTED_NON_LOCAL out
    
    

VXLAN Configuration for the Handoff Node

Follow these guidelines when configuring VXLAN for the handoff node:

  • Enable the following features:

    feature nv overlay
    feature ngmvpn
    feature interface-vlan
    feature vn-segment-vlan-based
    
    
  • Configure the required L3 VNI:

    L3VNIs are mapped to tenant VRF. 
    vlan 2501
      vn-segment 5001 <-- Associate VNI to a VLAN.
    
    
  • Configure the NVE interface:

    interface nve1
      no shutdown
      host-reachability protocol bgp
      source-interface loopback1 <-- This interface should not be the same as the MVPN source interface.
      global suppress-arp
    member vni 5001 associate-vrf <-- L3VNI
      mcast-group 233.1.1.1 <-- The underlay multicast group for VXLAN should be different from the MVPN default/data MDT.
    
    
  • Configure the tenant VRF:

    vrf context vrfVxLAN5001
      vni 5001 <-- Associate VNI to VRF.
      rd auto
    address-family ipv4 unicast
        route-target both auto
        route-target both auto mvpn
        route-target both auto evpn
    
    interface Vlan2501 <-- SVI interface associated with the L3VNI
      no shutdown
      mtu 9216 <-- The overlay header requires 58 byes, so the max tenant traffic is (Configured MTU – 58).
      vrf member vrfVxLAN5001
      no ip redirects
      ip forward
      ipv6 forward
      no ipv6 redirects
      ip pim sparse-mode <-- PIM is enabled.
    
    interface Vlan2 <-- SVI interface associated with L2 VNI
      no shutdown
      vrf member vrfVxLAN5001
      no ip redirects
      ip address 100.1.1.1/16
      no ipv6 redirects
      ip pim sparse-mode <-- PIM enabled on L2VNI
     fabric forwarding mode anycast-gateway
    
    

MVPN Configuration for the Handoff Node

Follow these guidelines when configuring MVPN for the handoff node:

  • Enable the following features:

    install feature-set mpls
    allow feature-set mpls
    feature-set mpls
    feature mpls l3vpn
    feature mvpn
    feature mpls ldp
    
    
  • MPLS LDP Configuration:

    • Enable MPLS LDP (mpls ip ) on all interfaces that are MPLS links.

    • Do not advertise loopback interfaces used for VXLAN as MPLS prefixes.

      • Configure a prefix list that contains IP addresses that identify the MVPN PE node.

        ip prefix-list LDP-LOOPBACK seq 51 permit 9.1.1.10/32
        ip prefix-list LDP-LOOPBACK seq 52 permit 9.1.2.10/32
        
        
      • Configure label allocation only for MVPN PE identifiers.

        mpls ldp configuration
          explicit-null
          advertise-labels for LDP-LOOPBACK
          label allocate global prefix-list LDP-LOOPBACK
        
        
  • Tenant VRF Configuration:

    • For the default MDT mode, make the underlay multicast group the same for all tenant multicast traffic under the VRF.

      vrf context vrfVxLAN5001
        vni 5001
        mdt default 225.1.100.1
        mdt source loopback100 <-- If the source interface is not configured, the BGP identifier is used as the source interface.
        mdt asm-use-shared-tree <-- If the underlay is configured in ASM mode
        no mdt enforce-bgp-mdt-safi <-- Enabled by befault but should be negated if BGP MDT should not be used for discovery.
        mdt mtu <mtu-value> <-- Overlay ENCAP Max MTU value
      
      
    • For the data MDT mode, configure a unique multicast group-set for a subset of or all tenant multicast traffic.

      mdt data 229.1.100.2/32 immediate-switch
      mdt data 232.1.10.4/24 immediate-switch
      route-map DATA_MDT_MAP permit 10
        match ip multicast group 237.1.1.1/32
      mdt data 235.1.1.1/32 immediate-switch route-map DATA_MDT_MAP
      
      
  • Enable MVPN tunnel statistics.

    hardware profile mvpn-stats module all

CoPP Configuration for the Handoff Node

Both TRM and MVPN are heavily dependent on the control plane. Make sure to set the CoPP policy bandwidth as per the topology.

The following CoPP classes are used for TRM and MVPN traffic:

  • copp-system-p-class-multicast-router (The default bandwidth is 3000 pps.)

  • copp-system-p-class-l3mc-data (The default bandwidth is 3000 pps.)

  • copp-system-p-class-l2-default (The default bandwidth is 50 pps.)

  • copp-class-normal-igmp (The default bandwidth is 6000 pps.)

The following configuration example shows CoPP policies that can be configured to avoid control packet drops with multicast route scale.


Note


The policer values in this example are approximations and might not be optimal for all topologies or traffic patterns. Configure the CoPP policies according to the MVPN/TRM traffic pattern.


copp copy profile strict prefix custom
  policy-map type control-plane custom-copp-policy-strict
    class custom-copp-class-normal-igmp 
      police cir 6000 pps bc 512 packets conform transmit violate drop
  control-plane
    service-policy input custom-copp-policy-strict

copp copy profile strict prefix custom 
  policy-map type control-plane custom-copp-policy-strict 
    class custom-copp-class-multicast-router
      police cir 6000 pps bc 512 packets conform transmit violate drop 
  control-plane 
    service-policy input custom-copp-policy-strict

copp copy profile strict prefix custom
  policy-map type control-plane custom-copp-policy-strict 
    class copp-system-p-class-l3mc-data 
      police cir 3000 pps bc 512 packets conform transmit violate drop
  control-plane
    service-policy input custom-copp-policy-strict

copp copy profile strict prefix custom
  policy-map type control-plane custom-copp-policy-strict
    class custom-copp-class-l2-default
      police cir 9000 pps bc 512 packets conform transmit violate drop
  control-plane
    service-policy input custom-copp-policy-strict

Configuration Example for Seamless Integration of EVPN (TRM) with MVPN

The following figure shows a sample topology with a VXLAN network on the left, an MVPN network on the right, and a centralized handoff node.

Figure 2. Sample Topology for Seamless Integration of EVPN (TRM) with MVPN

The following example show sample configurations for the VTEP, handoff node, and PE in this topology.

Configuration on VTEP1:

feature ngmvpn
feature interface-vlan
feature vn-segment-vlan-based
feature nv overlay
feature pim
nv overlay evpn
ip pim rp-address 90.1.1.100 group-list 225.0.0.0/8
ip pim ssm range 232.0.0.0/8

vlan 555
  vn-segment 55500

route-map ALL_ROUTES permit 10
interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback2
  member vni 55500 associate-vrf
    mcast-group 225.3.3.3

interface loopack1
  ip address 196.196.196.196/32

interface loopback2
  ip address 197.197.197.197/32
  ip pim sparse-mode

feature bgp
router bgp 1
    address-family l2vpn evpn
        maximum-paths 8
        maximum-paths ibgp 8
    neighbor 2.1.1.2
        remote-as 1
        update-source loopback 1
        address-family ipv4 unicast
          send-community extended
        address-family ipv6 unicast
          send-community extended
        address-family ipv4 mvpn
          send-community extended
        address-family l2vpn evpn
          send-community extended
    vrf vrfVxLAN5023
        address-family ipv4 unicast
          advertise l2vpn evpn
          redistribute direct route-map ALL_ROUTES
          maximum-paths 8
          maximum-paths ibgp 8

vrf context vpn1
  vni 55500
  ip pim rp-address 27.27.27.27 group-list 224.0.0.0/4
  ip pim ssm range 232.0.0.0/8
  ip multicast multipath s-g-hash next-hop-based
rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto mvpn
    route-target both auto evpn
  
interface Vlan555
  no shutdown
  vrf member vpn1
  ip forward
  ip pim sparse-mode

interface Ethernet 1/50
  ip pim sparse-mode

interface Ethernet1/5.1
  encapsulation dot1q 90
  vrf member vpn1
  ip address 10.11.12.13/24
  ip pim sparse-mode
  no shutdown

Configuration on the handoff node:

install feature-set mpls
  allow feature-set mpls
feature-set mpls
feature ngmvpn
feature bgp
feature pim
feature mpls l3vpn
feature mvpn
feature mpls ldp
feature interface-vlan
feature vn-segment-vlan-based
feature nv overlay
nv overlay evpn

ip pim rp-address 90.1.1.100 group-list 225.0.0.0/8
ip pim rp-address 91.1.1.100 group-list 232.0.0.0/8

interface loopback1
  ip address 90.1.1.100 /32
  ip pim sparse-mode

interface loopback2
  ip address 91.1.1.100 /32
  ip pim sparse-mode

ip prefix-list LDP-LOOPBACK seq 2 permit 20.20.20.20/32
ip prefix-list LDP-LOOPBACK seq 3 permit 30.30.30.30/32
mpls ldp configuration
    advertise-labels for LDP-LOOPBACK
    label allocate label global prefix-list LDP-LOOPBACK

interface Ethernet 1/50
    ip pim sparse-mode

interface Ethernet 1/51
    ip pim sparse-mode
    mpls ip

interface Ethernet1/4.1
  encapsulation dot1q 50
  vrf member vpn1
  ip pim sparse-mode
  no shutdown

interface loopback0
  ip address 20.20.20.20/32
  ip pim sparse-mode

vlan 555
  vn-segment 55500

route-map ALL_ROUTES permit 10

interface nve1
  no shutdown
  host-reachability protocol bgp
  source-interface loopback3
  member vni 55500 associate-vrf
    mcast-group 225.3.3.3

interface loopback3
  ip address 198.198.198.198/32
  ip pim sparse-mode

vrf context vpn1
  vni 55500
  ip pim rp-address 27.27.27.27 group-list 224.0.0.0/4
  ip pim ssm range 232.0.0.0/8
  ip multicast multipath s-g-hash next-hop-based
  mdt default 232.1.1.1
  mdt source loopback 0
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto mvpn
    route-target both auto evpn
  
interface Vlan555
  no shutdown
  vrf member vpn1
  ip forward
  ip pim sparse-mode

router bgp 1
    address-family l2vpn evpn
        maximum-paths 8
        maximum-paths ibgp 8
    address-family vpnv4 unicast
        no allocate-label option-b
    address-family ipv4 mdt
    address-family ipv4 mvpn
        maximum-paths 8
        maximum-paths ibgp 8
    neighbor 196.196.196.196
        remote-as 1
        address-family ipv4 unicast
          send-community extended
        address-family ipv6 unicast
          send-community extended
        address-family ipv4 mvpn
          send-community extended
        address-family l2vpn evpn
          send-community extended
          import vpn unicast reoriginate

router bgp 1
    neighbor 30.30.30.30
        remote-as 100
        update-source loopback0
        ebgp-multihop 255
        address-family ipv4 unicast
          send-community extended
        address-family vpnv4 unicast
          send-community
          send-community extended
          next-hop-self
          import l2vpn evpn reoriginate
        address-family ipv4 mdt
          send-community extended
          no next-hop-third-party

Configuration on PE2:

install feature-set mpls
  allow feature-set mpls
feature-set mpls
feature bgp
feature pim
feature mpls l3vpn
feature mpls ldp
feature interface-vlan

ip pim rp-address 91.1.1.100 group-list 232.0.0.0/8
ip prefix-list LDP-LOOPBACK seq 2 permit 20.20.20.20/32
ip prefix-list LDP-LOOPBACK seq 3 permit 30.30.30.30/32
mpls ldp configuration
    advertise-labels for LDP-LOOPBACK
    label allocate label global prefix-list LDP-LOOPBACK

interface Ethernet 1/51
    ip pim sparse-mode
    mpls ip

interface Ethernet1/6.1
  encapsulation dot1q 50
  vrf member vpn1
  ip pim sparse-mode
  no shutdown

interface loopback0
  ip address 30.30.30.30/32
  ip pim sparse-mode

vrf context vpn1
  ip pim rp-address 27.27.27.27 group-list 224.0.0.0/4
  ip pim ssm range 232.0.0.0/8
  ip multicast multipath s-g-hash next-hop-based
  mdt default 232.1.1.1
  mdt source loopback 0
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto mvpn
    route-target both auto evpn

router bgp 100
      router-id 30.30.30.30
      address-family vpnv4 unicast
            additional-paths send
            additional-paths receive
            no allocate-label option-b
      neighbor 20.20.20.20
            remote-as 1
            update-source loopback0
            address-family vpnv4 unicast
                send-community
                send-community extended
            address-family ipv4 mdt
                send-community extended
                no next-hop-third-party