Segment Routing Application (SR-APP) module is used to configure the segment routing functionality. Segment Routing Application (SR-APP) is a separate internal process that handles all the CLIs related to segment routing. It is responsible for reserving the SRGB range and for notifying the clients about it. It is also responsible for maintaining the prefix to SID mappings. The SR-APP support is also available for the BGP, IS-IS, and OSPF protocols.

The SR-APP module maintains the following information:

• Segment routing operation state

• Segment routing global block label ranges

• Prefix SID mappings

For more information, see Configuring Segment Routing.

NetFlow identifies packet flows for ingress IP packets and provides statistics that are based on these packet flows. NetFlow does not require any change to either the packets themselves or to any networking device. You can export the data that NetFlow gathers for your flow by using a flow exporter and export this data to a remote NetFlow Collector, such as Cisco Stealthwatch. Cisco NX-OS exports flow as part of a NetFlow export User Datagram Protocol (UDP) datagram. You can export the data that NetFlow gathers for your flow by using a flow exporter and export this data to a remote NetFlow Collector, such as Cisco Stealthwatch. Cisco NX-OS exports a flow as part of a NetFlow export User Datagram Protocol (UDP) datagram.

Beginning with Cisco NX-OS Release 9.3(1), NetFlow Collector over segment routing is supported on Cisco Nexus 9300-EX, 9300-FX, 9300-FX2, 9500-EX, and 9500-FX platform switches.

Beginning with Cisco NX-OS Release 9.3(5), NetFlow Collector over segment routing is supported on Cisco Nexus 9300-FX3 platform switches.

NetFlow is not supported on Cisco Nexus 9300-GX platform switches..

NetFlow Collector supports both, single and double MPLS labels. Both, default and the non-default VRF in the exporter destination configurations is supported. NetFlow does not support an MPLS data path.

Since segment routing does not support a single label, you must configure the address-family ipv4 labeled-unicast command under BGP neighbor and the allocate-label command under the bgp configuration.

Sampled flow (sFlow) allows you to monitor real-time traffic in data networks that contain switches and routers. It uses the sampling mechanism in the sFlow agent software on switches and routers to monitor traffic and to forward the sample data to the central data collector.

Beginning with Cisco NX-OS Release 9.3(1), sFlow collector over segment routing is supported on Cisco Nexus 9300-EX, 9300-FX, 9300-FX2, 9500-EX, and 9500-FX platform switches.

Beginning Cisco NX-OS Release 9.3(5), sFlow collector over segment routing is supported on Cisco Nexus 9300-FX3 platform switches.

sFlow is not supported on Cisco Nexus 9364C-GX, Cisco Nexus 9316D-GX, and Cisco Nexus 93600CD-GX switches.

For information on configuring sFlow, see the Configuring sFlow section in the Cisco Nexus 9000 Series NX-OS System Management Configuration Guide, Release 9.3(x).

The examples in this section show a common BGP prefix SID configuration between two routers.

This example shows how to advertise a BGP speaker configuration of 10.10.10.10/32 and 20.20.20.20/32 with a label index of 10 and 20, respectively. It uses the default segment routing global block (SRGB) range of 16000 to 23999.

hostname s1
install feature-set mpls
feature-set mpls

feature telnet
feature bash-shell
feature scp-server
feature bgp
feature mpls segment-routing

segment-routing 
  mpls
  vlan 1
segment-routing
  mpls
    connected-prefix-sid-map
    address-family ipv4
    2.1.1.1/32 absolute 100100

route-map label-index-10 permit 10
  set label-index 10
route-map label-index-20 permit 10
  set label-index 20

vrf context management
  ip route 0.0.0.0/0 10.30.108.1

interface Ethernet1/1
  no switchport
  ip address 10.1.1.1/24
  no shutdown

interface mgmt0
  ip address dhcp
  vrf member management
 
interface loopback1
  ip address 10.10.10.10/32

interface loopback2
  ip address 20.20.20.20/32

line console
line vty

router bgp 1
  address-family ipv4 unicast
    network 10.10.10.10/32 route-map label-index-10
    network 20.20.20.20/32 route-map label-index-20
    allocate-label all
  neighbor 10.1.1.2 remote-as 2
    address-family ipv4 labeled-unicast

This example shows how to receive the configuration from a BGP speaker.

hostname s2
install feature-set mpls
feature-set mpls

feature telnet
feature bash-shell
feature scp-server
feature bgp
feature mpls segment-routing

segment-routing mpls
vlan 1

vrf context management
  ip route 0.0.0.0/0 10.30.97.1
  ip route 0.0.0.0/0 10.30.108.1

interface Ethernet1/1
  no switchport
  ip address 10.1.1.2/24
  ipv6 address 10:1:1::2/64
  no shutdown

interface mgmt0
  ip address dhcp
  vrf member management

interface loopback1
  ip address 2.2.2.2/32
line console

line vty

router bgp 2
  address-family ipv4 unicast
    allocate-label all
  neighbor 10.1.1.1 remote-as 1
    address-family ipv4 labeled-unicast

This example shows how to display the configuration from a BGP speaker. The show command in this example displays the prefix 10.10.10.10 with label index 10 mapping to label 16010 in the SRGB range of 16000 to 23999.

switch# show bgp ipv4 labeled-unicast 10.10.10.10/32

BGP routing table information for VRF default, address family IPv4 Label Unicast
BGP routing table entry for 10.10.10.10/32, version 7
Paths: (1 available, best #1)
Flags: (0x20c001a) on xmit-list, is in urib, is best urib route, is in HW, , has label
  label af: version 8, (0x100002) on xmit-list
  local label: 16010

  Advertised path-id 1, Label AF advertised path-id 1
  Path type: external, path is valid, is best path, no labeled nexthop, in rib
  AS-Path: 1 , path sourced external to AS
    10.1.1.1 (metric 0) from 10.1.1.1 (10.10.10.10)
      Origin IGP, MED not set, localpref 100, weight 0
      Received label 0
      Prefix-SID Attribute: Length: 10
        Label Index TLV: Length 7, Flags 0x0 Label Index 10

  Path-id 1 not advertised to any peer
  Label AF advertisement
  Path-id 1 not advertised to any peer

This example shows how to configure egress peer engineering on a BGP speaker.

hostname epe-as-1
install feature-set mpls
feature-set mpls

feature telnet
feature bash-shell
feature scp-server
feature bgp
feature mpls segment-routing

segment-routing mpls
vlan 1

vrf context management
  ip route 0.0.0.0/0 10.30.97.1
  ip route 0.0.0.0/0 10.30.108.1

interface Ethernet1/1
  no switchport
  ip address 10.1.1.1/24
  no shutdown

interface Ethernet1/2
  no switchport
  ip address 11.1.1.1/24
  no shutdown

interface Ethernet1/3
  no switchport
  ip address 12.1.1.1/24
  no shutdown

interface Ethernet1/4
  no switchport
  ip address 13.1.1.1/24
  no shutdown

interface Ethernet1/5
  no switchport
  ip address 14.1.1.1/24
  no shutdown

The following is an example of show ip route vrf 2 command.

show ip route vrf 2
IP Route Table for VRF "2"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

41.11.2.0/24, ubest/mbest: 1/0
    *via 1.1.1.9%default, [20/0], 13:26:48, bgp-2, external, tag 11 (mpls-vpn)
42.11.2.0/24, ubest/mbest: 1/0, attached
    *via 42.11.2.1, Vlan2, [0/0], 13:40:52, direct
42.11.2.1/32, ubest/mbest: 1/0, attached
    *via 42.11.2.1, Vlan2, [0/0], 13:40:52, local

The following is an example of show forwarding route vrf 2 command.

slot  1
=======

IPv4 routes for table 2/base

------------------+-----------------------------------------+----------------------+-----------------+-----------------
Prefix            | Next-hop                                | Interface            | Labels          | Partial Install 
------------------+-----------------------------------------+----------------------+-----------------+-----------------
0.0.0.0/32           Drop                                      Null0
127.0.0.0/8          Drop                                      Null0
255.255.255.255/32   Receive                                   sup-eth1
*41.11.2.0/24        27.1.31.4                                 Ethernet1/3            PUSH  30002 492529 
                     27.1.32.4                                 Ethernet1/21           PUSH  30002 492529 
                     27.1.33.4                                 port-channel23         PUSH  30002 492529 
                     27.11.31.4                                Ethernet1/3.11         PUSH  30002 492529 
                     27.11.33.4                                port-channel23.11      PUSH  30002 492529 
                     37.1.53.4                                 Ethernet1/53/1         PUSH  29002 492529 
                     37.1.54.4                                 Ethernet1/54/1         PUSH  29002 492529 
                     37.2.53.4                                 Ethernet1/53/2         PUSH  29002 492529 
                     37.2.54.4                                 Ethernet1/54/2         PUSH  29002 492529 
                     80.211.11.1                               Vlan801                PUSH  30002 492529 



 

The following is an example of show bgp l2vpn evpn summary command.

show bgp l2vpn evpn summary 
BGP summary information for VRF default, address family L2VPN EVPN
BGP router identifier 2.2.2.3, local AS number 2
BGP table version is 17370542, L2VPN EVPN config peers 4, capable peers 1
1428 network entries and 1428 paths using 268464 bytes of memory
BGP attribute entries [476/76160], BGP AS path entries [1/6]
BGP community entries [0/0], BGP clusterlist entries [0/0]
476 received paths for inbound soft reconfiguration
476 identical, 0 modified, 0 filtered received paths using 0 bytes

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
1.1.1.1         4    11       0       0        0    0    0 23:01:53 Shut (Admin)
1.1.1.9         4    11    4637    1836 17370542    0    0 23:01:40 476       
1.1.1.10        4    11       0       0        0    0    0 23:01:53 Shut (Admin)
1.1.1.11        4    11       0       0        0    0    0 23:01:52 Shut (Admin)


 

The following is an example of show bgp l2vpn evpn command.

show bgp l2vpn evpn 41.11.2.0 
BGP routing table information for VRF default, address family L2VPN EVPN
Route Distinguisher: 14.1.4.1:115
BGP routing table entry for [5]:[0]:[0]:[24]:[41.11.2.0]:[0.0.0.0]/224, version 17369591
Paths: (1 available, best #1)
Flags: (0x000002) on xmit-list, is not in l2rib/evpn, is not in HW

  Advertised path-id 1
  Path type: external, path is valid, received and used, is best path
             Imported to 2 destination(s)
  AS-Path: 11 , path sourced external to AS
    1.1.1.9 (metric 0) from 1.1.1.9 (14.1.4.1)
      Origin incomplete, MED 0, localpref 100, weight 0
      Received label 492529
      Extcommunity: RT:2:20

  Path-id 1 not advertised to any peer

Route Distinguisher: 2.2.2.3:113
BGP routing table entry for [5]:[0]:[0]:[24]:[41.11.2.0]:[0.0.0.0]/224, version 17369595
Paths: (1 available, best #1)
Flags: (0x000002) on xmit-list, is not in l2rib/evpn, is not in HW

  Advertised path-id 1
  Path type: external, path is valid, is best path
             Imported from 14.1.4.1:115:[5]:[0]:[0]:[24]:[41.11.2.0]:[0.0.0.0]/224 
  AS-Path: 11 , path sourced external to AS
    1.1.1.9 (metric 0) from 1.1.1.9 (14.1.4.1)

IS-IS is an Interior Gateway Protocol (IGP) based on Standardization (ISO)/International Engineering Consortium (IEC) 10589 and RFC 1995. Cisco NX-OS supports Internet Protocol version 4 (IPv4) and IPv6. IS-IS is a dynamic link-state routing protocol that can detect changes in the network topology and calculate loop-free routes to other nodes in the network. Each router maintains a link-state database that describes the state of the network and sends packets on every configured link to discover neighbors. IS-IS floods the link-state information across the network to each neighbor. The router also sends advertisements and updates on the link-state database through all the existing neighbors

Segment routing on the IS-IS protocol supports the following:

• IPv4

• Level 1, level 2, and multi-level routing

• Prefix SIDs

• Multiple IS-IS instances on the same loopback interface for domain border nodes

• Adjacency SIDs for adjacencies

Open Shortest Path First (OSPF) is an Interior Gateway Protocol (IGP) developed by the OSPF working group of the Internet Engineering Task Force (IETF). Designed expressly for IP networks, OSPF supports IP subnetting and tagging of externally derived routing information. OSPF also allows packet authentication and uses IP multicast when sending and receiving packets.

Segment routing configuration on the OSPF protocol can be applied at the process or the area level. If you configure segment routing at the process level, it is enabled for all the areas. However, you can enable ore disable it per area level.

Segment routing on the OSPF protocol supports the following:

• OSPFv2 control plane

• Multi-area

• IPv4 prefix SIDs for host prefixes on loopback interfaces

• Adjacency SIDs for adjacencies

OSPF supports the advertisement of segment routing adjacency SID. An Adjacency Segment Identifier (Adj-SID) represents a router adjacency in Segment Routing.

A segment routing-capable router may allocate an Adj-SID for each of its adjacencies and an Adj-SID sub-TLV is defined to carry this SID in the Extended Opaque Link LSA.

OSPF allocates the adjacency SID for each OSPF neighbor if the OSPF adjacency which are in two way or in FULL state. OSPF allocates the adjacency SID only if the segment routing is enabled. The label for adjacency SID is dynamically allocated by the system. This eliminates the chances of misconfiguration, as this has got only the local significance.

OSPFv2 supports the advertisement of prefix SID for address associated with the loopback interfaces. In order to achieve this, OSPF uses Extended Prefix Sub TLV in its opaque Extended prefix LSA. When OSPF receives this LSA from its neighbor, SR label is added to the RIB corresponding to received prefix based upon the information present in extended prefix sub TLV.

For configuration, segment-routing has to be enabled under OSPF and corresponding to loopback interface that is configured with OSPF, prefix-sid mapping is required under the segment routing module.

Note	SID will only be advertised for loopback addresses and only for intra-area and inter-area prefix types. No SID value will be advertised for external or NSSA prefixes.

To provide segment routing support across the area boundary, OSPF is required to propagate SID values between areas. When OSPF advertises the prefix reachability between areas, it checks if the SID has been advertised for the prefix. In a typical case, the SID value come from the router, which contributes to the best path to the prefix in the source area. In this case, OSPF uses such SID and advertises it between the areas. If the SID value is not advertised by the router which contributes to the best path inside the area, OSPF will use the SID value coming from any other router inside the source area.

OSPF advertises it's segment routing capability in terms of advertising the SID/Label Range TLV. In OSPFv2, SID/Label Range TLV is a carried in Router Information LSA.

The segment routing global range configuration will be under the “segment-routing mpls” configuration. When the OSPF process comes, it will get the global range values from segment-routing and subsequent changes should be propagated to it.

When OSPF segment routing is configured, OSPF must request an interaction with the segment routing module before OSPF segment routing operational state can be enabled. If the SRGB range is not created, OSPF will not be enabled. When an SRGB change event occurs, OSPF makes the corresponding changes in it's sub-block entries.

In an ideal situation, each prefix should have unique SID entries assigned.

When there is a conflict between the SID entries and the associated prefix entries use any of the following methods to resolve the conflict:

• Multiple SIDs for a single prefix - If the same prefix is advertised by multiple sources with different SIDs, OSPF will install the unlabeled path for the prefix. The OSPF takes into consideration only those SIDs that are from reachable routers and ignores those from unreachable routers. When multiple SIDs are advertised for a prefix, which is considered as a conflict, no SID will be advertised to the attached-areas for the prefix. Similar logic will be used when propagating the inter-area prefixes between the backbone and the non-backbone areas.

• Out of Range SID - For SIDs that do not fit in our SID range, labels are not used while updating the RIB.

MPLS forwarding must be enabled before segment routing can use an interface. OSPF is responsible for enabling MPLS forwarding on an interface.

When segment routing is enabled for a OSPF topology, or OSPF segment routing operational state is enabled, it enables MPLS for any interface on which the OSPF topology is active. Similarly, when segment routing is disabled for a OSPF topology, it disables the MPLS forwarding on all interfaces for that topology.

MPLS forwarding is not supported on an interface which terminates at the IPIP/GRE tunnel.

Segment routing for traffic engineering (SR-TE) takes place through a tunnel between a source and destination pair. Segment routing for traffic engineering uses the concept of source routing, where the source calculates the path and encodes it in the packet header as a segment. A Traffic Engineered (TE) tunnel is a container of TE LSPs instantiated between the tunnel ingress and the tunnel destination. A TE tunnel can instantiate one or more SR-TE LSPs that are associated with the same tunnel.

With segment routing for traffic engineering (SR-TE), the network no longer needs to maintain a per-application and per-flow state. Instead, it simply obeys the forwarding instructions provided in the packet.

SR-TE utilizes network bandwidth more effectively than traditional MPLS-TE networks by using ECMP at every segment level. It uses a single intelligent source and relieves remaining routers from the task of calculating the required path through the network.

SR-TE has the following guidelines and limitations:

• SR-TE ODN for both, IPv4 and IPv6 overlay is supported.

• SR-TE ODN is supported only with IS-IS underlay.

• Forwarding does not support routes with recursive next hops, where the recursive next hop resolves to a route with a binding SID.

• Forwarding does not support mixing paths with binding labels and paths without binding labels for the same route.

• The affinity and disjoint constraints are applicable only to those SR-TE policies that have a dynamic PCEP option.

• XTC supports only two policies with disjointness in the same group.

• When configuring the SR-TE affinity interfaces, the interface range is not supported.

• A preference cannot have both, the dynamic PCEP and the explicit segment lists configured together for the same preference.

• Only one preference can have a dynamic PCEP option per policy.

• For explicit policy, when configuring ECMP paths under same preference, if the first hop (NHLFE) is same for both the ECMP paths, ULIB will only install one path in switching. This occurs because both the ECMP paths create the same SRTE FEC as the NHLFE is same for both.

• In Cisco NX-OS Release 9.3(1), unprotected mode with affinity configuration is not supported by PCE (XTC).

• Beginning with Cisco NX-OS Release 9.3(3), SR-TE ODN, policies, policy paths, and the affinity and disjoint constraints are supported on Cisco Nexus 9364C-GX, Cisco Nexus 9316D-GX, and Cisco Nexus 93600CD-GX switches.

• Beginning with Cisco NX-OS Release 10.2(2)F, few new show commands for SR-TE policy are introduced and the autocomplete feature is provided for some of the existing SR-TE policy commands to improve usability. This feature is supported on Cisco Nexus 9300-EX, 9300-FX, 9300-FX2, 9300-GX, and N9K-C9332D-GX2B platform switches.

Note	For more information about the Cisco Nexus 9000 switches that support various features spanning release 7.0(3)I7(1) to the current release, refer to Nexus Switch Platform Support Matrix.

The examples in this section show affinity and disjoint configurations.

segment-routing
 traffic-eng
  affinity-map
   color green bit-position 0
   color blue bit-position 2
   color red bit-position 3

segment-routing
 traffic-eng
  interface eth1/1
   affinity
    color red
    color green
  !
  interface eth1/2
   affinity
    color green

segment-routing
  traffic-engineering
    affinity-map
      color blue bit-position 0
      color red  bit-position 1
    on-demand color 10
      candidate-paths
        preference 100 
          dynamic
            pcep
          constraints
            affinity
              [include-any|include-all|exclude-any]
                color <col_name>
                color <col_name>
    policy new_policy
      color 201 endpoint 2.2.2.0
      candidate-paths
        preference 200 
          dynamic
            pcep
          constraints
            affinity
              include-all
                color red

segment-routing
 traffic-eng
  on-demand color 99
   candidate-paths
     preference 100
       dynamic
         pcep
       constraints
         association-group
           disjoint
             type link
             id 1

Perform the following steps to configure ODN for SR-TE. The following figure is used as a reference to explain the configuration steps.

1. Configure all links with IS-IS point-to-point session from PE1 to PE2. Also, configure the domains as per the above topology.

2. Enable “distribute link-state” for IS-IS session on R1, R3, and R6.

   router isis 1
     net 31.0000.0000.0000.712a.00
     log-adjacency-changes
     distribute link-state
     address-family ipv4 unicast
       bfd
       segment-routing mpls
       maximum-paths 32
       advertise interface loopback0

3. Configure the router R1 (headend) and R6 (tailend) with a VRF interface.

   VRF configuration on R1:

   interface Ethernet1/49.101
   encapsulation dot1q 201
     vrf member sr
     ip address 101.10.1.1/24
     no shutdown
    
   vrf context sr
     rd auto
     address-family ipv4 unicast
       route-target import 101:101
       route-target import 101:101 evpn
       route-target export 101:101
       route-target export 101:101 evpn
   router bgp 6500
     vrf sr
       bestpath as-path multipath-relax
       address-family ipv4 unicast
         advertise l2vpn evpn

4. Tags VRF prefix with BGP community on R6 (tailend).

   route-map color1001 permit 10
     set extcommunity color 1001

5. Enable BGP on R6 (tailend) and R1 (headend) to advertise and receive VRF SR prefix and match on community set on R6 (tailend).

   R6 < EVPN > R3 < EVPN > R1

   BGP Configuration R6:

   router bgp 6500
     address-family ipv4 unicast
        allocate-label all
     neighbor 53.3.3.3
       remote-as 6500
       log-neighbor-changes
       update-source loopback0
       address-family l2vpn evpn
         send-community extended
        route-map Color1001 out
         encapsulation mpls
    

   BGP Configuration R1:

   router bgp 6500
     address-family ipv4 unicast
        allocate-label all
     neighbor 53.3.3.3
       remote-as 6500
       log-neighbor-changes
       update-source loopback0
       address-family l2vpn evpn
         send-community extended
          encapsulation mpls

6. Enable BGP configuration on R3 and BGP LS with XTC on R1, R3.abd

   BGP Configuration R3:

   router bgp 6500
     router-id 2.20.1.2
   address-family ipv4 unicast
   allocate-label all
   address-family l2vpn evpn
   retain route-target all
     neighbor 56.6.6.6
       remote-as 6500
       log-neighbor-changes
       update-source loopback0
       address-family l2vpn evpn
         send-community extended
          route-reflector-client
          route-map NH_UNCHANGED out
         encapsulation mpls
     neighbor 51.1.1.1
       remote-as 6500
       log-neighbor-changes
       update-source loopback0
       address-family l2vpn evpn
         send-community extended
         route-reflector-client
         route-map NH_UNCHANGED out
         encapsulation mpls
   neighbor 58.8.8.8
       remote-as 6500
       log-neighbor-changes
       update-source loopback0
       address-family link-state
    
   route-map NH_UNCHANGED permit 10
     set ip next-hop unchanged

   BGP Configuration R1:

   router bgp 6500
   neighbor 58.8.8.8
                 remote-as 6500
                  log-neighbor-changes
                  update-source loopback0
                  address-family link-state

   BGP Configuration R6: 


   outer bgp 6500
      neighbor 58.8.8.8
       remote-as 6500
          log-neighbor-changes
          update-source loopback0
          address-family link-state

7. Enable PCE and SR-TE tunnel configurations on R1.

   segment-routing
     traffic-engineering
       pcc
         source-address ipv4 51.1.1.1
         pce-address ipv4 58.8.8.8
       on-demand color 1001
         metric-type igp

The following guidelines and limitations apply to the SR-TE manual preference selection feature:

• Beginning with Cisco NX-OS Release 10.2(2)F, the SR-TE manual preference selection feature allows you to lockdown, shutdown, or perform both on an SRTE policy or an on-demand color template; shutdown preference(s) of an SR-TE policy or an on-demand color template. Furthermore, this feature also allows you to force a specific preference to be active for the SR-TE policy and force path re-optimization for all or a specific SR-TE policy.

  This feature is supported on Cisco Nexus 9300-EX, 9300-FX, 9300-FX2, 9300-GX, and N9K-C9332D-GX2B platform switches.

Beginning with Cisco NX-OS Release 10.2(2)F, you can perform the following actions as appropriate:

• Lockdown an SRTE policy – You can enable lockdown under on-demand color templates or explicit policies. Lockdown disables auto re-optimization of path preferences for a policy. In case a new higher preferred path comes up for a policy which is locked down, then it does not automatically switch to use the new path and continues to use the current active path option until it is valid.

  Note	If an explicit policy configuration exists for the same color as the on-demand template, then the policy configuration takes precedence over the template configuration for the lockdown.

  Example

  Consider a scenario where there are multiple preferences on a policy. Assume that the higher preference path goes down due to some fault in the network. The fault could be an impending failure of a node in the higher preference path. When investigating and rectifying the fault, the operations team may need to reload or disable the problematic node and prevent any disruptions while this occurs. Then, locking down the lower preference path and preventing switching back to the higher preference path is a good option to use.

• Shutdown an SRTE policy – You can enable shutdown under on-demand color templates or explicit policies. The policy state changes to admin down, and a policy down notification is sent to all the clients interested in the policy. Disabling shutdown under on-demand color configuration changes the policy state to up or down based on the path validity of the policy.

  Note	If an explicit policy configuration exists for the same color as the on-demand template, then the policy configuration takes precedence over the template configuration for the shutdown.

• Shutdown preference[s] of an SRTE policy – You can shut down a path preference under an on-demand color template configuration or under a path preference of explicit policy configuration. This disables that path preference and stops it from entering any future path re-optimization until the preference is unshut. The path preference is shown as admin down or up in the output of show srte policy based on whether it is shut or unshut in the configuration.

To force a specific preference to be the active path option for an SRTE policy, use the segment-routing traffic-engineering switch name <policy_name> pref <preference_number> execution command. This command uses the preference until it is valid.

A sample output is as follows:

NX2# show srte policy Green_White
Policy: 8.8.8.0|801
Name: Green_White
Source: 2.2.2.0
End-point: 8.8.8.0
State: UP
Color: 801
Authorized: Y
Binding-sid Label: 22
Policy-Id: 3
Path type = MPLS Active path option
Path-option Preference:180 ECMP path count: 1
1. PCE Weighted: No
Delegated PCE: 11.11.11.11
Index: 1 Label: 16005
Index: 2 Label: 16008
NX2# segment-routing traffic-engineering switch name Green_White preference 170
NX2(cfg-pref)# show srte policy Green_white detail
Policy: 8.8.8.0|801
Name: Green_White
…..
Path type = MPLS Path options count: 4
Path-option Preference:180 ECMP path count: 1 Admin: UP Forced: No
1. PCE Weighted: No
Delegated PCE: 11.11.11.11
Index: 1 Label: 16005
Index: 2 Label: 16008
Path-option Preference:170 ECMP path count: 1 Admin: UP Forced: Yes Active path option
1. Explicit Weighted: No
Name: Yellow
Index: 1 Label: 16006
Index: 2 Label: 16008

To undo this manually selected preference, you can perform any one of the following options:

• Use the segment-routing traffic-engineering reoptimize name <policy_name> command. For more information, see the Force path re-optimization for an SRTE Policy or All SRTE Policies section.

• Switch to another preference

• Shut this policy

• Shut the selected preference

When there are multiple preferences for an SRTE policy, you can re-optimize a policy, that is, pick the best preferred available path.

To force path re-optimization for a specific SRTE policy, use the segment-routing traffic-engineering reoptimize name <policy_name> command. The <policy_name> can be the name or alias name of the policy. This command undoes the preference switch command explained in the previous section and overrides lockdown if configured.

A sample output is as follows:

NX2# show srte policy Green_White
Policy: 8.8.8.0|801
Name: Green_White
Source: 2.2.2.0
End-point: 8.8.8.0
State: UP
Color: 801
Authorized: Y
Binding-sid Label: 22
Policy-Id: 3
Path type = MPLS Active path option
Path-option Preference:170 ECMP path count: 1
1. Explicit Weighted: Yes Weight: 1
Name: Yellow
Index: 1 Label: 16006
Index: 2 Label: 16008
NX2# segment-routing traffic-engineering reoptimize name Green_White
NX2# show srte policy Green_White
Policy: 8.8.8.0|801
Name: Green_White
Source: 2.2.2.0
End-point: 8.8.8.0
State: UP
Color: 801
Authorized: Y
Binding-sid Label: 22
Policy-Id: 3
Path type = MPLS Active path option
Path-option Preference:180 ECMP path count: 1
1. PCE Weighted: No
Delegated PCE: 11.11.11.11
Index: 1 Label: 16005
Index: 2 Label: 16008

To force path re-optimization for all SRTE policies, use the segment-routing traffic-engineering reoptimize all command to force path re-optimization for all SRTE policies present on the system. This command undoes the preference switch command explained in the previous point and overrides lockdown if configured.

The Flow-based Traffic Steering feature for Cisco NX-OS release 10.1(2) provides an alternate method of choosing the traffic to be steered, which is direct and flexible. This method allows configuring the source routing on the headend node directly, rather than on the egress node. Flow-based traffic steering allows the user to select which packets will be steered into an SRTE policy by matching fields in the incoming packet such as destination address, UDP or TCP port, DSCP bits and other properties. The matching is done by programming an ACL to steer packets into the policy.

To match and steer traffic, the Policy-Based Routing (PBR) feature is enhanced to support SRTE policies. The current PBR feature involves the RPM, ACL Manager, and AclQoS components. Beginning from Cisco NX-OS release 10.1(2), to add SRTE support, the RPM component also communicates with the SRTE and ULIB, and the communication with URIB is enhanced.

Thus, the flow-based traffic steering feature for SRTE includes the following:

• MPLS SR dataplane

• Steering IPv4 traffic is supported in default VRF, and steering IPv4 as well as IPv6 traffic is supported in non-default VRF

• Matching traffic by ACL based on a combination of the 5 tuple fields (source addr, destination addr, protocol, tcp/udp source port, tcp/udp destination port)

• Steering matched traffic into an SRTE policy

• Matching on the DSCP/TOS bits in the packet for IPv4 packets. Beginning with Cisco NX-OS Release 10.3(1)F, matching on the DSCP/TOS bits for the outer header for the VXLAN packets is also supported.

• Matching on the Traffic Class field of the packet for IPv6 packets

• Automatic enabling and disabling of ACLs based on time-period definitions

• When steering VRF cases, support steering into an SRTE policy without specifying a next hop

• Overlay ECMP using anycast endpoints

• Packets matched by ACL take precedence over regular routes

• Flow selection based on ToS/DSCP and timer-based ACL

• The next-hop-ip is used in steering traffic to SRTE policy from one endpoint to another

The following guidelines and limitations apply to the Flow-based Traffic Steering for SRTE feature:

• Beginning with Cisco NX-OS release 10.1(2), the flow-based traffic steering features for SRTE are supported on the Cisco Nexus 9000-FX, 9000-FX2, 9000-FX3, 9000-GX, and 9300 platform switches.

• When the SRTE policy is applied to a route-map assigned to an interface in a VRF (to steer L3VPN/L3EVPN traffic), if the next hop in the set statement resolves to a BGP prefix, and that BGP prefix is already using an SRTE policy to steer traffic, then the route-map does not steer traffic.

• Underlay ECMP is only supported if label stack is the same for each active SRTE path (ECMP member) in the policy. The 9000-GX platforms do not have this limitation.

• The route-map tracking feature is not supported.

• When steering into SRTE policies, having multiple set next-hop in a single route-map sequence entry is not supported.

• When the SRTE policy is applied to a route-map assigned to an interface in a VRF (to steer L3VPN/L3EVPN traffic), if the next hop in the set statement resolves to a BGP route (overlay route) that has multiple next hops in RIB, the traffic is only steered to the first next hop in the route and will not ECMP over all next hops.

• When the SRTE policy name is used in the route-map set statement, rather than color and endpoint, it can only be used for default VRF steering. If not, you must select an SRTE path that is defined explicitly. Specifically, this cannot be used to select SRTE policies defined to use a segment-list containing the policy-endpoint keyword in place of a label.

• The following keywords, which are applicable for the next hop-ip specified in the set ip next-hop <>, are not supported in the route-map when steering into SRTE policies:

  • verify-availability

  • drop-on-fail

  • force-order

  • load-share

• Route-map with srte-policy can be applied on the interface even if the required features (segmenting-routing, l3 evpn or l3vpn) are not enabled on the device. But the set-actions with srte-policy are kept down, that is, default-routing will be done for those flows.

• A route-map can have set commands with srte-policy and without srte-policy.

• For set-commands without srte-policy information, steering is done only if the reachability to the next-hop-ip does not require MPLS label.

• When a route-map is associated with an interface in a non-default VRF, and that route-map contains a sequence that specifies a next hop IP address N and an SRTE policy, then all other sequences on that route-map and all other route-maps associated with the same VRF that also use the same next hop IP address must also have an SRTE policy. Associating another route-map or route-map sequence using the same next hop IP and a different SRTE policy to the same VRF is not allowed.

• Similarly, when a route-map is associated with an interface in a non-default VRF, and that route-map does not specify an SRTE policy but specifies a next hop IP address N, then another sequence in that route-map or a separate route-map is not applied that uses the same next hop IP address N and specifies an SRTE policy.

• The SRTE flow-based traffic steering cannot be used at the same time as VXLAN or EoMPLS PBR.

• The SR label stats are not supported for policy based routed traffic on the SRTE ingress node. However, ACL redirect stats are supported.

• The IPv6 traffic in the default VRF cannot be steered into an SRTE policy. The MPLS SR underlay is only supported for IPv4. However, if an IPv6 SR underlay is required, use SRv6 instead.

• The 9000-FX, 9000-FX2, 9000-FX3, and 9300 platform hardware are unable to push unique underlay label stack per ECMP member, which impacts underlay ECMP on those platforms. In other words, if there are multiple active segment-lists on an SRTE policy (a single preference is configured with multiple segment-lists) where the first hop of the segment lists is different, then such a configuration is not supported. In such cases, as a workaround, configure anycast SID to make the label stack same across all ECMP members.

• Modular platforms are not supported in Cisco NX-OS release 10.1(2).

• Beginning with Cisco NX-OS release 10.2(2)F, the flow-based traffic steering features for SRTE are supported on the Cisco N9K-C9332D-GX2B platform switches.

• Beginning with Cisco NX-OS Release 10.3(1)F, the DSCP Based SR-TE Flow Steering feature allows source routing of VXLAN packets that are matched using the DSCP fields in the IP header and steered into an SRTE path. Following are the guidelines and limitations for this feature:

  • This feature is supported only on the Cisco Nexus 9300-FX2, 9300-FX3, 9300-GX, 9300-GX2 TOR switches.

  • In cases where the VXLAN packets are not terminated, the ACL filters are applied on the VXLAN Packet Outer IP Header fields (IPv4).

• Beginning with Cisco NX-OS release 10.3(2)F, the flow-based traffic steering features for SRTE are supported on Cisco Nexus 9700-FX and 9700-GX line cards. Following are the guidelines and limitations for this feature:

  • When Cisco Nexus 9508 platform switch is in a VXLAN EVPN to MPLS SR L3VPN hand-off mode and the MPLS encapsulated packets are forwarded on an L2 port, the dot1q header does not get added.

  • SVI/Sub-interfaces are not supported for core facing uplinks (MPLS or VXLAN) when Cisco Nexus 9500 platform switch is configured as EVPN to MPLS SR L3VPN hand off mode.

  • The DSCP to MPLS EXP promotion does not work on the FX TOR/line cards in DCI Mode. The copying of Inner DSCP value to MPLS EXP does not work on the FX TOR/line cards in this hand off mode. The MPLS EXP will be set to 0x7.

• Beginning with Cisco NX-OS Release 10.3(2)F, the DSCP based SR-TE flow steering feature is supported on Cisco Nexus 9300-FX platform switches and Cisco Nexus 9700-FX and 9700-GX line cards. Following are the guidelines and limitations for this feature:

  • When Cisco Nexus 9500 platform switch is in a VXLAN EVPN to MPLS SR L3VPN hand-off mode and the MPLS encapsulated packets are forwarded on an L2 port, the dot1q header does not get added.

  • SVI/Sub-interfaces are not supported for core facing uplinks (MPLS or VXLAN) when Cisco Nexus 9500 platform switch is configured as EVPN to MPLS SR L3VPN hand off mode.

  • The DSCP to MPLS EXP promotion does not work on the FX TOR/line cards in DCI Mode. The copying of Inner DSCP value to MPLS EXP does not work on the FX TOR/line cards in this hand off mode. The MPLS EXP will be set to 0x7.

The configuration process for the SRTE Flow-based Traffic Steering feature is as follows:

1. Configure the IP access lists, especially matching the criteria on the IP access list.

   For more information, see the Configuring IP ACLs chapter in the Cisco Nexus Series NX-OS Security Configuration Guide.

2. Define the SRTE policy.

   For more information about configuring SRTE, see the Configuring Segment Routing for Traffic Engineering chapter in the Cisco Nexus 9000 series NX-OS Label Switching Configuration Guide.

3. Configure the route map that binds the match (IP access list configured in step 1) and action. The match refers to the fields to match on the packet, and the action refers to what SRTE policy to steer into and the VPN label to use, if any.

This section includes the following examples for configuring flow-based traffic steering for SRTE:

Beginning with Cisco NX-OS release 10.1(2), MPLS OAM monitoring allows the switch on which one or more SRTE policies are configured to proactively detect if the active path or paths of an SRTE policy have failed. If the paths in the currently active preference have all failed, SRTE will consider that preference down and so make the next highest preference on the policy active, if there is such a preference, or otherwise mark the policy as down.

Before this feature, the state of an SRTE preference and policy was only determined by the state of the first hop (the first MPLS label) of the paths in the preference. If the label was programmed the path was considered up, and if the label was missing or invalid the path was considered down.

The MPLS OAM monitoring augments this validation by sending MPLS LSPV Nil-FEC ping requests continuously along the SRTE path. Each ping request contains the same label stack as would be imposed on traffic that follows the SRTE policy, making the pings take the same path. The pings are sent with a configurable interval between each ping, and a response to the ping from the final node of the path is expected within the interval. If a failure response is returned from the final node or no response is received within the interval, it is counted as a failed interval. After a configurable number of failed intervals occur in sequence, the path is considered down. If all paths in a preference are down, then the preference is considered down.

The MPLS OAM monitoring for SRTE policies has the following guidelines and limitations:

• Beginning with Cisco NX-OS Release 10.1(2), MPLS OAM monitoring (continuous and proactive path) is introduced and supported on Cisco Nexus 9300 EX, 9300-FX, 9300-FX2, and 9300-GX platform switches.

• On the head-end node where the SRTE policies are configured, both SRTE and MPLS OAM must be separately enabled as part of feature mpls segment-routing traffic-engineering and feature mpls oam respectively. If not, the user cannot configure the monitoring of SRTE policies using OAM. In addition, the remaining nodes in the SR fabric must have MPLS OAM enabled using feature mpls oam to respond to the pings sent by MPLS OAM monitoring.

• SRTE limits the maximum number of monitoring sessions to 1000.

• The minimum interval between pings is 1000 milliseconds.

• When SRTE OAM monitoring policies are running on a device, feature mpls oam cannot be disabled. Only when all the SRTE OAM monitoring policies are disabled, the feature mpls oam can be disabled from the device. Otherwise, the following error message is displayed:

  "SRTE MPLS liveness detection is either enabled for all policies, is enabled for at least one policy, or is enabled for an on-demand color. Please ensure liveness detection is completely disabled before disabling MPLS OAM."

• In Cisco NX-OS Release 10.1(2) SRTE OAM monitoring is supported for static policies and on-demand color having explicit path configured.

• The OAM sessions do not run for paths that are configured with dynamic option using PCEP.

This section describes the CLIs required to enable proactive path monitoring for policies.

• Global Configuration

  This configuration enables OAM path monitoring for all configured policies.

• Policy-specific Configuration

  This configuration enables OAM path monitoring for a specific policy.

The following example shows how to configure MPLS OAM monitoring:

• Configuration example for global enablement with user specified multiplier and interval:

  segment-routing
    traffic-engineering
      liveness-detection
          interval 6000
          multiplier 5
        mpls
          oam
      segment-list name blue
        index 10 mpls label 16004
        index 20 mpls label 16005
      segment-list name green
        index 10 mpls label 16003
        index 20 mpls label 16006
      segment-list name red
        index 10 mpls label 16002
        index 20 mpls label 16004
        index 30 mpls label 16005
      policy customer-1
        color 1 endpoint 5.5.5.5
        candidate-paths
          preference 100
            explicit segment-list red
      on-demand color 211
        candidate-paths
          preference 100
            explicit segment-list green
  

• Configuration example for policy enablement with user specified multiplier, interval, index-limit and shutdown option:

  segment-routing
    traffic-engineering
      liveness-detection
          interval 6000
          multiplier 5
      segment-list name blue
        index 10 mpls label 16004
        index 20 mpls label 16005
      segment-list name green
        index 10 mpls label 16003
        index 20 mpls label 16006
      segment-list name red
        index 10 mpls label 16002
        index 20 mpls label 16004
        index 30 mpls label 16005
      policy customer-1
        color 1 endpoint 5.5.5.5
        candidate-paths
          preference 100
            explicit segment-list red
        liveness-detection
          index-limit 20
          shutdown
          mpls
            oam
      on-demand color 211
        candidate-paths
          preference 100
            explicit segment-list green
        liveness-detection
            index-limit 20
            shutdown
            mpls
              oam
  

BFD for SRTE is similar to MPLS OAM Monitoring for SRTE Policies. BFD for SRTE allows the switch on which one or more SRTE policies are configured to proactively detect if the active path or paths of an SRTE policy have failed. If the paths in the currently active preference have all failed, SRTE considers that preference down and so make the next highest preference of the policy active, if there is such a preference, or otherwise mark the policy as down.

BFD for SRTE performs the detection by sending BFD probes continuously along the SRTE path. Each probe is encapsulated in MPLS with the same label stack as would be imposed on traffic that follows the SRTE policy, making the probes take the same path. In addition, one more label is imposed innermost in the label stack of the probe that causes the probe to be returned to the sender by the data plane of the final node of the policy when it is reached. This differs from MPLS OAM Monitoring for SRTE Policies in which the probe is received by the final node, processed in the control plane, and a response sent back.

The probes are sent with a configurable interval between each probe, and a probe is expected to loop back to the sender within the interval. After a configurable number of failed intervals occur in sequence, the path is considered down. If all paths in a preference are down, then the preference is considered down.

The guidelines and limitations for configuring BFD monitoring for SRTE policies are as follows:

• Beginning with Cisco NX-OS Release 10.3(2)F, BFD Monitoring for SRTE Policies is introduced and supported on 9300-FX, 9300-FX2, 9300-FX3, 9300-GX, 9300-GX2, N9K-C9364C, and N9K-C9332C TOR platforms only.

• Only SRTE MPLS with IPv4 underlay will be supported for monitoring using BFD. SRv6 policies are not supported.

• vPC is not supported on the headend when using this form of monitoring.

• Only one of OAM or BFD monitoring may be enabled at one time. That is, it is not possible to have some policies monitored using OAM and some using BFD.

• IP redirects must be disabled on disabled on SR enabled core interfaces of the node where the BFD probe loops back to the sender since it may need to exit the same interface it just arrived on.

• The innermost label that SRTE uses for the monitoring path (the headend label) must not be an anycast SID, it must be the unique SID for that node so that responses are not directed to another node sharing the same anycast address.

• The total number of ECMP members for a given policy when programmed to forwarding is 8, which includes the primary and backup ECMP members. If there are more than 8 ECMP members between the primary and backup preferences on the policy combined, only 8 will be used.

• The SRGB range defined on the SRTE headend node (where the policy is defined) and the SRGB range defined on the final node of all paths monitored by BFD liveness detection must be the same, and it is recommended that the SRGB range be the same on all nodes. The return-to-sender label added to the BFD probe packets is learned locally on the SRTE headend node from the connected-prefix-sid-map SR configuration for the prefix of the local loopback interface, so the value of that label must be the same on the node returning the packet.

• BFD monitoring is not supported for path preference with dynamic pcep option.

This section describes the commands required to enable proactive path monitoring for policies using BFD protection for SRTE policies. The configuration tasks can be performed in following ways based on whether you want to configure for all policies or a specific policy:

• Global Configuration - This configuration enables BFD protection for all configured policies.

• Policy-specific Configuration - This configuration enables BFD protection for a specific policy.

The following example shows how to configure BFD for SRTE:

feature mpls segment-routing traffic-engineering segment-routing
traffic-engineering
liveness-detection
      multiplier NUM 
      interval NUM 
      mpls 
        bfd
    segment-list name SEGLIST1      
      index 100 mpls label 16001 
      index 200 mpls label 16002
      index 300 mpls label 16003
    on-demand color 702
      explicit segment-list SEGLIST1
      liveness-detection
        mpls
          bfd
        index-limit 200
    policy name POL1
      color 20 endpoint 1.1.1.1
      liveness-detection
        mpls
          bfd
        index-limit 200

In order to support segment routing, BGP requires the ability to advertise a segment identifier (SID) for a BGP prefix. A BGP prefix SID is always global within the segment routing BGP domain and identifies an instruction to forward the packet over the ECMP-aware best path computed by BGP to the related prefix. The BGP prefix SID identifies the BGP prefix segment.

The adjacency segment Identifier (SID) is a local label that points to a specific interface and a next hop out of that interface. No specific configuration is required to enable adjacency SIDs. Once segment routing is enabled over BGP for an address family, for any interface that BGP runs over, the address family automatically allocates an adjacency SID toward every neighbor out of that interface.

In-service software upgrades (ISSUs) are minimally supported with BGP graceful restart. All states (including the segment routing state) must be relearned from the BGP router's peers. During the graceful restart period, the previously learned route and label state are retained.

Cisco Nexus 9000 Series switches are often deployed in massive scale data centers (MSDCs). In such environments, there is a requirement to support BGP Egress Peer Engineering (EPE) with Segment Routing (SR).

Segment Routing (SR) leverages source routing. A node steers a packet through a controlled set of instructions, known as segments, by prepending the packet with an SR header. A segment can represent any topological or service-based instruction. SR allows steering a flow through any topological path or any service chain while maintaining per-flow state only at the ingress node of the SR domain. For this feature, the Segment Routing architecture is applied directly to the MPLS data plane.

In order to support Segment Routing, BGP requires the ability to advertise a Segment Identifier (SID) for a BGP prefix. A BGP prefix is always global within the SR or BGP domain and it identifies an instruction to forward the packet over the ECMP-aware best-path that is computed by BGP to the related prefix. The BGP prefix is the identifier of the BGP prefix segment.

The SR-based Egress Peer Engineering (EPE) solution allows a centralized (SDN) controller to program any egress peer policy at ingress border routers or at hosts within the domain.

In the following example, all three routers run iBGP and they advertise NRLI to one another. The routers also advertise their loopback as the next-hop and it is recursively resolved. This provides an ECMP between the routers as displayed in the illustration.

The SDN controller receives the Segment IDs from the egress router 1.1.1.1 for each of its peers and adjacencies. It can then intelligently advertise the exit points to the other routers and the hosts within the controller’s routing domain. As displayed in the illustration, the BGP Network Layer Reachability Information (NLRI) contains both the Node-SID to Router 1.1.1.1 and the Peer-Adjacency-SID 24003 indicating that the traffic to 7.7.7.7 should egress over the link 12.1.1.1->12.1.1.3.

BGP Egress Peer Engineering has the following guidelines and limitations:

• BGP Egress Peer Engineering is only supported for IPv4 BGP peers. IPv6 BGP peers are not supported.

• BGP Egress Peer Engineering is only supported in the default VPN Routing and Forwarding (VRF) instance.

• Any number of Egress Peer Engineering (EPE) peers may be added to an EPE peer set. However, the installed resilient per-CE FEC is limited to 32 peers.

• A given BGP neighbor can only be a member of a single peer-set. Peer-sets are configured. Multiple peer-sets are not supported. An optional peer-set name may be specified to add neighbor to a peer-set. The corresponding RPC FEC load-balances the traffic across all the peers in the peer-set. The peer-set name is a string that is a maximum length of 63 characters (64 NULL terminated). This length is consistent with the NX-OS policy name lengths. A peer can only be a member of a single peer-set.

• Adjacencies for a given peer are not separately assignable to different peer-sets.

• Beginning with Cisco NX-OS Release 9.3(3), BGP Egress Peer Engineering is supported on Cisco Nexus 9300-GX platform switches.

See the Egress Peer Engineering sample configuration for the BGP speaker 1.1.1.1. Note that the neighbor 20.20.20.20 is the SDN controller.

 hostname epe-as-1
install feature-set mpls
feature-set mpls

feature telnet
feature bash-shell
feature scp-server
feature bgp
feature mpls segment-routing

segment-routing mpls
vlan 1

vrf context management
  ip route 0.0.0.0/0 10.30.97.1
  ip route 0.0.0.0/0 10.30.108.1

interface Ethernet1/1
  no switchport
  ip address 10.1.1.1/24
  no shutdown

interface Ethernet1/2
  no switchport
  ip address 11.1.1.1/24
  no shutdown

interface Ethernet1/3
  no switchport
  ip address 12.1.1.1/24
  no shutdown

interface Ethernet1/4
  no switchport
  ip address 13.1.1.1/24
  no shutdown

interface Ethernet1/5
  no switchport
  ip address 14.1.1.1/24
  no shutdown

interface mgmt0
  ip address dhcp
  vrf member management


interface loopback1
  ip address 1.1.1.1/32
line console

line vty
ip route 2.2.2.2/32 10.1.1.2
ip route 3.3.3.3/32 11.1.1.3
ip route 3.3.3.3/32  12.1.1.3
ip route 4.4.4.4/32  13.1.1.4
ip route 20.20.20.20/32 14.1.1.20 

router bgp 1
  address-family ipv4 unicast
  address-family link-state
 neighbor 10.1.1.2
    remote-as 2
    address-family ipv4
    egress-engineering
 neighbor 3.3.3.3
   remote-as 3
   address-family ipv4
   update-source loopback1
   ebgp-multihop 2
   egress-engineering
 neighbor 4.4.4.4
   remote-as 4
   address-family ipv4
   update-source loopback1
   ebgp-multihop 2
   egress-engineering
neighbor 20.20.20.20
   remote-as 1
   address-family link-state
   update-source loopback1
   ebgp-multihop 2
neighbor 124.11.50.5
    bfs
    remote-as 6
    update-source port-channel50.11
    egress-engineering peer-set pset2 <<<<<<<
    address-family ipv4 unicast
neighbor 124.11.101.2
    bfd
    remote-as 6
    update-source Vlan2401
    egress-engineering  
    address-family ipv4 unicast

This example shows sample output for the show bgp internal epe command.

switch# show bgp internal epe 
BGP Egress Peer Engineering (EPE) Information:
Link-State Server: Inactive
Link-State Client: Active
Configured EPE Peers: 26
Active EPE Peers: 3
EPE SID State:
RPC SID Peer or Set Assigned
ID Type Set Name ID Label Adj-Info, iod
1 Node 124.1.50.5 1 1600 
2 Set pset1 2 1601 
3 Node 6.6.6.6 3 1602 
4 Node 124.11.50.5 4 1603 
5 Set pset2 5 1604 
6 Adj 6.6.6.6 6 1605 124.11.50.4->124.11.50.5/0x1600b031, 80
7 Adj 6.6.6.6 7 1606 124.1.50.4->124.1.50.5/0x16000031, 78
EPE Peer-Sets:
IPv4 Peer-Set: pset1, RPC-Set 2, Count 7, SID 1601
Peers: 124.11.116.2 124.11.111.2 124.11.106.2 124.11.101.2 
124.11.49.5 124.1.50.5 124.1.49.5 
IPv4 Peer-Set: pset2, RPC-Set 5, Count 5, SID 1604
Peers: 124.11.117.2 124.11.112.2 124.11.107.2 124.11.102.2 
124.11.50.5 
IPv4 Peer-Set: pset3, RPC-Set 0, Count 4, SID unspecified
Peers: 124.11.118.2 124.11.113.2 124.11.108.2 124.11.103.2 
IPv4 Peer-Set: pset4, RPC-Set 0, Count 4, SID unspecified
Peers: 124.11.119.2 124.11.114.2 124.11.109.2 124.11.104.2 
IPv4 Peer-Set: pset5, RPC-Set 0, Count 4, SID unspecified
Peers: 124.11.120.2 124.11.115.2 124.11.110.2 124.11.105.2 
switch# 

In the simple example below, all three routers are running iBGP and advertising Network Layer Reachability Information (NRLI) to one another. The routers are also advertising their loopback interface as the next hop, which provides the ECMP between routers 2.2.2.2 and 3.3.3.3.

Ethernet VPN (EVPN) is a next generation solution that provides ethernet multipoint services over MPLS networks. EVPN operates in contrast to the existing Virtual Private LAN Service (VPLS) by enabling control-plane based MAC learning in the core. In EVPN, PEs participating in the EVPN instances learn customer MAC routes in control-plane using MP-BGP protocol. Control-plane MAC learning brings several benefits that allow EVPN to address the VPLS shortcomings, including support for multihoming with per-flow load balancing.

In a data center network, the EVPN control plane provides:

• Flexible workload placement that is not restricted with the physical topology of the data center network. Therefore, you can place virtual machines (VM) anywhere within the data center fabric.

• Optimal East-West traffic between servers within and across data centers. East-West traffic between servers, or virtual machines, is achieved by most specific routing at the first hop router. First hop routing is done at the access layer. Host routes must be exchanged to ensure most specific routing to and from servers or hosts. VM mobility is supported by detecting new endpoint attachment when a new MAC address or the IP address is directly connected to the local switch. When the local switch sees the new MAC or the IP address, it signals the new location to rest of the network.

• Segmentation of Layer 2 and Layer 3 traffic, where traffic segmentation is achieved using MPLS encapsulation and the labels (per-BD label and per-VRF labels) act as the segment identifier.

Layer 2 EVPN over segment routing MPLS has the following guidelines and limitations:

• Segment routing Layer 2 EVPN flooding is based on the ingress replication mechanism. MPLS core does not support multicast.

• ARP suppression is not supported.

• Consistency checking on vPC is not supported.

• The same Layer 2 EVI and Layer 3 EVI cannot be configured together.

• Beginning with Cisco NX-OS Release 9.3(1), Layer 2 EVPN is supported on Cisco Nexus 9300-FX2 platform switches.

• Beginning with Cisco NX-OS Release 9.3(5), Layer 2 EVPN over segment routing MPLS is supported on Cisco Nexus 9300-GX and Cisco Nexus 9300-FX3 platform switches.