Troubleshoot SDA Forwarding East-West Traffic Flow
Available Languages
Contents
Introduction
This document describes how to validate East-West Traffic Flow as part of Software Defined Access (SDA).
Prerequisites
Requirements
Cisco recommends that you have knowledge of these topics:
- Internet Protocol (IP) Forwarding
- Locator/ID Separation Protocol (LISP)
Components Used
The information in this document is based on these software and hardware versions:
- C9000v on Cisco IOS® XE 17.10.1
- SDA 1.0 (not LISP PubSub)
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.
Related Products
This document can also be used with these hardware and software versions:
- C9200
- C9300
- C9400
- C9500
- C9600
- Cisco IOS® XE 16.12 and later
Background Information
SDA East-West Traffic Flow refers to the concept where an endpoint within the SDA Fabric wants to communicate with another endpoint within the same fabric. There are caveats as to what is and is not considered an East-West flow. An East-West traffic flow can be these examples:
- Endpoints that are in the same subnet (172.17.10.2 talking to 172.19.10.3) this is considered L2LISP Extension
- Endpoints that are in the same VRF (VN) (172.19.10.2 talking to 172.19.11.2 and both are in VRF Campus) this is considered L3 LISP
- Endpoint that is within the fabric talking to a host that is connected to an L2 Handoff Border, exactly the same as L2LISP
East-West traffic flows do not refer to these examples:
- Traffic originated from the SDA Fabric to outside the fabric, that is North-South
- Inter-VRF routing is also not considered East-West (An endpoint in VRF Campus, IP address 172.19.10.2 talking to an endpoint in VRF Guest, IP address 172.19.11.2)
- SD-WAN Integrated Domains
- SDA Transit
- Border Affinity
- Extranet
Topology
For the purpose of this example, C9000v switches function as the Fabric Edges and Collocated Borders. All endpoints are within the same Virtual Network (VN), red_vn. Endpoints at 10.47.4.2 and 10.47.4.2 are in the same subnet, the endpoint at 10.47.10.2 is in a different subnet but same VN.
Configuration
It is assumed that CIsco DNA-Center is used to provision SDA fabric with the default settings:
- Layer-2 extension is enabled (this forces traffic to be forwarded based on MAC address lookups rather than IP address lookups).
- Layer-2 flooding is disabled (this enables ARP supression on edge devices and LISP-assisted ARP learning).
After proper host onboarding process, the interface configuration contains several sections:
Fabric Edge (10.47.1.12) Interface Configuration:
interface GigabitEthernet1/0/3 switchport access vlan 1026 switchport mode access device-tracking attach-policy IPDT_POLICY spanning-tree portfast spanning-tree bpduguard enable end interface Vlan1026 description Configured from Cisco DNA-Center mac-address 0000.0c9f.f341 vrf forwarding red_vn ip address 10.47.4.1 255.255.255.0 ip helper-address 10.47.9.9 no ip redirects ip route-cache same-interface no lisp mobility liveness test lisp mobility red-IPV4 end
Fabric Edge (10.47.1.12) LISP Configuration:
router lisp locator-table default locator-set rloc_222e1707-175d-4019-a783-060404f8bc2f IPv4-interface Loopback0 priority 10 weight 10 exit-locator-set
!
instance-id 4099 remote-rloc-probe on-route-change dynamic-eid red-IPV4 database-mapping 10.47.4.0/24 locator-set rloc_222e1707-175d-4019-a783-060404f8bc2f exit-dynamic-eid ! dynamic-eid red-helpdesk-IPV4 database-mapping 10.47.10.0/24 locator-set rloc_222e1707-175d-4019-a783-060404f8bc2f exit-dynamic-eid ! service ipv4 eid-table vrf red_vn map-cache 0.0.0.0/0 map-request sgt distribution sgt exit-service-ipv4 ! exit-instance-id
!
! instance-id 8190 remote-rloc-probe on-route-change service ethernet eid-table vlan 1026 database-mapping mac locator-set rloc_222e1707-175d-4019-a783-060404f8bc2f dynamic-eid detection multiple-addr bridged-vm exit-service-ethernet ! exit-instance-id ! instance-id 8192 remote-rloc-probe on-route-change service ethernet eid-table vlan 1028 database-mapping mac locator-set rloc_222e1707-175d-4019-a783-060404f8bc2f dynamic-eid detection multiple-addr bridged-vm exit-service-ethernet ! exit-instance-id
Fabric Edge (10.47.1.13) Interface Configuration:
interface GigabitEthernet1/0/3 switchport access vlan 1026 switchport mode access device-tracking attach-policy IPDT_POLICY spanning-tree portfast spanning-tree bpduguard enable end ! interface GigabitEthernet1/0/5 switchport access vlan 1028 switchport mode access device-tracking attach-policy IPDT_POLICY spanning-tree portfast spanning-tree bpduguard enable end ! interface Vlan1026 description Configured from Cisco DNA-Center mac-address 0000.0c9f.f341 vrf forwarding red_vn ip address 10.47.4.1 255.255.255.0 ip helper-address 10.47.9.9 no ip redirects ip route-cache same-interface no lisp mobility liveness test lisp mobility red-IPV4 end ! interface Vlan1028 description Configured from Cisco DNA-Center mac-address 0000.0c9f.f800 vrf forwarding red_vn ip address 10.47.10.1 255.255.255.0 ip helper-address 10.47.9.9 no ip redirects ip route-cache same-interface no lisp mobility liveness test lisp mobility red-helpdesk-IPV4 end
Fabric Edge (10.47.1.13) LISP Configuration
router lisp locator-table default locator-set rloc_691b1fe4-5264-44c2-bb1b-0903b3eb2c51 IPv4-interface Loopback0 priority 10 weight 10 exit-locator-set ! instance-id 4099 remote-rloc-probe on-route-change dynamic-eid red-IPV4 database-mapping 10.47.4.0/24 locator-set rloc_691b1fe4-5264-44c2-bb1b-0903b3eb2c51 exit-dynamic-eid ! dynamic-eid red-helpdesk-IPV4 database-mapping 10.47.10.0/24 locator-set rloc_691b1fe4-5264-44c2-bb1b-0903b3eb2c51 exit-dynamic-eid ! service ipv4 eid-table vrf red_vn map-cache 0.0.0.0/0 map-request sgt distribution sgt exit-service-ipv4 ! exit-instance-id ! instance-id 8190 remote-rloc-probe on-route-change service ethernet eid-table vlan 1026 database-mapping mac locator-set rloc_691b1fe4-5264-44c2-bb1b-0903b3eb2c51 dynamic-eid detection multiple-addr bridged-vm exit-service-ethernet ! exit-instance-id ! instance-id 8192 remote-rloc-probe on-route-change service ethernet eid-table vlan 1028 database-mapping mac locator-set rloc_691b1fe4-5264-44c2-bb1b-0903b3eb2c51 dynamic-eid detection multiple-addr bridged-vm exit-service-ethernet ! exit-instance-id
Host Onboarding Verification
As part of host on-boarding process, several structures are created:
IPDT / IP Device Tracking Entry
After successful host onboarding, there is a valid entry in the IP Device Tracking (IPDT) table and the end-host is be marked as REACHABLE:
Edge-1#show device-tracking database interface g1/0/3
portDB has 2 entries for interface Gi1/0/3, 2 dynamic
Codes: L - Local, S - Static, ND - Neighbor Discovery, ARP - Address Resolution Protocol, DH4 - IPv4 DHCP, DH6 - IPv6 DHCP, PKT - Other Packet, API - API created
Preflevel flags (prlvl):
0001:MAC and LLA match 0002:Orig trunk 0004:Orig access
0008:Orig trusted trunk 0010:Orig trusted access 0020:DHCP assigned
0040:Cga authenticated 0080:Cert authenticated 0100:Statically assigned
Network Layer Address Link Layer Address Interface vlan prlvl age state Time left
DH4 10.47.4.2 5254.0019.93e9 Gi1/0/3 1026 0024 3mn REACHABLE 28 s try 0(15198 s)MAC / ARP Entry
When the end-host is successfully onboarded, it can ping the default gateway (or can be pinged from the default gateway if no firewall is installed on the end-point blocking this communication):
Edge-1#ping vrf red_vn 10.47.4.2 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.47.4.2, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 142/150/161 ms
On the Edge Node, there is a MAC address, as well as corresponding ARP entry in the table (in VRF):
Edge-1#show mac address-table interface g1/0/3
Mac Address Table
-------------------------------------------
Vlan Mac Address Type Ports
---- ----------- -------- -----
1026 5254.0019.93e9 DYNAMIC Gi1/0/3
Total Mac Addresses for this criterion: 1
Edge-1#show ip arp vrf red_vn
Protocol Address Age (min) Hardware Addr Type Interface
Internet 10.47.4.1 - 0000.0c9f.f341 ARPA Vlan1026
Internet 10.47.4.2 1 5254.0019.93e9 ARPA Vlan1026
Internet 10.47.10.1 - 0000.0c9f.f800 ARPA Vlan1028Software FED MAC Address Programming**
To check the MAC address in FED use the command show platform software fed switch active matm macTable vlan <vlan id> mac <mac address>
Edge-1#show platform software fed switch active matm macTable vlan 1026 mac 5254.0019.93e9 VLAN MAC Type Seq# EC_Bi Flags machandle siHandle riHandle diHandle *a_time *e_time ports Con ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 1026 5254.0019.93e9 0x1 9 0 0 0x7f65ec7bda68 0x7f65ec7c21f8 0x0 0x7f65ec6e1368 300 7 GigabitEthernet1/0/3 Yes ======platform hardware details ====== Asic: 0 htm-handle = 0x7f65ec95dc68 MVID = 7 gpn = 1 SI = 0xc3 RI = 0x25 DI = 0x526e DI = 0x526e pmap = 0x00000000 0x00000004 pmap_intf : [GigabitEthernet1/0/3] Asic: 1 SI = 0xc3 RI = 0x25 DI = 0x526e DI = 0x526e pmap = 0x00000000 0x00000000
**MAC Address macHandle Programming**
Take the macHandle value from the previous command (0x7f65ec7bda68) and use in show platform hardware fed switch active fwd-asic abstraction print-resource-handle <macHandle> 1
Edge-1#show platform hardware fed switch active fwd-asic abstraction print-resource-handle 0x7f65ec7bda68 1
Handle:0x7f65ec7bda68 Res-Type:ASIC_RSC_HASH_TCAM Res-Switch-Num:0 Asic-Num:255 Feature-ID:AL_FID_L2 Lkp-ftr-id:LKP_FEAT_L2_SRC_MAC_VLAN ref_count:1
priv_ri/priv_si Handle: (nil)Hardware Indices/Handles: handle [ASIC: 0]: 0x7f65ec95dc68
Features sharing this resource:Cookie length: 12
19 00 54 52 e9 93 07 80 07 00 00 00
Detailed Resource Information (ASIC_INSTANCE# 0)
----------------------------------------
Number of HTM Entries: 1
Entry 0: (handle 0x7f65ec95dc68)
Absolute Index: 6778
Time Stamp: 4
KEY - vlan:7 mac:0x5254001993e9 l3_if:0 gpn:3 epoch:0 static:0 flood_en:0 vlan_lead_wless_flood_en: 0 client_home_asic: 0 learning_peerid 0, learning_peerid_valid 0 lvx:1
MASK - vlan:0 mac:0x0 l3_if:0 gpn:0 epoch:0 static:0 flood_en:0 vlan_lead_wless_flood_en: 0 client_home_asic: 0 learning_peerid 0, learning_peerid_valid 0 lvx:0
SRC_AD - need_to_learn:0 lrn_v:0 catchall:0 static_mac:0 chain_ptr_v:0 chain_ptr: 0 static_entry_v:0 auth_state:0 auth_mode:0 auth_behavior_tag:0 traf_m:0 is_src_ce:0
DST_AD - si:0xb7 bridge:0 replicate:0 blk_fwd_o:0 v4_rmac:0 v6_rmac:0 catchall:0 ign_src_lrn:0 port_mask_o:0 afd_cli_f:0 afd_lbl:0 prio:3 dest_mod_idx:0 destined_to_us:0 pv_trunk:0 smr:0
==============================================================
**MVID Verification**
The number 7 in the previous output is the Mapped VLAN ID (MVID) in hardware. To verify that they match up to the "real" vlan, use show platform software fed switch active vlan <vlan number>
Edge-1#show platform software fed switch active vlan 1026 VLAN Fed Information Vlan Id IF Id LE Handle STP Handle L3 IF Handle SVI IF ID MVID ----------------------------------------------------------------------------------------------------------------------- 1026 0x0000000000420011 0x00007f65ec6a08b8 0x00007f65ec6a1138 0x00007f65ec77e838 0x000000000000001d 7
**Global Port Number (GPN) Verification**
To correlate the GPN with a "real" interface, use the command show platform software fed switch active ifm mappings gpn
Edge-1#show platform software fed switch active ifm mappings gpn
Mappings Table
GPN Interface IF_ID IF_TYPE
--------------------------------------------------------------------------------------------------
1 GigabitEthernet1/0/1 0x0000001a ETHER
2 GigabitEthernet1/0/2 0x0000001b ETHER
3 GigabitEthernet1/0/3 0x0000000b ETHER <-- GPN 3 lines up with the expected Egress interface**MAC Address siHandle Programming**
Take the siHandle value from the previous command (0x7f65ec7c21f8) and utilize in show platform hardware fed switch active fwd-asic abstraction print-resource-handle <si_handle> 1
Edge-1#show platform hardware fed switch active fwd-asic abstraction print-resource-handle 0x7f65ec7c21f8 1
Handle:0x7f65ec7c21f8 Res-Type:ASIC_RSC_SI Res-Switch-Num:255 Asic-Num:255 Feature-ID:AL_FID_L3_UNICAST_IPV4 Lkp-ftr-id:LKP_FEAT_INVALID ref_count:2
priv_ri/priv_si Handle: 0x7f65ec7c2498Hardware Indices/Handles: index0:0xc3 mtu_index/l3u_ri_index0:0x0 index1:0xc3 mtu_index/l3u_ri_index1:0x0
Features sharing this resource:66 (1)]
57 (1)]
Cookie length: 56
00 00 00 00 00 00 00 00 02 04 00 00 00 00 00 00 00 00 00 00 07 00 52 54 00 19 93 e9 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Detailed Resource Information (ASIC_INSTANCE# 0)
----------------------------------------
Station Index (SI) [0xc3] <-- Station Index is comprised of the Rewrite Index (RI) and Destination Index (DI)
RI = 0x25 <-- Rewrite Index contains the forwarding information
DI = 0x526e <-- Destination Index contains information related to the outgoing interface
stationTableGenericLabel = 0
stationFdConstructionLabel = 0x7
lookupSkipIdIndex = 0
rcpServiceId = 0
dejaVuPreCheckEn = 0x1
Replication Bitmap: LD <-- Local Data (LD) indicates that the destination is on this ASIC
Detailed Resource Information (ASIC_INSTANCE# 1)
----------------------------------------
Station Index (SI) [0xc3] <-- Station Index is comprised of the Rewrite Index (RI) and Destination Index (DI)
RI = 0x25 <-- Rewrite Index contains the forwarding information
DI = 0x526e <-- Destination Index contains information related to the outgoing interface
stationTableGenericLabel = 0
stationFdConstructionLabel = 0x7
lookupSkipIdIndex = 0
rcpServiceId = 0
dejaVuPreCheckEn = 0x1
Replication Bitmap: CD <-- Core Data (CD) indicates that the destination is on the same ASIC, different core
==============================================================
**MAC Address Rewrite-Index Verification**
Take the RI value from the previous command (0x25) and utilize in show platform hardware fed switch active fwd-asic resource asic all rewrite-index range <RI> <RI>
Edge-1#show platform hardware fed switch active fwd-asic resource asic all rewrite-index range 0x25 0x25
ASIC#:0 RI:37 Rewrite_type:AL_RRM_REWRITE_L3_UNICAST_IPV4_SHARED(1) Mapped_rii:L3_UNICAST_IPV4(9)
MAC Addr: MAC Addr: 52:54:00:19:93:e9,
L3IF LE Index 41
ASIC#:0 RI:38 Rewrite_type:AL_RRM_REWRITE_L3_UNICAST_IPV4_SHARED(1) Mapped_rii:L3_UNICAST_IPV4(9)
MAC Addr: MAC Addr: 01:00:5e:00:00:00,
L3IF LE Index 40
ASIC#:0 RI:39 Rewrite_type:AL_RRM_REWRITE_L3_UNICAST_IPV4_SHARED(1) Mapped_rii:L3_UNICAST_IPV4(9)
MAC Addr: MAC Addr: 52:54:00:00:50:17,
L3IF LE Index 40
ASIC#:1 RI:37 Rewrite_type:AL_RRM_REWRITE_L3_UNICAST_IPV4_SHARED(1) Mapped_rii:L3_UNICAST_IPV4(9)
MAC Addr: MAC Addr: 52:54:00:19:93:e9,
L3IF LE Index 41
ASIC#:1 RI:38 Rewrite_type:AL_RRM_REWRITE_L3_UNICAST_IPV4_SHARED(1) Mapped_rii:L3_UNICAST_IPV4(9)
MAC Addr: MAC Addr: 01:00:5e:00:00:00,
L3IF LE Index 40
ASIC#:1 RI:39 Rewrite_type:AL_RRM_REWRITE_L3_UNICAST_IPV4_SHARED(1) Mapped_rii:L3_UNICAST_IPV4(9)
MAC Addr: MAC Addr: 52:54:00:00:50:17,
L3IF LE Index 40**MAC Address Destination-Index Verification**
Take the DI value from the previous command (0x526e) and utilize in show platform hardware fed switch active fwd-asic resource asic all destination-index range <DI> <DI>
Edge-1#show platform hardware fed switch active fwd-asic resource asic all destination-index range 0x526e 0x526e
ASIC#0:
Destination index = 0x526e
pmap = 0x00000000 0x00000004 <-- Convert decimal 4 to binary, which is 0100. Count this binary right to left, zero-based, so Port 2.
pmap_intf : [GigabitEthernet1/0/3]
cmi = 0x0
rcp_pmap = 0x0
al_rsc_cmi
CPU Map Index (CMI) [0]
ctiLo0 = 0
ctiLo1 = 0
ctiLo2 = 0
cpuQNum0 = 0
cpuQNum1 = 0
cpuQNum2 = 0
npuIndex = 0
stripSeg = 0
copySeg = 0
ASIC#1:
Destination index = 0x526e
pmap = 0x00000000 0x00000000
cmi = 0x0
rcp_pmap = 0x0
al_rsc_cmi
CPU Map Index (CMI) [0]
ctiLo0 = 0
ctiLo1 = 0
ctiLo2 = 0
cpuQNum0 = 0
cpuQNum1 = 0
cpuQNum2 = 0
npuIndex = 0
stripSeg = 0
copySeg = 0**Port Verification**
To correlate the port that was seen previously, use the commandshow platform software fed switch active ifm mappings and look at the Port column.
Edge-1#show platform software fed switch active ifm mappings ------------------ show platform software fed switch active ifm mappings ------------------ Interface IF_ID Inst Asic Core Port SubPort Mac Cntx LPN GPN Type Active GigabitEthernet1/0/1 0x1a 0 0 0 0 0 1 0 1 1 NIF Y GigabitEthernet1/0/2 0x1b 0 0 0 1 0 2 1 2 2 NIF Y GigabitEthernet1/0/3 0xb 0 0 0 2 0 3 2 3 3 NIF Y <-- Matches port 2 from previous output
**Hardware FED MAC Address Verification**
This output in a working/ideal scenario matches what the macHandle decode provided.
Edge-1#show platform hardware fed switch active matm macTable vlan 1026 mac 5254.0019.93e9 HEAD: MAC address 5254.0019.93e9 in VLAN 1026 KEY: vlan 7, mac 0x5254001993e9, l3_if 0, gpn 3, epoch 0, static 0, flood_en 0, vlan_lead_wless_flood_en 0, client_home_asic 0, learning_peerid 0, learning_peerid_valid 0 MASK: vlan 0, mac 0x0, l3_if 0, gpn 0, epoch 0, static 0, flood_en 0, vlan_lead_wless_flood_en 0, client_home_asic 0 learning_peerid 0, learning_peerid_valid 0 SRC_AD: need_to_learn 0, lrn_v 0, catchall 0, static_mac 0, chain_ptr_v 0, chain_ptr 0, static_entry_v 0, auth_state 0, auth_mode 0, traf_mode 0, is_src_ce 0 DST_AD: si 0xb7, bridge 0, replicate 0, blk_fwd_o 0, v4_mac 0, v6_mac 0, catchall 0, ign_src_lrn 0, port_mask_o 0, afd_cli_f 0, afd_lbl 0, priority 3, dest_mod_idx 0, destined_to_us 0, pv_trunk 0 Total Mac number of addresses:: 1
- The VLAN ID in hardware (MVID) is 7
- MAC Address: 5254.0019.93e9
- GPN: 3
LISP Entry
After successful host onboarding LISP entries for the end-host are created locally on Edge Node, as well as registered on Control Nodes (LISP MSMR - LISP Map Server / Map Resolver). All LISP checks needs to be done in regards to specific instance-id scope that can be checked for L2 and for L3:
Edge-1#show vlan id 1026 VLAN Name Status Ports ---- -------------------------------- --------- ------------------------------- 1026 red active L2LI0:8190, Gi1/0/3 <-- L2 LISP Instance ID tied to VLAN 1026
**L2 LISP Database Verification**
To check the L2 LISP Database use the command show lisp instance-id <L2 LISP IID> ethernet database <mac address>
Edge-1#show lisp instance-id 8190 ethernet database 5254.0019.93e9 LISP ETR MAC Mapping Database for LISP 0 EID-table Vlan 1026 (IID 8190), LSBs: 0x1 Entries total 1, no-route 0, inactive 0, do-not-register 2 5254.0019.93e9/48, dynamic-eid Auto-L2-group-8190, inherited from default locator-set rloc_222e1707-175d-4019-a783-060404f8bc2f -----> Endpoint MAC Uptime: 2d17h, Last-change: 2d17h Domain-ID: local Service-Insertion: N/A Locator Pri/Wgt Source State 10.47.1.12 10/10 cfg-intf site-self, reachable -----> Our own RLOC Map-server Uptime ACK Domain-ID 10.47.1.10 1d11h Yes 0 -----> RLOC of upstream collocated border 10.47.1.11 2d17h Yes 0 -----> RLOC of upstream collocated border
**LISP L2 Address Resolution (AR) Database Verification**
To check the LISP L2 AR Database, use the command show lisp instance-id <LISP L2 IID> ethernet database address-resolution <mac address>
Edge-1#show lisp instance-id 8190 ethernet database address-resolution 5254.0019.93e9 LISP ETR Address Resolution for LISP 0 EID-table Vlan 1026 (IID 8190) (*) -> entry being deleted Hardware Address L3 InstID Host Address 5254.0019.93e9 4099 10.47.4.2/32 <-- Endpoint MAC Address, LISP L3 Instance ID, Endpoint IPv4 Address, respectively
**LISP L3 Database Verfication**
To check the LISP L3 Database, use the command show lisp instance-id <LISP L3 IID> ipv4 database <IP address/Subnet Mask>
Edge-1#show lisp instance-id 4099 ipv4 database 10.47.4.2/32 LISP ETR IPv4 Mapping Database for LISP 0 EID-table vrf red_vn (IID 4099), LSBs: 0x1 Entries total 1, no-route 0, inactive 0, do-not-register 1 10.47.4.2/32, dynamic-eid red-IPV4, inherited from default locator-set rloc_222e1707-175d-4019-a783-060404f8bc2f -----> Endpoint IPv4 Address Uptime: 2d18h, Last-change: 2d18h Domain-ID: local Service-Insertion: N/A Locator Pri/Wgt Source State 10.47.1.12 10/10 cfg-intf site-self, reachable -----> Our own RLOC Map-server Uptime ACK Domain-ID 10.47.1.10 1d11h Yes 0 -----> RLOC of upstream collocated border 10.47.1.11 2d17h Yes 0 -----> RLOC of upstream collocated border
**CEF Verification**
To check CEF, use the command show ip cef vrf <vrf name> <IP address> internal
Edge-1#show ip cef vrf red_vn 10.47.4.2 internal
10.47.4.2/32, epoch 1, flags [att, sc], RIB[D], refcnt 6, per-destination sharing
sources: RIB, Adj, IPL
feature space:
IPRM: 0x00058000
Broker: linked, distributed at 3rd priority
subblocks:
SC owned,sourced: LISP local EID -
SC inherited: LISP remote EID - locator status bits 0x00000000
SC inherited: LISP cfg dyn-EID - LISP configured dynamic-EID
LISP EID attributes: localEID Yes, c-dynEID Yes, d-dynEID Yes, a-dynEID No
SC owned,sourced: LISP generalised SMR - [disabled, not inheriting, 0x7F06D0A67E40 locks: 1]
Adj source: IP adj out of Vlan1026, addr 10.47.4.2 7F06D300B738
Dependent covered prefix type adjfib, cover 10.47.4.0/24
2 IPL sources [no flags]
ifnums:
Vlan1026(29): 10.47.4.2
path list 7F06CEE8D720, 3 locks, per-destination, flags 0x49 [shble, rif, hwcn]
path 7F06D0A900C8, share 1/1, type attached nexthop, for IPv4
nexthop 10.47.4.2 Vlan1026, IP adj out of Vlan1026, addr 10.47.4.2 7F06D300B738
output chain:
IP adj out of Vlan1026, addr 10.47.4.2 7F06D300B738
In addition to local LISP entries on the SDA Edge Node, SDA Control Nodes (LISP MS/MR) also contain extra information about the endpoints:
Collocated Border L2 LISP Server Verification:
To check the L2 LISP Server, use the command show lisp instance-id <L2 LISP IID> ethernet server <MAC Address>
Border-1#show lisp instance-id 8190 ethernet server 5254.0019.93e9
LISP Site Registration Information
Site name: site_uci
Description: map-server configured from Cisco DNA-Center
Allowed configured locators: any
Requested EID-prefix:
EID-prefix: 5254.0019.93e9/48 instance-id 8190 <-- Endpoint MAC Address
First registered: 2w5d
Last registered: 3d16h
Routing table tag: 0
Origin: Dynamic, more specific of any-mac
Merge active: No
Proxy reply: Yes
Skip Publication: No
Force Withdraw: No
TTL: 1d00h
State: complete
Extranet IID: Unspecified
Registration errors:
Authentication failures: 0
Allowed locators mismatch: 0
ETR 10.47.1.12:21038, last registered 3d16h, proxy-reply, map-notify <-- Egress Tunnel Router (Fabric Edge IP address)
TTL 1d00h, no merge, hash-function sha1
state complete, no security-capability
nonce 0xB60C4314-0x97BB332D
xTR-ID 0xAB3179F6-0xC774F22C-0x00F2C82E-0x3A66738D
site-ID unspecified
Domain-ID local
Multihoming-ID unspecified
sourced by reliable transport
Locator Local State Pri/Wgt Scope
10.47.1.12 yes up 10/10 IPv4 none <--(Fabric Edge IP address)Collocated Border L2 LISP Address Resolution (AR) Server Verification:
To check the L2 LISP AR Server use the command show lisp instance-id <LISP L2 IID> ethernet server address-resolution <IP address>
To check the registration history use the command show lisp instance-id <LISP L2 IID> ethernet server address-resolution <IP address> registration-history
Border-1#show lisp instance-id 8190 ethernet server address-resolution 10.47.4.2 Address-resolution data for router lisp 0 instance-id 8190 Site name: site_uci Host Address: 10.47.4.2/32 Hardware Address: 5254.0019.93e9 First registered: 2w5d Last registered: 3d16h Registration errors: Authentication failures: 0 ETR 10.47.1.12:21038 Last registered: 3d16h TTL: 1d00h xTR-ID: 0xAB3179F6-0xC774F22C-0x00F2C82E-0x3A66738D Site-ID: unspecified Registered addr: 5254.0019.93e9 L3 Instance ID: 4099
Border-1#show lisp instance-id 8190 ethernet server address-resolution 10.47.4.2 registration-history Map-Server registration history Roam = Did host move to a new location? WLC = Did registration come from a Wireless Controller? Prefix qualifier: + = Register Event, - = Deregister Event, * = AR register event Timestamp (UTC) Instance Proto Roam WLC Source EID prefix / Locator *Sep 29 16:50:27.762 8190 TCP No No 10.47.1.12 +*10.47.4.2/32 / 5254.0019.93e9 *Oct 1 21:05:11.086 8190 TCP No No 10.47.1.12 +*10.47.4.2/32 / 5254.0019.93e9 *Oct 2 06:51:11.882 8190 TCP No No 10.47.1.12 +*10.47.4.2/32 / 5254.0019.93e9 *Oct 3 00:56:33.642 8190 TCP No No 10.47.1.12 +*10.47.4.2/32 / 5254.0019.93e9 *Oct 3 01:53:45.934 8190 TCP No No 10.47.1.12 +*10.47.4.2/32 / 5254.0019.93e9 *Oct 6 04:36:08.685 8190 TCP No No 10.47.1.12 +*10.47.4.2/32 / 5254.0019.93e9
Collocated Border L3 LISP Server Verification
To check the L3 LISP Server, use the command show lisp instance-id <LISP L3 IID> ipv4 server <IP address>
To check the L3 LISP Server registration history, use the command show lisp instance-id <LISP L3 IID> ipv4 server <IP address> registration-history
Border-1#show lisp instance-id 4099 ipv4 server 10.47.4.2
LISP Site Registration Information
Site name: site_uci
Description: map-server configured from Cisco DNA-Center
Allowed configured locators: any
Requested EID-prefix:
EID-prefix: 10.47.4.2/32 instance-id 4099
First registered: 2w5d
Last registered: 02:39:39
Routing table tag: 0
Origin: Dynamic, more specific of 10.47.4.0/24
Merge active: No
Proxy reply: Yes
Skip Publication: No
Force Withdraw: No
TTL: 1d00h
State: complete
Extranet IID: Unspecified
Registration errors:
Authentication failures: 0
Allowed locators mismatch: 0
ETR 10.47.1.12:21038, last registered 02:39:39, proxy-reply, map-notify
TTL 1d00h, no merge, hash-function sha1
state complete, no security-capability
nonce 0x128CB668-0xF7B85F77
xTR-ID 0xAB3179F6-0xC774F22C-0x00F2C82E-0x3A66738D
site-ID unspecified
Domain-ID local
Multihoming-ID unspecified
sourced by reliable transport
Locator Local State Pri/Wgt Scope
10.47.1.12 yes up 10/10 IPv4 none
Border-1#show lisp instance-id 4099 ipv4 server 10.47.4.2/32 registration-history
Map-Server registration history
Roam = Did host move to a new location?
WLC = Did registration come from a Wireless Controller?
Prefix qualifier: + = Register Event, - = Deregister Event, * = AR register event
Timestamp (UTC) Instance Proto Roam WLC Source
EID prefix / Locator
*Oct 6 04:36:01.548 4099 UDP No No 10.47.1.12
+ 10.47.4.2/32
*Oct 6 04:36:08.686 4099 TCP No No 10.47.1.12
+ 10.47.4.2/32
*Oct 9 18:35:48.058 4099 TCP No No 10.47.1.12
+ 10.47.4.2/32 ARP Resolution in SDA
It is assumed that CIsco Catalyst Center has been used to provision SDA fabric with default settings. This means that Layer-2 extension is enabled and that all traffic within the Fabric (in same VLAN / VN) is forwarded based on MAC address lookups / LISP Ethernet instance, rather than IP address lookups / LISP IP instance.
From troubleshooting perspective, it can be useful to configure static ARP entries on both hosts to quickly check if the issue is with generic connectivity in the fabric (in such case ping does not work between hosts) or only with ARP resolution.
ARP process in SDA Fabric leverages LISP to resolve the identify and location of hosts and is different from ARP behavior in traditional Routing/Switching environments.
Step 1: Fabric Endpoint sends an ARP request to determine the MAC/IP binding for the other Fabric Endpoint
Packet capture can be configured on the ingress interface to confirm that ARP packet is received from the host:
Edge-1#monitor capture 1 interface g1/0/3 in match any
Edge-1#mon cap 1 start
Started capture point : 1
Edge-1#mon cap 1 stop
Capture statistics collected at software:
Capture duration - 22 seconds
Packets received - 13
Packets dropped - 0
Packets oversized - 0
Number of Bytes dropped at asic not collected
Capture buffer will exists till exported or cleared
Stopped capture point : 1
Edge-1#show monitor capture 1 buffer brief
Starting the packet display ........ Press Ctrl + Shift + 6 to exit
1 0.000000 52:54:00:19:93:e9 -> ff:ff:ff:ff:ff:ff ARP 60 Who has 10.47.4.3? Tell 10.47.4.2
2 1.028893 52:54:00:19:93:e9 -> ff:ff:ff:ff:ff:ff ARP 60 Who has 10.47.4.3? Tell 10.47.4.2
3 2.058244 52:54:00:19:93:e9 -> ff:ff:ff:ff:ff:ff ARP 60 Who has 10.47.4.3? Tell 10.47.4.2
Edge-1#show monitor capture 1 buffer display-filter arp detailed
Starting the packet display ........ Press Ctrl + Shift + 6 to exit
Frame 1: 60 bytes on wire (480 bits), 60 bytes captured (480 bits) on interface /tmp/epc_ws/wif_to_ts_pipe, id 0
Interface id: 0 (/tmp/epc_ws/wif_to_ts_pipe)
Interface name: /tmp/epc_ws/wif_to_ts_pipe
Encapsulation type: Ethernet (1)
Arrival Time: Oct 10, 2023 14:52:03.659290000 UTC
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1696949523.659290000 seconds
[Time delta from previous captured frame: 0.000000000 seconds]
[Time delta from previous displayed frame: 0.000000000 seconds]
[Time since reference or first frame: 0.000000000 seconds]
Frame Number: 1
Frame Length: 60 bytes (480 bits)
Capture Length: 60 bytes (480 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ethertype:arp]
Ethernet II, Src: 52:54:00:19:93:e9 (52:54:00:19:93:e9), Dst: ff:ff:ff:ff:ff:ff (ff:ff:ff:ff:ff:ff) <-- SMAC/DMAC respectively
Destination: ff:ff:ff:ff:ff:ff (ff:ff:ff:ff:ff:ff)
Address: ff:ff:ff:ff:ff:ff (ff:ff:ff:ff:ff:ff)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...1 .... .... .... .... = IG bit: Group address (multicast/broadcast)
Source: 52:54:00:19:93:e9 (52:54:00:19:93:e9)
Address: 52:54:00:19:93:e9 (52:54:00:19:93:e9)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Type: ARP (0x0806)
Padding: 000000000000000000000000000000000000
Address Resolution Protocol (request)
Hardware type: Ethernet (1)
Protocol type: IPv4 (0x0800)
Hardware size: 6
Protocol size: 4
Opcode: request (1)
Sender MAC address: 52:54:00:19:93:e9 (52:54:00:19:93:e9)
Sender IP address: 10.47.4.2
Target MAC address: 00:00:00:00:00:00 (00:00:00:00:00:00)
Target IP address: 10.47.4.3Step 2. Edge Node consumes ARP packet and generate LISP request to determine MAC address of HOST-02.
Edge-1 sends a LISP Map-Request to resolve the MAC Address of 10.47.4.3 to the LISP Control-Planes (Collocated Borders):
Edge-1#debug lisp control-plane all
Edge-1#debug l2lisp all
LISP[REMT ]-0: Map Request: Delay is over for IID 8190 EID 10.47.4.3/32, requester 'AR'. LISP[REMT ]-0 IID 8190: Schedule processing of Map-Requests from 'remote EID prefix' in IPv4. LISP[REMT ]-0: Map Request: Sending request for IID 8190 EID 10.47.4.3/32, requester 'AR'.
Step 3. Control Node receives LISP request for IP/MAC mapping and send a response back to SDA Edge Node
LISP Map-Request is received from the Fabric Edge and responds with a LISP Map-Reply with the MAC address that ties to 10.47.4.3
Border-1#debug lisp control-plane all
Border-1#debug l2lisp all
LISP[TRNSP]-0: Processing received Map-Request(1) message on GigabitEthernet1/0/3 from 10.47.4.3:4342 to 10.47.4.3:4342. LISP[MR ]-0: Received Map-Request with 1 records, first EID IID 8190 10.47.4.3/32, source EID UNSPEC, nonce 0x73F75F36-0x061CAF58. LISP[MR ]-0 IID 8190 Eth-ARP: MS EID 10.47.4.3/32: Sending proxy reply to 10.47.1.12.
The LISP Control-Plane responds with a Proxy Reply based on the address-resolution entry stored in its local database
Border-1#show lisp instance-id 8190 ethernet server address-resolution 10.47.4.3 Address-resolution data for router lisp 0 instance-id 8190 Site name: site_uci Host Address: 10.47.4.3/32 Hardware Address: 5254.001e.ad00 First registered: 21:11:17 Last registered: 21:11:17 Registration errors: Authentication failures: 0 ETR 10.47.1.13:16056 Last registered: 21:11:17 TTL: 1d00h xTR-ID: 0x8CEE6478-0x9358E248-0xE935FF07-0x8C3C5450 Site-ID: unspecified Registered addr: 5254.001e.ad00 L3 Instance ID: 4099
Step 4. Edge Node receives LISP reply with MAC address of 10.47.4.3
LISP Proxy Reply is received by the Fabric Edge Node:
LISP[REMT ]-0: Processing Map-Reply mapping record for IID 8190 MAC 5254.001e.ad00/48 LCAF 2, ttl 1440, action none, not authoritative, 1 locator. LISP[REMT ]-0: Processing mapping information for EID prefix IID 8190 5254.001e.ad00/48.
Step 5. Edge Node sends LISP Map-Request packet to determine RLOC location for MAC address
After successful completion of first three steps, Edge Node knows MAC address of 10.47.4.3 for which ARP was initially generated for. As Layer-2 extension is enabled, Edge Node does not reply back with this information to 10.47.4.2, but rather use it to determine RLOC location of Egress Node Edge, so that it can forward ARP towards 10.47.4.3 as in a traditional Layer-2 network.
For this reason, Edge Node generates yet another LISP Map Request packet in Ethernet Instance, this time requesting RLOC information for 10.47.4.2's MAC address:
Edge-1#debug lisp control-plane all
Edge-1#debug l2lisp all
*Oct 10 17:01:41.430: LISP[REMT ]-0 IID 8190: Schedule processing of Map-Requests from 'remote EID prefix' in IPv4. *Oct 10 17:01:41.430: LISP[REMT ]-0: Map Request: Sending request for IID 8190 EID 5254.001e.ad00/48, requester 'remote EID prefix'
Step 6: LISP Map-Request packet is received by Control Node to determine RLOC location for MAC address
Control Node receives the LISP packet and reply to it based on its local database state
Border-1#debug lisp control-plane all Border-1#debug l2lisp all *Oct 10 16:04:42.055: LISP[MR ]-0 IID 8190 Eth-ARP: MS EID 10.47.4.3/32: Sending proxy reply to 10.47.1.12. *Oct 10 16:04:42.407: LISP[MR ]-0: Received Map-Request with 1 records, first EID IID 8190 5254.001e.ad00/48, source EID 0000.0c9f.f341, nonce 0x75E13E8C-0x40DE7912. *Oct 10 16:04:42.408: LISP[MR ]-0 IID 8190 MAC: MS EID 5254.001e.ad00/48: Sending proxy reply to 10.47.1.12.
Step 7: LISP Map-Reply is received by Edge Node
LISP Map Reply generated by Control Node is received by Edge Node:
Edge-1#debug lisp control-plane all
Edge-1#debug l2lisp all
*Oct 10 17:44:00.181: LISP[TRNSP]-0: Processing received Map-Reply(2) message on GigabitEthernet1/0/2 from 10.47.1.13:4342 to 10.47.1.12:4342. *Oct 10 17:44:00.181: LISP[REMT ]-0: Received Map-Reply with nonce 0xF954EC80-0x039D7E4A, 1 records. *Oct 10 17:44:00.181: LISP[REMT ]-0: Map-Reply nonce matches pending request for IID 8190 EID 5254.001e.ad00/48, requester 'remote EID RLOC'. *Oct 10 17:44:00.181: LISP[REMT ]-0: Processing Map-Reply mapping record for IID 8190 MAC 5254.001e.ad00/48 LCAF 2, ttl 1440, action none, authoritative, 1 locator. *Oct 10 17:44:00.181: LISP[REMT ]-0: Map Request: Received reply with rtt 560ms. *Oct 10 17:44:00.181: LISP[REMT ]-0: Processing mapping information for EID prefix IID 8190 5254.001e.ad00/48.
This finally creates an entry in LISP Ethernet instance map-cache and allows ARP packet to be forwarded towards Edge-2 where 10.47.4.3 is connected to
Edge-1#show lisp instance-id 8190 ethernet map-cache 5254.001e.ad00
LISP MAC Mapping Cache for LISP 0 EID-table Vlan 1026 (IID 8190), 1 entries
5254.001e.ad00/48, uptime: 00:04:11, expires: 23:55:48, via map-reply, complete
Sources: map-reply
State: complete, last modified: 00:04:11, map-source: 10.47.1.13
Active, Packets out: 8(0 bytes), counters are not accurate (~ 00:00:04 ago)
Encapsulating dynamic-EID traffic
Locator Uptime State Pri/Wgt Encap-IID
10.47.1.13 00:04:11 up 10/10 -
Last up-down state change: 00:04:11, state change count: 1
Last route reachability change: 00:04:11, state change count: 1
Last priority / weight change: never/never
RLOC-probing loc-status algorithm:
Last RLOC-probe sent: 00:04:11 (rtt 560ms)Step 8. ARP is encapsulated in VXLAN and send towards HOST-02
All of the LISP related steps were required to determine where 10.47.4.3 is located, so that Edge Node can sent original ARP (broadcast) packet as unicast towards proper Edge Node. The original ARP request is cached (not dropped) by Edge Node CPU until all of the steps are completed allowing proper ARP resolution even when single ARP packet was sent from 10.47.4.2.
ARP packet is encapsulated in VXLAN, as seen in the example:
Edge-2#show monitor capture 1 buffer display-filter arp brief Starting the packet display ........ Press Ctrl + Shift + 6 to exit
67 15.149181 52:54:00:19:93:e9 -> 52:54:00:1e:ad:00 ARP 110 Who has 10.47.4.3? Tell 10.47.4.2 68 15.155511 52:54:00:19:93:e9 -> 52:54:00:1e:ad:00 ARP 110 Who has 10.47.4.3? Tell 10.47.4.2
The ARP request has been encapsulated in VXLAN and also been converted from a broadcast ARP request to a unicast ARP request.
Frame 68: 110 bytes on wire (880 bits), 110 bytes captured (880 bits) on interface /tmp/epc_ws/wif_to_ts_pipe, id 0
Interface id: 0 (/tmp/epc_ws/wif_to_ts_pipe)
Interface name: /tmp/epc_ws/wif_to_ts_pipe
Encapsulation type: Ethernet (1)
Arrival Time: Oct 10, 2023 17:56:43.256570000 UTC
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1696960603.256570000 seconds
[Time delta from previous captured frame: 0.006330000 seconds]
[Time delta from previous displayed frame: 0.006330000 seconds]
[Time since reference or first frame: 15.155511000 seconds]
Frame Number: 68
Frame Length: 110 bytes (880 bits)
Capture Length: 110 bytes (880 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ethertype:ip:udp:vxlan:eth:ethertype:arp]
Ethernet II, Src: 52:54:00:0a:42:11 (52:54:00:0a:42:11), Dst: 52:54:00:17:fe:65 (52:54:00:17:fe:65)
Destination: 52:54:00:17:fe:65 (52:54:00:17:fe:65)
Address: 52:54:00:17:fe:65 (52:54:00:17:fe:65)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Source: 52:54:00:0a:42:11 (52:54:00:0a:42:11)
Address: 52:54:00:0a:42:11 (52:54:00:0a:42:11)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 10.47.1.12, Dst: 10.47.1.13 <-- 10.47.1.12 is Edge-1 RLOC, 10.47.1.13 is Edge-2 RLOC
0100 .... = Version: 4
.... 0101 = Header Length: 20 bytes (5)
Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
0000 00.. = Differentiated Services Codepoint: Default (0)
.... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
Total Length: 96
Identification: 0x1781 (6017)
Flags: 0x4000, Don't fragment
0... .... .... .... = Reserved bit: Not set
.1.. .... .... .... = Don't fragment: Set
..0. .... .... .... = More fragments: Not set
Fragment offset: 0
Time to live: 253
Protocol: UDP (17)
Header checksum: 0x4f95 [validation disabled]
[Header checksum status: Unverified]
Source: 10.47.1.12
Destination: 10.47.1.13
User Datagram Protocol, Src Port: 65354, Dst Port: 4789
Source Port: 65354
Destination Port: 4789
Length: 76
[Checksum: [missing]]
[Checksum Status: Not present]
[Stream index: 0]
[Timestamps]
[Time since first frame: 15.155511000 seconds]
[Time since previous frame: 0.006330000 seconds]
Virtual eXtensible Local Area Network
Flags: 0x8800, GBP Extension, VXLAN Network ID (VNI)
1... .... .... .... = GBP Extension: Defined
.... .... .0.. .... = Don't Learn: False
.... 1... .... .... = VXLAN Network ID (VNI): True
.... .... .... 0... = Policy Applied: False
.000 .000 0.00 .000 = Reserved(R): 0x0000
Group Policy ID: 0
VXLAN Network Identifier (VNI): 8190 <-- L2 LISP IID
Reserved: 0
Ethernet II, Src: 52:54:00:19:93:e9 (52:54:00:19:93:e9), Dst: 52:54:00:1e:ad:00 (52:54:00:1e:ad:00) <--Unicast ARP Request
Destination: 52:54:00:1e:ad:00 (52:54:00:1e:ad:00)
Address: 52:54:00:1e:ad:00 (52:54:00:1e:ad:00)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Source: 52:54:00:19:93:e9 (52:54:00:19:93:e9)
Address: 52:54:00:19:93:e9 (52:54:00:19:93:e9)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Type: ARP (0x0806)
Trailer: 000000000000000000000000000000000000
Address Resolution Protocol (request)
Hardware type: Ethernet (1)
Protocol type: IPv4 (0x0800)
Hardware size: 6
Protocol size: 4
Opcode: request (1)
Sender MAC address: 52:54:00:19:93:e9 (52:54:00:19:93:e9)
Sender IP address: 10.47.4.2
Target MAC address: 00:00:00:00:00:00 (00:00:00:00:00:00)
Target IP address: 10.47.4.3
Step 9. ARP Reply is generated by 10.47.4.3 and sent towards 10.47.4.2
Edge-2#show monitor capture 1 buffer display-filter arp brief
Starting the packet display ........ Press Ctrl + Shift + 6 to exit
1 0.000000 52:54:00:1e:ad:00 -> 52:54:00:19:93:e9 ARP 60 10.47.4.3 is at 52:54:00:1e:ad:00
2 0.069429 52:54:00:1e:ad:00 -> 52:54:00:19:93:e9 ARP 60 10.47.4.3 is at 52:54:00:1e:ad:00
11 5.960508 52:54:00:1e:ad:00 -> 52:54:00:19:93:e9 ARP 60 Who has 10.47.4.2? Tell 10.47.4.3At this time packet is destined not to broadcast address (as original ARP request) but to 10.47.4.2's MAC address, when it reaches ingress Edge Node (Edge-2), normal LISP operation is triggered. Initially MAC address of 10.47.4.2 is missing in LISP Ethernet instance of Edge Node, packet is punted to CPU to generate LISP Map Request to determine RLOC for HOST-01. This behavior is exactly the same as described in other sections in this document and allow to create LISP Map Cache entry for 10.47.4.2 on Edge-2:
Edge-2#show lisp instance-id 8190 ethernet map-cache 5254.0019.93e9
LISP MAC Mapping Cache for LISP 0 EID-table Vlan 1026 (IID 8190), 1 entries
5254.0019.93e9/48, uptime: 03:18:28, expires: 20:41:32, via map-reply, complete
Sources: map-reply
State: complete, last modified: 03:18:28, map-source: 10.47.1.12
Active, Packets out: 386(0 bytes), counters are not accurate (~ 00:00:12 ago)
Encapsulating dynamic-EID traffic
Locator Uptime State Pri/Wgt Encap-IID
10.47.1.12 03:18:28 up 10/10 -
Last up-down state change: 03:18:28, state change count: 1
Last route reachability change: 03:18:28, state change count: 1
Last priority / weight change: never/never
RLOC-probing loc-status algorithm:
Last RLOC-probe sent: 03:18:28 (rtt 710ms)The entry allows ARP Reply to be successfully sent towards Edge-1 in VXLAN Encapsulation and forwarded further to 10.47.4.2 competing whole ARP resolution process.
Basic Host Reachability in SDA Fabric (Same VLAN / Same VN)
It is assumed that ARP resolution completed successfully and both hosts 10.47.4.2 and 10.47.4.3 have proper ARP entries for each other.
From troubleshooting perspective, it is very useful to configure static ARP entries on both hosts to quickly check if the issue is with generic connectivity in the fabric (in such case ping does not work between hosts) or only with ARP process.
10.47.4.2 generates an ICMP Request towards 10.47.4.3:
Edge-1#show monitor capture 1 buffer brief
Starting the packet display ........ Press Ctrl + Shift + 6 to exit
1 0.000000 10.47.4.2 -> 10.47.4.3 ICMP 98 Echo (ping) request id=0x0040, seq=3/768, ttl=64
Edge-1#show monitor capture 1 buffer detail
Starting the packet display ........ Press Ctrl + Shift + 6 to exit
Frame 1: 98 bytes on wire (784 bits), 98 bytes captured (784 bits) on interface /tmp/epc_ws/wif_to_ts_pipe, id 0
Interface id: 0 (/tmp/epc_ws/wif_to_ts_pipe)
Interface name: /tmp/epc_ws/wif_to_ts_pipe
Encapsulation type: Ethernet (1)
Arrival Time: Oct 10, 2023 18:21:21.484694000 UTC
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1696962081.484694000 seconds
[Time delta from previous captured frame: 0.000000000 seconds]
[Time delta from previous displayed frame: 0.000000000 seconds]
[Time since reference or first frame: 0.000000000 seconds]
Frame Number: 1
Frame Length: 98 bytes (784 bits)
Capture Length: 98 bytes (784 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ethertype:ip:icmp:data]
Ethernet II, Src: 52:54:00:19:93:e9 (52:54:00:19:93:e9), Dst: 52:54:00:1e:ad:00 (52:54:00:1e:ad:00) <-- Endpoint MAC, Anycast GW MAC respectively
Destination: 52:54:00:1e:ad:00 (52:54:00:1e:ad:00)
Address: 52:54:00:1e:ad:00 (52:54:00:1e:ad:00)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Source: 52:54:00:19:93:e9 (52:54:00:19:93:e9)
Address: 52:54:00:19:93:e9 (52:54:00:19:93:e9)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 10.47.4.2, Dst: 10.47.4.3
0100 .... = Version: 4
.... 0101 = Header Length: 20 bytes (5)
Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
0000 00.. = Differentiated Services Codepoint: Default (0)
.... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
Total Length: 84
Identification: 0x7321 (29473)
Flags: 0x4000, Don't fragment
0... .... .... .... = Reserved bit: Not set
.1.. .... .... .... = Don't fragment: Set
..0. .... .... .... = More fragments: Not set
Fragment offset: 0
Time to live: 64
Protocol: ICMP (1)
Header checksum: 0xab25 [validation disabled]
[Header checksum status: Unverified]
Source: 10.47.4.2
Destination: 10.47.4.3
Internet Control Message Protocol
Type: 8 (Echo (ping) request)
Code: 0
Checksum: 0x02ea [correct]
[Checksum Status: Good]
Identifier (BE): 64 (0x0040)
Identifier (LE): 16384 (0x4000)
Sequence number (BE): 3 (0x0003)
Sequence number (LE): 768 (0x0300)
Data (56 bytes)
0000 68 95 8c 3d 00 00 00 00 00 00 00 00 00 00 00 00 h..=............
0010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0030 00 00 00 00 00 00 00 00 ........
Data: 68958c3d0000000000000000000000000000000000000000b^@&
[Length: 56]ICMP Packet is sent towards 10.47.4.3 to the Edge Node specified in Locator field: 10.47.1.13 (Edge-2) and can be captured via embedded packet capture.
As packet is received in VLAN where L2-extension is enabled, the lookup is done in LISP Ethernet instance:
Edge-1#show lisp instance-id 8190 ethernet map-cache 5254.001e.ad00
LISP MAC Mapping Cache for LISP 0 EID-table Vlan 1026 (IID 8190), 1 entries
5254.001e.ad00/48, uptime: 00:22:29, expires: 23:37:32, via map-reply, complete
Sources: map-reply
State: complete, last modified: 00:22:29, map-source: 10.47.1.13
Active, Packets out: 42(0 bytes), counters are not accurate (~ 00:00:58 ago)
Encapsulating dynamic-EID traffic
Locator Uptime State Pri/Wgt Encap-IID
10.47.1.13 00:22:29 up 10/10 -
Last up-down state change: 00:22:29, state change count: 1
Last route reachability change: 00:22:29, state change count: 1
Last priority / weight change: never/never
RLOC-probing loc-status algorithm:
Last RLOC-probe sent: 00:22:28 (rtt 1609ms)Check the MAC address of the remote endpoint, it points the L2LI0, which is expected
Edge-1#show mac add add 5254.001e.ad00
Mac Address Table
-------------------------------------------
Vlan Mac Address Type Ports
---- ----------- -------- -----
1026 5254.001e.ad00 CP_LEARN L2LI0
Total Mac Addresses installed by LISP: REMOTE: 1Check the MAC address in FED, additional information can be gleaned
Edge-1#show platform software fed sw active matm macTable vlan 1026 mac 5254.001e.ad00 VLAN MAC Type Seq# EC_Bi Flags machandle siHandle riHandle diHandle *a_time *e_time ports Con ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 1026 5254.001e.ad00 0x1000001 0 0 64 0x7f65ecfdd3a8 0x7f65ecfdd1f8 0x7f65ecfdd048 0x0 0 2 RLOC 10.47.1.13 adj_id 97 No ======platform hardware details ====== Asic: 0 htm-handle = 0x7f65ecc4d188 MVID = 7 gpn = 1 SI = 0xc7 RI = 0x12 DI = 0x5012 Asic: 1 SI = 0xc7 RI = 0x12 DI = 0x5013
MAC Address macHandle Decode
Take the macHandle (0x7f65ecfdd3a8) from the previous command and use in the command show platform hardware fed switch active fwd-asic abstraction print-resource-handle <macHandle> 1
Edge-1#show platform hardware fed switch active fwd-asic abstraction print-resource-handle 0x7f65ecfdd3a8 1 Handle:0x7f65ecfdd3a8 Res-Type:ASIC_RSC_HASH_TCAM Res-Switch-Num:0 Asic-Num:255 Feature-ID:AL_FID_L2_WIRELESS Lkp-ftr-id:LKP_FEAT_L2_SRC_MAC_VLAN ref_count:1 priv_ri/priv_si Handle: (nil)Hardware Indices/Handles: handle [ASIC: 0]: 0x7f65ecc4d188 Features sharing this resource:Cookie length: 12 1e 00 54 52 00 ad 07 80 07 00 00 00 Detailed Resource Information (ASIC_INSTANCE# 0) ---------------------------------------- Number of HTM Entries: 1 Entry 0: (handle 0x7f65ecc4d188) Absolute Index: 4706 Time Stamp: 14 KEY - vlan:7 mac:0x5254001ead00 l3_if:0 gpn:3401 epoch:0 static:0 flood_en:0 vlan_lead_wless_flood_en: 0 client_home_asic: 0 learning_peerid 0, learning_peerid_valid 0 lvx:1 MASK - vlan:0 mac:0x0 l3_if:0 gpn:0 epoch:0 static:0 flood_en:0 vlan_lead_wless_flood_en: 0 client_home_asic: 0 learning_peerid 0, learning_peerid_valid 0 lvx:0 SRC_AD - need_to_learn:0 lrn_v:0 catchall:0 static_mac:0 chain_ptr_v:0 chain_ptr: 0 static_entry_v:0 auth_state:0 auth_mode:0 auth_behavior_tag:0 traf_m:0 is_src_ce:0 DST_AD - si:0xc7 bridge:0 replicate:0 blk_fwd_o:0 v4_rmac:0 v6_rmac:0 catchall:0 ign_src_lrn:0 port_mask_o:0 afd_cli_f:0 afd_lbl:0 prio:3 dest_mod_idx:0 destined_to_us:0 pv_trunk:0 smr:1 ==============================================================
MAC Address siHandle Decode
Take the siHandle (0x7f65ecfdd1f8) from the previous command and use in the command show platform hardware fed switch active fwd-asic abstraction print-resource-handle <siHandle> 1
Edge-1#show platform hardware fed switch active fwd-asic abstraction print-resource-handle 0x7f65ecfdd1f8 1 Handle:0x7f65ecfdd1f8 Res-Type:ASIC_RSC_SI Res-Switch-Num:255 Asic-Num:255 Feature-ID:AL_FID_L2_WIRELESS Lkp-ftr-id:LKP_FEAT_INVALID ref_count:1 priv_ri/priv_si Handle: 0x7f65ecfdd048Hardware Indices/Handles: index0:0xc7 mtu_index/l3u_ri_index0:0x0 index1:0xc7 mtu_index/l3u_ri_index1:0x0 Features sharing this resource:58 (1)] Cookie length: 56 00 00 00 00 00 00 00 00 02 04 00 00 00 00 00 00 00 00 00 00 07 00 61 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 Detailed Resource Information (ASIC_INSTANCE# 0) ---------------------------------------- Station Index (SI) [0xc7] <-- Contains the Rewrite Index (RI) and Outgoing Interface Information (DI) RI = 0x12 <-- Rewrite Index = Contains information for forwarding DI = 0x5012 <-- Destination Index = Outgoing Interface stationTableGenericLabel = 0 stationFdConstructionLabel = 0x7 lookupSkipIdIndex = 0 rcpServiceId = 0 dejaVuPreCheckEn = 0 Replication Bitmap: LD Detailed Resource Information (ASIC_INSTANCE# 1) ---------------------------------------- Station Index (SI) [0xc7] <-- Contains the Rewrite Index (RI) and Outgoing Interface Information (DI)
RI = 0x12 <-- Rewrite Index = Contains information for forwarding
DI = 0x5013 <-- Destination Index = Outgoing Interface stationTableGenericLabel = 0 stationFdConstructionLabel = 0x7 lookupSkipIdIndex = 0 rcpServiceId = 0 dejaVuPreCheckEn = 0 Replication Bitmap: LD ==============================================================
Rewrite Index Decode
Take the RI (0x12) and use in the command show platform hardware fed switch active fwd-asic resource asic all rewrite-index range <RI> <RI>
Edge-1#show platform hardware fed switch active fwd-asic resource asic all rewrite-index range 0x12 0x12 ASIC#:0 RI:18 Rewrite_type:AL_RRM_REWRITE_L2_PAYLOAD_L2LISP_ENCAP(115) Mapped_rii:LVX_L2_ENCAP_L2_PAYLOAD_ROUTED(133) Src IP: 10.47.1.12 <-- Local RLOC Dst IP: 10.47.1.13 <-- Remote RLOC iVxlan dstMac: 0x5254:0x01c:0x7de0 iVxlan srcMac: 0x00:0x00:0x00 IPv4 TTL: 0 iid present: 1 lisp iid: 0 lisp flags: 0 dst Port: 4789 update only l3if: 0 is Sgt: 1 is TTL Prop: 0 L3if LE: 0 (0) Port LE: 0 (0) Vlan LE: 7 (0) ASIC#:1 RI:18 Rewrite_type:AL_RRM_REWRITE_L2_PAYLOAD_L2LISP_ENCAP(115) Mapped_rii:LVX_L2_ENCAP_L2_PAYLOAD_ROUTED(133) Src IP: 10.47.1.12 <-- Local RLOC Dst IP: 10.47.1.13 <-- Remote RLOC iVxlan dstMac: 0x5254:0x01c:0x7de0 iVxlan srcMac: 0x00:0x00:0x00 IPv4 TTL: 0 iid present: 1 lisp iid: 0 lisp flags: 0 dst Port: 4789 update only l3if: 0 is Sgt: 1 is TTL Prop: 0 L3if LE: 0 (0) Port LE: 0 (0) Vlan LE: 7 (0)
Destination-Index Decode
Take the DI (0x5012) and use in the command show platform hardware fed switch active fwd-asic resource asic all destination-index range <DI> <DI>
Edge-1#show platform hardware fed switch active fwd-asic resource asic all destination-index range 0x5012 0x5012 ASIC#0: Destination index = 0x5012 DI_RCP_PORT1 <-- Recirculation port for VXLAN imposition pmap = 0x00000000 0x00000000 cmi = 0x0 rcp_pmap = 0x1 al_rsc_cmi CPU Map Index (CMI) [0] ctiLo0 = 0 ctiLo1 = 0 ctiLo2 = 0 cpuQNum0 = 0 cpuQNum1 = 0 cpuQNum2 = 0 npuIndex = 0 stripSeg = 0 copySeg = 0 ASIC#1: Destination index = 0x5012 DI_RCP_PORT1 <-- Recirculation port for VXLAN imposition pmap = 0x00000000 0x00000000 cmi = 0x0 rcp_pmap = 0x0 al_rsc_cmi CPU Map Index (CMI) [0] ctiLo0 = 0 ctiLo1 = 0 ctiLo2 = 0 cpuQNum0 = 0 cpuQNum1 = 0 cpuQNum2 = 0 npuIndex = 0 stripSeg = 0 copySeg = 0
siHandle Decode
Take the siHandle (0x7f65ecfdd048) and use in the command show platform hardware fed switch active fwd-asic abstraction print-resource-handle <siHandle> 1
Edge-1#show platform hardware fed switch active fwd-asic abstraction print-resource-handle 0x7f65ecfdd048 1 Handle:0x7f65ecfdd048 Res-Type:ASIC_RSC_RI Res-Switch-Num:255 Asic-Num:255 Feature-ID:AL_FID_L2_WIRELESS Lkp-ftr-id:LKP_FEAT_INVALID ref_count:1 priv_ri/priv_si Handle: 0x7f65ecfdcd78Hardware Indices/Handles: index0:0x12 mtu_index/l3u_ri_index0:0x0 index1:0x12 mtu_index/l3u_ri_index1:0x0 Features sharing this resource:58 (1)] Cookie length: 56 00 00 00 00 00 00 00 00 02 04 00 00 00 00 00 00 00 00 00 00 07 00 61 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 Detailed Resource Information (ASIC_INSTANCE# 0) ---------------------------------------- ASIC#:0 RI:18 Rewrite_type:AL_RRM_REWRITE_L2_PAYLOAD_L2LISP_ENCAP(115) Mapped_rii:LVX_L2_ENCAP_L2_PAYLOAD_ROUTED(133) Src IP: 10.47.1.12 <-- Local RLOC Dst IP: 10.47.1.13 <-- Remote RLOC iVxlan dstMac: 0x610:0x00:0x00 iVxlan srcMac: 0x00:0x00:0x00 IPv4 TTL: 0 iid present: 1 lisp iid: 0 lisp flags: 0 dst Port: 4789 update only l3if: 0 is Sgt: 1 is TTL Prop: 0 L3if LE: 0 (0) Port LE: 279 (0) Vlan LE: 7 (0) Detailed Resource Information (ASIC_INSTANCE# 1) ---------------------------------------- ASIC#:1 RI:18 Rewrite_type:AL_RRM_REWRITE_L2_PAYLOAD_L2LISP_ENCAP(115) Mapped_rii:LVX_L2_ENCAP_L2_PAYLOAD_ROUTED(133)
Src IP: 10.47.1.12 <-- Local RLOC
Dst IP: 10.47.1.13 <-- Remote RLOC iVxlan dstMac: 0x610:0x00:0x00 iVxlan srcMac: 0x00:0x00:0x00 IPv4 TTL: 0 iid present: 1 lisp iid: 0 lisp flags: 0 dst Port: 4789 update only l3if: 0 is Sgt: 1 is TTL Prop: 0 L3if LE: 0 (0) Port LE: 279 (0) Vlan LE: 7 (0) ==============================================================
Underlay Route Verification
Traffic is encapsulated in VXLAN with IID 8190 using 10.47.1.12 and has the ability to be load-balanced out Gig1/0/1 and G1/0/2
Edge-1#show ip route 10.47.1.13
Routing entry for 10.47.1.13/32
Known via "isis", distance 115, metric 30, type level-2
Redistributing via isis
Last update from 10.47.1.4 on GigabitEthernet1/0/2, 2d22h ago
Routing Descriptor Blocks:
10.47.1.4, from 10.47.1.13, 2d22h ago, via GigabitEthernet1/0/2
Route metric is 30, traffic share count is 1
* 10.47.1.0, from 10.47.1.13, 2d22h ago, via GigabitEthernet1/0/1
Route metric is 30, traffic share count is 1
Edge-1#show ip cef 10.47.1.13
10.47.1.13/32
nexthop 10.47.1.0 GigabitEthernet1/0/1
nexthop 10.47.1.4 GigabitEthernet1/0/2To get si_hdl, ri_hdl information, use the command show platform software fed switch active ip adj
Edge-1#show platform software fed switch active ip adj IPV4 Adj entries dest if_name dst_mac si_hdl ri_hdl pd_flags adj_id Last-modified ---- ------- ------- ------ ------ -------- ------ ------------- 225.0.0.0 GigabitEthernet1/0/1 0100.5e00.0000 0x7f65ec958128 0x7f65ec957e18 0x0 0xf80001a1 2023/09/19 17:57:41.399 10.47.1.10 LISP0.4100 4500.0000.0000 0x7f65ec895ed8 0x7f65ec895a68 0x60 0x5c 2023/09/19 17:58:16.545 225.0.0.0 GigabitEthernet1/0/2 0100.5e00.0000 0x7f65ec958f68 0x7f65ec95b2b8 0x0 0xf80001b1 2023/09/19 17:57:41.938 10.47.1.4 GigabitEthernet1/0/2 5254.001c.7de0 0x7f65ec8a5458 0x7f65ec8a4eb8 0x0 0x4c 2023/09/19 17:58:02.150 225.0.0.0 Null0 f800.0011.0000 0x7f65ec3740c8 0x0 0x0 0xf8000011 2023/09/19 17:57:29.404 10.47.1.0 GigabitEthernet1/0/1 5254.000a.42f3 0x7f65ec8b8468 0x7f65ec8b8158 0x0 0x55 2023/09/19 17:58:08.864
Underlay Next-Hop si_hdl Decode
To check the si_hdl (0x7f65ec8a5458) use in the command show platform hardware fed switch active fwd-asic abstraction print-resource-handle <si_hdl> 1
Edge-1#show platform hardware fed switch active fwd-asic abstraction print-resource-handle 0x7f65ec8a5458 1 Handle:0x7f65ec8a5458 Res-Type:ASIC_RSC_SI Res-Switch-Num:255 Asic-Num:255 Feature-ID:AL_FID_L3_UNICAST_IPV4 Lkp-ftr-id:LKP_FEAT_INVALID ref_count:1 priv_ri/priv_si Handle: 0x7f65ec8a4eb8Hardware Indices/Handles: index0:0xbc mtu_index/l3u_ri_index0:0x0 index1:0xbc mtu_index/l3u_ri_index1:0x0 Features sharing this resource:66 (1)] Cookie length: 56 00 00 00 00 00 00 00 00 26 00 00 00 00 00 00 00 00 00 00 00 08 00 52 54 00 1c 7d e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 Detailed Resource Information (ASIC_INSTANCE# 0) ---------------------------------------- Station Index (SI) [0xbc] -----> Contains RI and DI information RI = 0x1a -----> Rewrite Index = MAC address rewrite information for L3 forwarding to the next-hop adjacency DI = 0x526d -----> Destination Index = Outgoing Interface stationTableGenericLabel = 0 stationFdConstructionLabel = 0x7 lookupSkipIdIndex = 0 rcpServiceId = 0 dejaVuPreCheckEn = 0 Replication Bitmap: LD -----> Local Data, indicating that this ASIC is directly connected to the adjacency interface Detailed Resource Information (ASIC_INSTANCE# 1) ---------------------------------------- Station Index (SI) [0xbc] -----> Contains RI and DI information RI = 0x1a -----> Rewrite Index = MAC address rewrite information for L3 forwarding to the next-hop adjacency DI = 0x526d -----> Destination Index = Outgoing Interface stationTableGenericLabel = 0 stationFdConstructionLabel = 0x7 lookupSkipIdIndex = 0 rcpServiceId = 0 dejaVuPreCheckEn = 0 Replication Bitmap: CD -----> Core Data, indicating that this instance of the ASIC is on the same ASIC, but different core. ==============================================================
Underlay Next-Hop Rewrite Index Decode
To decode the RI (0x1a) use in the command show platform hardware fed switch active fwd-asic resource asic all rewrite-index range <RI> <RI>
Edge-1#show platform hardware fed switch active fwd-asic resource asic all rewrite-index range 0x1a 0x1a ASIC#:0 RI:26 Rewrite_type:AL_RRM_REWRITE_L3_UNICAST_IPV4_SHARED(1) Mapped_rii:L3_UNICAST_IPV4(9) -----> Decimal 26 is hex 0x1a MAC Addr: MAC Addr: 52:54:00:1c:7d:e0, -----> MAC address 5254.001c.7de0 for the next-hop adjacency L3IF LE Index 38 ASIC#:1 RI:26 Rewrite_type:AL_RRM_REWRITE_L3_UNICAST_IPV4_SHARED(1) Mapped_rii:L3_UNICAST_IPV4(9) MAC Addr: MAC Addr: 52:54:00:1c:7d:e0, -----> MAC address 5254.001c.7de0 for the next-hop adjacency L3IF LE Index 38
Underlay Next-Hop Destination Index Decode
To decode the DI (0x526d) use in show platform hardware fed switch active fwd-asic resource asic all destination-index range <DI> <DI>
Edge-1#show platform hardware fed switch active fwd-asic resource asic all destination-index range 0x526d 0x526d ASIC#0: Destination index = 0x526d pmap = 0x00000000 0x00000002 <-- Convert decimal 2 to binary, which is 0010. Count this binary right to left, zero-based, so Port 1. pmap_intf : [GigabitEthernet1/0/2] cmi = 0x0 rcp_pmap = 0x0 al_rsc_cmi CPU Map Index (CMI) [0] ctiLo0 = 0 ctiLo1 = 0 ctiLo2 = 0 cpuQNum0 = 0 cpuQNum1 = 0 cpuQNum2 = 0 npuIndex = 0 stripSeg = 0 copySeg = 0 ASIC#1: Destination index = 0x526d pmap = 0x00000000 0x00000000 cmi = 0x0 rcp_pmap = 0x0 al_rsc_cmi CPU Map Index (CMI) [0] ctiLo0 = 0 ctiLo1 = 0 ctiLo2 = 0 cpuQNum0 = 0 cpuQNum1 = 0 cpuQNum2 = 0 npuIndex = 0 stripSeg = 0 copySeg = 0
Edge-1#show platform software fed switch active ifm mappings Interface IF_ID Inst Asic Core Port SubPort Mac Cntx LPN GPN Type Active GigabitEthernet1/0/1 0x1a 0 0 0 0 0 1 0 1 1 NIF Y GigabitEthernet1/0/2 0x1b 0 0 0 1 0 2 1 2 2 NIF Y <-- Port 1 lines up to G1/0/2 GigabitEthernet1/0/3 0xb 0 0 0 2 0 3 2 3 3 NIF Y GigabitEthernet1/0/4 0xc 0 0 0 3 0 4 3 4 4 NIF Y GigabitEthernet1/0/5 0xd 0 0 0 4 0 5 4 5 5 NIF Y GigabitEthernet1/0/6 0xe 0 0 0 5 0 6 5 6 6 NIF Y GigabitEthernet1/0/7 0xf 0 0 0 6 0 7 6 7 7 NIF Y GigabitEthernet1/0/8 0x10 0 0 0 7 0 8 7 8 8 NIF Y
Underlay Next-Hop ri_hdl Decode
To decode the ri_hdl (0x7f65ec8a4eb8) use in show platform hardware fed switch active fwd-asic abstraction print-resource-handle (ri_hdl) 1
Edge-1#show platform hardware fed switch active fwd-asic abstraction print-resource-handle 0x7f65ec8a4eb8 1 Handle:0x7f65ec8a4eb8 Res-Type:ASIC_RSC_RI Res-Switch-Num:255 Asic-Num:255 Feature-ID:AL_FID_L3_UNICAST_IPV4 Lkp-ftr-id:LKP_FEAT_INVALID ref_count:1 priv_ri/priv_si Handle: 0x7f65ec903b28Hardware Indices/Handles: index0:0x1a mtu_index/l3u_ri_index0:0x0 index1:0x1a mtu_index/l3u_ri_index1:0x0 Features sharing this resource:66 (1)] Cookie length: 56 00 00 00 00 00 00 00 00 26 00 00 00 00 00 00 00 00 00 00 00 08 00 52 54 00 1c 7d e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 Detailed Resource Information (ASIC_INSTANCE# 0) ---------------------------------------- ASIC#:0 RI:26 Rewrite_type:AL_RRM_REWRITE_L3_UNICAST_IPV4_SHARED(1) Mapped_rii:L3_UNICAST_IPV4(9) <-- Decimal 26 is 0x1a in hex MAC Addr: MAC Addr: 52:54:00:1c:7d:e0, <-- MAC address 5254.001c.7de0 for the next-hop adjacency L3IF LE Index 38 Detailed Resource Information (ASIC_INSTANCE# 1) ---------------------------------------- ASIC#:1 RI:26 Rewrite_type:AL_RRM_REWRITE_L3_UNICAST_IPV4_SHARED(1) Mapped_rii:L3_UNICAST_IPV4(9)<-- Decimal 26 is 0x1a in hex MAC Addr: MAC Addr: 52:54:00:1c:7d:e0, MAC Addr: MAC Addr: 52:54:00:1c:7d:e0,<-- MAC address 5254.001c.7de0 for the next-hop adjacency
L3IF LE Index 38 ==============================================================
Basic Host Reachability in SDA Fabric (Different VLANs / Same VN)
In this section, communication between 10.47.4.2 and 10.47.10.2 is examined. As these hosts belong to different VLANs, both need to have default-gateway configured that points to default gateway. For 10.47.4.2 it is 10.47.4.1 and 10.47.10.2 it is 10.47.10.1.
Step 1. Confirm that connectivity between the endpoint and the default gateway works:
Edge-1#ping vrf red_vn 10.47.4.2 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.47.4.2, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 155/164/181 ms
Edge-2#ping vrf red_vn 10.47.10.1 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.47.10.1, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 41/46/62 ms
Step 2. Confirm that packet from 10.47.4.2 is received successfully by Edge-1:
Packet can be captured on the ingress interface facing 10.47.4.2:
Edge-1#monitor capture 1 interface g1/0/3 in match any Edge-1#mon cap 1 start Started capture point : 1 Edge-1#mon cap 1 stop Capture statistics collected at software: Capture duration - 12 seconds Packets received - 9 Packets dropped - 0 Packets oversized - 0 Number of Bytes dropped at asic not collected Capture buffer will exists till exported or cleared Stopped capture point : 1
Edge-1#show monitor capture 1 buffer brief Starting the packet display ........ Press Ctrl + Shift + 6 to exit 1 0.000000 10.47.4.2 -> 10.47.10.2 ICMP 98 Echo (ping) request id=0x0041, seq=0/0, ttl=64 2 0.023447 10.47.4.2 -> 10.47.10.2 ICMP 98 Echo (ping) request id=0x0041, seq=0/0, ttl=64
Edge-1#show monitor capture 1 buffer detailed Starting the packet display ........ Press Ctrl + Shift + 6 to exit Frame 1: 98 bytes on wire (784 bits), 98 bytes captured (784 bits) on interface /tmp/epc_ws/wif_to_ts_pipe, id 0 Interface id: 0 (/tmp/epc_ws/wif_to_ts_pipe) Interface name: /tmp/epc_ws/wif_to_ts_pipe Encapsulation type: Ethernet (1) Arrival Time: Oct 11, 2023 15:27:46.033825000 UTC [Time shift for this packet: 0.000000000 seconds] Epoch Time: 1697038066.033825000 seconds [Time delta from previous captured frame: 0.000000000 seconds] [Time delta from previous displayed frame: 0.000000000 seconds] [Time since reference or first frame: 0.000000000 seconds] Frame Number: 1 Frame Length: 98 bytes (784 bits) Capture Length: 98 bytes (784 bits) [Frame is marked: False] [Frame is ignored: False] [Protocols in frame: eth:ethertype:ip:icmp:data] Ethernet II, Src: 52:54:00:19:93:e9 (52:54:00:19:93:e9), Dst: 00:00:0c:9f:f3:41 (00:00:0c:9f:f3:41) <-- SMAC and DMAC respectively Destination: 00:00:0c:9f:f3:41 (00:00:0c:9f:f3:41) Address: 00:00:0c:9f:f3:41 (00:00:0c:9f:f3:41) .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default) .... ...0 .... .... .... .... = IG bit: Individual address (unicast) Source: 52:54:00:19:93:e9 (52:54:00:19:93:e9) Address: 52:54:00:19:93:e9 (52:54:00:19:93:e9) .... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default) .... ...0 .... .... .... .... = IG bit: Individual address (unicast) Type: IPv4 (0x0800) Internet Protocol Version 4, Src: 10.47.4.2, Dst: 10.47.10.2 0100 .... = Version: 4 .... 0101 = Header Length: 20 bytes (5) Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT) 0000 00.. = Differentiated Services Codepoint: Default (0) .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0) Total Length: 84 Identification: 0x395e (14686) Flags: 0x4000, Don't fragment 0... .... .... .... = Reserved bit: Not set .1.. .... .... .... = Don't fragment: Set ..0. .... .... .... = More fragments: Not set Fragment offset: 0 Time to live: 64 Protocol: ICMP (1) Header checksum: 0xdee9 [validation disabled] [Header checksum status: Unverified] Source: 10.47.4.2 Destination: 10.47.10.2 Internet Control Message Protocol Type: 8 (Echo (ping) request) Code: 0 Checksum: 0x248a [correct] [Checksum Status: Good] Identifier (BE): 65 (0x0041) Identifier (LE): 16640 (0x4100) Sequence number (BE): 0 (0x0000) Sequence number (LE): 0 (0x0000) Data (56 bytes) 0000 2a 46 a8 ee 00 00 00 00 00 00 00 00 00 00 00 00 *F.............. 0010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 0020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 0030 00 00 00 00 00 00 00 00 ........ Data: 2a46a8ee0000000000000000000000000000000000000000b^@& [Length: 56]
Step 3 - LISP Lookup
Ingress Edge Node must determine the location (RLOC) of HOST-03 to which it sends packets to. As in this case the end-host HOST-03 is in different VLAN (but same VN / VRF: USERS), LISP IPv4 instance is used as the lookup is based on IP address (MAC address belong to Edge Node itself).
Edge-1#debug lisp control-plane all
LISP[REMT ]-0: Map Request: Sending request for IID 4099 EID 10.47.10.2/32, requester 'remote EID prefix'. LISP[REMT ]-0: Map-Reply nonce matches pending request for IID 4099 EID 10.47.10.2/32, requester 'remote EID prefix'.
LISP Map Request reaches Control Node (LISP Map Server) Border-1:
Border-1#debug lisp control-plane all
LISP[TRNSP]-0: Processing received Map-Request(1) message on GigabitEthernet1/0/3 from 10.47.10.2:4342 to 10.47.10.2:4342.
LISP[MR ]-0: Received Map-Request with 1 records, first EID IID 4099 10.47.10.2/32, source EID 10.47.4.2, nonce 0x1CFAB931-0x8207CB93. LISP[MR ]-0 IID 4099 IPv4: MS EID 10.47.10.2/32: Sending proxy reply to 10.47.1.12.
LISP Map-Reply reaches Edge Node:
LISP[REMT ]-0: Processing Map-Reply mapping record for IID 4099 IPv4 10.47.10.2/32 LCAF 2, ttl 1440, action none, not authoritative, 1 locator. LISP[REMT ]-0: Processing mapping information for EID prefix IID 4099 10.47.10.2/32.
The Fabric Edge queries for the RLOC for 10.47.10.2 and process the Map-Reply
LISP[REMT ]-0: Map Request: Sending request for IID 4099 EID 10.47.10.2/32, requester 'remote EID RLOC'.
LISP[REMT ]-0: Processing Map-Reply mapping record for IID 4099 IPv4 10.47.10.2/32 LCAF 2, ttl 1440, action none, authoritative, 1 locator. LISP[REMT ]-0: Processing mapping information for EID prefix IID 4099 10.47.10.2/32.
In case entry does not exist, debugs needs to be collected from LISP process perspective. There is also a tool, called LIG (LISP Grouper) which can be used to trigger manually LISP process (this is very effective way to test redundant Control Node configuration and database consistency between both Control Nodes):
Edge-1#lig instance-id 4099 10.47.10.2 to 10.47.1.10 Mapping information for EID 10.47.10.2 from 10.47.1.10 with RTT 334 msecs 10.47.10.2/32, uptime: 00:00:00, expires: 23:59:59, via map-reply, complete Locator Uptime State Pri/Wgt Encap-IID 10.47.1.13 00:00:00 up 10/10 -
Edge-1#lig instance-id 4099 10.47.10.2 to 10.47.1.11 Mapping information for EID 10.47.10.2 from 10.47.1.11 with RTT 327 msecs 10.47.10.2/32, uptime: 00:00:06, expires: 23:59:59, via map-reply, complete Locator Uptime State Pri/Wgt Encap-IID 10.47.1.13 00:00:06 up 10/10 -
Route Verification
CEF uses LISP, and LISP utilizes the map-cache entry it has received
Edge-1#show ip cef vrf red_vn 10.47.10.2 10.47.10.2/32 nexthop 10.47.1.13 LISP0.4099
Edge-1#show ip route 10.47.1.13 Routing entry for 10.47.1.13/32 Known via "isis", distance 115, metric 30, type level-2 Redistributing via isis Last update from 10.47.1.4 on GigabitEthernet1/0/2, 3d19h ago Routing Descriptor Blocks: 10.47.1.4, from 10.47.1.13, 3d19h ago, via GigabitEthernet1/0/2 Route metric is 30, traffic share count is 1 * 10.47.1.0, from 10.47.1.13, 3d19h ago, via GigabitEthernet1/0/1 Route metric is 30, traffic share count is 1
Edge-1#show lisp instance-id 4099 ipv4 map-cache 10.47.10.2 LISP IPv4 Mapping Cache for LISP 0 EID-table vrf red_vn (IID 4099), 1 entries 10.47.10.2/32, uptime: 00:08:48, expires: 23:51:17, via map-reply, complete Sources: map-reply State: complete, last modified: 00:08:48, map-source: 10.47.1.11 Active, Packets out: 51(29376 bytes), counters are not accurate (~ 00:00:15 ago) Encapsulating dynamic-EID traffic Locator Uptime State Pri/Wgt Encap-IID 10.47.1.13 00:08:48 up 10/10 - Last up-down state change: 00:08:48, state change count: 1 Last route reachability change: 22:07:12, state change count: 1 Last priority / weight change: never/never RLOC-probing loc-status algorithm: Last RLOC-probe sent: 00:08:48 (rtt 931ms)
LISP Next-Hop Verification
Since this packet is VXLAN encapsulated, verification of the LISP next-hop needs to occur. Use the command show platform software fed switch active ip adj to get additional information about 10.47.1.13, the LISP next-hop
Edge-1#show platform software fed switch active ip adj IPV4 Adj entries dest if_name dst_mac si_hdl ri_hdl pd_flags adj_id Last-modified ---- ------- ------- ------ ------ -------- ------ ------------- 10.47.1.10 LISP0.4100 4500.0000.0000 0x7f65ec895ed8 0x7f65ec895a68 0x60 0x5c 2023/09/19 17:58:16.545 10.47.1.4 GigabitEthernet1/0/2 5254.001c.7de0 0x7f65ec8a5458 0x7f65ec8a4eb8 0x0 0x4c 2023/09/19 17:58:02.150 10.47.1.0 GigabitEthernet1/0/1 5254.000a.42f3 0x7f65ec8b8468 0x7f65ec8b8158 0x0 0x55 2023/09/19 17:58:08.864 10.47.4.2 Vlan1026 5254.0019.93e9 0x7f65ec7c21f8 0x7f65ec7c2498 0x0 0x1a 2023/09/19 23:59:34.081 10.47.1.13 LISP0.4099 4500.0000.0000 0x7f65ed00f668 0x7f65ed00fd58 0x60 0x20 2023/10/11 15:36:06.243
LISP Next-Hop si_hdl Decode
Take the si_hdl (0x7f65ed00f668) and use in show platform hardware fed switch active fwd-asic abstraction print-resource-handle <si_hdl> 1
Edge-1#show platform hardware fed switch active fwd-asic abstraction print-resource-handle 0x7f65ed00f668 1 Handle:0x7f65ed00f668 Res-Type:ASIC_RSC_SI Res-Switch-Num:255 Asic-Num:255 Feature-ID:AL_FID_LISP Lkp-ftr-id:LKP_FEAT_INVALID ref_count:1 priv_ri/priv_si Handle: 0x7f65ed00fd58Hardware Indices/Handles: index0:0xc8 mtu_index/l3u_ri_index0:0x0 index1:0xc8 mtu_index/l3u_ri_index1:0x0 Features sharing this resource:109 (1)] Cookie length: 56 00 00 00 00 00 00 00 00 38 5f 84 ec 0a 2f 01 0d ff ff ff ff 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 Detailed Resource Information (ASIC_INSTANCE# 0) ---------------------------------------- Station Index (SI) [0xc8] <-- Contains the RI and DI RI = 0x2c <-- Rewrite Index contains information for L3 Forwarding DI = 0x5012 <-- Destination Index contains information for the destination port stationTableGenericLabel = 0 stationFdConstructionLabel = 0x7 lookupSkipIdIndex = 0xc rcpServiceId = 0 dejaVuPreCheckEn = 0 Replication Bitmap: LD Detailed Resource Information (ASIC_INSTANCE# 1) ----------------------------------------
Station Index (SI) [0xc8] <-- Contains the RI and DI
RI = 0x2c <-- Rewrite Index contains information for L3 Forwarding
DI = 0x5013 <-- Destination Index contains information for the destination port stationTableGenericLabel = 0 stationFdConstructionLabel = 0x7 lookupSkipIdIndex = 0xc rcpServiceId = 0 dejaVuPreCheckEn = 0 Replication Bitmap: LD ==============================================================
LISP Next-Hop RI Decode
Take the RI (0x2c) and use in show platform hardware fed switch active fwd-asic resource asic all rewrite-index range <RI> <RI>
Edge-1#show platform hardware fed switch active fwd-asic resource asic all rewrite-index range 0x2c 0x2c ASIC#:0 RI:44 Rewrite_type:AL_RRM_REWRITE_IPV4_VXLAN_INNER_IPV4_ENCAP(110) Mapped_rii:LVX_L3_ENCAP_L2_PAYLOAD(123) Dst Mac: MAC Addr: ba:25:cd:f4:ad:38, Src IP: 10.47.1.12 <-- Local RLOC Dst IP: 10.47.1.13 <-- RLOC of Edge-2 IPv4 TTL: 0 LISP INSTANCEID: 0 L3IF LE Index: 46 ASIC#:1 RI:44 Rewrite_type:AL_RRM_REWRITE_IPV4_VXLAN_INNER_IPV4_ENCAP(110) Mapped_rii:LVX_L3_ENCAP_L2_PAYLOAD(123) Dst Mac: MAC Addr: ba:25:cd:f4:ad:38,
Src IP: 10.47.1.12 <-- Local RLOC
Dst IP: 10.47.1.13 <-- RLOC of Edge-2 IPv4 TTL: 0 LISP INSTANCEID: 0 L3IF LE Index: 46
LISP Next-Hop DI Decode
Take the DI (0x5012) and use in show platform hardware fed switch active fwd-asic resource asic all destination-index range <DI> <DI>
Edge-1#show platform hardware fed switch active fwd-asic resource asic all destination-index range 0x5012 0x5012
ASIC#0:
Destination index = 0x5012 DI_RCP_PORT1 <-- Expected, this means the packet is recirculated for VXLAN imposition
pmap = 0x00000000 0x00000000
cmi = 0x0
rcp_pmap = 0x1
al_rsc_cmi
CPU Map Index (CMI) [0]
ctiLo0 = 0
ctiLo1 = 0
ctiLo2 = 0
cpuQNum0 = 0
cpuQNum1 = 0
cpuQNum2 = 0
npuIndex = 0
stripSeg = 0
copySeg = 0
ASIC#1:
Destination index = 0x5012 DI_RCP_PORT1 <-- Expected, this means the packet is recirculated for VXLAN imposition
pmap = 0x00000000 0x00000000
cmi = 0x0
rcp_pmap = 0x0
al_rsc_cmi
CPU Map Index (CMI) [0]
ctiLo0 = 0
ctiLo1 = 0
ctiLo2 = 0
cpuQNum0 = 0
cpuQNum1 = 0
cpuQNum2 = 0
npuIndex = 0
stripSeg = 0
copySeg = 0LISP Next-Hop ri_hdl Decode
Take the ri_hdl (0x7f65ed00fd58) and use in the command show platform hardware fed switch active fwd-asic abstraction print-resource-handle <ri_hdl> 1
Edge-1#show platform hardware fed switch active fwd-asic abstraction print-resource-handle 0x7f65ed00fd58 1 Handle:0x7f65ed00fd58 Res-Type:ASIC_RSC_RI Res-Switch-Num:255 Asic-Num:255 Feature-ID:AL_FID_LISP Lkp-ftr-id:LKP_FEAT_INVALID ref_count:1 priv_ri/priv_si Handle: 0x7f65ed00b618Hardware Indices/Handles: index0:0x2c mtu_index/l3u_ri_index0:0x0 index1:0x2c mtu_index/l3u_ri_index1:0x0 Features sharing this resource:109 (1)] Cookie length: 56 00 00 00 00 00 00 00 00 2e 00 00 00 0a 2f 01 0d ff ff ff ff 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 Detailed Resource Information (ASIC_INSTANCE# 0) ---------------------------------------- ASIC#:0 RI:44 Rewrite_type:AL_RRM_REWRITE_IPV4_VXLAN_INNER_IPV4_ENCAP(110) Mapped_rii:LVX_L3_ENCAP_L2_PAYLOAD(123) Dst Mac: MAC Addr: ba:25:cd:f4:ad:38, Src IP: 10.47.1.12 <-- Local RLOC Dst IP: 10.47.1.13 <-- Edge-2 RLOC IPv4 TTL: 0 LISP INSTANCEID: 0 L3IF LE Index: 46 Detailed Resource Information (ASIC_INSTANCE# 1) ---------------------------------------- ASIC#:1 RI:44 Rewrite_type:AL_RRM_REWRITE_IPV4_VXLAN_INNER_IPV4_ENCAP(110) Mapped_rii:LVX_L3_ENCAP_L2_PAYLOAD(123) Dst Mac: MAC Addr: ba:25:cd:f4:ad:38,
Src IP: 10.47.1.12 <-- Local RLOC
Dst IP: 10.47.1.13 <-- Edge-2 RLOC IPv4 TTL: 0 LISP INSTANCEID: 0 L3IF LE Index: 46 ==============================================================
Underlay Next-Hop Verification
To get to the LISP Next-Hop, there are two possible paths in the underlay, verification occurs for one, the same logic applies in verification of the other underlay next-hop.
Edge-1#show ip route 10.47.1.13
Routing entry for 10.47.1.13/32
Known via "isis", distance 115, metric 30, type level-2
Redistributing via isis
Last update from 10.47.1.4 on GigabitEthernet1/0/2, 3d19h ago
Routing Descriptor Blocks:
10.47.1.4, from 10.47.1.13, 3d19h ago, via GigabitEthernet1/0/2
Route metric is 30, traffic share count is 1
* 10.47.1.0, from 10.47.1.13, 3d19h ago, via GigabitEthernet1/0/1
Route metric is 30, traffic share count is 1To get more information about the next-hops, use show platform software fed switch active ip adj
Edge-1#show platform software fed switch active ip adj IPV4 Adj entries dest if_name dst_mac si_hdl ri_hdl pd_flags adj_id Last-modified ---- ------- ------- ------ ------ -------- ------ ------------- 10.47.1.4 GigabitEthernet1/0/2 5254.001c.7de0 0x7f65ec8a5458 0x7f65ec8a4eb8 0x0 0x4c 2023/09/19 17:58:02.150 10.47.1.0 GigabitEthernet1/0/1 5254.000a.42f3 0x7f65ec8b8468 0x7f65ec8b8158 0x0 0x55 2023/09/19 17:58:08.864 <snip>
Underlay Next-Hop si_hdl Decode
Take the si_hdl (0x7f65ec8a5458) and use in the command show platform hardware fed switch active fwd-asic abstraction print-resource-handle <si_hdl> 1
Edge-1#show platform hardware fed switch active fwd-asic abstraction print-resource-handle 0x7f65ec8a5458 1 Handle:0x7f65ec8a5458 Res-Type:ASIC_RSC_SI Res-Switch-Num:255 Asic-Num:255 Feature-ID:AL_FID_L3_UNICAST_IPV4 Lkp-ftr-id:LKP_FEAT_INVALID ref_count:1 priv_ri/priv_si Handle: 0x7f65ec8a4eb8Hardware Indices/Handles: index0:0xbc mtu_index/l3u_ri_index0:0x0 index1:0xbc mtu_index/l3u_ri_index1:0x0 Features sharing this resource:66 (1)] Cookie length: 56 00 00 00 00 00 00 00 00 26 00 00 00 00 00 00 00 00 00 00 00 08 00 52 54 00 1c 7d e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 Detailed Resource Information (ASIC_INSTANCE# 0) ---------------------------------------- Station Index (SI) [0xbc] <-- Contains the RI and DI RI = 0x1a <-- Rewrite index contains information for L3 Forwarding DI = 0x526d <-- Destination index contains information for the destination port stationTableGenericLabel = 0 stationFdConstructionLabel = 0x7 lookupSkipIdIndex = 0 rcpServiceId = 0 dejaVuPreCheckEn = 0 Replication Bitmap: LD Detailed Resource Information (ASIC_INSTANCE# 1) ----------------------------------------
Station Index (SI) [0xbc] <-- Contains the RI and DI
RI = 0x1a <-- Rewrite index contains information for L3 Forwarding
DI = 0x526d <-- Destination index contains information for the destination port stationTableGenericLabel = 0 stationFdConstructionLabel = 0x7 lookupSkipIdIndex = 0 rcpServiceId = 0 dejaVuPreCheckEn = 0 Replication Bitmap: CD ==============================================================
Underlay Next-Hop RI Decode
Take the RI (0x1a) and use in the command show platform hardware fed switch active fwd-asic resource asic all rewrite-index range <RI> <RI>
Edge-1#show platform hardware fed switch active fwd-asic resource asic all rewrite-index range 0x1a 0x1a ASIC#:0 RI:26 Rewrite_type:AL_RRM_REWRITE_L3_UNICAST_IPV4_SHARED(1) Mapped_rii:L3_UNICAST_IPV4(9) <-- Decimal 26 is hex 0x1a MAC Addr: MAC Addr: 52:54:00:1c:7d:e0, <-- MAC Address 5254.001c.7de0 corresponds to the next-hop L3IF LE Index 38 ASIC#:1 RI:26 Rewrite_type:AL_RRM_REWRITE_L3_UNICAST_IPV4_SHARED(1) Mapped_rii:L3_UNICAST_IPV4(9)<-- Decimal 26 is hex 0x1a MAC Addr: MAC Addr: 52:54:00:1c:7d:e0, <-- MAC Address 5254.001c.7de0 corresponds to the next-hop L3IF LE Index 38
Underlay Next Hop DI Decode
Take the DI (0x526d) and use in the command show platform hardware fed switch active fwd-asic resource asic all destination-index range <DI> <DI>
Edge-1#show platform hardware fed switch active fwd-asic resource asic all destination-index range 0x526d 0x526d ASIC#0: Destination index = 0x526d pmap = 0x00000000 0x00000002 <-- Take decimal 2 and convert to binary, so 0010, and then count this binary right to left zero-based, so Port 1 pmap_intf : [GigabitEthernet1/0/2] cmi = 0x0 rcp_pmap = 0x0 al_rsc_cmi CPU Map Index (CMI) [0] ctiLo0 = 0 ctiLo1 = 0 ctiLo2 = 0 cpuQNum0 = 0 cpuQNum1 = 0 cpuQNum2 = 0 npuIndex = 0 stripSeg = 0 copySeg = 0 ASIC#1: Destination index = 0x526d pmap = 0x00000000 0x00000000 cmi = 0x0 rcp_pmap = 0x0 al_rsc_cmi CPU Map Index (CMI) [0] ctiLo0 = 0 ctiLo1 = 0 ctiLo2 = 0 cpuQNum0 = 0 cpuQNum1 = 0 cpuQNum2 = 0 npuIndex = 0 stripSeg = 0 copySeg = 0
Edge-1#show platform software fed switch active ifm mappings Interface IF_ID Inst Asic Core Port SubPort Mac Cntx LPN GPN Type Active GigabitEthernet1/0/1 0x1a 0 0 0 0 0 1 0 1 1 NIF Y GigabitEthernet1/0/2 0x1b 0 0 0 1 0 2 1 2 2 NIF Y <-- Port 1 maps to Gig1/0/2 GigabitEthernet1/0/3 0xb 0 0 0 2 0 3 2 3 3 NIF Y GigabitEthernet1/0/4 0xc 0 0 0 3 0 4 3 4 4 NIF Y GigabitEthernet1/0/5 0xd 0 0 0 4 0 5 4 5 5 NIF Y GigabitEthernet1/0/6 0xe 0 0 0 5 0 6 5 6 6 NIF Y GigabitEthernet1/0/7 0xf 0 0 0 6 0 7 6 7 7 NIF Y GigabitEthernet1/0/8 0x10 0 0 0 7 0 8 7 8 8 NIF Y
Underlay Next-Hop ri_hdl Decode
Take the ri_hdl (0x7f65ec8b8158) and use in the command show platform hardware fed switch active fwd-asic abstraction print-resource-handle <ri_hdl> 1
Edge-1#show platform hardware fed switch active fwd-asic abstraction print-resource-handle 0x7f65ec8b8158 1 Handle:0x7f65ec8b8158 Res-Type:ASIC_RSC_RI Res-Switch-Num:255 Asic-Num:255 Feature-ID:AL_FID_L3_UNICAST_IPV4 Lkp-ftr-id:LKP_FEAT_INVALID ref_count:1 priv_ri/priv_si Handle: 0x7f65ec7a6338Hardware Indices/Handles: index0:0x1b mtu_index/l3u_ri_index0:0x0 index1:0x1b mtu_index/l3u_ri_index1:0x0 Features sharing this resource:66 (1)] Cookie length: 56 00 00 00 00 00 00 00 00 25 00 00 00 00 00 00 00 00 00 00 00 08 00 52 54 00 0a 42 f3 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 Detailed Resource Information (ASIC_INSTANCE# 0) ---------------------------------------- ASIC#:0 RI:27 Rewrite_type:AL_RRM_REWRITE_L3_UNICAST_IPV4_SHARED(1) Mapped_rii:L3_UNICAST_IPV4(9) MAC Addr: MAC Addr: 52:54:00:0a:42:f3, L3IF LE Index 37 Detailed Resource Information (ASIC_INSTANCE# 1) ---------------------------------------- ASIC#:1 RI:27 Rewrite_type:AL_RRM_REWRITE_L3_UNICAST_IPV4_SHARED(1) Mapped_rii:L3_UNICAST_IPV4(9) MAC Addr: MAC Addr: 52:54:00:0a:42:f3, L3IF LE Index 37 ==============================================================
Packets are encapsulated in VXLAN and sent based on load-balancing rules. Embedded Packet Capture (EPC) can be used to capture traffic on all interfaces at the same time. Remember at this point the packet is VXLAN encapsulated, the EPC filter must be against RLOC to RLOC, not inner IPv4 addresses.
Edge-1#monitor capture 1 interface range g1/0/1-2 out match ipv4 host 10.47.1.12 host 10.47.1.13 Edge-1#monitor capture 1 start Started capture point : 1 Edge-1# Edge-1#monitor capture 1 stop Capture statistics collected at software: Capture duration - 18 seconds Packets received - 4 Packets dropped - 0 Packets oversized - 0 Number of Bytes dropped at asic not collected Capture buffer will exists till exported or cleared Stopped capture point : 1
Edge-1#show monitor capture 1 buffer brief Starting the packet display ........ Press Ctrl + Shift + 6 to exit 1 0.000000 10.47.4.2 -> 10.47.10.2 ICMP 148 Echo (ping) request id=0x0046, seq=0/0, ttl=63 2 0.980849 10.47.4.2 -> 10.47.10.2 ICMP 148 Echo (ping) request id=0x0046, seq=1/256, ttl=63 3 1.984077 10.47.4.2 -> 10.47.10.2 ICMP 148 Echo (ping) request id=0x0046, seq=2/512, ttl=63 4 2.999989 10.47.4.2 -> 10.47.10.2 ICMP 148 Echo (ping) request id=0x0046, seq=3/768, ttl=63
Edge-1#show monitor capture 1 buffer detailed Starting the packet display ........ Press Ctrl + Shift + 6 to exit Frame 1: 148 bytes on wire (1184 bits), 148 bytes captured (1184 bits) on interface /tmp/epc_ws/wif_to_ts_pipe, id 0 Interface id: 0 (/tmp/epc_ws/wif_to_ts_pipe) Interface name: /tmp/epc_ws/wif_to_ts_pipe Encapsulation type: Ethernet (1) Arrival Time: Oct 11, 2023 16:50:52.262553000 UTC [Time shift for this packet: 0.000000000 seconds] Epoch Time: 1697043052.262553000 seconds [Time delta from previous captured frame: 0.000000000 seconds] [Time delta from previous displayed frame: 0.000000000 seconds] [Time since reference or first frame: 0.000000000 seconds] Frame Number: 1 Frame Length: 148 bytes (1184 bits) Capture Length: 148 bytes (1184 bits) [Frame is marked: False] [Frame is ignored: False] [Protocols in frame: eth:ethertype:ip:udp:vxlan:eth:ethertype:ip:icmp:data] Ethernet II, Src: 00:00:00:00:00:00 (00:00:00:00:00:00), Dst: 00:00:00:00:00:00 (00:00:00:00:00:00) <-- EPC does not capture L3 rewrite on egress properly, this is OK Destination: 00:00:00:00:00:00 (00:00:00:00:00:00) Address: 00:00:00:00:00:00 (00:00:00:00:00:00) .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default) .... ...0 .... .... .... .... = IG bit: Individual address (unicast) Source: 00:00:00:00:00:00 (00:00:00:00:00:00) Address: 00:00:00:00:00:00 (00:00:00:00:00:00) .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default) .... ...0 .... .... .... .... = IG bit: Individual address (unicast) Type: IPv4 (0x0800) Internet Protocol Version 4, Src: 10.47.1.12, Dst: 10.47.1.13 <-- RLOC to RLOC 0100 .... = Version: 4 .... 0101 = Header Length: 20 bytes (5) Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT) 0000 00.. = Differentiated Services Codepoint: Default (0) .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0) Total Length: 134 Identification: 0x1d6f (7535) Flags: 0x4000, Don't fragment 0... .... .... .... = Reserved bit: Not set .1.. .... .... .... = Don't fragment: Set ..0. .... .... .... = More fragments: Not set Fragment offset: 0 Time to live: 64 Protocol: UDP (17) Header checksum: 0x0682 [validation disabled] [Header checksum status: Unverified] Source: 10.47.1.12 Destination: 10.47.1.13 User Datagram Protocol, Src Port: 65354, Dst Port: 4789 Source Port: 65354 Destination Port: 4789 Length: 114 [Checksum: [missing]] [Checksum Status: Not present] [Stream index: 0] [Timestamps] [Time since first frame: 0.000000000 seconds] [Time since previous frame: 0.000000000 seconds] Virtual eXtensible Local Area Network Flags: 0x8800, GBP Extension, VXLAN Network ID (VNI) 1... .... .... .... = GBP Extension: Defined .... .... .0.. .... = Don't Learn: False .... 1... .... .... = VXLAN Network ID (VNI): True .... .... .... 0... = Policy Applied: False .000 .000 0.00 .000 = Reserved(R): 0x0000 Group Policy ID: 0 VXLAN Network Identifier (VNI): 4099 <-- LISP L3 IID Reserved: 0 Ethernet II, Src: 00:00:00:00:61:00 (00:00:00:00:61:00), Dst: ba:25:cd:f4:ad:38 (ba:25:cd:f4:ad:38) <-- Dummy Ethernet header for VXLAN Destination: ba:25:cd:f4:ad:38 (ba:25:cd:f4:ad:38) Address: ba:25:cd:f4:ad:38 (ba:25:cd:f4:ad:38) .... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default) .... ...0 .... .... .... .... = IG bit: Individual address (unicast) Source: 00:00:00:00:61:00 (00:00:00:00:61:00) Address: 00:00:00:00:61:00 (00:00:00:00:61:00) .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default) .... ...0 .... .... .... .... = IG bit: Individual address (unicast) Type: IPv4 (0x0800) Internet Protocol Version 4, Src: 10.47.4.2, Dst: 10.47.10.2 <-- True IPv4 addresses 0100 .... = Version: 4 .... 0101 = Header Length: 20 bytes (5) Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT) 0000 00.. = Differentiated Services Codepoint: Default (0) .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0) Total Length: 84 Identification: 0x92f6 (37622) Flags: 0x4000, Don't fragment 0... .... .... .... = Reserved bit: Not set .1.. .... .... .... = Don't fragment: Set ..0. .... .... .... = More fragments: Not set Fragment offset: 0 Time to live: 63 Protocol: ICMP (1) Header checksum: 0x8651 [validation disabled] [Header checksum status: Unverified] Source: 10.47.4.2 Destination: 10.47.10.2 Internet Control Message Protocol Type: 8 (Echo (ping) request) Code: 0 Checksum: 0xa383 [correct] [Checksum Status: Good] Identifier (BE): 70 (0x0046) Identifier (LE): 17920 (0x4600) Sequence number (BE): 0 (0x0000) Sequence number (LE): 0 (0x0000) Data (56 bytes) 0000 78 1e dc 17 00 00 00 00 00 00 00 00 00 00 00 00 x............... 0010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 0020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 0030 00 00 00 00 00 00 00 00 ........ Data: 781edc170000000000000000000000000000000000000000b^@& [Length: 56]
Encapsulated VXLAN packet reaches Edge-2:
Edge-2#monitor capture 1 interface range g1/0/1-2 in match ipv4 host 10.47.1.12 host 10.47.1.13 Edge-2#monitor capture 1 start Started capture point : 1 Edge-2#monitor capture 1 stop Capture statistics collected at software: Capture duration - 7 seconds Packets received - 6 Packets dropped - 0 Packets oversized - 0 Number of Bytes dropped at asic not collected Capture buffer will exists till exported or cleared Stopped capture point : 1
Edge-2#show monitor capture 1 buffer brief Starting the packet display ........ Press Ctrl + Shift + 6 to exit 1 0.000000 10.47.4.2 -> 10.47.10.2 ICMP 148 Echo (ping) request id=0x0047, seq=0/0, ttl=63 2 0.007826 10.47.4.2 -> 10.47.10.2 ICMP 148 Echo (ping) request id=0x0047, seq=0/0, ttl=63 3 0.086345 10.47.4.2 -> 10.47.10.2 ICMP 148 Echo (ping) request id=0x0047, seq=1/256, ttl=63 4 0.097490 10.47.4.2 -> 10.47.10.2 ICMP 148 Echo (ping) request id=0x0047, seq=1/256, ttl=63 5 1.150969 10.47.4.2 -> 10.47.10.2 ICMP 148 Echo (ping) request id=0x0047, seq=2/512, ttl=63 6 1.163817 10.47.4.2 -> 10.47.10.2 ICMP 148 Echo (ping) request id=0x0047, seq=2/512, ttl=63
Edge-2#show monitor capture 1 buffer detailed Starting the packet display ........ Press Ctrl + Shift + 6 to exit Frame 1: 148 bytes on wire (1184 bits), 148 bytes captured (1184 bits) on interface /tmp/epc_ws/wif_to_ts_pipe, id 0 Interface id: 0 (/tmp/epc_ws/wif_to_ts_pipe) Interface name: /tmp/epc_ws/wif_to_ts_pipe Encapsulation type: Ethernet (1) Arrival Time: Oct 11, 2023 16:58:12.702159000 UTC [Time shift for this packet: 0.000000000 seconds] Epoch Time: 1697043492.702159000 seconds [Time delta from previous captured frame: 0.000000000 seconds] [Time delta from previous displayed frame: 0.000000000 seconds] [Time since reference or first frame: 0.000000000 seconds] Frame Number: 1 Frame Length: 148 bytes (1184 bits) Capture Length: 148 bytes (1184 bits) [Frame is marked: False] [Frame is ignored: False] [Protocols in frame: eth:ethertype:ip:udp:vxlan:eth:ethertype:ip:icmp:data] Ethernet II, Src: 52:54:00:0a:42:11 (52:54:00:0a:42:11), Dst: 52:54:00:17:fe:65 (52:54:00:17:fe:65) <-- True MAC addresses post L3 rewrite Destination: 52:54:00:17:fe:65 (52:54:00:17:fe:65) Address: 52:54:00:17:fe:65 (52:54:00:17:fe:65) .... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default) .... ...0 .... .... .... .... = IG bit: Individual address (unicast) Source: 52:54:00:0a:42:11 (52:54:00:0a:42:11) Address: 52:54:00:0a:42:11 (52:54:00:0a:42:11) .... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default) .... ...0 .... .... .... .... = IG bit: Individual address (unicast) Type: IPv4 (0x0800) Internet Protocol Version 4, Src: 10.47.1.12, Dst: 10.47.1.13 <-- RLOC to RLOC 0100 .... = Version: 4 .... 0101 = Header Length: 20 bytes (5) Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT) 0000 00.. = Differentiated Services Codepoint: Default (0) .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0) Total Length: 134 Identification: 0x1d7b (7547) Flags: 0x4000, Don't fragment 0... .... .... .... = Reserved bit: Not set .1.. .... .... .... = Don't fragment: Set ..0. .... .... .... = More fragments: Not set Fragment offset: 0 Time to live: 62 Protocol: UDP (17) Header checksum: 0x0876 [validation disabled] [Header checksum status: Unverified] Source: 10.47.1.12 Destination: 10.47.1.13 User Datagram Protocol, Src Port: 65354, Dst Port: 4789 Source Port: 65354 Destination Port: 4789 Length: 114 [Checksum: [missing]] [Checksum Status: Not present] [Stream index: 0] [Timestamps] [Time since first frame: 0.000000000 seconds] [Time since previous frame: 0.000000000 seconds] Virtual eXtensible Local Area Network Flags: 0x8800, GBP Extension, VXLAN Network ID (VNI) 1... .... .... .... = GBP Extension: Defined .... .... .0.. .... = Don't Learn: False .... 1... .... .... = VXLAN Network ID (VNI): True .... .... .... 0... = Policy Applied: False .000 .000 0.00 .000 = Reserved(R): 0x0000 Group Policy ID: 0 VXLAN Network Identifier (VNI): 4099 <-- LISP L3 IID Reserved: 0 Ethernet II, Src: 00:00:00:00:61:00 (00:00:00:00:61:00), Dst: ba:25:cd:f4:ad:38 (ba:25:cd:f4:ad:38) <-- Dummy Ethernet header for VXLAN Destination: ba:25:cd:f4:ad:38 (ba:25:cd:f4:ad:38) Address: ba:25:cd:f4:ad:38 (ba:25:cd:f4:ad:38) .... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default) .... ...0 .... .... .... .... = IG bit: Individual address (unicast) Source: 00:00:00:00:61:00 (00:00:00:00:61:00) Address: 00:00:00:00:61:00 (00:00:00:00:61:00) .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default) .... ...0 .... .... .... .... = IG bit: Individual address (unicast) Type: IPv4 (0x0800) Internet Protocol Version 4, Src: 10.47.4.2, Dst: 10.47.10.2 0100 .... = Version: 4 .... 0101 = Header Length: 20 bytes (5) Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT) 0000 00.. = Differentiated Services Codepoint: Default (0) .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0) Total Length: 84 Identification: 0x1abb (6843) Flags: 0x4000, Don't fragment 0... .... .... .... = Reserved bit: Not set .1.. .... .... .... = Don't fragment: Set ..0. .... .... .... = More fragments: Not set Fragment offset: 0 Time to live: 63 Protocol: ICMP (1) Header checksum: 0xfe8c [validation disabled] [Header checksum status: Unverified] Source: 10.47.4.2 Destination: 10.47.10.2 Internet Control Message Protocol Type: 8 (Echo (ping) request) Code: 0 Checksum: 0x044f [correct] [Checksum Status: Good] Identifier (BE): 71 (0x0047) Identifier (LE): 18176 (0x4700) Sequence number (BE): 0 (0x0000) Sequence number (LE): 0 (0x0000) Data (56 bytes) 0000 e8 37 0b 32 00 00 00 00 00 00 00 00 00 00 00 00 .7.2............ 0010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 0020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 0030 00 00 00 00 00 00 00 00 ........ Data: e8370b320000000000000000000000000000000000000000b^@& [Length: 56]
Edge-2 decapsulates the VXLAN header and consult its ARP table to forward the ICMP Request towards 10.47.10.2
Edge-2#show ip cef vrf red_vn 10.47.10.2 10.47.10.2/32 nexthop 10.47.10.2 Vlan1028
Edge-2#show platform software fed switch active ip adj IPV4 Adj entries dest if_name dst_mac si_hdl ri_hdl pd_flags adj_id Last-modified ---- ------- ------- ------ ------ -------- ------ ------------- 10.47.10.2 Vlan1028 5254.0002.cbf5 0x7f5744f89988 0x7f5744f8afa8 0x0 0x26 2023/10/09 18:57:59.026
<snip>
Endpoint si_hdl decode
Take the si_hdl (0x7f5744f89988) and use in show platform hardware fed switch active fwd-asic abstraction print-resource-handle <si_hdl> 1
Edge-2#show platform hardware fed switch active fwd-asic abstraction print-resource-handle 0x7f5744f89988 1 Handle:0x7f5744f89988 Res-Type:ASIC_RSC_SI Res-Switch-Num:255 Asic-Num:255 Feature-ID:AL_FID_L3_UNICAST_IPV4 Lkp-ftr-id:LKP_FEAT_INVALID ref_count:2 priv_ri/priv_si Handle: 0x7f5744f8afa8Hardware Indices/Handles: index0:0xc8 mtu_index/l3u_ri_index0:0x0 index1:0xc8 mtu_index/l3u_ri_index1:0x0 Features sharing this resource:66 (1)] 57 (1)] Cookie length: 56 00 00 00 00 00 00 00 00 04 04 00 00 00 00 00 00 00 00 00 00 07 00 52 54 00 02 cb f5 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 Detailed Resource Information (ASIC_INSTANCE# 0) ---------------------------------------- Station Index (SI) [0xc8] <-- Station Index contains RI and DI RI = 0x2c <-- Rewrite Index contains information for L2 Forwarding DI = 0x526e <-- Rewrite Index contains destination port information stationTableGenericLabel = 0 stationFdConstructionLabel = 0x7 lookupSkipIdIndex = 0 rcpServiceId = 0 dejaVuPreCheckEn = 0x1 Replication Bitmap: LD Detailed Resource Information (ASIC_INSTANCE# 1) ----------------------------------------
Station Index (SI) [0xc8] <-- Station Index contains RI and DI
RI = 0x2c <-- Rewrite Index contains information for L2 Forwarding
DI = 0x526e <-- Rewrite Index contains destination port information stationTableGenericLabel = 0 stationFdConstructionLabel = 0x7 lookupSkipIdIndex = 0 rcpServiceId = 0 dejaVuPreCheckEn = 0x1 Replication Bitmap: CD ==============================================================
Endpoint RI Decode
Take the RI (0x2c) and use in the command show platform hardware fed switch active fwd-asic resource asic all rewrite-index range <RI> <RI>
Edge-2#show platform hardware fed switch active fwd-asic resource asic all rewrite-index range 0x2c 0x2c ASIC#:0 RI:44 Rewrite_type:AL_RRM_REWRITE_L3_UNICAST_IPV4_SHARED(1) Mapped_rii:L3_UNICAST_IPV4(9) <-- Decimal 44 is hex 0x2c MAC Addr: MAC Addr: 52:54:00:02:cb:f5, <-- MAC Address 5254.0002.cbf5 is 10.47.10.2 L3IF LE Index 50 ASIC#:1 RI:44 Rewrite_type:AL_RRM_REWRITE_L3_UNICAST_IPV4_SHARED(1) Mapped_rii:L3_UNICAST_IPV4(9) <-- Decimal 44 is hex 0x2c MAC Addr: MAC Addr: 52:54:00:02:cb:f5,<-- MAC Address 5254.0002.cbf5 is 10.47.10.2 L3IF LE Index 50
Endpoint DI Decode
Take the DI (0x526e) and use in show platform hardware fed switch active fwd-asic resource asic all destination-index range <DI> <DI>
Edge-2#show platform hardware fed switch active fwd-asic resource asic all destination-index range 0x526e 0x526e ASIC#0: Destination index = 0x526e pmap = 0x00000000 0x00000010 <-- Convert 10 into binary, 0001 and 0000, so 00010000, and count from right to left, zero-based, so Port 4 pmap_intf : [GigabitEthernet1/0/5] cmi = 0x0 rcp_pmap = 0x0 al_rsc_cmi CPU Map Index (CMI) [0] ctiLo0 = 0 ctiLo1 = 0 ctiLo2 = 0 cpuQNum0 = 0 cpuQNum1 = 0 cpuQNum2 = 0 npuIndex = 0 stripSeg = 0 copySeg = 0 ASIC#1: Destination index = 0x526e pmap = 0x00000000 0x00000000 cmi = 0x0 rcp_pmap = 0x0 al_rsc_cmi CPU Map Index (CMI) [0] ctiLo0 = 0 ctiLo1 = 0 ctiLo2 = 0 cpuQNum0 = 0 cpuQNum1 = 0 cpuQNum2 = 0 npuIndex = 0 stripSeg = 0 copySeg = 0
Edge-2#show platform software fed switch active ifm mappings Interface IF_ID Inst Asic Core Port SubPort Mac Cntx LPN GPN Type Active GigabitEthernet1/0/1 0x1a 0 0 0 0 0 1 0 1 1 NIF Y GigabitEthernet1/0/2 0x1b 0 0 0 1 0 2 1 2 2 NIF Y GigabitEthernet1/0/3 0xb 0 0 0 2 0 3 2 3 3 NIF Y GigabitEthernet1/0/4 0xc 0 0 0 3 0 4 3 4 4 NIF Y GigabitEthernet1/0/5 0xd 0 0 0 4 0 5 4 5 5 NIF Y <-- Port 4 corresponds to Gig1/0/5 GigabitEthernet1/0/6 0xe 0 0 0 5 0 6 5 6 6 NIF Y GigabitEthernet1/0/7 0xf 0 0 0 6 0 7 6 7 7 NIF Y GigabitEthernet1/0/8 0x10 0 0 0 7 0 8 7 8 8 NIF Y
Edge-2 decapsulates the packet and send it towards the egress interface where HOST-03 is connected to:
Edge-2#monitor capture 1 interface g1/0/5 out match ipv4 host 10.47.4.2 host 10.47.10.2 Edge-2#monitor capture 1 start Started capture point : 1 Edge-2#monitor capture 1 stop Capture statistics collected at software: Capture duration - 6 seconds Packets received - 3 Packets dropped - 0 Packets oversized - 0 Number of Bytes dropped at asic not collected Capture buffer will exists till exported or cleared Stopped capture point : 1
Edge-2#show monitor capture 1 buffer brief Starting the packet display ........ Press Ctrl + Shift + 6 to exit 1 0.000000 10.47.4.2 -> 10.47.10.2 ICMP 106 Echo (ping) request id=0x0048, seq=0/0, ttl=62 2 0.984985 10.47.4.2 -> 10.47.10.2 ICMP 106 Echo (ping) request id=0x0048, seq=1/256, ttl=62 3 1.985357 10.47.4.2 -> 10.47.10.2 ICMP 106 Echo (ping) request id=0x0048, seq=2/512, ttl=62
Edge-2#show monitor capture 1 buffer detailed Starting the packet display ........ Press Ctrl + Shift + 6 to exit Frame 1: 106 bytes on wire (848 bits), 106 bytes captured (848 bits) on interface /tmp/epc_ws/wif_to_ts_pipe, id 0 Interface id: 0 (/tmp/epc_ws/wif_to_ts_pipe) Interface name: /tmp/epc_ws/wif_to_ts_pipe Encapsulation type: Ethernet (1) Arrival Time: Oct 11, 2023 17:22:20.730331000 UTC [Time shift for this packet: 0.000000000 seconds] Epoch Time: 1697044940.730331000 seconds [Time delta from previous captured frame: 0.000000000 seconds] [Time delta from previous displayed frame: 0.000000000 seconds] [Time since reference or first frame: 0.000000000 seconds] Frame Number: 1 Frame Length: 106 bytes (848 bits) Capture Length: 106 bytes (848 bits) [Frame is marked: False] [Frame is ignored: False] [Protocols in frame: eth:ethertype:cmd:ethertype:ip:icmp:data] Ethernet II, Src: 00:00:00:00:61:00 (00:00:00:00:61:00), Dst: ff:ff:ff:ff:ff:ff (ff:ff:ff:ff:ff:ff) <-- Dummy Ethernet header, EPC does not capture it properly Destination: ff:ff:ff:ff:ff:ff (ff:ff:ff:ff:ff:ff) Address: ff:ff:ff:ff:ff:ff (ff:ff:ff:ff:ff:ff) .... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default) .... ...1 .... .... .... .... = IG bit: Group address (multicast/broadcast) Source: 00:00:00:00:61:00 (00:00:00:00:61:00) Address: 00:00:00:00:61:00 (00:00:00:00:61:00) .... ..0. .... .... .... .... = LG bit: Globally unique address (factory default) .... ...0 .... .... .... .... = IG bit: Individual address (unicast) Type: CiscoMetaData (0x8909) Cisco MetaData Version: 1 Length: 1 Options: 0x0001 SGT: 0 Type: IPv4 (0x0800) Internet Protocol Version 4, Src: 10.47.4.2, Dst: 10.47.10.2 <-- True IP addresses 0100 .... = Version: 4 .... 0101 = Header Length: 20 bytes (5) Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT) 0000 00.. = Differentiated Services Codepoint: Default (0) .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0) Total Length: 84 Identification: 0x35e4 (13796) Flags: 0x4000, Don't fragment 0... .... .... .... = Reserved bit: Not set .1.. .... .... .... = Don't fragment: Set ..0. .... .... .... = More fragments: Not set Fragment offset: 0 Time to live: 62 Protocol: ICMP (1) Header checksum: 0xe463 [validation disabled] [Header checksum status: Unverified] Source: 10.47.4.2 Destination: 10.47.10.2 Internet Control Message Protocol Type: 8 (Echo (ping) request) Code: 0 Checksum: 0x2693 [correct] [Checksum Status: Good] Identifier (BE): 72 (0x0048) Identifier (LE): 18432 (0x4800) Sequence number (BE): 0 (0x0000) Sequence number (LE): 0 (0x0000) Data (56 bytes) 0000 69 9c 67 88 00 00 00 00 00 00 00 00 00 00 00 00 i.g............. 0010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 0020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 0030 00 00 00 00 00 00 00 00 ........ Data: 699c67880000000000000000000000000000000000000000b^@& [Length: 56]
Revision History
| Revision | Publish Date | Comments |
|---|---|---|
1.0 |
12-Oct-2023 |
Initial Release |
FeedbackContact Cisco
- Open a Support Case
- (Requires a Cisco Service Contract)