Troubleshoot ARP Resolution in SD-Access Fabric
Available Languages
Contents
Introduction
This document describes how to troubleshoot Address Resolution Protocol (ARP) in SD-Access fabric.
Prerequisites
Requirements
Cisco recommends that you have knowledge of these topics:
- Internet Protocol (IP) Forwarding
- Locator/ID Separation Protocol (LISP)
- Address Resolution Protocol (ARP)
Components Used
The information in this document is based on these software and hardware versions:
- C9000v on Cisco IOS® XE 17.10.1
- SDA 1.0 (not LISP PubSub)
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.
This document can also be used with these hardware and software versions:
- C9200
- C9300
- C9400
- C9500
- C9600
- Cisco IOS® XE 16.12 and later
Topology
In this exercise:
- 10.47.1.10 and 10.47.1.11 are Collocated Borders
- 10.47.1.12 and 10.47.1.13 are Fabric Edge Nodes
- 10.47.4.2 and 10.47.4.3 are endpoints within the SDA Fabric and are in the same VLAN/subnet and same VN (red_vn)
Two main use-cases are presented in this document:
- ARP resolution between 10.47.4.2 and 10.47.4.3 through a unicast path
- ARP resolution between 10.47.4.2 and 10.47.4.3 through a flooding path
Unicast Path Initial State
Within SDA, there are two main methods for ARP resolution within the fabric. What is referred to as the unicast path indicates that there is no configuration of Layer 2 (L2) ARP Flooding or lack of "flood arp nd" configured under the LISP L2 Instance ID (IID) that that corresponds to the VLAN in question. In addition to this caveat, to qualify for the unicast ARP path, there is no configuration of "IP local proxy ARP" or no presence of a Layer 3 Only Pool.
If these caveats are met, continue with this troubleshooting section and it is assumed that both endpoints (10.47.4.2 and 10.47.4.3) can reach their respective default gateways that exist on the Fabric Edge switches which exist as 10.47.4.1
Edge-1#ping vrf red_vn 10.47.4.2 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.47.4.2, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 257/292/344 ms
Edge-2#ping vrf red_vn 10.47.4.3 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.47.4.3, timeout is 2 seconds: !!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 221/321/566 ms
Next, verify that each Fabric Edge node has their respective endpoint within IP Device-Tracking, (IPDT) the LISP L2 Database, L2 Address Resolution (AR) Database, ad L3 Database.
Edge-1:
To verify the endpoint within IPDT, use the command show device-tracking database interface <interface connecting to endpoint>
Edge-1#show device-tracking database interface g1/0/3
portDB has 2 entries for interface Gi1/0/3, 2 dynamic
Codes: L - Local, S - Static, ND - Neighbor Discovery, ARP - Address Resolution Protocol, DH4 - IPv4 DHCP, DH6 - IPv6 DHCP, PKT - Other Packet, API - API created
Preflevel flags (prlvl):
0001:MAC and LLA match 0002:Orig trunk 0004:Orig access
0008:Orig trusted trunk 0010:Orig trusted access 0020:DHCP assigned
0040:Cga authenticated 0080:Cert authenticated 0100:Statically assigned
Network Layer Address Link Layer Address Interface vlan prlvl age state Time left
DH4 10.47.4.2 5254.0019.93e9 Gi1/0/3 1026 0024 15s REACHABLE 228 s try 0(6374 s)
To verify the endpoint in the LISP L2 Database, use the command show lisp eid-table vlan <vlan id> ethernet database <mac address> use the VLAN and MAC address from the previous command
Edge-1#show lisp eid-table vlan 1026 ethernet database 5254.0019.93e9 LISP ETR MAC Mapping Database for LISP 0 EID-table Vlan 1026 (IID 8190), LSBs: 0x1 Entries total 1, no-route 0, inactive 0, do-not-register 2 5254.0019.93e9/48, dynamic-eid Auto-L2-group-8190, inherited from default locator-set rloc_222e1707-175d-4019-a783-060404f8bc2f Uptime: 3w5d, Last-change: 3w5d Domain-ID: local Service-Insertion: N/A Locator Pri/Wgt Source State 10.47.1.12 10/10 cfg-intf site-self, reachable <-- Edge-1's RLOC Map-server Uptime ACK Domain-ID 10.47.1.10 1w3d Yes 0 10.47.1.11 3w5d Yes 0
To verify the L2 AR Database use the command show lisp eid-table vlan <vlan-id> ethernet database address-resolution <mac address>
Edge-1#show lisp eid-table vlan 1026 ethernet database address-resolution 5254.0019.93e9 LISP ETR Address Resolution for LISP 0 EID-table Vlan 1026 (IID 8190) (*) -> entry being deleted Hardware Address L3 InstID Host Address 5254.0019.93e9 4099 10.47.4.2/32 <-- Endpoint MAC address, LISP L3 IID, and IPv4 address, respectively
To verify the LISP L3 Database, use the command show lisp instance-id <L3 IID> ipv4 database <IPv4 address>/subnet mask use the LISP L3 IID from the previous command
Edge-1#show lisp instance-id 4099 ipv4 database 10.47.4.2/32 LISP ETR IPv4 Mapping Database for LISP 0 EID-table vrf red_vn (IID 4099), LSBs: 0x1 Entries total 1, no-route 0, inactive 0, do-not-register 1 10.47.4.2/32, dynamic-eid red-IPV4, inherited from default locator-set rloc_222e1707-175d-4019-a783-060404f8bc2f Uptime: 3w5d, Last-change: 3w5d Domain-ID: local Service-Insertion: N/A Locator Pri/Wgt Source State 10.47.1.12 10/10 cfg-intf site-self, reachable <-- Edge-1's own RLOC Map-server Uptime ACK Domain-ID 10.47.1.10 1w0d Yes 0 10.47.1.11 1w0d Yes 0
Edge-2:
To verify the endpoint within IPDT, use the command show device-tracking database interface <interface connecting to local endpoint>
Edge-2#show device-tracking database interface g1/0/3
portDB has 3 entries for interface Gi1/0/3, 3 dynamic
Codes: L - Local, S - Static, ND - Neighbor Discovery, ARP - Address Resolution Protocol, DH4 - IPv4 DHCP, DH6 - IPv6 DHCP, PKT - Other Packet, API - API created
Preflevel flags (prlvl):
0001:MAC and LLA match 0002:Orig trunk 0004:Orig access
0008:Orig trusted trunk 0010:Orig trusted access 0020:DHCP assigned
0040:Cga authenticated 0080:Cert authenticated 0100:Statically assigned
Network Layer Address Link Layer Address Interface vlan prlvl age state Time left
DH4 10.47.4.3 5254.001e.ad00 Gi1/0/3 1026 0024 122s REACHABLE 124 s try 0(5810 s)To verify the endpoint in the LISP L2 Database, use the command show lisp eid-table vlan <vlan id> ethernet database <mac address> use the VLAN and MAC address from the previous command
Edge-2#show lisp eid-table vlan 1026 ethernet database 5254.001e.ad00 LISP ETR MAC Mapping Database for LISP 0 EID-table Vlan 1026 (IID 8190), LSBs: 0x1 Entries total 1, no-route 0, inactive 0, do-not-register 2 5254.001e.ad00/48, dynamic-eid Auto-L2-group-8190, inherited from default locator-set rloc_691b1fe4-5264-44c2-bb1b-0903b3eb2c51 Uptime: 3w5d, Last-change: 3w5d Domain-ID: local Service-Insertion: N/A Locator Pri/Wgt Source State 10.47.1.13 10/10 cfg-intf site-self, reachable <-- Edge-2's RLOC Map-server Uptime ACK Domain-ID 10.47.1.10 1w2d Yes 0 10.47.1.11 1w2d Yes 0
To verify the L2 AR Database use the command show lisp eid-table vlan <vlan-id> ethernet database address-resolution <mac address>
Edge-2#show lisp eid-table vlan 1026 ethernet database address-resolution 5254.001e.ad00 LISP ETR Address Resolution for LISP 0 EID-table Vlan 1026 (IID 8190) (*) -> entry being deleted Hardware Address L3 InstID Host Address 5254.001e.ad00 4099 10.47.4.3/32 <-- Endpoint MAC address, LISP L3 IID, and IPv4 address, respectively
To verify the LISP L3 Database, use the command show lisp instance-id <L3 IID> ipv4 database <IPv4 address>/subnet mask use the LISP L3 IID from the previous command
Edge-2#show lisp instance-id 4099 ipv4 database 10.47.4.3/32 LISP ETR IPv4 Mapping Database for LISP 0 EID-table vrf ed_vn (IID 4099), LSBs: 0x1 Entries total 1, no-route 0, inactive 0, do-not-register 2 10.47.4.3/32, dynamic-eid red-IPV4, inherited from default locator-set rloc_691b1fe4-5264-44c2-bb1b-0903b3eb2c51 Uptime: 1w0d, Last-change: 1w0d Domain-ID: local Service-Insertion: N/A Locator Pri/Wgt Source State 10.47.1.13 10/10 cfg-intf site-self, reachable <-- Edge-2, RLOC Map-server Uptime ACK Domain-ID 10.47.1.10 1w0d Yes 0 10.47.1.11 1w0d Yes 0
Collocated Borders:
Both endpoints ideally are registered from a L2 LISP, L2 AR, and L3 perspective to both Fabric Control-Planes (Border-1 and Border-2). To derive the L2 LISP IID, check a Fabric Edge node and use the command show vlan id <vlan id>
Edge-1#show vlan id 1026 VLAN Name Status Ports ---- -------------------------------- --------- ------------------------------- 1026 red active L2LI0:8190, Gi1/0/3 <-- L2 LISP IID VLAN Type SAID MTU Parent RingNo BridgeNo Stp BrdgMode Trans1 Trans2 ---- ----- ---------- ----- ------ ------ -------- ---- -------- ------ ------ 1026 enet 101026 1500 - - - - - 0 0 Remote SPAN VLAN ---------------- Disabled Primary Secondary Type Ports ------- --------- ----------------- ------------------------------------------
Utilize the L2 IID in the command show lisp instance-id <L2 IID> ethernet server to verify the server from a L2 LISP perspective
Border-1#show lisp instance-id 8190 ethernet server
LISP Site Registration Information
* = Some locators are down or unreachable
# = Some registrations are sourced by reliable transport
Site Name Last Up Who Last Inst EID Prefix
Register Registered ID
site_uci never no -- 8190 any-mac
1w3d yes# 10.47.1.12:21038 8190 5254.0019.93e9/48 <-- RLOC of the FE node, EID prefix that was registered
1w2d yes# 10.47.1.13:16056 8190 5254.001e.ad00/48 <-- RLOC of the FE node, EID prefix that was registeredTo verify the L2 AR server, use the command show lisp instance-id <L2 IID> ethernet server address-resolution
Border-1#show lisp instance-id 8190 ethernet server address-resolution
Address-resolution data for router lisp 0 instance-id 8190
L3 InstID Host Address Hardware Address
4099 10.47.4.2/32 5254.0019.93e9 <-- L3 LISP IID, Endpoint IPv4 Address, MAC Address, respectively
4099 10.47.4.3/32 5254.001e.ad00 <-- L3 LISP IID, Endpoint IPv4 Address, MAC Address, respectivelyTo verify the L3 server, use the command show lisp instance-id <L3 IID> ipv4 server
Border-1#show lisp instance-id 4099 ipv4 server
LISP Site Registration Information
* = Some locators are down or unreachable
# = Some registrations are sourced by reliable transport
Site Name Last Up Who Last Inst EID Prefix
Register Registered ID
site_uci never no -- 4099 0.0.0.0/0
6d01h yes# 10.47.1.11:22876 4099 8.8.8.8/32
1w0d yes# 10.47.1.10:21610 4099 10.47.2.4/30
1w0d yes# 10.47.1.11:22876 4099 10.47.2.12/30
never no -- 4099 10.47.4.0/24
1w0d yes# 10.47.1.12:21038 4099 10.47.4.2/32
1w0d yes# 10.47.1.13:16056 4099 10.47.4.3/32
1w0d yes# 10.47.1.11:22876 4099 10.47.6.0/24
1w0d yes# 10.47.1.11:22876 4099 10.47.7.0/24
1w0d yes# 10.47.1.11:22876 4099 10.47.9.8/29
never no -- 4099 10.47.10.0/24
1w0d yes# 10.47.1.13:16056 4099 10.47.10.2/32Control Planes have a brief history of the registration events for L3, L2, and AR entries, which can be useful when troubleshooting roaming clients or a possible loop where clients are unexpectedly registered by several Fabric Edges in a very short time.
Border-1#show lisp instance-id 8190 ethernet server resolution registration-history | include Timestamp|10.47.4.2|10.47.4.3
Timestamp (UTC) Instance Proto Roam WLC Source
*Oct 9 19:14:39.183 8190 TCP No No 10.47.1.12
+*10.47.4.2/32 / 5254.0019.93e9 <-- Last registered at Oct 9 19:14:39
*Oct 9 19:14:41.183 8190 TCP No No 10.47.1.13
+*10.47.4.3/32 / 5254.001e.ad00 <-- Last registered at OCt 9 19:14:31Unicast Path ARP Request High Level Workflow
Unicast Path ARP Request Verification
The endpoint that owns IP Address 10.47.4.2 sends a Broadcast ARP Request, confirm via an Embedded Packet Capture (EPC) on Edge-1
Edge-1 (10.47.1.12)
Edge-1#monitor capture 1 interface g1/0/3 in match any
Edge-1#monitor capture 1 start
Edge-1#monitor capture 1 stop
Capture statistics collected at software:
Capture duration - 39 seconds
Packets received - 21
Packets dropped - 0
Packets oversized - 0
Number of Bytes dropped at asic not collected
Capture buffer will exists till exported or cleared
Stopped capture point : 1
Edge-1#show monitor capture 1 buffer brief
Starting the packet display ........ Press Ctrl + Shift + 6 to exit
1 0.000000 52:54:00:19:93:e9 -> ff:ff:ff:ff:ff:ff ARP 60 Who has 10.47.4.3? Tell 10.47.4.
Edge-1#show monitor capture 1 buffer detailed
Starting the packet display ........ Press Ctrl + Shift + 6 to exit
Frame 1: 60 bytes on wire (480 bits), 60 bytes captured (480 bits) on interface /tmp/epc_ws/wif_to_ts_pipe, id 0
Interface id: 0 (/tmp/epc_ws/wif_to_ts_pipe)
Interface name: /tmp/epc_ws/wif_to_ts_pipe
Encapsulation type: Ethernet (1)
Arrival Time: Oct 19, 2023 23:43:31.893095000 UTC
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1697759011.893095000 seconds
[Time delta from previous captured frame: 0.000000000 seconds]
[Time delta from previous displayed frame: 0.000000000 seconds]
[Time since reference or first frame: 0.000000000 seconds]
Frame Number: 1
Frame Length: 60 bytes (480 bits)
Capture Length: 60 bytes (480 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ethertype:arp]
Ethernet II, Src: 52:54:00:19:93:e9 (52:54:00:19:93:e9), Dst: ff:ff:ff:ff:ff:ff (ff:ff:ff:ff:ff:ff)
Destination: ff:ff:ff:ff:ff:ff (ff:ff:ff:ff:ff:ff)
Address: ff:ff:ff:ff:ff:ff (ff:ff:ff:ff:ff:ff)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...1 .... .... .... .... = IG bit: Group address (multicast/broadcast)
Source: 52:54:00:19:93:e9 (52:54:00:19:93:e9)
Address: 52:54:00:19:93:e9 (52:54:00:19:93:e9)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Type: ARP (0x0806)
Padding: 000000000000000000000000000000000000
Address Resolution Protocol (request)
Hardware type: Ethernet (1)
Protocol type: IPv4 (0x0800)
Hardware size: 6
Protocol size: 4
Opcode: request (1)
Sender MAC address: 52:54:00:19:93:e9 (52:54:00:19:93:e9)
Sender IP address: 10.47.4.2
Target MAC address: 00:00:00:00:00:00 (00:00:00:00:00:00)
Target IP address: 10.47.4.3This ARP Request is punted up to the CPU for further processing. Utilize the FED Punject Capture to gain additional information.
Edge-1#debug platform software fed switch active punt packet-capture set-filter “arp" Edge-1#debug platform software fed switch active punt packet-capture start Edge-1#debug platform software fed switch active punt packet-capture stop Edge-1#show platform software fed switch active punt packet-capture brief Punt packet capturing: disabled. Buffer wrapping: disabled Total captured so far: 8 packets. Capture capacity : 4096 packets Capture filter : "arp" ------ Punt Packet Number: 1, Timestamp: 2023/10/19 23:55:03.552 ------ interface : physical: GigabitEthernet1/0/3[if-id: 0x0000000b], pal: GigabitEthernet1/0/3 [if-id: 0x0000000b] <-- Physical interface the ARP Request was received from metadata : cause: 109 [snoop packets], sub-cause: 1, q-no: 16, linktype: MCP_LINK_TYPE_IP [1] <-- Punted for cause snoop packets to CPU queue 16 ether hdr : dest mac: ffff.ffff.ffff, src mac: 5254.0019.93e9 ether hdr : ethertype: 0x0806 (ARP)
To see what CPU queue 16 is, use the command show platform software fed switch active punt cpuq 16
Edge-1#show platform software fed switch active punt cpuq 16 Punt CPU Q Statistics =========================================== CPU Q Id : 16 CPU Q Name : CPU_Q_PROTO_SNOOPING Packets received from ASIC : 49054 Send to IOSd total attempts : 49054 <-- Same number as received from ASIC
Send to IOSd failed count : 0 <-- No failures
Ideally there are no drops in the Proto Snooping Queue in the CPU policer (Queue 16), use the command show platform hardware fed switch active qos queue stats internal cpu policer
Edge-1#show platform hardware fed switch active qos queue stats internal cpu policer | include QId|Proto QId PlcIdx Queue Name Enabled Rate Rate Drop(Bytes) Drop(Frames) 16 12 Proto Snooping No 2000 2000 0 0 <-- No drops
Next, the punted ARP Request is processed by Punt Service. You can see this behavior with the debug, debug platform software infrastructure punt
Edge-1#debug platform software infrastructure punt
*Oct 20 00:07:01.509: PUNT RX: mcprp_process_receive_packet: pak->vlan_id: 1026 *Oct 20 00:07:01.509: Punt: IP proto src 147.233.10.47, dst 4.2.0.0, from table 0, intf Gi1/0/3, encap ARPA, size 60, cause snoop packets(L3) <-- The IP address is obscured, but it is 10.47.4.2 *Oct 20 00:07:01.509: punt cause:snoop packets invoking reg_invoke_mcprp_punt_feature_msg *Oct 20 00:07:01.509: punt cause:snoop packets MCPRP_PUNT_PAK_PROC_OK_DONE
After Punt Service processes the packet, it hands the packet to the Proto Snoop and Proto ARP snoop process. This creates an entry in the ARP snooping table and the packet is sent to Device-Tracking/SISF. To see this, debug arp as well as debug platform fhs all
Edge-1#debug arp
Edge-1#debug platform fhs all
*Oct 20 00:12:06.908: ARP packet received from ARP snooper(Gi1/0/3 10.47.4.2 (5254.0019.93e9) VLAN:1026)
Use the command show platform arpsnooping client <MAC address> to see what happens to the ARP request
Edge-1#show platform arpsnooping client 5254.0019.93e9 PLAT_DAI : Platform DAI shim FWDPLANE : Dataplane forwarding BRIDGE : Packet to be bridged ARPSN : Arp Snooping Packet Trace for client MAC 5254.0019.93E9: Timestamp Sender Mac Sender IP Target Mac Target IP Opcode VLAN Handler:Action ----------------------- --------------- ------------- ----------------- -------------- ------------ ------- ------------------------------ 2023/10/24 14:37:15.045 5254.0019.93e9 10.47.4.2 0000.0000.0000 10.47.4.3 ARP_REQUEST 1026 PSISF_PRE:PSP_BRNEUTRAL 2023/10/24 14:37:15.045 5254.0019.93e9 10.47.4.2 0000.0000.0000 10.47.4.3 ARP_REQUEST 1026 INJECT:INJ_VLAN_IFINPUT_TO_BDI 2023/10/24 14:37:15.045 5254.0019.93e9 10.47.4.2 0000.0000.0000 10.47.4.3 ARP_REQUEST 1026 INJECT:BD_DPIDX_TO_FWDPLANE
As IPDT/SISF processes the packet, it verifies the contents of the ARP Request and create a temporary entry while it awaits resolution via LISP.
Edge-1#show device-tracking messages detailed 255 | i 5254.0019.93e9 [Tue Oct 24 14:37:12.000] VLAN 1026, From Gi1/0/3 seclvl [guard], MAC 5254.0019.93e9: ARP::REQ, [Tue Oct 24 14:37:13.000] VLAN 1026, From Gi1/0/3 seclvl [guard], MAC 5254.0019.93e9: ARP::REQ,
If you use debug device-tracking you see that that there is an entry pointing towards 0000.0000.00fd in device-tracking. All this means is that the host behind Gi1/0/3 is querying for a host that has yet to be resolved, it is in a transient state (or permanent in case it is querying for a host that does not exist, this is not necessarily a bad thing)
Edge-1#debug device-tracking Device-tracking - General debugging is on *Oct 24 14:55:02.967: SISF[POL]: vlan 1026 matches vlan list on policy IPDT_POLICY for target Gi1/0/3 *Oct 24 14:55:02.967: SISF[POL]: Found matching policy IPDT_POLICY for feature Device-tracking on Gi1/0/3
*Oct 24 14:55:02.967:SISF[GLN]: Checking if ARP ownership can be taken by device-tracking
*Oct 24 14:55:02.967:SISF[GLN]: Not an ARP reply, do not take over
*Oct 24 14:55:02.967:SISF[POL]: Found matching policy LISP-AR-RELAY-VLAN for feature Address Resolution Relay on vlan 1026 *Oct 24 14:55:02.967: SISF[MAC]: Creating new MAC entry for 0000.0000.00fd on interface if none exists *Oct 24 14:55:02.967: SISF[MAC]: Number of MAC entries in MAC-CREATING state incremented to 1 *Oct 24 14:55:02.968: SISF[BT ]: Attaching 0000.0000.00fd entry in MacAdrDB for 10.47.4.3 *Oct 24 14:55:02.968: SISF[GLN]: Binding entry event 1 for 10.47.4.3
SISF triggers ARP Resolution using LISP because of the LISP-AR-RELAY-VLAN policy, after this LISP Control-Plane processes can take over.
Edge-1#debug lisp control-plane all Edge-1#debug l2lisp all *Oct 24 15:10:27.677: LISP Client 'SISF client': SISF request to resolve 10.47.4.3 in Vlan 1026.
Edge-1 sends a LISP Map-Request to resolve the MAC address of 10.47.4.3 via the LISP Control-Plane(s)
Edge-1#debug lisp control-plane all Edge-1#debug l2lisp all *Oct 24 15:10:27.681: LISP[REMT ]-0: Map Request: Delay is over for IID 8190 EID 10.47.4.3/32, requester 'AR'. *Oct 24 15:10:27.681: LISP[REMT ]-0 IID 8190: Schedule processing of Map-Requests from 'remote EID prefix' in IPv4. *Oct 24 15:10:27.681: LISP[REMT ]-0: Map Request: Sending request for IID 8190 EID 10.47.4.3/32, requester 'AR'.
A bidirectional Embedded Packet Capture (EPC) done at the CPU (control-plane) of Edge-1 demonstrates the incoming ARP request and the subsequent LISP Map-Request
Edge-1#monitor capture 1 control-plane both match any Edge-1#monitor capture 1 start Started capture point : 1 Edge-1#monitor capture 1 stop Edge-1#show monitor capture 1 buffer display-filter "arp.dst.proto_ipv4==10.47.4.3 or lisp" Starting the packet display ........ Press Ctrl + Shift + 6 to exit 60 10.110293 52:54:00:19:93:e9 -> ff:ff:ff:ff:ff:ff ARP 60 Who has 10.47.4.3? Tell 10.47.4.2 61 10.111714 10.47.4.3 -> 10.47.4.3 LISP 114 Encapsulated Map-Request for Unknown LCAF Type (53)/32
You can take a more detailed view of the LISP Map-Request
Edge-1#show monitor capture 1 buffer display-filter "frame.number==61" detailed
Starting the packet display ........ Press Ctrl + Shift + 6 to exit
Frame 61: 114 bytes on wire (912 bits), 114 bytes captured (912 bits) on interface /tmp/epc_ws/wif_to_ts_pipe, id 0
Interface id: 0 (/tmp/epc_ws/wif_to_ts_pipe)
Interface name: /tmp/epc_ws/wif_to_ts_pipe
Encapsulation type: Ethernet (1)
Arrival Time: Oct 24, 2023 15:20:08.948469000 UTC
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1698160808.948469000 seconds
[Time delta from previous captured frame: 0.001421000 seconds]
[Time delta from previous displayed frame: 0.000000000 seconds]
[Time since reference or first frame: 10.111714000 seconds]
Frame Number: 61
Frame Length: 114 bytes (912 bits)
Capture Length: 114 bytes (912 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ethertype:ip:udp:lisp:ip:udp:lisp]
Ethernet II, Src: 00:00:00:00:00:00 (00:00:00:00:00:00), Dst: 00:00:00:00:00:00 (00:00:00:00:00:00) <-- Ignore the SMAC/DMAC this is done up at the CPU, not final MAC addresses
Destination: 00:00:00:00:00:00 (00:00:00:00:00:00)
Address: 00:00:00:00:00:00 (00:00:00:00:00:00)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Source: 00:00:00:00:00:00 (00:00:00:00:00:00)
Address: 00:00:00:00:00:00 (00:00:00:00:00:00)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 10.47.1.12, Dst: 10.47.1.10 <-- Edge-1 RLOC and one of the collocated border RLOC, respectively
0100 .... = Version: 4
.... 0101 = Header Length: 20 bytes (5)
Differentiated Services Field: 0xc0 (DSCP: CS6, ECN: Not-ECT)
1100 00.. = Differentiated Services Codepoint: Class Selector 6 (48)
.... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
Total Length: 100
Identification: 0x599c (22940)
Flags: 0x0000
0... .... .... .... = Reserved bit: Not set
.0.. .... .... .... = Don't fragment: Not set
..0. .... .... .... = More fragments: Not set
Fragment offset: 0
Time to live: 255
Protocol: UDP (17)
Header checksum: 0x4ab9 [validation disabled]
[Header checksum status: Unverified]
Source: 10.47.1.12
Destination: 10.47.1.10
User Datagram Protocol, Src Port: 4342, Dst Port: 4342
Source Port: 4342
Destination Port: 4342
Length: 80
Checksum: 0x6393 [unverified]
[Checksum Status: Unverified]
[Stream index: 0]
[Timestamps]
[Time since first frame: 0.000000000 seconds]
[Time since previous frame: 0.000000000 seconds]
Locator/ID Separation Protocol
1000 .... .... .... .... .... = Type: Encapsulated Control Message (8)
.... 0... .... .... .... .... .... .... = S bit (LISP-SEC capable): Not set
.... .0.. .... .... .... .... .... .... = D bit (DDT-originated): Not set
.... ..00 0000 0000 0000 0000 0000 0000 = Reserved bits: 0x0000000
Internet Protocol Version 4, Src: 10.47.4.3, Dst: 10.47.4.3 <-- MAP Request to resolve the MAC address tied to 10.47.4.3
0100 .... = Version: 4
.... 0101 = Header Length: 20 bytes (5)
Differentiated Services Field: 0xc0 (DSCP: CS6, ECN: Not-ECT)
1100 00.. = Differentiated Services Codepoint: Class Selector 6 (48)
.... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
Total Length: 68
Identification: 0x599b (22939)
Flags: 0x0000
0... .... .... .... = Reserved bit: Not set
.0.. .... .... .... = Don't fragment: Not set
..0. .... .... .... = More fragments: Not set
Fragment offset: 0
Time to live: 255
Protocol: UDP (17)
Header checksum: 0x44ea [validation disabled]
[Header checksum status: Unverified]
Source: 10.47.4.3
Destination: 10.47.4.3
User Datagram Protocol, Src Port: 4342, Dst Port: 4342
Source Port: 4342
Destination Port: 4342
Length: 48
Checksum: 0x9622 [unverified]
[Checksum Status: Unverified]
[Stream index: 1]
[Timestamps]
[Time since first frame: 0.000000000 seconds]
[Time since previous frame: 0.000000000 seconds]
Locator/ID Separation Protocol
0001 .... .... .... .... .... = Type: Map-Request (1)
.... 0000 00.. .... .... .... = Flags: 0x00
.... 0... .... .... .... .... = A bit (Authoritative): Not set
.... .0.. .... .... .... .... = M bit (Map-Reply present): Not set
.... ..0. .... .... .... .... = P bit (Probe): Not set
.... ...0 .... .... .... .... = S bit (Solicit-Map-Request): Not set
.... .... 0... .... .... .... = p bit (Proxy ITR): Not set
.... .... .0.. .... .... .... = s bit (SMR-invoked): Not set
.... .... ..00 0000 000. .... = Reserved bits: 0x000
.... .... .... .... ...0 0000 = ITR-RLOC Count: 0
Record Count: 1
Nonce: 0xcffee30fb39a05b7
Source EID AFI: Reserved (0)
Source EID: not set
ITR-RLOC 1: 10.47.1.12
ITR-RLOC AFI: IPv4 (1)
ITR-RLOC Address: 10.47.1.12
Map-Request Record 1: Unknown LCAF Type (53)/32
Reserved: 0x00
Prefix Length: 32
Prefix AFI: LISP Canonical Address Format (LCAF) (16387)
Prefix: Unknown LCAF Type (53)
LCAF: Unknown (53)
LCAF Header: 00003520000a
Reserved bits: 0x00
Flags: 0x00
Type: Unknown (53)
Reserved bits: 0x20
Length: 10
[Expert Info (Error/Protocol): LCAF type 53 is not defined in draft-ietf-lisp-lcaf-05]
[LCAF type 53 is not defined in draft-ietf-lisp-lcaf-05]
[Severity level: Error]
[Group: Protocol]
Collocated Border(s) (10.47.1.10 and 10.47.1.11)
Next, the LISP Control-Plane responds to Edge-1 with a LISP Map-Reply. This can be seen in debugs as well as CPU EPC.
Border-2#show monitor capture 1 buffer display-filter lisp Starting the packet display ........ Press Ctrl + Shift + 6 to exit 113 12.767420 10.47.4.3 -> 10.47.4.3 LISP 114 Encapsulated Map-Request for Unknown LCAF Type (53)/32 114 12.774428 10.47.1.11 -> 10.47.1.12 LISP 96 Map-Reply for Unknown LCAF Type (53)/32
As we look at the packets in more detail, we can see the LISP Map-Request and the subsequent LISP Map-Reply
Border-2#show monitor capture 1 buffer display-filter frame.number==113 detailed
Starting the packet display ........ Press Ctrl + Shift + 6 to exit
Frame 113: 114 bytes on wire (912 bits), 114 bytes captured (912 bits) on interface /tmp/epc_ws/wif_to_ts_pipe, id 0
Interface id: 0 (/tmp/epc_ws/wif_to_ts_pipe)
Interface name: /tmp/epc_ws/wif_to_ts_pipe
Encapsulation type: Ethernet (1)
Arrival Time: Oct 24, 2023 15:41:06.566253000 UTC
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1698162066.566253000 seconds
[Time delta from previous captured frame: 0.013424000 seconds]
[Time delta from previous displayed frame: 0.000000000 seconds]
[Time since reference or first frame: 12.767420000 seconds]
Frame Number: 113
Frame Length: 114 bytes (912 bits)
Capture Length: 114 bytes (912 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ethertype:ip:udp:lisp:ip:udp:lisp]
Ethernet II, Src: 52:54:00:04:84:a3 (52:54:00:04:84:a3), Dst: 52:54:00:1c:7d:e0 (52:54:00:1c:7d:e0) <-- True MAC addresses
Destination: 52:54:00:1c:7d:e0 (52:54:00:1c:7d:e0)
Address: 52:54:00:1c:7d:e0 (52:54:00:1c:7d:e0)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Source: 52:54:00:04:84:a3 (52:54:00:04:84:a3)
Address: 52:54:00:04:84:a3 (52:54:00:04:84:a3)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 10.47.1.12, Dst: 10.47.1.11 <-- Edge-1 RLOC and Border-2 RLOC, respectively
0100 .... = Version: 4
.... 0101 = Header Length: 20 bytes (5)
Differentiated Services Field: 0xc0 (DSCP: CS6, ECN: Not-ECT)
1100 00.. = Differentiated Services Codepoint: Class Selector 6 (48)
.... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
Total Length: 100
Identification: 0x5e19 (24089)
Flags: 0x0000
0... .... .... .... = Reserved bit: Not set
.0.. .... .... .... = Don't fragment: Not set
..0. .... .... .... = More fragments: Not set
Fragment offset: 0
Time to live: 255
Protocol: UDP (17)
Header checksum: 0x463b [validation disabled]
[Header checksum status: Unverified]
Source: 10.47.1.12
Destination: 10.47.1.11
User Datagram Protocol, Src Port: 4342, Dst Port: 4342
Source Port: 4342
Destination Port: 4342
Length: 80
Checksum: 0x6392 [unverified]
[Checksum Status: Unverified]
[Stream index: 1]
[Timestamps]
[Time since first frame: 0.000000000 seconds]
[Time since previous frame: 0.000000000 seconds]
Locator/ID Separation Protocol
1000 .... .... .... .... .... = Type: Encapsulated Control Message (8)
.... 0... .... .... .... .... .... .... = S bit (LISP-SEC capable): Not set
.... .0.. .... .... .... .... .... .... = D bit (DDT-originated): Not set
.... ..00 0000 0000 0000 0000 0000 0000 = Reserved bits: 0x0000000
Internet Protocol Version 4, Src: 10.47.4.3, Dst: 10.47.4.3 <-- LISP MAP Request for 10.47.4.3
0100 .... = Version: 4
.... 0101 = Header Length: 20 bytes (5)
Differentiated Services Field: 0xc0 (DSCP: CS6, ECN: Not-ECT)
1100 00.. = Differentiated Services Codepoint: Class Selector 6 (48)
.... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
Total Length: 68
Identification: 0x5e18 (24088)
Flags: 0x0000
0... .... .... .... = Reserved bit: Not set
.0.. .... .... .... = Don't fragment: Not set
..0. .... .... .... = More fragments: Not set
Fragment offset: 0
Time to live: 255
Protocol: UDP (17)
Header checksum: 0x406d [validation disabled]
[Header checksum status: Unverified]
Source: 10.47.4.3
Destination: 10.47.4.3
User Datagram Protocol, Src Port: 4342, Dst Port: 4342
Source Port: 4342
Destination Port: 4342
Length: 48
Checksum: 0xe9a8 [unverified]
[Checksum Status: Unverified]
[Stream index: 2]
[Timestamps]
[Time since first frame: 0.000000000 seconds]
[Time since previous frame: 0.000000000 seconds]
Locator/ID Separation Protocol
0001 .... .... .... .... .... = Type: Map-Request (1)
.... 0000 00.. .... .... .... = Flags: 0x00
.... 0... .... .... .... .... = A bit (Authoritative): Not set
.... .0.. .... .... .... .... = M bit (Map-Reply present): Not set
.... ..0. .... .... .... .... = P bit (Probe): Not set
.... ...0 .... .... .... .... = S bit (Solicit-Map-Request): Not set
.... .... 0... .... .... .... = p bit (Proxy ITR): Not set
.... .... .0.. .... .... .... = s bit (SMR-invoked): Not set
.... .... ..00 0000 000. .... = Reserved bits: 0x000
.... .... .... .... ...0 0000 = ITR-RLOC Count: 0
Record Count: 1
Nonce: 0x50c5f2b60b41ca1c
Source EID AFI: Reserved (0)
Source EID: not set
ITR-RLOC 1: 10.47.1.12
ITR-RLOC AFI: IPv4 (1)
ITR-RLOC Address: 10.47.1.12
Map-Request Record 1: Unknown LCAF Type (53)/32
Reserved: 0x00
Prefix Length: 32
Prefix AFI: LISP Canonical Address Format (LCAF) (16387)
Prefix: Unknown LCAF Type (53)
LCAF: Unknown (53)
LCAF Header: 00003520000a
Reserved bits: 0x00
Flags: 0x00
Type: Unknown (53)
Reserved bits: 0x20
Length: 10
[Expert Info (Error/Protocol): LCAF type 53 is not defined in draft-ietf-lisp-lcaf-05]
[LCAF type 53 is not defined in draft-ietf-lisp-lcaf-05]
[Severity level: Error]
[Group: Protocol]
We can also look at the LISP Map-Reply that is sent back
Border-2#show monitor capture 1 buffer display-filter frame.number==114 detailed
Starting the packet display ........ Press Ctrl + Shift + 6 to exit
Frame 114: 96 bytes on wire (768 bits), 96 bytes captured (768 bits) on interface /tmp/epc_ws/wif_to_ts_pipe, id 0
Interface id: 0 (/tmp/epc_ws/wif_to_ts_pipe)
Interface name: /tmp/epc_ws/wif_to_ts_pipe
Encapsulation type: Ethernet (1)
Arrival Time: Oct 24, 2023 15:41:06.573261000 UTC
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1698162066.573261000 seconds
[Time delta from previous captured frame: 0.007008000 seconds]
[Time delta from previous displayed frame: 0.000000000 seconds]
[Time since reference or first frame: 12.774428000 seconds]
Frame Number: 114
Frame Length: 96 bytes (768 bits)
Capture Length: 96 bytes (768 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ethertype:ip:udp:lisp]
Ethernet II, Src: 00:00:00:00:00:00 (00:00:00:00:00:00), Dst: 00:00:00:00:00:00 (00:00:00:00:00:00) <-- CPU Inject does not properly show MAC addresses
Destination: 00:00:00:00:00:00 (00:00:00:00:00:00)
Address: 00:00:00:00:00:00 (00:00:00:00:00:00)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Source: 00:00:00:00:00:00 (00:00:00:00:00:00)
Address: 00:00:00:00:00:00 (00:00:00:00:00:00)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 10.47.1.11, Dst: 10.47.1.12 <-- Border-2 RLOC and Edge-1 RLOC, respectively
0100 .... = Version: 4
.... 0101 = Header Length: 20 bytes (5)
Differentiated Services Field: 0xc0 (DSCP: CS6, ECN: Not-ECT)
1100 00.. = Differentiated Services Codepoint: Class Selector 6 (48)
.... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
Total Length: 82
Identification: 0xe231 (57905)
Flags: 0x0000
0... .... .... .... = Reserved bit: Not set
.0.. .... .... .... = Don't fragment: Not set
..0. .... .... .... = More fragments: Not set
Fragment offset: 0
Time to live: 255
Protocol: UDP (17)
Header checksum: 0xc234 [validation disabled]
[Header checksum status: Unverified]
Source: 10.47.1.11
Destination: 10.47.1.12
User Datagram Protocol, Src Port: 4342, Dst Port: 4342
Source Port: 4342
Destination Port: 4342
Length: 62
Checksum: 0xe1d6 [unverified]
[Checksum Status: Unverified]
[Stream index: 1]
[Timestamps]
[Time since first frame: 0.007008000 seconds]
[Time since previous frame: 0.007008000 seconds]
Locator/ID Separation Protocol
0010 .... .... .... .... .... = Type: Map-Reply (2)
.... 0... .... .... .... .... = P bit (Probe): Not set
.... .0.. .... .... .... .... = E bit (Echo-Nonce locator reachability algorithm enabled): Not set
.... ..0. .... .... .... .... = S bit (LISP-SEC capable): Not set
.... ...0 0000 0000 0000 0000 = Reserved bits: 0x00000
Record Count: 1
Nonce: 0x50c5f2b60b41ca1c
Mapping Record 1, EID Prefix: Unknown LCAF Type (53)/32, TTL: 1440, Action: No-Action, Not Authoritative
Record TTL: 1440
Locator Count: 1
EID Mask Length: 32
000. .... .... .... = Action: No-Action (0)
...0 .... .... .... = Authoritative bit: Not set
.... .000 0000 0000 = Reserved: 0x000
0000 .... .... .... = Reserved: 0x0
.... 0000 0000 0000 = Mapping Version: 0
EID Prefix AFI: LISP Canonical Address Format (LCAF) (16387)
EID Prefix: Unknown LCAF Type (53)
LCAF: Unknown (53)
LCAF Header: 00003520000a
Reserved bits: 0x00
Flags: 0x00
Type: Unknown (53)
Reserved bits: 0x20
Length: 10
[Expert Info (Error/Protocol): LCAF type 53 is not defined in draft-ietf-lisp-lcaf-05]
[LCAF type 53 is not defined in draft-ietf-lisp-lcaf-05]
[Severity level: Error]
[Group: Protocol]
Locator Record 1, RLOC: 52:54:00:1e:ad:00, Unreachable, Priority/Weight: 1/100, Multicast Priority/Weight: 1/100
Priority: 1
Weight: 100
Multicast Priority: 1
Multicast Weight: 100
Flags: 0x0000
0000 0000 0000 0... = Reserved: 0x0000
.... .... .... .0.. = Local: Not set
.... .... .... ..0. = Probe: Not set
.... .... .... ...0 = Reachable: Not set
AFI: 802 (includes all 802 media plus Ethernet) (6)
Locator: 52:54:00:1e:ad:00
Border-2#debug lisp control-plane all All LISP control debugging is on at verbose level Border-2#debug l2lisp all All L2Lisp debugging is on *Oct 24 16:02:17.854: LISP[TRNSP]-0: Processing received Encap-Control(8) message on GigabitEthernet1/0/3 from 10.47.1.12:4342 to 10.47.1.11:4342. *Oct 24 16:02:17.854: LISP[TRNSP]-0: Processing received Map-Request(1) message on GigabitEthernet1/0/3 from 10.47.4.3:4342 to 10.47.4.3:4342. *Oct 24 16:02:17.855: LISP[MR ]-0: Received Map-Request with 1 records, first EID IID 8190 10.47.4.3/32, source EID UNSPEC, nonce 0x3847C91F-0xA966DDEA. *Oct 24 16:02:17.855: LISP[MR ]-0 IID 8190 Eth-ARP: MS EID 10.47.4.3/32: Sending proxy reply to 10.47.1.12.
Now that Edge-1 has received a LISP Map-Reply for the Address Resolution (AR) Request saying that 10.47.4.3 is 5254.001e.ad00, Edge-1 generates another LISP Map-Request to determine the RLOC for the endpoint MAC Address
Edge-1#debug lisp control-plane all
Edge-1#debug l2lisp all
*Oct 24 16:19:54.843: LISP[REMT ]-0: Received Map-Reply with nonce 0x37F890B9-0xAC60D2B9, 1 records.
*Oct 24 16:19:54.843: LISP[MS ]-0: This is a Address Resolution message.
*Oct 24 16:19:54.843: LISP[REMT ]-0: Map-Reply nonce matches pending request for IID 8190 EID 10.47.4.3/32, requester 'AR'.
*Oct 24 16:19:54.843: LISP[REMT ]-0: Processing Map-Reply mapping record for IID 8190 Eth-ARP 10.47.4.3/32 LCAF 53, ttl 1440, action none, not authoritative, 1 locator.
*Oct 24 16:19:54.843: LISP[REMT ]-0: 5254.001e.ad00 pri/wei/dID/mID/met/si_type/si_id/si_flg/afn_id=1/100/0/0/4294967295/none/0/UNSPEC/UNSPEC lpr.
<snip>
*Oct 24 17:11:24.056: LISP[REMT ]-0 IID 8190: Schedule processing of Map-Requests from 'remote EID prefix' in IPv4.
*Oct 24 17:11:24.056: LISP[REMT ]-0: Map Request: Sending request for IID 8190 EID 5254.001e.ad00/48, requester 'remote EID prefix'.>LISP Control-Plane receives the LISP Map-Request, which is for the MAC address of 10.47.4.3, consults the Ethernet server table for L2 LISP IID 8190 and send a LISP Map-Reply with the MAC-RLOC binding
Border-1#show monitor capture 1 buff display-filter lisp brief Starting the packet display ........ Press Ctrl + Shift + 6 to exit
250 28.656076 0.0.0.0 -> 0.0.0.0 LISP 176 Encapsulated Map-Request for [8190] 52:54:00:1e:ad:00/48 251 28.658851 10.47.1.10 -> 10.47.1.12 LISP 96 Map-Reply for [8190] 52:54:00:1e:ad:00/48
We can take a closer look at the LISP Map-Request and Map-Reply
Border-1#show monitor capture 1 buffer display-filter frame.number==250 detailed
Starting the packet display ........ Press Ctrl + Shift + 6 to exit
Frame 250: 176 bytes on wire (1408 bits), 176 bytes captured (1408 bits) on interface /tmp/epc_ws/wif_to_ts_pipe, id 0
Interface id: 0 (/tmp/epc_ws/wif_to_ts_pipe)
Interface name: /tmp/epc_ws/wif_to_ts_pipe
Encapsulation type: Ethernet (1)
Arrival Time: Oct 24, 2023 17:37:11.647755000 UTC
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1698169031.647755000 seconds
[Time delta from previous captured frame: 0.315724000 seconds]
[Time delta from previous displayed frame: 0.000000000 seconds]
[Time since reference or first frame: 28.656076000 seconds]
Frame Number: 250
Frame Length: 176 bytes (1408 bits)
Capture Length: 176 bytes (1408 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ethertype:ip:udp:lisp:ip:udp:lisp]
Ethernet II, Src: 52:54:00:04:84:b1 (52:54:00:04:84:b1), Dst: 52:54:00:0a:42:f3 (52:54:00:0a:42:f3)
Destination: 52:54:00:0a:42:f3 (52:54:00:0a:42:f3)
Address: 52:54:00:0a:42:f3 (52:54:00:0a:42:f3)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Source: 52:54:00:04:84:b1 (52:54:00:04:84:b1)
Address: 52:54:00:04:84:b1 (52:54:00:04:84:b1)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 10.47.1.12, Dst: 10.47.1.10 <-- Edge-1 RLOC and Border-1 RLOC, respectively
0100 .... = Version: 4
.... 0101 = Header Length: 20 bytes (5)
Differentiated Services Field: 0xc0 (DSCP: CS6, ECN: Not-ECT)
1100 00.. = Differentiated Services Codepoint: Class Selector 6 (48)
.... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
Total Length: 162
Identification: 0x75e5 (30181)
Flags: 0x0000
0... .... .... .... = Reserved bit: Not set
.0.. .... .... .... = Don't fragment: Not set
..0. .... .... .... = More fragments: Not set
Fragment offset: 0
Time to live: 255
Protocol: UDP (17)
Header checksum: 0x2e32 [validation disabled]
[Header checksum status: Unverified]
Source: 10.47.1.12
Destination: 10.47.1.10
User Datagram Protocol, Src Port: 4342, Dst Port: 4342
Source Port: 4342
Destination Port: 4342
Length: 142
Checksum: 0x46f1 [unverified]
[Checksum Status: Unverified]
[Stream index: 4]
[Timestamps]
[Time since first frame: 0.000000000 seconds]
[Time since previous frame: 0.000000000 seconds]
Locator/ID Separation Protocol
1000 .... .... .... .... .... = Type: Encapsulated Control Message (8)
.... 0... .... .... .... .... .... .... = S bit (LISP-SEC capable): Not set
.... .0.. .... .... .... .... .... .... = D bit (DDT-originated): Not set
.... ..00 0000 0000 0000 0000 0000 0000 = Reserved bits: 0x0000000
Internet Protocol Version 4, Src: 0.0.0.0, Dst: 0.0.0.0
0100 .... = Version: 4
.... 0101 = Header Length: 20 bytes (5)
Differentiated Services Field: 0xc0 (DSCP: CS6, ECN: Not-ECT)
1100 00.. = Differentiated Services Codepoint: Class Selector 6 (48)
.... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
Total Length: 130
Identification: 0x75e4 (30180)
Flags: 0x0000
0... .... .... .... = Reserved bit: Not set
.0.. .... .... .... = Don't fragment: Not set
..0. .... .... .... = More fragments: Not set
Fragment offset: 0
Time to live: 255
Protocol: UDP (17)
Header checksum: 0x44c7 [validation disabled]
[Header checksum status: Unverified]
Source: 0.0.0.0
Destination: 0.0.0.0
User Datagram Protocol, Src Port: 4342, Dst Port: 4342
Source Port: 4342
Destination Port: 4342
Length: 110
Checksum: 0x18bb [unverified]
[Checksum Status: Unverified]
[Stream index: 5]
[Timestamps]
[Time since first frame: 0.000000000 seconds]
[Time since previous frame: 0.000000000 seconds]
Locator/ID Separation Protocol
0001 .... .... .... .... .... = Type: Map-Request (1)
.... 0100 00.. .... .... .... = Flags: 0x10
.... 0... .... .... .... .... = A bit (Authoritative): Not set
.... .1.. .... .... .... .... = M bit (Map-Reply present): Set
.... ..0. .... .... .... .... = P bit (Probe): Not set
.... ...0 .... .... .... .... = S bit (Solicit-Map-Request): Not set
.... .... 0... .... .... .... = p bit (Proxy ITR): Not set
.... .... .0.. .... .... .... = s bit (SMR-invoked): Not set
.... .... ..00 0000 000. .... = Reserved bits: 0x000
.... .... .... .... ...0 0000 = ITR-RLOC Count: 0
Record Count: 1
Nonce: 0x86438e956066d3ca
Source EID AFI: LISP Canonical Address Format (LCAF) (16387)
Source EID: [8190] 00:00:0c:9f:f3:41
LCAF: Instance ID: 8190, Address: 00:00:0c:9f:f3:41
LCAF Header: 00000220000c
Reserved bits: 0x00
Flags: 0x00
Type: Instance ID (2)
Reserved bits: 0x20
Length: 12
Instance ID: 8190
Address AFI: 802 (includes all 802 media plus Ethernet) (6)
Address: 00:00:0c:9f:f3:41 (00:00:0c:9f:f3:41)
ITR-RLOC 1: 10.47.1.12
ITR-RLOC AFI: IPv4 (1)
ITR-RLOC Address: 10.47.1.12
Map-Request Record 1: [8190] 52:54:00:1e:ad:00/48 <-- Map-Request for this specific MAC address
Reserved: 0x00
Prefix Length: 48
Prefix AFI: LISP Canonical Address Format (LCAF) (16387)
Prefix: [8190] 52:54:00:1e:ad:00
LCAF: Instance ID: 8190, Address: 52:54:00:1e:ad:00
LCAF Header: 00000220000c
Reserved bits: 0x00
Flags: 0x00
Type: Instance ID (2)
Reserved bits: 0x20
Length: 12
Instance ID: 8190
Address AFI: 802 (includes all 802 media plus Ethernet) (6)
Address: 52:54:00:1e:ad:00 (52:54:00:1e:ad:00)
Map-Reply Record
Mapping Record 1, EID Prefix: [8190] 00:00:0c:9f:f3:41/48, TTL: 1440, Action: No-Action, Authoritative
Record TTL: 1440
Locator Count: 1
EID Mask Length: 48
000. .... .... .... = Action: No-Action (0)
...1 .... .... .... = Authoritative bit: Set
.... .000 0000 0000 = Reserved: 0x000
0000 .... .... .... = Reserved: 0x0
.... 0000 0000 0000 = Mapping Version: 0
EID Prefix AFI: LISP Canonical Address Format (LCAF) (16387)
EID Prefix: [8190] 00:00:0c:9f:f3:41
LCAF: Instance ID: 8190, Address: 00:00:0c:9f:f3:41
LCAF Header: 00000220000c
Reserved bits: 0x00
Flags: 0x00
Type: Instance ID (2)
Reserved bits: 0x20
Length: 12
Instance ID: 8190
Address AFI: 802 (includes all 802 media plus Ethernet) (6)
Address: 00:00:0c:9f:f3:41 (00:00:0c:9f:f3:41)
Locator Record 1, Local RLOC: 10.47.1.12, Reachable, Priority/Weight: 10/10, Multicast Priority/Weight: 10/10
Priority: 10
Weight: 10
Multicast Priority: 10
Multicast Weight: 10
Flags: 0x0005
0000 0000 0000 0... = Reserved: 0x0000
.... .... .... .1.. = Local: Set
.... .... .... ..0. = Probe: Not set
.... .... .... ...1 = Reachable: Set
AFI: IPv4 (1)
Locator: 10.47.1.12Border-1#show monitor capture 1 buffer display-filter frame.number==251 detailed
Starting the packet display ........ Press Ctrl + Shift + 6 to exit
Frame 251: 96 bytes on wire (768 bits), 96 bytes captured (768 bits) on interface /tmp/epc_ws/wif_to_ts_pipe, id 0
Interface id: 0 (/tmp/epc_ws/wif_to_ts_pipe)
Interface name: /tmp/epc_ws/wif_to_ts_pipe
Encapsulation type: Ethernet (1)
Arrival Time: Oct 24, 2023 17:37:11.650530000 UTC
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1698169031.650530000 seconds
[Time delta from previous captured frame: 0.002775000 seconds]
[Time delta from previous displayed frame: 0.000000000 seconds]
[Time since reference or first frame: 28.658851000 seconds]
Frame Number: 251
Frame Length: 96 bytes (768 bits)
Capture Length: 96 bytes (768 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ethertype:ip:udp:lisp]
Ethernet II, Src: 00:00:00:00:00:00 (00:00:00:00:00:00), Dst: 00:00:00:00:00:00 (00:00:00:00:00:00)
Destination: 00:00:00:00:00:00 (00:00:00:00:00:00)
Address: 00:00:00:00:00:00 (00:00:00:00:00:00)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Source: 00:00:00:00:00:00 (00:00:00:00:00:00)
Address: 00:00:00:00:00:00 (00:00:00:00:00:00)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 10.47.1.10, Dst: 10.47.1.12 <-- Border-1 RLOC, Edge-1 RLOC, respectively
0100 .... = Version: 4
.... 0101 = Header Length: 20 bytes (5)
Differentiated Services Field: 0xc0 (DSCP: CS6, ECN: Not-ECT)
1100 00.. = Differentiated Services Codepoint: Class Selector 6 (48)
.... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
Total Length: 82
Identification: 0x12a9 (4777)
Flags: 0x0000
0... .... .... .... = Reserved bit: Not set
.0.. .... .... .... = Don't fragment: Not set
..0. .... .... .... = More fragments: Not set
Fragment offset: 0
Time to live: 255
Protocol: UDP (17)
Header checksum: 0x91be [validation disabled]
[Header checksum status: Unverified]
Source: 10.47.1.10
Destination: 10.47.1.12
User Datagram Protocol, Src Port: 4342, Dst Port: 4342
Source Port: 4342
Destination Port: 4342
Length: 62
Checksum: 0xd63e [unverified]
[Checksum Status: Unverified]
[Stream index: 4]
[Timestamps]
[Time since first frame: 0.002775000 seconds]
[Time since previous frame: 0.002775000 seconds]
Locator/ID Separation Protocol
0010 .... .... .... .... .... = Type: Map-Reply (2)
.... 0... .... .... .... .... = P bit (Probe): Not set
.... .0.. .... .... .... .... = E bit (Echo-Nonce locator reachability algorithm enabled): Not set
.... ..0. .... .... .... .... = S bit (LISP-SEC capable): Not set
.... ...0 0000 0000 0000 0000 = Reserved bits: 0x00000
Record Count: 1
Nonce: 0x86438e956066d3ca
Mapping Record 1, EID Prefix: [8190] 52:54:00:1e:ad:00/48, TTL: 1440, Action: No-Action, Not Authoritative
Record TTL: 1440
Locator Count: 1
EID Mask Length: 48
000. .... .... .... = Action: No-Action (0)
...0 .... .... .... = Authoritative bit: Not set
.... .000 0000 0000 = Reserved: 0x000
0000 .... .... .... = Reserved: 0x0
.... 0000 0000 0000 = Mapping Version: 0
EID Prefix AFI: LISP Canonical Address Format (LCAF) (16387)
EID Prefix: [8190] 52:54:00:1e:ad:00
LCAF: Instance ID: 8190, Address: 52:54:00:1e:ad:00
LCAF Header: 00000220000c
Reserved bits: 0x00
Flags: 0x00
Type: Instance ID (2)
Reserved bits: 0x20
Length: 12
Instance ID: 8190
Address AFI: 802 (includes all 802 media plus Ethernet) (6)
Address: 52:54:00:1e:ad:00 (52:54:00:1e:ad:00)
Locator Record 1, RLOC: 10.47.1.13, Reachable, Priority/Weight: 10/10, Multicast Priority/Weight: 10/10
Priority: 10
Weight: 10
Multicast Priority: 10
Multicast Weight: 10
Flags: 0x0001
0000 0000 0000 0... = Reserved: 0x0000
.... .... .... .0.. = Local: Not set
.... .... .... ..0. = Probe: Not set
.... .... .... ...1 = Reachable: Set
AFI: IPv4 (1)
Locator: 10.47.1.13 <-- This RLOC owns the MAC address Border-1#debug lisp control-plane all
Border-1#debug l2lisp all
*Oct 24 18:03:00.361: LISP[TRNSP]-0: Processing received Map-Request(1) message on GigabitEthernet1/0/3 from 0.0.0.0:4342 to 0.0.0.0:4342. *Oct 24 18:03:00.361: LISP[MR ]-0: Received Map-Request with 1 records, first EID IID 8190 5254.001e.ad00/48, source EID 0000.0c9f.f341, nonce 0xAD90C521-0x348A478D. *Oct 24 18:03:00.361: LISP[MR ]-0 IID 8190 MAC: MS EID 5254.001e.ad00/48: Sending proxy reply to 10.47.1.12.
Edge-1 receives the LISP Map-Reply from Border-1
Edge-1#debug lisp control-plane all
Edge-1#debug l2lisp all
*Oct 24 17:11:24.558: LISP[TRNSP]-0: Processing received Map-Reply(2) message on GigabitEthernet1/0/1 from 10.47.1.10:4342 to 10.47.1.12:4342. *Oct 24 17:11:24.558: LISP[REMT ]-0: Received Map-Reply with nonce 0x38A78BA8-0xC378149D, 1 records. *Oct 24 17:11:24.558: LISP[REMT ]-0: Map-Reply nonce matches pending request for IID 8190 EID 5254.001e.ad00/48, requester 'remote EID prefix'. *Oct 24 17:11:24.558: LISP[REMT ]-0: Processing Map-Reply mapping record for IID 8190 MAC 5254.001e.ad00/48 LCAF 2, ttl 1440, action none, not authoritative, 1 locator. *Oct 24 17:11:24.559: LISP[REMT ]-0: 10.47.1.13 pri/wei/dID/mID/met/si_type/si_id/si_flg/afn_id=10/10/0/0/4294967295/none/0/UNSPEC/UNSPEC lpR.
The entire ARP Request through LISP/SISF exchange can be viewed on Edge-1 via CPU EPC
Edge-1#show monitor capture 1 buffer display-filter "arp.dst.proto_ipv4==10.47.4.3 or lisp" Starting the packet display ........ Press Ctrl + Shift + 6 to exit
120 18.415474 52:54:00:19:93:e9 -> ff:ff:ff:ff:ff:ff ARP 60 Who has 10.47.4.3? Tell 10.47.4.2 <-- Broadcast ARP Request punted up to the CPU 121 18.416092 10.47.4.3 -> 10.47.4.3 LISP 114 Encapsulated Map-Request for Unknown LCAF Type (53)/32 <-- LISP Map-Request to obtain the MAC address of 10.47.4.3 135 19.598041 10.47.1.11 -> 10.47.1.12 LISP 96 Map-Reply for Unknown LCAF Type (53)/32 <-- LISP Map-Reply providing the MAC address of 10.47.4.3 136 19.613072 0.0.0.0 -> 0.0.0.0 LISP 176 Encapsulated Map-Request for [8190] 52:54:00:1e:ad:00/48 <-- LISP Map-Request to obtain the RLOC for MAC address 5254.001e.ad00 138 20.119722 10.47.1.10 -> 10.47.1.12 LISP 96 Map-Reply for [8190] 52:54:00:1e:ad:00/48 <-- LISP Map-Reply for the RLOC that owns MAC address 5254.001e.ad00 143 20.477618 52:54:00:19:93:e9 -> 52:54:00:1e:ad:00 ARP 60 Who has 10.47.4.3? Tell 10.47.4.2 <-- Unicast ARP Request injected down from the CPU
Once the control-plane on Edge-1 has converged, there is a map-cache entry as well as a SISF Remote Entry (RMT)
Edge-1#show lisp instance-id 8190 ethernet map-cache 5254.001e.ad00
LISP MAC Mapping Cache for LISP 0 EID-table Vlan 1026 (IID 8190), 1 entries
5254.001e.ad00/48, uptime: 00:06:26, expires: 23:53:34, via map-reply, complete
Sources: map-reply
State: complete, last modified: 00:06:26, map-source: 10.47.1.13
Active, Packets out: 11(0 bytes), counters are not accurate (~ 00:00:00 ago)
Encapsulating dynamic-EID traffic
Locator Uptime State Pri/Wgt Encap-IID
10.47.1.13 00:06:26 up 10/10 -
Last up-down state change: 00:06:26, state change count: 1
Last route reachability change: 2w0d, state change count: 1
Last priority / weight change: never/never
RLOC-probing loc-status algorithm:
Last RLOC-probe sent: 00:06:25 (rtt 1104ms)
Edge-1#show device-tracking database address 10.47.4.3
Codes: L - Local, S - Static, ND - Neighbor Discovery, ARP - Address Resolution Protocol, DH4 - IPv4 DHCP, DH6 - IPv6 DHCP, PKT - Other Packet, API - API created
Preflevel flags (prlvl):
0001:MAC and LLA match 0002:Orig trunk 0004:Orig access
0008:Orig trusted trunk 0010:Orig trusted access 0020:DHCP assigned
0040:Cga authenticated 0080:Cert authenticated 0100:Statically assigned
Network Layer Address Link Layer Address Interface vlan prlvl age state Time left
RMT 10.47.4.3 5254.001e.ad00 L2LI0 1026 0005 7mn STALE try 0 731 s Next, the unicast ARP Request is injected down from the CPU. Remember, CPU injected packets cannot be captured in the egress direction with EPC on physical interfaces, Switchport Port Analyzer (SPAN) or an ingress EPC on the receiving Fabric Edge node can be used to confirm receipt of the VXLAN encapsulated unicast ARP Request
Edge-2 (10.47.1.13)
First, verify that the LISP or Tunnel interface is listed in the VLAN ID output for VLAN 1026
Edge-2#show vlan id 1026 VLAN Name Status Ports ---- -------------------------------- --------- ------------------------------- 1026 red active L2LI0:8190, Gi1/0/3 <-- L2 LISP IID is associated VLAN Type SAID MTU Parent RingNo BridgeNo Stp BrdgMode Trans1 Trans2 ---- ----- ---------- ----- ------ ------ -------- ---- -------- ------ ------ 1026 enet 101026 1500 - - - - - 0 0 Remote SPAN VLAN ---------------- Disabled Primary Secondary Type Ports ------- --------- ----------------- ------------------------------------------
Now, an ingress EPC on Edge-2 demonstrates the VXLAN encapsulated ARP Reply received. Since the unicast ARP Request is VXLAN encapsulated, you can leverage an IP ACL to match against Edge-1 RLOC send towards Edge-2 RLOC (10.47.1.12 towards 10.47.1.13, respectively) to help filter the traffic.
Edge-2(config)#ip access-list extended tac Edge-2(config-ext-nacl)#permit ip host 10.47.1.12 host 10.47.1.13
Edge-2#monitor capture 1 interface g1/0/1 in access-list tac Edge-2#monitor capture 1 interface g1/0/2 in access-list tac Edge-2#monitor capture 1 start Started capture point : 1 Edge-2#monitor capture 1 stop Capture statistics collected at software: Capture duration - 20 seconds Packets received - 10 Packets dropped - 0 Packets oversized - 0 Number of Bytes dropped at asic not collected Capture buffer will exists till exported or cleared Stopped capture point : 1
Edge-2#show monitor capture 1 buffer brief Starting the packet display ........ Press Ctrl + Shift + 6 to exit 1 0.000000 52:54:00:19:93:e9 -> 52:54:00:1e:ad:00 ARP 110 Who has 10.47.4.3? Tell 10.47.4.2
Upon closer inspection of this ARP Request, you can see there is VXLAN encapsulation, UDP header, and other headers, as an ARP frame is small, 60 bytes normally.
Edge-2#show monitor capture 1 buffer display-filter frame.number==1 detailed
Starting the packet display ........ Press Ctrl + Shift + 6 to exit
Frame 1: 110 bytes on wire (880 bits), 110 bytes captured (880 bits) on interface /tmp/epc_ws/wif_to_ts_pipe, id 0
Interface id: 0 (/tmp/epc_ws/wif_to_ts_pipe)
Interface name: /tmp/epc_ws/wif_to_ts_pipe
Encapsulation type: Ethernet (1)
Arrival Time: Oct 24, 2023 18:57:34.642468000 UTC
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1698173854.642468000 seconds
[Time delta from previous captured frame: 0.000000000 seconds]
[Time delta from previous displayed frame: 0.000000000 seconds]
[Time since reference or first frame: 0.000000000 seconds]
Frame Number: 1
Frame Length: 110 bytes (880 bits)
Capture Length: 110 bytes (880 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ethertype:ip:udp:vxlan:eth:ethertype:arp]
Ethernet II, Src: 52:54:00:0a:42:11 (52:54:00:0a:42:11), Dst: 52:54:00:17:fe:65 (52:54:00:17:fe:65)
Destination: 52:54:00:17:fe:65 (52:54:00:17:fe:65)
Address: 52:54:00:17:fe:65 (52:54:00:17:fe:65)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Source: 52:54:00:0a:42:11 (52:54:00:0a:42:11)
Address: 52:54:00:0a:42:11 (52:54:00:0a:42:11)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 10.47.1.12, Dst: 10.47.1.13 <-- Edge-1 RLOC and Edge-2 RLOC, respectively
0100 .... = Version: 4
.... 0101 = Header Length: 20 bytes (5)
Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
0000 00.. = Differentiated Services Codepoint: Default (0)
.... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
Total Length: 96
Identification: 0x798a (31114)
Flags: 0x4000, Don't fragment
0... .... .... .... = Reserved bit: Not set
.1.. .... .... .... = Don't fragment: Set
..0. .... .... .... = More fragments: Not set
Fragment offset: 0
Time to live: 253
Protocol: UDP (17)
Header checksum: 0xed8b [validation disabled]
[Header checksum status: Unverified]
Source: 10.47.1.12
Destination: 10.47.1.13
User Datagram Protocol, Src Port: 65354, Dst Port: 4789
Source Port: 65354
Destination Port: 4789
Length: 76
[Checksum: [missing]]
[Checksum Status: Not present]
[Stream index: 0]
[Timestamps]
[Time since first frame: 0.000000000 seconds]
[Time since previous frame: 0.000000000 seconds]
Virtual eXtensible Local Area Network
Flags: 0x8800, GBP Extension, VXLAN Network ID (VNI)
1... .... .... .... = GBP Extension: Defined
.... .... .0.. .... = Don't Learn: False
.... 1... .... .... = VXLAN Network ID (VNI): True
.... .... .... 0... = Policy Applied: False
.000 .000 0.00 .000 = Reserved(R): 0x0000
Group Policy ID: 0
VXLAN Network Identifier (VNI): 8190 <-- LISP L2 IID
Reserved: 0
Ethernet II, Src: 52:54:00:19:93:e9 (52:54:00:19:93:e9), Dst: 52:54:00:1e:ad:00 (52:54:00:1e:ad:00) <-- True source and destination endpoint MAC address
Destination: 52:54:00:1e:ad:00 (52:54:00:1e:ad:00)
Address: 52:54:00:1e:ad:00 (52:54:00:1e:ad:00)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Source: 52:54:00:19:93:e9 (52:54:00:19:93:e9)
Address: 52:54:00:19:93:e9 (52:54:00:19:93:e9)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Type: ARP (0x0806)
Trailer: 000000000000000000000000000000000000
Address Resolution Protocol (request)
Hardware type: Ethernet (1)
Protocol type: IPv4 (0x0800)
Hardware size: 6
Protocol size: 4
Opcode: request (1)
Sender MAC address: 52:54:00:19:93:e9 (52:54:00:19:93:e9)
Sender IP address: 10.47.4.2
Target MAC address: 00:00:00:00:00:00 (00:00:00:00:00:00)
Target IP address: 10.47.4.3Edge-2 pops the VXLAN encapsulation off and punt the unicast ARP Request up to the CPU for further processing. This can be seen via a FED Punject capture.
Edge-2#debug platform software fed switch active punt packet-capture start Punt packet capturing started. Edge-2#debug platform software fed switch active punt packet-capture stop Punt packet capturing stopped. Captured 21 packet(s) Edge-2#show platform software fed sw active punt packet-capture display-filter "arp" brief Punt packet capturing: disabled. Buffer wrapping: disabled Total captured so far: 21 packets. Capture capacity : 4096 packets ------ Punt Packet Number: 6, Timestamp: 2023/10/24 19:14:32.930 ------ interface : physical: [if-id: 0x00000000], pal: L2LISP0 [if-id: 0x00000017] metadata : cause: 109 [snoop packets], sub-cause: 1, q-no: 16, linktype: MCP_LINK_TYPE_IP [1] ether hdr : dest mac: 5254.001e.ad00, src mac: 5254.0019.93e9 ether hdr : ethertype: 0x0806 (ARP)
The ARP Request is sent to the ARP Snooper process.
Edge-2#debug platform software infrastructure punt
*Oct 24 19:18:38.916: PUNT RX: mcprp_process_receive_packet: pak->vlan_id: 1026 *Oct 24 19:18:38.916: Punt: IP proto src 147.233.10.47, dst 4.2.0.0, from table 0, intf L2LI0, encap LISP, size 60, cause snoop packets(L3) <-- You can see the 10.47.4.2 *Oct 24 19:18:38.916: punt cause:snoop packets invoking reg_invoke_mcprp_punt_feature_msg *Oct 24 19:18:38.916: punt cause:snoop packets MCPRP_PUNT_PAK_PROC_OK_DONE
Since this ARP Request comes from a L2 LISP/Tunnel interface, Edge-2 never learns 10.47.4.2 as a local endpoint of a dynamic EID in LISP
Edge-2#show platform arpsnooping client 5254.0019.93e9 PLAT_DAI : Platform DAI shim FWDPLANE : Dataplane forwarding BRIDGE : Packet to be bridged ARPSN : Arp Snooping Packet Trace for client MAC 5254.0019.93E9: Timestamp Sender Mac Sender IP Target Mac Target IP Opcode VLAN Handler:Action ----------------------- --------------- ------------- ----------------- -------------- ------------ ------- ------------------------------ 2023/10/24 15:57:01.129 5254.0019.93e9 10.47.4.2 0000.0000.0000 10.47.4.3 ARP_REQUEST 1026 PLATF_DAI:RECEIVED INPUT 2023/10/24 15:57:01.129 5254.0019.93e9 10.47.4.2 0000.0000.0000 10.47.4.3 ARP_REQUEST 1026 PLATF_DAI:RECEIVED_TUNNEL 2023/10/24 15:57:01.129 5254.0019.93e9 10.47.4.2 0000.0000.0000 10.47.4.3 ARP_REQUEST 1026 PLATF_DAI:SHUNTED
Now, the ARP Request is injected from the CPU down into VLAN 1026, specifically Gi1/0/3, where 10.47.4.3 is connected to.
Edge-2#show mac address-table address 5254.001e.ad00
Mac Address Table
-------------------------------------------
Vlan Mac Address Type Ports
---- ----------- -------- -----
1026 5254.001e.ad00 DYNAMIC Gi1/0/3
Total Mac Addresses for this criterion: 1
Edge-2#show platform arpsnooping client 5254.001e.ad00
PLAT_DAI : Platform DAI shim
FWDPLANE : Dataplane forwarding
BRIDGE : Packet to be bridged
ARPSN : Arp Snooping
Packet Trace for client MAC 5254.001E.AD00:
Timestamp Sender Mac Sender IP Target Mac Target IP Opcode VLAN Handler:Action
----------------------- --------------- ------------- ----------------- -------------- ------------ ------- ------------------------------
2023/10/24 15:57:01.129 5254.0019.93e9 10.47.4.2 0000.0000.0000 10.47.4.3 ARP_REQUEST 1026 INJECT:INJ_VLAN_IFINPUT_TO_BDI
2023/10/24 15:57:01.129 5254.0019.93e9 10.47.4.2 0000.0000.0000 10.47.4.3 ARP_REQUEST 1026 INJECT:BD_DPIDX_TO_FWDPLANEUnicast Path ARP Reply High Level Workflow
Unicast Path ARP Reply Verification
Edge-2 (10.47.1.13)
Endpoint that owns 10.47.4.3 responds with an unicast ARP Reply, the ARP Reply is punted up to the CPU due to the presence of IPDT. Initial verification occurs via EPC on the interface facing the endpoint.
Edge-2#show monitor capture 1 buffer display-filter arp
Starting the packet display ........ Press Ctrl + Shift + 6 to exit
2 88.712035 52:54:00:1e:ad:00 -> 00:00:0c:9f:f3:41 ARP 60 10.47.4.3 is at 52:54:00:1e:ad:00Next, verify the punt action with a FED Punject
Edge-2#debug platform software fed sw active punt packet-capture start Punt packet capturing started. Edge-2#debug platform software fed sw active punt packet-capture stop Punt packet capturing stopped. Captured 22 packet(s) Edge-2#show platform software fed sw active punt packet-capture display-filter "arp" brief Punt packet capturing: disabled. Buffer wrapping: disabled Total captured so far: 22 packets. Capture capacity : 4096 packets ------ Punt Packet Number: 6, Timestamp: 2023/10/24 20:32:35.634 ------ interface : physical: [if-id: 0x00000000], pal: L2LISP0 [if-id: 0x00000017] metadata : cause: 109 [snoop packets], sub-cause: 1, q-no: 16, linktype: MCP_LINK_TYPE_IP [1] <-- Punted for Snoop Packets to CPU queue 16 ether hdr : dest mac: 5254.001e.ad00, src mac: 5254.0019.93e9 ether hdr : ethertype: 0x0806 (ARP)
Next, the ARP reply is passed to ARP Snooper and Device-Tracking
Edge-2#debug platform software infrastructure punt
*Oct 24 19:18:39.101: PUNT RX: mcprp_process_receive_packet: pak->vlan_id: 1026 *Oct 24 19:18:39.101: Punt: IP proto src 173.0.10.47, dst 4.3.82.84, from table 0, intf Gi1/0/3, encap ARPA, size 60, cause snoop packets(L3) <-- 10.47.4.3 is obscured *Oct 24 19:18:39.101: punt cause:snoop packets invoking reg_invoke_mcprp_punt_feature_msg *Oct 24 19:18:39.101: punt cause:snoop packets MCPRP_PUNT_PAK_PROC_OK_DONE
Edge-2#debug platform fhs Edge-2#debug platform fhs all ARP packet received from ARP snooper(Gi1/0/3 10.47.4.3 (5254.001e.ad00) VLAN:10
Edge-2#debug device-tracking
*Oct 24 20:42:22.554: SISF[CLA]: Interest on target vlan 1026 *Oct 24 20:42:22.554: SISF[CLA]: feature Device-tracking *Oct 24 20:42:22.554: SISF[CLA]: feature Address Resolution Relay
*Oct 24 20:42:22.555: SISF[SWI]: Gi1/0/3 vlan 1026 Feature_0 Device-tracking priority 128 *Oct 24 20:42:22.555: SISF[SWI]: Gi1/0/3 vlan 1026 Feature_1 Address Resolution Relay priority 81 *Oct 24 20:42:22.555: SISF[PRS]: ARP-REPLY target set to 10.47.4.2 *Oct 24 20:42:22.556: SISF[SWI]: Gi1/0/3 vlan 1026 Feature Device-tracking rc: OK *Oct 24 20:42:22.556: SISF[ARR]: Gi1/0/3 vlan 1026 Receive a msg in AR *Oct 24 20:42:22.557: SISF[ARR]: Gi1/0/3 vlan 1026 Not ARP Request or NS, return OK *Oct 24 20:42:22.557: SISF[SWI]: Gi1/0/3 vlan 1026 Feature Address Resolution Relay rc: OK *Oct 24 20:42:22.557: SISF[SWI]: Gi1/0/3 vlan 1026 Features execution OK
Since the ARP reply already points to a real destination MAC address (not like a placeholder as seen temporarily on Edge-1) Edge-2 can trigger a LISP Map-Request to resolve RLOC-MAC association.
Edge-2#debug lisp control-plane all
Edge-2#debug l2lisp all
*Oct 24 20:47:34.400: LISP[REMT ]-0 IID 8190: Schedule processing of Map-Requests from 'remote EID prefix' in IPv4. *Oct 24 20:47:34.401: LISP[REMT ]-0: Map Request: Sending request for IID 8190 EID 5254.0019.93e9/48, requester 'remote EID prefix'. *Oct 24 20:47:35.166: LISP[TRNSP]-0: Processing received Map-Reply(2) message on GigabitEthernet1/0/1 from 10.47.1.11:4342 to 10.47.1.13:4342. *Oct 24 20:47:35.166: LISP[REMT ]-0: Received Map-Reply with nonce 0x5879579E-0xCAFC0AA5, 1 records. *Oct 24 20:47:35.166: LISP[REMT ]-0: Processing Map-Reply mapping record for IID 8190 MAC 5254.0019.93e9/48 LCAF 2, ttl 1440, action none, not authoritative, 1 locator. *Oct 24 20:47:35.166: LISP[REMT ]-0: 10.47.1.12 pri/wei/dID/mID/met/si_type/si_id/si_flg/afn_id=10/10/0/0/4294967295/none/0/UNSPEC/UNSPEC lpR.
Use the command show lisp instance-id <L2 IID> ethernet map-cache <destination MAC address> to verify which RLOC this ARP Reply is be sent towards with VXLAN encapsulation
Edge-2#show lisp instance-id 8190 ethernet map-cache 5254.0019.93e9
LISP MAC Mapping Cache for LISP 0 EID-table Vlan 1026 (IID 8190), 1 entries
5254.0019.93e9/48, uptime: 00:03:45, expires: 23:56:15, via map-reply, complete
Sources: map-reply
State: complete, last modified: 00:03:45, map-source: 10.47.1.12
Active, Packets out: 6(0 bytes), counters are not accurate (~ 00:00:59 ago)
Encapsulating dynamic-EID traffic
Locator Uptime State Pri/Wgt Encap-IID
10.47.1.12 00:03:45 up 10/10 -
Last up-down state change: 00:03:45, state change count: 1
Last route reachability change: 2w0d, state change count: 1
Last priority / weight change: never/never
RLOC-probing loc-status algorithm:
Last RLOC-probe sent: 00:03:45 (rtt 861ms)After LISP resolution, the ARP Reply can be injected from the CPU towards 10.47.1.12 RLOC in the underlay
Edge-2#show ip cef 10.47.1.12 10.47.1.12/32 nexthop 10.47.1.2 GigabitEthernet1/0/2 nexthop 10.47.1.6 GigabitEthernet1/0/1
The entire flow can be seen on Edge-2 CPU via EPC, the difference between an ARP Reply and ARP Request is that there is no LISP AR Resolution in this flow.
Edge-2#show monitor capture 1 buffer display-filter "arp.src.proto_ipv4==10.47.4.3 or lisp" Starting the packet display ........ Press Ctrl + Shift + 6 to exit 62 9.355185 52:54:00:1e:ad:00 -> 52:54:00:19:93:e9 ARP 60 10.47.4.3 is at 52:54:00:1e:ad:00 <-- ARP Reply punted up to the CPU 63 9.355486 0.0.0.0 -> 0.0.0.0 LISP 176 Encapsulated Map-Request for [8190] 52:54:00:19:93:e9/48 <-- LISP Map-Request to resolve RLOC-MAC association 88 12.058412 10.47.1.10 -> 10.47.1.13 LISP 96 Map-Reply for [8190] 52:54:00:19:93:e9/48 <-- LISP Map-Reply providing the RLOC-MAC association 90 12.072455 52:54:00:1e:ad:00 -> 52:54:00:19:93:e9 ARP 110 10.47.4.3 is at 52:54:00:1e:ad:00 <-- VXLAN Encapsulated ARP Reply that is injected by the CPU
You can use the command show platform arpsnooping client <source MAC address> to see the actions taken on Edge-2 in relation to the ARP Reply
Edge-2#show platform arpsnooping client 5254.001e.ad00 PLAT_DAI : Platform DAI shim FWDPLANE : Dataplane forwarding BRIDGE : Packet to be bridged ARPSN : Arp Snooping Packet Trace for client MAC 5254.001E.AD00: Timestamp Sender Mac Sender IP Target Mac Target IP Opcode VLAN Handler:Action ----------------------- --------------- ------------- ----------------- -------------- ------------ ------- ------------------------------
2023/10/24 20:47:38.151 5254.001e.ad00 10.47.4.3 5254.0019.93e9 10.47.4.2 ARP_REPLY 1026 PLATF_DAI:RECEIVED INPUT 2023/10/24 20:47:38.151 5254.001e.ad00 10.47.4.3 5254.0019.93e9 10.47.4.2 ARP_REPLY 1026 PLATF_DAI:RECEIVED 2023/10/24 20:47:38.152 5254.001e.ad00 10.47.4.3 5254.0019.93e9 10.47.4.2 ARP_REPLY 1026 PLATF_DAI:TO_ARPSND 2023/10/24 20:47:38.152 5254.001e.ad00 10.47.4.3 5254.0019.93e9 10.47.4.2 ARP_REPLY 1026 PSISF_PRE:PSP_INJNEUTRAL 2023/10/24 20:47:38.152 5254.001e.ad00 10.47.4.3 5254.0019.93e9 10.47.4.2 ARP_REPLY 1026 INJECT:INJ_VLAN_IFINPUT_TO_BDI 2023/10/24 20:47:38.152 5254.001e.ad00 10.47.4.3 5254.0019.93e9 10.47.4.2 ARP_REPLY 1026 INJECT:BD_DPIDX_TO_FWDPLANE
Edge-1 (10.47.1.12)
Edge-1 receives the VXLAN encapsulated ARP Reply, pop VXLAN header off and punt the ARP Reply up to the CPU for further processing.
Edge-1#debug platform software infrastructure punt
*Oct 24 21:42:11.303: PUNT RX: mcprp_process_receive_packet: pak->vlan_id: 1026 *Oct 24 21:42:11.303: Punt: IP proto src 173.0.10.47, dst 4.3.82.84, from table 0, intf L2LI0, encap LISP, size 60, cause snoop packets(L3) <-- Can see 10.47.4.3 IP address that has been obscured *Oct 24 21:42:11.303: punt cause:snoop packets invoking reg_invoke_mcprp_punt_feature_msg *Oct 24 21:42:11.303: punt cause:snoop packets MCPRP_PUNT_PAK_PROC_OK_DONE
You can use the command show platform arpsnooping client <source MAC address> to get additional information about how the ARP Reply is handled on Edge-1
Edge-1#show platform arpsnooping client 5254.001e.ad00 PLAT_DAI : Platform DAI shim FWDPLANE : Dataplane forwarding BRIDGE : Packet to be bridged ARPSN : Arp Snooping Packet Trace for client MAC 5254.001E.AD00: Timestamp Sender Mac Sender IP Target Mac Target IP Opcode VLAN Handler:Action ----------------------- --------------- ------------- ----------------- -------------- ------------ ------- ------------------------------
2023/10/24 20:40:33.741 5254.001e.ad00 10.47.4.3 5254.0019.93e9 10.47.4.2 ARP_REPLY 1026 PLATF_DAI:RECEIVED INPUT 2023/10/24 20:40:33.741 5254.001e.ad00 10.47.4.3 5254.0019.93e9 10.47.4.2 ARP_REPLY 1026 PLATF_DAI:RECEIVED_TUNNEL 2023/10/24 20:40:33.741 5254.001e.ad00 10.47.4.3 5254.0019.93e9 10.47.4.2 ARP_REPLY 1026 PLATF_DAI:SHUNTED 2023/10/24 20:40:33.741 5254.001e.ad00 10.47.4.3 5254.0019.93e9 10.47.4.2 ARP_REPLY 1026 INJECT:INJ_VLAN_IFINPUT_TO_BDI 2023/10/24 20:40:33.741 5254.001e.ad00 10.47.4.3 5254.0019.93e9 10.47.4.2 ARP_REPLY 1026 INJECT:BD_DPIDX_TO_FWDPLANE
Edge-1 takes the punted ARP Reply and send the ARP Reply into VLAN 1026, to the port where the endpoint 10.47.4.2 lives
Edge-1#show mac address-table address 5254.0019.93e9
Mac Address Table
-------------------------------------------
Vlan Mac Address Type Ports
---- ----------- -------- -----
1026 5254.0019.93e9 DYNAMIC Gi1/0/3
Total Mac Addresses for this criterion: 1ARP Flooding (L2 Flooding) Path
With L2 Flooding, ARP resolution can also occur where all Fabric Edge nodes in the fabric as well as L2 Handoff/IP Directed Broadcast enabled Borders join a common underlay multicast group. Every single time a packet/frame that is eligible to be flooded arrives to an Edge node, it is VXLAN encapsulated with the destination IP address of the underlay multicast group. L2 Flooding can apply to ARP resolution in certain scenarios:
- The command flood arp-nd is configured under the L2 LISP instance for the VLAN, broadcast ARP frames are flooded to all Fabric Edges using the broadcast-underlay multicast group.
- The command flood arp-nd is configured bt Cisco Catalyst Center when L2 flooding is enabled in a pool, and the pool is not marked as a wireless pool
- Underlay multicast must be configured either through LAN Automation or manual configuration. None of the fabric multicast workflows configure underlay multicast automatically.
Once underlay multicast is configured and flood arp-nd is enabled, this changes how the ARP Request is handled, which originally used LISP/SISF based resolution. Once flood arp-nd is configured under the L2 LISP instance, this disables the LISP-ARP-RELAY-VLAN IPDT policy for the VLAN, and is not used.
Edge-1#show device-tracking policies vlan 1026 Target Type Policy Feature Target range vlan 1026 VLAN DT-PROGRAMMATIC Device-tracking vlan all vlan 1026 VLAN LISP-DT-GLEAN-VLAN-MULTI-IP Device-tracking vlan all
Flooding Path ARP Request Verification
Edge-1 (10.47.1.12)
After an ARP Request is received from the client in L2 flooding enabled VLAN with flood arp-nd configured, it no longer is handled by ARP snooping. The ARP Request is punted up to the CPU for IPDT learning purposes but not for forwarding.
ARP Snooper does not process the ARP Request, which can be seen with the command show platform arpsnooping client <MAC address>
Edge-1#show platform arpsnooping client 5254.0019.93e9
PLAT_DAI : Platform DAI shim
FWDPLANE : Dataplane forwarding
BRIDGE : Packet to be bridged
ARPSN : Arp Snooping
Packet Trace for client MAC 5254.0019.93E9:
Timestamp Sender Mac Sender IP Target Mac Target IP Opcode VLAN Handler:Action
----------------------- --------------- ------------- ----------------- -------------- ------------ ------- ------------------------------
Filtered entries counters:
ARPSN_FILTER_SVI: 0
Edge-1 does not create a RMT IPDT entry for the endpoint 10.47.4.3 as evidenced in the output
Edge-1#show monitor capture 1 buffer display-filter arp brief
Starting the packet display ........ Press Ctrl + Shift + 6 to exit
1 0.000000 52:54:00:19:93:e9 -> ff:ff:ff:ff:ff:ff ARP 60 Who has 10.47.4.3? Tell 10.47.4.2
Edge-1#show device-tracking database address 10.47.4.3
Codes: L - Local, S - Static, ND - Neighbor Discovery, ARP - Address Resolution Protocol, DH4 - IPv4 DHCP, DH6 - IPv6 DHCP, PKT - Other Packet, API - API created
Preflevel flags (prlvl):
0001:MAC and LLA match 0002:Orig trunk 0004:Orig access
0008:Orig trusted trunk 0010:Orig trusted access 0020:DHCP assigned
0040:Cga authenticated 0080:Cert authenticated 0100:Statically assigned
Network Layer Address Link Layer Address Interface vlan prlvl age state Time left Now, the ARP Request is VXLAN encapsulated into the broadcast underlay multicast group. Edge-1 has a mroute with Loopback0 as the source, and the group the broadcast underlay group.
Edge-1#show run int lo0 Building configuration... Current configuration : 135 bytes ! interface Loopback0 ip address 10.47.1.12 255.255.255.255 no ip redirects ip pim sparse-mode <-- PIM must be enabled ip router isis clns mtu 1400 end
Edge-1#show ip mroute 239.0.17.2
IP Multicast Routing Table
Flags: D - Dense, S - Sparse, B - Bidir Group, s - SSM Group, C - Connected,
L - Local, P - Pruned, R - RP-bit set, F - Register flag,
T - SPT-bit set, J - Join SPT, M - MSDP created entry, E - Extranet,
X - Proxy Join Timer Running, A - Candidate for MSDP Advertisement,
U - URD, I - Received Source Specific Host Report,
Z - Multicast Tunnel, z - MDT-data group sender,
Y - Joined MDT-data group, y - Sending to MDT-data group,
G - Received BGP C-Mroute, g - Sent BGP C-Mroute,
N - Received BGP Shared-Tree Prune, n - BGP C-Mroute suppressed,
Q - Received BGP S-A Route, q - Sent BGP S-A Route,
V - RD & Vector, v - Vector, p - PIM Joins on route,
x - VxLAN group, c - PFP-SA cache created entry,
* - determined by Assert, # - iif-starg configured on rpf intf,
e - encap-helper tunnel flag, l - LISP decap ref count contributor
Outgoing interface flags: H - Hardware switched, A - Assert winner, p - PIM Join
t - LISP transit group
Timers: Uptime/Expires
Interface state: Interface, Next-Hop or VCD, State/Mode
(*, 239.0.17.2), 5w1d/00:02:05, RP 10.47.1.14, flags: SJC
Incoming interface: GigabitEthernet1/0/2, RPF nbr 10.47.1.4
Outgoing interface list:
L2LISP0.8190, Forward/Sparse-Dense, 01:56:41/00:00:18, flags:
L2LISP0.8192, Forward/Sparse-Dense, 2w2d/00:00:58, flags:
L2LISP0.8188, Forward/Sparse-Dense, 5w1d/00:01:58, flags:
(10.47.1.12, 239.0.17.2), 00:02:53/00:00:06, flags: PFT <-- Lo0 interface of Edge-1
Incoming interface: Null0, RPF nbr 0.0.0.0, <-- Incoming interface Null0 is expected
Outgoing interface list:
GigabitEthernet1/0/2, Forward/Sparse, 00:04:40/00:02:45, flags: <-- Outgoing interface Gig1/0/2 faces the fabric underlay In reality, flood arp-nd toggles IPDT/SISF rather than flooding itself. L2 flooding already floods broadcasts but the trick is to disable the LISP AR policy from device-tracking, ownership of ARP is now released and can be flooded just like any other broadcast.
To verify hardware programming for L2 flooding, use the command show platform software dpidb l2lisp <L2 LISP IID>
Edge-1#show platform software dpidb l2lisp 8190 Instance Id:8190, dpidx:25, vlan:1026, Parent Interface:L2LISP0(if_id:23) <-- dpidx value used in the next command
Take the dpidx value from the previous command and use in the command show platform software fed switch active ifm if-id <dpidx value>
Edge-1#show platform software fed switch active ifm if-id 25 Interface IF_ID : 0x0000000000000019 Interface Name : L2LISP0.8190 Interface Block Pointer : 0x7f65ec85ba78 Interface Block State : READY Interface State : Enabled Interface Status : ADD, UPD Interface Ref-Cnt : 2 Interface Type : L2_LISP Created Time : 2023/09/19 17:57:32.046 Last Modified Time : 2023/10/25 17:59:09.265 Current Time : 2023/10/25 20:15:44.624 Is top interface : FALSE Asic_num : 0 Switch_num : 0 AAL port Handle : 7a00003a Parent interface id : 17 Multicast Tunnel IP : 239.0.17.2 Mcast Tunnel Handle : 0x7f65ed356918 L2 Multicast Tunnel IP : 0.0.0.0 L2 Multicast Vlan Id : 0 L2 Multicast Tunnel Hdl : NULL Vlan Id : 1026 Instance Id : 8190 Dest Port : 4789 SGT : Enable Underlay VRF (V4) : 0 Underlay VRF (V6) : 0 Flood Access-tunnel : Disable Flood unknown ucast : Enable Broadcast : Enable Multicast Flood : Enable L2 Multicast Flood : Disable Host Activity report: Enabled <snip>
You can utilize EPC on Gi1/0/2 in the egress direction, because this ARP Request is forwarded in and out without requiring a CPU Inject, you can trust EPC captures in the egress decision this time.
Edge-1#monitor capture 1 start Started capture point : 1 Edge-1#monitor capture 1 stop Capture statistics collected at software: Capture duration - 22 seconds Packets received - 5 Packets dropped - 0 Packets oversized - 0 Number of Bytes dropped at asic not collected Capture buffer will exists till exported or cleared Stopped capture point : 1
Edge-1#show monitor capture 1 buffer brief Starting the packet display ........ Press Ctrl + Shift + 6 to exit 1 0.000000 52:54:00:19:93:e9 -> ff:ff:ff:ff:ff:ff ARP 110 Who has 10.47.4.3? Tell 10.47.4.2 <-- Size 110 because VXLAN, UDP, and other headers
You can take a closer look at the VXLAN encapsulated ARP Request
Edge-1#show monitor capture 1 buffer display-filter frame.number==1 detailed
Starting the packet display ........ Press Ctrl + Shift + 6 to exit
Frame 1: 110 bytes on wire (880 bits), 110 bytes captured (880 bits) on interface /tmp/epc_ws/wif_to_ts_pipe, id 0
Interface id: 0 (/tmp/epc_ws/wif_to_ts_pipe)
Interface name: /tmp/epc_ws/wif_to_ts_pipe
Encapsulation type: Ethernet (1)
Arrival Time: Oct 25, 2023 20:44:36.578645000 UTC
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1698266676.578645000 seconds
[Time delta from previous captured frame: 0.000000000 seconds]
[Time delta from previous displayed frame: 0.000000000 seconds]
[Time since reference or first frame: 0.000000000 seconds]
Frame Number: 1
Frame Length: 110 bytes (880 bits)
Capture Length: 110 bytes (880 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ethertype:ip:udp:vxlan:eth:ethertype:arp]
Ethernet II, Src: 00:00:00:00:00:00 (00:00:00:00:00:00), Dst: 00:00:00:00:00:00 (00:00:00:00:00:00) <-- Ignore the all 0s MAC, not accurate
Destination: 00:00:00:00:00:00 (00:00:00:00:00:00)
Address: 00:00:00:00:00:00 (00:00:00:00:00:00)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Source: 00:00:00:00:00:00 (00:00:00:00:00:00)
Address: 00:00:00:00:00:00 (00:00:00:00:00:00)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 10.47.1.12, Dst: 239.0.17.2 <-- Source is Edge-1 RLOC, Destination is the broadcast underlay group
0100 .... = Version: 4
.... 0101 = Header Length: 20 bytes (5)
Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
0000 00.. = Differentiated Services Codepoint: Default (0)
.... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
Total Length: 96
Identification: 0x8dab (36267)
Flags: 0x4000, Don't fragment
0... .... .... .... = Reserved bit: Not set
.1.. .... .... .... = Don't fragment: Set
..0. .... .... .... = More fragments: Not set
Fragment offset: 0
Time to live: 255
Protocol: UDP (17)
Header checksum: 0xe2a3 [validation disabled]
[Header checksum status: Unverified]
Source: 10.47.1.12
Destination: 239.0.17.2
User Datagram Protocol, Src Port: 65280, Dst Port: 4789
Source Port: 65280
Destination Port: 4789
Length: 76
[Checksum: [missing]]
[Checksum Status: Not present]
[Stream index: 0]
[Timestamps]
[Time since first frame: 0.000000000 seconds]
[Time since previous frame: 0.000000000 seconds]
Virtual eXtensible Local Area Network
Flags: 0x8800, GBP Extension, VXLAN Network ID (VNI)
1... .... .... .... = GBP Extension: Defined
.... .... .0.. .... = Don't Learn: False
.... 1... .... .... = VXLAN Network ID (VNI): True
.... .... .... 0... = Policy Applied: False
.000 .000 0.00 .000 = Reserved(R): 0x0000
Group Policy ID: 0
VXLAN Network Identifier (VNI): 8190 <-- L2 LISP IID
Reserved: 0
Ethernet II, Src: 52:54:00:19:93:e9 (52:54:00:19:93:e9), Dst: ff:ff:ff:ff:ff:ff (ff:ff:ff:ff:ff:ff) <-- SMAC and DMAC of the ARP Request
Destination: ff:ff:ff:ff:ff:ff (ff:ff:ff:ff:ff:ff)
Address: ff:ff:ff:ff:ff:ff (ff:ff:ff:ff:ff:ff)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...1 .... .... .... .... = IG bit: Group address (multicast/broadcast)
Source: 52:54:00:19:93:e9 (52:54:00:19:93:e9)
Address: 52:54:00:19:93:e9 (52:54:00:19:93:e9)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Type: ARP (0x0806)
Trailer: 000000000000000000000000000000000000
Address Resolution Protocol (request)
Hardware type: Ethernet (1)
Protocol type: IPv4 (0x0800)
Hardware size: 6
Protocol size: 4
Opcode: request (1)
Sender MAC address: 52:54:00:19:93:e9 (52:54:00:19:93:e9)
Sender IP address: 10.47.4.2
Target MAC address: 00:00:00:00:00:00 (00:00:00:00:00:00)
Target IP address: 10.47.4.3
Edge-2 (10.47.1.13)
Edge-2 joins the broadcast underlay group, 239.0.17.2 and has S,G for Edge-1, it receives the VXLAN encapsulated multicast packet on Gig1/0/1 and LISP0.8190 sub-interface is in the outgoing interface list. Earlier versions of code like 17.3 or older use a Tunnel interface instead of a LISP0 sub-interface.
Edge-2#show ip mroute 239.0.17.2
IP Multicast Routing Table
Flags: D - Dense, S - Sparse, B - Bidir Group, s - SSM Group, C - Connected,
L - Local, P - Pruned, R - RP-bit set, F - Register flag,
T - SPT-bit set, J - Join SPT, M - MSDP created entry, E - Extranet,
X - Proxy Join Timer Running, A - Candidate for MSDP Advertisement,
U - URD, I - Received Source Specific Host Report,
Z - Multicast Tunnel, z - MDT-data group sender,
Y - Joined MDT-data group, y - Sending to MDT-data group,
G - Received BGP C-Mroute, g - Sent BGP C-Mroute,
N - Received BGP Shared-Tree Prune, n - BGP C-Mroute suppressed,
Q - Received BGP S-A Route, q - Sent BGP S-A Route,
V - RD & Vector, v - Vector, p - PIM Joins on route,
x - VxLAN group, c - PFP-SA cache created entry,
* - determined by Assert, # - iif-starg configured on rpf intf,
e - encap-helper tunnel flag, l - LISP decap ref count contributor
Outgoing interface flags: H - Hardware switched, A - Assert winner, p - PIM Join
t - LISP transit group
Timers: Uptime/Expires
Interface state: Interface, Next-Hop or VCD, State/Mode
(*, 239.0.17.2), 5w1d/stopped, RP 10.47.1.14, flags: SJC
Incoming interface: GigabitEthernet1/0/1, RPF nbr 10.47.1.6
Outgoing interface list:
L2LISP0.8190, Forward/Sparse-Dense, 02:28:57/00:01:02, flags:
L2LISP0.8192, Forward/Sparse-Dense, 2w2d/00:00:32, flags:
L2LISP0.8188, Forward/Sparse-Dense, 5w1d/00:02:54, flags:
(10.47.1.12, 239.0.17.2), 00:00:03/00:02:56, flags: JT
Incoming interface: GigabitEthernet1/0/1, RPF nbr 10.47.1.6 <-- Interface that faces the fabric underlay and the RPF interface towards 10.47.1.12
Outgoing interface list:
L2LISP0.8188, Forward/Sparse-Dense, 00:00:03/00:02:56, flags:
L2LISP0.8192, Forward/Sparse-Dense, 00:00:03/00:02:56, flags:
L2LISP0.8190, Forward/Sparse-Dense, 00:00:03/00:02:56, flags: Edge-2 receives the VXLAN encapsulated packet, pop the VXLAN header off and flood the packet into VLAN 1026, which can be seen in an EPC on the incoming interface as well as the interface facing the endpoint.
Edge-2#monitor capture 1 interface gig1/0/1 in match any Edge-2#monitor capture 1 int g1/0/3 out Edge-2#monitor capture 1 start Started capture point : 1 Edge-2#monitor capture 1 stop Capture statistics collected at software: Capture duration - 22 seconds Packets received - 43 Packets dropped - 0 Packets oversized - 0 Number of Bytes dropped at asic not collected Capture buffer will exists till exported or cleared Stopped capture point : 1 Edge-2#show monitor capture 1 buffer display-filter arp Starting the packet display ........ Press Ctrl + Shift + 6 to exit 10 6.230153 52:54:00:19:93:e9 -> ff:ff:ff:ff:ff:ff ARP 110 Who has 10.47.4.3? Tell 10.47.4.2 <-- Size 110 is the VXLAN encapsulated ARP Request 11 6.404781 52:54:00:19:93:e9 -> ff:ff:ff:ff:ff:ff ARP 60 Who has 10.47.4.3? Tell 10.47.4.2 <-- Size 60 is the original ARP Request
Flooding Path ARP Reply Verification
ARP Replies are almost always unicast, unless they are gratuitous ARPs. For unicast ARP Replies, there is no distinction between the workflow for Unicast Path based on LISP/SISF or the Flooding Path based on flood arp-nd, both use the same unicast path which includes SISF/IPDT detection. You can leverage the Unicast Path ARP Reply Verification section for troubleshooting.
Revision History
| Revision | Publish Date | Comments |
|---|---|---|
2.0 |
26-Oct-2023 |
Initial Release |
1.0 |
12-Aug-2020 |
Initial Release |
FeedbackContact Cisco
- Open a Support Case
- (Requires a Cisco Service Contract)