EX硬體：ACI資料包轉發深入分析。
目錄
簡介
本文檔介紹在應用中心基礎設施(ACI)中使用基於「EX」的ACI交換機的不同轉發方案。它將說明如何驗證硬體是否已正確程式設計，以及我們正在將資料包轉發到相應終端組(EPG)中的正確目標終端(EP)。
必要條件
需求
本文件沒有特定需求。
採用元件
本檔案中的資訊是根據以下硬體和軟體版本：
- ACI交換矩陣，包括使用EX硬體的兩個主幹交換機和兩個枝葉交換機
- 一台帶有兩個上行鏈路的ESXi主機，分別連線到每台枝葉交換機
- 充當路由器的Nexus 5000裝置。
- 用於初始設定的應用策略基礎設施控制器(APIC)
本文中的資訊是根據特定實驗室環境內的裝置所建立。文中使用到的所有裝置皆從已清除（預設）的組態來啟動。如果您的網路正在作用，請確保您已瞭解任何指令可能造成的影響。
案例
2個EP位於同一EPG/相同枝葉 — 交換幀中
拓撲
根據此拓撲，從EP1到EP2的流是L2流，應在源流量進入的任何枝葉上本地交換。 要檢查第2層(L2)資料流，首先要檢查mac地址表，以確定交換機是否接收幀以及在何處收到幀：
leaf4# show mac address-table | grep fccc * 30 0050.56a5.fccc dynamic - F F po3 leaf4# show mac address-table | grep 6794 * 30 0050.56a5.6794 dynamic - F F po4
若要檢視封裝VLAN，您還可以檢查EP資料庫：
leaf4# show endpoint mac 0050.56a5.fccc
Legend:
O - peer-attached H - vtep a - locally-aged S - static
V - vpc-attached p - peer-aged L - local M - span
s - static-arp B - bounce
+-----------------------------------+---------------+-----------------+--------------+-------------+
VLAN/ Encap MAC Address MAC Info/ Interface
Domain VLAN IP Address IP Info
+-----------------------------------+---------------+-----------------+--------------+-------------+
30 vlan-2268 0050.56a5.fccc LV po3
Joey-Tenant:Joey-Internal vlan-2268 192.168.20.2 LV po3
calo2-leaf4# show endpoint mac 0050.56a5.6794
Legend:
O - peer-attached H - vtep a - locally-aged S - static
V - vpc-attached p - peer-aged L - local M - span
s - static-arp B - bounce
+-----------------------------------+---------------+-----------------+--------------+-------------+
VLAN/ Encap MAC Address MAC Info/ Interface
Domain VLAN IP Address IP Info
+-----------------------------------+---------------+-----------------+--------------+-------------+
30 vlan-2268 0050.56a5.6794 LV po4
Joey-Tenant:Joey-Internal vlan-2268 192.168.20.3 LV po4
我們知道FD_VLAN 30匹配，但我們可以始終在軟體中驗證對映：
leaf4# show vlan extended | grep 2268 30 enet CE vlan-2268
當然，我們可以檢查硬體，以確保VLAN 30對映到作為前面板封裝的VLAN 2268。
leaf4# vsh_lc
module-1# show system internal eltmc info vlan 30 vlan_id: 30 ::: hw_vlan_id: 22 vlan_type: FD_VLAN ::: bd_vlan: 28 access_encap_type: 802.1q ::: access_encap: 2268 fabric_encap_type: VXLAN ::: fabric_encap: 11960 sclass: 32778 ::: scope: 11 untagged: 0 acess_encap_hex: 0x8dc ::: fabric_enc_hex: 0x2eb8 pd_vlan_ft_mask: 0x8 fd_learn_disable: 0 qos_class_id: 0 ::: qos_pap_id: 0 qq_met_ptr: 25 ::: ipmc_index: 0 ingressBdAclLabel: 0 ::: ingBdAclLblMask: 0 egressBdAclLabel: 0 ::: egrBdAclLblMask: 0 qos_map_idx: 0 ::: qos_map_pri: 0 qos_map_dscp: 0 ::: qos_map_tc: 0 vlan_ft_mask: 0xe30 hw_bd_idx: 0 ::: hw_epg_idx: 11267 intf_count: 2 ::: glbl_scp_if_cnt: 2 <SNIPPED>
鑑於已經從軟體中學習了電子產品，我們還可以驗證硬體是否也對這些電子產品的二級資訊進行程式設計。 在新硬體中，硬體抽象層(HAL)是硬體的軟體狀態。 HAL的工作是接受軟體程式設計請求，並將其推送到硬體上。
為了檢視有關終端的L2硬體資訊，我們可以檢視HAL中給定mac地址的L2表：
leaf4# vsh_lc
module-1# show platform internal hal ep l2 mac 0050.56a5.fccc LEGEND: ------- BDId: BD Id BD Name: BD Name T: EP Type (Pl: Physical Vl: Virtual Xr: Remote EP Mac: Mac L2 IfId: L2 Interface L2 IfName: L2 IfName FDId: FD Id FD Name: FD Name S Class: S Class Age Intvl: Age Interval P A: Packet Action (F: Forward, T: Trap to CPU, L: Log & Forward, D: Drop, N: None) S T: Static Ep S E: Secure EP L D: Learn Disable B N D: Bind Notify Disable E N D: Epg Notify Disable B E: Bounce Enable I D L: IVxlan Dont Learn SPI: Source Policy Incomplete DPI: Dest Policy Incomplete SPA: Source Policy Applied DPA: Dest Policy Applied DSS: Dest Shared Service IL: Is Local VUB: Vnid Use Bd SO: SA Only L2 EP Count: 1 ====================================================================================================================== B E I S D S D D V BD EP L2 L2 FD S Age P S S L N N B D P P P P S I U S BdId Name T Mac IfId Ifname FDId Name Class Intvl A T E D D D E L I I A A S L B O ====================================================================================================================== 1c BD-28 Pl 00:50:56:a5:fc:cc 16000002 Po3 1e FD-30 800a 29f F 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 module-1# show platform internal hal ep l2 mac 0050.56a5.6794 ====================================================================================================================== B E I S D S D D V BD EP L2 L2 FD S Age P S S L N N B D P P P P S I U S BdId Name T Mac IfId Ifname FDId Name Class Intvl A T E D D D E L I I A A S L B O ====================================================================================================================== 1c BD-28 Pl 00:50:56:a5:67:94 16000003 Po4 1e FD-30 800a 29f F 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0
現在我們已經規劃了硬體，讓我們來執行ELAM並檢視資料包應該傳送到哪裡。
伊拉姆語
leaf4# vsh_lc
module-1# debug platform internal tah elam asic 0 module-1(DBG-TAH-elam)# trigger reset module-1(DBG-TAH-elam)# trigger init in-select 6 out-select 0 module-1(DBG-TAH-elam-insel6)# set outer l2 src_mac 0050.56a5.fccc dst_mac 0050.56a5.6794 module-1(DBG-TAH-elam-insel6)# start module-1(DBG-TAH-elam-insel6)# stat ELAM STATUS =========== Asic 0 Slice 0 Status Armed Asic 0 Slice 1 Status Triggered module-1(DBG-TAH-elam-insel6)# report | grep ovec sug_elam_out_sidebnd_no_spare_vec.ovector_idx: 0x9E
很好，因此Leaf4在Asic 0片1上收到該幀。在新硬體上使用ELAM時，出現了一個在故障排除時非常重要的新的欄位：ovector_idx。 此索引是轉發幀/資料包時應使用的物理埠索引。 收到ovector_idx後，我們可以使用此命令查詢它對映到的埠：
module-1(DBG-TAH-elam-insel6)# show platform internal hal l2 port gpd
Legend:
-------
IfId: Interface Id IfName: Interface Name
I P: Is PC Mbr IfId: Interface Id
Uc PC Cfg: UcPcCfg Idx Uc PC MbrId: Uc Pc Mbr Id
As: Asic AP: Asic Port
Sl: Slice Sp: Slice Port
Ss: Slice SrcId Ovec: Ovector (slice | srcid)
L S: Local Slot Reprogram:
L3: Is L3
P: PifTable Xla Idx: Xlate Idx
RP: Rw PifTable Ovx Idx: OXlate Idx
IP: If Profile Table N L3: Num. of L3 Ifs
RS: Rw SrcId Table NI L3: Num. of Infra L3 Ifs
DP: DPort Table Vif Tid: Vif Tid
SP: SrcPortState Table RwV Tid: RwVif Tid
RSP: RwSrcPortstate Table Ing Lbl: Ingress Acl Label
UC: UCPcCfg Egr Lbl: Egress Acl Label
UM: UCPcMbr Reprogram:
PROF ID: Lport Profile Id
VS: VifStateTable HI: LportProfile Hw Install
RV: Rw VifTable
Num. of Sandboxes: 1
Sandbox_ID: 0, BMP: 0x0
Port Count: 8
============================================================================================================================================
Uc Uc | Reprogram | | Rep |
I PC Pc L | R I R D R U U X | L Xla Ovx N NI Vif RwV Ing Egr | V R | PROF H
IfId Ifname P Cfg MbrID As AP Sl Sp Ss Ovec S | P P P S P Sp Sp C M L | 3 Idx Idx L3 L3 Tid Tid Lbl Lbl | S V | ID I
=============================================================================================================================================
1a004000 Eth1/5 1 0 1d 0 d 0 c 18 18 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 - - 800 0 0 1 0 0
1a005000 Eth1/6 1 0 b 0 e 0 d 1a 1a 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 - - 800 0 0 1 0 0
1a006000 Eth1/7 0 26 5 0 f 0 e 1c 1c 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 D-256 - 800 0 0 1 e 0
1a007000 Eth1/8 0 2e 7 0 10 0 f 1e 1e 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 D-84 - 800 0 0 1 30 0
1a01e000 Eth1/31 1 0 2d 0 37 1 e 1c 9c 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 - - 0 0 0 1 0 0
1a01f000 Eth1/32 1 0 3d 0 38 1 f 1e 9e 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 - - 0 0 0 1 0 0
1a030000 Eth1/49 0 2 1 0 49 1 20 38 b8 1 0 0 0 0 0 0 0 0 0 0 1 8 6 2 2 D-24d - 400 0 0 0 1 0
1a031000 Eth1/50 0 3 3 0 29 1 0 0 80 1 0 0 0 0 0 0 0 0 0 0 1 9 7 2 2 D-350 - 400 0 0 0 1 0
交換器認為封包應該從介面Ethernet 1/32轉送。是在得知MAC位址的PO4嗎？
leaf4# show port-channel summary
Flags: D - Down P - Up in port-channel (members)
I - Individual H - Hot-standby (LACP only)
s - Suspended r - Module-removed
S - Switched R - Routed
U - Up (port-channel)
M - Not in use. Min-links not met
F - Configuration failed
-------------------------------------------------------------------------------
Group Port- Type Protocol Member Ports
Channel
-------------------------------------------------------------------------------
1 Po1(SU) Eth LACP Eth1/5(P)
2 Po2(SU) Eth LACP Eth1/6(P)
3 Po3(SU) Eth LACP Eth1/31(P)
4 Po4(SU) Eth LACP Eth1/32(P)
會，因此封包會從介面1/32轉送到目的地主機。
不同EPG/相同枝葉中的2個EP — 路由資料包
拓撲
在本示例中，我們將跟蹤資料包從EP1到EP2的流量，其中資料包存在於同一個vPC枝葉對中。 兩個EP使用不同的BD位於不同的EPG中。
首先要始終檢查EP資料庫，看我們是否學習了EP:
leaf4# show endpoint ip 192.168.20.2
Legend:
O - peer-attached H - vtep a - locally-aged S - static
V - vpc-attached p - peer-aged L - local M - span
s - static-arp B - bounce
+-----------------------------------+---------------+-----------------+--------------+-------------+
VLAN/ Encap MAC Address MAC Info/ Interface
Domain VLAN IP Address IP Info
+-----------------------------------+---------------+-----------------+--------------+-------------+
30 vlan-2268 0050.56a5.fccc LV po3
Joey-Tenant:Joey-Internal vlan-2268 192.168.20.2 LV po3
calo2-leaf4# show endpoint ip 192.168.21.2
Legend:
O - peer-attached H - vtep a - locally-aged S - static
V - vpc-attached p - peer-aged L - local M - span
s - static-arp B - bounce
+-----------------------------------+---------------+-----------------+--------------+-------------+
VLAN/ Encap MAC Address MAC Info/ Interface
Domain VLAN IP Address IP Info
+-----------------------------------+---------------+-----------------+--------------+-------------+
8 vlan-2200 0050.56a5.0c11 LV po4
Joey-Tenant:Joey-Internal vlan-2200 192.168.21.2 LV po4
由於我們已經學習了EP並瞭解了IP資訊，因此我們應當能夠檢視硬體中的EP學習資訊：
leaf4# vsh_lc
module-1# show platform internal hal ep l3 all
LEGEND:
-------
VrfName: Vrf Name T: Type (Pl: Physical, Vl: Virtual, Xr: Remote)
EP IP: Endpoint IP
S Class: S Class Age Intvl: Age Interval
S T: Static Ep S E: Secure EP
L D: Learn Disable B N D: Bind Notify Disable
E N D: Epg Notify Disable B E: Bounce Enable
I D L: IVxlan Dont Learn SPI: Source Policy Incomplete
DPI: Dest Policy Incomplete SPA: Source Policy Applied
DPA: Dest Policy Applied DSS: Dest Shared Service
IL: Is Local VUB: Vnid Use Bd
SO: SA Only EP NH L3IfName: EP Next Hop L3 If Name
NHT: Next Hop Type (L2: L2 Entry L3: L3 Next Hop) BD Name: L2 NH BD Name
EP Mac: EP Mac L3 IfName: L3 NH If Name
L2 IfName: L2 If Name FD Name: L2 Entry FD Name
IP: L3 NH IP
L3 EP Count: 12
===========================================================================================================================================================================================
B E I S D S D D V EP-NH N |
Vrf EP S Age S S L N N B D P P P P S I U S L3 H | BD EP L3 L2 FD
Name T IP Class Intvl T E D D D E L I I A A S L B O IfName T | Name Mac IfName Ifname Name IP
===========================================================================================================================================================================================
common*rewall Pl 10.6.112.1 1 0 1 0 0 0 0 0 1 1 0 0 0 0 1 0 0 - L3 - 00:00:00:00:00:00 - - - 0.0.0.0
common*rewall Pl 10.6.114.1 1 0 1 0 0 0 0 0 1 1 0 0 0 0 1 0 0 - L3 - 00:00:00:00:00:00 - - - 0.0.0.0
common*rewall Pl 10.6.114.129 1 0 1 0 0 0 0 0 1 1 0 0 0 0 1 0 0 - L3 - 00:00:00:00:00:00 - - - 0.0.0.0
common*efault Pl 100.100.101.1 1 0 1 0 0 0 0 0 1 1 0 0 0 0 1 0 0 - L3 - 00:00:00:00:00:00 - - - 0.0.0.0
Joey-T*ternal Pl 192.168.1.1 1 0 1 0 0 0 0 0 1 1 0 0 0 0 1 0 0 - L3 - 00:00:00:00:00:00 - - - 0.0.0.0
Joey-T*ternal Xr 192.168.1.100 8013 128 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 - L3 - 00:0c:0c:0c:0c:0c Tunnel2 Tunnel2 - 0.0.0.0
Joey-T*ernal2 Pl 192.168.3.1 1 0 1 0 0 0 0 0 1 1 0 0 0 0 1 0 0 - L3 - 00:00:00:00:00:00 - - - 0.0.0.0
Joey-T*ternal Pl 192.168.20.1 1 0 1 0 0 0 0 0 1 1 0 0 0 0 1 0 0 - L3 - 00:00:00:00:00:00 - - - 0.0.0.0
Joey-T*ternal Pl 192.168.20.2 800a 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 - L2 BD-28 00:50:56:a5:fc:cc - Po3 FD-30 -
Joey-T*ternal Pl 192.168.21.1 1 0 1 0 0 0 0 0 1 1 0 0 0 0 1 0 0 - L3 - 00:00:00:00:00:00 - - - 0.0.0.0
Joey-T*ternal Pl 192.168.21.2 800c 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 - L2 BD-7 00:50:56:a5:0c:11 - Po4 FD-8 -
Joey-T*ternal Pl 2001:0:0:100::1 1 0 1 0 0 0 0 0 1 1 0 0 0 0 1 0 0 - L3 - 00:00:00:00:00:00 - - - 0.0.0.0
HAL第3層(l3)表非常有用，因為它為我們提供了第3層獲知EP的VLAN/埠資訊。 我們知道目的地是Po4，因此應該將封包從Po4中的任何連線埠轉送。
我們運行一個ELAM並檢視結果！
伊拉姆語
leaf4# vsh_lc
module-1# debug platform internal tah elam asic 0 module-1(DBG-TAH-elam)# trigger init in-select 6 out-select 0 module-1(DBG-TAH-elam-insel6)# set outer ipv4 src_ip 192.168.20.2 dst_ip 192.168.21.2 module-1(DBG-TAH-elam-insel6)# start module-1(DBG-TAH-elam-insel6)# stat ELAM STATUS =========== Asic 0 Slice 0 Status Armed Asic 0 Slice 1 Status Armed module-1(DBG-TAH-elam-insel6)# stat ELAM STATUS =========== Asic 0 Slice 0 Status Armed Asic 0 Slice 1 Status Triggered module-1(DBG-TAH-elam-insel6)# report | grep ovec sug_elam_out_sidebnd_no_spare_vec.ovector_idx: 0x9E
很好，所以我們觸發了資料包，我們發現「ovector_idx」是0x9E。ovector索引是應該將資料包轉發出來的傳出物理介面索引。 讓我們看看哪個連線埠具有此索引：
module-1(DBG-TAH-elam-insel6)# show platform internal hal l2 port gpd
Legend:
-------
IfId: Interface Id IfName: Interface Name
I P: Is PC Mbr IfId: Interface Id
Uc PC Cfg: UcPcCfg Idx Uc PC MbrId: Uc Pc Mbr Id
As: Asic AP: Asic Port
Sl: Slice Sp: Slice Port
Ss: Slice SrcId Ovec: Ovector (slice | srcid)
L S: Local Slot Reprogram:
L3: Is L3
P: PifTable Xla Idx: Xlate Idx
RP: Rw PifTable Ovx Idx: OXlate Idx
IP: If Profile Table N L3: Num. of L3 Ifs
RS: Rw SrcId Table NI L3: Num. of Infra L3 Ifs
DP: DPort Table Vif Tid: Vif Tid
SP: SrcPortState Table RwV Tid: RwVif Tid
RSP: RwSrcPortstate Table Ing Lbl: Ingress Acl Label
UC: UCPcCfg Egr Lbl: Egress Acl Label
UM: UCPcMbr Reprogram:
PROF ID: Lport Profile Id
VS: VifStateTable HI: LportProfile Hw Install
RV: Rw VifTable
Num. of Sandboxes: 1
Sandbox_ID: 0, BMP: 0x0
Port Count: 8
============================================================================================================================================
Uc Uc | Reprogram | | Rep |
I PC Pc L | R I R D R U U X | L Xla Ovx N NI Vif RwV Ing Egr | V R | PROF H
IfId Ifname P Cfg MbrID As AP Sl Sp Ss Ovec S | P P P S P Sp Sp C M L | 3 Idx Idx L3 L3 Tid Tid Lbl Lbl | S V | ID I
=============================================================================================================================================
1a004000 Eth1/5 1 0 1d 0 d 0 c 18 18 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 - - 800 0 0 1 0 0
1a005000 Eth1/6 1 0 b 0 e 0 d 1a 1a 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 - - 800 0 0 1 0 0
1a006000 Eth1/7 0 26 5 0 f 0 e 1c 1c 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 D-256 - 800 0 0 1 c 0
1a007000 Eth1/8 0 2f 7 0 10 0 f 1e 1e 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 D-199 - 800 0 0 1 2e 0
1a01e000 Eth1/31 1 0 2d 0 37 1 e 1c 9c 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 - - 0 0 0 1 0 0
1a01f000 Eth1/32 1 0 3d 0 38 1 f 1e 9e 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 - - 0 0 0 1 0 0
1a030000 Eth1/49 0 2 1 0 49 1 20 38 b8 1 0 0 0 0 0 0 0 0 0 0 1 6 4 2 2 D-24d - 400 0 0 0 1 0
1a031000 Eth1/50 0 3 3 0 29 1 0 0 80 1 0 0 0 0 0 0 0 0 0 0 1 5 3 2 2 D-350 - 400 0 0 0 1 0
似乎應該從埠1/32傳送它，對嗎？
leaf4# show port-channel summary
Flags: D - Down P - Up in port-channel (members)
I - Individual H - Hot-standby (LACP only)
s - Suspended r - Module-removed
S - Switched R - Routed
U - Up (port-channel)
M - Not in use. Min-links not met
F - Configuration failed
-------------------------------------------------------------------------------
Group Port- Type Protocol Member Ports
Channel
-------------------------------------------------------------------------------
1 Po1(SU) Eth LACP Eth1/5(P)
2 Po2(SU) Eth LACP Eth1/6(P)
3 Po3(SU) Eth LACP Eth1/31(P)
4 Po4(SU) Eth LACP Eth1/32(P)
是的，這是正確的。
不同EPG/不同枝葉中的2個EP — 路由資料包
拓撲
在本示例中，我們將跟蹤資料包從EP1到EP2的資料包流，其中EP1存在於EX vPC對上，EP2存在於遠端第1代vPC枝葉對上。 兩個EP使用不同的BD位於不同的EPG中。
再來，我們看看環保部門從哪裡學到：
leaf4# show endpoint ip 192.168.20.2
Legend:
O - peer-attached H - vtep a - locally-aged S - static
V - vpc-attached p - peer-aged L - local M - span
s - static-arp B - bounce
+-----------------------------------+---------------+-----------------+--------------+-------------+
VLAN/ Encap MAC Address MAC Info/ Interface
Domain VLAN IP Address IP Info
+-----------------------------------+---------------+-----------------+--------------+-------------+
30 vlan-2268 0050.56a5.fccc LV po3
Joey-Tenant:Joey-Internal vlan-2268 192.168.20.2 LV po3
calo2-leaf4# show endpoint ip 192.168.1.100
Legend:
O - peer-attached H - vtep a - locally-aged S - static
V - vpc-attached p - peer-aged L - local M - span
s - static-arp B - bounce
+-----------------------------------+---------------+-----------------+--------------+-------------+
VLAN/ Encap MAC Address MAC Info/ Interface
Domain VLAN IP Address IP Info
+-----------------------------------+---------------+-----------------+--------------+-------------+
Joey-Tenant:Joey-Internal 192.168.1.100 tunnel2
現在，讓我們驗證硬體已程式設計：
leaf4# vsh_lc
module-1# show platform internal hal ep l3 all LEGEND: ------- VrfName: Vrf Name T: Type (Pl: Physical, Vl: Virtual, Xr: Remote) EP IP: Endpoint IP S Class: S Class Age Intvl: Age Interval S T: Static Ep S E: Secure EP L D: Learn Disable B N D: Bind Notify Disable E N D: Epg Notify Disable B E: Bounce Enable I D L: IVxlan Dont Learn SPI: Source Policy Incomplete DPI: Dest Policy Incomplete SPA: Source Policy Applied DPA: Dest Policy Applied DSS: Dest Shared Service IL: Is Local VUB: Vnid Use Bd SO: SA Only EP NH L3IfName: EP Next Hop L3 If Name NHT: Next Hop Type (L2: L2 Entry L3: L3 Next Hop) BD Name: L2 NH BD Name EP Mac: EP Mac L3 IfName: L3 NH If Name L2 IfName: L2 If Name FD Name: L2 Entry FD Name IP: L3 NH IP L3 EP Count: 12 =========================================================================================================================================================================================== B E I S D S D D V EP-NH N | Vrf EP S Age S S L N N B D P P P P S I U S L3 H | BD EP L3 L2 FD Name T IP Class Intvl T E D D D E L I I A A S L B O IfName T | Name Mac IfName Ifname Name IP =========================================================================================================================================================================================== common*rewall Pl 10.6.112.1 1 0 1 0 0 0 0 0 1 1 0 0 0 0 1 0 0 - L3 - 00:00:00:00:00:00 - - - 0.0.0.0 common*rewall Pl 10.6.114.1 1 0 1 0 0 0 0 0 1 1 0 0 0 0 1 0 0 - L3 - 00:00:00:00:00:00 - - - 0.0.0.0 common*rewall Pl 10.6.114.129 1 0 1 0 0 0 0 0 1 1 0 0 0 0 1 0 0 - L3 - 00:00:00:00:00:00 - - - 0.0.0.0 common*efault Pl 100.100.101.1 1 0 1 0 0 0 0 0 1 1 0 0 0 0 1 0 0 - L3 - 00:00:00:00:00:00 - - - 0.0.0.0 Joey-T*ternal Pl 192.168.1.1 1 0 1 0 0 0 0 0 1 1 0 0 0 0 1 0 0 - L3 - 00:00:00:00:00:00 - - - 0.0.0.0 Joey-T*ternal Xr 192.168.1.100 8013 128 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 - L3 - 00:0c:0c:0c:0c:0c Tunnel2 Tunnel2 - 0.0.0.0 Joey-T*ernal2 Pl 192.168.3.1 1 0 1 0 0 0 0 0 1 1 0 0 0 0 1 0 0 - L3 - 00:00:00:00:00:00 - - - 0.0.0.0 Joey-T*ternal Pl 192.168.20.1 1 0 1 0 0 0 0 0 1 1 0 0 0 0 1 0 0 - L3 - 00:00:00:00:00:00 - - - 0.0.0.0 Joey-T*ternal Pl 192.168.20.2 800a 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 - L2 BD-28 00:50:56:a5:fc:cc - Po3 FD-30 - Joey-T*ternal Pl 192.168.21.1 1 0 1 0 0 0 0 0 1 1 0 0 0 0 1 0 0 - L3 - 00:00:00:00:00:00 - - - 0.0.0.0 Joey-T*ternal Pl 192.168.21.2 800c 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 - L2 BD-7 00:50:56:a5:0c:11 - Po4 FD-8 - Joey-T*ternal Pl 2001:0:0:100::1 1 0 1 0 0 0 0 0 1 1 0 0 0 0 1 0 0 - L3 - 00:00:00:00:00:00 - - - 0.0.0.0
硬體認為EP存在於通道2上。通道2的目的地是什麼？
module-1# show system internal eltmc info interface tunnel2
IfInfo:
interface: Tunnel2 ::: ifindex: 402718722
iod: 66 ::: state: up
Mod: 0 ::: Port: 0
Tunnel Index: 0 ::: Tunnel Dst ip: 0xc0a87843
Tunnel Encap: ivxlan ::: Tunnel VPC Peer: 0
Tunnel Dst ip str: 192.168.120.67 ::: Tunnel ept: 0x1
[SDK Info]:
tunnl_name:
vrf_id: 2 ::: if_index: 0x18010002
hwencapidx: 0 ::: encaptype: 1
mac_proxy: 0 ::: v4_proxy: 0
v6_proxy: 0 ::: ip_addr_type: 0
ipv4_address: 0xc0a87843
[SDB INFO]:
iod: 66
pc_if_index: 0
fab_if_index: 0
sv_if: 0
src_idx: 0
int_vlan: 0
encap_vlan: 0
mod_port_status: 0x41620003
v6_tbl_id: 0x80000002
v4_tbl_id: 0x2
router_mac:00.00.00.00.00.00
unnumbered: 0
trunk_id: 0
tunnel_mod: 0
tunnel_port: 0
tep_ip: 0xc0a87843
ip_if_mode: 0
sdk_vrf_id: 2
mtu: 9366 ::: ipmtu_id: 0
is_fex_fabric: 0
由於目標位於vPC之外，因此該目標IP應該是遠端枝葉的vPC虛擬IP。 讓我們檢查遠端枝葉並檢視：
leaf1# show system internal epm vpc Local TEP IP : 192.168.160.95 Peer TEP IP : 192.168.160.93 vPC configured : Yes vPC VIP : 192.168.120.67 MCT link status : Up Local vPC version bitmap : 0x7 Peer vPC version bitmap : 0x7 Negotiated vPC version : 3 Peer advertisement received : Yes Tunnel to vPC peer : Up
非常好，因此它從遠端vPC對獲取了目標EP。 我們來看看ELAM看到什麼並驗證我們是否正確轉發資料包：
伊拉姆語
module-1# debug platform internal tah elam asic 0 module-1(DBG-TAH-elam)# trigger init in-select 6 out-select 0 module-1(DBG-TAH-elam-insel6)# set outer ipv4 src_ip 192.168.20.2 dst_ip 192.168.1.100 module-1(DBG-TAH-elam-insel6)# start module-1(DBG-TAH-elam-insel6)# stat ELAM STATUS =========== Asic 0 Slice 0 Status Armed Asic 0 Slice 1 Status Triggered
現在，對於EX硬體上的遠端目標，有2個ELAM值在排除資料包流故障時非常重要。 與之前類似的ovector_idx和encap_idx:
module-1(DBG-TAH-elam-insel6)# report | grep ovec sug_elam_out_sidebnd_no_spare_vec.ovector_idx: 0xB8 module-1(DBG-TAH-elam-insel6)# report | grep encap sug_lurw_vec.encap_l2_idx: 0x0 sug_lurw_vec.encap_pcid: 0x0 sug_lurw_vec.encap_idx: 0x6 sug_lurw_vec.encap_vld: 0x1
在EX硬體上，我們確實能夠驅動應轉發資料包的目的埠。 以前，我們通常只檢查封裝idx並驗證目標idx是否是正確的隧道。 在此我們可以驗證哪些埠對映到8B:
module-1(DBG-TAH-elam-insel6)# show platform internal hal l2 port gpd
Legend:
-------
IfId: Interface Id IfName: Interface Name
I P: Is PC Mbr IfId: Interface Id
Uc PC Cfg: UcPcCfg Idx Uc PC MbrId: Uc Pc Mbr Id
As: Asic AP: Asic Port
Sl: Slice Sp: Slice Port
Ss: Slice SrcId Ovec: Ovector (slice | srcid)
L S: Local Slot Reprogram:
L3: Is L3
P: PifTable Xla Idx: Xlate Idx
RP: Rw PifTable Ovx Idx: OXlate Idx
IP: If Profile Table N L3: Num. of L3 Ifs
RS: Rw SrcId Table NI L3: Num. of Infra L3 Ifs
DP: DPort Table Vif Tid: Vif Tid
SP: SrcPortState Table RwV Tid: RwVif Tid
RSP: RwSrcPortstate Table Ing Lbl: Ingress Acl Label
UC: UCPcCfg Egr Lbl: Egress Acl Label
UM: UCPcMbr Reprogram:
PROF ID: Lport Profile Id
VS: VifStateTable HI: LportProfile Hw Install
RV: Rw VifTable
Num. of Sandboxes: 1
Sandbox_ID: 0, BMP: 0x0
Port Count: 8
============================================================================================================================================
Uc Uc | Reprogram | | Rep |
I PC Pc L | R I R D R U U X | L Xla Ovx N NI Vif RwV Ing Egr | V R | PROF H
IfId Ifname P Cfg MbrID As AP Sl Sp Ss Ovec S | P P P S P Sp Sp C M L | 3 Idx Idx L3 L3 Tid Tid Lbl Lbl | S V | ID I
=============================================================================================================================================
1a004000 Eth1/5 1 0 1d 0 d 0 c 18 18 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 - - 800 0 0 1 0 0
1a005000 Eth1/6 1 0 b 0 e 0 d 1a 1a 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 - - 800 0 0 1 0 0
1a006000 Eth1/7 0 26 5 0 f 0 e 1c 1c 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 D-256 - 800 0 0 1 c 0
1a007000 Eth1/8 0 2f 7 0 10 0 f 1e 1e 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 D-199 - 800 0 0 1 2e 0
1a01e000 Eth1/31 1 0 2d 0 37 1 e 1c 9c 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 - - 0 0 0 1 0 0
1a01f000 Eth1/32 1 0 3d 0 38 1 f 1e 9e 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 - - 0 0 0 1 0 0
1a030000 Eth1/49 0 2 1 0 49 1 20 38 b8 1 0 0 0 0 0 0 0 0 0 0 1 6 4 2 2 D-24d - 400 0 0 0 1 0
1a031000 Eth1/50 0 3 3 0 29 1 0 0 80 1 0 0 0 0 0 0 0 0 0 0 1 5 3 2 2 D-350 - 400 0 0 0 1 0
Switch認為它應該將其轉發到介面Eth1/49上的主幹。但我們如何驗證封裝是否正確？
我們首先需要檢視有關隧道的硬體資訊。 可通過運行以下HAL命令執行此操作：
module-1(DBG-TAH-elam-insel6)# show platform internal hal tunnel rtep pi
Non-Sandbox Mode
LEGEND:
-------
Tun Ifid: Tunnel Ifid IfName: Tunnel If Name
Lid: Logical Id ET: Encap Type V: Vxlan I: IVxlan N: NVGRE
VrfId: Vrf Id Vrf Name: Vrf Name
IP: Tunnel's IP
Hw Enc: Hw Encap Idx IVP: Is VPC Peer
IL: Is Local P4: Proxy for v4
P6: Proxy for V6 PM: Proxy for Mac
II: Is Ingress Only IC: Is Copy Service
C OBd: Copy Service Outer Bd U D: Use DF
NBT: Next Base Type E: ECMP N: Next-Hop NB Id: Next Base Id
NH cnt: Next Hop Count VrfId: Vrf Id
Vrf Name: Vrf Name IP: IP Address
Mac: Mac L3 IfId: L3 IfId
L3IfName: L3 If Name L2 IfId: L2 IfId
L2IfName: L2 If Name
Num. of Sandboxes: 1
Sandbox_ID: 0, BMP: 0x0
Remote Tep Count: 15
================================================================================================================================================================================================
=======
I N N |
E Vrf Hw V I P P P I I C U B B NH | Vrf L3 L3 L2 L2
IfId Ifname T Lid VrfId Name IP Enc P L 4 6 M I C OBd D T Id Cnt | VrfId Name IP Mac IfId IfName IfId IfName
================================================================================================================================================================================================
=======
18010002 Tunnel2 I 3005 2 overlay-1 192.168.120.670 0 0 0 0 0 0 0 1 0 E 2 2 2 overlay-1 0.0.0.0 0d:0d:0d:0d:0d:00 1a030001 Eth1/49.1 1a030000 Eth1/4
9
2 overlay-1 0.0.0.0 0d:0d:0d:0d:0d:00 1a031002 Eth1/50.2 1a031000 Eth1/5
0
此輸出提供我們關心的幾個值：
IfId — 分配給通道的介面ID
IP — 目的地的IP。 這應該與ELTMC相符。
L3 IfId — 交換器可用來轉送到適當目的地的第3層介面。
得知IfId後，我們可以驗證在群組中取得的封包是否與通道目的地相符：
module-1(DBG-TAH-elam-insel9)# show platform internal hal tunnel rtep apd
Non-Sandbox Mode
LEGEND:
-------
ifId: Interface Id IP: IP address
HwVrfId: Hardware Vrf Id SrcTepIdx: Source Tep Index
BDXlate: Egress BDXlate DstInfoIdx: Destination info index
RwEncapIdx: Rw Encap Index ECMPIdx: ECMP Index
Num: Number of hops ECMPMbrIdx: ECMP member Index
L2 Index: L2 Index RwDmacIdx: Rw Dmax Index
Num. of Sandboxes: 1
Sandbox_ID: 0, BMP: 0x0
Remote Tep Count: 15
==============================================================================================================================
ifId IP HwVrfId BDXlate SrcTepIdx DstInfoIdx RwEncapIdx ECMPIdx ECMPMbrIdx Num L2Index RwDmacIdx
==============================================================================================================================
18010002 192.168.120.67 2 1 3a9a 3005 6 0 0 2 1a030000 0 <---- RwEncapIdx is 6! Same as the "encap_idx" in the ELAM Report.
1a031000 1
此隧道的RwEncapIdx（重寫封裝索引）為6，這是在elam中顯示的內容。
1 EP —> L3輸出 — 路由流
拓撲
在本示例中，我們將跟蹤資料包的流量，該資料包從EP1傳送ICMP到運行OSPF的N5K上的環回。N5K通過同一對EX交換機上的L3Out連線。
由於我們已經在本文檔開頭驗證了本地EP程式設計，因此假定在硬體中正確學習了EP，然後繼續路由驗證。
首先檢查OSPF狀態和路由表：
leaf6# show ip ospf neighbors vrf jr:sb OSPF Process ID default VRF jr:sb Total number of neighbors: 2 Neighbor ID Pri State Up Time Address Interface 27.27.27.1 1 FULL/BDR 00:22:39 10.10.27.1 Vlan28 <---- Leaf5 27.27.27.3 1 FULL/DROTHER 00:22:37 10.10.27.3 Vlan28 <---- N5K
leaf6# show ip route vrf jr:sb 100.100.100.100
IP Route Table for VRF "jr:sb"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>
100.100.100.100/32, ubest/mbest: 1/0
*via 10.10.27.3, vlan28, [110/5], 00:16:58, ospf-default, intra
因此，我們知道路由表在10.10.27.3上將下一跳顯示為5K。這算是良好的開端，但我們如何驗證哪些硬體有呢？
首先檢查硬體中的鄰接表，確保ARP解析為10.10.27.3，並且已使用正確的介面進行程式設計：
leaf6# vsh_lc module-1# show forwarding adjacency IPv4 adjacency information, adjacency count 20 next-hop rewrite info interface phy i/f -------------- -------------- --------------- --------------- 10.10.27.1 0022.bdf8.19ff Vlan28 Tunnel3 10.10.27.3 8c60.4f02.88fc Vlan28 port-channel5
MAC地址與5K:
ACI-5548-B# show interface vlan 3117 Vlan3117 is up, line protocol is up Hardware is EtherSVI, address is 8c60.4f02.88fc Internet Address is 10.10.27.3/29 MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec
在EX平台上，有一個分配給VRF的「hw_vrf_idx」。在我們驗證硬體程式設計時，將引用此索引。 讓我們找到索引：
module-1# show system internal eltmc info vrf jr:sb
VRF-TABLE: jr:sb
vrf_type: tenant ::: context_id: 6
overlay_index: 0 ::: vnid: 2129921
scope: 5 ::: sclass: 16386
v4_table_id: 0x5 ::: v6_table_id: 0x80000005
intf_count: 5 ::: intrn_vlan_id: 0
VRF Intf: Vlan11 ::: src_plcy_incomp: 0
vnid_hex: 0x208001 ::: ingress_policy: 0x1
vrf_intf_list: Vlan28,Vlan16,Vlan9,Vlan11,loopback2,
hw_vrf_idx: 4612 ::: nb_egr_outer_bd: 0
sb_egr_outer_bd: 0
vrf_bd_list: 28,16,11,9,
sb_egr_outer_bd: 0 ::: sdk_vrf_id: 5
[SDK Info]:
vrf_name: jr:sb
vrf_id: 5 ::: hw_vrf_idx: 4612
vrf_vnid: 2129921 ::: is_infra: 0
tornbinfrahwbd: 0 ::: torsbinfrahwbd: 0
ingressBdAclLabel: 0 ::: ingBdAclLblMask: 0
egressBdAclLabel: 0 ::: egrBdAclLblMask: 0
sg_label: 5 ::: sclass: 16386
sp_incomplete: 1 ::: sclassprio: 3
[SDB INFO]:
v4 table
vrf type: 1
vrf id: 5
vnid: 2129921
internal infra vlan: 0
external router mac:00:22:bd:f8:19:ff
v6 table
vrf type: 1
vrf id: 5
vnid: 2129921
internal infra vlan: 0
external router mac:00:22:bd:f8:19:ff
::::
在檢測到鄰接關係後，HAL應該對路由進行程式設計。 我們可以使用以下命令檢查這一點：
module-1# show platform internal hal l3 routes | head ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ LEGEND: | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ LID: Logical ID RID: Route ID PID: Physical ID NB-ID:Next-Base ID HIT IDX: Next-Hop HitIndex CLP : Class Priority TBI: Trie Base Index | SC : Sup-Copy SSR: Src Sup-Redirect DSR: Dst Sup-Redirect TDD :TTL Disable NB: NextBaseType SDC : Src Direct Connect TRO: Trie Offset | SPI: Src Policy Inc DPI: Dst Policy Inc DR : Default Route LE :Learn Enable [E:Ecmp/A:Adj] ILL : Is Link Local ISS: Is Shared Services | RT : Route Type FWD: Forwarding HR : Host Routes EP :Ext Prefixes DLR: Default Lpm Route CLSS: Class Id RDEL: Route in Deletion | BNE: Bind Notify Enable SNE: Sclass Notify Enable BE : Bounce Enable IDL :Ivxlan DoNotLearn DL : Dest Local SA : Src Only AI : Age Interval | SF : Static Flag SH : Src Hit DH: Dest Hit | module-1# show platform internal hal l3 routes ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ LEGEND: | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ LID: Logical ID RID: Route ID PID: Physical ID NB-ID:Next-Base ID HIT IDX: Next-Hop HitIndex CLP : Class Priority TBI: Trie Base Index | SC : Sup-Copy SSR: Src Sup-Redirect DSR: Dst Sup-Redirect TDD :TTL Disable NB: NextBaseType SDC : Src Direct Connect TRO: Trie Offset | SPI: Src Policy Inc DPI: Dst Policy Inc DR : Default Route LE :Learn Enable [E:Ecmp/A:Adj] ILL : Is Link Local ISS: Is Shared Services | RT : Route Type FWD: Forwarding HR : Host Routes EP :Ext Prefixes DLR: Default Lpm Route CLSS: Class Id RDEL: Route in Deletion | BNE: Bind Notify Enable SNE: Sclass Notify Enable BE : Bounce Enable IDL :Ivxlan DoNotLearn DL : Dest Local SA : Src Only AI : Age Interval | SF : Static Flag SH : Src Hit DH: Dest Hit | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | | | | | LID |<------------------ Trie ------------->|<Dleft Trie>| | | VRF | Prefix/Len | RT| RID | LID | Type| PID | FPID/| HIT |N| NB-ID | NB Hw | PID | FPID/| TBI |TRO|Ifindex|CLSS|CLP| AI |SH|DH| Flags | |-----|-------------------------------------------|---|-----|----------|-----| | TID | IDX |B| | Idx | | TID |---------|---|-------|----|---|----|--|--|------------------| |-----|-------------------------------------------|---|-----|----------|-----|<------------------ DLEFT ------------>|-----|------|---------|---|-------|----|---|----|--|--|------------------| | | | | | | | PID | FPID/| HIT |N| NB-ID | NB Hw | | | | | | | | | | | | | | | | | | | | TID | IDX |B| | Idx | | | | | | | | | | | | | | | | | | |<------------------ TCAM ------------->| | | | | | | | | | | | | | | | | | | PID | TCAM | HIT |N| NB-ID | NB Hw | | | | | | | | | | | | | | | | | | | | ID | IDX |B| | Idx | | | | | | | | | | | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |Sandbox_ID: 0 Asic Bitmap: 0x0 -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- module-1# show platform internal hal l3 routes | egrep 100.100.100.100 | 4612| 100.100.100.100/ 32| UC| e4| 4a04| TRIE| 10| 5/ 0| 6010|A| 7567| 802e| 186a| 1/ 2| 10| 0| 0| f| 3| 0| 0| 0|spi,dpi
此輸出提供下一跳路由的相關資訊。 4612是jr:sb VRF的hw_vrf_idx。為了驗證下一跳，我們將使用TCAM中的「NB硬體Idx」來驗證下一表：
module-1# show platform internal hal l3 nexthops
Non-Sandbox Mode
LEGEND:
-------
NHOP ID : Nhop Identifier (Hex) CONS : H/W S/W info Consistency
TYPE : Nexthop Type ACTN : Nexthop Action
Vrf : L3 Vrf of the Nhop L3 INTF : L3 interface index (Hex)
L2 INTF : L2 interface index (Hex) BDID Or RwVRF : Bridge Domain Id Or Rewrite Vrfid (Hex)
INFR : ACI Infra valid PVRF : Preserve VRF
LRN : Learn Enabled VRFR : VRF Rewrite
PID : Physical ID FPID : FP of this nexthop
TLID : Tile Id within FP HIT IDX : Location of this Nhop (Hex)
Mac Entry:
TYP : Type INTF : Interface related Info (Hex)
LRN : Learn Info DL : Destination Local
MLD : Unused VNB : Vnid use BD
DFL : Default Entry VLD : MacKey Valid
FT : FID Type FV : FID Valid
FID : FID value (Hex) Mac : L2 MAC Address
L2 Ifabric Info:
CLSS : Source Class CLP : Source Class Priority
EPG : EndPoint Group BNE : Bind Notification Enabled
SNE : Source Address Notification Enabled CNE : Source class Notification Enabled
DL : iVxlan DL SPI : Source Policy Incomplete
DPI : Dest Policy Incomplete
IP Address : IP address
Sandbox_ID: 0 Asic Bitmap: 0x0
Summary info for 31 L3 Nexthop objects
C T A BDID I P V T |---------------------Mac Entry-------------------|----L2 Ifabric Info----|
NHOP O Y C L3 L2 Or N V L R L HIT|T L M V D V|--------------Mac Key-------| C B S C S D|
ID N P T INTF INTF RwVRF F R R F FP I IDX|Y INTF R D L N F L|F F FID | L N N N D P P|
(Hex) S E N Vrf (H) (H) (H) R F N R PID ID D (H)|P (H) N L D B L D|T V (H) Mac |CLSS P EPG E E E L I I| IP Address
--------+-+-+-+----+--------+--------+----+-+-+-+-+----+--+-+----+-+------+-+-+-+-+-+-+-+-+------+-----------------+----+-+----+-+-+-+-+-+-+-----------------------------
module-1# show platform internal hal l3 nexthops | grep 802e
7567 N I F 5 901001c 16000004 1c 0 0 0 0 2e 9 0 802e 0 22 0 0 0 0 0 1 1 1 1214 8c:60:4f:02:88:fc 0 0 2c0d 0 0 0 0 0 0 10.10.27.3
這裡，我們選擇「NB硬體IDX」並將其對映到「HIT IDX」。這將向我們顯示與下一跳MAC/IP對應的條目。這相當於檢視第1代ACI枝葉交換機上Broadcom中的「l3 defip show」和「l3 egress show」。
我們可以看到，該表包含正確的資訊：
L2 INTF:0x16000004 —> Port-channel 5的ifIndex
命中IDX: Hal l3路由中Nb硬體IDX驅動的索引
MAC:8c:60:4f:02:88:fc —> 5K上下一跳SVI的MAC
EPG: L3 EPG的SCLASS
IP 位址:10.10.27.3 —> 5K上SVI的下一跳IP
伊拉姆語
leaf6# pwd
/var/sysmgr/tmp_logs
leaf6# cat elam_report.txt | grep ip.da
sug_pr_lu_vec_l3v.ip.da: 0x0000000000000000064646464
leaf6# cat elam_report.txt | grep ip.sa
sug_pr_lu_vec_l3v.ip.sa: 0x00000000000000000C0A8140A
leaf6# cat elam_report.txt | grep adj
sug_lurw_vec.dst_addr.adj: 0x8C604F0288FC
sug_lurw_vec.dst_addr.adj.padfield: 0x04F0288FC
sug_lurw_vec.dst_addr.adj.idx: 0x2318
sug_lurw_vec.adj_vld: 0x0
leaf6# cat elam_report.txt | grep macdarslt.hit_idx
sug_fpc_lookup_vec.fplu_vec.rslt.macdarslt.hit_idx: 0x802E
1 EP —>遠端EP或SVI — 主幹驗證
拓撲
邏輯
在本示例中，我們將跟蹤從EP1發往遠端BD交換虛擬介面(SVI)的資料包流。本示例的目的是驗證主幹轉發，以確保將該資料包傳送到正確的枝葉。 讓我們假設封包已傳送到輸入枝葉上的主幹代理。
在骨幹上，我們首先驗證目的地IP的Council of Oracles Protocol(COOP)，因為封包已傳送到骨幹代理進行查詢：
calo1-spine1# show coop internal info ip-db | grep -A 10 192.168.20.1 IP address : 192.168.20.1 Vrf : 2129921 Flags : 0 EP vrf vnid : 2129921 EP IP : 192.168.20.1 Publisher Id : 10.0.224.88 Record timestamp : 11 04 2016 16:41:16 422062712 Publish timestamp : 11 04 2016 16:41:16 424633605 Seq No: 0 Remote publish timestamp: 01 01 1970 00:00:00 0 URIB Tunnel Info Num tunnels : 1 Tunnel address : 10.0.224.88 <---- REMOTE LEAF Tunnel ref count : 1
讓我們驗證哪個枝葉具有該TEP地址：
spine1# acidiag fnvread | grep 10.0.224.88
105 1 calo1-leaf5 FDO20160TPS 10.0.224.88/32 leaf active 0
由於我們知道資料包進入模組2上的骨幹（埠6），因此我們可以連線到模組2並檢視埠佈局。
spine1# vsh
Cisco iNX-OS Debug Shell
This shell should only be used for internal commands and exists
for legacy reasons. User should use ibash infrastructure as this
will be deprecated.
calo1-spine1# attach module 2
Attaching to module 2 ...
To exit type 'exit', to abort type '$.'
No directory, logging in with HOME=/
Bad terminal type: "xterm-256color". Will assume vt100.
Cisco iNX-OS Debug Shell
This shell should only be used for internal commands and exists
for legacy reasons. User should use ibash infrastructure as this
will be deprecated.
Loading parse tree (LC). Please be patient...
module-2#
module-2# show platform internal hal l2 port gpd
Legend:
-------
IfId: Interface Id IfName: Interface Name
I P: Is PC Mbr IfId: Interface Id
Uc PC Cfg: UcPcCfg Idx Uc PC MbrId: Uc Pc Mbr Id
As: Asic AP: Asic Port
Sl: Slice Sp: Slice Port
Ss: Slice SrcId Ovec: Ovector (slice | srcid)
L S: Local Slot Reprogram:
L3: Is L3
P: PifTable Xla Idx: Xlate Idx
RP: Rw PifTable Ovx Idx: OXlate Idx
IP: If Profile Table N L3: Num. of L3 Ifs
RS: Rw SrcId Table NI L3: Num. of Infra L3 Ifs
DP: DPort Table Vif Tid: Vif Tid
SP: SrcPortState Table RwV Tid: RwVif Tid
RSP: RwSrcPortstate Table Ing Lbl: Ingress Acl Label
UC: UCPcCfg Egr Lbl: Egress Acl Label
UM: UCPcMbr Reprogram:
PROF ID: Lport Profile Id
VS: VifStateTable HI: LportProfile Hw Install
RV: Rw VifTable
Num. of Sandboxes: 1
Sandbox_ID: 0, BMP: 0x0
Port Count: 7
============================================================================================================================================
Uc Uc | Reprogram | | Rep |
I PC Pc L | R I R D R U U X | L Xla Ovx N NI Vif RwV Ing Egr | V R | PROF H
IfId Ifname P Cfg MbrID As AP Sl Sp Ss Ovec S | P P P S P Sp Sp C M L | 3 Idx Idx L3 L3 Tid Tid Lbl Lbl | S V | ID I
=============================================================================================================================================
1f5 SpInBndMgmt 0 9de 1a 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 D-2d4 D-3e1 0 0 0 0 1 0
1a080000 Eth2/1 0 9a 1c 0 11 0 10 20 20 1 0 0 0 0 0 0 0 0 0 0 1 b b 1 1 D-f3 D-61 100 0 0 0 1 0
1a081000 Eth2/2 0 9b 22 0 d 0 c 18 18 1 0 0 0 0 0 0 0 0 0 0 1 c c 1 1 D-1ee D-30b 100 0 0 0 1 0
1a084000 Eth2/5 0 9e 1e 0 3d 1 14 28 a8 1 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 D-19a D-2ee 100 0 0 0 1 0
1a085000 Eth2/6 0 9f 24 0 39 1 10 20 a0 1 0 0 0 0 0 0 0 0 0 0 1 e e 1 1 D-87 D-184 100 0 0 0 1 0
1a086000 Eth2/7 0 a0 26 0 35 1 c 18 98 1 0 0 0 0 0 0 0 0 0 0 1 d d 1 1 D-1d0 D-357 100 0 0 0 1 0
1a088000 Eth2/9 0 a2 20 1 d 0 c 18 18 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 D-3ea D-1a9 100 0 0 0 1 0
Ethernet 2/6是連線到枝葉6的介面位於ASIC 0 SLICE 1上
現在我們知道哪個ASIC運行我們的ELAM。 ASIC 0.
module-2# debug platform internal tah elam asic 0 module-2(DBG-TAH-elam)# trigger reset module-2(DBG-TAH-elam)# trigger init in-select 13 out-select 0 module-2(DBG-TAH-elam-insel13)# set inner ipv4 src_ip 10.100.17.11 dst_ip 192.168.20.1 module-2(DBG-TAH-elam-insel13)# start stat module-2(DBG-TAH-elam-insel13)# stat ELAM STATUS =========== Asic 0 Slice 0 Status Armed Asic 0 Slice 1 Status Armed module-2(DBG-TAH-elam-insel13)# stat ELAM STATUS =========== Asic 0 Slice 0 Status Triggered <---- Packet triggered from FM Asic 0 Slice 1 Status Triggered <---- Packet triggered from Front Panel
通過ELAM可以找到向量索引：
Front Panel ELAM drove sug_elam_out_sidebnd_no_spare_vec.ovector_idx: 0xB8
現在，如何將0xb8對映到埠？ 由於我們知道資料包應該傳送到交換矩陣模組(FM)進行查詢，因此我們可以檢視內部埠對映以查詢目的FM:
module-2# show platform internal hal l2 internal-port pi
Num. of Sandboxes: 1
Legend:
-------
IfId: Interface Id IfName: Interface Name
As: Asic AP: Asic Port
Sl: Slice SP: Slice Port
Ss: Slice SrcId Ovec: Ovector
UcPcCfgId: Uc Pc CfgId Lb Mbrid: LB MbrId
Sandbox_ID: 0, BMP: 0x0
Internal Port Count: 32
======================================================================
UcPc Lb
IfId IfName As AP Sl SP Ss Ovec CfgId MbrId
======================================================================
7d - 0 21 0 20 38 38 0 4
7e - 0 29 1 0 0 80 0 8
7f - 1 21 0 20 38 38 0 c
80 - 1 29 1 0 0 80 0 10
81 - 2 21 0 20 38 38 0 14
82 - 2 29 1 0 0 80 0 18
83 - 3 21 0 20 38 38 0 1c
84 - 3 29 1 0 0 80 0 20
95 - 0 19 0 18 30 30 0 3
96 - 0 49 1 20 38 b8 0 7
97 - 1 19 0 18 30 30 0 b
98 - 1 49 1 20 38 b8 0 f
99 - 2 19 0 18 30 30 0 13
9a - 2 49 1 20 38 b8 0 17
9b - 3 19 0 18 30 30 0 1b
9c - 3 49 1 20 38 b8 0 1f
ad - 0 25 0 24 40 40 0 1
ae - 0 41 1 18 30 b0 0 6
af - 1 25 0 24 40 40 0 9
b0 - 1 41 1 18 30 b0 0 e
b1 - 2 25 0 24 40 40 0 11
b2 - 2 41 1 18 30 b0 0 16
b3 - 3 25 0 24 40 40 0 19
b4 - 3 41 1 18 30 b0 0 1e
dd - 0 15 0 14 28 28 0 2
de - 0 4d 1 24 40 c0 0 5
df - 1 15 0 14 28 28 0 a
e0 - 1 4d 1 24 40 c0 0 d
e1 - 2 15 0 14 28 28 0 12
e2 - 2 4d 1 24 40 c0 0 15
e3 - 3 15 0 14 28 28 0 1a
e4 - 3 4d 1 24 40 c0 0 1d
使用ASIC0/Ovc B8時，我們得到MbrId 0x7,Slice並不重要。
此MbrId是USD上對映到FM上的介面的介面。 請記住，此MbrId以十六進位制表示，必須轉換為十進位制。
通過檢視USD介面並檢查埠7，我們可以找到哪個FM:
module-2# show platform internal usd port info | grep -A 3 "Int 7"(if the interface has multiple digits, will be "Int##" with no space)
Port 73.0 (Int 7) : Admin UP Link UP Remote slot22.asic0
slice:1 slice port:32 lcl srcid:56 gbl srcid:184
asic mrl:0xd07c010, mac mrl:0x12c84010, mac:16, chan:0
speed 106G serdes: 0x328 0x329 0x32a 0x32b
「slot」以0為基礎，而FM編號以1為基礎，因此我們需要將1新增到此處列出的數字中。 這意味著資料包應傳送到FM 23。
合成IP
與Alpine一樣，有一個合成IP用作外部IP地址，用於確定COOP查詢的雜湊。 若要尋找此內容，您需要對內部DST IP運行此命令和grep:
module-2(DBG-TAH-elam-insel7)# show forwarding route synthetic vrf all | grep 192.168.20.1 SYNTH-88 1.203.211.185/32 0x208001 192.168.20.1
這表明1.203.211.185是我們的合成IP。基於此，我們還可以將FM鏈路上的「外部DST IP」設定為此。 我們應該在FM上觸發：
光纖模組ELAM
module-23(DBG-TAH-elam-insel7)# trigger reset module-23(DBG-TAH-elam)# trigger init in-select 13 out-select 0 module-23(DBG-TAH-elam-insel13)# set outer ipv4 dst_ip 1.203.211.185 <----- DST IP IS THE SYNTHETIC IP module-23(DBG-TAH-elam-insel13)# set inner ipv4 src_ip 10.100.17.11 dst_ip 192.168.20.1 module-23(DBG-TAH-elam-insel13)# start stat module-23(DBG-TAH-elam-insel13)# stat ELAM STATUS =========== Asic 0 Slice 0 Status Armed Asic 0 Slice 1 Status Armed Asic 0 Slice 2 Status Armed Asic 0 Slice 3 Status Armed Asic 0 Slice 4 Status Armed Asic 0 Slice 5 Status Armed module-23(DBG-TAH-elam-insel13)# stat ELAM STATUS =========== Asic 0 Slice 0 Status Armed Asic 0 Slice 1 Status Armed Asic 0 Slice 2 Status Triggered <---- Triggered on SLICE 2 Asic 0 Slice 3 Status Armed Asic 0 Slice 4 Status Armed Asic 0 Slice 5 Status Armed
顯然，請轉儲完整的報告，但讓我們看一下我們觸發的此資料包的ovector_idx:
lac_elam_out_sidebnd_no_spare_vec.ovector_idx:0x20 <以下-----令中使用的向量索引
我們如何確定哪個介面具有此向量？ 在FM上運行以下命令：
**錯誤CSCvf42796 
，附加所有FM命令，並加上「|不更多」。 否則，某些表條目可能不會顯示在最終輸出中。
module-23(DBG-TAH-elam-insel13)# show platform internal hal l2 port gpd | no-more
Legend:
-------
IfId: Interface Id IfName: Interface Name
I P: Is PC Mbr IfId: Interface Id
Uc PC Cfg: UcPcCfg Idx Uc PC MbrId: Uc Pc Mbr Id
As: Asic AP: Asic Port
Sl: Slice Sp: Slice Port
Ss: Slice SrcId Ovec: Ovector (slice | srcid)
L S: Local Slot Reprogram:
L3: Is L3
P: PifTable Xla Idx: Xlate Idx
RP: Rw PifTable Ovx Idx: OXlate Idx
IP: If Profile Table N L3: Num. of L3 Ifs
RS: Rw SrcId Table NI L3: Num. of Infra L3 Ifs
DP: DPort Table Vif Tid: Vif Tid
SP: SrcPortState Table RwV Tid: RwVif Tid
RSP: RwSrcPortstate Table Ing Lbl: Ingress Acl Label
UC: UCPcCfg Egr Lbl: Egress Acl Label
UM: UCPcMbr Reprogram:
PROF ID: Lport Profile Id
VS: VifStateTable HI: LportProfile Hw Install
RV: Rw VifTable
Num. of Sandboxes: 1
Sandbox_ID: 1, BMP: 0x1
Port Count: 8
============================================================================================================================================
Uc Uc | Reprogram | | Rep |
I PC Pc L | R I R D R U U X | L Xla Ovx N NI Vif RwV Ing Egr | V R | PROF H
IfId Ifname P Cfg MbrID As AP Sl Sp Ss Ovec S | P P P S P Sp Sp C M L | 3 Idx Idx L3 L3 Tid Tid Lbl Lbl | S V | ID I
=============================================================================================================================================
ae fc0-lc1:0-0 1 0 3 0 11 0 10 20 20 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 - - 0 0 0 0 0 0
af fc0-lc1:0-1 1 0 4 0 3d 2 c 18 98 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 - - 0 0 0 0 0 0
b0 fc0-lc1:1-0 1 0 13 0 d 0 c 18 18 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 - - 0 0 0 0 0 0
b1 fc0-lc1:1-1 1 0 14 0 39 2 8 10 90 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 - - 0 0 0 0 0 0
b2 fc0-lc1:2-0 1 0 23 0 5d 3 14 28 e8 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 - - 0 0 0 0 0 0
b3 fc0-lc1:2-1 1 0 24 0 21 1 8 10 50 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 - - 0 0 0 0 0 0
b4 fc0-lc1:3-0 1 0 33 0 51 3 8 10 d0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 - - 0 0 0 0 0 0
該向量對映到ASIC 0/SLICE 0上的LC1（插槽2中的線卡，因為它基於0）。我們從最初在LC上運行的ELAM中得知，我們在此片上觸發：
module-2# debug platform internal tah elam asic 0 module-2(DBG-TAH-elam)# trigger reset module-2(DBG-TAH-elam)# trigger init in-select 13 out-select 0 module-2(DBG-TAH-elam-insel13)# set inner ipv4 src_ip 10.100.17.11 dst_ip 192.168.20.1 module-2(DBG-TAH-elam-insel13)# start stat module-2(DBG-TAH-elam-insel13)# stat ELAM STATUS =========== Asic 0 Slice 0 Status Armed Asic 0 Slice 1 Status Armed module-2(DBG-TAH-elam-insel13)# stat ELAM STATUS =========== Asic 0 Slice 0 Status Triggered <---- Packet triggered from FM Asic 0 Slice 1 Status Triggered <---- Packet triggered from Front Panel
此ELAM上的向量是sug_elam_out_sidebnd_no_spare_vec.ovector_idx:0x98(我們通過「hal l2埠gpd」得知)對映到LC上的正確介面：
============================================================================================================================================
Uc Uc | Reprogram | | Rep |
I PC Pc L | R I R D R U U X | L Xla Ovx N NI Vif RwV Ing Egr | V R | PROF H
IfId Ifname P Cfg MbrID As AP Sl Sp Ss Ovec S | P P P S P Sp Sp C M L | 3 Idx Idx L3 L3 Tid Tid Lbl Lbl | S V | ID I
=============================================================================================================================================
1f5 SpInBndMgmt 0 9de 1a 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 D-2d4 D-3e1 0 0 0 0 1 0
1a080000 Eth2/1 0 9a 1c 0 11 0 10 20 20 1 0 0 0 0 0 0 0 0 0 0 1 b b 1 1 D-f3 D-61 100 0 0 0 1 0
1a081000 Eth2/2 0 9b 22 0 d 0 c 18 18 1 0 0 0 0 0 0 0 0 0 0 1 c c 1 1 D-1ee D-30b 100 0 0 0 1 0
1a084000 Eth2/5 0 9e 1e 0 3d 1 14 28 a8 1 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 D-19a D-2ee 100 0 0 0 1 0
1a085000 Eth2/6 0 9f 24 0 39 1 10 20 a0 1 0 0 0 0 0 0 0 0 0 0 1 e e 1 1 D-87 D-184 100 0 0 0 1 0
1a086000 Eth2/7 0 a0 26 0 35 1 c 18 98 1 0 0 0 0 0 0 0 0 0 0 1 d d 1 1 D-1d0 D-357 100 0 0 0 1 0
1a088000 Eth2/9 0 a2 20 1 d 0 c 18 18 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 D-3ea D-1a9 100 0 0 0 1 0
Ethernet 2/7是連線到枝葉5的介面。
額外方案：獲取不在「hal internal-port pi」輸出中的Ovector
拓撲
邏輯
在某些情況下，我們會在「show platform internal hal2 internal-port pi」表中捕獲沒有Ovector的資料包。 在下面的場景中，我們實際上捕獲的是從FM傳回的資料包，因此我們需要檢視不同的表格，以檢視資料包選擇的前面板埠。
請注意，上面的拓撲是學習傳輸流量的完全不同的環境（無代理路由）。 模組是N9K-X9732C-EX。
@module-1# debug platform internal tah elam asic 3
@module-1(DBG-elam)# trigger reset
@module-1(DBG-elam)# trigg init in-select 13 out-select 0
@module-1(DBG-elam-insel13)# set inner ipv4 src_ip 192.85.1.2 dst_ip 192.85.2.67
@module-1(DBG-elam-insel13)# star
@module-1(DBG-elam-insel13)# stat
ELAM STATUS
===========
Asic 3 Slice 0 Status Armed
Asic 3 Slice 1 Status Triggered
@module-1(DBG-elam-insel13)# report | grep ovector
sug_elam_out_sidebnd_no_spare_vec.ovector_idx: 0xA0 <<<<<<<<<<<<<<<<< now we look for this in the "hal internal-port pi" command
@module-1# show platform internal hal l2 internal-port pi
No sandboxes exist
Num. of Sandboxes: 1
Legend:
-------
IfId: Interface Id IfName: Interface Name
As: Asic AP: Asic Port
Sl: Slice SP: Slice Port
Ss: Slice SrcId Ovec: Ovector
UcPcCfgId: Uc Pc CfgId Lb Mbrid: LB MbrId
Sandbox_ID: 0, BMP: 0x0
Internal Port Count: 24
======================================================================
UcPc Lb
IfId IfName As AP Sl SP Ss Ovec CfgId MbrId
======================================================================
7d - 0 21 0 20 38 38 0 4
7e - 0 29 1 0 0 80 0 8
7f - 1 21 0 20 38 38 0 c
80 - 1 29 1 0 0 80 0 10
81 - 2 21 0 20 38 38 0 14
82 - 2 29 1 0 0 80 0 18
83 - 3 21 0 20 38 38 0 1c
84 - 3 29 1 0 0 80 0 20
ad - 0 25 0 24 40 40 0 1
ae - 0 41 1 18 30 b0 0 6
af - 1 25 0 24 40 40 0 9
b0 - 1 41 1 18 30 b0 0 e
b1 - 2 25 0 24 40 40 0 11
b2 - 2 41 1 18 30 b0 0 16
b3 - 3 25 0 24 40 40 0 19
b4 - 3 41 1 18 30 b0 0 1e
dd - 0 15 0 14 28 28 0 2
de - 0 4d 1 24 40 c0 0 5
df - 1 15 0 14 28 28 0 a
e0 - 1 4d 1 24 40 c0 0 d
e1 - 2 15 0 14 28 28 0 12
e2 - 2 4d 1 24 40 c0 0 15
e3 - 3 15 0 14 28 28 0 1a
e4 - 3 4d 1 24 40 c0 0 1d <<<<<<< we cant find an entry that matches 0xA0
@module-1# show platform internal hal l2 port gpd
Legend:
-------
<snip>
Sandbox_ID: 0, BMP: 0x0
Port Count: 6
============================================================================================================================================
Uc Uc | Reprogram | | Rep |
I PC Pc L | R I R D R U U X | L Xla Ovx N NI Vif RwV Ing Egr | V R | PROF H
IfId Ifname P Cfg MbrID As AP Sl Sp Ss Ovec S | P P P S P Sp Sp C M L | 3 Idx Idx L3 L3 Tid Tid Lbl Lbl | S V | ID I
=============================================================================================================================================
1f5 SpInBndMgmt 0 9de 1a 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 D-2d4 D-3e1 0 0 0 0 1 0
1a000000 Eth1/1 0 1b 1c 0 11 0 10 20 20 1 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 D-13b D-33b 500 0 1 0 3 0
1a01c000 Eth1/29 0 37 1e 3 3d 1 14 28 a8 1 0 0 0 0 0 0 0 0 0 0 1 8 8 1 1 D-3f2 D-7a 100 0 0 0 2 0
1a01d000 Eth1/30 0 38 20 3 39 1 10 20 a0 1 0 0 0 0 0 0 0 0 0 0 1 5 5 1 1 D-36e D-362 100 0 0 0 2 0
1a01e000 Eth1/31 0 39 22 3 35 1 c 18 98 1 0 0 0 0 0 0 0 0 0 0 1 9 9 1 1 D-273 D-8 100 0 0 0 2 0
1a01f000 Eth1/32 0 3a 24 3 31 1 8 10 90 1 0 0 0 0 0 0 0 0 0 0 1 a a 1 1 D-154 D-5d 100 0 0 0 2 0
1/30是連線到枝葉102的phy介面，由拓撲、ASIC 3、片1驗證
意見