The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.
This document describes how Cisco Catalyst 6500 with Supervisor Sup2T programs the (Cisco Express Forwarding) CEF entries configured on Cisco IOS software in the line cards hardware used to achieve packet forwarding.
Cisco recommends that you have knowledge of these topics:
Cisco Catalyst 6500 Series Switches
The information in this document is based on these hardware and software versions:
Cisco Catayst 6500 WS-X6848-GE-TX (with DFC4) line card.
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.
CEF as a Layer 3 switching mechanism is used by most Cisco multilayer switches.It is imperative for network engineers to understand how CEF works in order to troubleshoot network outages, packet loss or packet delay scenarios on a daily basis.
Sup2T supervisor in standalone mode or as VSS is currently deployed by many enterprise networks as a core switch, aggregates practically all other routing or switching devices. This also means that forwards most intra and inter domain traffic in order to successfully deliver the packets to its destinations. For this to be achieved, Sup2T must have proper routing information learned statically or dynamically via routing protocols.
In a modular chassis, multiple forwarding engines might exist besides the supervisor's. Certain line cards (specially the new generation ones such as C6800-32P10G) already include their own forwarding engine in order to enhance packet switching performance, the lookup of CEF entries is exectued locally and causes resources to be best distributed for traffic that ingresses over different line cards.These are known as Disributed Forwarding Cards (DFCs).
These CEF entries shared across all forwarding engines might fail to be allocated in HW for multiple reasons, from a software defect condition, resources exhaustion to high CPU conditions and prevents the switch to have enough time to update all entries, this can causes a series of undesirable events.
Network:
Switch#show module 3
---------------------- ----------------------------- Mod Ports Card Type Model Serial No. --- ----- -------------------------------------- ------------------ ----------- 3 48 CEF720 48 port 10/100/1000mb Ethernet WS-X6848-GE-TX SAL2003X5AH ---- --------------------------- ------------------ ----------- ------- ------- 3 Distributed Forwarding Card WS-F6K-DFC4-A SAL2003X5AH 1.4 Ok
In the diagram, a standalone 6506 switch has a Supervisor 2T installed as well as a line card WS-6848-GE-TX with a DFC in slot 3. Host 3750X which is connected to the line card through port G3/1 sends traffic to 3850's Loopback 0 address 1.1.1.1.
For this, 3750X has a static route to IP address 1.1.1.1 through next hop 10.1.1.10 which is the SVI of VLAN 1 in the Sup2T switch. The Sup2T switch needs to route this traffic to 3850 switch based in a static route entry for IP 1.1.1.1/32 via next hop 10.1.2.1 which is the 3850 interface connected to the Sup2T in VLAN 2.
MXC.CALO.3750X#show ip route | inc 1.1.1.1 S 1.1.1.1 [1/0] via 10.1.1.10 MXC.CALO.Sup2T#show ip route | inc 1.1.1.1 S 1.1.1.1 [1/0] via 10.1.2.1 CALO.MXC.3850#show ip route | inc 1.1.1.1 C 1.1.1.1 is directly connected, Loopback1
Be aware that for sake of simplicity, both 3750X and 3850 switches are connected to the 6500 through the same line card. This means that traffic is looked up locally and locally forwarded too.
A packet ingresses Sup2T switch via Gi3/1, eventually reaches the forwarding engine (since this is a DFC). The forwarding engine parses the destination IP address field in this packet and a lookup over the programmed CEF entries for the best match (Longest Mask).
Since this is a DFC card, it means it has its own CEF entries and to verify them, it is necessary for us to attach to the linecard with command attach [dec] or attach switch [1-2] mod [dec] for VSS.
Now, you should be in the DFC prompt, command show platform hardware cef or show platform hardware cef vpn 0 return all CEF entries programmed for general routing table (VPN 0/ No VRF).
Since the objective is prefix 1.1.1.1/32 you use command show platform hardware cef vpn 0 lookup 1.1.1.1. The command returns the best match for prefix 1.1.1.1 and the one it uses to actually forward traffic:
MXC.CALO.Sup2T#attach 3 Trying Switch ... Entering CONSOLE for Switch Type "^C^C^C" to end this session MXC.CALO.Sup2T-dfc3#show platform hardware cef vpn 0 Codes: decap - Decapsulation, + - Push Label Index Prefix Adjacency 32 0.0.0.0/32 receive 33 255.255.255.255/32 receive 34 10.1.85.254/32 glean 35 10.1.85.5/32 receive 36 10.1.86.5/32 receive [snip...] MXC.CALO.Sup2T-dfc3#show platform hardware cef vpn 0 lookup 1.1.1.1 Codes: decap - Decapsulation, + - Push Label Index Prefix Adjacency 262 1.1.1.1/32 Vl2 ,0c11.678b.f6f7
The CEF entry is there, it got programmed as the result of our static entry programmed in IOS software via command ip route 1.1.1.1 255.255.255.255 10.1.2.1.
You can also verify that this entry gets hits and traffic is forwarded with this entry via commands show platform hardware cef 1.1.1.1 detail which returns an adjacency entry:
MXC.CALO.Sup2T-dfc3#show platform hardware cef 1.1.1.1 detail Codes: M - mask entry, V - value entry, A - adjacency index, NR- no_route bit LS - load sharing count, RI - router_ip bit, DF: default bit CP - copy_to_cpu bit, AS: dest_AS_number, DGTv - dgt_valid bit DGT: dgt/others value Format:IPV4 (valid class vpn prefix) M(262 ): 1 F 2FFF 255.255.255.255 V(262 ): 1 0 0 1.1.1.1 (A:114689, LS:0, NR:0, RI:0, DF:0 CP:0 DGTv:1, DGT:0)
Finally, the adjacency entry shows how the packet is rewritten and if traffic is rewritten by this adjacency entry:
MXC.CALO.Sup2T-dfc3#show platform hardware cef adjacencies entry 114689 detail RIT fields: The entry has a Layer2 Format _________________________________________________________ |decr_ttl = YES | pipe_ttl = 0 | utos = 0 |_________________|__________________|____________________ |l2_fwd = 0 | rmac = 0 | ccc = L3_REWRITE |_________________|__________________|____________________ |rm_null_lbl = YES| rm_last_lbl = YES| pv = 0 |_________________|__________________|____________________ |add_shim_hdr= NO | rec_findex = N/A | rec_shim_op = N/A |_________________|__________________|____________________ |rec_dti_type = N/A | rec_data = N/A |____________________________________|____________________ |modify_smac = YES| modify_dmac = YES| egress_mcast = NO |____________________________________|____________________ |ip_to_mac = NO |_________________________________________________________ |dest_mac = 0c11.678b.f6f7 | src_mac = d8b1.902c.9680 |___________________________|_____________________________ | Statistics: Packets = 642 Bytes = 75756 <<<<
The dest_mac and src_mac are the values of main interest, which indicate the new L2 headers that are written for this packet. Destination MAC address 0c11.678b.f6f7 is 10.1.2.1 which is the 3850 (Next hop to reach 1.1.1.1):
MXC.CALO.Sup2T#show ip arp 10.1.2.1 Protocol Address Age (min) Hardware Addr Type Interface Internet 10.1.2.1 30 0c11.678b.f6f7 ARPA Vlan2
Also, the Statistics field shows that traffic is actually hits this adjacency entry and L2 headers are rewritten accordingly.
Delete CEF entries can help us to delete any entry that might be wrongly programmed (to a wrong adjacency entry for example) or even for training purposes. It also provides a way to modify a routing path.
In order to delete a CEF entry, you need to understand that CEF entries are programmed sequentially and have a Hardware Index assigned, for example:
MXC.CALO.Sup2T-dfc3#show platform hardware cef vpn 0
Codes: decap - Decapsulation, + - Push Label
MXC.CALO.Sup2T-dfc3#show platform hardware cef vpn 0
...
Index Prefix Adjacency 259 10.1.2.255/32 receive 260 10.1.1.1/32 Vl1 ,a0ec.f930.3f40 261 10.1.2.1/32 Vl2 ,0c11.678b.f6f7 262 1.1.1.1/32 Vl2 ,0c11.678b.f6f7 <<<< Our CEF entry of interest has a HW index of 262.
...
This Hardware Index is the most important element to delete a CEF entry since it's used as a reference. However, in order to do any change on it, it must be converted to a software handle. You can achieve this with the command test platform hardware cef index-conv hw_to_sw [hw index]
MXC.CALO.Sup2T-dfc3#test platform hardware cef index-conv hw_to_sw 262 hw index: 262 ----> sw handle: 101
Now that you know the software handle, you can proceed with the CEF entry deletion with command test platform hardware cef v4-delete [sw handle] mask [mask length] vpn [dec]
MXC.CALO.s2TVSS-sw2-dfc3#test platform hardware cef v4-delete 101 mask 32 vpn 0 test_ipv4_delete: done.
Note: Mask length value is 32 since this is a host specific route (1.1.1.1/32)
Now, our CEF entry is deleted:
MXC.CALO.Sup2T-dfc3#show platform hard cef vpn 0 1.1.1.1 Codes: decap - Decapsulation, + - Push Label Index Prefix Adjacency MXC.CALO.Sup2T-dfc3#show platform hard cef vpn 0 [snip...] 259 10.1.2.255/32 receive 260 10.1.1.1/32 Vl1 ,a0ec.f930.3f40 261 10.1.2.1/32 Vl2 ,0c11.678b.f6f7 288 224.0.0.0/24 receive <<<<<<< Index 262 no longer exists in the CEF entries. 289 10.1.85.0/24 glean
Notice the test platform hardware cef vpn 0 command was executed under the DFC prompt. This way, the CEF entry was removed from the DFC's CEF table and NOT from the supervisor, you must be really careful on which forwarding engine the entries are removed from.
A change in the traffic has the risk of no visibility (in case of a lab test), this can be due to the hit of another CEF entry. Consider to always match the most exact one (longest mask). In this lab, it hits:
MXC.CALO.Sup2T-dfc3#show plat hard cef vpn 0 lookup 1.1.1.1 Codes: decap - Decapsulation, + - Push Label Index Prefix Adjacency 262048 0.0.0.0/0 glean
So what does this entry actually do with the packet?:
MXC.CALO.Sup2T-dfc3#show platform hardware cef adjacencies entry 262048
RIT fields: The entry has a Recirc. Format _________________________________________________________ |decr_ttl=NO | l2_fwd=NO | ccc = 6 | add_shim_hdr = YES |_____________|____________|_________|____________________ |rc_fidx=0 | rc_shimop=1 | rc_dti_type=4 | rc_data = 0x10B |____________|_____________|_______________|______________ Statistics: Packets = 2163 Bytes = 255234
Taken from a CPU packet capture using Catlayst 6500 NETDR tool. For NETDR capture tool details refer to: Catalyst 6500 Series Switches Netdr Tool for CPU-Bound Packet Captures ------- dump of incoming inband packet ------- l2idb Po1, l3idb Vl1, routine inband_process_rx_packet, timestamp 01:00:17.841 dbus info: src_vlan 0x1(1), src_indx 0xB40(2880), len 0x82(130) bpdu 0, index_dir 0, flood 0, dont_lrn 0, dest_indx 0x5FA4(24484), CoS 0 cap1 0, cap2 0 78020800 00018400 0B400100 82000000 1E000464 2E000004 00000010 5FA45BDD destmac D8.B1.90.2C.96.80, srcmac A0.EC.F9.30.3F.40, shim ethertype CCF0 earl 8 shim header IS present: version 0, control 64(0x40), lif 1(0x1), mark_enable 1, feature_index 0, group_id 0(0x0), acos 0(0x0), ttl 14, dti 4, dti_value 267(0x10B) 10000028 00038080 010B ethertype 0800 protocol ip: version 0x04, hlen 0x05, tos 0x00, totlen 100, identifier 51573 df 0, mf 0, fo 0, ttl 255, src 10.1.1.1, dst 1.1.1.1 icmp type 8, code 0 ------- dump of outgoing inband packet ------- l2idb NULL, l3idb Vl2, routine etsec_tx_pak, timestamp 01:03:56.989 dbus info: src_vlan 0x2(2), src_indx 0x380(896), len 0x82(130) bpdu 0, index_dir 0, flood 0, dont_lrn 0, dest_indx 0x0(0), CoS 0 cap1 0, cap2 0 00020000 0002A800 03800000 82000000 00000000 00000000 00000000 00000000 destmac 0C.11.67.8B.F6.F7, srcmac D8.B1.90.2C.96.80, shim ethertype CCF0 earl 8 shim header IS present: version 0, control 0(0x0), lif 16391(0x4007), mark_enable 0, feature_index 0, group_id 0(0x0), acos 0(0x0), ttl 15, dti 0, dti_value 540674(0x84002) 000800E0 0003C008 4002 ethertype 0800 protocol ip: version 0x04, hlen 0x05, tos 0x00, totlen 100, identifier 50407 df 0, mf 0, fo 0, ttl 254, src 10.1.1.1, dst 1.1.1.1 icmp type 8, code 0
Now, all traffic with destination of 1.1.1.1 that ingresses through line card 3 is recirculated with shim header and punted to CPU. Sometimes, instead of this CEF entry, another 0.0.0.0/0 with drop adjaceny is seen and does the exact same thing.
Note: Evaluate which CEF entries are removed. A High CPU utilization can be caused due to this. Commonly a default route 0.0.0.0/0 is configured and traffic is forwarded based on it (and causes packet loss).
When a CEF entry is added, in most cases resolves any misprogramming issue that causes packet loss, packet delay or high CPU utilization. Knowledge of how to install the CEF entries in hardware, not only provides the ability to correct a misprogrammed entry, but also to manipulate any packet forwarding through recirculation of the packet, point it to a completely different interface or next hop, rewrite a routed packet as desired and/or drop it, etc. All of this, without a reload of the box, remove and set configuration or any apparent alteration. The CEF entry addition can be done without getting into configuration mode too. (As you also did with the CEF entry removal procedure explained in past section).
Basically, there are two situations here, when you have a valid ARP entry to the next hop, in this case 10.1.2.1 and when you don't (For any reason). The second situation forces you to actually create a valid ARP entry (via static ARP):
Step 1. There is an ARP entry in the switch for 10.1.2.1 which is the next hop for 1.1.1.1.
MXC.CALO.Sup2T#show ip arp 10.1.2.1 Protocol Address Age (min) Hardware Addr Type Interface Internet 10.1.2.1 2 0c11.678b.f6f7 ARPA Vlan2 MXC.CALO.Sup2T#show ip route | inc 1.1.1.1 S 1.1.1.1 [1/0] via 10.1.2.1
An ARP entry is programmed as a host route ( /32 ) in the CEF table:
MXC.CALO.Sup2T-dfc3#show plat hard cef vpn 0 look 10.1.2.1 Codes: decap - Decapsulation, + - Push Label Index Prefix Adjacency 53 10.1.2.1/32 Vl2 ,0c11.678b.f6f7 And of course, there is an index for this which again will tell us how a packet should be rewritten to reach 10.1.2.1: MXC.CALO.Sup2T-sw2-dfc3#show plat hard cef vpn 0 10.1.2.1 detail [snip...] Format:IPV4 (valid class vpn prefix) M(53 ): 1 F 2FFF 255.255.255.255 V(53 ): 1 0 0 10.1.2.1 (A:114689, LS:0, NR:0, RI:0, DF:0 CP:0 DGTv:1, DGT:0) Wait, wasn't 114689 adj entry the same used for 1.1.1.1?: MXC.CALO.Sup2T-sw2-dfc3#show plat hard cef 1.1.1.1 de [snip...] Format:IPV4 (valid class vpn prefix) M(54 ): 1 F 2FFF 255.255.255.255 V(54 ): 1 0 0 1.1.1.1 (A:114689, LS:0, NR:0, RI:0, DF:0 CP:0 DGTv:1, DGT:0)
Any packet with any destination IP address that has the same data link next hop should be forwarded through the same interface and rewritten with the same L2 headers.
Even though this might seem pretty obvious at first, it is actually the most important element to add a CEF entry, you need to tell it how a packet should be rewritten with a specific CEF Adjacency Entry.
Step 2. Now, suppose there is no ARP entry automatically created for this, so you need to create a static ARP entry.
In order to do this, you need to know the MAC address of the device that is used as next-hop for prefix 10.1.2.1, so it is sent to 0c11.678b.f6f7. If there is already a MAC Address entry in the show mac address-table address 0c11.678b.f6f7 command output that is fine, if not then you need to create a static MAC entry:
MXC.CALO.Sup2T(config)#mac address-table static 0c11.678b.f6f7 vlan 2 int Gi3/21 Displaying entries from DFC switch [2] linecard [3]: vlan mac address type learn age ports ----+----+---------------+-------+-----+----------+----------------------------- 2 0c11.678b.f6f7 static No - Gi3/21
Step 3. Finally, a static ARP entry needs to be created in order for a CEF entry to be programmed:
MXC.CALO.Sup2T(config)#arp 10.1.2.1 0c11.678b.f6f7 arpa <<< Static ARP configuration MXC.CALO.Sup2T#show ip arp 10.1.2.1 Protocol Address Age (min) Hardware Addr Type Interface Internet 10.1.2.1 - 0c11.678b.f6f7 ARPA <<< Now the static ARP entry is complete
// Attaching to DFC3...
MXC.CALO.Sup2T-sw2-dfc3#show plat hard cef 10.1.2.1 detail [snip...] Format:IPV4 (valid class vpn prefix) M(53 ): 1 F 2FFF 255.255.255.255 V(53 ): 1 0 0 10.1.2.1 (A:114689, LS:0, NR:0, RI:0, DF:0 CP:0 DGTv:1, DGT:0)
The ARP entry exist in CEF table for DFC3. Same Adjacency Index result as before...
Now that you understand what these adjacency entries do, you can finally proceed to add a CEF entry. In the last section, the CEF entry for prefix 1.1.1.1/32 was deleted through the test platform hardware cef v4-delete command. Now, add it back through the command test platform hardware cef v4-insert [prefix] [mask length] vpn [vpn number] adjacency [adjacency index]
In order to verify this, use the command test platform hardware cef v4-insert 1.1.1.1 32 vpn 0 adjacency 114689. The entry has been added back in the DFC CEF table:
MXC.CALO.Sup2T-sw2-dfc3#test platform hardware cef v4-insert 1.1.1.1 32 vpn 0 adjacency 114689 test_ipv4_insert: done: sw_index = 42 MXC.CALO.Sup2T-sw2-dfc3#show plat hard cef vpn 0 1.1.1.1 Codes: decap - Decapsulation, + - Push Label Index Prefix Adjacency 54 1.1.1.1/32 Vl2 ,0c11.678b.f6f7 Ping from the 3750X to Loopback 0 is successful and HW forwarded by 6500 DFC. MXC.CALO.Sup2T-sw2-dfc3#show platform hard cef adj entry 114689 Index: 114689 -- Valid entry (valid = 1) -- RIT fields: The entry has a Layer2 Format _________________________________________________________ |decr_ttl=YES | l2_fwd=NO | ccc = 4 | add_shim_hdr = NO |_____________|____________|_________|____________________ Statistics: Packets = 684 Bytes = 80712
// Logs in 3850
CALO.MXC.385024XU#show logging [snip...] *Jan 23 05:59:56.911: ICMP: echo reply sent, src 1.1.1.1, dst 10.1.1.1, topology BASE, dscp 0 topoid 0 *Jan 23 05:59:57.378: ICMP: echo reply sent, src 1.1.1.1, dst 10.1.1.1, topology BASE, dscp 0 topoid 0 *Jan 23 05:59:57.390: ICMP: echo reply sent, src 1.1.1.1, dst 10.1.1.1, topology BASE, dscp 0 topoid 0
Throughout the configuration made from all previous steps, the vpn 0 string in the show platform hardware cef commands has been enforced. Even if it seems completely unnecessary since the command by default returns the entries for general routing table or vpn 0, this was done on purpose in order to always have in mind that entries are added or deleted from specific routing table instances (VRFs), through the document you added and deleted CEF entry 1.1.1.1/32. However, certain prefixes are very likely to exist in different VRFs (i. e. 10.x.x.x) and delete, add or modify a CEF entry for a wrong VRF can cause a negative impact.
Delete a CEF entry with prefix 1.1.1.1/32 for VRF TEST_VRF. For a detailed description of the addtion of CEF entries, refer to the Add a CEF Entry section of this document.
In order to add the VRF, change SVIs in 6500 switch to the proposed VRF with command ip vrf forwarding [VRF-NAME] and finally add the same static route in our TEST_VRF table:
MXC.CALO.Sup2T(config)#ip vrf TEST_VRF MXC.CALO.Sup2T(config-vrf)#int vlan 1 MXC.CALO.Sup2T(config-if)#ip vrf forwarding TEST_VRF % Interface Vlan1 IPv4 disabled and address(es) removed due to enabling VRF TEST_VRF MXC.CALO.Sup2T(config-if)#ip add 10.1.1.10 255.255.255.0 MXC.CALO.Sup2T(config-if)#int vlan 2 MXC.CALO.Sup2T(config-if)#ip vrf forwarding TEST_VRF % Interface Vlan2 IPv4 disabled and address(es) removed due to enabling VRF TEST_VRF MXC.CALO.Sup2T(config-if)#ip add 10.1.2.10 255.255.255.0 MXC.CALO.Sup2T(config)#ip route vrf TEST_VRF 1.1.1.1 255.255.255.255 10.1.2.1
MXC.CALO.Sup2T#show ip vrf
Name Default RD Interfaces
TEST_VRF <not set> Vl1
Vl2
The VRFs are programmed sequentially too. This was the first VRF in the switch (no other VRF was configured previously) so the vpn number for this VRF instance should be 1. Run the show platform hardware cef vpn 1 command in order to verify this is true:
MXC.CALO.Sup2T-sw2-dfc3#show plat hard cef vpn 1 Codes: decap - Decapsulation, + - Push Label Index Prefix Adjacency 34 10.1.1.10/32 receive 35 10.1.1.0/32 receive 36 10.1.1.255/32 receive 38 10.1.2.10/32 receive 43 10.1.2.0/32 receive 44 10.1.2.255/32 receive 53 10.1.2.1/32 Vl2 ,0c11.678b.f6f7 54 1.1.1.1/32 Vl2 ,0c11.678b.f6f7 [snip...] However, usually, switches have hundred or thousands of VRFs and just count them in the 'show ip vrf' command output would be quite difficult. In order to know which VPN number is assigned to a VRF we will run the command "show platform hardware cef vrf [VRF name] [prefix] detail", it will return the actual vpn number for that VRF: Format:IPV4 (valid class vpn prefix) M(54 ): 1 F 2FFF 255.255.255.255 V(54 ): 1 0 1 1.1.1.1 <<<<<<<<<<< The number in red determines the VPN this prefix belongs to. (A:114689, LS:0, NR:0, RI:0, DF:0 CP:0 DGTv:1, DGT:0)
It is important to know the actual VPN number and the software index for this entry so you can proceed to delete it or add it from/to this VRF instance:
MXC.CALO.Sup2T-sw2-dfc3#test platform hardware cef index-conv hw_to_sw 54 hw index: 54 ----> sw handle: 42 MXC.CALO.Sup2T-sw2-dfc3#test platform hardware cef v4-delete 42 mask 32 vpn 1 test_ipv4_delete: done. Result: MXC.CALO.Sup2T-sw2-dfc3#show platform hardware cef vpn 1 lookup 1.1.1.1 Codes: decap - Decapsulation, + - Push Label Index Prefix Adjacency 262049 0.0.0.0/0 drop Traffic is now getting punted, and the effects are seen in the 3750X pings to 1.1.1.1: MXC.CALO.3750X#ping 1.1.1.1 repe 5000000 Sending 5000000, 100-byte ICMP Echos to 1.1.1.1, timeout is 2 seconds: !!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!! [snip...]
// Packet loss
Consider that in a production network, packet loss and choppy audio or bad video is experienced due to these CEF entries condition. Therefore, it is recommended to perform these tests in a maintenance window.