Out of Resource Handling of FIB Resources
The main function of the FIB within routers/line-cards is to map destination IP prefixes/labels to potential egress interfaces. In order to achieve this, the FIB maintains multiple databases or hardware tables, such as:
-
Forward Equivalence Class (FEC) and the Equal Cost Multi-Path Forward Equivalence Class (ECMP FEC): This database mainly stores information related to the IPv4 and IPv6 prefixes that the router has learnt. When a traffic packet reaches the router, it performs a FIB lookup on the destination prefix of the incoming packet. This lookup results in the FEC or ECMP FEC object for that particular prefix and it provides information about the egress interface on the router through which the packet can reach its destination.
-
Egress Encapsulation Data Base (EEDB): This database stores adjacency details such as the link-local details from Address Resolution Protocol (ARP) or Neighbor Discovery (ND) Protocol, Multiprotocol Label Switching (MPLS) labels, Generic Routing Encapsulation (GRE) headers and so on.
OOR Protection Mode
When the FIB resource usage exceeds the red threshold, the system enters OOR protection mode.
In FEC OOR protection mode, the router doesn’t allocate any FEC objects when it learns a new prefix, but assigns a pre-created drop FEC instead. The router drops incoming packets with destination IP addresses that were assigned the drop FEC.
In ECMP FEC OOR protection mode, the router chooses only the first path among multiple paths to forward traffic. When FEC or ECMP FEC usage drops below yellow threshold, the router reprograms the affected prefixes to normal FEC or ECMP FEC. Thereafter, the router resumes traffic flow.
In EEDB OOR protection mode, the router will not program any more egress enacapsulation information. This in turn results in traffic drop. When EEDB usage drops below yellow threshold, the router resumes programing new egress encapsulation data as well as traffic flow.
Configuration Steps to Change OOR Threshold Levels
The following section shows how to change the default threshold levels for OOR handling.
Router# conf t
Router(config)# oor hw threshold red 90
Router(config)# oor hw threshold yellow 75
Router(config)# commitRelease Stale FEC Resources
To release the stale FEC resources, execute the following command:
Router# clear cefVerification of FEC Resources
|
Feature Name |
Release Information |
Feature Description |
|---|---|---|
|
Hierarchical Forward Equivalence Class (HFEC) Out of Resource (OOR) Enhancements |
Release 7.5.1 |
With this enhancement, you can view the details of the utilization of Forwarding Information Base (FIB) hardware resources, such as hierarchical FEC and hierarchical Equal Cost Multi-Path (ECMP) FEC, in the output of the command show controllers npu resources. This feature also enables the router to display system logs on the console that alert you when FEC resources have crossed the OOR threshold levels. These logs help you to take corrective action and free up FEC resources, to minimize traffic loss. . |
Use the show controllers npu resources command to verify FIB resources.
The OOR State in the output of the show controllers npu resources command changes when the router reaches an OOR situation. The OOR State changes from Green to Yellow, and finally to Red depending on the utilization of FEC or ECMP FEC resources.
Router# show controllers npu resources fec location 0/0/CPU0
HW Resource Information
Name : fec
Asic Type : Qumran
NPU-0
OOR Summary
Estimated Max Entries : 126976
Red Threshold : 95 %
Yellow Threshold : 80 %
OOR State : Green
Bank Info : FEC
OFA Table Information
(May not match HW usage)
ipnhgroup : 43058
ip6nhgroup : 2
edpl : 0
limd : 0
punt : 19
iptunneldecap : 0
ipmcroute : 1
ip6mcroute : 0
ipnh : 0
ip6nh : 0
mplsmdtbud : 0
ipvrf : 2
ippbr : 0
redirectvrf : 0
l2protect : 0
l2bridgeport : 0
Current Hardware Usage
Name: fec
Estimated Max Entries : 126976
Total In-Use : 43082 (33 %)
OOR State : Green
Bank Info : FEC
Name: hier_0
Estimated Max Entries : 126976
Total In-Use : 43082 (33 %)
OOR State : Green
Bank Info : FECFIB has three FEC hierarchies and 3 ECMP FEC hierarchies. From Cisco IOS XR Release 7.5.1 onwards, you can view these hierarchies in the output of the commands show controllers npu resources fec and show controllers npu resources ecmpfec . The router records the OOR State of each of these hierarchies in the command outputs.
Router# show controllers npu resources fec location 1/0/CPU0
HW Resource Information
Name : fec
Asic Type : Jericho 2
NPU-0
OOR Summary
Estimated Max Entries : 576704
Red Threshold : 95 %
Yellow Threshold : 80 %
OOR State : Green
Bank Info : FEC
OFA Table Information
(May not match HW usage)
ipnhgroup : 157800
ip6nhgroup : 17377
edpl : 0
limd : 0
punt : 18
iptunneldecap : 0
ipmcroute : 1
ip6mcroute : 0
ipnh : 0
ip6nh : 0
mplsmdtbud : 0
ipvrf : 1053
ippbr : 0
redirectvrf : 1
l2protect : 42
l2bridgeport : 58
Current Hardware Usage
Name: fec
Estimated Max Entries : 576704
Total In-Use : 176350 (30 %)
OOR State : Green
Bank Info : FEC
Name: hier_0
Estimated Max Entries : 131072
Total In-Use : 3207 (2 %)
OOR State : Green
Bank Info : H1 FEC
Name: hier_1
Estimated Max Entries : 262144
Total In-Use : 63 (0 %)
OOR State : Green
Bank Info : H2 FEC
Name: hier_2
Estimated Max Entries : 183488
Total In-Use : 173080 (94 %)
OOR State : Yellow
OOR State Change Time : 2021.Aug.29 22:14:16 PDT
Bank Info : H3 FECRouter# show controllers npu resources ecmpfec location 0/0/CPU0
HW Resource Information
Name : ecmp_fec
Asic Type : Jericho 2
NPU-0
OOR Summary
Estimated Max Entries : 32768
Red Threshold : 95 %
Yellow Threshold : 80 %
OOR State : Yellow
OOR State Change Time : 2021.Aug.29 23:07:53 PDT
Bank Info : ECMP
OFA Table Information
(May not match HW usage)
ipnhgroup : 30654
ip6nhgroup : 4
Current Hardware Usage
Name: ecmp_fec
Estimated Max Entries : 32768
Total In-Use : 30658 (93 %)
OOR State : Yellow
OOR State Change Time : 2021.Aug.29 23:07:53 PDT
Bank Info : ECMP
Name: hier_0
Total In-Use : 0
OOR State : Green
Bank Info : H1 ECMP
Name: hier_1
Total In-Use : 1
OOR State : Green
Bank Info : H2 ECMP
Name: hier_2
Total In-Use : 30657
OOR State : Yellow
OOR State Change Time : 2021.Aug.29 23:07:53 PDT
Bank Info : H3 ECMPSystem Log Alerts for HFEC OOR
When utilization of any of the FEC or ECMP FEC hierarchies transition from the current OOR state to another state, the router generates system logs to alert the user. You can then take corrective action if the router is approaching OOR state to ensure that FEC resources get freed up before traffic loss takes place.
LC/0/0/CPU0: fia_driver[170]: %PLATFORM-OFA-1-OOR_RED : NPU 0, Table ipnhgroup, Resource fec
LC/0/0/CPU0: fia_driver[170]: %PLATFORM-OFA-4-OOR_YELLOW : NPU 0, Table ipnhgroup, Resource fec
LC/0/0/CPU0: fia_driver[170]: %PLATFORM-OFA-5-OOR_GREEN : NPU 0, Table ipnhgroup, Resource fec
LC/0/0/CPU0: fia_driver[140]: %PLATFORM-OFA-4-OOR_YELLOW : NPU 0, Table ipnhgroup, Resource ecmp_fec
LC/0/0/CPU0: fib_mgr[133]: %PLATFORM-PLAT_FIB-4-OOR_PROT_STOP_WARNING : ECMP_FEC resources are now available. OOR protection stopped
LC/0/0/CPU0: fia_driver[140]: %PLATFORM-OFA-5-OOR_GREEN : NPU 0, Table ipnhgroup, Resource ecmp_fecVerification of EEDB Resources
|
Feature Name |
Release Information |
Feature Description |
|---|---|---|
|
Verification of EEDB OOR |
Release 7.6.1 |
If the Egress Encapsulation Database (EEDB) resources exceed the Out of Resource (OOR) threshold levels, the router generates system logs that indicate the resource utilization of EEDB hardware. With these details, you can take corrective action by freeing up EEDB resources, thereby minimizing traffic loss. This feature also updates the show output of the following command to display utilization of EEDB hardware resources: |
The router processes several entries and often stores these entries in the form of tables. These tables are further divided into smaller tables. These smaller tables are called as banks. The banks are often named as bank_0, bank_1, bank_2, and so on. Router segregates the entries through these banks.
 Note |
bank_0 will always be indicated as completely utilized and in Red OOR state on the following routers and line cards:
This complete utilization of bank_0 must be ignored since it is reserved for internal usage regardless of the router configuration. |
Use show controllers npu resources encap command to verify the usage of EEDB resources.
The OOR State in the output of the show controllers npu resources encap command changes when the router reaches an OOR situation. The OOR State changes from Green to Yellow, and finally to Red depending on the utilization of the EEDB resources.
Router# show controllers npu resources encap location 1/0/CPU0
HW Resource Information
Name : encap
Asic Type : Jericho
NPU-0
OOR Summary
Red Threshold : 95 %
Yellow Threshold : 80 %
OFA Table Information
(May not match HW usage)
ipnh : 13
ip6nh : 0
mplsnh : 0
llnh : 0
srv6nh : 0
ipvrf : 0
mplsmdtbud : 0
iptunnelencap : 0
tep : 0
Current Hardware Usage
Name: encap
Name: bank_0
Estimated Max Entries : 4096
Total In-Use : 4096 (100 %)
OOR State : Red
OOR State Change Time : 2022.Mar.15 05:33:14 UTC
Bank Info : phase=2 extended=no
Name: bank_1
Estimated Max Entries : 4096
Total In-Use : 4 (0 %)
OOR State : Green
Bank Info : phase=8 extended=no
Name: bank_2
Estimated Max Entries : 4096
Total In-Use : 0 (0 %)
OOR State : Green
Bank Info : phase=0 extended=no
Name: bank_3
Estimated Max Entries : 4096
Total In-Use : 0 (0 %)
OOR State : Green
Bank Info : phase=0 extended=no
From Cisco IOS XR Release 7.6.1 onwards, the output of the command show controllers npu resources encap has been enhanced to provide more fields such as Current Hardware Usage and Cluster Bank Pair details.
Router# show controllers npu resources encap location 1/0/CPU0
HW Resource Information
Name : encap
Asic Type : Jericho Two
NPU-0
OOR Summary
Red Threshold : 95 %
Yellow Threshold : 80 %
OFA Table Information
(May not match HW usage)
ipnh : 13
ip6nh : 0
mplsnh : 0
llnh : 0
srv6nh : 0
ipvrf : 0
mplsmdtbud : 0
iptunnelencap : 0
tep : 0
Current J2 Hardware Usage
Cluster Bank Pair: EEDB_S2_XL
Max-Entries : 163840
Total In-use : 0
Logical_phase : 1 (Encap_Rif)
Cluster Bank
Bank Size : S2
Estimated Max Entries : 163840
OOR State : Green
Total In-Use : 0
EEDB Bank:
Estimated Max Entries : 8192
OOR State : Green
Total In-Use : 0
Logical_phase : 6 (Tunnel4)
Cluster Bank
Bank Size : XL
Estimated Max Entries : 122880
OOR State : Green
Total In-Use : 0
EEDB Bank:
Estimated Max Entries : 61440
OOR State : Green
Total In-Use : 0
Cluster Bank Pair: EEDB_L1_S1
Max-Entries : 98304
Total In-use : 26
Logical_phase : 2 (Encap_NativeArp)
Cluster Bank
Bank Size : L1
Estimated Max Entries : 81920
OOR State : Green
Total In-Use : 0
EEDB Bank:
Estimated Max Entries : 30720
OOR State : Green
Total In-Use : 0
Logical_phase : 8 (Encap_Ac)
Cluster Bank
Bank Size : S1
Estimated Max Entries : 98304
OOR State : Green
Total In-Use : 26
EEDB Bank:
Estimated Max Entries : 8192
OOR State : Green
Total In-Use : 0
Cluster Bank Pair: EEDB_M1_M2
Max-Entries : 131072
Total In-use : 62
Logical_phase : 3 (Encap_NativeAc or Tunnel1)
Cluster Bank
Bank Size : M1
Estimated Max Entries : 122880
OOR State : Green
Total In-Use : 0
EEDB Bank:
Estimated Max Entries : 16384
OOR State : Green
Total In-Use : 0 System Log Alerts for EEDB OOR
When utilization of EEDB resources transition from the current OOR state to another state, the router generates system logs to alert the user. You can then take corrective action if the router is approaching OOR state to ensure that EEDB resources get freed up before traffic loss takes place.
The router logs the following log messages to inform you of the OOR state of the encap resource when it exceeds thresholds. The syslog also indicates the Network Processing Unit (NPU) of the affected encap resource, which is NPU0, in this case.
LC/0/0/CPU0: fia_driver[164]: %PLATFORM-OFA-4-OOR_YELLOW : NPU 0, Table mplsnh, Resource encap
LC/0/0/CPU0: fia_driver[164]: %PLATFORM-OFA-1-OOR_RED : NPU 0, Table mplsnh, Resource encap 
Feedback