CoPP on Nexus 7000 Series Switches

Introduction

This document describes what, how, and why Control Plane Policing (CoPP) is used on the Nexus 7000 Series Switches, which include the F1, F2, M1, and M2 Series Modules and line cards (LCs). It also includes best practice policies, as well as how to customize a CoPP policy.

Prerequisites

Requirements

Cisco recommends that you have knowledge of Nexus operating system CLI.

Components Used

The information in this document is based on the Nexus 7000 Series Switches with Supervisor 1 Module.

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, make sure that you understand the potential impact of any command.

CoPP on the Nexus 7000 Series Switch Overview

The CoPP is critical to network operation. A Denial of Service (DoS) attack to the Control/Management Plane, which can be perpetrated either inadvertently or maliciously, typically involves high rates of traffic that result in excessive CPU utilization. The Supervisor module spends an inordinate amount of time handling the packets.

Examples of such attacks include:

• Internet Control Message Protocol (ICMP) echo requests.
  
  
• Packets sent with ip-options set.

This can lead to:

• Loss of keep-alive messages and routing protocol updates.
  
  
• Filling of packet queues, which results in indiscriminate drops.
  
  
• Slow or unresponsive interactive sessions.

Attacks can overwhelm the network stability and availability and lead to business-impacting network outages.

CoPP is a hardware-based feature that protects the Supervisor from DoS attacks. It controls the rate at which packets are allowed to reach the Supervisor. The CoPP feature is modeled like an input QoS policy attached to the special interface called the control-plane. However, CoPP is a security feature and not part of QoS. In order to protect the Supervisor, the CoPP separates data plane packets from the control plane packets (Exception Logic). It identifies DoS attack packets from valid packets (Classification). CoPP allows for classification of these packets:

• Receive packets
• Multicast packets
• Exception packets
• Redirect packets
• Broadcast MAC + non-IP packets
• Broadcast MAC + IP packets (See Cisco Bug ID CSCub47533 - Packets in L2 Vlan (No SVI) hitting CoPP)
• Mcast MAC + IP packets
• Router MAC + non-IP packets
• ARP packets

After a packet is classified, the packet can also be marked and used to assign different priorities based on the type of packets. Conform, exceed, and violate actions (transmit, drop, mark-down) can be set. If no policer is attached to a class, then a default policer is added whose conform action is drop. Glean packets are policed with default-class. One rate, two color, and two rate, three color policing are supported.

Traffic that hits the CPU on the Supervisor module can come in through four paths:

1. Inband interfaces (front panel port) for traffic sent by line cards.
   
   
2. Management Interface (mgmt0) used for management traffic.
   
   
3. Control and Monitoring Processor (CMP) interface used for the console.
   
   
4. Switched Ethernet Out Band Channel (EOBC) to control the line cards from the Supervisor module and exchange status messages.

Only the traffic sent through the Inband interface is subject to CoPP, because this is the only traffic that reaches the Supervisor module through the forwarding engines (FEs) on the line cards. The Nexus 7000 Series Switch implementation of CoPP is hardware-based only, which means that CoPP is not performed in software by the Supervisor module. CoPP functionality (policing) is implemented on each FE independently. When the various rates are configured for CoPP policy-map, consideration must be taken in regard to the number of line cards in the system.

The total traffic received by the Supervisor is N times X, where N is the number of FEs on the Nexus 7000 system, and X is the rate allowed for the particular class. The configured policer values apply on a per FE basis, and the aggregate traffic prone to hit the CPU is the sum of the conformed and transmitted traffic on all of the FEs. In other words, traffic that hits the CPU equals the configured conform rate multiplied by the number of FEs.

• N7K-M148GT-11/L LC has 1 FE
• N7K-M148GS-11/L LC has 1 FE
• N7K-M132XP-12/L LC has 1 FE
• N7K-M108X2-12L LC has 2 FE
• N7K-F248XP-15 LC has 12 FE (SOC)
• N7K-M235XP-23L LC has 2 FE
• N7K-M206FQ-23L LC has 2 FE
• N7K-M202CF-23L LC has 2 FE

CoPP configuration is only implemented in the default virtual device context (VDC); however, the CoPP policies are applicable for all VDCs. The same global policy is applied for all line cards. CoPP applies resource sharing between VDCs if ports of the same FEs belong to different VDCs (M1 Series or M2 Series LC). For example, ports of one FE, even in different VDCs, count against the same threshold for CoPP.

If the same FE is shared between different VDCs and a given class of control plane traffic exceeds the threshold, this affects all VDCs on the same FE. It is recommended to dedicate one FE per VDC in order to isolate CoPP enforcement, if possible.

When the switch comes up first time, the default policy must be programmed to protect the control-plane. CoPP provides the default policies, which are applied to control-plane as part of the initial startup sequence.

Why CoPP on the Nexus 7000 Series Switch

The Nexus 7000 Series Switch is deployed as an aggregation or core switch. Hence, it is the ear and brain of the network. It handles the maximum load in the network. It must handle frequent and burst requests. Some of requests include:

• Spanning Tree Bridge Protocol Data Unit (BPDU) Processing - Default is every two seconds.
  
  
• First Hop Redundancy - This includes Hot Standby Router Protocol (HSRP), Virtual Router Redundancy Protocol (VRRP), and Gateway Load Balancing Protocol (GLBP) - Default is every three seconds.
  
  
• Address Resolution - This includes Address Resolution Protocol/Neighbor-Discovery (ARP/ND), Forwarding Information Base (FIB) Glean - Up to one request per second, per host , such as network interface controller (NIC) teaming.
  
  
• Dynamic Host Control Protocol (DHCP) - DHCP Request, Relay - Up to one request per second, per host.
  
  
• Routing Protocols for Layer 3 (L3).
  
  
• Data Center Interconnect - Overlay Transport Virtualization (OTV), Multiprotocol Label Switching (MPLS), and Virtual Private LAN Service (VPLS).

CoPP is essential in order to protect the CPU against misconfigured servers or potential DoS attacks, which allows the CPU to have enough cycle to process critical control plane messages.

Control Plane Processing on the Nexus 7000 Series Switch

The Nexus 7000 Series Switch takes a distributed control plane approach. It has a multi-core on each I/O module, as well as a multi-core for switch control plane on the Supervisor module. It offloads intensive tasks to the I/O module CPU for access control lists (ACL) and FIB programming. It scales the control plane capacity with the number of line cards. This avoids Supervisor CPU bottleneck, which is seen in a centralized approach. Hardware rate limiters and hardware-based CoPP protects the control plane from bad or malicious activity.

CoPP Best Practices Policy

CoPP Best Practices Policy (BPP) was introduced in Cisco NX-OS Release 5.2. The show running-config command output does not display the content of the CoPP BPP. The show run all command displays the content of CoPP BPP.

------------------------------------SNIP-----------------------------------------
SITE1-AGG1# show run copp
 
!! Command: show running-config copp
!! Time: Mon Nov  5 22:21:04 2012
 
version 5.2(7)
copp profile strict
 
 
SITE1-AGG1# show run copp all
 
!! Command: show running-config copp all
!! Time: Mon Nov  5 22:21:15 2012
 
version 5.2(7)
------------------------------SNIP---------------------
control-plane
  service-policy input copp-system-p-policy-strict
copp profile strict

CoPP provides four options to the user for default policies:

• Strict
• Moderate
• Lenient
• Dense (introduced in Release 6.0(1))

If no option is selected or if set up is skipped, then strict policing is applied. All of these options use the same class-maps and classes, but different Committed Information Rate (CIR) and Burst Count (BC) values for policing. In Cisco NX-OS releases earlier than 5.2.1, the setup command was used to change the option. Cisco NX-OS Release 5.2.1 introduced an enhancement to the CoPP BPP so that the option can be changed without the setup command; use the copp profile command.

SITE1-AGG1# conf t
Enter configuration commands, one per line.  End with CNTL/Z.
SITE1-AGG1(config)# copp profile ?
  dense     The Dense   Profile
  lenient   The Lenient Profile
  moderate  The Moderate Profile
  strict    The Strict Profile
SITE1-AGG1(config)# copp profile strict
SITE1-AGG1(config)# exit

Use the show copp profile <profile-type> command to view the default CoPP BPP configuration. Use the show copp status command to verify that the CoPP policy has been applied correctly.

SITE1-AGG1# show copp status
Last Config Operation: copp profile strict
Last Config Operation Timestamp: 20:40:27 PST Nov  5 2012
Last Config Operation Status: Success
Policy-map attached to the control-plane: copp-system-p-policy-strict

In order to view the difference between two CoPP BPPs, use the show copp diff profile <profile-type 1> profile <profile-type 2> command:

SITE1-AGG1# show copp diff profile strict profile moderate
A '+' represents a line that has been added and
a '-' represents a line that has been removed.
    -policy-map type control-plane copp-system-p-policy-strict
    -  class copp-system-p-class-critical
    -    set cos 7
    -    police cir 39600 kbps bc 250 ms conform transmit violate drop
    -  class copp-system-p-class-important
    -    set cos 6
    -    police cir 1060 kbps bc 1000 ms conform transmit violate drop
----------------------SNIP---------------------------------------
    +policy-map type control-plane copp-system-p-policy-moderate
    +  class copp-system-p-class-critical
    +    set cos 7
    +    police cir 39600 kbps bc 310 ms conform transmit violate drop
    +  class copp-system-p-class-important
    +    set cos 6
    +    police cir 1060 kbps bc 1250 ms conform transmit violate drop
----------------------SNIP---------------------------------------

How to Customize a CoPP Policy

Users can create a customized CoPP policy. Clone the default CoPP BPP, and attach it to the control-plane interface because the CoPP BPP is read-only.

SITE2-AGG1(config)# policy-map type control-plane copp-system-p-policy-strict
                                                                             ^
% String is invalid, 'copp-system-p-policy-strict' is not an allowed string at 
   '^' marker.

The copp copy profile <profile-type> <prefix> [suffix] command creates a clone of the CoPP BPP. This is used in order to modify the default configurations. The copp copy profile command is an exec mode command. User can choose a prefix or suffix for the access-list, class-maps, and policy-map name. For instance, copp-system-p-policy-strict is changed to [prefix]copp-policy-strict[suffix]. Cloned configurations are treated as user configurations and are included in the show run output.

SITE1-AGG1# copp copy profile ?
  dense     The Dense   Profile
  lenient   The Lenient Profile
  moderate  The Moderate Profile
  strict    The Strict Profile
SITE1-AGG1# copp copy profile strict ?
  prefix  Prefix for the copied policy
  suffix  Suffix for the copied policy
SITE1-AGG1# copp copy profile strict suffix ?
  WORD  Enter prefix/suffix for the copied policy (Max Size 20)
SITE1-AGG1# copp copy profile strict suffix CUSTOMIZED-COPP
SITE1-AGG1# show run copp | grep policy-map
policy-map type control-plane copp-policy-strict-CUSTOMIZED-COPP
SITE1-AGG1#

It is possible to mark down traffic that exceeds and violates a specified Permitted Information Rate (PIR) with these commands:

SITE1-AGG1(config)# policy-map type 
   control-plane copp-policy-strict-CUSTOMIZED-COPP
SITE1-AGG1(config-pmap)# class copp-class-critical-CUSTOMIZED-COPP
SITE1-AGG1(config-pmap-c)# police cir 59600 kbps bc 250 ms ?
  <CR>    
  conform  Specify a conform action
  pir      Specify peak information rate
 
SITE1-AGG1(config-pmap-c)# police cir 59600 kbps bc 250 ms pir ?
  <1-80000000000>  Peak Information Rate in bps/kbps/mbps/gbps
 
SITE1-AGG1(config-pmap-c)# police cir 59600 kbps bc 250 ms pir 100 mbps ?
  <CR>          
  <1-512000000>  Peak Burst Size in bytes/kbytes/mbytes/packets/ms/us
  be             Specify extended burst
  conform        Specify a conform action
 
SITE1-AGG1(config-pmap-c)# police cir 59600 kbps bc 250 ms pir 100 mbps conform ?
  drop               Drop the packet
  set-cos-transmit   Set conform action cos val
  set-dscp-transmit  Set conform action dscp val
  set-prec-transmit  Set conform action precedence val
  transmit           Transmit the packet
 
SITE1-AGG1(config-pmap-c)# police cir 59600 kbps bc 250 ms pir 100 mbps conform
 set-dscp-transmit ef exceed set dscp1 dscp2 table cir-markdown-map violate
 set1 dscp3 dscp4 table1 pir-markdown-map
SITE1-AGG1(config-pmap-c)#

Apply the customized CoPP policy to the global interface control-plane. Use the show copp status command in order to verify that the CoPP policy has been applied correctly.

SITE1-AGG1# conf t
Enter configuration commands, one per line.  End with CNTL/Z.
SITE1-AGG1(config)# control-plane
SITE1-AGG1(config-cp)# service-policy input ?
  copp-policy-strict-CUSTOMIZED-COPP 
 
SITE1-AGG1(config-cp)# service-policy input copp-policy-strict-CUSTOMIZED-COPP
SITE1-AGG1(config-cp)# exit
SITE1-AGG1# sh copp status
Last Config Operation: service-policy input copp-policy-strict-CUSTOMIZED-COPP
Last Config Operation Timestamp: 18:04:03 UTC May 15 2012
Last Config Operation Status: Success
Policy-map attached to the control-plane: copp-policy-strict-CUSTOMIZED-COPP

Customized CoPP Policy Case Study

This section describes a real example in which the customer requires multiple monitoring devices in order to frequently ping the local interfaces. Difficulty is encountered in this scenario when the customer wants to modify the CoPP policy in order to:

• Increase the CIR so that these specific addresses can ping the local device and not violate the policy.
  
  
• Allow the other IP addresses to maintain the ability to ping the local device, but at a lower CIR for troubleshooting purposes.

The solution is shown in the next example, which is to create a customized policy with a separate class-map. The separate class-map contains the specified IP addresses of the monitoring devices and the class-map has a higher CIR. This also leaves the original class-map monitoring, which captures ICMP traffic for all of the other IP addresses at a lower CIR.

F340.13.19-Nexus7000-1#
F340.13.19-Nexus7000-1#
F340.13.19-Nexus7000-1# conf t
Enter configuration commands, one per line. End with CNTL/Z.
F340.13.19-Nexus7000-1(config)# copp copy profile strict prefix TAC_CHANGE
F340.13.19-Nexus7000-1(config)#
F340.13.19-Nexus7000-1(config)#
F340.13.19-Nexus7000-1(config)# ip access-list TAC_CHANGE-copp-acl-specific-icmp
F340.13.19-Nexus7000-1(config-acl)#
F340.13.19-Nexus7000-1(config-acl)# permit icmp host 1.1.1.1 host 2.2.2.2 echo
F340.13.19-Nexus7000-1(config-acl)# permit icmp host 1.1.1.1 host 2.2.2.2 echo-reply
F340.13.19-Nexus7000-1(config-acl)#
F340.13.19-Nexus7000-1(config-acl)# exit
F340.13.19-Nexus7000-1(config)# sho ip access-lists TAC_CHANGE-copp-acl-specific-
icmp IP access list TAC_CHANGE-copp-acl-specific-icmp
 10 permit icmp 1.1.1.1/32 2.2.2.2/32 echo
 20 permit icmp 1.1.1.1/32 2.2.2.2/32 echo-reply
F340.13.19-Nexus7000-1(config)#
F340.13.19-Nexus7000-1(config)#
F340.13.19-Nexus7000-1(config)# class-map type control-plane match-any
 TAC_CHANGE-copp-class-specific-icmp
F340.13.19-Nexus7000-1(config-cmap)# match access-group name TAC_CHANGE-copp
-acl-specific-icmp
F340.13.19-Nexus7000-1(config-cmap)#exit
F340.13.19-Nexus7000-1(config)#
F340.13.19-Nexus7000-1(config)#policy-map type control-plane TAC_CHANGE-copp-
policy-strict
F340.13.19-Nexus7000-1(config-pmap)# class TAC_CHANGE-copp-class-specific-icmp 
insert-before
 TAC_CHANGE-copp-class-monitoring
F340.13.19-Nexus7000-1(config-pmap-c)# set cos 7 
F340.13.19-Nexus7000-1(config-pmap-c)# police cir 5000 kbps bc 250 ms conform transmit
 violate drop
F340.13.19-Nexus7000-1(config-pmap-c)# exit
F340.13.19-Nexus7000-1(config-pmap)#
F340.13.19-Nexus7000-1(config-pmap)#
F340.13.19-Nexus7000-1(config-pmap)#
F340.13.19-Nexus7000-1(config-pmap)#
F340.13.19-Nexus7000-1(config-pmap)# exit
F340.13.19-Nexus7000-1(config)#
F340.13.19-Nexus7000-1(config)#
F340.13.19-Nexus7000-1(config)# control-plane
F340.13.19-Nexus7000-1(config-cp)# service-policy input TAC_CHANGE-copp-policy-strict
F340.13.19-Nexus7000-1(config-cp)# end
F340.13.19-Nexus7000-1#
F340.13.19-Nexus7000-1# sho policy-map interface control-plane
Control Plane
 service-policy input TAC_CHANGE-copp-policy-strict
<abbreviated output>
class-map TAC_CHANGE-copp-class-specific-icmp (match-any)
 match access-group name TAC_CHANGE-copp-acl-specific-icmp
 set cos 7
 police cir 5000 kbps bc 250 ms
 conform action: transmit
 violate action: drop
 module 4:
 conformed 0 bytes,
 5-min offered rate 0 bytes/sec
 peak rate 0 bytes/sec
 violated 0 bytes,
 5-min violate rate 0 bytes/sec
 peak rate 0 bytes/sec
 module 7:
 conformed 0 bytes,
 5-min offered rate 0 bytes/sec
 peak rate 0 bytes/sec
 violated 0 bytes,
 5-min violate rate 0 bytes/sec
 peak rate 0 bytes/secclass-map TAC_CHANGE-copp-class-monitoring (match-any)
 match access-group name TAC_CHANGE-copp-acl-icmp
 match access-group name TAC_CHANGE-copp-acl-icmp6
 match access-group name TAC_CHANGE-copp-acl-mpls-oam
 match access-group name TAC_CHANGE-copp-acl-traceroute
 match access-group name TAC_CHANGE-copp-acl-http-response
 match access-group name TAC_CHANGE-copp-acl-smtp-response
 match access-group name TAC_CHANGE-copp-acl-http6-response
 match access-group name TAC_CHANGE-copp-acl-smtp6-response
 set cos 1
 police cir 130 kbps bc 1000 ms
 conform action: transmit
 violate action: drop
 module 4:
 conformed 0 bytes,
 5-min offered rate 0 bytes/sec
 peak rate 0 bytes/sec
 violated 0 bytes,
 5-min violate rate 0 bytes/sec
 peak rate 0 bytes/sec
 module 7:
 conformed 0 bytes,
 5-min offered rate 0 bytes/sec
 peak rate 0 bytes/sec
 violated 0 bytes,
 5-min violate rate 0 bytes/sec
 peak rate 0 bytes/sec
<abbreviated output>

CoPP Data Structure

CoPP BPP data structure is constructed as:

• ACL configuration: IP ACL and MAC ACL.
  
  
• Classifier configuration: Class-map matching IP ACL or MAC ACL.
  
  
• Policer configuration: Set CIR, BC, conform action, and violate action. The policer has two rates (CIR and BC), and two colors (conform and violate).

mac access-list copp-system-p-acl-mac-fabricpath-isis
  permit any 0180.c200.0015 0000.0000.0000
  permit any 0180.c200.0014 0000.0000.0000
 
ip access-list copp-system-p-acl-bgp
  permit tcp any gt 1024 any eq bgp
  permit tcp any eq bgp any gt 1024
 
class-map type control-plane match-any copp-system-p-class-critical
  match access-group name copp-system-p-acl-bgp
  match access-group name copp-system-p-acl-pim
  <snip>
  match access-group name copp-system-p-acl-mac-fabricpath-isis
policy-map type control-plane copp-system-p-policy-dense
  class copp-system-p-class-critical
    set cos 7
    police cir 5000 kbps bc 250 ms conform transmit violate drop

CoPP Scale Factor

Scale factor configuration introduced in Cisco NX-OS Release 6.0 is used to scale the policer rate of the applied CoPP policy for a particular line card. This increases or reduces the policer rate for a particular line card, but does not change the current CoPP policy. The changes are effective immediately, and there is no need to reapply the CoPP policy.

scale factor option configured within control-plane interface:
                Scale-factor <scale factor value> module <module number>
                <scale factor value>: from 0.10 to 2.00
Scale factor is recommended when a chassis is loaded with both F2 and M 
   Series modules.
SITE1-AGG1# conf t
Enter configuration commands, one per line.  End with CNTL/Z.
SITE1-AGG1(config)# control-plane
SITE1-AGG1(config-cp)# scale-factor ?
  <whole>.<decimal>  Specify scale factor value from 0.10 to 2.00
 
SITE1-AGG1(config-cp)# scale-factor 1.0 ?
  module  Module
 
SITE1-AGG1(config-cp)# scale-factor 1.0 module ?
  <1-10>  Specify module number
 
SITE1-AGG1(config-cp)# scale-factor 1.0 module 4
SITE1-AGG1# show system internal copp info
<snip>
Linecard Configuration:
-----------------------
Scale Factors
Module 1: 1.00
Module 2: 1.00
Module 3: 1.00
Module 4: 1.00
Module 5: 1.00
Module 6: 1.00
Module 7: 1.00
Module 8: 1.00
Module 9: 1.00
Module 10: 1.00

CoPP Monitoring and Management

With Cisco NX-OS Release 5.1, it is possible to configure a drop threshold per CoPP class name that triggers a Syslog message in the event the threshold is exceeded. The command is logging drop threshold <dropped bytes count> level <logging level>.

SITE1-AGG1(config)# policy-map type control-plane 
   copp-policy-strict-CUSTOMIZED-COPP
SITE1-AGG1(config-pmap)# class copp-class-critical-CUSTOMIZED-COPP
SITE1-AGG1(config-pmap-c)# logging ?
  drop  Logging for dropped packets
 
SITE1-AGG1(config-pmap-c)# logging drop ?
  threshold  Threshold value for dropped packets
 
SITE1-AGG1(config-pmap-c)# logging drop threshold ?
  <CR>            
  <1-80000000000>  Dropped byte count
 
SITE1-AGG1(config-pmap-c)# logging drop threshold 100 ?
  <CR>  
  level  Syslog level
 
SITE1-AGG1(config-pmap-c)# logging drop threshold 100 level ?
  <1-7>  Specify the logging level between 1-7
 
SITE1-AGG1(config-pmap-c)# logging drop threshold 100 level 7

Here is an example of a Syslog message:

%COPP-5-COPP_DROPS5: CoPP drops exceed threshold in class: 
   copp-system-class-critical, 
check show policy-map interface control-plane for more info.

CoPP Counters

CoPP supports the same QoS statistics as any other interface. It shows the statistics of the classes that form the service policy for every I/O module that supports CoPP. Use the show policy-map interface control-plane command to view the statistics for CoPP.

Note: All classes should be monitored in terms of violated packets.

SITE1-AGG1# show policy-map interface control-plane
Control Plane
 
  service-policy  input: copp-policy-strict-CUSTOMIZED-COPP
 
    class-map copp-class-critical-CUSTOMIZED-COPP (match-any)
      match access-group name copp-acl-bgp-CUSTOMIZED-COPP
      match access-group name copp-acl-bgp6-CUSTOMIZED-COPP
      match access-group name copp-acl-eigrp-CUSTOMIZED-COPP
      match access-group name copp-acl-igmp-CUSTOMIZED-COPP
      match access-group name copp-acl-msdp-CUSTOMIZED-COPP
      match access-group name copp-acl-ospf-CUSTOMIZED-COPP
      match access-group name copp-acl-ospf6-CUSTOMIZED-COPP
      match access-group name copp-acl-pim-CUSTOMIZED-COPP
      match access-group name copp-acl-pim6-CUSTOMIZED-COPP
      match access-group name copp-acl-rip-CUSTOMIZED-COPP
      match access-group name copp-acl-rip6-CUSTOMIZED-COPP
      match access-group name copp-acl-vpc-CUSTOMIZED-COPP
      match access-group name copp-acl-eigrp6-CUSTOMIZED-COPP
      match access-group name copp-acl-mac-l2pt-CUSTOMIZED-COPP
      match access-group name copp-acl-mpls-ldp-CUSTOMIZED-COPP
      match access-group name copp-acl-mpls-oam-CUSTOMIZED-COPP
      match access-group name copp-acl-mpls-rsvp-CUSTOMIZED-COPP
      match access-group name copp-acl-otv-as-CUSTOMIZED-COPP
      match access-group name copp-acl-mac-otv-isis-CUSTOMIZED-COPP
      match access-group name copp-acl-mac-fabricpath-isis-CUSTOMIZED-COPP
      match protocol mpls router-alert
      match protocol mpls exp 6
      set cos 7
      threshold: 100, level: 7
      police cir 39600 kbps , bc 250 ms
      module 1 :
        conformed 22454 bytes; action: transmit
        violated 0 bytes; action: drop
 
      module 2 :
        conformed 0 bytes; action: transmit
        violated 0 bytes; action: drop
 
      module 3 :
        conformed 19319 bytes; action: transmit
        violated 0 bytes; action: drop
 
      module 4 :
        conformed 0 bytes; action: transmit
        violated 0 bytes; action: drop

In order to obtain an aggregate view of conformed and violated counters for all class-map and I/O modules, use the show policy-map interface control-plane | i "class|conform|violated" command.

SITE1-AGG1# show policy-map interface control-plane | i "class|conform|violated"
    class-map copp-class-critical-CUSTOMIZED-COPP (match-any)
        conformed 123126534 bytes; action: transmit
        violated 0 bytes; action: drop
        conformed 0 bytes; action: transmit
        violated 0 bytes; action: drop
        conformed 107272597 bytes; action: transmit
        violated 0 bytes; action: drop
        conformed 0 bytes; action: transmit
        violated 0 bytes; action: drop
    class-map copp-class-important-CUSTOMIZED-COPP (match-any)
        conformed 0 bytes; action: transmit
        violated 0 bytes; action: drop
        conformed 0 bytes; action: transmit
        violated 0 bytes; action: drop
        conformed 0 bytes; action: transmit
        violated 0 bytes; action: drop
        conformed 0 bytes; action: transmit
        violated 0 bytes; action: drop

The class copp-class-l2-default and class-default should be monitored to ensure that there are no high increases, even for conformed counters. Ideally, these two classes must have low values for conformed counter and at least no violated counter increase.

ACL Counters

The statistics per-entry command is not supported for IP ACL or MAC ACL used in CoPP class-map, and it has no effect when applied to CoPP IP ACL or MAC ACL. (There is no CLI check done by the CLI Parser). In order to view the CoPP MAC ACL or IP ACL hits on an I/O module, use the show system internal access-list input entries detail command.

Here is an example:

!! 0180.c200.0041 is the destination MAC used for FabricPath IS-IS
 
SITE1-AGG1# show system internal access-list input entries det | grep 0180.c200.0041
  [00fc:00f7:00f7] qos 0000.0000.0000 0000.0000.0000 0180.c200.0041 ffff.ffff.ffff  [0]
  [0148:00fe:00fe] qos 0000.0000.0000 0000.0000.0000 0180.c200.0041 ffff.ffff.ffff  [0]
  [0148:00fe:00fe] qos 0000.0000.0000 0000.0000.0000 0180.c200.0041 ffff.ffff.ffff  [30042]
  [0148:00fe:00fe] qos 0000.0000.0000 0000.0000.0000 0180.c200.0041 ffff.ffff.ffff  [29975]
  [0148:00fe:00fe] qos 0000.0000.0000 0000.0000.0000 0180.c200.0041 ffff.ffff.ffff  [8965]
  [0148:00fe:00fe] qos 0000.0000.0000 0000.0000.0000 0180.c200.0041 ffff.ffff.ffff  [8935]
  [0148:00fe:00fe] qos 0000.0000.0000 0000.0000.0000 0180.c200.0041 ffff.ffff.ffff  [0]
  [0148:00fe:00fe] qos 0000.0000.0000 0000.0000.0000 0180.c200.0041 ffff.ffff.ffff  [58233]
  [0148:00fe:00fe] qos 0000.0000.0000 0000.0000.0000 0180.c200.0041 ffff.ffff.ffff  [27689]
  [0148:00fe:00fe] qos 0000.0000.0000 0000.0000.0000 0180.c200.0041 ffff.ffff.ffff  [0]
  [0148:00fe:00fe] qos 0000.0000.0000 0000.0000.0000 0180.c200.0041 ffff.ffff.ffff  [0]
  [0148:00fe:00fe] qos 0000.0000.0000 0000.0000.0000 0180.c200.0041 ffff.ffff.ffff  [0]
  [0148:00fe:00fe] qos 0000.0000.0000 0000.0000.0000 0180.c200.0041 ffff.ffff.ffff  [0]
  [00fc:00f7:00f7] qos 0000.0000.0000 0000.0000.0000 0180.c200.0041 ffff.ffff.ffff  [0]
  [0148:00fe:00fe] qos 0000.0000.0000 0000.0000.0000 0180.c200.0041 ffff.ffff.ffff  [0]
  [0148:00fe:00fe] qos 0000.0000.0000 0000.0000.0000 0180.c200.0041 ffff.ffff.ffff  [0]
  [0148:00fe:00fe] qos 0000.0000.0000 0000.0000.0000 0180.c200.0041 ffff.ffff.ffff  [0]
  [0148:00fe:00fe] qos 0000.0000.0000 0000.0000.0000 0180.c200.0041 ffff.ffff.ffff  [0]
  [0148:00fe:00fe] qos 0000.0000.0000 0000.0000.0000 0180.c200.0041 ffff.ffff.ffff  [0]
  [0148:00fe:00fe] qos 0000.0000.0000 0000.0000.0000 0180.c200.0041 ffff.ffff.ffff  [0]
  [0148:00fe:00fe] qos 0000.0000.0000 0000.0000.0000 0180.c200.0041 ffff.ffff.ffff  [0]
  [0148:00fe:00fe] qos 0000.0000.0000 0000.0000.0000 0180.c200.0041 ffff.ffff.ffff  [0]
  [0148:00fe:00fe] qos 0000.0000.0000 0000.0000.0000 0180.c200.0041 ffff.ffff.ffff  [0]
  [0148:00fe:00fe] qos 0000.0000.0000 0000.0000.0000 0180.c200.0041 ffff.ffff.ffff  [0]
  [0148:00fe:00fe] qos 0000.0000.0000 0000.0000.0000 0180.c200.0041 ffff.ffff.ffff  [0]
  [0148:00fe:00fe] qos 0000.0000.0000 0000.0000.0000 0180.c200.0041 ffff.ffff.ffff  [0]

CoPP Configuration Best Practices

These are best practice recommendations for CoPP configuration:

• Use strict CoPP mode by default.
  
  
• Dense CoPP profile is recommended when the chassis is fully loaded with F2 Series Modules or loaded with more F2 Series Modules than any other I/O modules.
  
  
• It is not recommended to disable CoPP. Tune the default CoPP, as needed.
  
  
• Monitor unintended drops, and add or modify the default CoPP policy in accordance to the expected traffic.
  
  
• Based on the number of FEs in the chassis, the CIR and BC settings for CoPP can be increased or decreased. This is also based on the role of the devices on the network, protocols running, and so on.
  
  
• Because traffic patterns constantly change in a Data Center, the customization of a CoPP is a constant process.
  
  
• CoPP and VDC: All ports of the same FE should belong to the same VDC, which is easy for an F2 Series LC, but not as easy for an M2 Series or M108 LC. This is because the CoPP resource sharing between VDCs if ports of the same FE belong to different VDCs (M1 Series or M2 Series LC). The ports of one FE, even in different VDCs, count against the same threshold for CoPP.
  
  
• The scale factor configuration is recommended when a chassis is loaded with both F2 Series and M Series Modules.

CoPP Monitoring Best Practices

These are best practice recommendations for CoPP monitoring:

• Configure a syslog message threshold for CoPP (Cisco NX-OS Release 5.1) in order to monitor drops enforced by CoPP.
  
  
• Syslog messages are generated if drops within a traffic class exceed the user-configured threshold.
  
  
• The logging threshold and level can be customized within each traffic class with use of the logging drop threshold <packet-count> level <level> command.
  
  
• Because the "statistics per-entry" option for CoPP MAC ACL or IP ACL is not supported, use the show system internal access-list input entries det command to monitor Access Control Entries (ACE) hits.
  
  
• The class copp-class-l2-default and class-default command should be monitored to ensure that there are no high increases, even for conformed counters.
  
  
• All classes should be monitored in terms of violated packets.
  
  
• Because copp-class-critical is highly vital but has a violate drop policy, it is a good practice to monitor the rate of conformed packets in order to receive an early indication when the class becomes near to the moment where it begins violation. If the violated counter increases for this class, it does not necessarily mean a red alert. Rather, it means this situation must be investigated in short term.
  
  
• Use the copp profile strict command after each Cisco NX-OS code upgrade, or at least after each major Cisco NX-OS code upgrade; if a CoPP modification was previously completed, it must be reapplied.

Conclusions

• CoPP is a hardware-based feature that protects the Supervisor from DoS attacks.
  
  
• M1, F2, and M2 Series LCs support CoPP. F1 Series LCs do not support CoPP.
  
  
• CoPP configuration is similar to MQC (Modular QoS CLI).
  
  
• CoPP configuration and monitoring is performed only in a default VDC.
  
  
• Default CoPP BPP can be used with strict, moderate, lenient, and dense options.
  
  
• Clone CoPP BPP to customized CoPP rules in order to match specific network requirements.
  
  
• CoPP counters (conformed and violated in bytes per class-map) are displayed with the show policy-map interface control-plane command.
  
  
• Traffic received by the CPU of the Supervisor module equals the total number of FEs times the allowed rate.
  
  
• Try to avoid shared ports of one FE across different VDCs.
  
  
• Follow CoPP best practices in order to successfully implement and monitor the features.

Unsupported Features

These features are not supported:

• Distributed aggregate policing.
  
  
• Microflow policing.
  
  
• Egress exception policing.
  
  
• CoPP support for BPDU that comes from a dot1q-tunnel port (QinQ): Cisco Discovery Protocol (CDP), DOT1x, Spanning Tree Protocol (STP), and VLAN Trunk Protocol (VTP).