Troubleshooting ISG with Session Monitoring and Distributed Conditional Debugging

Last Updated: December 19, 2012

Intelligent Services Gateway (ISG) is a Cisco IOS software feature set that provides a structured framework in which edge devices can deliver flexible and scalable services to subscribers.This document describes ISG session monitoring and distributed conditional debugging. Conditional debugging facilitates debug filtering for ISG and is available as distributed conditional debugging.

Finding Feature Information

Your software release may not support all the features documented in this module. For the latest caveats and feature information, see Bug Search Tool and the release notes for your platform and software release. To find information about the features documented in this module, and to see a list of the releases in which each feature is supported, see the feature information table at the end of this module.

Use Cisco Feature Navigator to find information about platform support and Cisco software image support. To access Cisco Feature Navigator, go to www.cisco.com/go/cfn. An account on Cisco.com is not required.

Prerequisites for ISG Session Monitoring and Distributed Conditional Debugging

Before using the information in this module, it is recommended that you be familiar with the use of Cisco IOS debug commands and conditional debugging. See the "Additional References" section to find information about these topics.

Restrictions for Distributed Conditional Debugging

Conditions that are set for an active session take effect only when the session is terminated and reestablished.


Caution

Because debugging output is assigned high priority in the CPU process, it can render the system unusable. For this reason, use the Cisco IOS debugcommands only to troubleshoot specific problems or during troubleshooting sessions with Cisco technical support staff. Moreover, it is best to use debug commands during periods of lower network traffic and fewer users, or on a debug chassis with a single active session. Debugging during these periods decreases the likelihood that increased debug command processing overhead will affect system use.

Information About ISG Session Monitoring and Distributed Conditional Debugging

ISG Session and Flow Monitoring

ISG introduces a mechanism that allows an administrator to monitor ISG sessions and flows continuously. The show interface monitor command, which displays interface statistics, and the show process cpu monitor command, which displays information about CPU usage, both update the information in their displays at specified intervals. These commands also provide the ability to freeze or clear the information in the display.

ISG Distributed Conditional Debugging

Benefits of Enhanced Conditional Debugging for the ISG Platforms

Because thousands of user sessions run on the ISG platforms, it is not practical to troubleshoot a problem with a session by enabling the various component debug commands that are available and trace through the messages for a single session or user. Instead, it is more practical to filter debugging messages for a single session or call across the various Cisco IOS XE components that a session traverses. For this reason, the conditional debugging previously offered in the Cisco IOS XE software has been enhanced to facilitate debug filtering for ISG and is available as distributed conditional debugging.

Cisco IOS Software Components Supported by Distributed Conditional Debugging

The following components are supported for ISG distributed conditional debugging:

  • Authentication, authorization, and accounting (AAA) and RADIUS
  • ATM components
  • Feature Manager
  • Policy Manager
  • PPP
  • PPP over Ethernet (PPPoE)
  • Session Manager
  • Virtual Private Dialup Network (VPDN)

See Table 1 and Table 2 for specific commands that are supported for distributed conditional debugging.

How to Enable ISG Session Monitoring and Distributed Conditional Debugging

Monitoring ISG Sessions and Flows

Perform this task to monitor interface and CPU statistics. The show commands are not required and may be entered in any order.

SUMMARY STEPS

1.    enable

2.    show interface type number monitor [interval seconds]

3.    show processes cpu monitor [interval seconds]


DETAILED STEPS
  Command or Action Purpose
Step 1
enable


Example:

Router> enable

 

Enables privileged EXEC mode.

  • Enter your password if prompted.
 
Step 2
show interface type number monitor [interval seconds]


Example:

Router# show interface ethernet 3/0 monitor interval 10

 

Displays interface statistics that are updated at specified intervals.

 
Step 3
show processes cpu monitor [interval seconds]


Example:

Router# show processes cpu monitor

 

Displays detailed CPU utilization statistics that are updated at specified intervals.

 

Configuring Distributed Conditional Debugging

Two main tasks are required for configuring distributed conditional debugging: enabling conditional debugging, and issuing one or more supported debug commands. These required tasks are described in the following sections:

ISG Debug Condition Commands

The table below lists the debug condition commands that you can issue at the EXEC prompt to enable distributed conditional debugging. You can set more than one condition.

Table 1 Supported Conditional Debug Commands

Command

Purpose

debug condition domain domain-name

Filters messages on the specified domain name.

debug condition interface atm ATM-interface vc {vci / vpi|vci}

Filters messages on the specified virtual circuit.

debug condition interface {atm ATM-interface vc {vci / vpi | vci} | Ethernet | Fast Ethernet | Gigabit Ethernet} vlan-id ID

Filters messages on the specified VLAN identifier.

debug condition mac-address hexadecimal-MAC-address

Filters messages on the specified MAC address.

debug condition portbundle ip IP-address bundle bundle-number

Filters messages on the specified Port-Bundle Host Key (PBHK).

debug condition session-id session-ID

Filters messages on the specified session identifier.

Note    The session identifier can be obtained by entering the show subscriber session command.

debug condition username email-address

Filters messages on the specified Internet username.

Debug Commands That Are Supported by ISG Conditional Debug

The table below lists the Cisco IOS debugging commands that are supported for distributed conditional debugging. The commands are listed by component. One or more of these commands can be issued after enabling one of the debug condition commands listed in the table above.

Table 2 Debug Commands Supported by ISG Distributed Conditional Debugging

AAA Debug Commands

debug aaa accounting

debug aaa authentication

debug aaa authorization

debug aaa id

ATM Debug Commands

debug atm arp

debug atm error

debug atm event

debug atm oam

debug atm packet

debug atm state

PPP Debug Commands

debug ppp authentication

debug ppp bap error

debug ppp bap events

debug ppp bap negotiation

debug ppp cbcp

debug ppp error

debug ppp mppe detailed

debug ppp mppe events

debug ppp mppe pack

debug ppp multi data

debug ppp multi events

debug ppp multi frag

debug ppp negotiation

debug ppp pack

debug ppp subscriber

PPPoE Debug Commands

debug pppoe data

debug pppoe error

debug pppoe event

debug pppoe packet

Session Manager Debug Commands

debug subscriber aaa authorization event

debug subscriber aaa authorization fsm

debug subscriber error

debug subscriber event

Feature Manager Debug Commands

debug subscriber feature access-list error

debug subscriber feature access-list event

debug subscriber feature compression detail

debug subscriber feature compression error

debug subscriber feature compression event

debug subscriber feature detail

debug subscriber feature error

debug subscriber feature event

debug subscriber feature interface-config error

debug subscriber feature interface-config event

debug subscriber feature modem-on-hold detail

debug subscriber feature modem-on-hold error

debug subscriber feature modem-on-hold event

debug subscriber feature portbundle error

debug subscriber feature portbundle event

debug subscriber feature portbundle packet

debug subscriber feature qos-policy error

debug subscriber feature qos-policy event

debug subscriber feature static-routes error

debug subscriber feature static-routes event

debug subscriber feature traffic-classification detail

debug subscriber feature traffic-classification error

debug subscriber feature traffic-classification event

Policy Manager Debug Commands

debug subscriber fsm

debug subscriber policy condition

debug subscriber policy detail

debug subscriber policy error

debug subscriber policy event

debug subscriber policy fsm

debug subscriber policy rule

debug subscriber session error

debug subscriber session event

VPDN Debug Commands

debug vpdn call event

debug vpdn call fsm

debug vpdn error

debug vpdn event

debug vpdn event disconnect

Restrictions

The debug condition session-id command filters a session only after the session has been established. The session identifier is a unique dynamic number generated internally by the Cisco IOS software and assigned to each session when the session is established.

In VPDN, the debug commands and messages associated with tunnels cannot be filtered because they are not associated with a session, but are displayed during the tunnel-establishment phase. The debugging messages will be displayed even if filtering is enabled by one of the conditions.

If multiple conditions are set, the debugging messages corresponding to all the sessions that meet any of the conditions will be displayed. Some conditions, such as domain name, will trigger debugging messages for all the sessions that belong to the particular domain.

Enabling Distributed Conditional Debugging

Perform this task to enable distributed conditional debugging for ISG.

SUMMARY STEPS

1.    enable

2.    debug condition command

3.    debug command


DETAILED STEPS
  Command or Action Purpose
Step 1
enable


Example:

Router> enable

 

Enables privileged EXEC mode.

  • Enter your password if prompted.
 
Step 2
debug condition command


Example:

Router# debug condition username user@cisco.com

 

Enter one or more of the debug condition commands listed in Table 1 to enable distributed conditional debugging.

 
Step 3
debug command


Example:

Router# debug subscriber aaa authorization fsm

 

Enter one or more of the supported debug commands from Table 2.

 

Displaying Debugging Conditions

To display the debugging conditions that have been set, perform the following task:

SUMMARY STEPS

1.    enable

2.    show debug condition


DETAILED STEPS
  Command or Action Purpose
Step 1
enable


Example:

Router> enable

 

Enables privileged EXEC mode.

  • Enter your password if prompted.
 
Step 2
show debug condition


Example:

Router# show debug condition

 

Displays conditions that have been set for debugging.

 

Troubleshooting Tips

The Cisco IOS software displays messages as you set the conditions for filtering the debugging.

When a condition is set, it is assigned a number, as follows:

Condition 1 set

If a condition has already been set, the following message is displayed:

% Condition already set

The following messages and prompt are displayed when you attempt to disable the last condition using the no form of a debug condition command:

This condition is the last interface condition set.
Removing all conditions may cause a flood of debugging messages
to result, unless specific debugging flags are first removed.
 
Proceed with removal? [yes/no]: yes
Condition 1 has been removed

Tip

Use the no form of the commands to disable all debug commands before disabling all of the debugging conditions that have been set.

Configuration Examples for ISG Distributed Conditional Debugging

Monitoring Interface Statistics Example

The following example shows sample output for the show interface monitor command. The display will be updated every 10 seconds.

Router> show interface ethernet 0/0 monitor interval 10
 
Router Name:  Scale3-Router8       Update Secs: 10
Interface Name:   Ethernet 0/0          Interface Status: UP, line is up 
 
Line Statistics:          Total:        Rate(/s)    Delta
Input Bytes:              123456        123         7890
Input Packets:              3456         56          560
Broadcast:                  1333          6           60
OutputBytes:               75717        123         1230
Output Packets:              733         44          440
 
Error Statistics:         Total:        Delta:
Input Errors:              0               0
CRC  Errors:               0               0
Frame Errors:              0               0
Ignored:                   0               0
Output Errors:             0               0
Collisions:                0               0
 
No. Interface Resets:  2
End = e       Clear = c      Freeze = f 
Enter Command:

Monitoring CPU Statistics Example

The following example shows sample output for the show processes cpu monitor command:

Router> show processes cpu monitor
 
CPU utilization for five seconds: 0%/0%; one minute: 0%; five minutes: 0%
 PID Runtime(ms)   Invoked      uSecs    5Sec   1Min     5Min    TTY Process 
   3      772          712       1084    0.08%  0.04%    0.02%    0    Exec             
  67      276         4151         66    0.08%  0.03%    0.01%    0 L2TP mgmt daemon 
 116      604         2263        266    0.16%  0.05%    0.01%    0 IDMGR CORE       
 
End = e    Freeze = f

Enter Command:

Enabling ISG Distributed Conditional Debugging Example

The following example shows how to filter PPP, PPPoE, and Session Manager debugs for a PPPoE session with username "user@cisco.com". Only debugging messages for the defined user are displayed on the console. Any other debugging messages associated with other users will not be displayed.

Router# debug condition username user@cisco.com
Condition 1 set
 
Router# debug ppp negotiation
Router# debug pppoe event
Router# debug subscriber session event

Displaying Debugging Conditions Example

The following example shows how to display debugging conditions that have been set.

Router# show debug condition

Condition 1: domain cisco.com (0 flags triggered)
Condition 2: username user@cisco.com (0 flags triggered)
Condition 3: ip 172.19.200.10 (0 flags triggered)

Filtering Debug Output Example

In the following example, the output of the debug subscriber packet detail command is filtered on the basis of the username "cpe6_1@isp.com": 

Router# debug condition username cpe6_1@isp.com
Condition 1 set
Router# show debug
 
Condition 1: username cpe6_1@isp.com (0 flags triggered)
 
Router# debug subscriber packet detail
SSS packet detail debugging is on

Router# show debug
SSS:
  SSS packet detail debugging is on
 
Condition 1: username cpe6_1@isp.com (0 flags triggered)

Additional References

Related Documents

Related Topic

Document Title

ISG commands

Cisco IOS Intelligent Services Gateway Command Reference

Debug commands

Cisco IOS Debug Command Reference

Conditional debugging

"Conditionally Triggered Debugging" chapter in the Cisco IOS Debug Command Reference

Technical Assistance

Description

Link

The Cisco Support website provides extensive online resources, including documentation and tools for troubleshooting and resolving technical issues with Cisco products and technologies.

To receive security and technical information about your products, you can subscribe to various services, such as the Product Alert Tool (accessed from Field Notices), the Cisco Technical Services Newsletter, and Really Simple Syndication (RSS) Feeds.

Access to most tools on the Cisco Support website requires a Cisco.com user ID and password.

http://www.cisco.com/cisco/web/support/index.html

Feature Information for Distributed Conditional Debugging

The following table provides release information about the feature or features described in this module. This table lists only the software release that introduced support for a given feature in a given software release train. Unless noted otherwise, subsequent releases of that software release train also support that feature.

Use Cisco Feature Navigator to find information about platform support and Cisco software image support. To access Cisco Feature Navigator, go to www.cisco.com/go/cfn. An account on Cisco.com is not required.

Table 3 Feature Information for ISG Session Monitoring and Distributed Conditional Debugging

Feature Name

Releases

Feature Configuration Information

ISG: Instrumentation: Session and Flow Monitoring

12.2(28)SB 12.2(33)SRC 15.0(1)S

ISG provides a mechanism for continuously monitoring interface and CPU statistics. This feature introduces the show interface monitor and show processes cpu monitorcommands, which display statistics that are updated at specified intervals.

In Cisco IOS Release 12.2(28)SB, this feature was introduced.

This feature was integrated into Cisco IOS Release 12.2(33)SRC.

ISG: Instrumentation: Advanced Conditional Debugging

12.2(28)SB 12.2(33)SRC

ISG provides the ability to define various conditions for filtering debug output. Conditional debugging generates very specific and relevant information that can be used for session, flow, subscriber, and service diagnostics.

In Cisco IOS Release 12.2(28)SB, this feature was introduced.

This feature was integrated into Cisco IOS Release 12.2(33)SRC.

Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1110R)

Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional and coincidental.

© 2012 Cisco Systems, Inc. All rights reserved.