Using Trace Logs for Troubleshooting
Use traces to troubleshoot system issues with IM and Presence services and features. You can configure automated system tracing for a variety of services, features, and system components. The results are stored in system logs that you can browse and view using the Cisco Unified Real-Time Monitoring Tool. Alternatively, you can use the Command Line Interface to pull a subset of the system log files and upload them to your own PC or laptop for further analysis.
To use traces, you must first configure the system for tracing. For details on how to configure system tracing, refer to the "Traces" chapter of the Cisco Unified Serviceability Administration Guide.
Once tracing is configured, you can use one of two methods to view the contents of trace files:
-
Real-Time Monitoring Tool—With the Real-Time Monitoring Tool, you can browse and view the individual log files that are created as a result of system tracing. For details on how to use the Real-Time Monitoring Tool, see the Cisco Unified Real-Time Monitoring Tool Administration Guide.
-
Command Line Interface (CLI)—If system tracing is configured, use the CLI to build customized traces from your system logs. With the CLI, you can specify the specific days that you want to include in a customized trace file. The CLI pulls the associated trace files from your system and saves them in a compressed zip file that you can copy to a PC or laptop for further analysis, thereby ensuring that the logs don't get overwritten by the system.
The susequent tables and tasks in this section describe how to use CLI commands to build trace log files for the IM and Presence Service.
Common IM and Presence Issues via Trace
The following table lists common issues with the IM and Presence Service and which traces you can run to troubleshoot the issue.
|
Issues with... |
View Traces for These Services |
Additional Instructions |
||
|---|---|---|---|---|
|
Login and Authentication Traces |
Client Profile Agent Cisco XCP Connection Manager Cisco XCP Router Cisco XCP Authentication Service Cisco Tomcat Security Logs |
See Common Traces via CLI for CLI commands to build logs and output locations. |
||
|
Availability Status |
Cisco XCP Connection Manager Cisco XCP Router Cisco Presence Engine |
See Common Traces via CLI for CLI commands to build logs and output locations. |
||
|
Sending and Receiving IMs |
Cisco XCP Connection Manager Cisco XCP Router |
See Common Traces via CLI for CLI commands to build logs and output locations. |
||
|
Contact Lists |
Cisco XCP Connection Manager Cisco XCP Router Cisco Presence Engine |
See Common Traces via CLI for CLI commands to build logs and output locations. |
||
|
Chat Rooms |
Cisco XCP Connection Manager Cisco XCP Router Cisco XCP Text Conferencing Manager |
See Common Traces via CLI for CLI commands to build logs and output locations. |
||
|
Partitioned Intradomain Federation |
Cisco XCP Router Cisco XCP SIP Federation Connection Manager Cisco SIP Proxy Cisco Presence Engine |
See Common Traces via CLI for CLI commands to build logs and output locations.
|
||
|
Availability and IMs for XMPP Based Interdomain Federation Contact |
Cisco XCP Connection Manager Cisco XCP Router Cisco Presence Engine Cisco XCP XMPP Federation Connection Manager |
See Common Traces via CLI for CLI commands to build logs and output locations. Perform trace on each IM and Presence node on which XMPP Federation is enabled |
||
|
Availability and IMs for SIP Interdomain Federation Contact |
Cisco XCP Connection Manager Cisco XCP Router Cisco Presence Engine Cisco SIP Proxy Cisco XCP SIP Federation Connection Manager |
See Common Traces via CLI for CLI commands to build logs and output locations. |
||
|
Calendaring Traces |
Cisco Presence Engine |
See Common Traces via CLI for CLI commands to build logs and output locations. |
||
|
Intercluster Synchronization Traces and Intercluster Troubleshooter |
Cisco Intercluster Sync Agent Cisco AXL Web Service Cisco Tomcat Security Log Cisco Syslog Agent |
Run the system troubleshooter at to check for interclustering errors. |
||
|
SIP Federation Traces |
Cisco SIP Proxy Cisco XCP Router Cisco XCP SIP Federation Connection Manager |
See Common Traces via CLI for CLI commands to build logs and file output locations. |
||
|
XMPP Federation Traces |
Cisco XCP Router Cisco XCP XMPP Federation Connection Manager |
See Common Traces via CLI for CLI commands to build logs and file output locations. |
||
|
High CPU and Low VM Alert Troubleshooting |
Cisco XCP Router Cisco XCP SIP Federation Connection Manager Cisco SIP Proxy Cisco Presence Engine Cisco Tomcat Security Log Cisco Syslog Agent |
For additional troubleshooting, run the following CLI commands:
Run the following CLI to get RIS (Real-Time Information Service) data:
You can also setup Cisco Unified IM and Presence Serviceability alarms to provide information about runtime status and the state of the system to local system logs. |
Common Traces via CLI
Use the Command Line Interface to build trace log files to troubleshoot your system. With the CLI, you can choose the component for which you want to run a trace and specify the <duration>, which is the number of days looking backwards from today that you want to include in your log file.
The following two tables contain the CLI commands that you can use to build trace log files and the log output locations for:
-
IM and Presence Services
-
IM and Presence Features
 Note |
The CLI pulls a subset of the same individual traces files that you can view with the Cisco Unified Real-Time Monitoring Tool (RTMT), but groups and stores them in a single compressed zip file. For RTMT traces, see Common Traces via RTMT. |
|
Service |
CLI to Build Log |
CLI Output File |
|---|---|---|
|
Cisco Audit Logs |
file build log cisco_audit_logs <duration> |
/epas/trace/log_cisco_audit_logs_*.tar.gz |
|
Cisco Client Profile Agent |
file build log cisco_client_profile_agent <duration> |
/epas/trace/log_cisco_client_profile_agent_*.tar.gz |
|
Cisco Cluster Manager |
file build log cisco_config_agent <duration> |
/epas/trace/log_cisco_cluster_manager_*.tar.gz |
|
Cisco Config Agent |
file build log cisco_config_agent<duration> |
/epas/trace/log_cisco_config_agent_*.tar.gz |
|
Cisco Database Layer Monitor |
file build log cisco_database_layer_monitor <duration> |
/epas/trace/log_cisco_database_layer_monitor_*.tar.gz |
|
Cisco Intercluster Sync Agent |
file build log cisco_inter_cluster_sync_agent <duration> |
/epas/trace/log_cisco_inter_cluster_sync_agent_*.tar.gz |
|
Cisco OAM Agent |
file build log cisco_oam_agent <duration> |
/epas/trace/log_cisco_oam_agent_*.gz |
|
Cisco Presence Engine |
file build log cisco_presence_engine <duration> |
/epas/trace/log_cisco_presence_engine_*.tar.gz |
|
Cisco RIS (Real-time Information Service) Data Collector |
file build log cisco_ris_data_collector <duration> |
/epas/trace/log_cisco_ris_data_collector_*.tar.gz |
|
Cisco Service Management |
file build log cisco_service_management <duration> |
/epas/trace/log_cisco_service_management_*.tar.gz |
|
Cisco SIP Proxy |
file build log cisco_sip_proxy <duration> |
/epas/trace/log_cisco_sip_proxy_*.tar.gz |
|
Cisco Sync Agent |
file build log cisco_sync_agent <duration> |
/epas/trace/log_cisco_sync_agent_*.tar.gz |
|
Cisco XCP Config Manager |
file build log cisco_xcp_config_mgr <duration> |
/epas/trace/log_cisco_xcp_config_mgr_*.tar.gz |
|
Cisco XCP Router |
file build log cisco_xcp_router <duration> |
/epas/trace/log_cisco_xcp_router_*.tar.gz |
|
Feature Name |
CLI to Build Log |
CLI Output File |
|---|---|---|
|
Administration GUI |
file build log admin_ui <duration> |
/epas/trace/log_admin_ui_*.tar.gz |
|
Bulk Administration |
file build log bat <duration> |
/epas/trace/log_bat_*.tar.gz |
|
Bidirectional Streams over Synchronous HTTP |
file build log bosh <duration> |
/epas/trace/log_bosh_*.tar.gz |
|
Certificates |
file build log certificates <duration> |
/epas/trace/log_certificates_*.tar.gz |
|
Config Agent Core |
file build log cfg_agent_core <duration> |
/epas/trace/log_cfg_agent_core_*.tar.gz |
|
Customer Voice Portal |
file build log cvp <duration> |
/epas/trace/log_cvp_*.tar.gz |
|
Directory Groups |
file build log directory_groups <duration> |
/epas/trace/log_directory_groups_*.tar.gz |
|
Disaster Recovery |
file build log disaster_recovery <duration> |
/epas/trace/log_disaster_recovery_*.tar.gz |
|
Flexible IM address |
file build log flexable_im_address <duration> |
/epas/trace/log_flexible_im_address_*.tar.gz |
|
General core |
file build log general_core <duration> |
/epas/trace/log_general_core_*.tar.gz |
|
High Availability |
file build log ha <duration> |
/epas/trace/log_ha_*.tar.gz |
|
High CPU |
file build log high_cpu <duration> |
/epas/trace/log_high_cpu_*.tar.gz |
|
High Memory |
file build log high_memory <duration> |
/epas/trace/log_high_memory_*.tar.gz |
|
Instant Messaging Database Core |
file build log imdb <duration> |
/epas/trace/log_imdb_core_*.tar.gz |
|
Intercluster Peering |
file build log inter_cluster <duration> |
/epas/trace/log_inter_cluster_*.tar.gz |
|
Managed File Transfer |
file build log managed_file_transfer <duration> |
/epas/trace/log_managed_file_transfer_*.tar.gz |
|
Microsoft Exchange |
file build log msft_exchange <duration> |
/epas/trace/log_msft_exchange_*.tar.gz |
|
Message Archiver |
file build log msg_archiver <duration> |
/epas/trace/log_msg_archiver_*.tar.gz |
|
Presence Engine Core |
file build log pe_core <duration> |
/epas/trace/log_pe_core_*.tar.gz |
|
Presence and IM Message Exchange |
file build log presence_im_exchange <duration> |
/epas/trace/log_presence_im_exchange_*.tar.gz |
|
SIP Login Issues |
file build log pws <duration> |
/epas/trace/log_pws_*.tar.gz |
|
Remote Call Control |
file build log remote_call_control <duration> |
/epas/trace/log_remote_call_control_*.tar.gz |
|
Security Vulnerabilities |
file build log sec_vulnerability <duration> |
/epas/trace/log_sec_vulnerability_*.tar.gz |
|
Serviceability GUI |
file build log serviceability_ui <duration> |
/epas/trace/log_serviceability_ui_*.tar.gz |
|
SIP Interdomain Federation |
file build log sip_inter_federation <duration> |
/epas/trace/log_sip_inter_federation_*.tar.gz |
|
SIP Partitioned Intradomain Federation |
file build log sip_partitioned_federation <duration> |
/epas/trace/log_sip_partitioned_federation_*.tar.gz |
|
SIP Proxy Core |
file build log sipd_core <duration> |
/epas/trace/log_sipd_core_*.tar.gz |
|
Persistent Chat High Availability |
file build log tc_ha <duration> |
/epas/trace/log_tc_ha_*.tar.gz |
|
Persistent Chat |
file build log text_conference <duration> |
/epas/trace/log_text_conference_*.tar.gz |
|
Upgrade Issues |
file build log upgrade_issues <duration> |
/epas/trace/log_upgrade_issues_*.tar.gz |
|
User Connectivity |
file build log user_connectivity <duration> |
/epas/trace/log_user_connectivity_*.tar.gz |
|
Rosters |
file build log user_rosters <duration> |
/epas/trace/log_user_rosters_*.tar.gz |
|
XCP Router Core |
file build log xcp_core <duration> |
/epas/trace/log_xcp_core_*.tar.gz |
|
XMPP Interdomain Federation |
file build log xmpp_inter_federation <duration> |
/epas/trace/log_xmpp_inter_federation_*.tar.gz |
|
Deployment Info |
file build log deployment_info <duration> |
/epas/trace/log_deployment_info_*.tar.gz |
Run Traces via CLI
Use this procedure to create a customized trace file via the Command Line Interface (CLI). With the CLI, you can specify, via the duration parameter, the number of days looking backwards that you want to include in your trace. The CLI pulls a subset of the system logs.
Note |
Make sure to use SFTP servers only to transfer files. |
Before you begin
You must have trace configured for your system. For details on setting up trace, see the "Trace" chapter of the Cisco Unified Serviceability Administration Guide.
Review Common Traces via CLI for a list of traces that you can run.
Procedure
| Step 1 |
Log in to the Command Line Interface. |
| Step 2 |
To build the log, run the For example, |
| Step 3 |
To get the log, run the For example, file get activelog epas/trace/log_cisco_cluster_manager__2016-09-30-09h41m37s.tar.gz. |
| Step 4 |
To maintain a stable system, delete the log after you retrieve it. Run the For example, |
Common Traces via RTMT
The following table lists common traces that you can perform on your IM and Presence Service node and the resulting log files. You can view the trace log files using the Real-Time Monitoring Tool (RTMT).
Note |
The CLI can be used to pull a subset of the same individual traces files that you can view with RTMT, but groups and stores them in a single compressed zip file. For CLI traces, see Common Traces via CLI. |
Note |
The following table shows the output locations for 11.5(1). Please note that automatic log file compression with gzip was introduced for many of these services as of release 11.5(1)SU2. For 11.5(1)SU2 and later details, see Release Notes for Cisco Unified Communications Manager and the IM and Presence Service, Release 11.5(1)SU2. |
|
Service |
Trace Log Filename |
|---|---|
|
Cisco AXL Web Services |
/tomcat/logs/axl/log4j/axl*.log |
|
Cisco Intercluster Sync Agent |
/epas/trace/cupicsa/log4j/icSyncAgent*.log |
|
Cisco Presence Engine |
/epas/trace/epe/sdi/epe*.txt |
|
Cisco SIP Proxy |
/epas/trace/esp/sdi/esp*.txt |
|
Cisco Syslog Agent |
/cm/trace/syslogmib/sdi/syslogmib*.txt |
|
Cisco Tomcat Security Log |
/tomcat/logs/security/log4/security*.log |
|
Cisco XCP Authentication Service |
/epas/trace/xcp/log/auth-svc-1*.log |
|
Cisco XCP Config Manager |
/epas/trace/xcpconfigmgr/log4j/xcpconfigmgr*.log |
|
Cisco XCP Connection Manager |
/epas/trace/xcp/log/client-cm-1*.log |
|
Cisco XCP Router |
/epas/trace/xcp/log/rtr-jsm-1*.log |
|
Cisco XCP SIP Federation Connection Manager |
/epas/trace/xcp/log/sip-cm-3*.log |
|
Cisco XCP Text Conferencing Manager |
/epas/trace/xcp/log/txt-conf-1*.log |
|
Cisco XCP XMPP Federation Connection Manager |
/epas/trace/xcp/log/xmpp-cm-4*.log |
|
Cluster Manager |
/platform/log/clustermgr*.log |
|
Cisco Client Profile Agent (CPA) |
/tomcat/logs/epassoap/log4j/EPASSoap*.log |
|
dbmon |
/cm/trace/dbl/sdi/dbmon*.txt |
Feedback