Troubleshooting Cisco MNM

Revised: December 15, 2009, OL-18339-03

This section provides troubleshooting information for Cisco Media Gateway Controller (MGC) Node Manager (MNM) internal messages and for other common Cisco MNM issues.

Troubleshooting Cisco MNM Internal Messages

The following messages may be generated by Cisco MNM that reflect errors in deployment, discovery, or configuration. See the "Solving Deployment and Discovery Errors" section for information on how to correct deployment and discovery errors.

Table C-1 Cisco MNM Internal Events 

Message
Explanation
Action

(Cisco PGW 2200 Softswitch host) Failed to collect active configuration

(1) FTP failed and the information is not getting to Cisco MNM.

(2) The device is not generating the information.

On the Cisco PGW 2200 Softswitch host, run the prov-exp command to view the configuration information being generated. If it is OK, there is an FTP problem. If it is not OK, there is a problem with the Cisco PGW 2200 Softswitch host.

<Host name>: Could not collect inventory: Login ID or password or security policy is invalid.

Login or password is invalid for the deployed device, or the security policy attribute is mis-set. As a result, Cisco MNM cannot fully discover the device.

See Troubleshooting SSH-Related Errors, for help in pinpointing the problem.

Correct the login, password, or security policy attribute information (Accounts dialog box) and rediscover the device.

<Host name>: Could not collect inventory: Password not specified.

Password is not specified for the deployed device. As a result, Cisco MNM cannot fully discover the device.

Correct the password information and rediscover the device.

<Host name>: Could not get Host Device table. Check IP address and read-community string.

Cisco MNM failed to retrieve the device table from the device. The problem may be
(1) Wrong SNMP community strings.

(2) SNMP Agent or the hostagt process not running on the device.

(3) The device is not reachable.

(1) Check the SNMP read-community string and correct if needed.

(2) Check that the snmpdm and hostagt processes are running.

(3) Attempt to access the device using ping. If it is unreachable, there may be a problem in the network connection.

<Host name>: Could not get Host Files System. Check IP address and read-community string.

Cisco MNM failed to retrieve the device table from the device. The problem may be

(1) Wrong SNMP community strings.

(2) SNMP agent or the fsagt process not running on the device.

(3) The device is not reachable.

(1) Check the SNMP read-community string and correct if needed.

(2) Check that the snmpdm and fsagt processes are running.

(3) Attempt to access the device using ping. If it is unreachable, there may be a problem in the network connection.

<Host name>: Could not get Host Storage table. Check IP address and read-community string.

Cisco MNM failed to retrieve the device table from the device. The problem may

(1) Wrong SNMP community strings.

(2) SNMP Agent or the hostagt process not running on the device.

(3) The device is not reachable.

(1) Check the SNMP read-community string and correct if needed.

(2) Check that the snmpdm and hostagt processes are running.

(3) Attempt to access the device using ping. If it is unreachable, there may be a problem in the network connection.

Billing and Measurements Server (BAMS) is not configured to receive Call Data Records from any Cisco PGW 2200 Softswitch Host.

Since the BAMS is not configured to collect data from any Cisco PGW 2200 Softswitch Host, Cisco MNM cannot deploy the device to the right Cisco PGW 2200 Softswitch node, and its alarm status will not be propagated in the MGC-Node-View.

Check your BAMS configuration and check the Cisco PGW 2200 Softswitch status.

Cannot get IF description from the interface table.

The appropriate processes may not be running on the device.

On the Cisco PGW 2200 Softswitch host, determine the process IDs by entering this command

ps-ef|grep agt

Check that critagt, mibagt, hostagt, and snmpagt are running. If not, kill critagt and restart the processes.

If the problem is not resolved, check if the interface type is one of Serial, Ethernet, Loopback, and SCO/SLO.

Could not get BAMS Poll table.

Cisco MNM failed to retrieve the BAMS configuration via SNMP. The problem may be

(1) Wrong SNMP community strings.

(2) SNMP Agent does not run on the device,.

(3) The device is not reachable.

As a result, Cisco MNM cannot deploy the device to the correct Cisco PGW 2200 Softswitch node. Therefore, its alarm status will not be propagated in the MGC-Node-View.

(1) Check the SNMP community strings and correct if needed.

(2) Check that the snmpdm and sagt processes are running.

(3) Attempt to access the device using ping. If it is unreachable, there may be a problem in the network connection.

For more information, refer to the log file <CEMF_ROOT>/logs/mgcController.log.

Could not get IP Address table from <device name>. Check IP address and read-community string.

Cisco MNM failed to retrieve the interface table from the device. The problem may be

(1) Wrong SNMP community strings

(2) Invalid IP Address.

(3) The device is not reachable.

(1) Check the SNMP read-community string and correct if needed.

(2) Check the IP address.

(3) Attempt to access the device using ping. If it is unreachable, there may be a problem in the network connection.

(4) Check if the interface type is one of Serial, Ethernet, Loopback, and SCO/SLO.

Could not get password for host <IP Address>.

Password is not specified for the deployed Cisco PGW 2200 Softswitch host. As a result,
Cisco MNM cannot fully discover the device.

Correct the password information and rediscover the device.

Failed to launch action <Action name>. Perhaps hostController is not running.

The most probable cause is that the Cisco MNM process hostController is down while Cisco MNM is trying to discover a Cisco PGW 2200 Softswitch.

Verify that the hostController process is running. For example, enter:

ps -ef | grep hostController

If the hostController is running, rediscover the device. If not, contact the TAC.

Miscellaneous error messages upon deployment, such as demons not running...

—

(1) Verify that the correct software release and patch are installed on the device. See Chapter 1 of the installation guide for details and links to up-to-date information.

(2) Make sure that the device is running. For example, for Cisco PGW 2200 Softswitch enter

/etc/init.d/CiscoMGC start

If the device is already running, a message displays. Otherwise it should start.

No IP addresses defined on this device. All traps from it will be ignored.

Cisco MNM failed to find any address on this device via SNMP. The problem may be

(1) Wrong SNMP community strings

(2) SNMP Agent does not run on the device.

(3) The device is not reachable.

(1) Check the SNMP community strings and correct if needed.

(2) Check that the snmpdm and mib2agt processes are running.

(3) Attempt to access the device using ping. If it is unreachable, there may be a problem in the network connection.

Subrack discovery failed. Check logs

Cisco MNM failed to discover components on the device. The problem may be

(1) Wrong SNMP community strings.

(2) SNMP Agent does not run on the device.

(3) The device is not reachable.

(1) Check the SNMP community strings and correct if needed.

(2) If Cisco PGW 2200 Softswitch or Cisco BAMS, check that the snmpdm and mib2agt processes are running.

(3) Attempt to access the device using ping. If it is unreachable, there may be a problem in the network connection.

For more information, refer to the log file <CEMF_ROOT>/logs/mgcController.log.

Verify that the correct software release and patch are installed on the device. See Chapter 1 of the installation guide for details and links to up-to-date information.

The IP Address <IP Address> is not reachable.

Cisco MNM failed to do SNMP ping with this address.

Check the network connection.

This device is not reachable.

Cisco MNM cannot reach the device using SNMP. If the device has multiple IP addresses, then all of them are unreachable.

(1) Check the SNMP community strings and correct if needed.

(2) Attempt to access the device using ping. If it is unreachable, there may be a problem in the network connection.


Table C-2 Seed File Deployment Errors 

Message
Explanation
Action

Unknown device specified

—

—

Unbalanced braces

—

—

Duplicate object names

—

—

Missing required attribute: Attribute

A required attribute is missing.

—

Component is not valid: COMPONENT

Device information supplied is syntactically incorrect.

Check and fix device syntax in seed file.

Expected attribute value. Found

A required value is missing

—


Solving Deployment and Discovery Errors

If you receive errors when deploying a seed file, check the information in Table C-2 and correct the problem in the file. See Chapter 5, "Deploying Your Network in Cisco MNM," "Deploying a Network Using a Seed File" section on page 5-9 for details.

If you receive a message about a problem in manual device deployment or during the discovery process, use these procedures to change the deployment information or rediscover network elements

Changing Password or Community Strings

Use the following procedure to change the password or community strings for a device:

Step 1 In the Map Viewer window, select the object and right-click.

Step 2 From the pull-down menu, choose Accounts.

The Accounts dialog box opens.

Step 3 On the Accounts tab, check and if needed, change the password.

Step 4 On the SNMP tab, check and if needed, change the SNMP community strings.

Step 5 Click Save on the toolbar, and close the dialog box.

Step 6 If you changed the community strings on any device or the password for the Cisco PGW 2200 Softswitch host, rediscover the device as described in the "Rediscovering a Device After a Problem" section.

Changing IP Address

If the wrong IP address was entered, the device must be redeployed. Use the following steps to redeploy a device:

Step 1 In the Map Viewer window, select the object and right-click.

Step 2 From the pull-down menu, choose Deployment > Delete Objects.

The Deployment Wizard dialog box opens with the message, "Ready to delete 1 object."

Step 3 Click Finish. A message displays that the object has been deleted.

Step 4 Click OK.

Step 5 Redeploy the device by following the instructions in Chapter 5, "Deploying Your Network in Cisco MNM," "Manual Deployment" section on page 5-10.

Step 6 After deployment, rediscover the device as described in the "Rediscovering a Device After a Problem" section.

Rediscovering a Device After a Problem

Follow these steps to rediscover a device after correcting a problem that interfered with discovery. This synchronizes the Cisco MNM network object model with the real-world network.

Step 1 In the Map Viewer window, select the object and right-click.

Step 2 Choose States.

The States dialog box opens.

Step 3 On the States tab, choose Rediscover.

You are asked if you want to rediscover the device.

Step 4 Click Yes. Cisco MNM rediscovers the device. During discovery, Current State is discovering. When the discovery is complete, Current State changes to active.

Step 5 Close the dialog box.

Troubleshooting SSH-Related Errors

If you suspect an SSH security-policy error, such as a mismatch between the security policy defined for a component at deployment and its actual security policy, you can do one of two things:

•Check SSH-related alarms in the Event Browser. You can see SSH-related alarms, such as a mismatched security policy or an incorrect password, for the BAMS, Cisco PGW, and HSI server, in the Event Browser. These are Warning alarms. For a description of Cisco PGW 2200, the BAMS, and HSI alarms caused by login failures related to SSH problems, see the "Cisco PGW 2200 Security Enhancements" chapter in the Cisco Media Gateway Controller Software Installation and Configuration Guide (Release 9.7) at

http://www.cisco.com/en/US/docs/voice_ip_comm/pgw/9/installation/software/SW1/97.html

•For an IOS device, check the ssh protocol version or configuration with this command:

show ip ssh 

SSH Enabled - version 1.5

Authentication timeout: 120 secs; Authentication retries: 3

To see ssh users that are logged on, use this command

show ssh 

Connection   Version Encryption      State    Username

1              1.5     DES             Session started         lab

Troubleshooting Other Issues

Table C-3 lists common problems you may encounter along with suggested corrective actions. Other troubleshooting information is provided in the Installation Guide, Chapter 2, Troubleshooting Common Installation Problems.

Table C-3 Troubleshooting Symptoms and Suggested Steps 

Problem
Action

Alarms are not being received from a device.

Check that SNMP trap forwarding has been configured. To configure SNMP trap forwarding, see Chapter 2, "Configuring Network Devices for Management."

If trap forwarding has been configured, check the snmpd.cnf file (Cisco PGW 2200 Softswitch host or the BAMS) against the instructions in Chapter 2 for possible typing errors.

Cisco MNM cannot automatically clear some alarms of the Cisco BAMS.

Manually clear these alarms in Cisco MNM. To manully clear alarms, see Chapter 6, "Managing Faults with Cisco MNM."

Cisco MNM intermittently fails to discover BAMS Node 1 association between the BAMS and PGW.

Check if the Cisco BAMS has been configured to collect CDRs for the relevant Cisco PGW 2200 Softswitch host.

Check if the Cisco BAMS actively polls CDRs from that Cisco PGW 2200 Softswitch host

Check if <IP Address> <hostname> for Cisco PGW 2200 Softswitch is added to the /etc/inet/hosts file.

Rediscover the Cisco BAMS if needed.

See Chapter 5, "Deploying Your Network in Cisco MNM," "Discovery of Cisco PGW 2200 Softswitch Host, Cisco HSI Server, and Cisco BAMS Components" section on page 5-15 for details.