Troubleshooting SGM and the Network

Table Of Contents

Troubleshooting SGM and the Network

Verifying Discovery

Clearing a Locked-Up SGM Display

Investigating Data Problems

Viewing the SGM Troubleshooting Log

Viewing SGM Data on the Web

Diagnosing a Typical Network Problem

Troubleshooting SGM and the Network

---

This chapter provides the following information for troubleshooting basic SGM and network problems:

•Verifying Discovery

•Clearing a Locked-Up SGM Display

•Investigating Data Problems

•Viewing the SGM Troubleshooting Log

•Viewing SGM Data on the Web

•Diagnosing a Typical Network Problem

Verifying Discovery

After you discover the network (see the "Discovering the Network" section), examine the Discovered Nodes table to verify that SGM discovered all of the ITP nodes in the network. If you suspect that SGM did not discover all of the ITP nodes, verify the following conditions:

•Verify that the SGM server can ping the nodes.

•Verify that the nodes are running images that are compatible with the SGM server.

•Verify that the SNMP is enabled on the nodes.

•Verify that SGM is configured with the correct SNMP community name. See the "Configuring SNMP" section for details.

•Verify that the missing nodes are connected to the seed nodes by SCTP connections, not just serial connections. If they are not connected by SCTP connections, you must add the missing nodes to the seed file as seed nodes. See the "Configuring Seed Files" section for more information.

•Verify that you selected Entire Network when you ran Discovery. If you suspect that you did not, run Discovery again with Entire Network selected.

Clearing a Locked-Up SGM Display

In SGM, events might cause message popups to remain in the background of your display, preventing you from interacting with other windows. If you suspect that your display has locked up, look for an SGM message popup in the background before taking any other action. To do so, try to bring every SGM window to the foreground of the display.

Investigating Data Problems

If you suspect that there are problems with the data that SGM is displaying, perform the following tasks:

•Enter equivalent show commands on the router. Is the data the same as that displayed by SGM?

•Send SNMP queries to the nodes. Do all queries complete?

The results of these tasks can help you distinguish between an ITP problem and an SGM problem.

Viewing the SGM Troubleshooting Log

SGM stores troubleshooting information in the /opt/CSCOsgm/tmp/cisco_sgm_tshoot.log file on the SGM server. This log, which is updated each time the SGM Server Troubleshooting page is accessed or the sgm tac command is run, contains information that might be requested by Cisco customer support personnel.

If you want to view the SGM troubleshooting log, Cisco strongly recommends that you do so in a Web browser. To view the log in a Web browser, select System Troubleshooting from the SGM Server Home Page.

You can also view the log from the command line, but this method displays the entire log, which can contain thousands of lines of output, line-by-line on your workstation screen. Therefore, Cisco strongly recommends that you view the log from the Web, as indicated above, or redirect the output from this command to a separate file, using the redirection symbol (>).

To view the log from the command line:

---

Step 1 Log in as the root user, as described in the "Becoming the Root User (Solaris Only)" section, or as a super user, as described in the "Specifying a Super User (Solaris Only)" section.

Step 2 Enter the following commands:

# cd /opt/CSCOsgm/bin

# ./sgm tac

This command might take a minute or more to complete. When it completes, SGM displays the following message and prompt:

Output is in /opt/CSCOsgm/tmp/cisco_sgm_tshoot.log

Would you like to view it? [y]

Step 3 Press Enter. SGM displays the contents of the /opt/CSCOsgm/tmp/cisco_sgm_tshoot.log file.

---

Viewing SGM Data on the Web

SGM provides an enormous amount of Web-based troubleshooting information. From the SGM Server Home Page, you can access many Web pages containing SGM data, including server status, network status, installation logs, message logs, product documentation, and other important troubleshooting information about SGM. For full details, see the "Accessing SGM Data from a Web Browser" section.

Diagnosing a Typical Network Problem

When you use SGM to diagnose a problem in an ITP network, follow these basic steps:

1. Monitor the network using the Linkset, Node, and Topology windows. For example, an object in the topology map that changes color from green to yellow or red indicates a problem in the network.

2. Use SGM windows, especially the Details window, to begin investigating the problem.

3. As you identify the source of the problem, examine the messages logged by SGM for more detailed information about the sequence of events that led to the problem.

4. Telnet to the problematic router, if necessary.

The following real-life example provides detailed information about using SGM to diagnose a problem in an ITP network:

---

Step 1 A network operator (we'll call him Joe) is using SGM to monitor an ITP network. Joe has customized his view, limiting it to only those nodes for which he is responsible.

(For more information about customizing views, see the "Working with Views" section.)

Step 2 In the topology map, Joe notices a node that has changed color from green to yellow. Yellow indicates a status of Warning, which means that one or more links or linksets associated with that node is in Failed, Unavailable, Unknown, or Warning status and is not flagged as Ignored.

(For more information about node status, see the "Working with Nodes" section.)

Step 3 Joe single-clicks the node in the topology map.

SGM highlights the node in the topology map, and in the topology node table, in the left pane of the Topology window. With the node highlighted, Joe can easily see that the name of the node is sgm-7500j.

SGM also displays all associated linksets in the topology linkset table.

Joe double-clicks the node's name in the topology node table.

SGM redraws the topology map, centered on sgm-7500j, making it easier for Joe to see the relevant portion of the map.

(For more information about the Topology window and how to use it, see the "Viewing the Topology of the Network" section.)

Step 4 Joe notices that one of sgm-7500j's diamonds is red, indicating that the associated linkset is either Unavailable or Unknown. Joe single-clicks the red diamond.

SGM highlights the linkset in the topology map and in the topology linkset table. The table entry indicates that the linkset is Unavailable.

(For more information about linkset status, see the "Viewing Basic Information for Linksets" section.)

Step 5 Joe right-clicks the linkset in the topology map and selects View > Details in the right-click menu.

SGM opens the Details window, showing detailed information for the linkset.

In the Details window, detailed information for the selected linkset is displayed in the left column and for the adjacent linkset in the right column

Immediately, Joe sees that the left column is populated with SGM data, but the right column is not. The problem is in the adjacent node-to-primary node linkset.

(For more information about linkset details, see the "Viewing Detailed Information for a Linkset" section.)

Step 6 The turner beside Linkset in the left pane displays the list of links associated with the linkset, identified by their signaling link code IDs (SLCs). In this case, there is only one link listed, SLC 0, and it is red, meaning it is Failed and there is no traffic flowing on the link.

Joe selects SLC 0, and SGM displays detailed information for the link in the left column. Normally SGM also displays detailed information for links associated with the adjacent linkset in the right column, but in this case, that column is blank.

(For more information about linkset status, see the "Viewing Detailed Information for a Link" section.)

Step 7 Joe decides to investigate the adjacent node, so he selects AdjNode in the left pane of the Details window.

The resulting display shows that the adjacent node, sgm-2600a, is Unmanaged.

(For more information about node details, see the "Viewing Detailed Information for a Node" section.)

Step 8 Joe closes the Details window and returns to the Topology window. He tries to find sgm-2600a in the topology map, but the map is too complex. So Joe lets SGM find the node for him:

a. He selects Edit > Find in the SGM Main Menu. SGM displays the Find dialog.

b. He enters sgm-2600a in the Search string field, selects the Nodes checkbox, and makes sure the Name, checkbox is selected.

c. He clicks Search to launch the search. Almost immediately, SGM finds the node and displays the Choose dialog, listing all found objects.

d. Joe selects sgm-2600a, and SGM automatically highlights sgm-2600a in the topology node table and in the topology map, and redraws the map centered on sgm-2600a.

(For more information about finding objects in the topology map, see the "Viewing the Topology of the Network" section.)

Step 9 Joe wants to see recent events for sgm-2600a, so he clicks the node in the topology map and selects View > Events in the SGM Main Menu. SGM displays the Event Window, showing recent events for sgm-2600a.

(For more information about viewing events, see the "Working with Events" section.)

Step 10 Joe decides to see if SGM can manage the node. He right-clicks sgm-2600a in the topology map and selects Manage in the right-click menu.

SGM changes the status of the node from Unmanaged (red) to Warning (yellow), which means the node is active, but one or more associated links or linksets has a status of Failed, Unavailable, Unknown, or Warning and is not flagged as Ignored.

(For more informations, see the "Unmanaging and Managing a Node" section.)

Step 11 Joe wants to see status change messages for sgm-2600a, so he right-clicks the node again and selects Web Links > Status Change Messages in the right-click menu. SGM displays recent status change messages for the node in a Web browser.

Joe sees that many of the links and linksets associated with sgm-2600a have a status of Unknown.

(For more information about displaying messages on the Web, see the "Accessing SGM Data from a Web Browser" section.)

Step 12 At this point, Joe needs to determine why so many of the links and linksets are Unknown. He needs to verify that the SGM server can ping the node (see the "Telnetting to a Router" section), and that SGM is configured with the correct SNMP community name for the node (see the "Configuring SNMP" section).

(For a list of some other actions Joe can take, see the "Verifying Discovery" section.)

Step 13 Finally, Joe can use another product, such as CiscoView, to further investigate the problem.

(For more information about integrating SGM with CiscoView and other products, see the "Integrating SGM with Other Products" section.)

---