How to Maintain and Troubleshoot the Network Capacity Expansion Service Module


Contents

Restoring NCE Software

Image Recovery Using Boothelper

System-Level Troubleshooting

Troubleshooting the NCE module from the Cisco IOS

Application-Level Troubleshooting

Troubleshooting the NCE Module

NCE service module uses the module's CLI, which you access through the host-router console. The module CLI is similar but not identical to the router CLI:

Similarities—Standard Cisco IOS navigation and command-completion conventions apply (for example, entering ? lists options, pressing the TAB key completes a command, and entering a pipe [|] directs show command output).

Differences—Standard command names and options do not necessarily apply. A notable example is the command for accessing global configuration mode: the Cisco IOS command is configure terminal; the module command is configure terminal.

The NCE service module uses a last-one-wins rule. For example, if George and Frank both try to set the IP address for the same entity at the same time, the system starts and completes one operation before it starts the next. The last IP address set is the final result.

The NCE service module uses a data-preservation rule in the event of software or hardware failure. During subsequent bootstrap, each user is prompted to examine data that was preserved from the previous boot and to resolve inconsistencies with data from other users. Repair operations reconstruct missing data wherever feasible and notify you when reconstruction is impossible.

Restoring NCE Software

If NCE software fails to bootup, use the following procedure to restore the module to factory settings.


Note If you are upgrading or downgrading an existing system, do not use this procedure. Instead, use the procedure in the "Software Upgrade or Downgrade" section on page 2-6.


Prerequisites

Have available the IP address of your TFTP and FTP file servers and download the required software from the Cisco Software Center website.

SUMMARY STEPS

From the Host-Router CLI

1. service-module Transport-Opt-Service-Engine slot/port reset

2. service-module Transport-Opt-Service-Engine slot/port session

From the Service-Module Interface

3. Enter ***

4. config

5. show config

6. boot helper

7. Follow boothelper instructions for installing software.

8. Control+Shift+6 x

From the Host-Router CLI

9. service-module Transport-Opt-Service-Engine slot/port session clear

DETAILED STEPS


Step 1 Download the NCE installation-package files (containing a kernel image, a boothelper image, and the NCE application software files, TPO Version 2.0.1 or later) as follows:

a. Go to the NCE page of the Cisco Software Center website and locate the following files:

NME-TPO

tpo-k9-full.aggr.2.0.1.prt1

tpo-k9-installer.aggr.2.0.1.prt1

tpo-k9.aggr.2.0.1.pkg

tpo-boothelper.aggr.2.0.1

AIM-TPO-1/2

tpo-k9-full.aim.2.0.1.prt1

tpo-k9-installer.aim.2.0.1.prt1

tpo-k9.aim.2.0.1.pkg

tpo-boothelper.aim.2.0.1


Note Download the boothelper image to the TFTP server and the NCE module application software (pkg, prt1) to the FTP server.


Step 2 Reset the NCE module:

Router> enable
Router# service-module Transport-Opt-Service-Engine slot/port reset

Step 3 Open a session.

Router# service-module Transport-Opt-Service-Engine slot/port session

Step 4 As soon as you have completed Step 3, enter *** to interrupt the automatic boot sequence and access the bootloader:

Router# ***

Step 5 Configure the bootloader to load and launch the boothelper:

SE-Module bootloader> config

Prompts for the following appear in the order listed. For each, enter a value or accept the previously stored input that appears inside square brackets by pressing Enter.

Service-module IP address

Subnet mask

TFTP file-server IP address

Gateway-router IP address (normally the IP address for the ISR)

Default boothelper image filename

Ethernet interface: internal

Default boot option: USB (Table 10-1)

Table 10-1 Default Boot Options 

Boot Option
Function
Loads This Software...
From This Location

helper

Boothelper image

TFTP file server

none

Starts the bootloader by default

usb

NCE image

USB flash memory


Default bootloader file to be used on subsequent boot: primary or secondary.


Note Make sure the TFTP and FTP servers are accessible from the bootloader prompt.

Make sure that there is a route configured in the Cisco IOS to access the service module.


Example:
boot-loader > config

  IP Address [1.3.202.203] > 
  Subnet Mask [255.255.0.0] > 
  TFTP Server [1.3.202.250] > 
  Gateway [1.3.202.100] > 
  Default Helper-file [tpo-boothelper.aim.2.0.1] > 
  Ethernet Interface [internal] [internal] > 
  Default Boot [none|helper|usb] [helper] > 
  Default Bootloader [primary|secondary] [primary] > 
boot-loader > 

Step 6 (Optional) Verify your bootloader configuration settings:

SE-Module bootloader> show config
IP Address: 1.3.202.26
Subnet Mask: 255.255.0.0
TFTP Server: 1.3.202.250
Gateway: 1.3.202.33
Default Helper-file: aesop_helper_cachaggr
Ethernet Interface: internal
Default Boot: usb
Default Bootloader: primary
Bootloader Version: 1.0.4
boot-loader >

Step 7 Start the boothelper:

SE-Module bootloader> boot helper

Step 8 Follow boothelper for installing software:

a. Install the NCE software (choose option 1 and follow the instructions).

b. Use the pkg file as the file name - the rest of the files are downloaded automatically.

c. If the module does not automatically reboot after the software is installed, reload it (choose option 2).

Step 9 Close the session by pressing Control+Shift+6 x.

Step 10 From the host-router CLI, clear the session:

Router# service-module Transport-Opt-Service-Engine slot/port session clear

Image Recovery Using Boothelper

Boothelper Install Log

In the following example, requests for user input are shown in bold.


Note Parts of the boothelper install log in this example have been removed to show only specific sections.


A-3845-1#
.
.
.
Removed
.
.
.
ServicesEngine boot-loader > boot helper
max_sdram_size = 0x40000000 = 1024 MEG 
bl_boot_helper_cmd tftpboot 0x4000000 /tpo-boothelper.aim.2.0.1
Using octeth1 device
TFTP from server 1.3.202.250; our IP address is 1.3.202.26
Filename '/aesop_helper_cachnm+aim'.
Load address: 0x4000000
Loading: #################################################################
#################################################################
#################################################################
.
.
.
Removed
.
.
.

done
Bytes transferred = 20926969 (13f51f9 hex)
argv[2]: rw
argv[3]: rdinit=/sbin/init
argv[4]: plat=cachnm
argv[5]: mem=1024M
ELF file is 64 bit
Allocated memory for ELF segment: addr: 0x1100000, size 0x1357110
Loading .text @ 0x81100000 (0x3903e8 bytes)
Loading __ex_table @ 0x814903f0 (0x7660 bytes)
Loading .rodata @ 0x81497a50 (0x4d5a8 bytes)
Loading .pci_fixup @ 0x814e4ff8 (0x600 bytes)
Loading __ksymtab @ 0x814e55f8 (0x86b0 bytes)
Loading __ksymtab_gpl @ 0x814edca8 (0x1530 bytes)
Loading __ksymtab_strings @ 0x814ef1d8 (0xc470 bytes)
Loading __param @ 0x814fb648 (0x14f0 bytes)
Loading .data @ 0x81500000 (0x73c00 bytes)
Loading .data.cacheline_aligned @ 0x81574000 (0x4a00 bytes)
Loading .init.text @ 0x81579000 (0x302f8 bytes)
Loading .init.data @ 0x815a92f8 (0x7068 bytes)
Loading .init.setup @ 0x815b0360 (0x588 bytes)
Loading .initcall.init @ 0x815b08e8 (0x420 bytes)
Loading .con_initcall.init @ 0x815b0d08 (0x10 bytes)
Loading .init.ramfs @ 0x815b1000 (0xe65d36 bytes)
.
.
.
Removed
.
.
.

Router communications servers initializing...complete.
rsrcmgr: module/platnm: cachnm
IOS IP Address Registration complete.
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
1.3.0.0 * 255.255.0.0 U 0 0 0 eth1
default 1.3.202.33 0.0.0.0 UG 0 0 0 eth1
RDBL: Module 8 Type 3 Loc 0 Offset 8
Size of buff is: 65536
65536 bytes written
Reading License... Valid License file not found: Using default values
done
Reading Limits...
Processing: /lib/python2.3/startup/limits.xml
done
INIT: Entering runlevel: 2
********** rc.post_install ****************
rsrcmgr: module/platnm: cachnm
Changing owners and file permissions.
Change owners and permissions complete.
INIT: Switching to runlevel: 4
INIT: Sending processes the TERM signal
STARTED: dwnldr_startup.show
Welcome to Cisco Systems Service Engine Helper Software
Please select from the following
1 Install software
2 Reload module
3 Disk cleanup
4 Linux shell
(Type '?' at any time for help)
Choice: q
Choice: 1
Package name: tpo-k9.aggr.2.0.1.pkg
Server url: ftp://1.3.202.250/pub/
Username: 
Password: 
Downloading ftp tpo-k9.aggr.2.0.1.pkg
Bytes downloaded : 61548 
Validating package signature ... done
WARNING:: Software installation will clear disk contents
Continue [n]? y
cleaning fs
prepfs.sh: cachnm reiser /mnt clean
hd_name: /dev/sda
hd_part: /dev/sda1
umount: /dev/sda1: not mounted
check_partition_count: 1
Command (m for help): Selected partition 1
Command (m for help): Command action
.
.
.
Removed
.
.
.
System Now Booting ... test-base=0xbfc40000
Booting from flash...please wait.
Please enter '***' to change boot configuration:
0 
ServicesEngine boot-loader Version : 1.0.3
boot-loader > boot usb
kernel size: 0x37863000
Decompressing image. 931540992 bytes from Address 0xbf580008 to adress 0x86400000 .....
..........................................................................................
..........................................................................................
..........................................................................................
..........................................................................................
.
.
.
Removed
.
.
.
Creating /dev/ciscofilter 
254
INIT: Entering runlevel: 2
********** rc.post_install ****************
IMPORTANT::
IMPORTANT:: Welcome to Cisco Systems Service Engine 
IMPORTANT:: post installation configuration tool. 
IMPORTANT::
IMPORTANT:: This is a one time process which will guide 
IMPORTANT:: you through initial setup of your Service Engine.
IMPORTANT:: Once run, this process will have configured 
IMPORTANT:: the system for your location.
IMPORTANT::
IMPORTANT:: If you do not wish to continue, the system will be halted 
IMPORTANT:: so it can be safely removed from the router.
IMPORTANT::
Do you wish to start configuration now (y,n)? y
Are you sure (y,n)? y
Enter Hostname
(my-hostname, or enter to use se-1-3-202-26): BRANCH
Enter Domain Name
(mydomain.com, or enter to use localdomain): 
Using localdomain as default
IMPORTANT:: DNS Configuration:
IMPORTANT:: 
IMPORTANT:: This allows the entry of hostnames, for example foo.cisco.com, instead 
IMPORTANT:: of IP addresses like 1.100.10.205 for application configuration. In order 
IMPORTANT:: to set up DNS you must know the IP address of at least one of your
IMPORTANT:: DNS Servers. 
Would you like to use DNS (y,n)?n
WARNING: If DNS is not used, IP addresses will be required.
Are you sure (y,n)? y
Enter IP Address of the Primary NTP Server
(IP address, or enter for 1.3.202.33): 
***************************************************
I could not reach 1.3.202.33 using NTP.
1.3.202.33 might not have been configured as
NTP server.
***************************************************
Do you wish to continue with out Primary NTP server (y,n)? y
Enter IP Address of the Secondary NTP Server
(IP address, or enter to bypass): 
Please identify a location so that time zone rules can be set correctly.
Please select a continent or ocean.
1) Africa 4) Arctic Ocean 7) Australia 10) Pacific Ocean
2) Americas 5) Asia 8) Europe
3) Antarctica 6) Atlantic Ocean 9) Indian Ocean
#? 2
Please select a country.
1) Anguilla 18) Ecuador 35) Paraguay
2) Antigua & Barbuda 19) El Salvador 36) Peru
3) Argentina 20) French Guiana 37) Puerto Rico
4) Aruba 21) Greenland 38) St Kitts & Nevis
5) Bahamas 22) Grenada 39) St Lucia
6) Barbados 23) Guadeloupe 40) St Pierre & Miquelon
7) Belize 24) Guatemala 41) St Vincent
8) Bolivia 25) Guyana 42) Suriname
9) Brazil 26) Haiti 43) Trinidad & Tobago
10) Canada 27) Honduras 44) Turks & Caicos Is
11) Cayman Islands 28) Jamaica 45) United States
12) Chile 29) Martinique 46) Uruguay
13) Colombia 30) Mexico 47) Venezuela
14) Costa Rica 31) Montserrat 48) Virgin Islands (UK)
15) Cuba 32) Netherlands Antilles 49) Virgin Islands (US)
16) Dominica 33) Nicaragua
17) Dominican Republic 34) Panama
#? 45
Please select one of the following time zone regions.
1) Eastern Time
2) Eastern Time - Michigan - most locations
3) Eastern Time - Kentucky - Louisville area
4) Eastern Time - Kentucky - Wayne County
5) Eastern Standard Time - Indiana - most locations
6) Eastern Standard Time - Indiana - Crawford County
7) Eastern Standard Time - Indiana - Starke County
8) Eastern Standard Time - Indiana - Switzerland County
9) Central Time
10) Central Time - Michigan - Wisconsin border
11) Central Time - North Dakota - Oliver County
12) Mountain Time
13) Mountain Time - south Idaho & east Oregon
14) Mountain Time - Navajo
15) Mountain Standard Time - Arizona
16) Pacific Time
17) Alaska Time
18) Alaska Time - Alaska panhandle
19) Alaska Time - Alaska panhandle neck
20) Alaska Time - west Alaska
21) Aleutian Islands
22) Hawaii
#? 16
The following information has been given:
United States
Pacific Time
Therefore TZ='America/Los_Angeles' will be used.
Is the above information OK?
1) Yes
2) No
#? 1
Local time is now: Mon Sep 24 05:05:38 PDT 2007.
Universal Time is now: Mon Sep 24 12:05:38 UTC 2007.
No NTP servers configured.
Would you like to manually adjust the system time (y,n)? n
Configuring the system. Please wait...
Changing owners and file permissions.
Change owners and permissions complete.
INIT: Switching to runlevel: 4
INIT: Sending processes the TERM signal
telnet> 
STARTED: cli_server.sh
STARTED: ntp_startup.sh
STARTED: LDAP_startup.sh
STARTED: dwnldr_startup.sh
STARTED: HTTP_startup.sh
STARTED: probe
STARTED: superthread_startup.sh
STARTED: /bin/products/wpo/wanopt_startup.sh

System-Level Troubleshooting

To verify the status of an installation, upgrade, or downgrade or to troubleshoot problems, use commands as needed from the following list of common router and module commands (Table 10-2).


Note Among keyword options for many show commands is the provision to display diagnostic output on your screen or to pipe it to a file or a URL.


Table 10-2 Common Verification and Troubleshooting Commands

Command
Configuration Mode
Purpose

ping

Router# 
SE-Module>

Pings a specified IP address to check network connectivity (does not accept a hostname as destination).

show arp

Router#
SE-Module>

Shows the current Address Resolution Protocol (ARP) table.

show clock

Router#
SE-Module>

Shows the current date and time.

show controller transport-opt-Service-Engine slot/port

Router#

Shows interface debug information.

show diag

Router#

Shows standard Cisco IOS diagnostics information, including information about the NCE service module.

show hardware

Router#

Shows information about module and host-router hardware.

show hosts

Router#

Shows the default domain name, style of name lookup, list of name-server hosts, and cached list of hostnames and addresses

show interfaces

Router# 

Shows information about the module interfaces.

SE-Module>

Shows information about all hardware interfaces, including network and disk.

show interfaces transport-opt-Service-engine

Router#

Shows information about the module side of the router-module interface.

show ntp status

Router# 
SE-Module>

Shows information about Network Time Protocol (NTP).

show processes

Router# 
SE-Module>

Shows a list of the running application processes.

show running-config

Router# 
SE-Module>

Shows the configuration commands that are in effect.

show startup-config

Router# 
SE-Module>

Shows the startup configuration.

show tech-support

Router# 
SE-Module>

Shows general information about the module that is useful to Cisco technical support for problem diagnosis.

show version

Router# 
SE-Module>

Shows information about the loaded router, software or module, bootloader version, and hardware and device information.

test scp ping

Router#

Pings the module to check network connectivity.

show running-config

SE-Module>

Shows the configuration commands that are in effect.

show software packages

SE-Module>

Shows package information for installed packages.

show software versions

SE-Module>

Shows version information for installed software.


Troubleshooting the NCE module from the Cisco IOS

To troubleshooting the TPO service module:

Make sure the interception or redirection is configured on outgoing WAN interface.

Adjacencies are created for the TPO module and for each tpo ID configured on the WAN interface.

Use commands for troubleshooting the Cisco IOS:

show adjacency (detail)

debug tpo packets

debug tpo errors

debug ip wccp (events/packets)

Use show ip wccp commands to check the WCCP status.

Use the show ip wccp 61 bucket command to match to redirection table on the NCE.

Application-Level Troubleshooting

To configure logging options for the NCE service module, use commands from the following list of common module commands (Table 10-3).


Note Among keyword options for many log and trace commands is provision to display diagnostic output on your screen or to pipe it to a file or a URL.


Table 10-3 Common Logging Commands 

Configuration Mode
Command
Purpose
SE-Module>

log console monitor

Cofigures error logging by means of the console. The following are the different severity levels:

errors - Error messages, severity=3

info - Information messages, severity=6

notice - Notice messages, severity=5

warning - Warning messages, severity=4


There are two types of diagnostics:

System log (syslog)—Syslog is an industry-standard protocol for capturing the following events:

Fatal exceptions that cause an application or system crash, during which normal error-handling paths are typically nonfunctional.

Application run-time errors that cause unusual conditions and configuration changes.

The syslog file size is fixed at 1 MB for an AIM and 10 MB for an NME. Syslog configurations survive a power failure.

Traces—Trace logs capture events related to the progress of a request through the system.

Trace logs survive a CPU reset; trace configurations survive a power failure. To log and display these, use the trace commands.

To generate and display syslog and trace diagnostics, use commands as needed from the lists of common router and module commands in Table 10-4 and Table 10-5.

Use the trace command to help in debugging issues: trace tpo cli inout/debug/error.

Trace TPO gateway [ogw / tgw] errors.

If you are troubleshooting tpo ID configuration, turn on trace tpo jni gw-int. If you are troubleshooting policy map configuration, turn on trace tpo jni pm-int.

If the show command results in an error message from the CLI, it is likely that a process is not running. Check the messages log. See the "Messages Log" section.

Table 10-4 Common Syslog Commands 

Configuration Mode
Command
Purpose
Router#

show log

Shows the contents of the specified log.

SE-Module>

copy log

Saves the syslog to a destination that you choose.

SE-Module>

show log

Shows the contents of the specified log.

SE-Module>

show logs

Shows a list of the available log files.


Table 10-5 Common Trace Commands 

Configuration Mode
Command
Purpose
SE-Module>

clear trace

Clears the logged trace events for specified modules.

SE-Module>

log trace

Logs the configured traces to the module (can be done locally or remotely).

SE-Module>

no trace

Disables the tracing for specified modules, entities, or activities.

SE-Module>

show errors

Shows error statistics by module, entity, or activity.

SE-Module>

show trace

Shows the trace settings.

SE-Module>

show trace buffer

Shows the contents of the trace buffer.

SE-Module>

show trace store

Shows the contents of the traced messages that are stored.

SE-Module>

trace

Enables tracing (that is, generates error reports) for specified modules, entities, or activities.


Troubleshooting the NCE Module

When troubleshooting the NCE module, check the following:

1. SCTP pipe not coming up.

Ping the IP route to the peer IP address.

Check if the keepalives are received and sent out to the other end in gateway statistics using the show tpo statistics gateway command.

2. Traffic is not getting optimized.

Check the service policy / default policy on the tpo interface.

Check the SYNs statistics are getting bypassed using the show tpo stat filter command.

Use show commands to check the state of the incoming connections.

Check for tpo-lookup type.

3. WCCP debugging.

Check the redirection table on the Branch to make sure it's the same as the one populated on the Data Center side.

Check if the load balancing is configured correctly on all the modules which are part of the WCCP group.

Check the WCCP status on the Data Center NCE modules and WCCP router.

Check if the modules are registered correctly with the WCCP router on the Data Center side.

Check if the WCCP updates from the Data Center are received on the branch in the gateway statistics using the show tpo statistics gateway command.

Use error logging on the NCE console for error messages.

Debugging Cisco IOS

Make sure the interception or redirection is configured on the WAN interface

Adjacencies must be created for the TPO module and for each tpo ID configured on the WAN interface

Commands generally used for debugging the Cisco IOS

show adjacency (detail)

debug tpo packets

debug tpo errors

debug ip wccp (events/packets)

Use show ip wccp commands to check the WCCP status

show ip wccp 61 bucket - match to redirection table on the NCE

Common scenarios

1. SCTP pipe does not come up

Ping the IP route to peer IP address.

Check if the keepalives are received and sent out to the other end in gateway statistics using the show tpo statistics gateway command.

2. Traffic does not get optimized

Check the service policy / default policy on the tpo interface.

Check the SYNs statistics using the show tpo stats filter command - they should be getting bypassed.

Use show commands to check the state of the incoming connections.

Check for the tpo-lookup type.

WCCP debugging.

Check the redirection table on the Branch to make sure it is the same as the one populated on the Data Center side.

Check if the load-Balancing is configured correctly on all the modules which are part of the WCCP group.

Check WCCP status on HQ NCE modules and WCCP router.

Check if the modules are registered correctly with the WCCP router on the Data Center side.

Check if the WCCP updates from HQ are received on the branch in the gateway statistics using the show tpo statistics gateway command.

Use error logging on linux console for error messages.

Debugging commands

Use the tpo debug packets command to debug traffic by filtering based on the protocol type.

all

All packets

ip

All IP packets

sctp

SCTP packets

tcp

TCP packets


Use the tpo debug filter-events command

all

View all details

detailed-trace

View detailed trace

errors

View error details

events

View event details

informational

View informational details

nat

View NAT details


to debug all the events and errors at the filter level.

Use the show packets command debug connection issues. This command uses tcpdump on the NCE application to sniff the packets on the Ethernet interfaces.

Using Trace

Tracing can be enabled for different individual processes and for specific levels.

Use trace tpo [option] to turn on trace activity. The arguments for option are:
cli | gateway | statistics | cli | policymanager | procmanager | jni | all

To display the trace options that are set and the activity level, use the show trace command. To display the trace log, use the show trace buffer [option] command. The trace log can be saved to a file and decoded later. The following sample configuration lists available trace options:

service-module> trace tpo ?
  all          Every entity and activity
  cli          Command-Line-Interface Entity
  gateway     Gateway Entity 								[to debug gateway, such as issues with pipe status]
  jni          Entity
  policymgr    Policy Manager Entity [to debug service-policy issues]
  procmgr      Entity 								[to debug process manager issues]
  statsmgr     Entity                            [to debug statistics issues]

The following sample configuration lists available trace activity levels for each preceding option:

service-module> trace tpo gateway ?
  all          Every activity
  cli          command-line-interface 
  control      control activity
  control-msg  control-messages activity
  data-msg     data-message related activity
  error        errors
  event        events
  inout        all incoming/outgoing messages

se-1-3-252-181> trace tpo gateway all
se-1-3-252-181> show trace
MODULE           ENTITY          SETTING
tpo              gateway         ffffffff  {activity level - all: is for enabled tpo 
[option] all]

LOG NAME                         STATUS
atrace.log                       disabled

How to Read the Trace Buffer

To examine the trace buffer, first enable trace. For example:

trace tpo gateway event

show trace

Pass traffic; then do the following:

1. To display the trace buffer, enter the show trace buffer command.

2. To show the latest 10 events, enter the show trace buffer tail 10 command.

3. to save trace buffer to a file (/var/log/atrace_save.log), enter the log trace buffer save command.

Example:

Service-module>  show trace
MODULE           ENTITY          SETTING
No tracing active
LOG NAME                         STATUS
atrace.log                       disabled

se-1-3-252-181> show trace ?
  <cr>         
  buffer       Print recent system event messages
  store        Print system event messages from hard-drive store
  store-prev   Print system event messages from previous hard-drive store
  |            Pipe output to another command

In the following example, the messages with the keyword tpo *gw* are trace gateway all messages.

Messages with tpo cli keyword are the messages related to CLI messages.


Note Using the show trace buffer tail command creates a CPU-intensive process. We recommended that you use this command in troubleshooting issues when there is minimal traffic.


se-1-3-252-181> show trace buff
Press <CTRL-C> to exit...
1444 09/11 01:09:39.905 tpo *gw* 2 gw_sctp_conn_req():2804 will let 1.3.252.190 (peer) be 
the initiator
1444 09/11 01:09:39.905 tpo *gw* 2 gw_sctp_cfg_timer():1005 Retry interface 3 config tos 1 
status tos 0
1444 09/11 01:09:39.905 tpo *gw* 2 gw_sctp_conn_req():2794 Start: request to connect to 
1.3.252.191 tos 1 intf_id 3
1444 09/11 01:09:39.905 tpo *gw* 2 gw_sctp_conn_req():2802 this node 1.3.252.181 
1444 09/11 01:10:15.547 tpo *gw* 2 gw_sctp_peer_cfg_hdlr():800 inserting in tree
1444 09/11 01:10:15.547 tpo *gw* 2 gw_sctp_peer_cfg_hdlr():811 insert SUCCESS 
1444 09/11 01:10:15.547 tpo *gw* 2 gw_sctp_conn_req():2794 Start: request to connect to 
11.11.11.11 tos 1 intf_id 5
1444 09/11 01:10:15.547 tpo *gw* 2 gw_sctp_conn_req():2802 this node 1.3.252.181 
1444 09/11 01:10:15.547 tpo *gw* 2 gw_sctp_conn_req():2804 will let 11.11.11.11 (peer) be 
the initiator
1444 09/11 01:10:15.547 tpo *gw* 2 rearm_event():136 event added successfully
2071 09/11 01:10:15.547 tpo *gw* 2 gwlib_cli_intf_cfg: sendto done; fd= 64 toPort= 9990
2071 09/11 01:10:15.547 tpo cli 1 JNI: (AddPeerToInterface) Ret val: 0
2071 09/11 01:10:15.547 tpo cli 1 ===============================
ConfigSubmodeHandler/eval END.
2071 09/11 01:10:17.522 tpo cli 1 ===============================
EndMode/eval START.
2071 09/11 01:10:17.522 tpo cli 1 dirpath: /sw/apps/wpo/wpoIntDir/wpoIntDir11
2071 09/11 01:10:17.522 tpo cli 1 Utils/get_dir_count: next dir: sctpPeerDir0
2071 09/11 01:10:17.522 tpo cli 1 substring: wpoIntDir11
2071 09/11 01:10:17.522 tpo cli 1 last index 0
begin index 9
Dir ID: 11
2071 09/11 01:10:17.522 tpo cli 1 Interface ID: 11
2071 09/11 01:10:17.522 tpo cli 1 subdir count is: 1
2071 09/11 01:10:17.522 tpo cli 1 Utils/configure_default_peer START
2071 09/11 01:10:17.522 tpo cli 1 Utils/get_configured_tosbits START.
2071 09/11 01:10:17.522 tpo cli 1 dirPath: /sw/apps/wpo/wpoIntDir/wpoIntDir11/sctpPeerDir
2071 09/11 01:10:17.522 tpo cli 1 got sdir: 
/sw/apps/wpo/wpoIntDir/wpoIntDir11/sctpPeerDir/sctpPeerDir0

Messages Log

By default, all error messages and important events are written to the messages.log file.

To examine the contents of the file, use the show log name messages.log command.

The messages.log file has a file size limit of 5 MB. When the file reaches the file size limit, the messages are copied to messages.log.prev and cleaned from the messages.log file.

New messages are then written to the messages.log file.

The following is sample output of the messages.log file:

<4>Sep 11 11:53:00 localhost kernel: Opened (pid 1528) Filter Device
<27>Sep 11 11:53:00 localhost Transport-Opt:  setpriority successful
<27>Sep 11 11:53:00 localhost Transport-Opt:  Starting process: (/bin/gateway): pid = 1537
<27>Sep 11 11:53:00 localhost Transport-Opt:  number of restarts = 1
<27>Sep 11 11:53:00 localhost Transport-Opt:  setpriority successful
<27>Sep 11 11:53:00 localhost Transport-Opt:  Starting process: (/bin/statsmgr): pid = 
1538
<27>Sep 11 11:53:00 localhost Transport-Opt:  number of restarts = 1
<27>Sep 11 11:53:00 localhost gateway: Alert: appmain():719 GATEWAY STARTING
<27>Sep 11 11:53:00 localhost Transport-Opt:  setpriority successful
<27>Sep 11 11:53:00 localhost Transport-Opt:  Closing the file descriptors for process 
manager
<27>Sep 11 11:53:00 localhost Transport-Opt:  startup_sync: APP_ONLINE
<27>Sep 11 11:53:00 localhost gateway: gw_stats_startup: GW STATS INIT. fname(gw_stats) 
pid(1554)