Unified Contact Center Enterprise (UCCE) Peripheral Gateway (PG) TCP Port Range Assignment and duplex operations

Introduction

This document describes the troubleshooting method for the problem identification of Unified Contact Center Enterprise (UCCE) Peripheral Gateway (PG) remaining in isolated enabled state, especially after a fresh PG installation or the upgrade attempt. Typical causes of this problem are related to the incorrect PG port numbers assigned by the UCCE application installer during installation and upgrade run. This document also summarizes a list of ports used for maintaining PG duplex operations.

Background Information

For duplex PG pairs, when you run OPCTEST status command on either side of the PG, OPC status is visible only for the one side of PGs, and the other side is not reachable. As if each PG server of the PG pairs is running under simplex mode on its own.

Multilayer Data Switch (MDS) process log indicates MDS initiates registration process of all its clients, and duplex operation is requested but eventually failed. MDS goes into simplex operation.

Open Peripheral Controller (OPC) process log indicates the pairing side of OPC went offline due to MDS problem.

PGAG process logs also indicate TOS events, and eventually receiving reports from MDS process that the other side is not reachable, MDS in simplex mode operations etc.

Since these are catastrophic failures, these errors are obvious and readily available even with default traces turned up on PG services,

Troubleshoot steps first involves some basic network checks:

1. Checking IP addresses configuration on Pub and Private NICs
2. Name resolutions
3. Network connectivities
4. Window route table inspections

However, there are cases where the problem persists once all the steps are reviewed. Then our topic starts at this point onwards.

Note: All ports as referenced in this article are TCP ports.

Before the installer lays out the port range schema for a PG, it has to first inspect existing PG installation on the server to ensure that there are no port range conflicts after the installation.  In some cases, especially after an upgrade, migration, and re-installation of PG servers, assigned ports could be mismatched between a pairing PG servers. The mismatches are likely caused by the existing ICM installations or the remanence of it on the servers, and coverage installations are performed on top of existing installation.

Important system numbers

PG port ranges are assigned by ICM installer during installation time, and the values are based on certain factors, Enterprise Instance Number is one of such factor and the first thing to verify.

Enterprise Instance Number

Inherited from old Geotel Intelligent Call Management (ICM) software architecture, instance number is used to differentiate the software contexts. The default value is 0. It is usually determined during installation time. if the Instance Number is not consistently maintained across the entire enterprise setup of the software, port number mismatches will definitely occur.

HKEY_LOCAL_MACHINE\SOFTWARE\Cisco Systems, Inc\ICM\<Customer Name>\CurrentVersion\InstanceNumber 

PG port calculation based on Instance number n

port number (instance n) = port number (instance 0) + 40 * n.

PG DMP system id

PG DMP system id determines the designation of a PG pair in the UCCE Call Router, misconfiguration of this value will cause conflict in terms of communication failure between Call Routers and PGs. Different values if are configured across a PG pair by mistake will not allow PG pair to form duplex obviously.

HKEY_LOCAL_MACHINE\SOFTWARE\Cisco Systems, Inc\ICM\<Customer Name>\PGXX>\DMP\CurrentVersion\SystemID

PG1 has system ID 1

PG2 has system ID set to 2 by the installer.

PG3 has system ID as 3

PG duplex/simplex installation state

To confirm whether the PG is installed for duplex operation or simplex operation. If one side of PG pairs is installed with simplex oepration, no state transfer between PGs can be initiated.

HKEY_LOCAL_MACHINE\SOFTWARE\Cisco Systems,Inc.\ICM\<Customer Name>\PG<XX>\NodeManager\CurrentVersion\Dumplexed

Simplex PG the value is set to 0

Duplex PG the value is set to 1

Port Ranges on Each PG Host Server

By the design, there could only be 2 PG nodes on each host server. If the installer found an existing PG installation on the host server, it tries to use different sets of ports for the new PG.

Regardless the PG DMP or system id of the PG, Installer installs 43XXX range of these ports to the first pg installation on a host server. The 2nd PG node installed on the same host server uses the 45XXX port range. ie. if PG2 was installed before PG1 on the host server, the relevant ports associated to PG2 nodes take up 43XXX port range, whereas PG1 is given 45XXX port range.

This is where port mismatch could occur even with the order of PG installation.

Port Index

HKEY_LOCAL_MACHINE\SOFTWARE\Cisco Systems, Inc.\ICM\<instance-name>\PGXX\PortIndex

For instance, 0 1st installed PG, the PORT value is 0 on both sides of PG.

For instance, 0 2nd installed PG, the PORT value is 1 on both sides of PG.

PG PGAG Test Other Side port

This is the port used when PG pair loses the private network connectivity and sends out TOS messages through PGAG process via the Public Interfaces.

HKEY_LOCAL_MACHINE\SOFTWARE\Cisco Systems, Inc\ICM\<Customer Name>\PGXX\DMP\CurrentVersion\PGTestOtherSide

For instance, 0 1st installed PG, the PORT value is 43006 on both sides of PG.

For instance, 0 2nd installed PG, the PORT value is 45006 on both sides of PG.

PG MDS ports

A set of ports that are responsible for High, Medium, Low MDS traffics between the PG pairs via private interfaces.

HighPriority

HKEY_LOCAL_MACHINE\SOFTWARE\Cisco Systems, Inc\ICM\<Customer Name>\PGXX\MDS\CurrentVersion\DedicatedPath\HighPriority

For instance 0 and 1st PG, the PORT value is 43005 on both sides of PG
For instance 0 and 2nd PG, the PORT value is 45005 on both sides of PG

MediumPriority

HKEY_LOCAL_MACHINE\SOFTWARE\Cisco Systems, Inc\ICM\<Customer Name>\PGXX\MDS\CurrentVersion\DedicatedPath\MediumPriority

For instance 0 and 1st PG, the PORT value is 43016 on both sides of PG
For instance 0 and 2nd PG, the PORT value is 45016 on both sides of PG

LowPriority

HKEY_LOCAL_MACHINE\SOFTWARE\Cisco Systems, Inc\ICM\<Customer Name>\PGXX\MDS\CurrentVersion\DedicatedPath\LowPriority

For instance 0 and 1st PG, the PORT value is 43004 on both sides of PG
For instance 0 and 2nd PG, the PORT value is 45004 on both sides of PG

PG OPC state transfer port

This is the port used for OPC to conduct state transfer through MDS when the PG pair start up.

HKEY_LOCAL_MACHINE\SOFTWARE\Cisco Systems, Inc\ICM\<Customer Name>\PGXX\MDS\CurrentVersion\Clients\opc

For instance 0 and 1st PG, the StateXferPort value is 43023 on both sides of PG
For instance 0 and 2nd PG, the StateXferPort value is 45023 on both sides of PG

Conclusion

For PG duplex operation, state transfer issues, if the problems occur after a major upgrade, migration and coverage installation of PG servers, review above-mentioned ports and numbers, see if one of them are mismatched across both sides of PG pairs.