Configure and Troubleshoot UCCE Outbound Option High Availability - Quick Reference

Introduction

This document describes how to configure and troubleshoot Cisco Unified Contact Center Enterprise (UCCE) Outbound Option High Availability (OOHA).

Prerequisites

Requirements

Cisco recommends that you have knowledge of these topics:

• UCCE Outbound Option
• Microsoft SQL Transactional Replication

Components Used

The information in this document is based on these software and hardware versions:

• Cisco UCCE 11.6
• MS SQL Server 2014

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.

Background Information

Architecture

Outbound Option High-Availability (OOHA) feature was introduced in UCCE 11.6 version. OOHA is an optional feature. From UCCE 11.6 version Campaign Manager process can be redundant with Active-StandBy failover model. When OOHA is enabled in WebSetup, the system automatically does SQL bi-directional transactional replication between BA_A and BA_B databases.

These tables are replicated:

• Contact
• Dialing_List
• PCB
• Do_Not_Call

UCCE 11.6 OOHA Architecture

  

Overview of Failover Models

Campaign Managers Active - StandBy

• Active Campaign Manager process initiates failover if there is no dialer connection for more than 60 seconds by default. This timer can be changed by adding dword EMTClientTimeoutToFailover under Logger/BlendedAgent/CurrentVersion/ registry path; the value must be a wait time for dialer connection in seconds.
• Campaign Manager processes keep bouncing from A to B and vice versa if dialer cannot establish a connection to any of them.
• Campaign Manager failover can take up to 4,5 minutes if there is a huge replication queue between BA databases. 4,5 minutes is a hardcoded timer and cannot be changed.

Dialers Active - StandBy

• No changes from the previous versions. Dialer failover model remains the same, only one dialer active at a time.

BaImport - No Failover

• BaImport works only with the local Campaign Manager process and replicates its status. In case of BaImport process crash, failover on Campaign Manager level is triggered.

  

Configure

Preliminary Steps

Step 1. Ensure that SQL Server Replication feature is enabled.

• During installation of SQL, replication as a feature needs to be selected. In order to ensure that the replication feature is enabled on Logger server, navigate to SQL disk drive > setup.exe > Tools and run report Installed SQL Discovery Report
• If the feature is not listed in the report run this command in Windows CMD tool and provide SQL Server instance name in the respective command parameter

setup.exe /q /Features=Replication /InstanceName=<instancename> /ACTION=INSTALL /IAcceptSQLServerLicenseTerms

Step 2. Ensure that SQL Server user account is configured.

• The username and password must be the same on Logger Side A and Logger Side B.
• The user must have the SQL Server System Admin privilege.
• You use this username and password when you run WebSetup to configure Outbound Option and enable Outbound Option High Availability.
• The user does not have to be the SQL sa user. It can be another user but must have sysadmin privilege and remains enabled.

        

Step 3. In SQL user NT AUTHORITY\SYSTEM must have sysadmin role. 

Step 4. Logger server hostname and the SQL Server servername (@@servername) must be the same.

New Install Configuration

Step 1. Create BA databases on both Logger servers.

Step 2. Configure the same local SQL user with sysadmin role on both Loggers.

Step 3. Start WebSetup on LoggerA, edit Logger Component and enable Outbound Option and Outbound High Availability.

Note: Ensure you provide Loggers hostname in Logger Public Interface fields. This value must match SQL server name on respective Logger.

After WebSetup is succesfully completed, you must see Publication created and LoggerA SQL server and Subscribtion on LoggerB.

Check it from the SQL Server Management Studio (SSMS) under Replication > Local Publications on LoggerA and Local Subsciptions on LoggerB.

Run WebSetup on LoggerB, edit Logger component and enable Outbound Option and Outbound High Availability.

Publication must be created on LoggerB and Subscription on LoggerA. 

This image shows Publication and Subscription created on LoggerB server.

This image shows Publication and Subscription created on LoggerA server.

  

Troubleshoot

SQL Replication Health Check

Select Launch Replication Monitor Tool from SSMS to check Replication status.

Replication status must be OK.

Expand the publisher to get more information about performance and latency.

Navigate to second tab Tracer Tokens and select Insert Tracer. That tests the latency between Publisher and Distributor and between Distributor and Subscriber.

This must be checked on both Loggers.

Change SQL Servername 

Open SSMS and run this SQL query.

SELECT @@servername

Compare the output from the query with the Windows server hostname. They must match.

This image shows a problem scenario when hostname of LoggerA and SQL server name do not match. Ensure you fix it before OO HA setup.

In order to drop SQL servername run this command in SSMS against master DB.

EXEC sp_dropserver @server=<current wrong SQL servername>

To add a new SQL servername run this command.

EXEC sp_addserver @server=<new SQL servername>, @local=LOCAL

Restart SQL server and SQL Server Agent from Windows Services and check the output of select @@servername SQL query.

Enable SQL Replication Manually

Caution: Use this procedure only if WebSetup is not able to establish replication and the error(s) are not clear.

Execute this stored procedure against BA databases on both Loggers with respective variable values.

EXEC sp_ba_create_replication 
@instance=<ICM instancename>,
@publisher=<SQL publisher servername>,
@subscriber=<SQL subscriber servername>,
@working_directory =<default c:\Program Files\Microsoft SQL Server\MSSQL12.MSSQLSERVER\MSSQL>,
@login =<sql user with sysadmin role>,
@pwd =<password>

If you face an error "CREATE DATABASE failed", check if MSSQLSERVER account has full acces to the SQL working directory.

This image displays respective error on SQL server logs. 

Ensure MSSQLSERVER account has full access to the SQL working directory.

Ensure that Publication and Subscription are created on each Logger SQL server.

   

Disable SQL Replication Manually

Caution: Use this procedure only if WebSetup is not able to establish replication and the error(s) are not clear.

Execute this procedure against BA databses on both Loggers with respective variable values.

EXEC sp_ba_remove_replication 
@instance = <ICM instancename>,
@subscriber = <SubscriberPublicAddressHostname>

Check if Publications are removed from both Logger SQL servers.

To clear completly SQL server from replication configuration, you need to delete manually Subscriptions and drop Distribution databases on both Logger SQL servers.

USE master
EXEC sp_dropdistpublisher @publisher=<SQL servername>;
EXEC sp_dropdistributiondb @database=distribution;
EXEC sp_dropdistributor;
GO

In some cases the last command can fail with the error message "Cannot drop server name as Distributor Publisher because there are databases enabled for replication on that server".

EXEC sp_dropdistributor @no_checks = 1, @ignore_distributor =1

Related Information

• UCCE 11.6 Outbound Option Guide
• UCCE 11.6 Design Guide - Outbound Option High Availability Considerations
• SQL Transactional Replication Reference
• Technical Support & Documentation - Cisco Systems