• The figures provided here assume all necessary software licenses are applied.

• The figures are tested for specific, dedicated Expressway scenarios. Based on an Expressway or cluster being used for a single service or scenario, such as just for MRA or just for B2B calling. It's not possible to provide tested capacity guidelines for multi-service deployments.

• Up to six Expressway systems can be clustered, but this only increases capacity by a maximum factor of four (except Small VMs, which have no gain).

• For Small VMs, clustering is only for redundancy and not for scale and there is no capacity gain from clustering.

• The figures provided for video calls and audio-only calls are alternatives - the stated capacity is available either for video or for audio, not for both.

The figures for calls refer to concurrent calls.

Concurrent calls and Rich Media Session (RMS) licenses do not have a one-to-one relationship. Various factors determine RMS license usage, which means that some calls may be "free" and others may use multiple licenses.

To support 6000 TURN relays on a large system (Large VM or CE1200) you need to enable "TURN Port Multiplexing on Large Expressway" (Configuration > Traversal > TURN).

Small VMs are supported on the Cisco Business Edition 6000 platform, or on general purpose hardware / ESXi which matches the Cisco Business Edition 6000 specification. The figures for Small VMs are for M5-based BE6000 appliances.

This table shows the base capacity for a standalone Expressway.

This table illustrates the increased capacity for a clustered system with four Expressways (the maximum cluster size for scale gain).

To determine the capacity for clusters with two or three nodes, apply a factor of 2 or 3 respectively to the standalone figures. Except for Small VMs, where the figures for clustered systems and for standalone systems are always the same (because there's no capacity gain from clustering Small VMs).

Say you want to deploy a resilient cluster that can handle up to 750 concurrent desktop registrations and 250 Rich Media Sessions. In this case you could configure 4 peers as follows:

In this example it doesn't matter which peer an endpoint registers to, as the licenses are shared across all of the peers. If any one peer is temporarily taken out of service the full set of call licenses remain available to the entire cluster.

License usage when endpoints are registered to different peers in the same cluster, depends on call media traversal across the cluster:

• If call media does not traverse the cluster peers, a call between the endpoints does not use any RMS licenses (it's a "Registered" call).

  • If any of the endpoint is not registered to Cisco infrastructure then calls will use RMS license.

• If call media does traverse the cluster peers, a call between the endpoints uses an RMS license on the Expressway where the B2BUA is engaged.

  • If both the endpoints are registered to Cisco infrastructure then call will not use RMS license.

More information about how licenses are used in clustered systems is provided in the licensing section of this guide.

The Clustering page (System > Clustering) lists the IP addresses of all the peers in the cluster, to which this Expressway belongs, and identifies the configuration primary peer.

Most items of configuration are applied via the primary peer to all peers in a cluster. However, the following items (marked with a on the web interface) must be specified separately on each cluster peer.

Configuration data that applies to all peers should only be modified on the primary peer. Otherwise, at best the changes will be overwritten from the primary or at worst the cluster replication will fail.

Service setup wizard

Configuration settings made through the service setup wizard (including Select Type, Select Series, service selection, licensing for those services, and basic network settings) must be configured on each peer in a cluster.

Cluster configuration (System > Clustering)

The list of Peer N addresses (including the peer's own address) that make up the cluster must be specified on each peer and must be identical for all peers.

The Cluster name, Configuration primary, and Cluster IP version must also be specified on each peer and must be identical for all peers.

If you need to enable cluster address mapping, we recommend forming the cluster on IP addresses first. Then you only need to add the mappings on one peer.

Ethernet speed (System > Network interfaces > Ethernet)

The Ethernet speed is specific to each peer. Each peer may have slightly different requirements for the connection to their Ethernet switch.

IP configuration (System > Network interfaces > IP)

LAN configuration is specific to each peer.

• Each peer must have a unique IP address, whether that is an IPv4 address, an IPv6 address, or both.

• IP gateway configuration is peer-specific. Each peer can use a different gateway.

Note	The IP protocol is applied to all peers, because each peer must support the same protocols.

IP static routes (System > Network interfaces > Static routes)

Any static routes you add are peer-specific and you may create different routes on different peers if required. If you want all peers in the cluster to be able to use the same static route, you must create the route on each peer.

System name (System > Administration)

The System name must be different for each peer in the cluster.

DNS servers and DNS host name (System > DNS)

DNS servers are specific to each peer. Each peer can use a different set of DNS servers.

The System host name and Domain name are specific to each peer.

NTP servers and time zone (System > Time)

The NTP servers are specific to each peer. Each peer may use one or more different NTP servers.

The Time zone is specific to each peer. Each peer may have a different local time.

SNMP (System > SNMP)

SNMP settings are specific to each peer. They can be different for each peer.

Logging (Maintenance > Logging)

The Event Log and Configuration Log on each peer only report activity for that particular Expressway. The Log level and the list of Remote syslog servers are specific to each peer. We recommend that you set up a remote syslog server to which the logs of all peers can be sent. This allows you to have a global view of activity across all peers in the cluster.

Security certificates (Maintenance > Security)

The trusted CA certificate, server certificate and certificate revocation lists (CRLs) used by the Expressway must be uploaded individually per peer.

Administration access (System > Administration)

The following system administration access settings are specific to each peer:

• Serial port / console

• SSH service

• Web interface (over HTTPS)

• Redirect HTTP requests to HTTPS

• Automated protection service

Option keys (Maintenance > Option keys)

This section only applies to systems that use PAK-based licensing (option keys do not apply if your system uses Smart Licensing). Option keys can control licensing or specific features. They are gradually being phased out for Expressway and their use is diminishing.

Option keys that control licenses are pooled for use by the whole cluster.

Option keys that control features (such as advanced account security or Microsoft Interoperability) are specific to the peer where they are applied. Each peer must have an identical set of feature option keys installed, which means that if you use option keys for features you must purchase a key for each peer in the cluster.

License option keys can be applied to one or more peers in the cluster, and the sum of the installed licenses is available across the cluster. This license pooling behavior includes the following option keys:

• Rich media sessions

• Telepresence room systems

• Desktop systems

Note	In some cases a peer will raise an alarm that it has no key to enable licenses the peer needs, even though there are licenses available in the cluster. You can acknowledge and ignore this category of alarm, unless the only peer that has the required licenses is out of service.

Active Directory Service (Configuration > Authentication > Devices > Active Directory Service)

When configuring the connection to an Active Directory Service for device authentication, the NetBIOS machine name (override), and domain administrator Username and Password are specific to each peer.

Conference Factory template (Applications > Conference Factory)

The template used by the Conference Factory application to route calls to a conferencing server must be unique for each peer in the cluster.

When a cluster peer receives a search request (such as an INVITE), it checks its own and its peers' registration lists before responding. This allows all endpoints in the cluster to be treated as if they were registered with a single Expressway.

Peers are periodically queried to ensure they are still functioning.

H.323 registrations

All the peers in a cluster share responsibility for their H.323 endpoint community. When an H.323 endpoint registers with one peer, it receives a registration response which contains a list of alternate gatekeepers, populated with a randomly ordered list of the IP addresses of all the other peers in that cluster.

If the endpoint loses contact with the initial peer, it will seek to register with one of the other peers. The random ordering of the list of alternate peers ensures that endpoints that can only store a single alternate peer will failover evenly across the cluster.

When using a cluster, you may want to reduce the registration Time to live on all peers in the cluster from the default 30 minutes. This setting determines how often endpoints are required to re-register with their Expressway, and reducing it means that if a cluster peer is unavailable, the endpoint will failover more quickly to an available peer.

Note

By reducing the registration time to live too much, you risk flooding the Expressway with registration requests, which will severely impact performance. This impact is proportional to the number of endpoints, so you should balance the need for occasional quick failover against the need for continuous good performance.

To change this setting, go to Configuration > Protocols > H.323 > Gatekeeper > Time to live.

SIP registrations

The Expressway supports multiple client-initiated connections (also referred to as "SIP Outbound") as outlined in RFC 5626.

This allows SIP endpoints that support RFC 5626 to be simultaneously registered to multiple Expressway cluster peers. This provides extra resiliency: if the endpoint loses its connection to one cluster peer it will still be able to receive calls via one of its other registration connections.

You can also use DNS round-robin techniques to implement a registration failover strategy. Some SIP UAs, such as Jabber Video, can be configured with a SIP server address that is an FQDN. If the FQDN resolves to a round-robin DNS record populated with the IP addresses of all the peers in the cluster, then this could allow the endpoint to re-register with another peer if its connection to the original peer is lost.

When clustering has been configured, all peers share the bandwidth available to the cluster.

• Peers must be configured identically for all aspects of bandwidth control including subzones, links and pipes.

• Peers share their bandwidth usage information with all other peers in the cluster, so when one peer is consuming part or all of the bandwidth available within or from a particular subzone, or on a particular pipe, this bandwidth will not be available for other peers.

For general information on how the Expressway manages bandwidth, see the bandwidth control section.

Upgrading a cluster

See the Cisco Expressway Cluster Creation and Maintenance Deployment Guide, for your version, on the Cisco Expressway Series Configuration Guides page.

Note	If you are upgrading to X8.8 or later from an earlier version, clustering communications changed in X8.8 to use TLS connections between peers instead of IPSec. TLS verification is not enforced (by default) after you upgrade, and you'll see an alarm reminding you to enforce TLS verification.

Backing up a cluster

Use the backup and restore process to save cluster configuration information. The backup process saves all configuration information for the cluster, regardless of the Expressway used to make the backup.

Caution

Do not take VMware snapshots of Cisco Expressway systems. The process interferes with database timing and negatively impacts performance.

Restoring a cluster

To restore previously backed up cluster configuration data, follow this process.

Important

You can't restore data to an Expressway that is part of a cluster. As described here, first remove the Expressway peer from the cluster. Then do the restore. (After the restore you need to build a new cluster.)

1. Remove the Expressway peer from the cluster so that it becomes a standalone Expressway.

2. Restore the configuration data to the standalone Expressway. See Restoring a Previous Backup for details.

3. Build a new cluster using the Expressway that now has the restored data.

4. Take each of the other peers out of their previous cluster and add them to the new cluster. See Setting Up a Cluster for details.

Note	No additional steps are required if you are using FQDN's and have a valid cluster address mapping configured. Mappings will be configured on a restore action.

Cisco TMS version 13.2 or later is mandatory if your cluster is configured to use FindMe or Device Provisioning.

Size limitations for clusters and provisioning

An Expressway cluster of any size supports up to:

• 10,000 FindMe accounts

• 10,000 users for provisioning

• 200,000 phonebook entries

Note	Even if the device registration capacity limit of your system is greater, you are limited to 10,000 FindMe accounts/users and 10,000 provisioned devices per cluster.

If you need to provision more than 10,000 devices, your network will require additional Expressway clusters with an appropriately designed and configured dial plan.

See the Cisco Expressway Cluster Creation and Maintenance Deployment Guide, for your version, on the Cisco Expressway Series Configuration Guides page.

When two or more Expressways are clustered together, a new subzone is created within the cluster’s Local Zone. This is the Cluster Subzone (see the diagram in the About Clusters section). Any calls between two peers in the cluster will briefly pass via this subzone during call setup.

The Cluster Subzone is (like the Traversal Subzone) a virtual subzone used for call routing only, and endpoints cannot register to this subzone. After a call has been established between two peers, the Cluster Subzone will no longer appear in the call route and the call will appear as having come from (or being routed to) the Default Subzone.

The two situations in which a call will pass via the Cluster Subzone are:

• Calls between two endpoints registered to different peers in the cluster.

  For example, Endpoint A is registered in the Default Subzone to Peer 1. Endpoint B is also registered in the Default Subzone, but to Peer 2. When A calls B, the call route is shown on Peer 1 as Default Subzone -> Cluster Subzone, and on Peer 2 as Cluster Subzone -> Default Subzone.

• Calls received from outside the cluster by one peer, for an endpoint registered to another peer.

  For example, we have a single Expressway for the Branch Office, which is neighbored to a cluster of 4 Expressways at the Head Office. A user in the Branch Office calls Endpoint A in the Head Office. Endpoint A is registered in the Default Subzone to Peer 1. The call is received by Peer 2, as it has the lowest resource usage at that moment. Peer 2 then searches for Endpoint A within the cluster’s Local Zone, and finds that it is registered to Peer 1. Peer 2 then forwards the call to Peer 1, which forwards it to Endpoint A. In this case, on Peer 2 the call route will be shown as Branch Office -> Default Subzone -> Cluster Subzone, and on Peer 1 as Cluster Subzone -> Default Subzone.

Note	If Call signaling optimization is set to On and the call is H.323, the call will not appear on Peer 2, and on Peer 1 the route will be Branch Office > Default Subzone.

You can neighbor your local Expressway (or Expressway cluster) to a remote Expressway cluster. The remote cluster might be a neighbor, traversal client, or traversal server to the local system. When a call is received on the local Expressway and is passed via the relevant zone to the remote cluster, it gets routed to whichever peer in that neighbor cluster has the lowest resource usage (peers in maintenance mode are not considered). That peer then forwards the call to one of the following:

• A locally registered endpoint, if the endpoint is registered to that peer.

• A peer, if the endpoint is registered to another peer in the cluster.

• An external zone, if the endpoint is located elsewhere.

Lowest resource usage is determined by comparing the number of available media sessions (maximum - current use) on the peers, and choosing the peer with the highest number.

Expressways that are configured as peers must not also be configured as neighbors to each other, or the other way round.