Table Of Contents
Monitoring and Maintaining Multicast HA Operations (NSF/SSO and ISSU)
Prerequisites for Monitoring and Maintaining HA Operations (NSF/SSO and ISSU)
Restrictions for Monitoring and Maintaining HA Operations (NSF/SSO and ISSU)
Information About Monitoring and Maintaining Multicast HA Operations (NSF/SSO and ISSU)
Multicast HA Support Differences from Other Routing Protocols
Multicast Graceful Restart Overview
Multicast Checkpointing for HA Operations
PIM Triggered Joins Functional Overview
PIM Triggered Joins and Multicast Traffic Flow
Dynamic Multicast SSO Synchronization Events That Occur During Normal Operation (Steady State)
MFIB Interactions on the Active and Standby RPs Before an RP Switchover
MFIB Interactions During an RP Switchover
Unicast and Multicast NSF/SSO Events That Occur Following an RP Switchover
PIM and MFIB Interactions Following an RP Switchover to Replay DDEs
Operation After the RP Switchover
How to Monitor and Maintain Multicast HA Operations (NSF/SSO and ISSU)
Monitoring Multicast HA Events (NSF/SSO and ISSU)
Modifying the Stale Mroute Flush Timeout Period for Multicast HA Operations
Configuration Examples for Monitoring and Maintaining Multicast HA Operations (NSF/SSO and ISSU)
Example: Monitoring Multicast NSF/SSO Events During an RP Switchover
Unicast Convergence Detection and Multicast Route Control Plane Convergence
Example: Monitoring the Transition from Standby RP to Active RP Following a Switchover
Initial State on Standby RP Before Switchover
Standby RP Transition to Active RP After an RP Switchover
Feature Information for Monitoring and Maintaining Multicast HA Operations (NSF/SSO and ISSU)
Monitoring and Maintaining Multicast HA Operations (NSF/SSO and ISSU)
First Published: November 20, 2009Last Updated: July 30, 2010This module describes IPv4 and IPv6 multicast high availability (HA) support and the concepts and tasks necessary to monitor and maintain multicast HA operations.
Multicast HA capabilities enable Cisco nonstop forwarding (NSF) with stateful switchover (SSO) support for IPv4 and IPv6 multicast, which—following a Route Processor (RP) switchover—reduces the reconvergence time of the multicast control plane to a level that is transparent to most multicast-based applications and In Service Software Upgrade (ISSU) support for Protocol Independent Multicast (PIM).
Finding Feature Information
Your software release may not support all the features documented in this module. For the latest feature information and caveats, see the release notes for your platform and software release. To find information about the features documented in this module, and to see a list of the releases in which each feature is supported, see the "Feature Information for Monitoring and Maintaining Multicast HA Operations (NSF/SSO and ISSU)" section.
Use Cisco Feature Navigator to find information about platform support and Cisco software image support. To access Cisco Feature Navigator, go to http://www.cisco.com/go/cfn. An account on Cisco.com is not required.
Contents
•Prerequisites for Monitoring and Maintaining HA Operations (NSF/SSO and ISSU)
•Information About Monitoring and Maintaining Multicast HA Operations (NSF/SSO and ISSU)
•How to Monitor and Maintain Multicast HA Operations (NSF/SSO and ISSU)
•Configuration Examples for Monitoring and Maintaining Multicast HA Operations (NSF/SSO and ISSU)
•Feature Information for Monitoring and Maintaining Multicast HA Operations (NSF/SSO and ISSU)
Prerequisites for Monitoring and Maintaining HA Operations (NSF/SSO and ISSU)
•This module assumes that your device is configured for IP multicast and is participating in an IP multicast network. For more information about configuring IP multicast using PIM sparse mode (PIM-SM), Source Specific Multicast (PIM-SSM), or bidirectional PIM (bidir-PIM), see the "Configuring a Basic IP Multicast Network" module.
•SSO must be configured and working properly. If you do not have SSO enabled, see the "Stateful Switchover" module.
•This module assumes that you are familiar with NSF concepts. For more information about NSF, see the "Cisco Nonstop Forwarding" module.
•This module assumes that you are familiar with the Cisco IOS ISSU process. For more information, see the "Cisco IOS In Service Upgrade Software Upgrade Process" module.
Restrictions for Monitoring and Maintaining HA Operations (NSF/SSO and ISSU)
•Multicast IPv6 multicast SSO is supported only for PIM-SSM mode and PIM sparse mode using static RP configuration. SSO for bidir-PIM is not supported for IPv6 multicast.
Information About Monitoring and Maintaining Multicast HA Operations (NSF/SSO and ISSU)
•Multicast HA Support Differences from Other Routing Protocols
•Multicast Graceful Restart Overview
•Multicast Checkpointing for HA Operations
•ISSU Support for IP Multicast
Multicast HA Support Differences from Other Routing Protocols
Multicast HA support is different than HA support for other routing protocols because multicast routing (mroute) state is dynamic; that is, mroute state depends on the presence of sources and receivers. At the beginning of SSO, multicast state information known by downstream PIM neighbors is refreshed by the control plane. In addition, mroute state creation can be triggered by data driven events (DDEs) in the following cases:
•Mroute state creation triggered on the first hop designated router (DR) as a result of active source traffic.
•Shortest path tree (SPT) switchovers on the last hop DR; this occurs when traffic on the shared tree is detected on the last hop router.
Mroute states created in these data driven event cases are not learned from PIM join and prune messages from PIM neighbors.
Multicast Graceful Restart Overview
Multicast Graceful Restart (GR) is achieved with a combination of the NSF/SSO—IPv4 Multicast feature, the NSF/SSO—IPv6 feature, and the PIM Triggered Joins feature.
Multicast NSF
Multicast NSF ensures uninterrupted flow of multicast traffic during an RP failure.
Multicast SSO
Multicast SSO ensures that necessary information such as RP information, DDEs, and other multicast information is checkpointed to ensure the seamless takeover of the standby RP after an RP failover.
PIM Triggered Joins
The PIM Triggered Joins feature is used to trigger adjacent PIM neighbors on an interface to send PIM join messages for all (*, G) and (S, G) mroutes that use that interface as a reverse path forwarding (RPF) interface.
Multicast Checkpointing for HA Operations
The following multicast information is synchronized between the active and standby RPs:
•Dynamically learned group-to-RP mappings learned from either Auto-RP or bootstrap router (BSR) (IPv4 only).
•Bidir-PIM designated forwarder (DF) information and bidir-PIM RP route information (IPv4 only).
•Multicast Call Admission Control (MCAC) reservation synchronization information (IPv6 only).
•Multicast VPN (MVPN) tunnel information (IPv4 only).
•Multicast Distribution Tree (MDT) data group state information (IPv4 only).
•PIM register tunnel information (IPv4/IPv6).
•Multicast forwarding state created by DDEs (IPv4/IPv6).
PIM Triggered Joins
The PIM Triggered Joins features is a multicast HA enhancement that improves the convergence of mroutes after an RP switchover. In the event of an RP switchover, this feature utilizes the Generation ID (GenID) value (defined in RFC 4601) as a mechanism to trigger adjacent PIM neighbors on an interface to send PIM join messages for all (*, G) and (S, G) mroutes that use that interface as an RPF interface. This mechanism enables PIM neighbors to immediately reestablish those states on the newly active RP.
Generation IDs (GenIDs)
A GenID is a randomly generated 32-bit value that is regenerated each time PIM forwarding is started or restarted on an interface. In order to process the GenID value in PIM hello messages, PIM neighbors must be running Cisco IOS software with an implementation of PIM that is compliant with RFC 4601.
Note PIM neighbors that are not compliant with RFC 4601 and are unable to process GenID differences in PIM hello messages will ignore the GenIDs.
PIM Triggered Joins Functional Overview
Figure 1 illustrates the operations that occur after an RP switchover on routers that support the PIM Triggered Joins feature.
Figure 1 Operation of PIM Triggered Joins During an RP Switchover
The mechanics of the PIM Triggered Joins feature are as follows:
•In steady state, PIM neighbors exchange periodic PIM hello messages.
•An active RP receives PIM joins periodically to refresh mroute states.
•When an active RP fails, the standby RP takes over to become the new active RP.
•The new active RP then modifies the GenID value and sends the new GenID in PIM hello messages to adjacent PIM neighbors.
•Adjacent PIM neighbors that receive PIM hello messages on an interface with a new GenID then send PIM triggered joins for all (*, G) and (S, G) mroutes that use that interfaces as an RPF interface.
•Those mroute states are then immediately reestablished on the newly active RP.
PIM Triggered Joins and Multicast Traffic Flow
Multicast traffic flow on the neighbors is not affected if it detects a PIM triggered join or PIM hello message from a node with the failing RP within the default PIM hello hold-time interval. Multicast traffic flow on a failing RP is not affected if it is NSF capable.
Caution The default PIM hello hold-time interval is 3.5 x the PIM hello period. Multicast HA operations may not behave properly if the PIM hello interval is configured with a value lower than the default value of 30 seconds.
Multicast NSF/SSO Operations
The following sections describe the multicast NSF/SSO operations that occur before, during, and following an RP switchover:
•Dynamic Multicast SSO Synchronization Events That Occur During Normal Operation (Steady State)
•MFIB Interactions on the Active and Standby RPs Before an RP Switchover
•MFIB Interactions During an RP Switchover
•Unicast and Multicast NSF/SSO Events That Occur Following an RP Switchover
•PIM and MFIB Interactions Following an RP Switchover to Replay DDEs
Dynamic Multicast SSO Synchronization Events That Occur During Normal Operation (Steady State)
During normal operation (steady state), the Cisco IOS software dynamically synchronizes information corresponding to events that modify the multicast forwarding state on the standby RP. Instead of performing periodic bulk synchronization updates, the Cisco IOS software sends updates only for modified entities within internal databases. These updates are triggered by events that cause internal database changes related to the multicast forwarding state.
Note This functionality applies only to the dynamic synchronization on the standby RP for updates to the multicast forwarding state that occur during steady state operation. Bulk synchronization updates, however, are required whenever a standby RP is inserted, reloaded, or reset.
In steady state, the following internal multicast forwarding databases are dynamically synchronized on the standby RP:
•RP Mapping—Internal database that stores group-to-RP mapping information (IPv4 only).
•Bidirectional Route Information—Internal database that stores bidir-PIM RP route information (IPv4 only).
•Bootstrap Cache—Internal database that stores BSR candidate information (IPv4 only).
•Auto-RP Discovery IDB—Internal database that stores Auto-RP discovery message information (IPv4 only).
•RPDF—Internal database that stores the set of interfaces enabled for the reception of bidir-PIM packets for a given bidir-PIM RP (IPv4 only).
•MDT Tunnel—Internal database that stores MVPN MDT tunnel information (IPv4 only).
•PIM Register Tunnel—Internal database that stores PIM register tunnel information (IPv4/IPv6).
•MCAC Reservation—Internal database that stores the identity of IPv6 (S, G) multicast routes for which a MCAC cost is currently accrued for each interface on the active RP (IPv6 only).
MFIB Interactions on the Active and Standby RPs Before an RP Switchover
Before an RP switchover, each Multicast Forwarding Information Base (MFIB) instance keeps a permanent record of DDEs it generated that are passed through the Multicast Routing Information Base (MRIB) on the active RP to the MFIB on the standby RP.
Figure 2 illustrates the multicast NSF/SSO interactions between the MFIB components on the active and standby RPs before a switchover.
Figure 2 MFIB Interactions on the Active and Standby RPs Before an RP Switchover
Unicast and Multicast NSF/SSO Hold-Off Period and Resynchronization of the Data Plane Following an RP Switchover
Following an RP failure, data plane forwarding information is retained despite the fact that the new primary RP does not have a complete set of control plane information. The retention of this information enables forwarding to continue during unicast and multicast routing protocol reconvergence. While unicast and multicast routing protocol reconvergence is in progress, a hold-off period is observed during which no multicast forwarding updates are sent from the multicast routing protocol layer to the data plane layer. The hold-off period ends after unicast and multicast protocol convergence has completed.
Unicast routing protocol convergence begins before multicast protocol convergence. Multicast routing protocol (PIM) convergence does not begin until the multicast protocol layer receives explicit signaling that unicast routing protocol convergence has completed. Unicast protocols that are not SSO-aware are not covered by this signal and are not taken into account when waiting for convergence.
Note Some SSO-aware routing protocols (for example, Border Gateway Protocol) may generate logging messages indicating that the initial convergence has completed (based on an internal timer) before full convergence has occurred. PIM, however, does not provide any explicit indication of reconvergence.
It is possible that the hold-off period may terminate before full convergence of unicast routing protocols, which will result in null RPF interfaces for any affected IP addresses. As additional unicast routing updates are received, the affected multicast routes are updated as needed. This is expected and acceptable behavior for SSO-aware routing protocols that are slower in converging.
Note An RP switchover occurring on a system operating with unicast protocols that are not SSO-aware will cause undesirably long convergence times—but no routing loops—for multicast routes.
At the end of the hold-off period, the multicast data plane layer marks any existing data plane information as stale. That information is subsequently flushed if it is not refreshed through the downloading of the current reconverged control plane information.
MFIB Interactions During an RP Switchover
During an RP switchover, while the routing protocols are reconverging, no changes to the multicast tables will occur. All MFIB instances will enter NSF mode, as illustrated in Figure 3.
Figure 3 MFIB Interactions During an RP Switchover
Unicast and Multicast NSF/SSO Events That Occur Following an RP Switchover
In the event of an RP switchover, even with the continuous synchronization of unicast and multicast routing information from the primary to the standby RP, it is not possible to guarantee that the information most recently updated on the primary RP can be synchronized to the standby RP before a failure occurs on the primary RP. For this reason, following an RP switchover, both unicast and multicast routing protocols trigger the retransmission of routing information from neighboring routers to ensure that the unicast and multicast routing information is current.
For multicast protocol retransmission, the Cisco IOS software triggers a refresh of all multicast routing information available from PIM neighbors using the PIM GenID capability described in RFC 4601. GenID support enables fast mroute reconvergence after a switchover. A GenID is a randomly generated 32-bit value regenerated each time PIM forwarding is started or restarted on an interface. In the event of a switchover, the GenID value is used as a mechanism to trigger adjacent PIM neighbors on an interface to send PIM join messages for all (*, G) and (S, G) mroutes that use that interface as an RPF interface, immediately reestablishing those states on the new primary RP. Internet Group Management Protocol (IGMP) for IPv4 multicast and Multicast Listener Discovery (MLD) group membership information for IPv6 multicast is restored by executing IGMP/MLD queries on all IGMP/MLD interfaces.
Note For more information about GenID support, see the "PIM Triggered Joins" section.
The following multicast NSF/SSO events occur in parallel following an RP switchover:
•The Cisco IOS software empties the queue containing unprocessed synchronization messages for multicast sent by the previous primary RP and starts a unicast IGP convergence fail-safe timer to handle the possibility that unicast Interior Gateway Protocol (IGP) convergence never completes.
•As interfaces come up on the new primary RP, unicast routing protocol reconvergence processing proceeds.
•As each PIM-enabled interface comes up, PIM hello messages are sent out using a new GenID value for the interface. The modified GenID value triggers PIM join and prune messages from all adjacent PIM neighbors on the network to which the interface is attached. As these messages are received, information about mroute states that were missing on the new primary RP are restored except for last hop SPT (S, G) routes and mroutes associated with directly connected hosts with no other intermediate routers. Because this routing information begins to arrive before unicast IGP convergence has occurred, mroutes may initially have NULL RPF ingress interfaces. As this state information is learned, the multicast protocol layer sends the corresponding update messages to the MRIB.
•IGMP/MLD group membership information is restored by the execution of IGMP/MLD queries on all IGMP/MLD interfaces.
•Following IGMP/MLD reporting, the control plane then sends out requests for the MFIB replay of DDEs to retrigger multicast route information that cannot be obtained from PIM neighbors or directly connected hosts.
•After DDE replay, the hold-off period ends. At the end of the hold-off period, the multicast data plane layer marks any existing data plane information as stale and that information is subsequently flushed if it is not refreshed via the downloading of the current reconverged control plane information.
PIM and MFIB Interactions Following an RP Switchover to Replay DDEs
The underlying components that make up the MFIB infrastructure coordinate to ensure successful multicast NSF/SSO operations. In particular, the internal exchange of instructions between PIM and the MFIB, as illustrated in Figure 4, ensure error-free operation and the successful replay of DDEs.
Figure 4 PIM and MFIB Interactions Following an RP Switchover
Operation After the RP Switchover
The new RP (the previous active RP that went down) will work as the standby RP after the repair, reboot, or reinstallation, as shown in Figure 5.
Figure 5 PIM and MFIB Interactions Following an RP Switchover
ISSU Support for IP Multicast
The ISSU process allows Cisco IOS software to be updated or otherwise modified while packet forwarding continues. In most networks, planned software upgrades are a significant cause of downtime. ISSU allows Cisco IOS software to be modified while packet forwarding continues, which increases network availability and reduces downtime caused by planned software upgrades.
To provide the required ISSU and SSO support necessary for IP multicast, a PIM ISSU client was introduced. The PIM ISSU client resides on both the primary and the standby RPs and enables PIM synchronization message transmission between two RPs using different versions of software. The PIM ISSU client performs transformation of PIM dynamic state synchronization messages sent from or received by the RP having the most recent software version. If synchronization messages are sent to a down-level RP, the messages are translated to the older format used by the down-level RP. If messages are received from the down-level RP, the messages are translated to the newer format used by the receiving RP before being passed to the Cisco IOS PIM HA software for processing.
How to Monitor and Maintain Multicast HA Operations (NSF/SSO and ISSU)
This section contains the following tasks:
•Monitoring Multicast HA Events (NSF/SSO and ISSU) (optional)
•Modifying the Stale Mroute Flush Timeout Period for Multicast HA Operations (optional)
Monitoring Multicast HA Events (NSF/SSO and ISSU)
Perform this optional task to monitor multicast HA NSF/SSO and ISSU events.
SUMMARY STEPS
1. enable
2. debug ip multicast redundancy [verbose]
3. show ip pim neighbor
4. show ip multicast redundancy state [verbose]
5. show ip multicast redundancy statistics
6. clear ip multicast redundancy statistics
DETAILED STEPS
Step 1 enable
Enables privileged EXEC mode. Enter your password if prompted.
Router> enable
Step 2 debug ip multicast redundancy [verbose]
Use this command to display IP multicast redundancy events.
This command logs events that are important in verifying the operation of NSF/SSO operation for IP multicast. The classes of events logged by the debug ip multicast redundancy command include SSO events during an RP switchover and dynamic synchronization events that occur during steady state operation.
Use the optional verbose keyword to log events that may occur frequently during normal operation, but that may be useful for tracking in short intervals.
The following is output from the debug ip multicast redundancy command. The output displays the logging message that is displayed when the standby RP is recovered after a standby RP transition:
*Aug 7 02:36:07.843: MCAST-HA-RF: Status event: status=RF_STATUS_OPER_REDUNDANCY_MODE_CHANGE Op=7 RFState=ACTIVEStep 3 show ip pim neighbor
Use this command to display the PIM neighbors discovered by PIMv1 router query messages or PIMv2 hello messages that support the GenID capability.
The output of the show ip pim neighbor command displays the "G" flag to indicate GenID support status for each PIM neighbor. The "G" flag is displayed only if the neighbor supports the GenID capabilities provided by PIM.
GenID support enables fast mroute reconvergence after a switchover. A GenID is a randomly generated 32-bit value regenerated each time PIM forwarding is started or restarted on an interface. In the event of a switchover, the GenID value is used as a mechanism to trigger adjacent PIM neighbors on an interface to send PIM join messages for all (*, G) and (S, G) mroutes that use that interface as an RPF interface, immediately reestablishing those states on the newly active RP.
Router# show ip pim neighborPIM Neighbor TableMode: B - Bidir Capable, DR - Designated Router, N - Default DR Priority,P - Proxy Capable, S - State Refresh Capable, G - GenID CapableNeighbor Interface Uptime/Expires Ver DRAddress Prio/Mode192.168.10.5 Ethernet0/1 00:01:35/00:01:37 v2 1 / DR B S P GStep 4 show ip multicast redundancy state [verbose]
Use this command to display the current redundancy state for IP multicast.
The output displays information about the current multicast redundancy state of the RPs and the current synchronization state of the standby RP.
Router# show ip multicast redundancy state
Multicast IPv4 Redundancy Mode: SSOMulticast IPv6 Redundancy Mode: Not enabledMulticast IPv4 HA state machine status: IdleMulticast IPv6 HA state machine status: IdleSync message epoch: 0Sync message sequence number: 24Stale NSF state flush timeout: 30000 msCurrent sync state: SynchedMulticast ISSU Client Status:PIM MIC client ISSU compatibleMRIB MIC client ISSU compatibleMFIB IPv4 MIC client ISSU compatibleMFIB IPv6 MIC client No ISSU result reportedPLATFORM IPv4 MIC client Unregistered - ignoredPLATFORM IPv6 MIC client Unregistered - ignoredIPv4 SSO supported for: PIM, MRIB, MFIBV4IPv6 SSO blocked by: MFIBV6Step 5 show ip multicast redundancy statistics
Use this command to display IP multicast redundancy statistics.
The output displays the following information:
•A summary statistic showing the current number of synchronization messages awaiting transmission from the active RP to the standby RP. (This count is summed across all synchronization database types.)
•A summary statistic showing the current number of synchronization messages that have been sent from the active RP to the standby RP, but for which the active RP has not yet received acknowledgment from the standby for successful reception. (This count is summed across all synchronization database types.)
•The last two statistics, displaying the count of messages awaiting transmission or acknowledgment, provide a way to measure the load on the internal synchronization message-sending mechanism.
Router# show ip multicast redundancy statistics
Multicast Redundancy StatisticsSync Type Updates Syncs Sync failuresRP mapping 0 0 0Bidir. RP route info 0 0 0Bootstrap cache 4 4 0Autorp discovery IDB 4 4 0RPDF 0 0 0MDT tunnel 0 0 0PIM register tunnel 13 13 0MCAC Reservation 0 0 0Requests Awaiting Sync Msg Transmission: 0Requests Awaiting Sync Msg Acknowledgement: 0Average Sync Wait Time = 1 msAverage Sync Ack Time = 7 msStep 6 clear ip multicast redundancy statistics
Use this command to reset IP multicast redundancy statistics.
Router# clear ip multicast redundancy statistics
Modifying the Stale Mroute Flush Timeout Period for Multicast HA Operations
Perform this optional task to configure an additional timeout period before stale forwarding plane mroute information is flushed. This timeout period is added on to the default NSF route flush time as a delay between the downloading of refreshed multicast control plane route information to the forwarding plane and the flushing of "stale" NSF forwarding plane information retained from SSO before the RP switchover.
Caution It is not recommended that you configure this additional delay unless it is specifically required for your topology because it could increase the risk of routing loops during NSF.
Note You would need to perform this task only if you have a routing protocol that requires additional time to populate routing information after the signaling of unicast routing convergence (for example, BGP) in a configuration with a large number of VPN routing and forwarding (VRF) instances). The need to configure this timeout period may be determined during predeployment SSO stress testing.
SUMMARY STEPS
1. enable
2. configure terminal
3. ip multicast redundancy routeflush maxtime seconds
4. end
5. show ip multicast redundancy state
DETAILED STEPS
Configuration Examples for Monitoring and Maintaining Multicast HA Operations (NSF/SSO and ISSU)
This section provides the following monitoring examples:
•Example: Monitoring Multicast NSF/SSO Events During an RP Switchover
•Example: Monitoring the Transition from Standby RP to Active RP Following a Switchover
Example: Monitoring Multicast NSF/SSO Events During an RP Switchover
The following example shows how to monitor IP multicast NSF/SSO events during an RP switchover using the debug ip multicast redundancy command. The example shows IP multicast events occurring as a standby RP assumes the role of active RP during an SSO switchover. The events labeled "MCAST-HA" are logged by the IP multicast SSO debug facility.
Initial Switchover Detection
The following output is from the debug ip multicast redundancy command. The output shows the initial logging messages that display when the system detects an RP switchover.
00:10:33: %REDUNDANCY-3-SWITCHOVER: RP switchover (PEER_DOWN_INTERRUPT)00:10:33: %REDUNDANCY-5-PEER_MONITOR_EVENT: Standby received a switchover (raw-event=PEER_DOWN_INTERRUPT(11))*Aug 7 02:31:28.051: MCAST-HA: Received cf status CHKPT_STATUS_PEER_NOT_READY*Aug 7 02:31:28.063: MCAST-HA: Received cf status CHKPT_STATUS_PEER_NOT_READY*Aug 7 02:31:28.063: MCAST-HA-RF: Status event: status=RF_STATUS_PEER_COMM Op=0 RFState=STANDBY HOT*Aug 7 02:31:28.063: MCAST-HA-RF: Status event: status=RF_STATUS_OPER_REDUNDANCY_MODE_CHANGE Op=0 RFState=STANDBY HOT*Aug 7 02:31:28.063: MCAST-HA-RF: Status event: status=RF_STATUS_REDUNDANCY_MODE_CHANGE Op=0 RFState=STANDBY HOT*Aug 7 02:31:28.063: MCAST-HA-RF: Status event: status=RF_STATUS_PEER_PRESENCE Op=0 RFState=STANDBY HOT*Aug 7 02:31:28.063: MCAST-HA-RF: Status event: status=RF_STATUS_MAINTENANCE_ENABLE Op=0 RFState=ACTIVE-FAST*Aug 7 02:31:28.063: MCAST-HA-RF: Progression event: RF_Event=RF_PROG_ACTIVE_FAST RFState=ACTIVE-FAST*Aug 7 02:31:28.091: MCAST-HA-RF: Progression event: RF_Event=RF_PROG_ACTIVE_DRAIN RFState=ACTIVE-DRAIN*Aug 7 02:31:28.091: MCAST-HA-RF: Progression event: RF_Event=RF_PROG_ACTIVE_PRECONFIG RFState=ACTIVE_PRECONFIG*Aug 7 02:31:28.091: MCAST-HA-RF: Progression event: RF_Event=RF_PROG_ACTIVE_POSTCONFIG RFState=ACTIVE_POSTCONFIG*Aug 7 02:31:28.103: MCAST-HA: Received cf status CHKPT_STATUS_IPC_FLOW_ON*Aug 7 02:31:28.103: MCAST-HA-RF: Progression event: RF_Event=RF_PROG_ACTIVE RFState=ACTIVEUnicast Convergence Detection and Multicast Route Control Plane Convergence
The following output is from the debug ip multicast redundancy command. As interfaces come up on the new active RP, unicast convergence occurs in parallel with multicast route refresh from PIM neighbors. Unicast convergence is followed by RPF adjustments to the refreshed mroute information.
*Aug 7 02:31:28.107: MCAST-HA: Triggering unicast convergence notification process handling for MVRF IPv4 default*Aug 7 02:31:28.107: MCAST-HA: Triggering unicast convergence notification process handling for MVRF VRF1*Aug 7 02:31:28.107: MCAST-HA: Triggering unicast convergence notification process handling for MVRF VRF2*Aug 7 02:31:28.107: MCAST-HA: Triggering unicast convergence notification process handling for MVRF red*Aug 7 02:31:28.107: MCAST-HA: Triggering unicast convergence notification process handling for all MVRFs*Aug 7 02:31:28.111: MCAST-HA: Beginning unicast convergence notification process handling.*Aug 7 02:31:28.111: MCAST-HA: Unicast convergence completed for MVRF IPv4 default: Triggering RPF updates*Aug 7 02:31:28.111: MCAST-HA: Beginning unicast convergence notification process handling.*Aug 7 02:31:28.111: MCAST-HA: Unicast convergence completed for MVRF VRF1: Triggering RPF updates*Aug 7 02:31:28.111: MCAST-HA: Beginning unicast convergence notification process handling.*Aug 7 02:31:28.111: MCAST-HA: Unicast convergence completed for MVRF VRF2: Triggering RPF updates*Aug 7 02:31:28.111: MCAST-HA: Beginning unicast convergence notification process handling.*Aug 7 02:31:28.111: MCAST-HA: Unicast convergence completed for MVRF red: Triggering RPF updates*Aug 7 02:31:28.111: MCAST-HA: Unicast convergence notification has been received for the only unconverged VRF.Stopping the unicast routing convergence failsafe timer.*Aug 7 02:31:28.111: MCAST-HA: Beginning unicast convergence notification process handling.*Aug 7 02:31:28.111: MCAST-HA: Unicast convergence notification received for the wildcard tableid (all VRFs).Triggering RPF updates for all MVRFs and stopping the unicast IGP convergence failsafe timer.00:10:34: %PIM-5-DRCHG: DR change from neighbor 0.0.0.0 to 172.16.1.1 on interface Loopback000:10:34: %PIM-5-DRCHG: DR change from neighbor 0.0.0.0 to 172.31.10.1 on interface Loopback100:10:35: %PIM-5-DRCHG: VRF VRF2: DR change from neighbor 0.0.0.0 to 172.16.1.1 on interface Tunnel100:10:35: %PIM-5-DRCHG: VRF red: DR change from neighbor 0.0.0.0 to 172.16.1.1 on interface Tunnel200:10:35: %LINK-3-UPDOWN: Interface Null0, changed state to up00:10:35: %LINK-3-UPDOWN: Interface Loopback0, changed state to up00:10:35: %LINK-3-UPDOWN: Interface Loopback1, changed state to up00:10:35: %LINK-3-UPDOWN: Interface Tunnel0, changed state to up00:10:35: %LINK-3-UPDOWN: Interface Tunnel1, changed state to up00:10:35: %LINK-3-UPDOWN: Interface Tunnel2, changed state to up00:10:35: %LINK-5-CHANGED: Interface Ethernet0/0, changed state to administratively down00:10:35: %LINK-5-CHANGED: Interface Ethernet0/1, changed state to administratively down00:10:35: %LINK-5-CHANGED: Interface Ethernet0/2, changed state to administratively down00:10:35: %LINK-5-CHANGED: Interface Ethernet0/3, changed state to administratively down00:10:35: %LINK-5-CHANGED: Interface Ethernet1/0, changed state to administratively down00:10:35: %LINK-5-CHANGED: Interface Ethernet1/1, changed state to administratively down00:10:35: %LINK-5-CHANGED: Interface Ethernet1/2, changed state to administratively down00:10:35: %LINK-5-CHANGED: Interface Ethernet1/3, changed state to administratively down00:10:36: %LINEPROTO-5-UPDOWN: Line protocol on Interface Null0, changed state to up00:10:36: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback0, changed state to up00:10:36: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback1, changed state to up00:10:36: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel0, changed state to up00:10:36: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel1, changed state to up00:10:36: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel2, changed state to up00:10:38: %PIM-5-DRCHG: VRF VRF1: DR change from neighbor 0.0.0.0 to 172.16.1.1 on interface Tunnel0IGMP Queries, DDE Replay, Termination of the NSF Hold-Off Period, and Flushing of Stale Forwarding Information
The following output is from the debug ip multicast redundancy command. After the processing of unicast and multicast route convergence, time is allowed for IGMP reporting. Following IGMP reporting, the control plane then sends out requests for the MFIB replay of data driven events (DDEs) to retrigger multicast route information that cannot be obtained from PIM neighbors or directly connected hosts. After this processing completes, the control plane waits for the NSF hold-off time period to terminate. The refreshed multicast control plane information is then downloaded to the forwarding plane; once completed, the stale multicast forwarding plane information is subsequently flushed.
*Aug 7 02:31:43.651: MCAST-HA: IGMP response timer expired. Ready for DDE replay for MVRF red*Aug 7 02:31:43.651: MCAST-HA: Sending DDE replay request for MVRF red.*Aug 7 02:31:43.651: MCAST-HA: MFIB DDE replay completed for mvrf red*Aug 7 02:31:43.651: MCAST-HA: No NSF Holdoff extension requested for mvrf red at completion of DDE replay.*Aug 7 02:31:43.651: MCAST-HA: Terminating multicast NSF holdoff for MVRF red*Aug 7 02:31:43.651: MCAST-HA: Still awaiting MFIB DDE replay for mvrf VRF2DDE replay: NOT COMPLETED, MRIB update: NOT PENDING*Aug 7 02:31:43.651: MCAST-HA: IGMP response timer expired. Ready for DDE replay for MVRF VRF2*Aug 7 02:31:43.651: MCAST-HA: Sending DDE replay request for MVRF VRF2.*Aug 7 02:31:43.651: MCAST-HA: MFIB DDE replay completed for mvrf VRF2*Aug 7 02:31:43.651: MCAST-HA: No NSF Holdoff extension requested for mvrf VRF2 at completion of DDE replay.*Aug 7 02:31:43.651: MCAST-HA: Terminating multicast NSF holdoff for MVRF VRF2*Aug 7 02:31:43.651: MCAST-HA: Still awaiting MFIB DDE replay for mvrf VRF1DDE replay: NOT COMPLETED, MRIB update: NOT PENDING*Aug 7 02:31:43.651: MCAST-HA: IGMP response timer expired. Ready for DDE replay for MVRF VRF1*Aug 7 02:31:43.651: MCAST-HA: Sending DDE replay request for MVRF VRF1.*Aug 7 02:31:43.651: MCAST-HA: MFIB DDE replay completed for mvrf VRF1*Aug 7 02:31:43.651: MCAST-HA: No NSF Holdoff extension requested for mvrf VRF1 at completion of DDE replay.*Aug 7 02:31:43.651: MCAST-HA: Terminating multicast NSF holdoff for MVRF VRF1*Aug 7 02:31:43.651: MCAST-HA: Still awaiting MFIB DDE replay for mvrf IPv4 defaultDDE replay: NOT COMPLETED, MRIB update: NOT PENDING*Aug 7 02:31:43.651: MCAST-HA: IGMP response timer expired. Ready for DDE replay for MVRF IPv4 default*Aug 7 02:31:43.651: MCAST-HA: Sending DDE replay request for MVRF IPv4 default.*Aug 7 02:31:43.651: MCAST-HA: MFIB DDE replay completed for mvrf IPv4 default*Aug 7 02:31:43.651: MCAST-HA: No NSF Holdoff extension requested for mvrf IPv4 default at completion of DDE replay.*Aug 7 02:31:43.651: MCAST-HA: Terminating multicast NSF holdoff for MVRF IPv4 default*Aug 7 02:31:43.651: MCAST-HA: MFIB DDE replay completed for all MVRFs.*Aug 7 02:31:43.651: MCAST-HA: Stopping the MFIB DDE replay failsafe timer.*Aug 7 02:32:13.651: MCAST-HA: Flush timer expired. Starting final RPF check for MVRF IPv4 default*Aug 7 02:32:13.651: MCAST-HA: Flush timer expired. Starting final RPF check for MVRF VRF1*Aug 7 02:32:13.651: MCAST-HA: Flush timer expired. Starting final RPF check for MVRF VRF2*Aug 7 02:32:13.651: MCAST-HA: Flush timer expired. Starting final RPF check for MVRF red*Aug 7 02:32:14.151: MCAST-HA: Flushing stale mcast state. RP failover processing complete for MVRF IPv4 default.*Aug 7 02:32:14.151: MCAST-HA: Flushing stale mcast state. RP failover processing complete for MVRF VRF1.*Aug 7 02:32:14.151: MCAST-HA: Flushing stale mcast state. RP failover processing complete for MVRF VRF2.*Aug 7 02:32:14.151: MCAST-HA: Flushing stale mcast state. RP failover processing complete for MVRF red.*Aug 7 02:32:14.151: MCAST-HA: RP failover processing complete for all MVRFs.Standby RP Bringup
The following is sample output from the debug ip multicast redundancy command. This output shows events related to the reloading of the standby RP; in particular, events related to ISSU negotiation between the active and standby RP and events related to the synchronization of dynamic multicast forwarding information from the active RP to the standby RP. Synchronization events are also logged in steady state for events that occur which affect dynamic group-to-RP mapping information or dynamic tunnel state.
00:11:50: %HA-6-MODE: Operating RP redundancy mode is SSO*Aug 7 02:32:45.435: MCAST-HA-RF: Status event: status=RF_STATUS_OPER_REDUNDANCY_MODE_CHANGE Op=7 RFState=ACTIVE*Aug 7 02:32:45.435: MCAST-HA-RF: Status event: status=RF_STATUS_REDUNDANCY_MODE_CHANGE Op=7 RFState=ACTIVE*Aug 7 02:32:45.435: MCAST-HA-RF: Status event: status=RF_STATUS_PEER_PRESENCE Op=1 RFState=ACTIVE*Aug 7 02:32:45.463: MCAST-HA-RF: Status event: status=RF_STATUS_PEER_COMM Op=1 RFState=ACTIVE*Aug 7 02:32:45.563: MCAST-HA-RF: Progression event: RF_Event=RF_PROG_ISSU_NEGOTIATION RFState=ACTIVE*Aug 7 02:32:46.039: MCAST-HA-RF: Progression event: RF_Event=RF_PROG_PLATFORM_SYNC RFState=ACTIVE*Aug 7 02:32:46.979: MCAST-HA: Received cf status CHKPT_STATUS_PEER_READY*Aug 7 02:32:46.979: MCAST-ISSU Handling communication up transition for PIM HA transport type 0, RF comm = TRUE, renegotiation NOT PENDING*Aug 7 02:32:46.979: MCAST-HA: Received cf status CHKPT_STATUS_IPC_FLOW_ON*Aug 7 02:32:47.043: MCAST-HA-RF: Progression event: RF_Event=RF_PROG_STANDBY_ISSU_NEGOTIATION_LATE RFState=ACTIVE*Aug 7 02:32:50.943: MCAST-HA-RF: Progression event: RF_Event=RF_PROG_STANDBY_CONFIG RFState=ACTIVE*Aug 7 02:32:50.947: MCAST-ISSU Negotiation message sent from primary, rc = 0*Aug 7 02:32:50.947: MCAST-HA-RF: Started PIM ISSU negotiation on the primary RP.*Aug 7 02:32:50.947: MCAST-ISSU Negotiation message sent from primary, rc = 0*Aug 7 02:32:50.947: MCAST-ISSU Negotiation message sent from primary, rc = 0*Aug 7 02:32:50.951: MCAST-ISSU Negotiation message sent from primary, rc = 0*Aug 7 02:32:50.951: MCAST-ISSU Negotiation message sent from primary, rc = 0*Aug 7 02:32:50.951: MCAST-ISSU Negotiation message sent from primary, rc = 0*Aug 7 02:32:50.951: MCAST-ISSU Negotiation message sent from primary, rc = 0*Aug 7 02:32:50.955: MCAST-ISSU Negotiation message sent from primary, rc = 0*Aug 7 02:32:50.955: MCAST-ISSU Negotiation message sent from primary, rc = 0*Aug 7 02:32:50.955: MCAST-ISSU Negotiation message sent from primary, rc = 0*Aug 7 02:32:50.955: MCAST-ISSU Negotiation message sent from primary, rc = 0*Aug 7 02:32:50.959: MCAST-ISSU Negotiation message sent from primary, rc = 0*Aug 7 02:32:50.959: MCAST-ISSU Negotiation message sent from primary, rc = 0*Aug 7 02:32:50.959: MCAST-ISSU Negotiation message sent from primary, rc = 0*Aug 7 02:32:50.959: MCAST-ISSU Negotiation message sent from primary, rc = 0*Aug 7 02:32:50.959: MCAST-ISSU Negotiation message sent from primary, rc = 0*Aug 7 02:32:50.963: MCAST-ISSU Negotiation message sent from primary, rc = 0*Aug 7 02:32:50.963: MCAST-ISSU Negotiation message sent from primary, rc = 0*Aug 7 02:32:50.963: MCAST-ISSU Negotiation message sent from primary, rc = 0*Aug 7 02:32:50.963: MCAST-ISSU Negotiation message sent from primary, rc = 0*Aug 7 02:32:50.967: MCAST-ISSU Negotiation message sent from primary, rc = 0*Aug 7 02:32:50.967: MCAST-ISSU Negotiation message sent from primary, rc = 0*Aug 7 02:32:50.967: MCAST-ISSU Negotiation message sent from primary, rc = 0*Aug 7 02:32:50.967: MCAST-ISSU Negotiation message sent from primary, rc = 0*Aug 7 02:32:50.967: MCAST-ISSU Negotiation message sent from primary, rc = 0*Aug 7 02:32:50.971: MCAST-ISSU Negotiation message sent from primary, rc = 0*Aug 7 02:32:50.971: MCAST-ISSU Negotiation message sent from primary, rc = 0*Aug 7 02:32:50.971: MCAST-ISSU Negotiation completed for PIM Checkpoint Facility client, negotation rc = 4, negotiation result = COMPATIBLE*Aug 7 02:32:59.927: MCAST-HA-RF: Progression event: RF_Event=RF_PROG_STANDBY_FILESYS RFState=ACTIVE*Aug 7 02:32:59.963: MCAST-HA-RF: Progression event: RF_Event=RF_PROG_STANDBY_BULK RFState=ACTIVE*Aug 7 02:32:59.963: MCAST-HA-RF: Starting Bulk Sync.*Aug 7 02:32:59.963: MCAST-HA: Successfully created the bulk sync process*Aug 7 02:32:59.963: MCAST-HA: Starting Bulk sync*Aug 7 02:32:59.963: MCAST HA Executing RP mapping bulk sync.*Aug 7 02:32:59.963: MCAST HA Executing Bidir RP route bulk sync.*Aug 7 02:32:59.963: MCAST HA Executing BSR cache bulk sync.*Aug 7 02:32:59.963: MCAST-HA BSR cache sync request received for mvrf IPv4 default*Aug 7 02:32:59.963: MCAST-HA: Creating Bootstrap cache sync request chunk size=112 max=585 align=8*Aug 7 02:32:59.963: MCAST-HA: Allocating Bootstrap cache sync request sync request*Aug 7 02:32:59.963: MCAST-HA Formatting BSR cache sync message:search for mvrf IPv4 default result is 0 mvrf at 0x4A21680*Aug 7 02:32:59.971: MCAST-HA BSR cache sync request received for mvrf VRF1*Aug 7 02:32:59.971: MCAST-HA: Allocating Bootstrap cache sync request sync request*Aug 7 02:32:59.971: MCAST-HA Formatting BSR cache sync message:search for mvrf VRF1 result is 0 mvrf at 0x50EE660*Aug 7 02:32:59.983: MCAST-HA BSR cache sync request received for mvrf VRF2*Aug 7 02:32:59.983: MCAST-HA: Allocating Bootstrap cache sync request sync request*Aug 7 02:32:59.983: MCAST-HA Formatting BSR cache sync message:search for mvrf VRF2 result is 0 mvrf at 0x5103300*Aug 7 02:32:59.991: MCAST-HA BSR cache sync request received for mvrf red*Aug 7 02:32:59.991: MCAST-HA: Allocating Bootstrap cache sync request sync request*Aug 7 02:32:59.991: MCAST-HA Formatting BSR cache sync message:search for mvrf red result is 0 mvrf at 0x5135FE0*Aug 7 02:33:00.003: MCAST HA Executing AutoRP discovery IDB bulk sync.*Aug 7 02:33:00.003: MCAST-HA AutoRP discovery IDB sync request received formvrf IPv4 default*Aug 7 02:33:00.003: MCAST-HA: Creating Autorp discovery IDB sync request chunk size=112 max=585 align=8*Aug 7 02:33:00.003: MCAST-HA: Allocating Autorp discovery IDB sync request sync request*Aug 7 02:33:00.003: MCAST-HA Formatting AutoRP discovery IDB sync message:search for mvrf IPv4 default result is 0 mvrf at 0x4A21680*Aug 7 02:33:00.011: MCAST-HA AutoRP discovery IDB sync request received formvrf VRF1*Aug 7 02:33:00.011: MCAST-HA: Allocating Autorp discovery IDB sync request sync request*Aug 7 02:33:00.011: MCAST-HA Formatting AutoRP discovery IDB sync message:search for mvrf VRF1 result is 0 mvrf at 0x50EE660*Aug 7 02:33:00.023: MCAST-HA AutoRP discovery IDB sync request received formvrf VRF2*Aug 7 02:33:00.023: MCAST-HA: Allocating Autorp discovery IDB sync request sync request*Aug 7 02:33:00.023: MCAST-HA Formatting AutoRP discovery IDB sync message:search for mvrf VRF2 result is 0 mvrf at 0x5103300*Aug 7 02:33:00.031: MCAST-HA AutoRP discovery IDB sync request received formvrf VRF3*Aug 7 02:33:00.031: MCAST-HA: Allocating Autorp discovery IDB sync request sync request*Aug 7 02:33:00.031: MCAST-HA Formatting AutoRP discovery IDB sync message:search for mvrf VRF3 result is 0 mvrf at 0x5135FE0*Aug 7 02:33:00.043: MCAST HA Executing dummy bulk sync function.*Aug 7 02:33:00.043: MCAST HA Executing dummy bulk sync function.*Aug 7 02:33:00.043: MCAST HA Executing dummy bulk sync function.*Aug 7 02:33:00.043: MCAST HA Executing MDT tunnel bulk sync.*Aug 7 02:33:00.043: MCAST-HA MDT tunnel sync request received for mvrf VRF1*Aug 7 02:33:00.043: MCAST-HA: Creating MDT tunnel sync request chunk size=112 max=585 align=8*Aug 7 02:33:00.043: MCAST-HA: Allocating MDT tunnel sync request sync request*Aug 7 02:33:00.043: MCAST-HA Formatting MDT tunnel sync message:search for mvrf VRF1 result is 0 mvrf at 0x50EE660*Aug 7 02:33:00.051: MCAST-HA MDT tunnel sync request received for mvrf VRF2*Aug 7 02:33:00.051: MCAST-HA: Allocating MDT tunnel sync request sync request*Aug 7 02:33:00.051: MCAST-HA Formatting MDT tunnel sync message:search for mvrf VRF2 result is 0 mvrf at 0x5103300*Aug 7 02:33:00.063: MCAST-HA MDT tunnel sync request received for mvrf VRF3*Aug 7 02:33:00.063: MCAST-HA: Allocating MDT tunnel sync request sync request*Aug 7 02:33:00.063: MCAST-HA Formatting MDT tunnel sync message:search for mvrf VRF3 result is 0 mvrf at 0x5135FE0*Aug 7 02:33:00.071: MCAST HA Executing Bidir RP DF bulk sync.*Aug 7 02:33:00.071: MCAST HA Executing register tunnel bulk sync.*Aug 7 02:33:00.071: MCAST-HA: Completed enqueuing of bulk sync messages.*Aug 7 02:33:00.071: MCAST-HA: Bulk sync message queue has drained.*Aug 7 02:33:00.071: MCAST-HA: Received acknowledgement from standby for all bulk sync messages.*Aug 7 02:33:00.071: MCAST-HA Creating bulk sync completion message for peer.*Aug 7 02:33:00.071: MCAST-HA: Primary has notified standby of bulk sync completion. Waiting for final bulk sync ACK from stby.*Aug 7 02:33:00.075: MCAST-HA: Received cf status CHKPT_STATUS_SEND_OK*Aug 7 02:33:00.075: MCAST-HA: Sent message type is 2*Aug 7 02:33:00.075: MCAST-HA Searching for sync request corresponding to the successfully received message.*Aug 7 02:33:00.075: MCAST-HA Transmission from primary and reception by standby confirmed for sync type 2. Cleanup is complete.*Aug 7 02:33:00.075: MCAST-HA: Received cf status CHKPT_STATUS_SEND_OK*Aug 7 02:33:00.075: MCAST-HA: Sent message type is 2*Aug 7 02:33:00.075: MCAST-HA Searching for sync request corresponding to the successfully received message.*Aug 7 02:33:00.075: MCAST-HA Transmission from primary and reception by standby confirmed for sync type 2. Cleanup is complete.*Aug 7 02:33:00.075: MCAST-HA: Received cf status CHKPT_STATUS_SEND_OK*Aug 7 02:33:00.075: MCAST-HA: Sent message type is 2*Aug 7 02:33:00.075: MCAST-HA Searching for sync request corresponding to the successfully received message.*Aug 7 02:33:00.075: MCAST-HA Transmission from primary and reception by standby confirmed for sync type 2. Cleanup is complete.*Aug 7 02:33:00.087: MCAST-HA: Received cf status CHKPT_STATUS_SEND_OK*Aug 7 02:33:00.087: MCAST-HA: Sent message type is 2*Aug 7 02:33:00.087: MCAST-HA Searching for sync request corresponding to the successfully received message.*Aug 7 02:33:00.087: MCAST-HA Transmission from primary and reception by standby confirmed for sync type 2. Cleanup is complete.*Aug 7 02:33:00.087: MCAST-HA: Received cf status CHKPT_STATUS_SEND_OK*Aug 7 02:33:00.087: MCAST-HA: Sent message type is 3*Aug 7 02:33:00.087: MCAST-HA Searching for sync request corresponding to the successfully received message.*Aug 7 02:33:00.087: MCAST-HA Transmission from primary and reception by standby confirmed for sync type 3. Cleanup is complete.*Aug 7 02:33:00.087: MCAST-HA: Received cf status CHKPT_STATUS_SEND_OK*Aug 7 02:33:00.087: MCAST-HA: Sent message type is 3*Aug 7 02:33:00.087: MCAST-HA Searching for sync request corresponding to the successfully received message.*Aug 7 02:33:00.087: MCAST-HA Transmission from primary and reception by standby confirmed for sync type 3. Cleanup is complete.*Aug 7 02:33:00.087: MCAST-HA: Received cf status CHKPT_STATUS_SEND_OK*Aug 7 02:33:00.087: MCAST-HA: Sent message type is 3*Aug 7 02:33:00.087: MCAST-HA Searching for sync request corresponding to the successfully received message.*Aug 7 02:33:00.087: MCAST-HA Transmission from primary and reception by standby confirmed for sync type 3. Cleanup is complete.*Aug 7 02:33:00.087: MCAST-HA: Received cf status CHKPT_STATUS_SEND_OK*Aug 7 02:33:00.087: MCAST-HA: Sent message type is 3*Aug 7 02:33:00.087: MCAST-HA Searching for sync request corresponding to the successfully received message.*Aug 7 02:33:00.087: MCAST-HA Transmission from primary and reception by standby confirmed for sync type 3. Cleanup is complete.*Aug 7 02:33:00.087: MCAST-HA: Received cf status CHKPT_STATUS_SEND_OK*Aug 7 02:33:00.087: MCAST-HA: Sent message type is 8*Aug 7 02:33:00.087: MCAST-HA Searching for sync request corresponding to the successfully received message.*Aug 7 02:33:00.087: MCAST-HA Transmission from primary and reception by standby confirmed for sync type 8. Cleanup is complete.*Aug 7 02:33:00.087: MCAST-HA: Received cf status CHKPT_STATUS_SEND_OK*Aug 7 02:33:00.087: MCAST-HA: Sent message type is 8*Aug 7 02:33:00.087: MCAST-HA Searching for sync request corresponding to the successfully received message.*Aug 7 02:33:00.087: MCAST-HA Transmission from primary and reception by standby confirmed for sync type 8. Cleanup is complete.*Aug 7 02:33:00.087: MCAST-HA: Received cf status CHKPT_STATUS_SEND_OK*Aug 7 02:33:00.087: MCAST-HA: Sent message type is 8*Aug 7 02:33:00.087: MCAST-HA Searching for sync request corresponding to the successfully received message.*Aug 7 02:33:00.087: MCAST-HA Transmission from primary and reception by standby confirmed for sync type 8. Cleanup is complete.*Aug 7 02:33:00.087: MCAST-HA: Received cf status CHKPT_STATUS_SEND_OK*Aug 7 02:33:00.087: MCAST-HA: Sent message type is 11*Aug 7 02:33:00.087: MCAST-HA Process: Primary RP received standby ACK for reception of bulk sync completion message.*Aug 7 02:33:00.087: MCAST-HA Notifying RF to continue progression.*Aug 7 02:33:00.087: MCAST-HA: Wakeup received for bulk sync completion.major = 4, minor = 2.*Aug 7 02:33:00.091: MCAST-HA Process: Primary RP received bulk sync completion confirmation from standby.*Aug 7 02:33:00.091: MCAST-HA RF notification previously sent.*Aug 7 02:33:00.455: MCAST-HA-RF: Progression event: RF_Event=RF_PROG_STANDBY_HOT RFState=ACTIVE00:12:05: %HA_CONFIG_SYNC-6-BULK_CFGSYNC_SUCCEED: Bulk Sync succeeded00:12:05: %HA-6-STANDBY_READY: Standby RP in slot 7 is operational in SSO mode00:12:05: %RF-5-RF_TERMINAL_STATE: Terminal state reached for (SSO)Example: Monitoring the Transition from Standby RP to Active RP Following a Switchover
The following example shows how to monitor the transition from standby RP to active RP and confirm the IP multicast redundancy state and the status on the standby RP after it has resynchronized with the new active RP.
Note In this example scenario, a router is configured for IPv4 multicast routing operation, but not for IPv6 multicast. As a result, some of the output fields that are specific to IPv6 multicast will indicate status such as "Not enabled" or "Idle" in the example outputs.
Initial State on Standby RP Before Switchover
The following output is from the show ip multicast redundancy state command on a standby RP before an active RP goes down. In the sample output, notice that the "Current sync state" field displays "Not synching," indicating that the standby RP is not synchronizing data to the active RP. The standby RP serves only as a passive recipient of synchronization updates and does not initiate synchronization updates to the active RP.
Router_Standby# show ip multicast redundancy state
Multicast IPv4 Redundancy Mode: Not enabledMulticast IPv6 Redundancy Mode: Not enabledSync message epoch: 0Sync message sequence number: 24Stale NSF state flush timeout: 30000 msCurrent sync state: Not synchingMulticast ISSU Client Status:PIM MIC client No ISSU result reportedMRIB MIC client Unregistered - ignoredMFIB IPv4 MIC client Unregistered - ignoredMFIB IPv6 MIC client Unregistered - ignoredPLATFORM IPv4 MIC client Unregistered - ignoredPLATFORM IPv6 MIC client Unregistered - ignoredIPv4 SSO blocked by: PIMIPv6 SSO blocked by: PIMThe following output is unconditionally logged by the Cisco IOS Redundancy Facility (RF) software when the standby RP detects that it has become the active RP due to a failure of the original active RP. The output shows the message used to indicate that an RP switchover has occurred:
00:00:49: %REDUNDANCY-3-SWITCHOVER: RP switchover (PEER_DOWN_INTERRUPT)Standby RP Transition to Active RP After an RP Switchover
The following output is from the show ip multicast redundancy state command on the standby RP during its transition from standby RP to active RP. Notice that the "Multicast IPv4 HA state machine status" field displays "Unicast converging," indicating that unicast convergence on the new active RP has begun. At this point in the RP switchover, the standby RP is waiting for unicast convergence.
Router-Standby# show ip multicast redundancy state
Multicast IPv4 Redundancy Mode: Not enabledMulticast IPv6 Redundancy Mode: Not enabledMulticast IPv4 HA state machine status: Unicast convergingMulticast IPv6 HA state machine status: IdleSync message epoch: 0Sync message sequence number: 24Stale NSF state flush timeout: 30000 msCurrent sync state: Not synchingMulticast ISSU Client Status:PIM MIC client No ISSU result reportedMRIB MIC client No ISSU result reportedMFIB IPv4 MIC client No ISSU result reportedMFIB IPv6 MIC client No ISSU result reportedPLATFORM IPv4 MIC client Unregistered - ignoredPLATFORM IPv6 MIC client Unregistered - ignoredIPv4 SSO blocked by: PIM, MRIB, MFIBV4IPv6 SSO blocked by: PIM, MRIB, MFIBV6The following output from the debug ip multicast redundancy state command shows messages indicating that the interfaces on the new active RP are coming up. As interfaces come up on the new active RP, unicast convergence occurs in parallel with multicast route refresh from PIM neighbors. Unicast convergence is followed by RPF adjustments to the refreshed mroute information.
00:00:51: %LINK-3-UPDOWN: Interface Null0, changed state to up00:00:51: %LINK-3-UPDOWN: Interface Loopback0, changed state to up00:00:51: %LINK-3-UPDOWN: Interface Loopback1, changed state to up00:00:51: %LINK-3-UPDOWN: Interface Tunnel0, changed state to up00:00:51: %LINK-3-UPDOWN: Interface Tunnel1, changed state to up00:00:51: %LINK-3-UPDOWN: Interface Tunnel2, changed state to up00:00:51: %LINK-3-UPDOWN: Interface Tunnel3, changed state to up00:00:51: %LINK-3-UPDOWN: Interface Tunnel4, changed state to up00:00:51: %LINK-3-UPDOWN: Interface Tunnel5, changed state to up00:00:51: %LINK-3-UPDOWN: Interface Tunnel6, changed state to up00:00:51: %LINK-3-UPDOWN: Interface Tunnel7, changed state to up00:00:51: %LINK-3-UPDOWN: Interface Tunnel8, changed state to up00:00:51: %LINK-3-UPDOWN: Interface Tunnel9, changed state to up00:00:51: %LINK-3-UPDOWN: Interface Tunnel10, changed state to up00:00:51: %LINK-3-UPDOWN: Interface Tunnel11, changed state to up00:00:51: %LINK-3-UPDOWN: Interface Tunnel12, changed state to up00:00:51: %LINK-3-UPDOWN: Interface Tunnel13, changed state to up00:00:51: %LINK-3-UPDOWN: Interface Tunnel14, changed state to up00:00:51: %LINK-3-UPDOWN: Interface Tunnel15, changed state to up00:00:51: %LINK-5-CHANGED: Interface Ethernet0/0, changed state to administratively down00:00:51: %LINK-5-CHANGED: Interface Ethernet0/1, changed state to administratively down00:00:51: %LINK-5-CHANGED: Interface Ethernet0/2, changed state to administratively down00:00:51: %LINK-5-CHANGED: Interface Ethernet0/3, changed state to administratively down00:00:51: %LINK-5-CHANGED: Interface Ethernet1/0, changed state to administratively down00:00:51: %LINK-5-CHANGED: Interface Ethernet1/1, changed state to administratively down00:00:51: %LINK-5-CHANGED: Interface Ethernet1/2, changed state to administratively down00:00:51: %LINK-5-CHANGED: Interface Ethernet1/3, changed state to administratively down00:00:51: %LINK-5-CHANGED: Interface Serial2/0, changed state to administratively down00:00:51: %LINK-5-CHANGED: Interface Serial2/1, changed state to administratively downThe following is output from the show ip multicast redundancy state command during the transition from the standby RP to the new active RP. Notice that the "Multicast IPv4 HA state machine status" displays "DDE replaying," indicating that the MFIB is replaying DDEs. After the processing of unicast and multicast route convergence, time is allowed for IGMP reporting. Following IGMP reporting, the control plane then sends out requests for the MFIB replay of DDEs to retrigger multicast route information that cannot be obtained from PIM neighbors or directly connected hosts.
Router# show ip multicast redundancy state
Multicast IPv4 Redundancy Mode: Not enabledMulticast IPv6 Redundancy Mode: Not enabledMulticast IPv4 HA state machine status: DDE replayingMulticast IPv6 HA state machine status: IdleSync message epoch: 0Sync message sequence number: 24Stale NSF state flush timeout: 30000 msCurrent sync state: Not synchingMulticast ISSU Client Status:PIM MIC client No ISSU result reportedMRIB MIC client No ISSU result reportedMFIB IPv4 MIC client No ISSU result reportedMFIB IPv6 MIC client No ISSU result reportedPLATFORM IPv4 MIC client Unregistered - ignoredPLATFORM IPv6 MIC client Unregistered - ignoredIPv4 SSO blocked by: PIM, MRIB, MFIBV4IPv6 SSO blocked by: PIM, MRIB, MFIBV6After this processing completes, the control plane terminates the NSF hold-off or, if the platform multicast driver software requests an extension to the hold-off period, allows additional time for the platform multicast driver software to release the NSF hold-off extension.
The refreshed multicast control plane information is then downloaded to the forwarding plane. Although reconvergence is considered complete at this point, additional "refresh" updates may occur after this point in time. An additional time interval is provided for any remaining updates before stale multicast forwarding plane information is subsequently flushed.
The following is output from the show ip multicast redundancy state command. Notice that the "Multicast IPv4 HA state machine status" field displays, "Flush pending," indicating that stale NSF data plane state is still being temporarily retained to allow for any additional refreshed multicast control plane information to be downloaded to the forwarding plane.
Router# show ip multicast redundancy state
Multicast IPv4 Redundancy Mode: Not enabledMulticast IPv6 Redundancy Mode: Not enabledMulticast IPv4 HA state machine status: Flush pendingMulticast IPv6 HA state machine status: IdleSync message epoch: 0Sync message sequence number: 24Stale NSF state flush timeout: 30000 msCurrent sync state: Not synchingMulticast ISSU Client Status:PIM MIC client No ISSU result reportedMRIB MIC client No ISSU result reportedMFIB IPv4 MIC client No ISSU result reportedMFIB IPv6 MIC client No ISSU result reportedPLATFORM IPv4 MIC client Unregistered - ignoredPLATFORM IPv6 MIC client Unregistered - ignoredIPv4 SSO blocked by: PIM, MRIB, MFIBV4IPv6 SSO blocked by: PIM, MRIB, MFIBV6The following is output from the show ip multicast redundancy state command after the refreshed multicast control plane information has been downloaded to the forwarding plane and the stale multicast forwarding plane information has been flushed. Notice that at this stage in the RP switchover the "Multicast IPv4 HA state machine status" field displays "Idle" because multicast IPv4 HA state machine operations have completed.
Router# show ip multicast redundancy state
Multicast IPv4 Redundancy Mode: Not enabledMulticast IPv6 Redundancy Mode: Not enabledMulticast IPv4 HA state machine status: IdleMulticast IPv6 HA state machine status: IdleSync message epoch: 0Sync message sequence number: 24Stale NSF state flush timeout: 30000 msCurrent sync state: Not synchingMulticast ISSU Client Status:PIM MIC client No ISSU result reportedMRIB MIC client No ISSU result reportedMFIB IPv4 MIC client No ISSU result reportedMFIB IPv6 MIC client No ISSU result reportedPLATFORM IPv4 MIC client Unregistered - ignoredPLATFORM IPv6 MIC client Unregistered - ignoredIPv4 SSO blocked by: PIM, MRIB, MFIBV4IPv6 SSO blocked by: PIM, MRIB, MFIBV6Standby RP Resynchronization
The following is sample output from the debug ip multicast redundancy command. The output shows the messages used to indicate that a standby RP has been resynchronized.
00:25:42: %HA-6-MODE: Operating RP redundancy mode is SSO00:26:04: %HA_CONFIG_SYNC-6-BULK_CFGSYNC_SUCCEED: Bulk Sync succeeded00:26:04: %HA-6-STANDBY_READY: Standby RP in slot 7 is operational in SSO mode00:26:04: %RF-5-RF_TERMINAL_STATE: Terminal state reached for (SSO).00:15:28: %RF-5-RF_TERMINAL_STATE: Terminal state reached for (SSO)The following is output from the show ip multicast redundancy state command after the standby RP has completed resynchronization with the new active RP. Notice that the "Multicast IPv4 Redundancy Mode" field displays "SSO," indicating that all information between the standby RP and active RP has been synchronized. Also, notice that the "Current sync state" field displays "Synched," indicating that the standby has resynchronized with the new active RP.
Router# show ip multicast redundancy state
Multicast IPv4 Redundancy Mode: SSOMulticast IPv6 Redundancy Mode: Not enabledMulticast IPv4 HA state machine status: IdleMulticast IPv6 HA state machine status: IdleSync message epoch: 0Sync message sequence number: 24Stale NSF state flush timeout: 30000 msCurrent sync state: SynchedMulticast ISSU Client Status:PIM MIC client ISSU compatibleMRIB MIC client ISSU compatibleMFIB IPv4 MIC client ISSU compatibleMFIB IPv6 MIC client No ISSU result reportedPLATFORM IPv4 MIC client Unregistered - ignoredPLATFORM IPv6 MIC client Unregistered - ignoredIPv4 SSO supported for: PIM, MRIB, MFIBV4IPv6 SSO blocked by: MFIBV6Additional References
Related Documents
Related Topic Document TitleOverview of the IP multicast technology area
"IP Multicast Technology Overview" module
Concepts, tasks, and examples for configuring an IP multicast network using PIM
ISSU concepts, tasks, and examples
NSF concepts, tasks, and examples
"Cisco Nonstop Forwarding" module
Stateful switchover concepts, tasks, and examples
"Stateful Switchover" module
Multicast commands: complete command syntax, command mode, command history, defaults, usage guidelines, and examples
Standards
Standard TitleNo new or modified standards are supported by this feature, and support for existing standards has not been modified by this feature.
—
MIBs
RFCs
RFC TitleRFC 4601
Protocol Independent Multicast - Sparse Mode (PIM-SM): Protocol Specification (Revised)
Technical Assistance
Feature Information for Monitoring and Maintaining Multicast HA Operations (NSF/SSO and ISSU)
Table 1 lists the features in this module and provides links to specific configuration information.
Use Cisco Feature Navigator to find information about platform support and software image support. Cisco Feature Navigator enables you to determine which software images support a specific software release, feature set, or platform. To access Cisco Feature Navigator, go to http://www.cisco.com/go/cfn. An account on Cisco.com is not required.
Note Table 1 lists only the software release that introduced support for a given feature in a given software release train. Unless noted otherwise, subsequent releases of that software release train also support that feature.
Cisco and the Cisco Logo are trademarks of Cisco Systems, Inc. and/or its affiliates in the U.S. and other countries. A listing of Cisco's trademarks can be found at www.cisco.com/go/trademarks. Third party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1005R)
Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional and coincidental.
© 2009-2010 Cisco Systems, Inc. All rights reserved.