CPwE Scalable Time Distribution Overview
The prevailing trend in Industrial Automation and Control System (IACS) networking is the convergence of technology, specifically IACS operational technology (OT) with information technology (IT). Converged Plantwide Ethernet (CPwE) helps to enable IACS network and security technology and OT-IT persona convergence through the use of standard Ethernet, Internet Protocol (IP), network services, security services, time synchronization technologies, and EtherNet/IP. A real-time converged plant-wide IACS architecture helps to enable the Industrial Internet of Things (IIoT).
Business practices, corporate standards, policies, industry standards, and tolerance to risk are key factors in determining the degree of time synchronization required within a plant-wide IACS architecture. IACS networks differ from their IT counterparts in their need to support significantly lower latency (time delay between message sent and message received) and jitter (the variance of the latency) to help enable real-time IACS communications. Time synchronization helps to support time-critical data for the most demanding IACS applications.
Successful deployment of IIoT IACS applications within CPwE Architectures (Figure 1-1) depends on a network infrastructure design that addresses IACS application performance requirements. The content of CPwE, which is relevant to both OT and IT disciplines, consists of documented architectures and key tenets from OT and IT to help achieve real-time communications to support IIoT IACS applications. CPwE key tenets include:
- Smart IIoT Devices—controllers, I/O, drives, instrumentation, actuators and analytics
- Zoning (segmentation)—smaller connected LANs, functional areas and security groups
- Managed Infrastructure—managed industrial Ethernet switches (IES) and industrial firewalls
- Resiliency—robust physical layer and resilient or redundant topologies with resiliency protocols
- Time-critical Data—data prioritization and time synchronization via CIP Sync and IEEE-1588 Precision Time Protocol (PTP)
- Wireless—unified wireless LAN (WLAN) to enable mobility for personnel and equipment
- Holistic Defense-in-Depth Security—multiple layers of diverse technologies for threat detection and prevention, implemented by different persona (e.g., OT and IT) and applied at different levels of the plant-wide IACS architecture
- Convergence-ready—seamless plant-wide integration by trusted partner applications
The ODVA, Inc. CIP Sync technology uses the Common Industrial Protocol (CIP) application layer protocol and the IEEE 1588-2008 Precision Time Protocol (PTP) standard for time synchronization. CIP Sync and IEEE 1588-2008 are designed for local and plant-wide IACS applications requiring very high accuracies beyond those attainable with Network Time Protocol (NTP).
This Deploying Scalable Time Distribution within a Converged Plantwide Ethernet Architecture Design Guide (CPwE Time) outlines several use cases for designing and deploying IEEE 1588 PTP and CIP Sync technology throughout a plant-wide IACS network infrastructure. CPwE Time was architected, tested and verified by Cisco Systems and Rockwell Automation with assistance by Panduit.
CPwE Time Solution Use Cases
CPwE is the underlying architecture that provides standard network and security services for control and information disciplines, devices and equipment found in modern IACS applications. The CPwE architectures (Figure 1-1) were architected, tested and validated to provide design and implementation guidance, test results and documented configuration settings. This can help to achieve the real-time communication, reliability, scalability, security and resiliency requirements of modern IACS applications.
An IACS is deployed in a wide variety of discrete and process manufacturing industries such as automotive, pharmaceuticals, consumer packaged goods, pulp and paper, oil and gas, water/wastewater, mining and energy. IACS applications are composed of multiple control and information disciplines such as continuous process, batch and discrete, and hybrid combinations. One of the challenges facing industrial operations is the hardening of standard Ethernet and IP-converged IACS networking technologies to take advantage of the business benefits associated with IIoT.
This Deploying Scalable Time Distribution within a Converged Plantwide Ethernet Architecture Design Guide outlines the concepts, requirements and technology solutions for reference designs developed around a specific set of priority use cases. These use cases were architected and tested for solution functional verification with limited scale by Cisco Systems and Rockwell Automation with assistance by Panduit to help support time synchronization within a converged plant-wide EtherNet/IP IACS architecture.
The CPwE Time Design Guide includes:
–
IEEE 1588 Precision Time Protocol (PTP)
– Distributed motion (not included in CPwE Time)
– Design, configuration and diagnostic considerations for plant-wide (Levels 0-3) IEEE 1588 PTP and CIP Sync deployments
– Limited resiliency and reliability PoC testing
– Layer 2 IES—Allen-Bradley® Stratix® 5700/5400
– Layer 3 IES—Allen-Bradley Stratix 5410
CPwE Time Architecture Overview
ODVA, Inc. CIP Sync technology for time synchronization is used across a broad range of IACS applications to synchronize control system clocks (Figure 1-2) and helps enable applications such as event sequencing and logging. For example:
- A sequence of events or first fault system can use timestamps to determine the order in which faults occurred in the system. This allows for the tracking of faults to establish the first in a chain of faults. These types of applications use dedicated alarm instructions in the Programmable Automation Controller (PAC) to record events or time stamping inputs in order to log the change of state for a point.
- An in-PAC chassis historian module logs historical data at Level 1 (skid/machine).
- High-speed applications can use timestamps to process inputs and outputs asynchronously from the control loop. For example, an application can use time-synchronized inputs and outputs to trigger a diverter without the application scan time matching the part cycle time.
CIP Sync uses IEEE 1588 Precision Time Protocol (PTP) to synchronize clocks in the control system. In the PTP architecture, all clocks are synchronized to a single grandmaster clock. In turn, this clock must be synchronized to Coordinated Universal Time (UTC) to represent the time of day in the system.
Several approaches exist for setting time in the grandmaster clock that allow customers to meet different application requirements. For example, applications that only require rough correlation to UTC can use a handset grandmaster clock. In this case, the administrator simply sets the time in the grandmaster clock based on the time that is currently shown on another device such as their PC or smartphone. All clocks drift over time and need to be adjusted to accurately reflect UTC. Handset clocks do not have any inherent mechanism to compensate for drift. As such, the administrator will need to re-adjust the grandmaster time manually to ensure alignment with UTC time. This readjustment will need to be done on a regular basis to help prevent the two clocks from drifting too far apart.
Applications that require tight correlation to UTC can use a grandmaster clock with a built-in Global Navigation Satellite System (GNSS) receiver such as a Layer 3 IES or a dedicated PAC module (e.g., Allen-Bradley 1756-TIME module). These type of grandmaster clocks synchronize to the atomic clocks in the navigation satellites and automatically adjust their time to match UTC. However, the installation of these systems is complex and requires an antenna with an unobstructed view of the sky and low loss coaxial cable to connect to the receiver.
A final approach is the NTP-PTP flywheel feature available in some IES. This is a hybrid approach that uses Network Time Protocol (NTP) to synchronize and regulate the grandmaster clock to UTC. Correlation to UTC will not be as good as with a GNSS receiver; however, the flywheel does compensate for drift.
Figure 1-2 CPwE Time Architecture
In the CPwE Time solution (Figure 1-2), PTP time does not pass through the core and the PTP domain is bounded to the Cell/Area Zones connected to a single Layer 3 IES distribution pair. Within these bounds, the grandmaster clock will function as the reference for all PTP IACS devices in the plant-wide architecture. The preferred solution for the network topology is redundant star, however ring may work for some use cases. In either case, synchronization should be sufficient to support time stamping functionality of I/O modules that typically have a resolution of ±4µs to ±100µs. However, the accuracy of the timestamps to UTC will be limited by how well the grandmaster clock is correlated to UTC. Since PTP time does not synchronize across the core, correlation between PTP domains is also limited by how well the grandmaster in each of the PTP domains is correlated to UTC.
For more information on CIP Sync and CIP Sync applications, see Integrated Architecture and CIP Sync Configuration
https://literature.rockwellautomation.com/idc/groups/literature/documents/at/ia-at003_-en-p.pdf
Feedback