Deploying 90W Cisco UPOE+ with Cisco Catalyst 9000 Switches Deployment Guide

 

This document provides detailed guidelines for deploying the Cisco^® Smart Building Solution with the Cisco Catalyst^® 9000 switching family and enterprise Internet of Things (IoT) endpoints that use Power over Ethernet (PoE), such as LED lighting fixtures, motorized blinds, cameras, and USB-C dongles. It includes information about the system's architecture, possible deployment models, and configuration required on the Cisco network nodes. It also recommends best practices and potential issues to be aware of when deploying the solution. Vendor-specific information and implementation details are not covered here. Please refer to the vendor-specific documentation and recommendations that are specific to a particular endpoint. The deployment models and configuration recommendations provided in this document are applicable to different types of PoE-powered enterprise IoT endpoints.

The detailed implementation of the Cisco campus network architecture is beyond the scope of this document. For more details on the campus network architecture, refer to the Design Zone for Campus Wired and Wireless LAN.

Audience

This document is intended for system architects; network, computer, and IT design engineers; systems engineers; field consultants; and customers who want to understand how to deploy an indoor IoT infrastructure. It is written with the assumption that the reader is familiar with the basic concepts of IP protocols, switching, routing, and security.

1. Introduction

The Cisco Smart Building Solution helps different building systems converge on a single IP network. When the lighting and other building systems are connected via Cisco Catalyst 9000 switches, they can be monitored and managed together by the enterprise Cisco DNA Center. The Catalyst 9000 switches installed in the wiring closet can power endpoints in spaces such as audio privacy rooms, conference rooms, team rooms, sections of a floor, etc. The switches are available in multiple form factors and port densities to address different design models. They are capable of providing PoE up to 90W on each port based on the IEEE 802.3bt standards. The specific Catalyst 9000 switch models that support PoE up to 90W are covered later in “What is Cisco UPOE+?”

Two deployment models are typically used: a distributed deployment model and a centralized deployment model. In the centralized model, the IoT endpoints connect directly to the access switches residing in the wiring closet. In the distributed model, the IoT endpoints such as lights connect directly to the Cisco Catalyst Digital Building Series Switches deployed in the ceiling space. In the distributed model, the Digital Building Series switches further connect to enterprise access switches in the wiring closet. This document focuses on the centralized deployment model, in which the IoT endpoints are connected directly to the Catalyst 9000 switches.

Comparison of the distributed and centralized deployment models

2. What is Cisco UPOE+?

Cisco UPOE+ combines the latest IEEE 802.3bt PoE standard with support for all previous IEEE standards and Cisco UPOE^®. With the advent of 90W delivered by Cisco UPOE+, smart building transformation has accelerated by incorporating several measures to safely, efficiently, and cost-effectively deliver power and data using a common network structure. For detailed information on 90W Cisco UPOE+ and IEEE 802.3bt, please refer to the IT and OT convergence white paper.

Cisco UPOE+ uses the same cabling standard as PoE. However, instead of delivering power over two of the twisted pairs, it provides the capability to source up to 90W of power by using all four twisted pairs of standard Ethernet cabling (Category 5e or better). It does this by using two Power Sourcing Equipment (PSE) controllers to power both the signal pairs and the spare pairs. Cisco UPOE+ assures a minimum of 71W of power to the Powered Device (PD). Cisco recommends the use of a 22-gauge cable to efficiently carry the power.

Complete details on the 802.3bt standards can be found in the IEEE 802.3bt-2018 document.

Architectural differences between PoE/PoE+ and Cisco UPOE+

Table 1 summarizes the differences between PoE, PoE+, Cisco UPOE and UPOE+.

Table 1.        PoE, PoE+, Cisco UPOE, and UPOE+ comparison

For 802.3bt Type 4 installations specific to lighting and applications that are concerned with maximum power transmission efficiency and low data speed installs (i.e., 1gbps or less), Cisco recommends 22AWG conductors in the cable with a minimum of Cat 5e cable.  For IEEE 802.3bt Type 4 installations requiring higher data speeds, it is recommended to use Cat 6a with 23AWG or larger conductors

Other options that comply with the National Electric Code (NEC):

•        Cables with 24AWG conductors rated at 60 ℃, in bundle sizes of 37 or less

•        Cables with 23AWG conductors rated at 60 ℃, in bundle sizes of 61 or less

•        Cables with 24AWG conductors rated at 75 ℃, in bundle sizes of 91 or less

•        Cables with 23AWG conductors rated at 75 ℃, in bundle sizes of 192 or less

 

Catalyst 9000 switches are purpose-built for enterprise networks. The switches come in both fixed and modular form factors to meet different port densities and address different deployment models. Within the Catalyst 9000 family of switches, the following series support PoE capabilities.

Catalyst 9200 Series Switches

Catalyst 9300 Series Switches

Catalyst 9400 Series Switches

Within the above list of switches, the Catalyst 9300 and 9400 Series support Cisco UPOE+, providing power up to 90W.

2.1. 90W Cisco UPOE+ use cases

Traditionally, IT and OT had different and specific roles in the system network, but lately they have been in synergy and are increasingly merging. Additionally, numerous endpoints are now entering the network, such as USB-C dongles, digital signage, motorized shades, intelligent sensors, touch-screen PCs, badging systems, emergency alert systems, and high-definition security cameras, and the list is ever growing.

Daisy-chaining the PoE lights greatly reduces cost by using fewer ports, since more lights can be connected on a single switch port. Connecting PoE pass-through desktop switches really simplifies overall cabling and is ideal for space-constrained areas. PoE-powered USB-C dongles for charging laptops as well as providing data connectivity on the same cable have the potential to replace the existing power bricks. Finally, the PoE-powered IoT ecosystem is also seeing an addition of new high-power endpoints such as desktop monitors, UHD cameras, minibars, HVAC, etc. creating great use cases for 90W UPOE+. Cisco’s integration of 90W UPOE+ line cards in its Catalyst 9400 Series modular switches as well as Catalyst 9300 Series fixed switches is making all of the use cases above a reality.

The 90W UPOE+ standard is driving new use cases

3. Solution overview

This section provides an overview of the Smart Building Solution deployment using 90W-capable switches in a centralized deployment model.

3.1. System topology

Different network topologies can be designed based on your requirements. In this deployment we will cover the initial installation followed by a description of how it can be deployed in a campus network architecture.

Figure 4 shows an example of the network topology. It uses the centralized deployment model, in which the switches are in a wiring closet. At the access layer, Catalyst 9300 or 9400 Series switches can be used to support PoE up to 90W. Catalyst 9300 Series switches can be deployed standalone or in StackWise^®-480 mode, depending on the port density and requirements. The deployment model can be a 3-tier architecture with access, distribution, and core layer, or a 2-tier architecture with access and distribution/core layers. In the distribution/core layer, the Catalyst 9400, 9500, or 9600 Series can be used to provide IP connectivity to the IoT endpoints. These switches can be deployed in either standalone or StackWise virtual mode, depending on the requirements.

Network topology

In this deployment guide, Catalyst 9300 Series switches have been used at the access layer in standalone and StackWise-480 mode. At the distribution/core layer, Catalyst 9500 Series switches are used. Specific models are outlined in Table 2 of this document. Shared services such as Dynamic Host Configuration Protocol (DHCP)/DNS, Cisco DNA Center, and Identity Services Engine (ISE) are located in the data center. Additionally, the lighting management software is within the data center.

3.2. System components

The components of the topology are a mix of Cisco products and IoT endpoints such as third-party LED lighting endpoints, Cisco and third-party cameras, Cisco IP phones, and PoE-capable USB-C dongles that can power and provide data connectivity to laptops and PCs. The tables below list these components.

Table 2.        Cisco components

Table 3.        Third-party components

3.3. System networking

The network-powered IoT endpoints should be deployed on a separate logical network (VLAN) based on the endpoint type and requirements. For example, a lighting network should be deployed in its own dedicated network, as the light fixtures would need to talk only to their controller. When you have multiple third-party PoE lights, we recommended separating them into different VLANs to make sure that the control software from each vendor does not interfere with the other lighting endpoints.

The table below summarizes the logical networks (VLANs) that are configured to segregate the traffic on the access switches.

Table 4.        VLAN segmentation

Note:      The VLANs shown above are the ones used in this deployment guide. VLAN numbering and subnets will be based on your requirements.

3.4. PoE APIs

Cisco Catalyst 9000 switches support various industry-standard open programmable protocols such as NETCONF, RESTCONF, Google gNMI, etc. to be used to interface with external automation software toolchains. Cisco Catalyst 9000 supports the IETF-based YANG data modeling language to automate network configuration and to monitor network and system functions. The APIs support Remote Procedure Calls (RPCs) for communication using various industry-standard encoding formats, including XML, JSON, and Google gRPC.

To gather PoE statistics using APIs, the following YANG model is supported on the Catalyst 9000 switches:

Cisco-IOS-XE-poe-oper.yang

More information on the YANG model can be found at poe-oper Model. A complete list of supported YANG models for Cisco IOS^® XE is located at Github.

4. 90W endpoint ecosystem

The ecosystem of 90W IoT endpoints is continually expanding. We continue to see the evolution of PoE-powered USB-C dongles that can charge laptops and provide data connectivity at the same time. Similarly, lighting infrastructure is leveraging higher PoE power to daisy-chain multiple lights via a single switch port. For this deployment guide, we have used PoE-powered USB-C dongles and lighting endpoints that are 802.3bt compliant. In this section, we take a brief look at the specifications and characteristics of the PoE-powered IoT endpoints.

Power consumption of the switch and heat dissipation numbers can be derived using the Cisco Power Calculator.

USB-C dongle for PoE and data connectivity

A PoE-powered USB-C dongle provides PoE and data connectivity to a laptop over a single Ethernet connection. The endpoint (laptop in this case) is transparent to these changes, while the switch port provides required PoE and network connectivity.

There is no difference in how you configure the access port when using a USB-C dongle, other than enabling additional PoE capabilities such as Perpetual and Fast PoE. The configuration for the switch port looks like the one shown below.

 

Once the laptop is connected to the switch port, it starts to draw power to charge while getting data connectivity at the same time. The following is a snapshot of a Lenovo laptop that is connected to the Catalyst 9300 Series switch port via a USB-C dongle.

The snapshot below shows the power negotiation between the PD (USB-C dongle) and PSE (Catalyst 9300 Series switch).

 

USB-C dongle providing power and network connectivity

90W lighting fixtures

Multiple vendors have developed network gateways that can daisy-chain multiple light fixtures that connect to a single port of an access switch to leverage the 90W that is available on the port. This helps build a scalable lighting network within a campus network. In this section, we will go through the configuration that is needed on the Catalyst 9300 Series switch for initial installation of the lighting network.

Light fixture installation

The lighting vendor will provide a recommendation for installation and cabling between the light and the Cisco switch. For 802.3bt Type 4 installations specific to lighting and applications that are concerned with maximum power transmission efficiency and low data speed installs (i.e., 1gbps or less), Cisco recommends 22AWG conductors in the cable with a minimum of Cat 5e cable. For installs that need higher Ethernet speeds, either today or in the future, Cat6A cable is recommended.  

Initial installation of the Cisco Catalyst 9300 Series switch

The switch by default comes with a preloaded image that can be accessed via the console cable. The switch can be upgraded to the required image by following the software upgrade guide.

Configuring the Cisco Catalyst 9300 Series UPOE+ switch for initial installation

This section covers the basic configuration and network topology needed for the initial installation of the light fixtures.

Smart Building Solution initial setup in the wiring closet with Cisco Catalyst 9300 Series switches

The configuration below is required on the switch so that LED lights can be assigned an IP address and be discovered by the respective third-party control software.

●      Define a VLAN that can be used for the lighting network.

 

●      Enable Link Layer Discovery Protocol (LLDP) on the switch.

 

Light fixtures have to communicate with the controller to perform automated and sensor operations and firmware upgrades. This communication occurs over IP on the Ethernet network; hence each light fixture should be assigned an IP address. To assign the IP address dynamically, the Catalyst 9300 Series switch can act as a DHCP server.

●      Configure a DHCP server on the Catalyst 9300 Series switch.

 

In scenarios where the requirement is to maintain the same IP address for an endpoint connected to a switch port, port-based address allocation can be used on Catalyst 9000 switches so that the same IP address is assigned to the endpoint connected to the interface, regardless of interface bounce or endpoint replacement.

 

DHCP uses the subscriber ID configured on the interface to generate the client ID. If no subscriber ID is configured, the client ID is automatically generated based on the short name of the interface. The client ID already present in the message is ignored.

Once the light fixtures get an IP address, they can be discovered by the control software. Additional settings and firmware upgrades should be possible from the respective third-party control software.

As soon as the lighting endpoint is connected to the switch port, the power is granted to the PD based on the requested class.

 

In this case, we have more than one light in a daisy chain that is connected to port G1/0/14. We can see below that the PD has requested Class 8.

 

 

5. 90W Deployment in a campus network architecture

In an earlier section, we looked at how 90W endpoints such as light fixtures and USB-C dongles can be configured for initial installation and verification.

Once the endpoints are verified and validated, they can be migrated to an existing campus infrastructure. This section provides more details on how to design the endpoints’ subnet, controller software, and centralized DHCP/DNS server.

5.1. Network topology

The access switches in the wiring closet (Cisco Catalyst 9300 Series standalone and stack) connect to a production network distribution/core switch with separate logical networks for light fixtures.

Cisco campus network architecture with Catalyst 9000 switches

In this topology, we have used Catalyst 9300 Series switches (C9300-24H/48H) in the access layers, both in standalone and StackWise-480 mode.

Catalyst 9500 Series switches (9500-24Q) are configured for StackWise Virtual mode and are acting as the distribution/core layer for this deployment guide.

Shared services such as Cisco DNA Center, ISE, and third-party lighting management software are connected to the data center switch in its own specific subnet.

5.2. Configuring DNS/DHCP services

This section covers dynamic IP addressing for 90W IoT endpoints such as light fixtures.

During the preliminary tests, the DHCP server for light fixtures is configured on a Catalyst 9300 Series switch. When this environment is migrated to a campus network architecture, DHCP addressing will also be migrated to a centralized DHCP server that manages multiple DHCP pools across the network.

In this deployment guide, the DHCP server is connected to the data center switch and is serving multiple IP pools across the campus network.

Table 5.        IPv4 DHCP address pool configured on the Windows DHCP server

 

DHCP pools defined on the Windows server

Note:      There may be multiple requirements from third-party vendors on how the IoT endpoints are discovered within the network. For example, a third-party lighting vendor may require a custom DHCP option so that lighting endpoints can communicate with the management software for discovery, management, and firmware upgrade operations. Necessary options have to be predefined in the DHCP pool for the discovery to work.

5.3. Campus network core/aggregation configuration

The UPOE+ access switches (Catalyst 9300 Series standalone and stack) are connected to the campus network distribution/core switches in a converged campus/large-scale deployment. A detailed description of the implementation of the campus network architecture is beyond the scope of this document.

In this guide, Catalyst 9500 Series switches are deployed in StackWise Virtual mode to provide campus network core, distribution, and Layer 3 routing functionalities for lighting and other 90W PoE endpoints that are connected across multiple access layer switches.

5.3.1. Cisco StackWise Virtual

Cisco StackWise Virtual is a network system virtualization technology that pairs two switches into one virtual switch. Switches in a Cisco StackWise Virtual solution simplify operational efficiency with a single control and management plane, scale system bandwidth with a distributed forwarding plane, and assist in building resilient networks using the recommended network design. Cisco StackWise Virtual allows two physical switches to operate as a single logical virtual switch.

Catalyst 9500 Series switches deployed with StackWise Virtual

Within a StackWise Virtual domain, one device is designated as the StackWise Virtual active switch, and the other is designated as the StackWise Virtual standby switch (Figure 9). All control plane functions are centrally managed by the StackWise Virtual active switch, including:

●      Management (Simple Network Management Protocol [SNMP], Telnet, Secure Shell [SSH] Protocol, etc.)

●      Layer 2 protocols (Bridge Protocol Data Units [BPDUs], Protocol Data Units [PDUs], Link Aggregation Control Protocol [LACP], etc.)

●      Layer 3 protocols (routing protocols, etc.)

●      Software data path

Prerequisites for StackWise Virtual

●      All the switches in the Cisco StackWise Virtual solution should be the same switch model.

●      All the switches in the Cisco StackWise Virtual solution should be running the same license level.

●      All the switches in the Cisco StackWise Virtual solution should be running compatible software versions.

Both of the Catalyst 9500 Series switches have been configured to operate as a StackWise Virtual pair.

The steps below outline the configuration that is required to convert two standalone 9500 Series switches to a StackWise Virtual pair.

Cisco StackWise Virtual configuration

●      Configure the StackWise Virtual domain on 9500-1 and 9500-2.

 

●      Configure the StackWise Virtual link.

 

●      Configure StackWise Virtual dual-active detection. This has to be configured on a different port than the StackWise Virtual link ports.

 

●      Save the configuration and reload the switch.

This step will convert the standalone switches to the StackWise Virtual domain once the switches come back online after the reload.

5.3.2.     Configuring Layer 2 and Layer 3 services

This section defines the implementation of Layer 2 and Layer 3 logical interfaces on the Catalyst 9500 Series switch.

Below is the topology from the core/distribution layer to the access layer.

Distribution/core to access topology

●      Configure VLANs, which must be created along with port assignments on the Catalyst 9500 Series switch.

 

The Catalyst 9500 Series switch in StackWise Virtual mode is acting as the gateway for the lighting VLAN and other 90W IoT endpoints in this deployment.

Configure a Layer 3 Switched Virtual Interface (SVI) for the 90W endpoints that are connected on the access switch. We recommend segmenting the VLANs based on the endpoint types. In this deployment guide, we have used VLAN 10 for the lighting network for vendor 1, VLAN 20 for the camera/surveillance network, VLAN 30 for the lighting network for vendor 2, VLAN 40 for data connectivity, and VLAN 100 for management connectivity across the site.

Note:      When migrating the lighting or any 90W IoT endpoints from a smoke test to a converged campus network, remove the SVIs of any VLANs that you may have created on wiring closet access switches. SVIs for these endpoints are configured at core/aggregation switches to provide Layer 3 services to the lighting network.

Create port-channel interfaces on the Catalyst 9500 Series switch to the wiring closet switches (Catalyst 9300 Series standalone and stack), as shown in the topology. Configure the links as trunks and allow the lighting VLANs, management VLAN, and other IoT endpoint VLANs on the trunk links.

Make sure the port channel has at least one interface from each physical switch of the StackWise Virtual pair.

●      Below is the physical interface and port-channel configuration on the Catalyst 9500 Series switch to the 9300 Series standalone switch:

 

When connecting a Catalyst 9300 Series stack to the 9500 Series distribution/core, make sure that uplinks are distributed across active, standby, and member so that there is always redundancy during an active switch failover or when performing software upgrades using the Extended Fast Switch Upgrade feature.

Below, we have uplinks connected from all switches in the stack to the distribution/core. The number of uplinks has to be decided based on the bandwidth that would be needed from access to distribution/core.

●      Below is the physical interface and port-channel configuration on the Catalyst 9500 Series switch to the 9300 Series stack switch:

5.3.3.     DHCP relay agent

In a campus network architecture, IoT endpoints and the DHCP server are typically not located in the same subnet. A centralized DHCP server that resides in a shared environment serves multiple subnets on the access switches.

The gateway for these IoT endpoints resides on the Catalyst 9500 Series switches, which are operating in StackWise Virtual mode, as outlined in Figure 9. The gateways on the Catalyst 9500 Series have to relay these DHCP messages to the centralized DHCP server.

The configuration below shows how to enable a relay agent on the Catalyst 9500 Series switches:

 

Enable required routing on the 9500 Series switches to make sure the endpoint subnets have reachability to the shared services and lighting management subnets that reside in the data center network, as outlined in Figure 7.

In this deployment guide, we have used Open Shortest Path First (OSPF) as the interior gateway protocol to propagate local routes and learn remote routes from the data center switch.

 

5.4. Campus network access configuration

This section covers the necessary configuration required on the access switches. Create different VLANs based on the types of endpoints located in the network.

●      Define VLANs that can be used for the endpoint network.

 

●      Enable LLDP on the switch.

 

Autoconf or manual templates, as outlined in the “Autoconf to scale endpoint configurations” section, can be used to scale the configuration of the access ports on Catalyst 9300 Series standalone or stack switches. Below is the minimum configuration that is required on the switch port so that any 90W IoT endpoint that is connected to the downlink port can get an IP address from the DHCP server and be discovered by its controller.

6. Security best practices

The Cisco Catalyst 9000 switching platforms have many integrated security features that can provide threat defense capabilities for mitigating man-in-the-middle attacks and protecting the critical network infrastructure. This section details the switch configurations for basic Layer 2 security features that can be enabled to secure the network. These are optional features that can be enabled to secure the network.

6.1. DHCP snooping

Rogue DHCP servers can be detrimental to the security and usability of the network if not protected against properly. Rogue DHCP servers attack the network by assigning unrouteable IP addresses to clients, causing them to lose connectivity. They can also be used to issue malicious DNS servers. Users then looking to go to real websites will be sent to fake copies of these sites to steal credentials or information.

DHCP snooping is a tool used to combat rogue DHCP servers. It works by assigning one or more ports as trusted, meaning these ports lead to legitimate DHCP servers. The switch will then build a database of the untrusted hosts with leased IP addresses, MAC addresses, switch ports, and VLANs. Traffic being sent from these untrusted hosts will be filtered of any DHCP server messages, blocking any attempt to create a malicious DHCP server.

Perform the following steps on the Cisco Catalyst 9300 Series switch to configure IP DHCP snooping:

●      Configure the required port as a DHCP snooping trusted port. This is usually the uplink port to the core.

 

●      Enable IP DHCP snooping globally and for the required VLANs

6.2. PortFast and BPDU Guard

To prevent loops in Layer 2 networks, we use Spanning Tree Protocol (STP), but this can also hurt the network if it is not protected.

PortFast can be used on the switch ports where a single workstation or server is connected to allow the devices to connect to the network immediately, instead of waiting for the port to transition from the listening and learning states to the forwarding state. PortFast causes a switch port to enter the spanning tree forwarding state immediately, bypassing the listening and learning states.

In STP, a root bridge is elected. This decides which ports will be put in a forwarding or blocking state. If a device is added with a lower priority than the current root bridge, it will take over, causing a topology change and possibly blocking traffic from a wanted path. BPDU Guard is a protocol designed to solve this problem. When enabled, if a switch sees BDPU traffic coming from a port, it will automatically set it to the “errdisabled” state so that no traffic can pass.

We recommend enabling PortFast and BPDU Guard on the ports that are connected to lighting endpoints, as no BPDUs are expected on those ports.

●      Enable PortFast and BPDU Guard using the command-Line Interface (CLI) as shown below:

 

6.3. Dynamic ARP Inspection

ARP cache poisoning is a malicious tool used to stage man-in-the middle attacks. It works by sending a forged Address Resolution Protocol (ARP) packet with the IP address of another device and the MAC address of itself to poison the host’s ARP cache. This means traffic destined for the legitimate device will instead be sent to the attacker. The attacker can then forward the traffic to its intended destination, making it look as if the traffic was never interrupted.

Dynamic ARP Inspection (DAI) is a tool that can be used to mitigate this threat. DAI uses the DHCP snooping database for IP-to-MAC address bindings. DAI then intercepts all ARP packets and drops any packet in which the IP-to-MAC address binding is not valid.

●      Configure the uplink ports to be trusted:

 

●      Enable ARP Inspection per VLAN:

6.4. Storm control

Storm control prevents traffic on a LAN from being disrupted by a broadcast, multicast, or unicast storm on one of the physical interfaces. A LAN storm occurs when packets flood the LAN, creating excessive traffic and degrading network performance. Errors in the protocol-stack implementation, mistakes in network configurations, or users issuing a denial-of-service attack can cause a storm.

With storm control, the port blocks traffic when the rising threshold is reached. The port remains blocked until the traffic rate drops below the falling threshold (if one is specified) and then resumes normal forwarding.

An action can be taken when a storm is detected. The default is to filter out the traffic and not to send traps.

●      Select the shutdown keyword to error-disable the port during a storm.

●      Select the trap keyword to generate an SNMP trap when a storm is detected.

 

6.5. Port access control lists

PACLs filter incoming traffic on Layer 2 interfaces using Layer 3 information, Layer 4 header information, or non-IP Layer 2 information. The PACL feature uses standard or extended IP ACLs or named MAC-extended ACLs that you want to apply to the port.

The ports on which lights are connected should be able to filter packets based on specific Layer 4 port numbers so that unwanted traffic doesn't reach the light. PACLs, in this scenario, filter specifically the port numbers that third-party control software uses to communicate with lights.

To configure a PACL, do the following on the Catalyst 9300 Series switch:

Configure the IP access list to permit incoming traffic only for Layer 4 port numbers specific to communication between the lights and SCM.

 

Note:      The network administrator will need to Identify the ports that need to be open from the IoT endpoint to the controller for this feature to be used. The port numbers are used to build the port ACL, which allows communication from the endpoint only to these specific ports. This adds an extra layer of security by limiting the endpoint to talking only to the controller in the network.

7. Configuring PoE features

PoE enables the switch to provide power through the copper wire to an endpoint. This capability is referred to as inline power. The specification for PoE calls for two devices: the Power Sourcing Equipment (PSE) and the powered device (PD).

Terminology details

Power Sourcing Equipment (PSE): A device that provides power through a twisted-pair Ethernet connection. The switch is the PSE.

Powered Device (PD): A device powered by a PSE (for example, lights, IP phones, IP cameras, and wireless access points).

We support two major types of PoE: Perpetual PoE and Fast PoE. Certain restrictions apply to both. These are:

●      Configuration of Fast PoE or Perpetual PoE has to be done before any endpoint is physically connected. Alternatively, you can do a manual shut/no-shut of the ports drawing power.

●      Power to the ports will be interrupted in the event of a microcontroller (MCU) firmware upgrade, and the ports will be back up immediately after the upgrade.

●      Perpetual and Fast PoE are not supported with Cisco StackPower. Hence, StackPower is not recommended in deployments where the endpoint is not expected to lose power during switch reboots.

●      If the PD doesn’t support LLDP, the user can configure with either static or 2-event to receive required power as per the PD specification.

Note:      Both Perpetual PoE and Fast PoE can be enabled together. But to enable Fast PoE, Perpetual PoE must be enabled first.

7.1. Perpetual PoE

Perpetual PoE provides uninterrupted power to connected PDs even when the PSE is rebooting. This feature assists with maintaining power to the endpoint during a switch reload, unexpected crash, or software upgrade. This capability is critical in environments where the PD has to remain powered up all the time. We recommend enabling this feature on the switch port where the PD is connected.

Configuring Perpetual PoE:

Note:      Power to the ports will be interrupted in case of an MCU firmware upgrade, and ports will be back up immediately after the upgrade.

7.2. Fast PoE

When there is an unexpected power outage or failure, Fast PoE assists with restoring power to the PD within 25 seconds from the time that power is restored. This feature remembers the last power drawn from a particular PSE port and provides power to the PD without waiting for Cisco IOS to boot up. We recommend enabling Perpetual PoE and Fast PoE on the access ports where the PDs are connected.

Note:      Configuring Fast PoE requires that Perpetual PoE be configured first.

Configuring Fast PoE:

 

7.3. PoE port priority

By default, all PoE ports are treated as low priority. In scenarios where load shedding happens due to a power supply failure or insufficient power, all the ports are treated with equal priority. To preserve power for critical IoT endpoints, their respective switch ports can be configured with high priority. There are two allowed modes of priority: high and low. Setting port priority for switch ports allows the switch to determine which ports have a higher priority for PoE than others. This guarantees that ports with high priority will not lose power in load-shedding scenarios.

Below is the CLI command used to configure port priority on the interface:

 

Using the show power inline priority command shows the port priority for each port:

8. Autoconf to scale endpoint configurations

Configuring each individual port on the switch for 90W IoT endpoints is cumbersome and ineffective. These configurations can be applied dynamically using interface templates. The Autoconf feature permits hard binding between an end device and an interface. Autoconf falls under the umbrella of the Cisco Smart Operations solution. Smart Operations is a comprehensive set of capabilities that can simplify and improve LAN switch deployment and help organizations deliver operational excellence and scale services on the network.

The Autoconf feature automatically applies the necessary configurations on the device ports to enable the efficient performance of each directly connected end device using a set of interface configurations that are configured inside an interface template.

Autoconf is disabled by default in global configuration mode. When you enable Autoconf in global configuration mode, it is enabled by default at the interface level. The built-in template configurations are applied based on the end devices detected on all the interfaces.

 

Use the access-session inherit disable autoconf command to manually disable Autoconf at the interface level, even when Autoconf is enabled at the global level.

 

Autoconf uses the Device Classifier to identify the end devices that are connected to a port. The Autoconf feature uses the device classification information gleaned from Cisco Discovery Protocol, LLDP, DHCP, MAC addresses, and the organizationally unique identifier (OUI) that is identified by the Device Classifier.

Autoconf provides the following built-in templates:

●      AP_INTERFACE_TEMPLATE

●      DMP_INTERFACE_TEMPLATE

●      IP_CAMERA_INTERFACE_TEMPLATE

●      IP_PHONE_INTERFACE_TEMPLATE

●      LAP_INTERFACE_TEMPLATE

●      MSP_CAMERA_INTERFACE_TEMPLATE

●      MSP_VC_INTERFACE_TEMPLATE

●      PRINTER_INTERFACE_TEMPLATE

●      ROUTER_INTERFACE_TEMPLATE

●      SWITCH_INTERFACE_TEMPLATE

●      TP_INTERFACE_TEMPLATE

Note:      By default, built-in templates are not displayed under the running configuration. The built-in templates are displayed in the running configuration only if you edit them.

You can manually create templates that can detect an endpoint. The template that is selected is based on parameter map information applied to an interface. This information can be based on the following criteria:

●      End device type

●      MAC address

●      OUI

●      Platform type

●      User role

●      Username

Below, we have configured a user-defined template that can be used for the lighting endpoints. You can customize this to meet the needs of the respective IoT endpoint.

 

This user-defined template can be verified by using the commands below:

 

The Autoconf feature provides one built-in parameter map (BUILTIN_DEVICE_TO_TEMPLATE) with the following configuration:

 

You can also manually create policy maps, parameter maps, and templates. When a trigger is created based on specific user information, the associated template will be assigned to the port.

Customer mappings can be defined under the BUILTIN template to match the IoT or lighting endpoints, and then the user-defined templates can be applied as soon as the endpoint is connected to the network.

 

You will see the user-defined options applied to the default BUILTIN template.

 

As you can see below, as soon as the endpoint is connected to the network, Autoconf assists with applying the interface template dynamically based on the user-defined mappings.

9. Using MUD to classify and secure the network

9.1. What is MUD?

The Manufacturer Usage Description (MUD) is an embedded software standard defined by the IETF that allows IoT device makers to advertise device specifications, including the intended communication patterns for their device when it connects to the network. The network can then use this intent to author a context-specific access policy, so the device functions only within those parameters. In this manner, the MUD becomes the authoritative identifier and enforcer of policy for devices on the network.

Core MUD solution

The MUD solution consists of three components at its core:

●      A URL that an IoT device emits when it connects to the network.

●      An internet-hosted file that this URL points to. This file contains an abstracted policy that describes the level of communication access that the IoT device needs to perform its normal function.

●      A core process that receives the URL from the IoT device, retrieves the file from the MUD file server, and establishes appropriate access controls in the network for that IoT device.

Cisco’s MUD process flows

The figure above shows the MUD process.

●      When an IoT device first connects to a network access point, it sends a MUD URL embedded in either the LLDP, DHCP, or 802.1X request.

●      The network access device, which for Cisco deployments is restricted to Catalyst switches for now, extracts the URL, encapsulates it in a RADIUS packet, and sends it to the Authentication, Authorization, and Accounting (AAA) server. (When the MUD URL is embedded in a DHCP packet, some networks may instead forward the entire DHCP packet to the AAA server.)

●      The AAA server, which for Cisco deployments is the ISE, then passes this URL on to the MUD controller, which is also hosted on the ISE for now.

●      The MUD controller then contacts the internet-hosted MUD file server of the manufacturer that this URL points to over HTTPS.

●      After verifying that the MUD file was produced by the device manufacturer, the MUD file corresponding to that device is sent to the MUD controller. This file, which is a YANG data model represented as a JSON object, contains abstracted communication intent for the IoT device in question.

●      The MUD controller translates this abstract intent to a context-specific policy, which is passed on to the AAA server (ISE).

●      The ISE then enforces the policy on the network in the form of PACLs for the point of that IoT device's connection.

●      Based on the manufacturer’s stated intent, access to the device is then provided.

9.2. Endpoint requirements for MUD

The endpoint should be capable of sending a MUD URL embedded in either the LLDP, DHCP, or 802.1X request. When enabling MUD functionality on the switch, make sure the third-party lighting vendor is capable of sending the MUD URL.

9.3. MUD configuration on the switch

Below is an outline of the configuration that is required on the switch to enable MUDs. In this case, the lighting fixture is sending the MUD URL via the LLDP TVL 127.

 

●      Enable a change of authorization from RADIUS:

 

●      Enable the Device Sensor to track based on LLDP or DHCP. The Device Sensor configuration is based on Identity Based Networking Services (IBNS) 2.0.

 

●      Enable LLDP on the switch so that the switch can form an LLDP session to the lighting endpoint.

 

●      Configure IBNS 2.0 policy to trigger MAC Authentication Bypass (MAB) on the interface.

 

●      Configure the interface template to scale the configuration of multiple lighting endpoints.

Spanning tree PortFast is a requirement, as the MAB can time out before the port STP changes to the Forward state. Below is the minimum template configuration required for MUD to work.

 

●      Set the RADIUS server preshared key.

 

●      Apply the template to the interface.

9.4. Configure ISE to classify MUD

Profiling for MUD has to be enabled on the ISE for MUD functionality. It can be enabled in Work Centers à Settings.

 

Check “Enable profiling for MUD” and click Save.

Once the endpoint (lighting endpoint in this case) is connected, it will pass the MAB authentication based on the policy that has been defined on the switch.

 

The endpoint will be seen as active and authenticated on ISE in Context Visibility à Endpoints.

9.5. Policy profiling using ISE

Now that ISE can see and authenticate the client, you can create a profiling policy that matches the MUD URL sent by the lighting fixture. This can be done in Work Centers à Profiler à Profiling Policies.

9.6. Authorization policy using ISE

Authorization policy can be configured in ISE that can dynamically push an ACL to the port where the lighting fixture is connected. This helps limit the communication from the lighting fixture to the control software and other required services that the lighting fixture needs to communicate with. This policy can be defined to a highly granular level, up to limiting access to specific Layer 4 ports, depending on the third-party vendor requirements.

Create an authorization policy that can use the downloadable ACL that was defined earlier.

Enable the authorization policy so that the lighting fixture is authenticated via MAB and a dynamic ACL can be pushed to the Catalyst 9300 Series switch to limit the access by the lighting fixture based on the custom-defined ACL parameters.

 

 

 

10. Cisco DNA Center for day-0, day-N operations

Cisco DNA Center has various capabilities built in that help with network design, automation, assurance, provisioning, maintenance, and much more for your devices. In this section, we will cover a few of the various functionalities that Cisco DNA Center allows.

10.1. Cisco DNA Center for day-0 provisioning

This section describes how to provision the Catalyst 9300 Series switch via Cisco DNA Center. There are multiple ways to provision the switch. Here we are going to look at how Plug and Play (PnP) helps with day-0 provisioning.

●      For PnP to work successfully, you need a DHCP server with option 43 enabled. Option 43 is used to locate the PnP server, which resides on Cisco DNA Center. For our setup, the DHCP server has already been preconfigured with option 43. This DHCP server assigns an IP address to the switch in addition to providing the location of the Cisco DNA Center (PnP server).

●      By default, PnP discovery happens on VLAN 1. Since the management network that we are using in this design is VLAN 100, we have changed the PnP startup VLAN to 100 on the Catalyst 9500 Series switch.  The PnP startup VLAN can be changed using the command below:

 

●      Once the switch is powered on initially, the Catalyst 9300 Series switch learns about the PnP server and establishes a secure connection with it.

1.     The device will be seen in Cisco DNA Center in the Provision section under Plug and Play:

2.     On the Plug and Play tab, select the device or devices that need to be claimed.

3.     From the Actions menu, select Claim:

4.     Set the Device Name and select the site to which the device is to be assigned, then click Next:

 

5.     Select the software image and/or the template you wish to assign to the device, then click Next:

6.     You may check the day-0 configuration by clicking “Preview Day-0 Config,” then click Continue.

7.     Verify that the device is claimed.

More details and workflows regarding PnP using Cisco DNA Center can be found in the Network Device Onboarding Deployment Guide.

10.2. Software update via Cisco DNA Center

The software image of the switch can be updated using Cisco DNA Center. The image must be downloaded and loaded into the Cisco DNA Center repository beforehand and marked as the Golden Image. Once that is done, the task of updating the image simply involves clicking a few buttons. The steps are as follows:

1.     Go to the Inventory of devices within Cisco DNA Center.

 

2.     Select the devices that are to be updated.

 

3.     After selecting the devices, click the Action menu, choose Software Image, and then Update Image.

4.     Decide when to update the image (it can be updated now or scheduled for a later time). Then click Next.

5.     Select when to activate the image.

6.     Verify the checks that will be performed for validation.

 

7.     Verify the checks and activation timing. Then click Confirm.

10.3. Device discovery after initial installation

This section applies if the device has already been installed and needs to be integrated into Cisco DNA Center. To get Assurance benefits via Cisco DNA Center, NETCONF needs to be configured on the access switches to which the PoE devices are connected. Cisco DNA Center pushes the required telemetry configuration to the switch for PoE Assurance after the device is discovered successfully.

1.     Enable NETCONF on the switch prior to discovery.

2.     Navigate to the Discovery section.

3.     Either choose Add Discovery or edit an existing discovery.

4.     Add NETCONF into discovery.

5.     Verify the successful integration of NETCONF between the Catalyst 9300 Series switch and Cisco DNA Center.

6.     Verify that the following NETCONF telemetry (subscriptions 500 and 501) was pushed to the Catalyst 9300 Series switch:

11. Cisco DNA Center for PoE assurance

Cisco DNA Center allows for easy viewing and assessment of real-time analytical data of the connected PoE devices. Information includes the PoE device operational state, PoE powered device power distribution, various PoE insights, and power load distribution (see Figure 14).

POE Assurance dashboard

11.1. PoE operational state distribution

Within the PoE Assurance dashboard, you can see various information for the PoE devices. For example, by clicking the View Details link in the PoE Operational State Distribution section, you see the states of the various PoE endpoints connected to your switch (see Figure 15). You can view a timeline of when and where a fault occurred with an endpoint, view information on which devices are connected to which Catalyst 9300 Series switches, and view various PoE data such as allocated power and power consumed by the PoE endpoint (see Figure 16).

POE Operational State Distribution timeline

PoE Operational State Distribution details

11.2. PoE powered device distribution

Another section within the PoE Assurance dashboard is PoE Powered Device Distribution. By clicking View Details in this section, you can view the PoE endpoint power distribution on the switches (see Figure 17). You can also view how many devices are within which allocated power segments, allowing for an easy view and assessment of power load distribution.

PoE Powered Device Distribution graph

11.3. PoE insights

Within the PoE Insights section of the POE Assurance dashboard, you can also see a graphical view of the various PoE endpoints that are connected to your switch (see Figure 18). You can see the various device types and their power state, allowing for easy troubleshooting of vendor-specific endpoints.

A view of the different PoE endpoint types

11.4. Power load distribution

Power load management on the Catalyst 9300 Series switches can also be easily managed and assessed via Cisco DNA Center. Within the Power Load Distribution dashboard, you can view a timeline of power consumption as a percentage per Catalyst 9300 Series device (see Figure 19). Furthermore, you can see an exact percentage of power load and the total power budget per switch (see Figure 20).

Power load distribution on Catalyst 9300 Series switches

Power load distribution per switch

11.5. PoE port verification on switch

With the help of Cisco DNA Center, verification of which ports are currently powering PoE devices is made simple. In Cisco DNA Center, you can see a graphical representation of which ports on the Catalyst 9300 Series are powered on and supplying PoE to endpoints. Furthermore, you can view the port speed, PoE status, and much more in the details for each port on this page (see Figure 21).

View of which switch ports have PoE enabled

12. Troubleshooting PoE on Catalyst 9000 switches

This section covers the basic troubleshooting steps for resolving PoE issues seen on Catalyst 9000 switches.

Troubleshooting of PoE endpoints is made simpler with a GUI with the help of Cisco DNA Center. You can view an instance of a PoE endpoint and see the various data specific to that device (see Figure 22). You can view the various issues related to the device, and Cisco DNA Center will also help you resolve those issues by recommending possible resolutions to the particular issues and allowing you to resolve them within Cisco DNA Center by giving you the option to push particular CLI commands to the Catalyst 9300 Series switch (see Figure 23).

Furthermore, you get a timeline view of when major events occurred to the PoE endpoint, how critical they were, the reason for the failure, and much more (see Figure 22).

PoE endpoint health

Resolutions to PoE device issue

Reason for PoE device failure

Displaying 802.3bt relevant information

A show power inline upoe-plus command has been added to display the 802.3bt relevant information. The output will show 802.3bt-specific information such as PD type, state of each pair on the port, power allocated to the port, and power consumed by the PD.

For more detail information, the show power inline detail command can be used:

PoE power-on self-test (POST)

The POST performs a PoE controller functionality test to check the chip accessibility, firmware download, and health status of the PSE.

 

This test is run only during the bootup of the switch. For any failed status in POST results, contact Cisco TAC for further troubleshooting.

PoE diagnostics

With online diagnostics, you can test and verify the hardware functionality of a device while the device is connected to a live network. Online diagnostics contain packet-switching tests that check different hardware components and verify the data path and control signals. Online diagnostics detect problems related to components such as:

●      PoE hardware components

●      Interfaces

●      Solder joints and board integrity

A TestPoE diagnostic test can be used to detect hardware problems. These can be run on demand. This is a disruptive test and is off by default. Do not start this diagnostic test during normal switch operation unless recommended/assured by TAC. This test can be run if you experience PoE controller issues with a port, and it can be run only as an on-demand test.

 

For any PoE-related issues, gather the output from the following show tech-support command:

 

Controller port error

A port error reported by the PoE controller is detected by the Cisco switch. Controller errors have some common variants.

 

An Imax error occurs when a PoE-capable port on the switch draws more power than it negotiated. Additionally, some third-party devices might have an excessive surge in current when first connected to a PoE port, which could trigger an Imax error.

Please make sure that attached PDs are IEEE compliant.

ILPOWER_POWER_DENY error

 

This error means that there is not enough power remaining in the switch to supply the PoE port. This is likely due to total inline power being greater than available power. Verify power budgeting, and install more power supplies if needed. Changing power supply redundancy from redundant to combined will also help. For stacked systems, consider StackPower to pool total power across switches.

Controller POST error

 

This error means the switch decided to shut off PoE because the POST for the switch failed. Verify the PoE controller functionality test to learn the health status of the PSE.

The table below summarizes the high-level features validated as part of this deployment guide.

Table 6.        Features covered in this guide

 

The following vendor-specific lighting endpoints have been used in this deployment guide.

●      Igor

●      Molex

Anker and Bizlink USB-C 90W-capable dongles have also been used in this deployment guide.