District Office Design
There are four main differences in the district office design:
•
The use of the Supervisor 6—The Supervisor 6 supports hierarchical QoS.
•
The Metro Ethernet Switch Connection—The aggregation and QoS policy enforcement point for the Metro Ethernet WAN connection to the schools
•
The Services Block Switch Connection—The district office "mini-Data Center" for the management and services servers for the district and the schools
•
The ASA Firewall Connection—The firewall connection to the Internet
Figure 11-1 shows a schematic of the district office network. Aside from providing core/distribution services to the access switches, the Cisco 4500 Modular switch in the district office connects the school WAN to the district office, the services such as Internet access and the Services Block of the SRA.
Figure 11-1 District Office Partial Schematic
Metro Ethernet Connection Configuration
Table 11-1 shows an example of the port-channel configuration on the core/distribution 4500 Modular switch and the 3750 Metro Ethernet switch. This is a Layer-3 connection where both the core/distribution switch and the Metro Ethernet switch are part of the same EIGRP AS. The most significant difference in this configuration from the School design using the 4500 Modular switch is the difference in the QoS configuration on the 4500 interface; this is primarily due to the district office using hierarchical QoS features of a Supervisor 6 module, rather than the Supervisor 5 used in the School SRA.
Table 11-1 Port-Channel Configuration on the Core/Distribution 4500 Modular Switch and the 3750 Metro Ethernet Switch
Example 4500 Modular Switch Configuration
|
Example 3750ME Switch Configuraiton
|
interface Port-channel1
description Connected to 3750ME-DO
dampening
ip address 10.125.32.4 255.255.255.254
ip authentication mode eigrp 100 md5
ip authentication key-chain eigrp 100 eigrp-key
ip pim dr-priority 100
ip pim sparse-mode
ip summary-address eigrp 100 10.125.0.0 255.255.0.0
5
logging event link-status
load-interval 30
carrier-delay msec 0
service-policy output PQ-POLICER
|
interface Port-channel1
description Connected to 4507-DO
no switchport
dampening
ip address 10.125.32.5 255.255.255.254
ip authentication mode eigrp 100 md5
ip authentication key-chain eigrp 100 eigrp-key
ip pim sparse-mode
ip summary-address eigrp 100 10.127.0.0 255.255.0.0 5
ip summary-address eigrp 100 10.126.0.0 255.255.0.0 5
logging event bundle-status
load-interval 30
carrier-delay msec 0
hold-queue 2000 in
hold-queue 2000 out
|
interface GigabitEthernet3/3
no switchport
no ip address
load-interval 30
carrier-delay msec 0
udld port
channel-protocol pagp
channel-group 1 mode desirable
service-policy output EGRESS-POLICY
!
interface GigabitEthernet4/3
no switchport
no ip address
load-interval 30
carrier-delay msec 0
udld port
channel-protocol pagp
channel-group 1 mode desirable
service-policy output EGRESS-POLICY
|
interface GigabitEthernet1/0/1
description Connected to cr24-4507-DO
no switchport
no ip address
logging event bundle-status
load-interval 30
carrier-delay msec 0
srr-queue bandwidth share 1 30 35 5
priority-queue out
udld port
mls qos trust dscp
channel-protocol pagp
channel-group 1 mode desirable
!
interface GigabitEthernet1/0/2
description Connected to cr24-4507-DO
no switchport
no ip address
logging event bundle-status
load-interval 30
carrier-delay msec 0
srr-queue bandwidth share 1 30 35 5
priority-queue out
udld port
mls qos trust dscp
channel-protocol pagp
channel-group 1 mode desirable
|
policy-map PQ-POLICER
class PRIORITY-QUEUE
police cir 300000000
conform-action transmit
exceed-action drop policy-map EGRESS-POLICY
class PRIORITY-QUEUE
priority
class CONTROL-MGMT-QUEUE
bandwidth remaining percent 10
class MULTIMEDIA-CONFERENCING-QUEUE
bandwidth remaining percent 10
class MULTIMEDIA-STREAMING-QUEUE
bandwidth remaining percent 10
class TRANSACTIONAL-DATA-QUEUE
bandwidth remaining percent 10
dbl
class BULK-DATA-QUEUE
bandwidth remaining percent 4
dbl
class SCAVENGER-QUEUE
bandwidth remaining percent 1
class class-default
bandwidth remaining percent 25
dbl
|
|
ASA Connection
The ASA firewall connection to the 4500 Modular switch is fundamentally different from the other network device connections to this switch—it uses the redundant interface features of the ASA. The ASA redundant interface is a logical interface that pairs two physical interfaces, called active and standby interfaces. Under normal operation, the active interface is the only one passing traffic. The active interface uses the IP address defined at the redundant interface, and the MAC address of the first physical interface associated with the redundant interface. When the active interface fails, the standby interface becomes active and starts passing traffic. The same IP address and MAC address are maintained so that traffic is not disrupted. See Table 11-2.
Table 11-2 ASA Connection Configuration
Example 4500 Modular Switch Configuration
|
Example ASA Interface Configuration
|
interface GigabitEthernet4/4
<!-- /* Font Definitions */ @font-face
Unknown macro: {font-family}
Unknown macro: {font-family}
Unknown macro: {font-family}
/* Style Definitions */ p.MsoNormal,
li.MsoNormal, div.MsoNormal
Unknown macro: {mso-style-unhide}
Unknown macro: {mso-style-type}
-->description backup link to cr26-asa5520-DO
switchport access vlan 200
switchport mode access
switchport block unicast
load-interval 30
spanning-tree portfast
spanning-tree bpduguard enable
service-policy output EGRESS-POLICY
!
interface GigabitEthernet5/3
<!-- /* Font Definitions */ @font-face
Unknown macro: {font-family}
Unknown macro: {font-family}
Unknown macro: {font-family}
/* Style Definitions */ p.MsoNormal,
li.MsoNormal, div.MsoNormal
Unknown macro: {mso-style-unhide}
Unknown macro: {mso-style-type}
--> description Connected to cr26-asa5520-DO
switchport access vlan 200
switchport mode access
switchport block unicast
load-interval 30
media-type rj45
spanning-tree portfast
spanning-tree bpduguard enable
service-policy output EGRESS-POLICY
!
|
interface GigabitEthernet0/0
description Connected to cr24-4507-DO
no nameif
no security-level
no ip address
!
interface GigabitEthernet0/1
description backup to cr24-4507-DO
no nameif
no security-level
no ip address
!
! Defines logical redundant interface associated with
physical interfaces. Configures IP and logical interface
parameters.
interface Redundant1
description Connected to cr24-4507-DO
member-interface GigabitEthernet0/0
member-interface GigabitEthernet0/1
nameif inside
security-level 100
ip address 10.125.33.10 255.255.255.0
authentication key eigrp 100 <removed> key-id 1
authentication mode eigrp 100 md5
!
|
interface Vlan200
description cr24_4507_FW_Inside
ip address 10.125.33.9 255.255.255.0
ip authentication mode eigrp 100 md5
ip authentication key-chain eigrp 100
eigrp-key
ip pim sparse-mode
ip summary-address eigrp 100 10.125.0.0
255.255.0.0 5
logging event link-status
load-interval 30
carrier-delay msec 0
|
|
Services Block Connection
The Services Block supports the centralized servers and services for the district. The Cisco 4500 Modular switch connection to the Services Block switch uses EtherChannel, but in this case the connection between the switches is a Layer-3 connection, allowing the services block switch to keep its VLANs from stack is fundamentally the same as an access switch connection, with different VLANs. Table 11-3 provides sample configurations for the Cisco 4500 Modular switch and the Services Block switch.
Table 11-3 Service Block Configuration
Example 4500 Modular Switch Configuration
|
Example Services Block Switch Configuration
|
interface Port-channel17
description Connected to cr26-3750DC-DO
switchport
switchport trunk native vlan 806
switchport trunk allowed vlan 141-150,900
switchport mode trunk
logging event link-status
load-interval 30
carrier-delay msec 0
service-policy output PQ-POLICER
|
interface Port-channel1
description Connected to cr24-4507-DO
switchport trunk encapsulation dot1q
switchport trunk native vlan 806
switchport trunk allowed vlan 141-150,900
switchport mode trunk
logging event bundle-status
load-interval 30
carrier-delay msec 0
hold-queue 2000 in
hold-queue 2000 out
|
interface GigabitEthernet1/1
description Connected to cr24-2960-DO
switchport trunk native vlan 802
switchport trunk allowed vlan 101-110,900
switchport mode trunk
logging event link-status
load-interval 30
carrier-delay msec 0
udld port
channel-protocol pagp
channel-group 11 mode desirable
spanning-tree guard root
service-policy output EGRESS-POLICY
!
interface GigabitEthernet2/1
description Connected to cr24-2960-DO
switchport trunk native vlan 802
switchport trunk allowed vlan 101-110,900
switchport mode trunk
logging event link-status
load-interval 30
carrier-delay msec 0
udld port
channel-protocol pagp
channel-group 11 mode desirable
spanning-tree guard root
service-policy output EGRESS-POLICY
|
interface GigabitEthernet0/1
description Connected to cr24-4507-DO
switchport trunk native vlan 802
switchport trunk allowed vlan 101-110,201,900
switchport mode trunk
ip arp inspection trust
load-interval 30
srr-queue bandwidth share 1 30 35 5
priority-queue out
udld port
mls qos trust dscp
channel-protocol pagp
channel-group 1 mode desirable
hold-queue 2000 in
hold-queue 2000 out
ip dhcp snooping trust
!
interface GigabitEthernet0/2
description Connected to cr24-4507-DO
switchport trunk native vlan 802
switchport trunk allowed vlan 101-110,201,900
switchport mode trunk
ip arp inspection trust
load-interval 30
srr-queue bandwidth share 1 30 35 5
priority-queue out
udld port
mls qos trust dscp
channel-protocol pagp
channel-group 1 mode desirable
hold-queue 2000 in
hold-queue 2000 out
ip dhcp snooping trust
|
Core/Distribution Virtual Interfaces
The following is an example configuration of the Switch Virtual Interfaces configured on the core/distribution 4500 modular switch. This SVIs are trunked to the access switches as required, and access to the VLANs are controlled by the switchport trunk allowed vlan command applied on the port channels. The same basic configuration is used for the Server Switch.
interface Vlan101
description Connected to cr24_2960_Dept_1_VLAN
dampening
ip address 10.125.1.1 255.255.255.128
ip helper-address 10.125.31.2
no ip redirects
no ip unreachables
ip pim sparse-mode
load-interval 30
!
interface Vlan102
description Connected to cr24_2960_Dept_2_VLAN
dampening
ip address 10.125.1.129 255.255.255.128
ip helper-address 10.125.31.2
no ip redirects
no ip unreachables
ip pim sparse-mode
load-interval 30
!
...
interface Vlan110
description Connected to cr24_2960_Dept_10_VLAN
dampening
ip address 10.125.5.129 255.255.255.128
ip helper-address 10.125.31.2
no ip redirects
no ip unreachables
ip pim sparse-mode
load-interval 30
Table 11-4 provides examples of the port-channel configuration on core/distribution 4500 modular switch and an access switch. A similar configuration would be applied to each access switch connection with the same or different VLANs as required. From an IP routing or services level there is no requirement to span the same VLAN to multiple switches, but if there is a requirement to support legacy protocols such as AppleTalk at the school these AppleTalk VLANs can be easily spanned to different access switches as required.
Table 11-4 Core/Distribution Virtual Interfaces
Example 4500 Modular switch Port Channel Configuration
|
Example 2960 Port Channel Configuration
|
interface Port-channel11
description Connected to cr24-2960-DO
switchport
switchport trunk native vlan 802
switchport trunk allowed vlan 101-110,900
switchport mode trunk
logging event link-status
load-interval 30
carrier-delay msec 0
service-policy output PQ-POLICER
|
interface Port-channel1
description Connected to cr24-4507-DO
switchport trunk native vlan 802
switchport trunk allowed vlan 101-110,201,900
switchport mode trunk
ip arp inspection trust
load-interval 30
carrier-delay msec 0
hold-queue 2000 in
hold-queue 2000 out
ip dhcp snooping trust
|
interface GigabitEthernet1/1
description Connected to cr24-2960-DO
switchport trunk native vlan 802
switchport trunk allowed vlan 101-110,900
switchport mode trunk
logging event link-status
load-interval 30
carrier-delay msec 0
udld port
channel-protocol pagp
channel-group 11 mode desirable
spanning-tree guard root
service-policy output EGRESS-POLICY
interface GigabitEthernet2/1
description Connected to cr24-2960-DO
switchport trunk native vlan 802
switchport trunk allowed vlan 101-110,900
switchport mode trunk
logging event link-status
load-interval 30
carrier-delay msec 0
udld port
channel-protocol pagp
channel-group 11 mode desirable
spanning-tree guard root
service-policy output EGRESS-POLICY
|
interface GigabitEthernet0/1
description Connected to cr24-4507-DO
switchport trunk native vlan 802
switchport trunk allowed vlan 101-110,201,900
switchport mode trunk
ip arp inspection trust
load-interval 30
srr-queue bandwidth share 1 30 35 5
priority-queue out
udld port
mls qos trust dscp
channel-protocol pagp
channel-group 1 mode desirable
hold-queue 2000 in
hold-queue 2000 out
ip dhcp snooping trust
!
interface GigabitEthernet0/2
description Connected to cr24-4507-DO
switchport trunk native vlan 802
switchport trunk allowed vlan 101-110,201,900
switchport mode trunk
ip arp inspection trust
load-interval 30
srr-queue bandwidth share 1 30 35 5
priority-queue out
udld port
mls qos trust dscp
channel-protocol pagp
channel-group 1 mode desirable
hold-queue 2000 in
hold-queue 2000 out
ip dhcp snooping trust
|
WLC Connection
The WLC Connection to the Core/Distribution Stack is fundamentally the same as an Access Switch connection, with different VLANs, and the exception of using a different QoS trust mode, where the CoS values from the WLC, are trusted. Figure 7 shows an example of the configuration.
description Connected to WLC-SS2
switchport trunk encapsulation dot1q
switchport trunk native vlan 802
switchport trunk allowed vlan 111-120
interface GigabitEthernet1/0/48
description Connected to WLC-SS2
switchport trunk encapsulation dot1q
switchport trunk native vlan 802
switchport trunk allowed vlan 110-120
srr-queue bandwidth share 1 30 35 5
channel-group 11 mode active
interface GigabitEthernet3/0/48
description Connected to WLC-SS2
switchport trunk encapsulation dot1q
switchport trunk native vlan 802
switchport trunk allowed vlan 110-110,
srr-queue bandwidth share 1 30 35 5
channel-group 11 mode active
NAC CAS Connection
The NAC CAS connection to the core/distribution switch. This is not an EtherChannel connection, but two switch ports are consumed. The two ports consist of a untrusted port for connecting client VLANs to the CAS prior to them completing the NAC process, and a trusted port that connects the NAS to the client VLANs used once clients have successfully completed the NAC process. The two, trusted and untrusted, ports are required even if OOB NAC is used, as the CAS requires access to the trusted VLANs during the NAC process. The following is an example of the configuration.
interface GigabitEthernet 3/9
description NAC Trusted Eth0
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 48,57,62
spanning-tree portfast trunk
interface GigabitEthernet 4/9
description NAC Untrusted Eth1
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 61,248,257
spanning-tree portfast trunk
SRST Connection
The SRST connection to the core/distribution is another EtherChannel connection. The differences between the SRST connection and the access switch connections are that a trunk connection is not required, and that the SRST interfaces are router interfaces, requiring a slightly different connection. Table 11-5 provides an example of the configuration.
Table 11-5 SRST Connection
|
|
interface Port-channel1
description Connected to ISR
dampening
ip address 10.125.32.4 255.255.255.254
ip authentication mode eigrp 100 md5
ip authentication key-chain eigrp 100 eigrp-key
ip pim dr-priority 100
ip pim sparse-mode
logging event link-status
load-interval 30
carrier-delay msec 0
service-policy output PQ-POLICER
!
interface GigabitEthernet3/10
no switchport
no ip address
load-interval 30
carrier-delay msec 0
udld port
channel-protocol pagp
channel-group 1 mode desirable
service-policy output EGRESS-POLICY
!
interface GigabitEthernet4/10
no switchport
no ip address
load-interval 30
carrier-delay msec 0
udld port
channel-protocol pagp
channel-group 1 mode desirable
service-policy output EGRESS-POLICY
|
interface Port-channel3
description port-channel to 4500
ip address 10.125.32.3 255.255.255.254
hold-queue 150 in
Note: Need to add the routing information
!
interface GigabitEthernet0/0
description $ETH-LAN$$ETH-SW-LAUNCH$$INTF-INFO-GE 0/0$
no ip address
duplex auto
speed auto
media-type rj45
no keepalive
channel-group 3
!
interface GigabitEthernet0/1
no ip address
duplex auto
speed auto
media-type rj45
no keepalive
channel-group 3
|
NTP
The use of Network Time Protocol (NTP) to synchronize the clocks of network devices is a well established best practice, is fundamental for the analysis of logs/events and security, but might not warrant a mention in a design guide that is focused upon introducing new designs and practices to support new services in the network.
Given that a number of key components (for example, CUWN and Cisco NAC) of the Schools SRA rely upon or benefit from time synchronization, it was decided to include a short discussion on Time Synchronization as part of the Schools SRA.
The preferred mechanism for time synchronization in the network is NTP (other systems may use their own time synchronization mechanism) and this network NTP discussion is not proposed as a the design to synchronize all devices (hosts) in the network, its goal is synchronization of the network components of the Schools SRA. At the same time, whatever alternative times synchronization systems used in other parts of the network need to have agreement on the time, and should have a common time source at the beginning of their timing hierarchy. This will allow sufficient synchronization between hosts and network devices for the solutions deployed in the SRA.
The Schools SRA network has a hierarchy based upon the district office, as hub, and the schools as spokes. The NTP hierarchy should be the same, with the highest stratum NTP server located at the district office serving as the time reference for the district network. In order to spread the load, a hierarchy of NTP servers is used. The district office NTP server acting as the server for the district office network devices, and for the NTP server at each school, and the NTP Server for each school will act as the NTP server for network devices in that school.
The preferred network device to act as the NTP server in the schools SRA is the ISR router at each site. The ISR is the preferred device as routers have a greater CPU capacity than switches used in the Schools SRA due to many of the general purpose task that a router may be required it perform in CPU, compared to switches that have been more optimized to perform their more limited number of tasks in ASIC.
Figure 11-2 shows a schematic of the NTP hierarchy in the school district.
For more information upon NTP refer to the Network Time Protocol: Best Practices White Paper at the following URL:
http://www.cisco.com/en/US/tech/tk869/tk769/technologies_white_paper09186a0080117070.shtml
Figure 11-2 NTP School District Hierarchy
When creating the NTP configuration care should be taken to protect the NTP system.
The NTP associations should be limited, and controlled by an access list, to protect against DoS attacks. The NTP system should also use NTP authentication, where possible, to protected against spoofing attacks.
|
|
access-list 99 permit x.x.x.x 0.0.0.255
access-list 99 permit y.y.y.y 0.0.0.255
ntp authentication-key 2 md5 Riewoldt
ntp authenticate
ntp source Port-channel3
ntp max-associations 150
ntp server a.a.a.a
ntp access-group serve-only 99
|
access-list 98 permit z.z.z.z 0.0.0.255
ntp authentication-key 2 md5 Riewoldt
ntp trusted-key 2
ntp clock-period 17179685
ntp max-associations 150
ntp server <DO-ISR> key 2
ntp access-group serve-only 98
|