Configure SAN Port-Channel between UCS IMM and MDS

Introduction

This document describes the SAN port channel configuration between Fabric Interconnect 64108 managed by Intersight and a MDS 9148 16 G.

Prerequisites

Requirements

• Fabric Interconnect 64108 managed by Intersight
• MDS 9148S 16G 
• Fabric Interconnect 64108 connected to a MDS

Components

The information in this document is based on these software and hardware versions:

• Fabric Interconnect 64108    Version: 4.3(2a)

• MDS model:  MDS 9148S 16G    Version: 9.3(2)

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.

Background

SAN port channel

SAN port channels refer to the aggregation of multiple physical interfaces into one logical interface to provide higher aggregated bandwidth, load balancing, and link redundancy.

Vsan

A VSAN is a virtual storage area network (SAN). A SAN is a dedicated network that interconnects hosts and storage devices primarily to exchange SCSI traffic. In SANs you use the physical links to make these interconnections. A set of protocols run over the SAN to handle routing, naming, and zoning. You can design multiple SANs with different topologies.

Advantages

• Traffic isolation—Traffic is contained within VSAN boundaries and devices reside only in one VSAN ensuring absolute separation between user groups, if desired.
• Scalability—VSANs are overlaid on top of a single physical fabric. The ability to create several logical VSAN layers increases the scalability of the SAN.
• Per VSAN fabric services—Replication of fabric services on a per VSAN basis provides increased scalability and availability.
• Redundancy—Several VSANs created on the same physical SAN ensure redundancy. If one VSAN fails, redundant protection (to another VSAN in the same physical SAN) is configured using a backup path between the host and the device.
• Ease of configuration—Users can be added, moved, or changed between VSANs without changing the physical structure of a SAN. Moving a device from one VSAN to another only requires configuration at the port level, not at a physical level

Topology

This example shows san port channel configuration between FI managed by Intersight and MDS. Used fc1/1 and fc1/2 interfaces for Fabric Interconnect and fc1/11 and fc1/12 for MDS.

Configure

Before start with configuration.

Login into SSH session of the MDS device and login into Intersight account.

Intersight configuration

Port Policy

Step 1. Create a Port Policy.

Create Port Policy

 Step 2.  In the search field, look for port, select Port and click Start button. 

Select port policy

Step 3. In the name field, write the port policy name and select switch model (Fabric Interconnect model) and click in Next button.

Define port policy name and Fabric Interconnect model

Step 4.  Select the amount of port to carry Fiber Channel (FC) traffic. You can see blue circle once you select the FC ports and click Next button.

Ports selection

Step 5. Click in Next button.

Next button

Step 6. Find Port Channels tab and then click in Create Port Channel button.

Create port channel

Step 7. Select FC Uplink Port Channel option in Role field.

Select FC Uplink Port Channel role

Step 8.  In the Port Channel ID, write the port channel identifier, in VSAN ID write vsan identifier and select Admin Speed.

Select admin speed , portchannel ID and vsan ID

Step 9. Select the port(s) connected to the MDS to create the port channel configuration and select Save button.

Select the port(s) connected to the MDS

VSAN scope 

The roles for an FC port are:

• FC Uplink—FC traffic passes through the FC uplink port. To specify the role of an FC port as an FC Uplink port the VSAN scope of the port must have been created as Storage and Uplink, or as Uplink in the VSAN Cofiguration policy.

• FC Storage—FC port acts as a storage port. To specify the role of an FC port as an FC Storage port the VSAN scope of the port must have been created as Storage and Uplink, or as Storage in the VSAN Cofiguration policy. Moreover, the FC has to be in the switching mode.

• Unconfigured—Unconfigured is the default role of the port.

Vsan Policy

Step 1.  Select Create Policy.

Create policy

Step 2. In the search field, write vsan, select vsan and click in Start button.

Select vsan

Step 3. In field name, write the vsan policy name and click in Next button.

Define vsan policy name

Step 4. Enable Uplink trunking option and select Add VSAN.

Enable trunking and add vsan

Step 5. In the field name, write the VSAN name, in VSAN scope select uplink option and write the VSAN ID and FCoE vlan ID. Then, select Add button.

Add VSAN

Step 6. Select Create button.

Create vsan policy

UCS Domain Policy

Step 1. Select Profiles, look for UCS Domain Profiles and select Create UCS Domain Profile.

Create UCS Domain Profile

Step 2. In the field name, write the Domain Profile name and click Next button.

Define UCS domain profile

Step 3. Select the Domain Name to assign the UCS Domain Profile. Then, select Next button.

Note: Is important that Fabric Interconnect does not have a Domain Profile assigned. If is the case, you need to unassigned the UCS Domain Profile first.

Select Domain name

Step 4. In VSAN configuration of Fabric Interconnect A, click Select Policy.

Select vsan policy

Step 5. Find the VSAN policy created, select it and click Next button. 

Select vsan policy created

Step 6. Click in Select Policy in ports configuration tab.

Select port policy created

Step 7. Select the port policy configured and click Select button.

Select port policy created

Step 8. In network section, find the System QoS and click Select Policy.

Select QoS policy

Step 9. Select Create New.

Create new QoS

Step 10.  In name field, write the QoS policy name and select Next button.

Define QoS policy name

 Step 11. Modify the MTU values, select the QoS priority, and select Create.

Modify MTU and policy details

Step 12. Click Next.

Select next

Step 13. You can find a quick summary of the UCS Domain Profile configuration. Click Deploy button.

Deploy UCS Domain Profile

Note: UCS Domain Profile deployment requires the Fabric Interconnect  in the domain to be rebooted and can result in a traffic disruption through that fabric path.

MDS configuration

Open an SSH session to MDS and login as a local user. 

MDS# configure terminal 
MDS(config-if)#vsan database
MDS(config-vsan-db)#vsan 8 
MDS(config-vsan-db)#vsan 8 interface fc1/11-12
MDS(config)#interface fc1/11-12
MDS(config-if)#channel-group 8 
MDS(config-if)#no shutdown
MDS(config-if)#exit
MDS(config)#interface port-channel 8
MDS(config-if)#switchport trunk mode on
MDS(config-if)#switchport trunk allowed vsan 8
switchport trunk allowed vsan add 1
MDS(config-if)#exit
MDS(config-if)#npv enable

Verify in MDS

Useful commands: 

MDS# show npiv status
MDS# show interface brief
MDS# show fcdomain domain-list
MDS# show flogi database
MDS# show interface port-channel <id>
MDS# show flogi database
MDS# show port-channel summary
MDS# show vsan usage
MDS# show port-channel internal event-history errors
MDS# show port-channel database

Verify in UCS 

Useful commands: 

UCS# connect nxos
UCS(nx-os)# show interface brief
UCS(nx-os)# show san-portchannel summary
UCS(nx-os)# show vsan membership
UCS(nx-os)# show interface san-port-channel <id>
UCS(nx-os)# show interface fc <id>
UCS(nx-os)# show npv flogi-table
UCS(nx-os)# show vsan usage
UCS(nx-os)# show san-port-channel internal event-history errors
UCS(nx-os)# show san-port-channel database

Troubleshooting

• Verify vsan allowed in both sides match
• Check fc interfaces are up 
• Verify port channel status in both sides
• Make sure vsan are created in both sides
• Check port channel interfaces are configured as trunk
• Review npiv is enable
• Verify vsan membership
• Make sure interfaces are associated to a port channel
• Review which fc interfaces connected between Fabric Interconnect and MDS

Related info

Intersight VSAN Domain Policy configuration

Configuring Fibre Channel Interfaces

Configuring UCS Domain Profiles

Configuring Port Channels