Configuring IPStorage Services

Cisco MDS 9000 Series IPStorage (IPS) services extend the reach of Fibre Channel SANs by using open-standard, IP-based technology. The switch connects separated SAN islands using Fibre Channel over IP (FCIP) and it allows IP hosts to access Fibre Channel storage.

note.gif

Noteblank.gif The FCIP feature is available in the modules with IPS ports and available in Cisco MDS 9250i, MDS 9220i, and Cisco MDS 9700 Director switches.

The Cisco MDS 24/10 Port SAN Extension Module for MDS 9700 series switch allows you to use Fibre Channel and FCIP features. The Cisco MDS 24/10 port SAN Extension Module can be used in any of the Cisco MDS 9700 series switches.

This chapter includes the following sections:

Feature Information

This section briefly describes the new and updated features for releases, starting from Cisco MDS NX-OS Release 8.x.

Table 6-1 Feature Information Table
Feature
Release
Description

25-Gbps speed is supported from Cisco MDS NX-OS Release 9.3(1).

9.3(1)

This feature allows you to configure 40-Gbps speed on the IPS ports of Cisco MDS 24/10 port SAN Extension Module.

Configuring IPS Ports on Cisco MDS 9220i Switch

8.5(1)

Support for Cisco MDS 9220i Fabric Switch was added.

Changing Link Speed on Cisco MDS 9220i Switch

7.3(0)DY(1)

This feature enables users to change the link speed on IPStorage interfaces between 1 Gbps and 10 Gbps on the Cisco MDS 24/10 port SAN Extension Module.

Changing Link Speed on Cisco MDS 9250i Multiservice Fabric Switch

6.2(13)

This feature enables users to change the link speed on IPStorage interfaces between 1 Gbps and 10 Gbps on the Cisco MDS 9250i Multiservice Fabric Switch.

Supported Hardware

You can configure the FCIP and iSCSI features on the following types of hardware:

  • Cisco MDS 24/10 port SAN Extension Module in Cisco MDS 9700 Series Director Switches and Cisco MDS 9220i Fabric Switches support FCIP.
  • Cisco MDS 9250i Multiservice Fabric Switches support iSCSI.

Upgrading FCIP and iSCSI Interfaces

ISSU/D is disruptive for FCIP and iSCSI interfaces along with the update of the underlying IPS and GigabitEthernet interfaces supporting these upper layer interfaces. Fibre Channel interfaces (including FC interfaces on the same module or switch as the IPS and GigabitEthernet interfaces), FCoE Ethernet interfaces, and supervisor modules are all updated nondisruptively.

Within Cisco MDS 9700 Directors, the modules with IPS ports implement a 5 minute delay before the next module with IPS ports is upgraded.

Configuring IPS Ports on Cisco MDS 9220i Switch

Starting from Cisco MDS NX-OS Release 8.5(1), the Cisco MDS 9220i Fabric Switch is supported. Cisco MDS 9220i switch supports six IPS ports. Ports 1 through 4 support 1- and 10-Gbps speed and port 6 supports 40-Gbps speed. By default, the first two ports are operating at a 1-Gbps speed mode and are licensed. To enable the other ports and different speed modes, you must obtain a license.You cannot move the default license from the first two ports to any other ports. For information on port licensing, see the Cisco MDS 9000 Series Licensing Guide, Release 8.x.

Cisco MDS 9220i switch supports three 1 or 10 Gbps IPS ports, one 1, 10, 25 Gbps IPS port, one 25 Gbps IPS port and one 40 Gbps IPS port. All these ports are handled by a single Service Engine. Table 6-2 provides the different speed combinations that can be configured on the IPS ports of Cisco MDS 9220i switch. For configuring the link speed, see Changing Link Speed on Cisco MDS 9220i Switch.

Table 6-2 IPS Port Speed Configuration Matrix for Cisco MDS 9220i Switch
Speed mode
Port 1
Port 2
Port 3
Port 4
Port 5
Port 6

1 Gbps

1 Gbps

1 Gbps

1 Gbps

1 Gbps

Out-of-service

Out-of-service

10 Gbps

10 Gbps

10 Gbps

10 Gbps

10 Gbps

Out-of-service

Out-of-service

25 Gbps

Out-of-service

Out-of-service

Out-of-service

25 Gbps

25 Gbps

Out-of-service

40 Gbps

Out-of-service

Out-of-service

Out-of-service

Out-of-service

Out-of-service

40 Gbps
note.gif

Noteblank.gif 25-Gbps speed is supported from Cisco MDS NX-OS Release 9.3(1).

Configuring 40 Gbps Speed on Cisco MDS 24/10 port SAN Extension Module

From Cisco MDS NX-OS Release 8.5(1), you can configure 40 Gbps on the IPS ports 9 and 10 of Cisco MDS 24/10 port SAN Extension Module. Cisco MDS 24/10 port SAN Extension Module supports eight 1 or 10 Gbps IPS ports and two 40 Gbps IPS ports. Ports 1 through 4 and port 9 and handled by Service Engine 1 and Ports 5 through 8 and port 10 and handled by Service Engine 2. Each Service Engine has the capability of 40 Gbps. You can only configure 40 Gbps speed on ports 9 and 10. When you configure 40 Gbps speed on port 9, ports 1 through 4 are moved to out-of-service because the Service Engine capability is 40 Gbps. However, ports 5 through 8 can be configured for 1 or 10 Gbps speed provided port 10 is not configured for 40 Gbps speed. Similarly, when you configure 40 Gbps speed on port 10, ports 5 through 8 are moved to out-of-service. However, ports 1through 4 can be configured for 1 or 10 Gbps speed provided port 9 is not configured for 40 Gbps speed. Table 6-3 provides the different speed combinations that can be configured on the IPS ports of Cisco MDS 24/10 port SAN Extension Module. For configuring the link speed, see Changing Link Speed on Cisco MDS 9220i Switch.

The following are the recommended configurations for achieving maximum throughput on the 40 Gbps IPS port:

  • The maximum bandwidth of FCIP tunnel is 10 Gbps. Therefore, configure four FCIP tunnels on the 40 Gbps IPS port. To create the FCIP tunnels, create four VLAN sub interfaces on the 40 Gbps IPS port. On each VLAN create an FCIP tunnel.
  • Configure the number of TCP connections to 5.

 

Table 6-3 IPS Port Speed Configuration Matrix for Cisco MDS 24/10 port SAN Extension Module
Service Engine 1
Service Engine 2
Ports 1 through 4
Port 9
Ports 5 through 8
Port 10

1 Gbps

Out-of-service

1 Gbps

Out-of-service

10 Gbps

Out-of-service

10 Gbps

Out-of-service

1 Gbps

Out-of-service

10 Gbps

Out-of-service

10 Gbps

Out-of-service

1 Gbps

Out-of-service

Out-of-service

40 Gbps

1 Gbps

Out-of-service

Out-of-service

40 Gbps

10 Gbps

Out-of-service

1 Gbps

Out-of-service

Out-of-service

40 Gbps

10 Gbps

Out-of-service

Out-of-service

40 Gbps

Out-of-service

40 Gbps

Out-of-service

40 Gbps

Configuring IPStorage Interfaces for IPv4

Both FCIP and iSCSI rely on TCP/IP for network connectivity. On each Fibre Channel module with IPS ports, connectivity is provided in the form of IP storage ports on Cisco MDS 9250i switches and Cisco MDS 9700 series switches with 24/10 port SAN Extension modules that are appropriately configured. This section covers the steps required to configure IP for subsequent use by FCIP and iSCSI.

A new port mode, called IPS, is defined for IP storage ports on each Fibre Channel module with IPS ports. IP storage ports are implicitly set to IPS mode, so it can only be used to perform iSCSI and FCIP storage functions. IP storage ports do not bridge Ethernet frames or route other IP packets.

Each IPS port represents a single virtual Fibre Channel host in the Fibre Channel SAN. All the iSCSI hosts connected to this IPS port are merged and multiplexed through the single Fibre Channel host.

In large scale iSCSI deployments where the Fibre Channel storage subsystems require explicit LUN access control for every host device, use of proxy-initiator mode simplifies the configuration.

note.gif

Noteblank.gif To configure IPv6 on an IPStorage interface, see the Cisco Fabric Manager Security Configuration Guide. For information about configuring FCIP, see Chapter2, “Configuring Fibre Channel over IP” For information about configuring iSCSI, see Chapter4, “Configuring Internet Small Computer Systems Interface”

tip.gif

Tipblank.gif IPStorage ports on any Fibre Channel module with IPS ports should not be configured in the same IPS broadcast domain as the management IPS port—they should be configured in a different broadcast domain, either by using separate standalone hubs or switches or by using separate VLANs.

Basic IPStorage Configuration

Figure 6-1 shows an example of a basic IPStorage IP version 4 (IPv4) configuration.

Figure 6-1 IPStorage IPv4 Configuration Example

 

ips-5.jpg

 

note.gif

Noteblank.gif The port on the Ethernet switch to which the IPStorage interface is connected should be configured as a host port (also known as access port) instead of a switch port. Spanning tree configuration for that port (on the Ethernet switch) should disabled. This helps avoid the delay in the management port coming up due to delay from Ethernet spanning tree processing that the Ethernet switch would run if enabled. For Cisco Ethernet switches, use either the switchport host command in Cisco IOS or the set port host command in Catalyst OS.

Fibre Channel Module with IPS Ports Core Dumps

IPS core dumps are different from the system’s kernel core dumps for other modules. When the Fibre Channel module with IPS port’s operating system (OS) unexpectedly resets, it is useful to obtain a copy of the memory image (called a IPS core dump) to identify the cause of the reset. Under that condition, the Fibre Channel module with IPS ports sends the core dump to the supervisor module for storage. Cisco MDS switches have two levels of IPS core dumps:

  • Partial core dumps (default)—Each partial core dump consists of four parts (four files). All four files are saved in the active supervisor module.

In Cisco MDS 9700 Series Switches with 24/10 port SAN Extension Modules, each partial core dump consists of five parts (five files). All five files are saved in the active supervisor module.

Use the show cores command to list these files.

  • Full core dumps—Each full core dump of Cisco MDS 9250i Switches and SSN-16 modules consists of 64 parts (64 files), and each full core dump of Cisco MDS 9700 Series Switches with 24/10 port SAN Extension Modules consists of 67 parts (67 files). The IPS core dump for MSM-18/4 modules consists of 32 parts. This dump cannot be saved on the supervisor module because of its large space requirement. They are copied directly to an external TFTP server.

Use the system cores tftp: command to configure an external TFTP server to copy the IPS core dump (and other core dumps).

To configure IPS core dumps on the Fibre Channel module with IPS ports, follow these steps:

Command
Purpose

Step 1

switch# config terminal

switch(config)#

Enters configuration mode.

Step 2

switch(config)# ips core dump full

ips core dump full’ successfully set for module 9

Configures a dump of the full core generation for all Fibre Channel module with IPS ports in the switch.

switch(config)# no ips core dump full

ips core dump partial’ successfully set for module 9

Configures a dump of the partial core (default) generation for the Fibre Channel module with IPS ports in slot 9.

 

Configuring Interface Descriptions

See the Cisco Fabric Manager Interfaces Configuration Guide for details on configuring the switch port description for any interface.

Configuring Beacon Mode

See the Cisco Fabric Manager Interfaces Configuration Guide for details on configuring the beacon mode for any interface.

Configuring Autonegotiation

By default, autonegotiation is enabled on all the IPStorage interface. You can enable or disable autonegotiation for a specified IPStorage interface. When autonegotiation is enabled, the port automatically detects the speed or pause method, and duplex of incoming signals based on the link partner. You can also detect link up conditions using the autonegotiation feature.

Configuring the MTU Frame Size

You can configure the interfaces on a switch to transfer large (or jumbo) frames on a port. The default IP maximum transmission unit (MTU) frame size is 1500 bytes for all Ethernet ports. By configuring jumbo frames on a port, the MTU size can be increased up to 9000 bytes.

note.gif

Noteblank.gif The minimum MTU size is 576 bytes.

tip.gif

Tipblank.gif MTU changes are disruptive, all FCIP links and iSCSI sessions flap when the software detects a change in the MTU size.

Configuring Promiscuous Mode

You can enable or disable promiscuous mode on a specific IPStorage interface. By enabling the promiscuous mode, the IPStorage interface receives all the packets and the software then filters and discards the packets that are not destined for that IPStorage interface.

About VLANs for IPStorage

Virtual LANs (VLANs) create multiple virtual Layer 2 networks over a physical LAN network. VLANs provide traffic isolation, security, and broadcast control.

IPStorage ports automatically recognize Ethernet frames with IEEE 802.1Q VLAN encapsulation. If you need to have traffic from multiple VLANs terminated on one IPStorage port, configure subinterfaces—one for each VLAN.

If the Fibre Channel module with IPS ports or MPS-14/2 module is connected to a Cisco Ethernet switch, and you need to have traffic from multiple VLANs coming to one IPS port, verify the following requirements on the Ethernet switch:

  • The encapsulation is set to 802.1Q and not ISL, which is the default.

Use the VLAN ID as a subscription to the IPStorage interface name to create the subinterface name: <slot-number> / <port-number>. <VLAN-ID>.

Interface Subnet Requirements

IPStorage interfaces (major), subinterfaces (VLAN ID), and management interfaces (mgmt 0) can be configured in the same or different subnet depending on the configuration (see Table 6-4 ).

 

Table 6-4 Subnet Requirements for Ethernet Interfaces Within the Same Switch

Interface 1
Interface 2
Same Subnet
Allowed
Notes

IPStorage 1/1

IPStorage 1/2

Yes1

Two major interfaces can be configured in the same or different subnets.

IPStorage 1/1.100

IPStorage 1/2.100

Yes 1

Two subinterfaces with the same VLAN ID can be configured in the same or different subnets.

IPStorage 1/1.100

IPStorage 1/2.200

No

Two subinterfaces with different VLAN IDs cannot be configured in the same subnet.

IPStorage 1/1

IPStorage 1/1.100

No

A subinterface cannot be configured on the same subnet as the major interface.

mgmt0

IPStorage 1/1.100

No

The mgmt0 interface cannot be configured in the same subnet as the IPStorage interfaces or subinterfaces.

mgmt0

IPStorage 1/1

No

1.Except IPSec enabled interfaces, which should not be configured in the same IP subnet.
note.gif

Noteblank.gif The configuration requirements in Table 6-4 also apply to Ethernet PortChannels.

Verifying IPStorage Connectivity

Once the IPStorage interfaces are connected with valid IP addresses, verify the interface connectivity on each switch. Ping the IP host using the IP address of the host to verify that the static IP route is configured correctly.

note.gif

Noteblank.gif If the connection fails, verify the following, and ping the IP host again:
- The IP address for the destination (IP host) is correctly configured.
- The host is active (powered on).
- The IP route is configured correctly.
- The IP host has a route to get to the IPStorage interface subnet.
- The IPStorage interface is in the up state.

IPStorage IPv4-ACL Guidelines

tip.gif

Tipblank.gif If IPv4-ACLs are already configured in a IPStorage interface, you cannot add this interface to an Ethernet PortChannel group.

Follow these guidelines when configuring IPv4-ACLs for IPStorage interfaces:

  • Only use Transmission Control Protocol (TCP) or Internet Control Message Protocol (ICMP).
note.gif

Noteblank.gif Other protocols such as User Datagram Protocol (UDP) and HTTP are not supported in IPStorage interfaces. Applying an ACL that contains rules for these protocols to a IPStorage interface is allowed but those rules have no effect.

  • Apply IPv4-ACLs to the interface before you enable an interface. This ensures that the filters are in place before traffic starts flowing.
  • Be aware of the following conditions:

–blank.gif If you use the log-deny option, a maximum of 50 messages are logged per second.

–blank.gif The established, precedence, and fragments options are ignored when you apply IPv4-ACLs (containing these options) to IPStorage interfaces.

–blank.gif If an IPv4-ACL rule applies to a preexisting TCP connection, that rule is ignored. For example if there is an existing TCP connection between A and B, and an IPv4-ACL specifies dropping all packets whose source is A and destination is B is subsequently applied, it will have no effect.

Configuring IPStorage High Availability

Virtual Router Redundancy Protocol (VRRP) and Ethernet PortChannels are two IPStorage features that provide high availability for iSCSI and FCIP services.

VRRP for iSCSI and FCIP Services

VRRP provides a redundant alternate path to the IPStorage port for iSCSI and FCIP services. VRRP provides IP address failover protection to an alternate IPStorage interface so the IP address is always available (see Figure 6-2).

Figure 6-2 VRRP Scenario

 

ips-10.jpg

In Figure 6-2, all members of the VRRP group must be IPStorage ports. VRRP group members can be one or more of the following interfaces:

  • One or more interfaces in the same Fibre Channel module with IPS ports or MSM-18/4 module
  • Interfaces across Fibre Channel module with IPS ports or MSM-18/4 modules in one switch
  • Interfaces across Fibre Channel module with IPS ports or MSM-18/4 modules in different switches
  • IPStorage subinterfaces
  • Ethernet PortChannels and PortChannel subinterfaces
note.gif

Noteblank.gif You can configure no more than seven VRRP groups, both IPv4 and IPv6, on a IPStorage interface, including the main interface and all subinterfaces.

Configuring VRRP for IPStorage Interfaces

To configure VRRP for IPStorage interfaces using IPv4, follow these steps:

Command
Purpose

Step 1

switch1# config terminal

switch1(config)#

Enters configuration mode.

Step 2

switch(config)# interface IPStorage 2/2

switch(config-if)#

Enters the interface configuration mode on the IPStorage interface (slot 2, port 2).

Step 3

switch(config-if)# ip address 10.1.1.10 255.255.255.0

Assigns the IPv4 address (10.1.1.10) and subnet mask (255.255.255.0) for the IPStorage interface.

Step 4

switch(config-if)# no shutdown

Enables the selected interface.

Step 5

switch(config-if)# vrrp 100

switch(config-if-vrrp)

Creates VR ID 100.

Step 6

switch(config-if-vrrp) # address 10.1.1.100

Configures the virtual IPv4 address (10.1.1.100) for the selected VRRP group (identified by the VR ID).

Note The virtual IPv4 address must be in the same subnet as the IPv4 address of the IPStorage interface. All members of the VRRP group must configure the same virtual IPv4 address.

Step 7

switch(config-if-vrrp) # priority 10

Configures the priority for the selected interface within this VRRP group.

Note The interface with the highest priority is selected as the master.

Step 8

switch(config-if-vrrp) # no shutdown

Enables the VRRP protocol on the selected interface.

To configure VRRP for IPStorage interfaces using IPv6, follow these steps:

Command
Purpose

Step 1

switch1# config terminal

switch1(config)#

Enters configuration mode.

Step 2

switch(config)# interface IPStorage 2/2

switch(config-if)#

Enters the interface configuration mode on the IPStorage interface (slot 2, port 2).

Step 3

switch(config-if)# ipv6 address 2001:0db8:800:200c::417a/64

Assigns the IPv6 address for the IPStorage interface.

Step 4

switch(config-if)# no shutdown

Enables the selected interface.

Step 5

switch(config-if)# vrrp ipv6 100

switch(config-if-vrrp-ipv6)

Creates VR ID 100.

Step 6

switch(config-if-vrrp-ipv6) # address 2001:0db8:800:200c::417a

Assigns single primary link-local IPv6 address or one of the multiple secondary IPv6 addresses.

Note If this IPv6 address is the same as the physical IPv6 address, this switch is automatically the owner of this IPv6 address.

Step 7

switch(config-if-vrrp-ipv6) # priority 10

Configures the priority for the selected interface within this VRRP group.

Note The interface with the highest priority is selected as the master.

Step 8

switch(config-if-vrrp-ipv6) # no shutdown

Enables the VRRP protocol on the selected interface.
note.gif

Noteblank.gif The VRRP preempt option is not supported on IPS interfaces. However, if the virtual IPv4 IP address is also the IPv4 IP address for the interface, then preemption is implicitly applied.

Configuring CDP

The Cisco Discovery Protocol (CDP) is supported on the management Ethernet interface on the supervisor module and the IPStorage interfaces on the Fibre Channel module with IPS ports or MSM-18/4 module.

For information about configuring CDP, refer to the Cisco MDS 9000 Series NX-OS Fundamentals Configuration Guide.

Changing Link Speed on IPStorage Interfaces

Changing Link Speed on Cisco MDS 9250i Multiservice Fabric Switch

The Cisco MDS 9250i Multiservice Fabric Switch has two IPStorage interfaces that support 1 Gbps and 10 Gbps link speeds. By default, IPStorage interfaces are configured at 10 Gbps link speed.

note.gif

Noteblank.gif Switching between different link speeds is supported on Cisco 10 Gbps IPStorage platforms starting from Cisco MDS NX-OS Release 6.2(13). An ISSD to a release earlier than Cisco MDS NX-OS Release 6.2(13) when any of the IPStorage ports are configured at 1 Gbps, is disallowed. Reconfigure such ports back to the default link speed of 10 Gbps before attempting such a downgrade.

To configure 1 Gbps link speed on an IPStorage interface, follow these steps:

Command
Purpose

Step 1

switch1# config terminal

switch1(config)#

Enters configuration mode.

Step 2
switch(config)# interface IPStorage slot-number/port-number-range

switch(config-if)#

Enters IPStorage interface configuration mode.

Step 3

switch(config-if)# shutdown

Administratively disables the interface and stops traffic through the interface.

Step 4

switch(config-if)# switchport speed 10002

This speed change will disrupt FCIP/iSCSI traffic for 5 mins on all IPStorage ports. If FCIP tunnels are configured please make sure max-bw <= 1000 Mbps and tcp-connections set to 2. Do you want to continue(y/n) ? [n]

Sets the link speed of the interface and all subinterfaces to 1000 Mbps (1 Gbps).

Note

This command causes all IPStorage ports on the selected FCIP engine to be reset. This may cause traffic disruption for up to 5 minutes. By default, n is selected. Press Enter to abort the command. Enter y and press Enter to continue. 3

Step 5

switch(config-if) # no shutdown

Administratively enables the interface.

Step 6

switch(config-if)# end

switch#

Exits IPStorage interface configuration mode and returns to privileged EXEC mode.

Step 7

switch# show ips status

 

Displays the operational speed of the IPStorage port.

2.Configuring the link speed of an interface generates the following port software failure syslog message:

%IF_DOWN_SOFTWARE_FAILURE: %$VSAN 1%$ Interface fcip is down (Port software failure)

3.If the conditions specified in the warning message are not met, the configured link speed is still applied. However, issues such as packet drops, retransmissions, and FCIP tunnel flaps may occur.

To configure 10 Gbps link speed on an IPStorage interface, follow these steps:

Command
Purpose

Step 1

switch1# config terminal

switch1(config)#

Enters configuration mode.

Step 2
switch(config)# interface IPStorage slot-number/port-number-range

switch(config-if)#

Enters IPStorage interface configuration mode.

Step 3

switch(config-if)# shutdown

Administratively disables the interface and stops traffic through the interface.

Step 4

switch(config-if)# switchport speed 100004

“This speed change will disrupt FCIP/iSCSI traffic for 5 mins on all IPStorage ports. Do you want to continue(y/n) ? [n]”

Sets the link speed of the interface and all subinterfaces to 10000 Mbps (10 Gbps).

Note

This command causes all IPStorage ports on the selected FCIP engine to be reset. This may cause traffic disruption for up to 5 minutes. By default, n is selected. Press Enter to abort the command. Enter y and press Enter to continue. 5

Step 5

switch(config-if) # no shutdown

Administratively enables the interface.

Step 6

switch(config-if)# end

switch#

Exits IPStorage interface configuration mode and returns to privileged EXEC mode.

Step 7

switch# show ips status

 

Displays the operational speed of the IPStorage port.

4.Configuring the link speed of an interface generates the following port software failure syslog message:

%IF_DOWN_SOFTWARE_FAILURE: %$VSAN 1%$ Interface fcip is down (Port software failure)

5.If the conditions specified in the warning message are not met, the configured link speed is still applied. However, issues such as packet drops, retransmissions, and FCIP tunnel flaps may occur.

 

If there is a mismatch between the configured link speed and the small form-factor pluggable (SFP) speed capabilities, the port goes into an Error Disabled state and a corresponding syslog message is logged. In such a scenario, either the configured link speed or the SFP should be changed. If the link speed is changed, even if the port is already enabled, the shutdown and no shutdown commands must be explicitly issued for the change to be applied.

For more information about supported 1 Gbps SFPs for a Cisco MDS 9250i Multiservice Fabric Switch, see the Cisco MDS 9000 Family Pluggable Transceivers Data Sheet.

For information about configuring FCIP tunnels with IPStorage interfaces at 1 Gbps speed, see the Configuring Fibre Channel over IP chapter.

Changing Link Speed on Cisco MDS 9220i Switch

note.gif

Noteblank.gif For Cisco MDS 9220i switch, delete all the FCIP related configurations, such as profiles, tunnels, and so on, before switching from 1 or 10 Gbps speed to 40 Gbps speed or vice versa.

To configure 1 Gbps link speed on an IPStorage interface, follow these steps:

Command
Purpose

Step 1

switch1# config terminal

switch1(config)#

Enters configuration mode.

Step 2

switch(config)# interface IPStorage 1/1-6

Enters IPStorage interface configuration mode.

Step 3

switch(config-if)# 1G-speed-mode

This speed change will disrupt FCIP/iSCSI traffic for 60 seconds on selected IPStorage ports.If FCIP tunnels are configured please make sure max-bw <= 1000 Mbps and tcp-connections set to 2.

Do you wish to continue(y/n)? [n]

Sets the link speed of the interface IPS1/1-4 and all subinterfaces to 1000 Mbps (1 Gbps) and administratively enables the interface.

Sets IPS1/5-6 out-of-service.

Step 4

switch(config-if)# end

switch#

Exits IPStorage interface configuration mode and returns to privileged EXEC mode.

To configure 10 Gbps link speed on an IPStorage interface, follow these steps:

Command
Purpose

Step 1

switch1# config terminal

switch1(config)#

Enters configuration mode.

Step 2

switch(config)# interface IPStorage 1/1-6

Enters IPStorage interface configuration mode.

Step 3

switch(config-if)# 10G-speed-mode

This speed change will disrupt FCIP/iSCSI traffic for 60 seconds on select IPStorage ports.

Do you wish to continue(y/n)? [n]

Sets the link speed of the interface IPS1/1-4 and all subinterfaces to 10000 Mbps (10 Gbps) and administratively enables the interface.
Sets IPS1/5-6 out-of-service.

Step 4

switch(config-if)# end

switch#

Exits IPStorage interface configuration mode and returns to privileged EXEC mode.

To configure 25 Gbps link speed on an IPStorage interface, follow these steps:

Command
Purpose

Step 1

switch1# configure

Enters configuration mode.

Step 2

switch(config)# interface IPStorage 1/1-6

 

Enters IPStorage interface configuration mode.

note.gif

Noteblank.gif Configure the number of TCP connections to 5 for attaining maximum throughput.

Step 3

switch(config-if)# 25G-speed-mode

Sets the link speed of the interface IPS1/4-5 to 25000 Mbps (25 Gbps) and administratively enables the interface IPS1/4-5.
Sets IPS1/1-3 and IPS1/6 to out-of-service.

Step 4

switch(config-if)# end

switch#

Exits IPStorage interface configuration mode and returns to privileged EXEC mode.

To configure 40 Gbps link speed on an IPStorage interface, follow these steps:

Command
Purpose

Step 1

switch1# configure

Enters configuration mode.

Step 2

switch(config)# interface IPStorage 1/1-6

 

Enters IPStorage interface configuration mode.

note.gif

Noteblank.gif Configure the number of TCP connections to 5 for attaining maximum throughput.

Step 3

switch(config-if)# 40G-speed-mode

Sets the link speed of the interface IPS1/6 to 40000 Mbps (40 Gbps) and administratively enables the interface. Sets IPS1/1-5 to out-of-service.

Step 4

switch(config-if)# end

switch#

Exits IPStorage interface configuration mode and returns to privileged EXEC mode.

Changing Link Speed on Cisco MDS 24/10 port SAN Extension Module

note.gif

Noteblank.gif For Cisco MDS 24/10 port SAN Extension Module, delete all the FCIP related configurations, such as profiles, tunnels, and so on, before switching from 1 or 10 Gbps speed to 40 Gbps speed or vice versa.

To configure 1 Gbps link speed on an IPStorage interface, follow these steps:

Command
Purpose

Step 1

switch1# config terminal

switch1(config)#

Enters configuration mode.

Step 2

Releases prior to Cisco MDS NX-OS Release 8.5(1):

Note The values for port-number-range can be 1-4 or 5-8.

switch(config)# interface IPStorage slot-number/port-number-range

switch(config-if)#

 

Cisco MDS NX-OS Release 8.5(1) or later releases:

Note The values for port-number-range can be 1-4 or 5-8. The value for port number can be 9 if port-number-range is 1-4 and the value for port number can be 10 if port-number-range is 5-8.

switch(config)# interface IPStorage slot-number/port-number-range, IPStorage slot-number/port-number

Enters IPStorage interface configuration mode.

Step 3

switch(config-if)# 1G-speed-mode

This speed change will disrupt FCIP/iSCSI traffic for 60 seconds on selected IPStorage ports.If FCIP tunnels are configured please make sure max-bw <= 1000 Mbps and tcp-connections set to 2.

Do you wish to continue(y/n)? [n]

Sets the link speed of the interface and all subinterfaces to 1000 Mbps (1 Gbps) and administratively enables the interface.

Step 4

switch(config-if)# end

switch#

Exits IPStorage interface configuration mode and returns to privileged EXEC mode.

To configure 10 Gbps link speed on an IPStorage interface, follow these steps:

Command
Purpose

Step 1

switch1# config terminal

switch1(config)#

Enters configuration mode.

Step 2

Releases prior to Cisco MDS NX-OS Release 8.5(1):

Note The values for port-number-range can be 1-4 or 5-8.

switch(config)# interface IPStorage slot-number/port-number-range

switch(config-if)#

 

Cisco MDS NX-OS Release 8.5(1) or later releases:

Note The values for port-number-range can be 1-4 or 5-8. The value for port number can be 9 if port-number-range is 1-4 and the value for port number can be 10 if port-number-range is 5-8.

switch(config)# interface IPStorage slot-number/port-number-range, IPStorage slot-number/port-number

Enters IPStorage interface configuration mode.

Step 3

switch(config-if)# 10G-speed-mode

This speed change will disrupt FCIP/iSCSI traffic for 60 seconds on select IPStorage ports.

Do you wish to continue(y/n)? [n]

Sets the link speed of the interface and all subinterfaces to 10000 Mbps (10 Gbps) and administratively enables the interface.

Step 4

switch(config-if)# end

switch#

Exits IPStorage interface configuration mode and returns to privileged EXEC mode.

To configure 40 Gbps link speed on an IPStorage interface, follow these steps:

Command
Purpose

Step 1

switch1# configure

Enters configuration mode.

Step 2
switch(config)# interface IPStorage slot-number/port-number-range, IPStorage slot-number/port-number

Note The values for port-number-range can be 1-4 or 5-8. The value for port number can be 9 if port-number-range is 1-4 and the value for port number can be 10 if port-number-range is 5-8.

Enters IPStorage interface configuration mode.

Step 3

switch(config-if)# 40G-speed-mode

Sets the link speed of the interface to 40 Gbps and interfaces in the range to out-of-service.

Displaying Statistics

This section provides examples to verify IPStorage interfaces and TCP/IP statistics on the IPStorage ports.

Use the show interface IPStorage command on each switch to verify that the interfaces are up and functioning as desired. See Example 6-1.

Example 6-1 Displaying the IPStorage Interface

switch# show interface IPStorage 4/1
IPStorage4/1 is up
Hardware is IPStorage, address is 008e.7339.39e7
Internet address is 10.197.141.81/24
MTU 2500 bytes
Port mode is IPS
Speed is 10 Gbps
Beacon is turned off
Auto-Negotiation is turned on
5 minutes input rate 77012744 bits/sec, 9626593 bytes/sec, 112755 frames/sec
5 minutes output rate 2762915176 bits/sec, 345364397 bytes/sec, 175258 frames/sec
71187036 packets input, 6078261484 bytes
0 multicast frames, 0 compressed
0 input errors, 0 frame, 0 overrun 0 fifo
110617842 packets output, 217860230652 bytes, 0 underruns
0 output errors, 0 collisions, 0 fifo
0 carrier errors
 
 
 
 
switch# show interface IPStorage 5/1-10 brief
 
--------------------------------------------------------------------------------
Interface Status IP Address Speed MTU
--------------------------------------------------------------------------------
IPStorage5/1 up 1.1.1.1/24 10 Gbps 2500
IPStorage5/2 up 2.2.2.2/24 10 Gbps 2500
IPStorage5/3 up 3.3.3.3/24 10 Gbps 2500
IPStorage5/4 up 4.4.4.4/24 10 Gbps 2500
IPStorage5/5 up 6811::3456/64 1 Gbps 2300
IPStorage5/6 up 9.9.9.1/24 1 Gbps 2500
IPStorage5/7 up 7.7.7.1/24 1 Gbps 2500
IPStorage5/8 up 8.8.8.1/24 1 Gbps 2500
IPStorage5/9 outOfServc -- auto 1500
IPStorage5/10 outOfServc -- auto 1500
note.gif

Noteblank.gif In Cisco MDS NX-OS Release 7.3(0)DY(1), 40GE IPStorage interfaces are not supported.

Displaying Ethernet MAC Statistics

The show ips stats mac interface ips command takes the IPStorage interface as a parameter and returns the statistics for that interface. See Example 6-2.

note.gif

Noteblank.gif Use the physical interface, not the subinterface, to display Ethernet MAC statistics.

Example 6-2 Displaying IPStorage Interface MAC Statistics

switch# show ips stats mac interface ips 4/1
 
DPP HW IPStorage port 4 (octeon port 19) statistics
 
Rx stats : 
  dropped         : 0          octs,   0          pkts 
  l2 red pkts     : 0          mcast,  0          bcasts
  l3 red pkts     : 0          mcast,  0          bcasts
  oversize        : 0          pkts,   0          crcpkts
  runt            : 0          pkts,   0          crcpkts
  inband          : 0          octs,   0          pkts,      0      err
  pci raw         : 0          pkts
  fcs_align_err   : 0          pkts
 
  total           : 0          octs,   0          pkts
 
  length of [pkts]:- 
   [64B]          : 0            [65B-127B]       : 0           
   [128B-255B]    : 0            [256B-511B]      : 0         
   [512B-1023B]   : 0            [1024B-1518B]    : 0         
   [1519B-MAX]    : 0         
 
Tx stats : 
  total           : 0          octs,   0          pkts
  dropped         : 0          octs,   0          pkts
 
Hw-intf stats : 
  total           : 0            Rx_octs,   0            Rx_pkts
  dropped         : 0            Rx_pkts
  total           : 0            Tx_octs,   0            Tx_pkts
  dropped         : 0            Tx_pkts
 

Displaying TCP Statistics

Use the show ips stats tcp interface ips to display and verify TCP statistics. This command takes the main Ethernet interface as a parameter, and shows TCP stats along with the connection list and TCP state. The detail option shows all information maintained by the interface. See Example 6-3 and Example 6-4.

Example 6-3 Displaying TCP Statistics

switch# show ips stats tcp interface ips 4/1
 
TCP statistics for port IPStorage4/1
Connection Stats
0 active openings, 3 accepts
0 failed attempts, 12 reset received, 3 established
Segment stats
163 received, 355 sent, 0 retransmitted
0 bad segments received, 0 reset sent
TCP Active Connections
Local Address Remote Address State Send-Q Recv-Q
0.0.0.0:3260 0.0.0.0:0 LISTEN 0 0
 

Example 6-4 Displaying Detailed TCP Statistics

switch# show ips stats tcp interface ips 4/1 detail
 
TCP Statistics for port IPStorage4/1
TCP send stats
355 segments, 37760 bytes
222 data, 130 ack only packets
3 control (SYN/FIN/RST), 0 probes, 0 window updates
0 segments retransmitted, 0 bytes
0 retransmitted while on ethernet send queue, 0 packets split
0 delayed acks sent
TCP receive stats
163 segments, 114 data packets in sequence, 6512 bytes in sequence
0 predicted ack, 10 predicted data
0 bad checksum, 0 multi/broadcast, 0 bad offset
0 no memory drops, 0 short segments
0 duplicate bytes, 0 duplicate packets
0 partial duplicate bytes, 0 partial duplicate packets
0 out-of-order bytes, 1 out-of-order packets
0 packet after window, 0 bytes after window
0 packets after close
121 acks, 37764 ack bytes, 0 ack toomuch, 4 duplicate acks
0 ack packets left of snd_una, 0 non-4 byte aligned packets
8 window updates, 0 window probe
30 pcb hash miss, 0 no port, 0 bad SYN, 0 paws drops
TCP Connection Stats
0 attempts, 3 accepts, 3 established
3 closed, 2 drops, 0 conn drops
0 drop in retransmit timeout, 1 drop in keepalive timeout
0 drop in persist drops, 0 connections drained
TCP Miscellaneous Stats
115 segments timed, 121 rtt updated
0 retransmit timeout, 0 persist timeout
12 keepalive timeout, 11 keepalive probes
TCP SACK Stats
0 recovery episodes, 0 data packets, 0 data bytes
0 data packets retransmitted, 0 data bytes retransmitted
0 connections closed, 0 retransmit timeouts
TCP SYN Cache Stats
15 entries, 3 connections completed, 0 entries timed out
0 dropped due to overflow, 12 dropped due to RST
0 dropped due to ICMP unreach, 0 dropped due to bucket overflow
0 abort due to no memory, 0 duplicate SYN, 0 no-route SYN drop
0 hash collisions, 0 retransmitted
TCP Active Connections
Local Address Remote Address State Send-Q Recv-Q
0.0.0.0:3260 0.0.0.0:0 LISTEN 0 0

 

Use the show ips stats icmp interface ips to display and verify IP statistics. This command takes the main Ethernet interface as a parameter and returns the ICMP statistics for that interface. See Example 6-5.

Example 6-5 Displaying ICMP Statistics

switch# show ips stats icmp interface ips 4/1
 
ICMP Statistics for port IPStorage4/1
0 ICMP messages received
0 ICMP messages dropped due to errors
ICMP input histogram
0 destination unreachable
0 time exceeded
0 parameter problem
0 source quench
0 redirect
0 echo request
0 echo reply
0 timestamp request
0 timestamp reply
0 address mask request
0 address mask reply
ICMP output histogram
0 destination unreachable
0 time exceeded
0 parameter problem
0 source quench
0 redirect
0 echo request
0 echo reply
0 timestamp request
0 timestamp reply
0 address mask request
0 address mask reply

Displaying IPStorage Ports Speed

Use the show ips status command to verify the programmed speed of an IPStorage port.

Example 6-6 Displays IPStorage Port Speed

switch# show ips status
Port 1/1 READY 10G
Port 1/2 READY 1G

Default Settings for IPStorage Services Parameters

Table 6-5 lists the default settings for IPStorage services parameters.

 

Table 6-5 Default IPStorage Parameters

Parameters
Default

IPS core size

Partial