Table Of Contents
Troubleshooting SANTap
Overview
Definitions
Limitations
Interface Restrictions
Initial Troubleshooting Checklist
Common Troubleshooting Tools in Fabric Manager
Common Troubleshooting Commands in the CLI
Messages, Logs and Databases
SANTap Issues
Host Login Problems
ITL Problems
Common Mismatch Problems
Troubleshooting SANTap
This chapter describes how to identify and resolve problems that might occur when implementing SANTap. This chapter includes the following sections:
•Overview
•Definitions
•Limitations
•Initial Troubleshooting Checklist
•SANTap Issues
Overview
Note SANTap is not supported in Release 3.3.1.
SANTap is an Intelligent Storage Services feature supported on the Storage Services Module (SSM). The SSM supports SANTap in Cisco MDS SAN-OS Release 2.0(2b) and later. The SANTap feature allows third-party data storage applications, such as long distance replication and continuous backup, to be integrated into the SAN.
SANTap has a control path and a data path. The control path services requests that create and manipulate replication sessions that are sent by an appliance. The control path is implemented using a SCSI-based protocol. An appliance sends requests to a control virtual target (CVT) which the SANTap process creates and monitors. Responses are sent to the control LUN on the appliance. SANTap also allows LUN mapping to appliance virtual targets (AVTs). You can have a maximum of 512 target LUNs.
When introducing SANTap-based applications, SANTap does not require reconfiguration of either the host or the target, and neither the host initiator nor the target is required to be directly connected to an SSM. This accomplished by assigning Cisco-specific WWNs to the virtual initiators (VIs) and data virtual targets (DVTs).
A host initiator or a target can be connected directly to an SSM. However, you must partition the SAN using VSANs. You must configure the host initiator and the DVT in one VSAN and configure the virtual initiator (VI) and the target in another VSAN.
Definitions
Table 23-1includes brief definitions of some of the common SANTap acronyms and terms.
Table 23-1 SANTap Acronyms
Acronym / Term
|
Definition
|
AVT
|
Appliance virtual target. The portal through which an appliance can complete its synchronization with the target LUN. AVT can be thought of as a host proxy for the appliance.
|
CVT
|
Control virtual target. The portal through which an appliance communicates with SANTap. An initiator port on the appliance sends out SANTap Control Protocol requests to the SANTap process. When the request is processed, the response is sent back by the Cisco VI (virtual initiator) to a target port on the appliance.
|
DVT
|
Data virtual target. A DVT is created for every port on a multi-ported target that is included in SANTap-based services. The DVT is created in the host VSAN. Once a DVT is created and a host logs into the DVT, SANTap installs a DVTLUN for every configured LUN on the target for this host.
|
ITL
|
Initiator/target/LUN tuple. Uniquely identifies a LUN on a target.
|
Session
|
A record/object that is created for every ITL whose WRITE I/Os the appliance is interested in. A session can be thought of as a target LUN that requires SANTap-based services.
|
VI
|
Virtual initiator. SANTap creates 9 VIs in the appliance and target VSANs. In the appliance, VIs are used to send responses to SANTap CP requests and also to send copies of WRITE I/Os. In the target, VIs are used when the appliance is down and one of the SANTap recovery tools (ARL, PWL-BPR) is enabled.
Note If the appliance implementation chooses not to use these recovery tools, the VIs are not used.
|
Limitations
This section describes SANTap limitations. Table 23-2 describes the component limitations by SAN-OS and SSI releases.
•One Recover Point Appliance (RPA) cluster can have only one target VSAN.
•SANTap does not support IVR. You cannot run SANTap and IVR together.
Table 23-2 SANTap Attribute Limitations
Attribute
|
Limitation
|
Cisco SAN-OS and SSI Release
|
Hosts per DVT
|
16
|
Prior to Cisco SAN-OS 3.2
|
32
|
Cisco SAN-OS 3.2
|
LUNs per host
|
256
|
All releases
|
LUNs per DVT
|
1500
|
Cisco SAN-OS 3.2
|
1024
|
Prior to Cisco SAN-OS 3.2
|
DVTs per SSM
|
32
|
Cisco SAN-OS 3.1(2) with SSI 3.1(2m) and later
|
16
|
Prior to Cisco SAN-OS 3.1(2) with SSI 3.1(2m)
|
Sessions per SSM
|
1024
|
Cisco SAN-OS 3.0(2) with SSI 3.0(2j)
|
2048
|
Cisco SAN-OS 3.1(2b) with SSI 3.1(2m) and later
|
LUN ID address
|
32-bits
|
Cisco SAN-OS 3.2
|
16-bits
|
Prior to Cisco SAN-OS 3.2
|
DVT LUNs per SSM
|
4080
|
All releases
|
ITLs per DPP
|
1500
|
Cisco SAN-OS 3.2
|
1024
|
Prior to Cisco SAN-OS 3.2
|
ITLs per SSM
|
1024
|
Cisco SAN-OS 3.0(2) with SSI 3.0(2j)
|
2048
|
Cisco SAN-OS 3.1(2b) with SSI 3.1(2m)
|
4080
|
Cisco SAN-OS 3.1(3) with SSI 3.1(3) and later
|
Interface Restrictions
When enabled on a per-port basis, SANTap must be enabled on a group of four ports on an SSM. The following restrictions apply:
•The fewest number of interfaces that can be SANTap enabled is four.
•The first interfaces in a group must be 1, 5, 9, 13, 17, 21, 25, or 29. (You can specify fc5 through fc8, but not fc7 through fc10.)
•The groups of four interfaces do not need to be consecutive. For example, you can specify fc1 through fc8 and fc17 through fc20.
Initial Troubleshooting Checklist
Troubleshooting a SANTap problem involves gathering information about the configuration and connectivity of the various SANTap components. Begin your troubleshooting activity as follows:
Checklist
|
Check off
|
Verify licensing requirements. See Cisco MDS 9000 Family Fabric Manager Configuration Guide.
|
|
Verify that SANTap is enabled on the SSM module of the selected switch.
|
|
Verify the VSAN configuration and zones for the appliance, using the configuration and verification tools for the specific appliance.
|
|
Verify the physical connectivity for any problem ports or VSANs.
|
|
Common Troubleshooting Tools in Fabric Manager
Use the following Fabric Manager procedures to verify the VSAN configuration for the SANTap components.
•Choose Fabricxx > VSANxx to view the VSAN configuration in the Information pane.
•Choose Fabricxx > VSANxx and select the Host or Storage tab in the Information pane to view the VSAN members.
Common Troubleshooting Commands in the CLI
Use the show santap module command to display information about SANTap.
Example 23-1 Display SANTap CVT Information
switch# show santap module 2 cvt
cvt pwwn = 23:4f:00:0d:ec:09:3c:02
cvt nwwn = 23:9d:00:0d:ec:09:3c:02
Example 23-2 Display SANTap DVT Information
switch# show santap module 2 dvt
dvt pwwn = 50:06:0e:80:03:81:32:36
dvt nwwn = 50:06:0e:80:03:81:32:36
dvt lun size handling = 0
dvt app iofail behaviour = 1
dvt tgt iofail behavior = 0
dvt appio failover time = 50 secs
dvt inq data behavior = 0
Example 23-3 Display SANTap DVT LUN Information
switch# show santap module 2 dvtlun
dvt pwwn = 22:00:00:20:37:88:20:ef
tgt pwwn = 22:00:00:20:37:88:20:ef
Example 23-4 Display SANTap Session Information
switch# show santap module 2 session
host pwwn = 21:00:00:e0:8b:12:8b:7a
dvt pwwn = 50:06:0e:80:03:81:32:36
tgt pwwn = 50:06:0e:80:03:81:32:36
adt pwwn = 33:33:33:33:33:33:33:00
aci pwwn = 22:22:22:22:22:22:22:22
cvt pwwn = 23:4f:00:0d:ec:09:3c:02
MRL : vsan 8 RegionSize 4806720, DiskPWWN 0x234f000dec093c02, DiskLun 0x 1, startLBA 1
PWL : type 2, UpdatePol 2, RetirePolicy 4, pwl_start 1
Example 23-5 Display SANTap AVT Information
switch# show santap module 2 avt
avt pwwn = 2a:4b:00:05:30:00:22:25
avt nwwn = 2a:60:00:05:30:00:22:25
hi pwwn = 21:00:00:e0:8b:07:61:aa
tgt pwwn = 22:00:00:20:37:88:20:ef
Example 23-6 Display SANTap AVT LUN Information
switch# show santap module 2 avtlun
avt pwwn = 2a:4b:00:05:30:00:22:25
Example 23-7 Display SANTap Remote Virtual Terminal Information
switch# show santap module 2 rvt
rvt pwwn = 2a:61:00:05:30:00:22:25
rvt nwwn = 2a:62:00:05:30:00:22:25
Example 23-8 Display SANTap Remote Virtual Terminal LUN Information
switch# show santap module 2 rvtlun
rvt pwwn = 2a:61:00:05:30:00:22:25
app pwwn = 22:00:00:20:37:39:b1:00
Use the following commands to display more advanced troubleshooting information for SANTap.
•show tech-support
•show santap module 2 tech-support
•show isapi tech-support
•show santap vttbl dvt dvt-pwwn
Messages, Logs and Databases
The following log files and databases can provide helpful information when troubleshooting SANTap problems.
•Look for SSM_CRIT in Sup syslog messages.
•SANTap logs are available via the show isapi tech-support CLI command. Search for the strings "Error" and "failed" or "failure."
•Review FCNS and FLOGI databases, using the show fcns and show flogi CLI commands.
SANTap Issues
This section includes the following topics:
•Host Login Problems
•ITL Problems
•Common Mismatch Problems
Host Login Problems
Host login problems can be caused by DVT limitations, connectivity, or other issues.
To diagnose host login problems, follow these steps:
Step 1 Use the show santap vttbl dvt dvt-pwwn command to display SANTap information for the DVT.
Step 2 Use the show santap vttbl dvt dvt-pwwn host-pwwn command to display SANTap information for the DVT and the host.
Step 3 Enable logging on the appropriate SSM module.
Step 4 Issue the following debug commands:
a. debug vsd vfc-felogin
b. debug partner 0x92810000 d1
c. debug partner 0x92810000 d2
Step 5 Review the logs to determine the problem and then take the appropriate corrective action.
ITL Problems
An ITL problem may occur if the number of ITLs exceeds the limitations for the version of Cisco SAN-OS and SSI in use. For specific ITL limitations, see the "Limitations" section
To diagnose and resolve ITL problems, follow these steps:
Step 1 Use the show isapi dpp 4 queue command to display DPP queue information.
Step 2 Verify that the number of ITLs on a DPP is within the limitations for the version of Cisco SAN-OS and SSI in use. Use the show isapi dpp 4 queue incl LUN and show isapi dpp 4 queue count commands.
Step 3 Verify that the number of ITLs on an SSM is within the limitations for the version of Cisco SAN-OS and SSI in use. Issue the show isapi dpp all queue command, using the incl LUN and count parameters.
Common Mismatch Problems
Problems are often caused by mismatching component versions, or using devices that are not supported by the interoperability matrix.
The following lists common mismatch situations:
Table 23-3 Common SANTap Mismatch Problems
Problem
|
Description
|
Version mismatch between SSM and the RPA.
|
The version of SSM and the version of the replication appliance are not compatible.
|
Version mismatch between the Supervisor and the SSI image.
|
The version of the Sup and the SSI image are not compatible.
|
Hosts, targets, HBAs or switches are not supported by the interoperability matrix.
|
•Review the Cisco SANTap interoperability matrix at http://www.cisco.com/en/US/products/ps5989/products_device_support_tables_list.html.
|
CVT is in the host VSAN.
|
The CVT is the portal through which the appliance communicates with SANTap, and cannot be in the host VSAN.
|
IVR and SANTap are being used together.
|
IVR and SANTap both perform straddling across VSANs and cannot be used together.
|
VIs in a DVT VSAN (CVT and DVT in the same VSAN).
|
This results in the creation of one CPP VI and eight DPP VIs in the DVT VSAN. The VIs attempt to login to DVT, resulting in non-deterministic behavior.
|
Overprovisioning, including:
•Too many ITLs per SSM
•Too many hosts per DVT
•Too many ITLs per DPP
|
When replication is enabled, AVT LUNs are created, and that can increase the ITL count over the limit. (See "Limitations" section.)
If Reservation Support is not enabled on the RPA:
•26 AVT LUNs are created at a time.
•The appliance completes recovery of these LUNs and then deletes them before creating more.
•This behavior does not significantly increase the ITL count.
If Reservation Support is enabled on the RPA:
•All AVT and AVT LUNs are permanently created.
•In RPA 2.3, only half the appliance posts are zoned with AVTs. This does not increase the ITL count significantly.
•In RPA 2.4, all appliance ports are zoned with AVTs. This behavior can increase the ITL count significantly.
|
Same pWWN occurs more than once in the same VSAN.
|
Misconfiguration can result in two DVTs (or two VIs) with the same WWN in the same VSAN.
For example, assume that two DVTs are created on different SSMs or on different DPPs. Both of these DVTs have the same back-end VSAN. When a host HBA logs into both DVTs, an attempt is made to create two VIs with the same WWN in the same back-end VSAN. This results in non-deterministic behavior.
|
A host is moved from the front-end VSAN to the back-end VSAN.
|
There is a VI in the back-end VSAN with the same pWWN as the host. Before you can move the host:
•Shut the host port.
•Purge to remove VI from the back-end VSAN.
The host can now be safely moved.
|
Inaccurate zoning.
|
Zoning solutions differ, based on the Cisco SAN-OS and SSI versions in use.
•With SSI 3.0(2j), you must have default zoning in the back-end VSAN, or zone the target and VIs together in the back-end VSAN.
•With SSI 3.1(2), only the host VI and target need to be zoned together in the back-end VSAN.
|
Adding and removing hosts without performing a purge.
|
If you have 16 hosts and you remove one and add another, the new host will not see the LUNs. In this situation, perform a purge to clear one of the 16 entries after removing the host. Then you can add the new host to the DVT.
|