Fabrics
From Release 12.0.1a, SAN Controller allows you to create SAN Fabrics.
The following table describes the fields that appear on SAN Controller > SAN > Fabrics > Fabrics.
Field | Description |
---|---|
Fabric Name |
Specifies the name of the fabric. |
Seed Switch |
Specifies the seed switch used to discover switches in the fabric. |
State |
Specifies the state of the fabric. |
SNMPv3/SSH |
Specifies if SNMP and SSH access is allowed. |
User/Community |
Specifies the role of the user who created the fabric. |
Auth/Privacy |
Displays the authentication type. |
Licensed |
Specifies if all the switches in the fabric are licensed or not. |
Health |
Displays the health of the fabric. |
Performance Collection |
Specifies if performance collection is enabled or disabled on the fabric. |
Updated Time |
Specifies the time when the fabric was created or updated. |
Incl. VSANS |
Specifies the VSANS included with the fabric. |
Excl. VSANS |
Specifies the excluded VSANS. |
The following table describes the action items, in the Actions menu drop‐down list, that appear on SAN > Fabrics > Fabrics.
Action Item | Description |
---|---|
Add Fabric |
From the Actions drop-down list, select Add Fabric. For more instructions, see Adding a Fabric. |
Edit Fabrics |
Select a fabric to edit. From the Actions drop-down list, select Edit Fabrics. Make the necessary changes and click Apply. For more instructions, see Editing a Fabric. |
Delete Fabrics |
Select one or more fabrics to delete. From the Actions drop-down list, select Delete Fabrics. Click Confirm to delete the fabrics. For more instructions, see Deleting a Fabric. |
Rediscover Fabrics |
Allows you to rediscover the switches, links, and end devices associated with the fabric. Select one or more fabrics to rediscover. From the Actions drop-down list, select Rediscover Fabrics. A progress bar in the State column displays the rediscovery progress. For more instructions, see Rediscovering a Fabric. |
Purge Fabrics |
Allows you to purge non-existent switches, links, and end devices of the fabric. Select one or more fabrics to purge. From the Actions drop-down list, select Purge Fabrics. For more instructions, see Purging a Fabric. |
Configure Performance |
Allows you to enable performance monitoring on links, switch interfaces, and end devices associated with the fabric. Select one or more fabrics for performance monitoring. From the Actions drop-down list, select Configure Performance. Make the necessary changes and click Apply. For more instructions, see Configuring Performance. |
Configure SAN Insights |
Allows you configure SAN Insights on the selected fabric. For more instructions, see Configure SAN Insights. |
Configure Backup |
Allows you to configure and schedule backup for the fabric data. For more instructions, see Configuring Fabric Backup. |
This chapter contains below sections:
Adding a Fabric
Procedure
Step 1 |
Choose SAN > Fabrics > SAN Fabrics. |
||
Step 2 |
Choose . |
||
Step 3 |
In the Fabric Name field, enter a unique name for the fabric. |
||
Step 4 |
Select the Fabric Seed Switch Type. From Release 12.1.2e, NDFC allows you to discover Cisco and Non-Cisco switches to SAN Fabrics. |
||
Step 5 |
If you chose Cisco in the Fabric Seed Switch Type, perform the following: |
||
Step 6 |
If you chose Non-Cisco in the Fabric Seed Switch Type, perform the following: |
||
Step 7 |
Click Add to add a Fabric.
|
ESXi Networking for Promiscuous Mode
From Cisco NDFC Release 12.1.2e, you can run NDFC on top of virtual Nexus Dashboard (vND) instance with promiscuous mode that is disabled on port groups that are associated with Nexus Dashboard interfaces where External Service IP addresses are specified. vND comprises Nexus Dashboard management interface and data interface. By default, for fabric controller persona, two external service IP addresses are required for the Nexus Dashboard management interface subnet.
Before the NDFC Release 12.1.2e, if Inband management or Endpoint Locator or POAP feature was enabled on NDFC, you must also enable promiscuous mode for the Nexus Dashboard data or fabric interface port-group. This setting was mandatory for traffic flow that is associated for these features.
Enabling promiscuous mode raise risk of security issues in NDFC, it is recommended to set default setting for promiscuous mode.
Note |
|
If Inband management or EPL is enabled, you must specify External Service IP addresses in the Nexus Dashboard data interface subnet. You can disable promiscuous mode for the Nexus Dashboard data or fabric interface port-group. For more information, refer to Cisco Nexus Dashboard Deployment Guide
Note |
Default option for promiscuous mode is Reject. |
Procedure
Step 1 |
Log into your vSphere Client. |
Step 2 |
Navigate to the ESXi host. |
Step 3 |
Right-click the host and choose Settings. A sub-menu appears. |
Step 4 |
Choose Networking > Virtual Switches. All the virtual switches appear as blocks. |
Step 5 |
Click Edit Settings of the VM Network. |
Step 6 |
Navigate to the Security tab. |
Step 7 |
Update the Promiscuous mode settings as follows:
|
Step 8 |
Click OK. |
Editing a Fabric
To edit a fabric from the Cisco SAN Controller Web UI, perform the following steps:
Procedure
Step 1 |
Choose SAN > Fabrics > SAN Fabrics. |
Step 2 |
Choose check box to edit required fabric name, choose the Actions > Edit Fabrics. |
Step 3 |
You see the Edit Fabrics window. You can edit only one fabric at a time. |
Step 4 |
Enter a new fabric Fabric Name |
Step 5 |
(Optional) Check the SNMPV3 check box. If you check SNMPV3, the Community field change to Username and Password. |
Step 6 |
Enter the Username and Password, privacy and specify how you want SAN Controller Web Client to manage the fabric by selecting one of the status options. |
Step 7 |
Change the status to Managed, Unmanaged, or Managed Continuously. |
Step 8 |
(Optional) Check the Use UCS Credentials check box. If you want to modify UCS credentials. |
Step 9 |
Enter the Username and Password |
Step 10 |
Click Apply to save the changes. |
Deleting a Fabric
To delete a fabric using SAN Controller Web UI, perform the following steps:
Procedure
Step 1 |
Choose SAN > Fabrics > SAN Fabrics. |
Step 2 |
Choose to remove the fabric from the data source and to discontinue data collection for that fabric. |
Rediscovering a Fabric
To discover a fabric using Cisco SAN Controller Web UI, perform the following steps:
Procedure
Step 1 |
Choose SAN > Fabrics > SAN Fabrics. |
Step 2 |
Choose check box to rediscover required fabric name, choose the . |
Step 3 |
Click Yes in the dialog box. In a fabric window, State column displays the progress of rediscovery for selected fabric. The Fabric is rediscovered. |
Purging a Fabric
You can clean and update the fabric discovery table through the Purge option.
Procedure
Step 1 |
Choose SAN > Fabrics. |
Step 2 |
Choose the check box next to the fabric you want to purge. |
Step 3 |
Choose .The Fabric is purged. From SAN Controller Release 12.0.1a, you can purge fabric on Topology window.
|
Configuring Performance
If you are managing your switches with the performance manager, you must set up an initial set of flows and collections on the switch. You can use SAN Controller to add and remove performance collections. License the switch and keep it in the managedContinuously state before creating a collection for the switch. Only licensed fabrics appear in this window.
Procedure
Step 1 |
Choose SAN > Fabrics. |
Step 2 |
Choose the check box next to the fabric you want to configure performance collections. |
Step 3 |
Choose .The Performance Data Collection Settings window appears. |
Step 4 |
Choose check box Performance Collection, to enable other check boxes. |
Step 5 |
Choose required ISL/NPV Links, Hosts, Storage, and FC Ethernet, or choose box Select All to enable performance collection for these data types.
|
Step 6 |
Click Apply to save the configuration. |
Step 7 |
In the confirmation dialog box, click Yes to restart the performance collector. |
What to do next
After upgrading to Nexus Dashboard Fabric Controller, to view the restored old Performance Manager and high chart data, you must manually enable Performance Manager for each fabric. However, any old Temperature data is not restored.
To begin collecting Temperature data on the upgraded Nexus Dashboard Fabric Controller setup, go to Settings > Server Settings PM tab. Check Collect Temperature for LAN Switches checkbox and click Save.. Note that Enable LAN Sensor Discovery checkbox is enabled by default.
SAN Insights
The SAN Insights feature enables you to configure, monitor, and view the flow analytics in fabrics. SAN Insights features of SAN Controller enable you to visualize the health-related indicators in the interface so that you can quickly identify issues in fabrics. Also, the health indicators enable you to understand the problems in fabrics. The SAN Insights feature also provides more comprehensive end-to-end flow-based data from the host to LUN.
SAN Controller supports SAN Telemetry Streaming (STS) using compact GPB transport, for better telemetry performance and to improve the overall scalability of SAN Insights.
For SAN Insights streaming stability and performance, see Required System Memory for SAN Controller deployment. Ensure that the system RAM, vCPU, and SSDs are used for deploying SAN Insights. Use of NTP is recommended to maintain time synchronization between the SAN Controller and the switches. Enable PM collection for viewing counter statistics.
From Release 12.0.1a, you can create policy based alarms generation for SAN ITL/ITN flow. From Web UI, choose
to create policies.Prerequisites
-
SAN Insights is supported on virtual-data node and physical node.
-
The SAN Insights feature isn’t supported on app-node deployment for Nexus Dashboard.
-
Single node and three nodes deployments of Nexus Dashboard are supported for deploying SAN Insights.
-
If SAN Insights streaming was configured with KVGPB encoding using versions of Cisco SAN Insights older than 11.2(1), the switch continues to stream with KVGPB encoding while configuring streaming with SAN Insights versions 11.2(1) and above. Compact GPB streaming configuration for SAN Insights is supported starting from SAN Controller 11.2(1). To stream using Compact GPB, disable the old KVGPB streaming before configuring SAN Insights newly, after the upgrade. To disable analytics and telemetry, on the Cisco SAN Controller Web UI, choose Next. On the Switch Configuration screen, select required switch, choose to clear all the analytics and telemetry configuration on the selected switches.
, select a fabric, choose and click -
The SAN Insights feature is supported for Cisco MDS NX-OS Release 8.3(1) and later.
Configuring Persistent IP Address
Before you install or upgrade to SAN Controller Release 12.1.1e, you must configure persistent IP addresses on Cisco Nexus Dashboard.
Ensure that the services are allocated with IP pool addresses on the Cisco Nexus Dashboard. For more information, refer to Cluster Configuration section in Cisco Nexus Dashboard User Guide.
Note |
To configure SAN Insights on one node for SAN Controller deployment, the SAN Insights receiver requires one available Persistent IP. Similarly, to configure SAN Insights on three nodes for SAN Controller deployment, it requires three available Persistent IP addressees. |
To configure Persistent IP addresses on Cisco Nexus Dashboard, perform the following steps:
Procedure
Step 1 |
Choose . |
Step 2 |
On General tab, in External Service Pools card, click Edit icon. The External Service Pools window appears. |
Step 3 |
To configure IP addresses for SAN Controller, in Data Service IP's, click Add IP Address, enter required IP addresses and click check icon. |
Step 4 |
Click Save. |
Changing Persistent IP Address
From Cisco NDFC Release 12.1.2e, you can change persistent IP addresses which are assigned for mandatory pods such as POAP-SCP and SNMP trap. To change the persistent IP address, perform the following steps:
Procedure
Step 1 |
On Cisco NDFC Web UI, navigate to under LAN Device Management Connectivity drop-down list change Management to Data or conversely.Changing option results in migration of SNMP and POAP-SCP pods to the persistent IP addresses associated with External Service Pool on Nexus Dashboard connected with the new LAN Device Management Connectivity option. After the completion of this process, the following message is displayed: Some features have been updated. Reload the page to see latest changes. Click Reload the page. |
Step 2 |
On Cisco Nexus Dashboard Web UI, navigate to External Service Pools card, change the required IP addresses for Management Service IP Usage or Data Service IP Usage. , in |
Step 3 |
Navigate to NDFC Web UI Server Settings page, change the option in LAN Device Management Connectivity drop-down list to its initial selection. Restoring this option to initial settings, results in migration of the SNMP and POAP-SCP pods to use the updated persistent IP address from the appropriate External Service IP pool. |
Guidelines and Limitations
-
Ensure that the time configurations in SAN Controller and the supported switches are synchronized to the local NTP server for deploying the SAN Insights feature.
-
Any applicable daylight time savings settings must be consistent across the switches and SAN Controller.
-
To modify the streaming interval, use the CLI from the switch, and remove the installed query for SAN Controller. Modify the san.telemetry.streaming.interval property in the SAN Controller server properties. The allowed values for the interval are 30–300 seconds. The default value is 30 seconds. If there is an issue with the default value or to increase the value, set default value to 60 seconds. You can change the default value while configuring SAN Insights. On Switch configuration wizard in Interval(s) column select required value from drop-down list.
-
The port sampling window on the switch side should have all ports (default).
-
Use the ISL query installation type only for the switches that have storage connected (storage-edge switches).
-
For the ISL query installation type, in the Configure SAN Insights wizard, analytics can’t be enabled on interfaces that are members of port-channel ISL to non-MDS platform switches.
-
After installing the switch-based FM_Server_PKG license, the Configure SAN Insights wizard may take upto 5 minutes to detect the installed license.
For information about the SAN Insights dashboard, see SAN Insights Dashboard.
For information about configuring the SAN Insights, see Configuring SAN Insights.
Server Properties for SAN Insights
To modify server settings values, navigate to
on the Web UI.Note |
If you change the server properties, ensure that you restart the SAN Controller to use the new properties value. |
The following table describes the field names, descriptions, and its default values.
Field Name | Description | Default Value | ||
---|---|---|---|---|
Telemetry pages default protocol scsi/nvme | Specifies the required default protocol selection in the SAN Insights UI pages to view corresponding data: SCSI or NVMe. | SCSI | ||
SAN Insights ECT thread count | Specifies number of threads to use for ECT queries. | 4 | ||
Max. Aggregation bucket size | Specifies maximum number of buckets to use for aggregation queries. | 40,000 | ||
Data table download size | Specifies number of records for table download. | 1000 | ||
ECT Data limit | Specifies the ECT Data limit. |
14
|
||
SAN Telemetry deviation low threshold | Specifies the value that is the change point between normal and low. | 1 | ||
SAN Telemetry deviation med threshold | Specifies the value that is the change point between low and medium. | 15 | ||
SAN Telemetry deviation high threshold | Specifies the value that is the change point between medium and high. | 30 | ||
SAN Telemetry deviation low threshold for NVMe | Specifies the value that is the change point between normal and low for NVMe. | 1 | ||
SAN Telemetry deviation med threshold for NVMe | Specifies the value that is the change point between low and medium for NVMe. | 2 | ||
SAN Telemetry deviation high threshold for NVMe | Specifies the value that is the change point between medium and high for NVMe. | 5 | ||
SAN Telemetry training timeframe | Specifies the training time frame for flows ECT baseline. | 7 days | ||
SAN Telemetry training reset timeframe | Specifies the time duration to periodically restart the ECT baseline training after number of days. | 14 days | ||
SAN Telemetry retention policy - baseline / post processed | Specifies the retention policy - baseline / post processed. | 14 | ||
SAN Telemetry retention policy - hourly rollups | Specifies the retention policy - hourly rollups | 90 | ||
Telemetry Gap Reset Interval | Specify maximum valid time gap between records (before drop) time is in seconds | 750 | ||
Active Anomaly Capture | Specify maximum number of actively tracked anomalies per post processor. | 500 | ||
Baseline training include NOOP frames | Specify if the baseline learning should reference noop frames. | Not selected | ||
Baseline training includes negative deviation | Specify if the baseline deviation must include negatives. | Selected | ||
Use telemetry Gap Reset Interval | Specifies the use telemetry reset based on time gap between records | Selected |
The following table describes the system requirement for installation of SAN Controller:
Node Type | vCPUs | Memory | Storage |
---|---|---|---|
Virtual Data Node |
32 |
128 GB |
3 TB SSD |
Physical Data Node |
40 |
256 GB |
4*2.2 TB HDD, 370G SSD, 1.5 TB NVMe |
Deployment Type | Verified Limit1 2 |
---|---|
Cisco Virtual Nexus Dashboard (1 Node) |
1M ITLs/ITNs 80K ITLs/ITNs |
Cisco Physical Nexus Dashboard (1 Node) |
120K ITLs/ITNs |
Cisco Virtual Nexus Dashboard (3 Node) |
150K ITLs/ITNs |
Cisco Physical Nexus Dashboard (3 Node) |
500K ITLs/ITNs |
Note |
For one Node vND to support 1M ITLs/ITNs, it requires 3TB of SSD and 128 GB of memory Nexus Dashboard as system requirement. |
Configuring SAN Insights
From SAN Controller Release 12.0.1a, you can configure SAN fabrics on topology window, apart from configuring on fabric window.
On topology window, right-click on a SAN fabric, choose Configure SAN Insights and follow procedure to configure.
To configure SAN Insights on the SAN Controller Web UI, perform the following steps:Before you begin
Ensure that you configure persistent IP addresses, before you configure SAN Insights. Refer to Configuring persistent IP addresses.
Ensure that you have enabled SAN Insights feature for SAN Controller. Choose SAN Insights.
, choose check boxNote |
You must configure with sufficient system requirements and IP addresses. For more information on scale limits, refer to table Required System Memory for SAN deployment in Server Properties for SAN Insights. |
Procedure
Step 1 |
Choose . |
||||||
Step 2 |
Choose required fabric, click .The SAN Insights Configuration wizard appears. |
||||||
Step 3 |
In the SAN Insights Configuration wizard, click Next. The Switch Configuration wizard appears. |
||||||
Step 4 |
Select the switches where SAN Insights analytics and telemetry streaming need to be configured, after you select the appropriate values from the drop-down list as mentioned below. If the switches don’t have SAN Insights license, the status in the Licensed column shows No (install licenses). Click on Install licenses to apply license to the switch.
For the selected SAN Controller Receiver in the last column, the receiver can subscribe to telemetry: SCSI only, NVMe only, both SCSI & NVMe, or None. This allows you to configure one SAN Controller server to receive SCSI telemetry and another SAN Controller server to receive NVMe telemetry. In SAN Controller deployment, the IP address assigned to eth0 or eth1 can be used for receiving SAN Insights streaming from the switch. However, ensure that streaming is configured to the SAN Controller interface having IP reachability from the respective switches. In the Receiver column all the discovered interfaces are listed. Choose the corresponding interface IP address that is configured while installing SAN Controller for streaming analytics data from the switch. You can provide management IP eth0 and data IP eth1 for fabric access to bootstrap the SAN Controller. Therefore, the streaming must be configured to the persistent IP assigned in the data-IP subnet. Refer to the Configuring Persistent IP Address section for more information. To configure promiscuous mode to have multiple persistent IPs reachable on the same port group. See Cluster Configuration section in Nexus Dashboard Guide. The Subscription column allows you to specify which protocol to which the Receiver subscribes. You can choose from SCSI, NVMe, both or none from drop-down list.
You can click the i icon in the Switch Name column to get the configuration details for analytics and telemetry features from the switch (if Analytics Query and Telemetry features are configured). If Analytics Query of either type (dcnminitITL, dcnmtgtITL, dcnmislpcITL, dcnminitITN, dcnmtgtITN, or dcnmislpcITN) isn’t configured on the switch, the telemetry configurations won’t be displayed.
|
||||||
Step 5 |
Click Next. The switches that are capable of streaming analytics are listed in the Select Switches page. |
||||||
Step 6 |
Select the switches on which SAN Insights must be configured.
Choose single or multiple switches, click to clear all the analytics and telemetry configuration on the selected switches.Compact GPB streaming configuration for SAN Insights is supported. To stream using Compact GPB, the old KVGPB streaming must be disabled and removed before configuring SAN Insights, newly after the upgrade. In the Install Query column, type of port per switch is displayed. The port types are:ISL, host, or storage.
The following queries are used:
When the administrator selects the ISL\Host\Storage on the configure wizard, the respective ports are filtered and listed on the next step. |
||||||
Step 7 |
Click Next. You can see all the analytics supported modules on the switches selected in the previous view, listed with the respective instantaneous NPU load in the last column. Port-sampling configuration (optional) and port-sampling rotation interval for the module can be specified in this step. The default configuration on the switch is to monitor all analytics-enabled ports on the switch for analytics.
|
||||||
Step 8 |
In the Module Configuration tab, configure the module(s) for SAN Insights functionality. Beginning with Release 12.1.1e, Cisco NDFC supports discovery of 64G modules and can be selected during SAN Insights configuration. Port-sampling is not supported on these modules and NPU load is not applicable for 64G SAN analytics. Therefore, you cannot configure sample window and rotation interval for 64G modules. To change the values for Sample Window (ports) and Rotation Interval (seconds), click the row and enter the desired values.
The NPU Load column displays the network processing unit (NPU) within a module. |
||||||
Step 9 |
Click Next. |
||||||
Step 10 |
In the Interface Selection tab, select the interfaces that generate analytics data within the fabric. For each interface, you can enable or disable metrics. Choose check box in SCSI Metrics and NVMe Metrics column to enable or disable analytics on the desired port. |
||||||
Step 11 |
Click Next, and then review the changes that you have made. |
||||||
Step 12 |
Click Commit. The CLI is executed on the switch. |
||||||
Step 13 |
Review the results and see that the response is successful.
|
||||||
Step 14 |
Click Close to return to the home page. Close icon appears only after all CLI commands are executed on the switch. Navigate to the or topology page again, to modify the SAN Insights configurations. |
Configuring Fabric Backup
You can configure backup for selected fabric, from Fabric window, similarly you can configure backup on Fabric Overview window. Choose on main window, click Configure Backup.
You can back up all fabric configurations and intents automatically or manually. You can save configurations in SAN Controller, which are the intents. The intent may or may not be pushed on to the switches.
SAN Controller doesn’t back up the following fabrics:
-
External fabrics in monitor-only mode: You can take a backup of external fabrics in monitor-only mode, but can’t restore them. You can restore this backup when the external fabric isn’t in monitor-only mode.
-
Parent MSD fabric: You can take backups of MSD fabrics. When you initiate a backup from the parent fabric, the backup process is applicable for the member fabrics as well. However, SAN Controller stores all the backed-up information of the member fabrics and the MSD fabric together in a single directory.
The backed-up configuration files can be found in the corresponding directory with the fabric name. Each backup of a fabric is treated as a different version, regardless if it is backed up manually or automatically. You can find all versions of the backup in the corresponding fabric directories.
You can enable scheduled backup for fabric configurations and intents.
The backup has the information related to intent and fabric configurations in addition to associated state of the resource manager in terms of used resources on fabrics. SAN Controller backs up only when there’s a configuration push. SAN Controller triggers the automatic backup only if you didn’t trigger any manual backup after the last configuration push.
Golden Backup
You can now mark the backups that you don’t want to delete even after you reach the archiving limit. These backups are the golden backups. You can’t delete golden backups of fabrics. However, SAN Controller archives only up to 10 golden backups. You can mark a backup as golden backup while restoring the fabric. To mark a backup as golden backup, perform the following steps from the Web UI:
Procedure
Step 1 |
Choose a fabric and choose Fabrics > Fabric Overview > Backup. The Backup tab appears. |
Step 2 |
On main window, choose .The Scheduled Archive window appears. |
Step 3 |
Choose the time period from where you want to choose the backup. Valid values are 1m, 3m, 6m, YTD, 1y, and All. You can zoom into the graph. By default the backup information for 1m, which is one month, appears. You can also choose a custom date range. The backup information includes the following information:
|
Step 4 |
Choose the backup you want to mark as golden by clicking the backup. You can choose the automatic or manual backup. These backups are color-coded. Automatic backups are indicated in blue color. Manual backups are indicated in midnight blue color. Golden backups are indicated in orange color. The automatic backups have only the versions in their names. Whereas the manual backups have tag names, which you gave when you initiated a manual backup, along with the version in the backup name. Hover over a backup to see the name. The automatic backup is initiated from the Backup tab in the Fabric Overview window. The manual backup is initiated by clicking Backup Now from the Actions pane in the backup tab. |
Step 5 |
Navigate to switch window, choose check box for required switch name, choose to mark golden backup.A confirmation dialog box appears. |
Step 6 |
Click Yes. |
Step 7 |
Continue with rest of the fabric restore procedure as mentioned in the Restoring Fabrics section or exit the window. |