Hyperflex Health and Pre-upgrade Check Toolの実行

はじめに

このドキュメントでは、Hypercheck Health and Pre-upgradeツールを実行するプロセスについて説明します。

前提条件

要件

次の項目に関する専門知識があることが推奨されます。

• Hyperflex

使用するコンポーネント

このドキュメントの情報は、Hypercheck Health and Pre-upgradeツールに基づくものです。

このドキュメントの情報は、特定のラボ環境にあるデバイスに基づいて作成されました。このドキュメントで使用するすべてのデバイスは、クリアな（デフォルト）設定で作業を開始しています。本稼働中のネットワークでは、各コマンドによって起こる可能性がある影響を十分確認してください。

背景説明

このツールは、Hyperflexシステムの安定性と復元力を確保するために、システムのセルフチェックをプロアクティブに実行するユーティリティです。Hyperflexシステムの健全性チェックとアップグレード前のチェックのリストを自動化し、Hyperflexのアップグレードやメンテナンス作業にかかる時間を節約します。

注：ツールを使用する前に、必ず最新バージョンをダウンロードしてください。このツールは頻繁に拡張されるため、以前のバージョンを使用すると、重要なチェックが欠落する可能性があります。

サポートされるHXシステム

• Hyperflexバージョン – 1.8、2.0、2.1、2.5、2.6、3.0、3.5、4.0、4.5、5.0、5.5
• Hyperflex標準クラスタ
• Hyperflex拡張クラスタ
• Hyperflexエッジクラスタ（2ノード、3ノード、4ノード）
• VMWare ESXi上のHyperflexクラスタでのみサポート

注：Hyperflex HyperVクラスタでハイパーチェックを実行する方法については、『ハイパーチェック：Hyperflex Health & Pre-Upgrade Check Tool - HyperV』を参照してください。

使用するケース

Hyperflex Health and Pre-upgrade checkツールを使用する効果的な時期は次のとおりです。

• Hyperflexのアップグレード前
• メンテナンスウィンドウの前後のHyperFlexヘルスチェック
• 障害が発生したドライブやディスクを特定するため
• Cisco TACと連携する場合
• 予防的なヘルスチェックをいつでも実行

使用方法

HXバージョン4.5以降

ステップ 1：HX接続IPであるクラスタ管理IP(CMIP)を使用して、ストレージコントローラVM(SCVM)へのSSH接続を開始します。

ステップ 2：hypercheck コマンドを実行します。

admin:~$ hypercheck

ステップ 3：プロンプトが表示されたらSCVM管理者パスワードを入力し、ESXiのルートパスワードを入力します。

admin:~$ hypercheck

                  HX Health Check 4.5.0

Please enter below info of HX-Cluster:
Enter the HX-Cluster Root Password:
Enter the HX-Cluster Admin Password:
Enter the ESX Root Password:

HXバージョン4.0以前

ステップ 1：Cisco github devnetアカウントからHyperflex-Hypercheck.zipをダウンロードします。 最新の改善点とアップデートを含む最新のコピーを入手します。

注：シスコの内部ツール、ファイル、および情報にアクセスできるのは、登録ユーザのみです。

注:Cisco github devnetアカウントからダウンロードしたスクリプトのみを使用してください。

ステップ 2：CMIPでSCVMにアップロードします。

Hyperflex-Hypercheck.zipを/tmpディレクトリにコピーするには、任意の方法scp/sftp/ftp/tftpを使用します。

MACの場合：

CLIからSCPを実行します(Hyperflex-Hypercheck.zipがSCPを実行する場所と同じフォルダにあることを確認します)。

# scp Hyperflex-Hypercheck.zip root@<scvm-eth0:mgmtip>:/tmp/.

HX環境のクラスタ管理IPを特定するには、Hyperflexプレイブックを使用します。

Windows の場合：

ファイルを転送するには、次のようにWINSCPを使用できます。

ステップ 3：Hyperflex-Hypercheck.zipの内容を解凍します。

cd /tmpと入力して、/tmpディレクトリに移動します。



root@SpringpathController7PVQWP6WV1:~# cd /tmp/


unzip Hyperflex-Hypercheck.zip と入力して、ファイルを抽出します。



root@SpringpathController7PVQWP6WV1:/tmp# unzip Hyperflex-Hypercheck.zip 
Archive:  Hyperflex-Hypercheck.zip
b61c59f7962b72902692ce70548ba3d760efdf06
   creating: Hyperflex-Hypercheck/
  inflating: Hyperflex-Hypercheck/HXTool.py 
  inflating: Hyperflex-Hypercheck/LICENSE.txt 
  inflating: Hyperflex-Hypercheck/ReadMe.txt 
  inflating: Hyperflex-Hypercheck/TestInfo.txt 
  inflating: Hyperflex-Hypercheck/prettytable.py 
  inflating: Hyperflex-Hypercheck/progressbar.py 
root@SpringpathController7PVQWP6WV1:/tmp# 


ステップ 4：HXTool Pythonスクリプトを実行します。

「cd Hyperflex-Hypercheck 」と入力して、Hyperflex-Hypercheckディレクトリに移動します。



root@SpringpathControllerABCDE01234:/tmp# cd Hyperflex-Hypercheck


スクリプトを実行するには、python HXTool.py  と入力します。



root@SpringpathControllerABCDE01234:/tmp/Hyperflex-Hypercheck# python HXTool.py


ステップ 5：プロンプトが表示されたら、SCVMルートパスワードを入力します。



Enter this information of HX-Cluster:
Enter the HX-Cluster Root Password:
Enter the ESX Root Password:

注：スクリプトの実行を停止するには、キー(Ctrl+Z)を使用すると、スクリプトはすぐに停止します。

手順 6：Hyperflex-Hypercheckツールがチェックを開始します。実行が完了するまでに、クラスタ内のコンバージドノードの数に応じて約3 ～ 10分かかります。

手順 7：スクリプト出力のレポートを取得します。次のように受信できます。

Hypercheck Report tarファイルは/var/log/springpath  および/tmp/Hyperflex-Hypercheckに保存されます。したがって、/var/log/springpathまたは/tmp/Hyperflex-Hypercheck の下からtarバンドルをダウンロードできます。 または、ハイパーチェックレポートtarも含むstorfs-supportバンドルを生成してアップロードすることもできます。

レポートtarファイルの例：HX_Report_2020_08_30_10_43_50.tarはpath:/var/log/springpathにコピーされます。

ls -l | grep HX_Reportと入力して、Hyperflex-Hypercheckツールで作成されたファイルを確認します。



Under /var/log/springpath,

root@SpringpathControllerABCDE01234:/var/log/springpath# ls -l | grep HX_Report
-rw-r--r-- 1 root root 380K Sep 23 15:41 HX_Report_2020_08_30_10_43_50.tar
root@SpringpathControllerABCDE01234:/var/log/springpath#

Under /tmp/Hyperflex-Hypercheck,

root@SpringpathControllerABCDE01234:/tmp/Hyperflex-Hypercheck# ls
HX_Report_2020_08_30_10_43_50.tar prettytable.py HX_Report_2020_08_30_10_43_50 TestInfo.txt progressbar.py
HXTool.py prettytable.pyc ReadMe.txt progressbar.pyc LICENSE.txt
root@SpringpathControllerABCDE01234:/tmp/Hyperflex-Hypercheck# 


ハイパーチェックログバンドル内のファイルとログ：



root@SpringpathControllerABCDE01234:/tmp/Hyperflex-Hypercheck# ls HX_Report_2020_08_30_10_43_50/
HX_Tool_2020-08-30_10-43-50.log 
HX_Tool_Main_Report_2020-08-30_10-54-34.txt 
HX_Tool_Summary.json


ステップ 8：HX_YYYY_MM_DD_HH_MM_SS.tarをエクスポートし、TACで共有します。

希望する方法を使用して、SCVMからscp/sftp/ftp/tftpを使用してハイパーチェックログをエクスポートするか、または単にHX_Report tarバンドルが含まれているstorfs-supportバンドルをダウンロードします。

実行される出力とチェックについて

ハイパーチェックで実行されるチェック

これらのチェックは、Hyperflex-Hypercheckツールによって実行されます。



Hyperflex Checks: (Below checks are performed on all the storage controller VMs)
  Cluster services check - Verifies the status of storfs, stMgr and stNodeMgr services
    Enospc state check - Checks if the cluster space usage is above the warning threshold or no   
    Zookeeper check - Checks whether the Zookeeper is running or no
    Exhibitor check - Verfies the status of the Exhibitor service which manages the ZK
 System Disks Usage - Checks if /sda1, var/stv and /var/zookeeper is less than 80% 
    HDD health check - Reports if you have any blacklisted disk in your cluster      
    DNS check - Checks whether DNS is configured and reachable                
    vCenter reachability check - Checks whether the vCenter is reachable on the required ports
    Timestamp check - Checks if all the controller VMs have the exact same time           
    NTP sync check - Checks whether NTP is reachable from the storage controller VMs and synced
   Check package & versions - Checks for packages and versions on Storage Controller VMs
 Check Iptables count - Checks for Iptables count on and ensure it is same on all Storage Controller VMs.              
    Extra pnodes check - Looks for any extra/duplicate pnode entries in the cluster
    Out of memory check - Checks through the log files if the cluster had any oom event
    Supported vSphere versions - Shows all the vSphere Versions supported with your current HXDP version
 Permissions for /tmp - Checks if the /tmp permissions are set correctly
 Check Cluster Policy - Checks the Configured Cluster Policy
 Check springpath_keystore.jceks file - Check if All the SCVM have same keystore file
 SED Capable - Checks if the cluster is SED Capable
 SED Enabled - Checks if Encryption is enabled in the Cluster
 USB-0 Check - If Encryption is enabled, Checks that USB0 interface is present on all the SCVMs
 SED 5100/5200 Drive Check - If we have Micron SED 5100 drives and version is below 3.5.2b, we wont be able to replace or add new 5200 drives
 Disk Lock Check - If Encryption is enabled, Checks for any Locked drives
 Network Checks - Checks the connectivity in Storage network
 Check ZK-Cleanup-Script - Checks to identify ZKTxnCleanUp Script
 Replication Checks - If replication is enabled, we check the local and remote network connectivity (HX 4.5 Only)
 Stretched Cluster Checks - Checks the latency between the sites and the witness VM (HX 4.5 Only)

ESXi Checks: (Below checks are performed on each ESXI node)
    HX User Account check - Verifies if the HXUser is created on all the esxi hosts and has admin rights
    vMotion enabled check - Checks if the vMotion network is confirgued
 Check for ESXI Failback timer - Check for ESXi Failback timer on ESXi host
    Check connectivity between vmk1 and eth1 - Checks the connectivity between the Mgmt and Storage network
    No extra controller vm folders check - Checks for duplicate Controller SCVM Folders
 VMware Tools location check  - Checks for Non default VMware Tools location
 vfat Disk Usage check - Checks for vfat Disk Usage
 Check /tmp usage - Checking for /tmp usage
 Compute Node Checks - All the ESXI checks are also performed on Compute nodes (HX 4.5 Only)


4ノード拡張クラスタからのハイパーチェックの出力例



Enter this information of HX-Cluster: Enter the HX-Cluster Root Password: Enter the ESX Root Password: Cluster Name: HX-10-Stretched Site-100 Site-97 Cluster Type: STRETCH_CLUSTER SSH connection established to HX Node: 192.168.53.135 SSH connection established to HX Node: 192.168.53.136 SSH connection established to HX Node: 192.168.53.137 SSH connection established to HX Node: 192.168.53.138 HX Cluster Nodes: +-------+--------------+--------------------------------+ | Nodes | IP Address | HostName | +-------+--------------+--------------------------------+ | 1 | 14.39.53.134 | SpringpathControllerOHCWUK9X3N | +-------+--------------+--------------------------------+ | 2 | 14.39.53.135 | SpringpathController37MHMEIBCY | +-------+--------------+--------------------------------+ | 3 | 14.39.53.136 | SpringpathControllerDWRWWIBFLF | +-------+--------------+--------------------------------+ | 4 | 14.39.53.137 | SpringpathControllerWB4UNXDKX3 | +-------+--------------+--------------------------------+ SSH connection established to ESX Host: 14.39.53.133 SSH connection established to ESX Host: 14.39.53.130 SSH connection established to ESX Host: 14.39.53.132 SSH connection established to ESX Host: 14.39.53.131 HX Controller: 192.168.53.135 Cluster services check [########################] COMPLETED ZooKeeper & Exhibitor check [########################] COMPLETED HDD health check [########################] COMPLETED Pre-Upgrade Check [########################] COMPLETED Network check [########################] COMPLETED HX Controller: 192.168.53.136 Cluster services check [########################] COMPLETED ZooKeeper & Exhibitor check [########################] COMPLETED HDD health check [########################] COMPLETED Pre-Upgrade Check [########################] COMPLETED Network check [########################] COMPLETED HX Controller: 192.168.53.137 Cluster services check [########################] COMPLETED ZooKeeper & Exhibitor check [########################] COMPLETED HDD health check [########################] COMPLETED Pre-Upgrade Check [########################] COMPLETED Network check [########################] COMPLETED HX Controller: 192.168.53.138 Cluster services check [########################] COMPLETED ZooKeeper & Exhibitor check [########################] COMPLETED HDD health check [########################] COMPLETED Pre-Upgrade Check [########################] COMPLETED Network check [########################] COMPLETED HX Controller: 192.168.53.135 Test Summary: +--------------------------------------+-----------+----------------------------------------------------------------+ | Name | Result | Comments | +--------------------------------------+-----------+----------------------------------------------------------------+ | Cluster services check | PASS | Checks storfs, stMgr, sstNodeMgr service running on each node. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Enospc state check | PASS | Checks if the cluster storage utilization is above threshold. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Zookeeper check | PASS | Checks if Zookeeper service is running. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Exhibitor check | PASS | Checks if Exhibitor in running. | +--------------------------------------+-----------+----------------------------------------------------------------+ | System Disks Usage | PASS | Checks if /sda1, var/stv and /var/zookeeper is less than 80%. | +--------------------------------------+-----------+----------------------------------------------------------------+ | HDD Health check | PASS | Checks if any drive is in blacklisted state. | +--------------------------------------+-----------+----------------------------------------------------------------+ | DNS check | PASS | Checks if configured DNS is reachable. | +--------------------------------------+-----------+----------------------------------------------------------------+ | vCenter reachability check | PASS | Checks if vCenter is network reachable using PING. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Timestamp check | PASS | Checks if the timestamp is same across all Nodes. | +--------------------------------------+-----------+----------------------------------------------------------------+ | NTP sync check | PASS | Checks if the NTP is synced with NTP server. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check package & versions | PASS | Checks for count and version of HX packages on each node. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check Iptables count | PASS | Checks if the IP Table count matches on all nodes. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Extra pnodes check | PASS | Checks for any stale Node entry. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Memory usage check | PASS | Checks for available memory more than 2GB. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Incidence of OOM in the log file | PASS | Checks for any previous incidence of Out Of Memory Condition. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Supported vSphere versions | 6.0.0-U3 | Prints the supported ESXi versions. | | | 6.5.0-U1 | | | | 6.5.0-U2 | | | | 6.5.0-U3 | | | | 6.7.0-UGA | | | | 6.7.0-U1 | | | | 6.7.0-U2 | | | | 6.7.0-U3 | | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check permissions for /tmp | PASS | Checks if the /tmp permissions are set correctly. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check Cluster Policy | Lenient | Checks the Configured Cluster Policy | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check springpath_keystore.jceks file | PASS | All the SCVM have same keystore file. | +--------------------------------------+-----------+----------------------------------------------------------------+ | SED Capable | NO | Checks if the cluster is SED Capable. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check Witness Reachability | PASS | Checks Witness VM IP address is reachabile. | +--------------------------------------+-----------+----------------------------------------------------------------+
| Check ZK-Cleanup-Script | PASS | Checks to identify ZKTxnCleanUp Script. | 
+--------------------------------------+-----------+----------------------------------------------------------------+ HX Controller: 192.168.53.136 Test Summary: +--------------------------------------+-----------+----------------------------------------------------------------+ | Name | Result | Comments | +--------------------------------------+-----------+----------------------------------------------------------------+ | Cluster services check | PASS | Checks storfs, stMgr, sstNodeMgr service running on each node. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Enospc state check | PASS | Checks if the cluster storage utilization is above threshold. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Zookeeper check | PASS | Checks if Zookeeper service is running. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Exhibitor check | PASS | Checks if Exhibitor in running. | +--------------------------------------+-----------+----------------------------------------------------------------+ | System Disks Usage | PASS | Checks if /sda1, var/stv and /var/zookeeper is less than 80%. | +--------------------------------------+-----------+----------------------------------------------------------------+ | HDD Health check | PASS | Checks if any drive is in blacklisted state. | +--------------------------------------+-----------+----------------------------------------------------------------+ | DNS check | PASS | Checks if configured DNS is reachable. | +--------------------------------------+-----------+----------------------------------------------------------------+ | vCenter reachability check | PASS | Checks if vCenter is network reachable using PING. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Timestamp check | PASS | Checks if the timestamp is same across all Nodes. | +--------------------------------------+-----------+----------------------------------------------------------------+ | NTP sync check | PASS | Checks if the NTP is synced with NTP server. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check package & versions | PASS | Checks for count and version of HX packages on each node. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check Iptables count | PASS | Checks if the IP Table count matches on all nodes. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Extra pnodes check | PASS | Checks for any stale Node entry. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Memory usage check | PASS | Checks for available memory more than 2GB. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Incidence of OOM in the log file | PASS | Checks for any previous incidence of Out Of Memory Condition. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Supported vSphere versions | 6.0.0-U3 | Prints the supported ESXi versions. | | | 6.5.0-U1 | | | | 6.5.0-U2 | | | | 6.5.0-U3 | | | | 6.7.0-UGA | | | | 6.7.0-U1 | | | | 6.7.0-U2 | | | | 6.7.0-U3 | | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check permissions for /tmp | PASS | Checks if the /tmp permissions are set correctly. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check Cluster Policy | Lenient | Checks the Configured Cluster Policy | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check springpath_keystore.jceks file | PASS | All the SCVM have same keystore file. | +--------------------------------------+-----------+----------------------------------------------------------------+ | SED Capable | NO | Checks if the cluster is SED Capable. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check Witness Reachability | PASS | Checks Witness VM IP address is reachabile. | +--------------------------------------+-----------+----------------------------------------------------------------+
| Check ZK-Cleanup-Script | PASS | Checks to identify ZKTxnCleanUp Script. | 
+--------------------------------------+-----------+----------------------------------------------------------------+ HX Controller: 192.168.53.137 Test Summary: +--------------------------------------+-----------+----------------------------------------------------------------+ | Name | Result | Comments | +--------------------------------------+-----------+----------------------------------------------------------------+ | Cluster services check | PASS | Checks storfs, stMgr, sstNodeMgr service running on each node. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Enospc state check | PASS | Checks if the cluster storage utilization is above threshold. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Zookeeper check | PASS | Checks if Zookeeper service is running. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Exhibitor check | PASS | Checks if Exhibitor in running. | +--------------------------------------+-----------+----------------------------------------------------------------+ | System Disks Usage | PASS | Checks if /sda1, var/stv and /var/zookeeper is less than 80%. | +--------------------------------------+-----------+----------------------------------------------------------------+ | HDD Health check | PASS | Checks if any drive is in blacklisted state. | +--------------------------------------+-----------+----------------------------------------------------------------+ | DNS check | PASS | Checks if configured DNS is reachable. | +--------------------------------------+-----------+----------------------------------------------------------------+ | vCenter reachability check | PASS | Checks if vCenter is network reachable using PING. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Timestamp check | PASS | Checks if the timestamp is same across all Nodes. | +--------------------------------------+-----------+----------------------------------------------------------------+ | NTP sync check | PASS | Checks if the NTP is synced with NTP server. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check package & versions | PASS | Checks for count and version of HX packages on each node. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check Iptables count | PASS | Checks if the IP Table count matches on all nodes. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Extra pnodes check | PASS | Checks for any stale Node entry. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Memory usage check | PASS | Checks for available memory more than 2GB. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Incidence of OOM in the log file | PASS | Checks for any previous incidence of Out Of Memory Condition. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Supported vSphere versions | 6.0.0-U3 | Prints the supported ESXi versions. | | | 6.5.0-U1 | | | | 6.5.0-U2 | | | | 6.5.0-U3 | | | | 6.7.0-UGA | | | | 6.7.0-U1 | | | | 6.7.0-U2 | | | | 6.7.0-U3 | | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check permissions for /tmp | PASS | Checks if the /tmp permissions are set correctly. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check Cluster Policy | Lenient | Checks the Configured Cluster Policy | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check springpath_keystore.jceks file | PASS | All the SCVM have same keystore file. | +--------------------------------------+-----------+----------------------------------------------------------------+ | SED Capable | NO | Checks if the cluster is SED Capable. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check Witness Reachability | PASS | Checks Witness VM IP address is reachabile. | +--------------------------------------+-----------+----------------------------------------------------------------+
| Check ZK-Cleanup-Script | PASS | Checks to identify ZKTxnCleanUp Script. | 
+--------------------------------------+-----------+----------------------------------------------------------------+ HX Controller: 192.168.53.138 Test Summary: +--------------------------------------+-----------+----------------------------------------------------------------+ | Name | Result | Comments | +--------------------------------------+-----------+----------------------------------------------------------------+ | Cluster services check | PASS | Checks storfs, stMgr, sstNodeMgr service running on each node. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Enospc state check | PASS | Checks if the cluster storage utilization is above threshold. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Zookeeper check | PASS | Checks if Zookeeper service is running. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Exhibitor check | PASS | Checks if Exhibitor in running. | +--------------------------------------+-----------+----------------------------------------------------------------+ | System Disks Usage | PASS | Checks if /sda1, var/stv and /var/zookeeper is less than 80%. | +--------------------------------------+-----------+----------------------------------------------------------------+ | HDD Health check | PASS | Checks if any drive is in blacklisted state. | +--------------------------------------+-----------+----------------------------------------------------------------+ | DNS check | PASS | Checks if configured DNS is reachable. | +--------------------------------------+-----------+----------------------------------------------------------------+ | vCenter reachability check | PASS | Checks if vCenter is network reachable using PING. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Timestamp check | PASS | Checks if the timestamp is same across all Nodes. | +--------------------------------------+-----------+----------------------------------------------------------------+ | NTP sync check | PASS | Checks if the NTP is synced with NTP server. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check package & versions | PASS | Checks for count and version of HX packages on each node. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check Iptables count | PASS | Checks if the IP Table count matches on all nodes. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Extra pnodes check | PASS | Checks for any stale Node entry. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Memory usage check | PASS | Checks for available memory more than 2GB. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Incidence of OOM in the log file | PASS | Checks for any previous incidence of Out Of Memory Condition. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Supported vSphere versions | 6.0.0-U3 | Prints the supported ESXi versions. | | | 6.5.0-U1 | | | | 6.5.0-U2 | | | | 6.5.0-U3 | | | | 6.7.0-UGA | | | | 6.7.0-U1 | | | | 6.7.0-U2 | | | | 6.7.0-U3 | | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check permissions for /tmp | PASS | Checks if the /tmp permissions are set correctly. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check Cluster Policy | Lenient | Checks the Configured Cluster Policy | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check springpath_keystore.jceks file | PASS | All the SCVM have same keystore file. | +--------------------------------------+-----------+----------------------------------------------------------------+ | SED Capable | NO | Checks if the cluster is SED Capable. | +--------------------------------------+-----------+----------------------------------------------------------------+ | Check Witness Reachability | FAIL | Checks Witness VM IP address is reachabile. | +--------------------------------------+-----------+----------------------------------------------------------------+
| Check ZK-Cleanup-Script | PASS | Checks to identify ZKTxnCleanUp Script. | 
+--------------------------------------+-----------+----------------------------------------------------------------+ | Check Disk for SMART Failure. | PASS | Checks disks for SMART Failure. |
+--------------------------------------+-----------+----------------------------------------------------------------+ ################################################################################ Network check: ################################################################################ ESX vmk0: 14.39.53.130, 14.39.53.131, 14.39.53.132, 14.39.53.133 ESX vmk1: 192.168.53.130, 192.168.53.131, 192.168.53.132, 192.168.53.133 SCVM eth0: 14.39.53.134, 14.39.53.135, 14.39.53.136, 14.39.53.137 SCVM eth1: 192.168.53.135, 192.168.53.136, 192.168.53.137, 192.168.53.138 ESX Host: 14.39.53.130 +--------------------------------------+--------+-------------------------------------------------+ | Name | Result | Comments | +--------------------------------------+--------+-------------------------------------------------+ | HX User Account check | PASS | Checks if HXUSER is present. | +--------------------------------------+--------+-------------------------------------------------+ | vMotion enabled check | PASS | Checks if vMotion is enabled on the host. | +--------------------------------------+--------+-------------------------------------------------+ | Check for ESXI Failback timer | PASS | Checks for ESXi FAILBACK timer set to 30000ms. | +--------------------------------------+--------+-------------------------------------------------+ | Check vmk1 ping to eth1 | PASS | Checks Network between ESXi vmk1 and SCVM eth1. | +--------------------------------------+--------+-------------------------------------------------+ | No extra controller vm folders check | PASS | Checks for duplicate Controller SCVM Folders. | +--------------------------------------+--------+-------------------------------------------------+ | VMware Tools location check | PASS | Checks for Non default VMware Tools location. | +--------------------------------------+--------+-------------------------------------------------+ | vfat Disk Usage check | PASS | Checks for vfat Disk Usage. | +--------------------------------------+--------+-------------------------------------------------+ | Check /tmp usage | PASS | Checking for /tmp usage. | +--------------------------------------+--------+-------------------------------------------------+ ESX Host: 14.39.53.131 +--------------------------------------+--------+-------------------------------------------------+ | Name | Result | Comments | +--------------------------------------+--------+-------------------------------------------------+ | HX User Account check | PASS | Checks if HXUSER is present. | +--------------------------------------+--------+-------------------------------------------------+ | vMotion enabled check | PASS | Checks if vMotion is enabled on the host. | +--------------------------------------+--------+-------------------------------------------------+ | Check for ESXI Failback timer | PASS | Checks for ESXi FAILBACK timer set to 30000ms. | +--------------------------------------+--------+-------------------------------------------------+ | Check vmk1 ping to eth1 | PASS | Checks Network between ESXi vmk1 and SCVM eth1. | +--------------------------------------+--------+-------------------------------------------------+ | No extra controller vm folders check | PASS | Checks for duplicate Controller SCVM Folders. | +--------------------------------------+--------+-------------------------------------------------+ | VMware Tools location check | PASS | Checks for Non default VMware Tools location. | +--------------------------------------+--------+-------------------------------------------------+ | vfat Disk Usage check | PASS | Checks for vfat Disk Usage. | +--------------------------------------+--------+-------------------------------------------------+ | Check /tmp usage | PASS | Checking for /tmp usage. | +--------------------------------------+--------+-------------------------------------------------+ ESX Host: 14.39.53.132 +--------------------------------------+--------+-------------------------------------------------+ | Name | Result | Comments | +--------------------------------------+--------+-------------------------------------------------+ | HX User Account check | PASS | Checks if HXUSER is present. | +--------------------------------------+--------+-------------------------------------------------+ | vMotion enabled check | PASS | Checks if vMotion is enabled on the host. | +--------------------------------------+--------+-------------------------------------------------+ | Check for ESXI Failback timer | PASS | Checks for ESXi FAILBACK timer set to 30000ms. | +--------------------------------------+--------+-------------------------------------------------+ | Check vmk1 ping to eth1 | PASS | Checks Network between ESXi vmk1 and SCVM eth1. | +--------------------------------------+--------+-------------------------------------------------+ | No extra controller vm folders check | PASS | Checks for duplicate Controller SCVM Folders. | +--------------------------------------+--------+-------------------------------------------------+ | VMware Tools location check | PASS | Checks for Non default VMware Tools location. | +--------------------------------------+--------+-------------------------------------------------+ | vfat Disk Usage check | PASS | Checks for vfat Disk Usage. | +--------------------------------------+--------+-------------------------------------------------+ | Check /tmp usage | PASS | Checking for /tmp usage. | +--------------------------------------+--------+-------------------------------------------------+ Main Report File: HX_Tool_Main_Report_2020-08-26_09-54-59.txt Report tar file: HX_Report_2020_08_26_09_43_18.tar Report file copied to path: /var/log/springpath Release Notes: https://www.cisco.com/c/en/us/support/hyperconverged-systems/hyperflex-hx-data-platform-software/products-release-notes-list.html Upgrade Guides: https://www.cisco.com/c/en/us/support/hyperconverged-systems/hyperflex-hx-data-platform-software/products-installation-guides-list.html Note: 1) If upgrading to HX 4.0(2a), please review the following link and perform workaround – https://tinyurl.com/wc7j5qp 2) Please check the status of Compute nodes manually, script only verifies the config on the converged nodes. 3) Hypercheck doesnot perform FAILOVER TEST, so please ensure that the upstream is configured for network connectivity for JUMBO or NORMAL MTU size as needed.
4) If you have performed any activity on your vcenter(like upgrade, certificate replacement,etc.), it is recommended to reregister your cluster to the vcenter before performing upgrades. Refer to the Hyperflex CLI guide for relevant commands. 


分析ツールの出力

次の手順

• このツールは、Hyperflexシステムで手動コマンドを実行するプロセスを自動化します。
• ツールがOKを実行し、すべてのテストでPASSを返した場合、HXシステムはスクリプトが実行したすべてのチェックに対して有効です。
• 一部のチェックでツールが失敗する、またはツールが正常に動作しない場合は、CLIコマンド（以下に示す）を使用して、スクリプトで手動で実行するHyperflexシステム上での同じチェックを実行できます。
• このツールでは、古い、新しい、オープンな、または解決された注意事項はチェックされません。そのため、アップグレードまたはメンテナンスを行う前に、Hyperflexリリースノートとアップグレードガイドを確認することを強くお勧めします。

注：スクリプトの実行に失敗したため、TACケースをオープンしないでください。コマンドを手動で実行し、問題を特定し、特定した問題のSRを開きます。

CLI コマンド

Hyperflex SCVM上：



SSH to All Hyperflex SCVMs-

# service_status.sh

# sysmtool --ns cluster --cmd enospcinfo

# echo srvr | nc 0 2181

# pidof exhibitor

# stcli disk list --ip <Corresponding ESXi Mgmt IP Address> |grep -B 2 -A 8 blacklisted

# stcli services dns show (and ping the IPs listed)

# ping <vCenter IP Address>

# date ; compare the time on all SCVMs. They should ideally be identical

# stcli services ntp show

# stcli cleaner info

# ntpq -p -4 

# dpkg -l | grep -i springpath | grep -v storfs-support*

# sysmtool --ns disk --cmd list | grep -i blacklisted

# iptables -L -n | wc -l 

# stcli cluster info

# df -h ; check that /var/stv should be less than 80%

# zgrep -i “out of memory” /var/log/springpath/debug-storfs.*

# ping -I eth0 <eth0> of all SCVMs

# ping -I eth1 <eth1> of all SCVMs

# "ls -ld /tmp" check for 775 and 777

# stcli cluster info | grep -i 'clusterAccessPolicy:' | head -1

# md5sum /etc/springpath/secure/springpath_keystore.jceks

# cat /etc/springpath/sed_capability.conf

# cat /etc/springpath/sed.conf

# cat /var/log/springpath/diskslotmap-v2.txt

# stcli cluster info | grep dataZkIp (ping dataZkIp for latency)


ESXiシステムの場合：



SSH to all ESXi hosts

# esxcli system account list

# esxcli network firewall ruleset list | grep -i vMotion

# esxcli software vib list| egrep -i 'scvm|stHyper|stfs'

# chkconfig --list | grep -E 'ntpd|hostd|vpxa|stHypervisorSvc|scvmclient|hxctlvm' 

# esxcfg-vmknic -l ; confirm that vMotion VMK2 is created

# vmkping -I vmk1 <eth1> of all SCVMs

# cd /vmfs/volumes/Springpath-XXXXXXXXXX ; Ensure that it has only one Folder that has the Storage Controller VM

# df -h | grep vfat ; Ensure dir has free space

関連情報

• シスコのテクニカルサポートとダウンロード