Troubleshoot SSH Host Key Verification Failure

Introduction

This document describes the process to fix the SSH Host key verification failed issue.

Background

This error occurs when a user tries to SSH to DNACenter's grapevine after DNACenter image was upgraded. So far, this issue has only been seen in macOS users.

Description

After DNACenter is upgraded to a new image, the local user's SSH key registry file maintains the SSH key used to connect to the previous DNACenter's grapevine build. This is the error:

MPAVLOVI-M-902T:~ mpavlovi$ ssh grapevine@172.16.21.86 -p 2222
       @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
       @    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
       @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
       IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
       Someone could be eavesdropping on you right now (man-in-the-middle attack)!
       It is also possible that a host key has just been changed.
       The fingerprint for the ECDSA key sent by the remote host is
       SHA256:+dnn+NRIXTDMmgpUNbeqdjDQscabBBWmC35F01ZFnLs.
       Please contact your system administrator.
       Add correct host key in /Users/mpavlovi/.ssh/known_hosts to get rid of this message.
       Offending ECDSA key in /Users/mpavlovi/.ssh/known_hosts:16
       ECDSA host key for [172.16.21.86]:2222 has changed and you have requested strict checking.
       Host key verification failed.
MPAVLOVI-M-902T:~ mpavlovi$

Solution

In order to solve this issue, the previous grapevine SSH key needs to be deleted from the registry file known_host at the local directory ~/.ssh/.

To delete the previous SSH keys, go to terminal and use your text editor of choice to open the file:
      

MPAVLOVI-M-902T:~ mpavlovi$ vim ~/.ssh/known_hosts 

Proceed to delete the line that includes the DNACenter IP address in it

[10.197.218.12]:2222 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKkToWAUheuqt876tDOrWwDSH5HbL0TLMw5MAsOsLqCnb3jRn5oxIJn2yECB1HPamglO/m79o2W8mAGAjypyFkw=
[172.16.21.86]:2222 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKz4ObQLQE7VEHZLOYL0t6k8aqbvFGIFDXsVaTguchsSNyGScFa6PLJKCZj/S7YIultUTYH94NZv4pCl509svvk=

Note: Modifying other keys can affect your SSH access to other devices/sites. Delete ONLY the necessary SSH key/s.

[10.197.218.12]:2222 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKkToWAUheuqt876tDOrWwDSH5HbL0TLMw5MAsOsLqCnb3jRn5oxIJn2yECB1HPamglO/m79o2W8mAGAjypyFkw=

Save the changes in the text editor and retry to SSH to DNACenter grapevine. Type the word "yes" when prompt.

MPAVLOVI-M-902T:~ mpavlovi$ ssh grapevine@172.16.21.86 -p 2222 
The authenticity of host '[172.16.21.86]:2222 ([172.16.21.86]:2222)' can't be established.
ECDSA key fingerprint is SHA256:+dnn+NRIXTDMmgpUNbeqdjDQscabBBWmC35F01ZFnLs.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '[172.16.21.86]:2222' (ECDSA) to the list of known hosts.

Use grapevine credential password.

grapevine@172.16.21.86's password: 

Welcome to the Cisco APIC-EM Appliance - Powered by Grapevine

  System information as of Wed Jul 19 22:20:17 UTC 2017

  System load:  1.02                Users logged in:          1
  Usage of /:   11.7% of 365.80GB   IP address for eth0:      12.99.1.2
  Memory usage: 41%                 IP address for eth1:      172.16.21.86
  Swap usage:   0%                  IP address for grape-br0: 169.254.0.1
  Processes:    695

  APIC-EM Version:   2.0.0.3757
  Grapevine Version: 2.0.0.3757.dev1065-ge50d0c2

Last login: Wed Jul 19 21:45:22 2017 from 10.41.49.41
(grapevine)
[Wed Jul 19 22:20:18 UTC] grapevine@12.99.1.2 (grapevine-root-1) ~
$