This was an interesting "gotcha", thanks to my colleague @edmsanchez13 which I know happens in a fully patched 5.5 and 6.5.
PuTTY by default only shows SSH key fingerprints in the md5 format, such as
(By the way, you can see all SSH keys PuTTY has learned by going in regedit to HKEY_CURRENT_USER\Software\SimonTatham\PuTTY\SshHostKeys . If you want to see the fingerprint for a known host again like if you were connecting for the first time, just delete it from here).
It used to be you could confirm this md5 hash in your host console. You should see this on the host console, in the Troubleshooting, View Support Information, section:
Notice how the SSH key in md5 format aa:bb:cc:dd from PuTTY does not match the SSH key in SHA256 format shown in the host? That's because since OpenSSH 6.8 "the fingerprint is now displayed as base64 SHA256 (by default)", "The default changes from MD5 to SHA256 and format from hex to base64.". This means ESXi now uses the SHA256 format as well to present to you the SSH key fingerprint.
ESXi uses OpenSSH (as does the rest of the world, thanks to OpenBSD) and is correct in leaving this default on. All ssh binaries are in this directory and you can check the OpenSS version with -V
/usr/lib/vmware/openssh/bin] ssh -V
OpenSSH_7.3p1, OpenSSL 1.0.2j-fips 26 Sep 2016
I can't find an option for PuTTY to show me the new SHA256 fingerprint; so - how is anyone in Windows proving the SSH pub key hash is correct, before connecting to a host?
Off the bat, I can think of two ways:
1) Confirm from the host's console
You can verify the SSH fingerprint that PuTTY shows you by asking for the md5 fingerprint - this is done with this command, using the stored host keys:
/usr/lib/vmware/openssh/bin] ./ssh-keygen -l -f /etc/ssh/ssh_host_rsa_key.pub -E md5
2048 MD5:41:dd:b9:ec:ba:c0:ae:c7:9a:2a:21:f7:fd:23:96:91 no comment (RSA)
2) find a client that uses the new format. I had no problem from Ubuntu, for example
However, if you are in Windows, you are most probably using PuTTY (unless you are paying for SecureCRT). If PuTTY won't show the new SHA256 fingerprints, what clients will? I found one that is free to use, even inside organizations, called Bitvise. This client shows bothMD5 and SHA256 fingerprints; additionally, it seems to be quite handy as it immediately brings up a WinSCP like window for file transfers, so I'll be testing this as my SSH client going forward.
Good links that helped me do this post