Troubleshooting SSH keys

SSH is one of those workhorse utilities that make lots of things work. Whole applications are built on top of ssh (for instance, Ansible).  Being able to set up ssh keys is, from a functional perspective, what makes this really work well. Sometimes when setting up SSH keys things fail to work and it is not at all clear why not. A lot of these are fairly simple things that amount to user error. The problem with user error is that it can be hard to detect when you are falling into that.

First..just for completeness.. what is the procedure for setting up ssh keys?  Fortunately this process is well documented on the Interwebz. See this page for instance, which is pretty readable.  Where these pages stop is in troubleshooting (which is why I have this post here.)

So… here’s a list of things to check….

  1. When you copied your public key to the remote host, the permissions on the directory and/or file are too open. SSH requires your .ssh directory to only be readable by the user you are logging in as (well, and root..)
  2. When you try to ssh into your remote server, it fails because you didn’t specify the right key name, or made a typo.
  3. You copied your public key to the server with the name ‘authorized_keys2’ and your ssh daemon was expecting ‘authorized_keys’. (Or vice-versa.)  It seems that the preferred thing now is just to use ‘authorized_keys’.  The exact name is set in the /etc/ssh/sshd_config file with the parameter name ‘AuthorizedKeysFile’.
  4. Often you will have more than one set of ssh keys on your laptop for different servers. Maybe you have one set for work related machines and another for a side project. When generating ssh keys you are prompted to give the filename, and so you do that. So… then you try connecting to your server … and it isn’t working?  “Why am I still being prompted for my password?” Well, you didn’t send the right key.  Again, your local .ssh/config is your friend here (instead of having to specify the key all the time with -i).  Add an entry IdentityFile=~/.ssh/my_server.rsa and on your laptop the appropriate ssh key will be sent to the server.
  5. Bonus points if you set up a different port for your ssh daemon to listen on. An easy mistake to make though is to neglect to use that non-standard port when you connect.  In your ssh config you will want to add this by just specifying the port with simply ‘Port’.

Running sshd manually

If you are still having problems, and if you are able to become root and manually run the sshd daemon in debug mode then you may find that helpful. Then you can watch both the client side and server side interact and see if you get any helpful messages from the daemon.

To run the daemon manually:

/usr/sbin/sshd -p 8888 -D -d -e

Update

So with OpenSSH 7 DSA keys have been deprecated by default.  If you suddenly find that you are prompted for a password or you just plain can’t get in at all then this may be the problem.  If you run ssh with the -v verbose flag then you may see this message:

Skipping ssh-dss key .ssh/whatever_id_dsa - not in PubkeyAcceptedKeyTypes

(I happened to notice this after I upgraded to Mac OS X Sierra recently.)

So, you can override this behavior by adding a line in your ssh config on your own machine  by adding a ‘PubkeyAcceptedKeyTypes‘ line…

Host myserver.org
     User me
     PubkeyAcceptedKeyTypes +ssh-dss
     IdentitiesOnly yes
     IdentityFile=~/.ssh/myserver_id_dsa
     Hostname=myserver.org
     TCPKeepAlive yes
     ServerAliveInterval 15

But probably the better thing to do is to regenerate your ssh keys and use RSA instead.  Of course you might temporarily need to add this override in order to get back into your account to update the key 😉