Introduction

After rotating SSH keys - whether for security compliance, employee offboarding, or scheduled key renewal - users may encounter Permission denied (publickey) errors. This happens when the new public key is not properly added to the server's authorized_keys file, the private key permissions are incorrect, or the SSH agent is still presenting the old key. Key rotation is a security best practice but requires careful coordination to avoid lockout.

Symptoms

  • ssh user@server returns Permission denied (publickey).
  • ssh -v user@server shows Offering public key: ... Server accepts key: no
  • Authentication works with password but not with key
  • ssh-add -l shows the wrong key loaded in the agent
  • Multiple keys in ~/.ssh/ and SSH tries the wrong one first

Common Causes

  • New public key not added to ~/.ssh/authorized_keys on the server
  • authorized_keys file has wrong permissions (should be 600 or 644)
  • Private key file permissions too open (should be 600)
  • SSH home directory permissions too open (should be 755 or stricter)
  • SSH agent holding stale key, presenting it before the new key
  • Key algorithm not supported by the server (e.g., ssh-rsa disabled)

Step-by-Step Fix

  1. 1.Enable verbose SSH output to see which key is being offered:
  2. 2.```bash
  3. 3.ssh -v user@server
  4. 4.# Look for: "Offering public key:" and "Server accepts key:"
  5. 5.`
  6. 6.Verify the public key is in authorized_keys on the server (via console or alternate access):
  7. 7.```bash
  8. 8.cat ~/.ssh/authorized_keys
  9. 9.# Verify your new public key is present
  10. 10.# Compare with your local public key:
  11. 11.cat ~/.ssh/id_ed25519.pub
  12. 12.`
  13. 13.Fix permissions on the server:
  14. 14.```bash
  15. 15.chmod 700 ~/.ssh
  16. 16.chmod 600 ~/.ssh/authorized_keys
  17. 17.chmod 755 ~ # Must not be group or world writable
  18. 18.# Verify SELinux context if applicable
  19. 19.restorecon -R ~/.ssh
  20. 20.`
  21. 21.Fix permissions on the client:
  22. 22.```bash
  23. 23.chmod 600 ~/.ssh/id_ed25519
  24. 24.chmod 644 ~/.ssh/id_ed25519.pub
  25. 25.chmod 700 ~/.ssh
  26. 26.`
  27. 27.Refresh the SSH agent with the new key:
  28. 28.```bash
  29. 29.# Remove all keys from agent
  30. 30.ssh-add -D
  31. 31.# Add the new key
  32. 32.ssh-add ~/.ssh/id_ed25519
  33. 33.# Verify
  34. 34.ssh-add -l
  35. 35.`
  36. 36.If ssh-rsa was disabled, generate a new ed25519 key:
  37. 37.```bash
  38. 38.ssh-keygen -t ed25519 -C "user@hostname" -f ~/.ssh/id_ed25519
  39. 39.# Copy to server
  40. 40.ssh-copy-id -i ~/.ssh/id_ed25519.pub user@server
  41. 41.`

Prevention

  • Use ssh-copy-id for key deployment - it handles permissions automatically
  • Implement centralized SSH key management (HashiCorp Vault, AWS EC2 Instance Connect)
  • Test new keys before removing old ones: keep both active during transition
  • Automate key rotation with configuration management (Ansible, Puppet)
  • Document the key rotation procedure including rollback steps