Introduction

OpenSSH 7.8 (August 2018) changed the default private key format from PEM to the new OpenSSH format. Keys generated by newer versions of ssh-keygen may not be readable by older SSH clients (OpenSSH < 7.8), embedded systems, network equipment, or legacy applications. The error manifests as Load key ...: invalid format or unsupported key type when attempting to use the key with an older client.

Symptoms

  • ssh -i ~/.ssh/id_ed25519 user@old-server fails with Load key: invalid format
  • Older SSH client (OpenSSH 6.x) reports unsupported key type
  • ssh-keygen -l -f keyfile shows is not a key file
  • Network equipment (Cisco, Juniper) rejects the key when pasted into configuration
  • Java JSch library fails with invalid privatekey error

Common Causes

  • Key generated with OpenSSH 7.8+ using new format (begins with -----BEGIN OPENSSH PRIVATE KEY-----)
  • Older SSH client does not support the new OpenSSH key format
  • Third-party SSH libraries (JSch, paramiko older versions) lack support for new format
  • Converting from PEM to OpenSSH format accidentally with ssh-keygen -p
  • AWS, GCP metadata service accepting only PEM-format keys for some services

Step-by-Step Fix

  1. 1.Identify the key format:
  2. 2.```bash
  3. 3.head -1 ~/.ssh/id_rsa
  4. 4.# PEM format: -----BEGIN RSA PRIVATE KEY-----
  5. 5.# New OpenSSH format: -----BEGIN OPENSSH PRIVATE KEY-----
  6. 6.# RFC 4716 format: ---- BEGIN SSH2 PUBLIC KEY ----
  7. 7.`
  8. 8.Generate a new key in PEM format for legacy compatibility:
  9. 9.```bash
  10. 10.ssh-keygen -t rsa -b 4096 -m PEM -f ~/.ssh/id_rsa_legacy
  11. 11.# The -m PEM flag forces the older PEM format
  12. 12.`
  13. 13.Convert an existing OpenSSH format key to PEM:
  14. 14.```bash
  15. 15.ssh-keygen -p -m PEM -f ~/.ssh/id_rsa
  16. 16.# This converts the private key in-place to PEM format
  17. 17.# Warning: This modifies the original key file
  18. 18.`
  19. 19.For ed25519 keys, note that PEM format is not supported:
  20. 20.```bash
  21. 21.# ed25519 keys are always in OpenSSH format
  22. 22.# If you need legacy compatibility, use RSA instead:
  23. 23.ssh-keygen -t rsa -b 4096 -m PEM -f ~/.ssh/id_rsa_compat
  24. 24.`
  25. 25.Convert public key to RFC 4716 format for network equipment:
  26. 26.```bash
  27. 27.ssh-keygen -e -f ~/.ssh/id_rsa.pub -m RFC4716
  28. 28.# Output can be pasted into Cisco/Juniper device configs
  29. 29.`
  30. 30.Test compatibility with the target system:
  31. 31.```bash
  32. 32.ssh -V
  33. 33.# Check the SSH version on both client and server
  34. 34.ssh -i ~/.ssh/id_rsa_legacy -o PubkeyAcceptedKeyTypes=+ssh-rsa user@old-server
  35. 35.`

Prevention

  • Use -m PEM when generating keys that need to work with legacy systems
  • Standardize on ed25519 keys for modern systems and RSA PEM for legacy compatibility
  • Maintain a key inventory documenting which format is used for which target
  • Test new key generation against the oldest SSH version in your environment
  • Document key format requirements in your SSH key management policy