Introduction

HashiCorp Vault starts in a sealed state, meaning its encryption keys are not loaded into memory and all secret operations are blocked. To unseal Vault, operators must provide a threshold of unseal keys (Shamir's Secret Sharing). If fewer than the threshold number of keys are provided, Vault remains sealed and all API requests return a 503 error.

Symptoms

  • All Vault API requests return Error reading seal status: Vault is sealed
  • vault status shows Sealed: true and Progress: 2/3 (below threshold)
  • Applications cannot retrieve secrets, causing service disruptions
  • Vault UI shows Vault is Sealed banner
  • Error message: Error making API request: Code: 503. Vault is sealed

Common Causes

  • Vault restarted after maintenance or crash and unseal process was not completed
  • Insufficient key holders available to provide the threshold number of unseal keys
  • Unseal keys lost or not securely stored after initial Vault initialization
  • Auto-unseal (cloud KMS) configuration broken or KMS endpoint unreachable
  • Network partition preventing key holders from accessing the Vault instance

Step-by-Step Fix

  1. 1.Check the current seal status and progress: Determine how many keys have been provided.
  2. 2.```bash
  3. 3.vault status
  4. 4.# Output shows:
  5. 5.# Sealed: true
  6. 6.# Key Progress: 2/3
  7. 7.`
  8. 8.Provide the remaining unseal keys: Distribute keys to authorized operators.
  9. 9.```bash
  10. 10.# Each key holder runs this command separately
  11. 11.vault operator unseal <unseal-key-1>
  12. 12.vault operator unseal <unseal-key-2>
  13. 13.vault operator unseal <unseal-key-3>
  14. 14.`
  15. 15.Verify Vault is unsealed: Confirm the seal state changed.
  16. 16.```bash
  17. 17.vault status
  18. 18.# Should show: Sealed: false
  19. 19.`
  20. 20.If using auto-unseal, check the KMS connection: Verify the cloud KMS is accessible.
  21. 21.```bash
  22. 22.# Check Vault configuration for auto-unseal
  23. 23.grep -A5 "seal" /etc/vault.d/vault.hcl
  24. 24.# Verify KMS credentials
  25. 25.aws kms describe-key --key-id alias/vault-unseal
  26. 26.`
  27. 27.Verify applications can access secrets again: Test secret retrieval.
  28. 28.```bash
  29. 29.vault kv get secret/my-app/database
  30. 30.`

Prevention

  • Configure auto-unseal with cloud KMS (AWS KMS, Azure Key Vault, GCP KMS) to eliminate manual unseal
  • Store unseal keys in a secure, distributed key management system accessible to multiple operators
  • Set unseal threshold to balance security and availability (e.g., 3 of 5)
  • Document the unseal procedure in an incident runbook with key holder contact information
  • Monitor Vault seal status and alert immediately when Vault becomes sealed
  • Test the unseal process regularly in disaster recovery drills