Introduction
JWT signature validation failed errors occur when the token's signature cannot be verified against the expected signing key, indicating potential tampering, key mismatch, or configuration issues. The error manifests as 401 Unauthorized responses with messages like Invalid signature, Signature verification failed, or Invalid token. JWT validation involves checking the signature, expiration (exp), issuer (iss), audience (aud), and optionally the nbf (not before) claim. Failures can stem from key rotation issues, algorithm confusion, clock skew between services, or malformed tokens.
Symptoms
- API returns
401 UnauthorizedwithInvalid signatureorInvalid tokenerror - Token decoded successfully but signature verification fails
- Authentication works intermittently across service instances
- Error appears after key rotation, service restart, or certificate renewal
- Logs show
JwtSignatureException,InvalidJwtException, or similar - Same token validates on some services but not others
Common Causes
- Signing key rotated but some services still using old key
- Algorithm mismatch (token signed with RS256, validator expects HS256)
- Clock skew between token issuer and validator services
- Issuer (
iss) or audience (aud) claim doesn't match expected values - Token modified in transit (proxy, load balancer, or middleware)
- Using public key as secret key (algorithm confusion vulnerability)
- Token expired or not yet valid (
nbfclaim in future)
Step-by-Step Fix
### 1. Decode and inspect the JWT token
Decode the token to examine claims and header:
```bash # Decode JWT without verification (for inspection only) # JWT format: header.payload.signature
# Using jq and base64 TOKEN="eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiYWRtaW4iOnRydWUsImlhdCI6MTUxNjIzOTAyMn0.signature"
# Decode header echo $TOKEN | cut -d'.' -f1 | base64 -d 2>/dev/null || echo $TOKEN | cut -d'.' -f1 | base64 -D
# Decode payload echo $TOKEN | cut -d'.' -f2 | base64 -d 2>/dev/null || echo $TOKEN | cut -d'.' -f2 | base64 -D
# Or use jwt.io CLI or online decoder # https://jwt.io/#debugger-io ```
Expected header:
json
{
"alg": "RS256",
"typ": "JWT",
"kid": "key-id-123"
}
Expected payload:
json
{
"sub": "user-123",
"iss": "https://auth.example.com",
"aud": "https://api.example.com",
"exp": 1680300000,
"iat": 1680296400,
"nbf": 1680296400
}
### 2. Verify signing key configuration
Check that the validator is using the correct key:
```bash # For HMAC (HS256, HS384, HS512) # The secret must match exactly between issuer and validator
# Check environment variable echo $JWT_SECRET | base64 -d # If stored base64 encoded
# For RSA (RS256, RS384, RS512) # Verify public key format openssl rsa -pubin -in public_key.pem -text -noout
# For ECDSA (ES256, ES384, ES512) # Verify EC public key openssl ec -pubin -in ec_public_key.pem -text -noout
# For JWKS (JSON Web Key Set) curl https://auth.example.com/.well-known/jwks.json | jq . ```
Key format requirements:
``` # RSA Public Key (PEM format) -----BEGIN PUBLIC KEY----- MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA... -----END PUBLIC KEY-----
# NOT this (private key format): -----BEGIN RSA PRIVATE KEY----- ```
### 3. Check for key rotation issues
During key rotation, both old and new keys must be valid:
json
// JWKS response during rotation
{
"keys": [
{
"kty": "RSA",
"kid": "key-2024-03",
"use": "sig",
"alg": "RS256",
"n": "...",
"e": "AQAB"
},
{
"kty": "RSA",
"kid": "key-2024-02",
"use": "sig",
"alg": "RS256",
"n": "...",
"e": "AQAB"
}
]
}
Validator must check all keys:
```python # Python with PyJWT and jwks import jwt from jwt import PyJWKClient
jwks_client = PyJWKClient("https://auth.example.com/.well-known/jwks.json")
def verify_token(token): try: signing_key = jwks_client.get_signing_key_from_jwt(token) payload = jwt.decode( token, signing_key.key, algorithms=["RS256"], issuer="https://auth.example.com", audience="https://api.example.com" ) return payload except jwt.ExpiredSignatureError: return {"error": "Token expired"} except jwt.InvalidIssuerError: return {"error": "Invalid issuer"} except jwt.InvalidAudienceError: return {"error": "Invalid audience"} except jwt.InvalidSignatureError: return {"error": "Invalid signature"} ```
- Key rotation checklist:
- Generate new key pair
- Add new key to JWKS with new
kid - Issuer starts signing new tokens with new key
- Wait for old tokens to expire (or force re-auth)
- Remove old key from JWKS
### 4. Verify algorithm configuration
Algorithm mismatch is a common failure mode:
```python # WRONG: Algorithm doesn't match token # Token signed with RS256, but validator expects HS256 jwt.decode(token, secret, algorithms=["HS256"]) # Fails!
# CORRECT: Match the algorithm jwt.decode(token, public_key, algorithms=["RS256"]) ```
Algorithm confusion attack prevention:
```python # VULNERABLE: Accepting multiple algorithms allows attack # Attacker can forge token with HS256 using public key as secret jwt.decode(token, public_key, algorithms=["RS256", "HS256"]) # DON'T DO THIS
# SECURE: Only accept asymmetric algorithms for public key validation jwt.decode(token, public_key, algorithms=["RS256"]) # Only RS256
# Or explicitly verify algorithm matches expected type def safe_decode(token, public_key): unverified_header = jwt.get_unverified_header(token)
# Reject symmetric algorithms when using public key if unverified_header.get("alg") in ["HS256", "HS384", "HS512"]: raise ValueError("Symmetric algorithm not allowed")
return jwt.decode(token, public_key, algorithms=["RS256", "RS384", "RS512"]) ```
Algorithm compatibility:
| Algorithm | Key Type | Key Size | Usage | |-----------|----------|----------|-------| | HS256 | Symmetric | 256-bit | Shared secret | | HS384 | Symmetric | 384-bit | Shared secret | | HS512 | Symmetric | 512-bit | Shared secret | | RS256 | RSA | 2048+ bit | Public/private key | | RS384 | RSA | 2048+ bit | Public/private key | | RS512 | RSA | 2048+ bit | Public/private key | | ES256 | ECDSA | P-256 | Elliptic curve | | ES384 | ECDSA | P-384 | Elliptic curve | | ES512 | ECDSA | P-521 | Elliptic curve |
### 5. Handle clock skew between services
Clock differences cause exp and nbf validation failures:
```python # WRONG: No clock skew tolerance jwt.decode(token, key, algorithms=["RS256"]) # Fails if clocks differ by even 1 second
# CORRECT: Add leeway for clock skew jwt.decode( token, key, algorithms=["RS256"], options={ "verify_exp": True, "verify_nbf": True, "verify_iat": True }, leeway=60 # 60 seconds tolerance ) ```
Check and fix clock synchronization:
```bash # Check system time date timedatectl status
# Check NTP sync timedatectl show-timesync --all
# Force NTP sync sudo timedatectl set-ntp true sudo systemctl restart systemd-timesyncd
# Or use ntpd sudo ntpdate -s time.nist.gov
# Verify sync status timedatectl | grep "System clock synchronized" # Should show: yes ```
Clock skew sources: - VM time drift (common in virtualized environments) - Container clock differences - Geographic distribution across time zones - NTP server unreachable
### 6. Verify issuer and audience claims
Mismatched iss or aud claims cause validation failures:
```python # Configuration expected_issuer = "https://auth.example.com" expected_audience = "https://api.example.com"
# Validation jwt.decode( token, key, algorithms=["RS256"], issuer=expected_issuer, audience=expected_audience, require=["iss", "aud", "exp", "sub"] ) ```
Common mismatches:
``` # Environment differences Development: iss = "http://localhost:8080" Production: iss = "https://auth.example.com"
# Trailing slash matters Expected: "https://auth.example.com" Actual: "https://auth.example.com/" # MISMATCH!
# Audience format Expected: "https://api.example.com" Actual: ["https://api.example.com", "https://mobile.example.com"] # Array vs string ```
Debug claim values:
python
# Decode without verification to see claims
import jwt
unverified = jwt.decode(token, options={"verify_signature": False})
print(f"Issuer: {unverified.get('iss')}")
print(f"Audience: {unverified.get('aud')}")
### 7. Check for token modification in transit
Proxies or middleware may corrupt the token:
```bash # Check if Authorization header is being forwarded correctly # Add logging to middleware
# Nginx configuration location /api/ { proxy_set_header Authorization $http_authorization; proxy_pass http://backend;
# Debug: log the header access_log /var/log/nginx/auth.log; if ($http_authorization ~ "Bearer (.+)") { set $jwt_token $1; } }
# Check for URL encoding issues # Token should NOT be URL-encoded when passed in header # Authorization: Bearer <token> (correct) # NOT: Authorization: Bearer Bearer%20eyJ... (wrong) ```
Common corruption sources:
- Double Bearer prefix: Bearer Bearer eyJ...
- URL encoding in header (should be plain base64)
- Load balancer stripping Authorization header
- API gateway transforming headers
### 8. Verify token not expired or future-dated
Check exp and nbf claims:
```python from datetime import datetime, timezone
# Decode to check timestamps unverified = jwt.decode(token, options={"verify_signature": False})
exp = unverified.get('exp') nbf = unverified.get('nbf') iat = unverified.get('iat')
print(f"Expired: {datetime.fromtimestamp(exp, tz=timezone.utc)}") print(f"Not Before: {datetime.fromtimestamp(nbf, tz=timezone.utc) if nbf else 'N/A'}") print(f"Issued At: {datetime.fromtimestamp(iat, tz=timezone.utc)}")
# Check current time print(f"Current: {datetime.now(timezone.utc)}") ```
Common timestamp issues:
- Token issued in future (issuer clock ahead)
- Token already expired (validator clock ahead)
- nbf (not before) claim in the future
- Very short token lifetime (< 1 minute)
### 9. Handle malformed tokens
Validate token format before parsing:
```python import re import jwt
def validate_jwt_format(token): # JWT must have exactly 2 dots if token.count('.') != 2: raise ValueError("Invalid JWT format: expected 3 parts separated by dots")
# Each part must be valid base64url parts = token.split('.') base64url_pattern = re.compile(r'^[A-Za-z0-9_-]*={0,2}$')
for i, part in enumerate(parts): if not base64url_pattern.match(part): raise ValueError(f"Invalid base64url encoding in part {i}")
return True
# Usage try: validate_jwt_format(token) payload = jwt.decode(token, key, algorithms=["RS256"]) except ValueError as e: print(f"Format error: {e}") except jwt.InvalidSignatureError: print("Signature verification failed") ```
### 10. Enable detailed JWT validation logging
Add logging for debugging:
```python import logging import jwt
logger = logging.getLogger(__name__)
def verify_jwt_with_logging(token, key): try: # Log token metadata (not the full token!) header = jwt.get_unverified_header(token) logger.info(f"Validating JWT: alg={header.get('alg')}, kid={header.get('kid')}")
payload = jwt.decode( token, key, algorithms=[header.get('alg')], issuer="https://auth.example.com", audience="https://api.example.com", leeway=60 )
logger.info(f"JWT valid for subject: {payload.get('sub')}") return payload
except jwt.ExpiredSignatureError: logger.error("JWT expired") raise except jwt.InvalidIssuerError: logger.error(f"JWT issuer mismatch: got {jwt.decode(token, options={'verify_signature': False}).get('iss')}") raise except jwt.InvalidAudienceError: logger.error(f"JWT audience mismatch") raise except jwt.InvalidSignatureError: logger.error("JWT signature verification failed") raise except jwt.DecodeError as e: logger.error(f"JWT decode error: {e}") raise ```
Prevention
- Use JWKS for automatic key rotation
- Set clock skew leeway of 30-60 seconds
- Monitor token validation failure rates
- Implement structured logging for JWT errors
- Use asymmetric algorithms (RS256, ES256) for distributed systems
- Document expected claim values for each environment
- Test key rotation procedure before production deployment
Related Errors
- **Token expired**:
expclaim in the past - **Invalid issuer**:
issclaim doesn't match expected - **Invalid audience**:
audclaim doesn't match expected - **Algorithm not allowed**: Token algorithm not in allowed list