Fix Vault Token Renewal Failed

What's Actually Happening

HashiCorp Vault tokens fail to renew, causing applications to lose access to secrets when tokens expire.

The Error You'll See

Renewal failed:

```bash $ vault token renew

Error renewing token: Error making API request. Code: 400. Errors: * invalid token or lease ID ```

Token expired:

Error: permission denied
token expired

Lease renewal error:

```bash $ vault lease renew database/creds/myapp/abc123

Error renewing lease: lease not found ```

AppRole login error:

Error: failed to renew token: Error making API request.
URL: PUT https://vault:8200/v1/auth/token/renew-self
Code: 403. Errors:
* permission denied

Why This Happens

1. 1.Token expired - Token TTL exceeded without renewal
2. 2.Root token - Root tokens don't auto-renew
3. 3.No renewable flag - Token created without renewable=true
4. 4.Policy missing - Token lacks renewal capability
5. 5.Max TTL exceeded - Token reached maximum TTL
6. 6.Token revoked - Token explicitly revoked

Step 1: Check Token Status

```bash # Check current token: vault token lookup

# Output: # display_name approle # creation_time 2024-01-01T00:00:00Z # ttl 1h # renewable true # policies ["default", "myapp"]

# Check token by accessor: vault token lookup -accessor <accessor>

# Check specific token: vault token lookup <token>

# Check token capabilities: vault token capabilities secret/data/myapp

# Check if token renewable: vault token lookup -format=json | jq '.data.renewable' # Must be true to renew ```

Step 2: Check Token TTL

```bash # Check token creation and expiration: vault token lookup -format=json | jq '{ creation: .data.creation_time, ttl: .data.ttl, expire: .data.expire_time }'

# Calculate remaining time: CREATION=$(vault token lookup -format=json | jq -r '.data.creation_time') TTL=$(vault token lookup -format=json | jq -r '.data.ttl')

# Check max TTL configuration: vault read sys/config/leases

# Default max TTL: 32 days (768 hours)

# Check role-specific TTL: vault read auth/approle/role/myrole

# Output: # token_ttl 1h # token_max_ttl 24h

# If token near max TTL, cannot renew further # Must generate new token ```

Step 3: Check Renewable Flag

```bash # Check if token is renewable: vault token lookup -format=json | jq '.data.renewable'

# If false, token cannot be renewed: # 1. Root tokens are never renewable # 2. Tokens created with renewable=false

# Check token creation method: vault token lookup -format=json | jq '.data'

# For AppRole, ensure role creates renewable tokens: vault read auth/approle/role/myrole

# Check renewable setting: vault read auth/approle/role/myrole -format=json | jq '.data.token_renewable' # Should be true

# Update role to allow renewable tokens: vault write auth/approle/role/myrole \ token_ttl=1h \ token_max_ttl=24h \ renewable=true

# For token create: vault token create -ttl=1h -renewable ```

Step 4: Check Policy Permissions

```bash # Check token policies: vault token lookup -format=json | jq '.data.policies'

# View policy content: vault policy read myapp-policy

# Policy must allow self-renewal: path "auth/token/renew-self" { capabilities = ["update"] }

# Or renew by accessor: path "auth/token/renew" { capabilities = ["update"] }

# Add renewal capability to policy: cat << 'EOF' | vault policy write myapp-policy - path "secret/data/myapp/*" { capabilities = ["read"] }

path "auth/token/renew-self" { capabilities = ["update"] }

path "sys/leases/renew" { capabilities = ["update"] } EOF

# Apply policy to role: vault write auth/approle/role/myrole policies="myapp-policy" ```

Step 5: Renew Token Correctly

```bash # Renew token incrementally: vault token renew

# Renew with specific increment: vault token renew -increment=2h

# Renew by accessor (requires permission): vault token renew -accessor <accessor>

# Renew lease (for dynamic secrets): vault lease renew database/creds/myapp/abc123

# Check renewal worked: vault token lookup

# TTL should be extended

# For long-running processes, use token renewer: vault token renew -increment=24h ```

Step 6: Configure Token Renewal in Applications

```go // Go Vault client with auto-renewal: config := api.DefaultConfig() config.Address = "https://vault:8200"

client, _ := api.NewClient(config)

// Set up token renewal: renewer, _ := client.NewRenewer(&api.RenewerInput{ Secret: &api.Secret{ Auth: &api.SecretAuth{ ClientToken: token, LeaseDuration: 3600, }, }, Increment: 3600, })

go renewer.Renew() defer renewer.Stop()

// Watch for renewal errors: for { select { case err := <-renewer.DoneCh(): if err != nil { log.Printf("Token renewal failed: %v", err) // Re-authenticate } case renewal := <-renewer.RenewCh(): log.Printf("Token renewed, new TTL: %d", renewal.Secret.Auth.LeaseDuration) } } ```

```python # Python hvac client: import hvac import time

client = hvac.Client(url='https://vault:8200') client.auth.approle.login(role_id, secret_id)

# Start token renewal thread: def renew_token(): while True: try: client.auth.token.renew_self() time.sleep(1800) # Renew every 30 minutes except Exception as e: print(f"Renewal failed: {e}") # Re-authenticate client.auth.approle.login(role_id, secret_id)

import threading renew_thread = threading.Thread(target=renew_token, daemon=True) renew_thread.start() ```

Step 7: Check Lease Configuration

```bash # Check lease TTL for secrets engine: vault read database/config/mydb

# Output: # default_lease_ttl 1h # max_lease_ttl 24h

# Update lease TTLs: vault write database/config/mydb \ default_lease_ttl="2h" \ max_lease_ttl="72h"

# Check lease configuration globally: vault read sys/config/leases

# Output: # default_lease_ttl 72h # max_lease_ttl 72h

# Update global lease config: vault write sys/config/leases \ default_lease_ttl=24h \ max_lease_ttl=720h ```

Step 8: Handle Lease Renewal for Dynamic Secrets

```bash # Check lease status: vault lease lookup database/creds/myapp/abc123

# Output: # expire_time 2024-01-01T01:00:00Z # ttl 1h

# Renew lease: vault lease renew database/creds/myapp/abc123

# Renew with increment: vault lease renew -increment=4h database/creds/myapp/abc123

# Check lease list: vault list sys/leases/lookup/database/creds/myapp

# Revoke lease: vault lease revoke database/creds/myapp/abc123

# Revoke all leases for prefix: vault lease revoke -prefix database/creds/myapp

# For auto-renewal, use vault agent: vault agent -config=/etc/vault/agent.hcl ```

Step 9: Use Vault Agent for Auto-Renewal

```hcl # vault-agent.hcl: pid_file = "/var/run/vault-agent.pid"

auto_auth { method "approle" { config = { role_id_file_path = "/etc/vault/role-id" secret_id_file_path = "/etc/vault/secret-id" } }

sink "file" { config = { path = "/var/run/vault-token" } } }

# Token renewal: template { source = "/etc/vault/templates/config.tmpl" destination = "/etc/app/config.json" }

# Agent automatically renews token # Applications use sink token ```

Step 10: Monitor Token Renewals

```bash # Create monitoring script: cat << 'EOF' > /usr/local/bin/monitor-vault-tokens.sh #!/bin/bash

echo "=== Current Token Info ===" vault token lookup -format=json | jq '{ttl: .data.ttl, renewable: .data.renewable, policies: .data.policies}'

echo "" echo "=== Vault Audit Log ===" tail -20 /var/log/vault/audit.log | grep -i "renew|token"

echo "" echo "=== Active Tokens (by accessor) ===" vault list auth/token/accessors | head -10

echo "" echo "=== Token Counts ===" vault list auth/token/accessors | wc -l

echo "" echo "=== Lease TTLs ===" vault read sys/config/leases EOF

chmod +x /usr/local/bin/monitor-vault-tokens.sh

# Prometheus metrics: curl http://localhost:8200/v1/sys/metrics | jq '.[] | select(.name | contains("token"))'

# Key metrics: # vault_token_renew_count # vault_token_renew_error_count # vault_token_expiration_time_seconds

# Alerts: - alert: VaultTokenRenewalFailed expr: rate(vault_token_renew_error_count[5m]) > 0 for: 2m labels: severity: warning annotations: summary: "Vault token renewal failures detected" ```

Vault Token Renewal Checklist

Verify the Fix

```bash # After fixing token renewal

# 1. Test renewal vault token renew // Token renewed, TTL extended

# 2. Check token status vault token lookup // renewable: true, TTL extended

# 3. Test application access vault read secret/data/myapp // Access successful

# 4. Monitor renewal over time vault token lookup | grep ttl # After 30 minutes, TTL should still be adequate

# 5. Test auto-renewal # If using Vault Agent, check logs journalctl -u vault-agent | grep renew // Regular renewals happening

# 6. Check lease renewal vault lease renew database/creds/myapp/<lease> // Lease renewed ```

Related Issues

• [Fix Vault Unseal Failed No Key](/articles/fix-vault-unseal-failed-no-key)
• [Fix Vault Lease Not Renewed](/articles/fix-vault-lease-not-renewed)
• [Fix Vault Secret Rotation Failed](/articles/fix-vault-secret-rotation-failed)