Fix Velero Restore Failed

What's Actually Happening

Velero restore operation fails when attempting to restore Kubernetes resources from a backup. Resources are not recreated in the target cluster.

The Error You'll See

```bash $ velero restore create --from-backup my-backup

Restore failed: velero/restore-xxx Phase: Failed Errors: 5 Warnings: 10 ```

Resource conflict:

Error: resource "default/nginx-deploy" already exists and is not included in the backup

Hook error:

Error: post-hook execution failed: command terminated with exit code 1

Backup not found:

Error: backup "my-backup" not found

PV binding failed:

Error: persistentvolumeclaim "data-pvc" failed to bind: no matching persistent volume

Why This Happens

1. 1.Backup corrupted - Backup data incomplete or corrupted
2. 2.Resource conflicts - Target cluster has existing resources
3. 3.Namespace missing - Target namespace not exists
4. 4.Storage class mismatch - PV storage class not matching
5. 5.Hook failures - Restore hooks failing
6. 6.CRD missing - Custom Resource Definition not installed

Step 1: Check Restore Status

```bash # List all restores: velero restore get

# Check specific restore: velero restore describe restore-xxx

# Get restore details: velero restore get restore-xxx -o yaml

# Check restore logs: velero restore logs restore-xxx

# Check restore errors: velero restore describe restore-xxx | grep -A10 "Errors"

# View restore warnings: velero restore describe restore-xxx | grep -A10 "Warnings"

# Check restore phase: velero restore get restore-xxx -o jsonpath='{.status.phase}'

# Check resource counts: velero restore get restore-xxx -o jsonpath='{.status.progress}' ```

Step 2: Verify Backup Integrity

```bash # Check backup exists: velero backup get

# Describe backup: velero backup describe my-backup

# Check backup status: velero backup get my-backup -o jsonpath='{.status.phase}' # Should be "Completed"

# Check backup expiration: velero backup get my-backup -o jsonpath='{.status.expiration}'

# Verify backup in storage: # AWS S3: aws s3 ls s3://velero-backups/<bucket-prefix>/backups/my-backup/

# Azure Blob: az storage blob list --container-name velero-backups --prefix backups/my-backup

# GCS: gsutil ls gs://velero-backups/backups/my-backup/

# Check backup tarball: velero backup download my-backup --output my-backup.tar.gz tar -tzf my-backup.tar.gz | head -20

# Verify backup contents: velero backup describe my-backup --details ```

Step 3: Handle Resource Conflicts

```bash # Check if resources exist in target: kubectl get deploy -n default nginx-deploy

# Velero restore strategies:

# Option 1: Delete existing resources: kubectl delete deploy nginx-deploy -n default

# Option 2: Use --existing-resource-policy: velero restore create --from-backup my-backup \ --existing-resource-policy=update # Updates existing resources instead of error

# Option 3: Use --restore-volumes=false for PVC conflicts: velero restore create --from-backup my-backup \ --restore-volumes=false

# Option 4: Exclude resources: velero restore create --from-backup my-backup \ --exclude-resources=deployments.apps

# Option 5: Include only specific resources: velero restore create --from-backup my-backup \ --include-resources=secrets,configmaps

# Check what resources are in backup: velero backup describe my-backup --details | grep "Resources included" ```

Step 4: Create Target Namespace

```bash # Check namespace exists: kubectl get namespace my-namespace

# Velero doesn't create namespaces by default for cluster backups

# Create namespace before restore: kubectl create namespace my-namespace

# Or use Velero to restore namespace: velero restore create --from-backup my-backup \ --include-namespaces=my-namespace

# Check backup namespace scope: velero backup describe my-backup | grep "Namespaces included"

# If backup was namespace-scoped: velero restore create --from-backup my-backup \ --namespace-mappings=my-namespace:target-namespace

# Check namespace in restore result: velero restore describe restore-xxx | grep "Namespaces" ```

Step 5: Fix Storage Class Issues

```bash # Check storage classes in target cluster: kubectl get storageclass

# Check PVCs in backup: velero backup describe my-backup --details | grep persistentvolumeclaim

# Check storage class used in backup: velero backup download my-backup --output my-backup.tar.gz tar -xzf my-backup.tar.gz -O resources/persistentvolumeclaims/cluster/data-pvc.json | jq '.spec.storageClassName'

# Create matching storage class if missing: kubectl apply -f - <<EOF apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: standard provisioner: kubernetes.io/aws-ebs parameters: type: gp2 EOF

# Or change PVC storage class: # In restore, edit PVC to use existing storage class

# Use volume snapshot restore: velero restore create --from-backup my-backup \ --restore-volumes=true \ --volume-snapshot-locations=default

# Check volume snapshots: velero volume-snapshot get ```

Step 6: Check Restore Hooks

```bash # Check restore hooks in backup: velero backup describe my-backup | grep -A5 "Hooks"

# View hook definition: velero backup get my-backup -o yaml | grep -A20 "hooks"

# Hook execution errors: velero restore logs restore-xxx | grep -i "hook"

# Disable hooks: velero restore create --from-backup my-backup \ --hooks=false

# Or skip specific hooks: velero restore create --from-backup my-backup \ --restore-hook-resources= pods \ --restore-hook-actions= exec

# Check hook container: kubectl get pods -n my-namespace

# Verify hook command works manually: kubectl exec -n my-namespace my-pod -- /restore-hook.sh

# Add exec hook to restore: velero restore create --from-backup my-backup \ --post-hook-exec-container=main \ --post-hook-exec-command=/bin/bash,-c,restore.sh \ --post-hook-exec-on-error=fail ```

Step 7: Install Missing CRDs

```bash # Check CRDs in backup: velero backup describe my-backup --details | grep customresourcedefinition

# Check CRDs in target cluster: kubectl get crd

# Install missing CRDs before restore: kubectl apply -f crd-definition.yaml

# Common CRDs to check: # - PrometheusRules (monitoring) # - ServiceMonitors (monitoring) # - IngressRoutes (traefik) # - CertificateRequests (cert-manager) # - ArgoCD Applications

# Or restore CRDs first: velero restore create --from-backup my-backup \ --include-resources=customresourcedefinitions.apiextensions.k8s.io

# Then restore other resources: velero restore create --from-backup my-backup \ --exclude-resources=customresourcedefinitions.apiextensions.k8s.io ```

Step 8: Check Velero Server Logs

```bash # Check Velero server logs: kubectl logs -n velero deploy/velero

# Recent logs: kubectl logs -n velero deploy/velero --tail=100

# Filter restore errors: kubectl logs -n velero deploy/velero | grep -i restore

# Check restore process: kubectl logs -n velero deploy/velero -f

# Describe Velero pod: kubectl describe pods -n velero

# Check Velero resources: kubectl top pods -n velero

# Check Velero service: kubectl get svc -n velero

# Check backup storage location: velero backup-location get

# Check volume snapshot location: velero snapshot-location get ```

Step 9: Retry Restore with Debug

```bash # Enable debug logging: kubectl set env deploy/velero -n velero LOG_LEVEL=debug

# Retry restore: velero restore create --from-backup my-backup --wait

# Watch restore progress: velero restore get restore-xxx -w

# Check detailed restore output: velero restore describe restore-xxx --details

# Use dry-run mode: velero restore create --from-backup my-backup --dry-run

# Verify resources would restore: velero restore get restore-xxx -o yaml | grep -A10 "resourcesToRestore"

# Create incremental restore: velero restore create --from-backup my-backup \ --include-namespaces=my-namespace \ --include-resources=pods,deployments ```

Step 10: Velero Restore Verification Script

```bash # Create verification script: cat << 'EOF' > /usr/local/bin/check-velero-restore.sh #!/bin/bash

RESTORE=${1:-"restore-xxx"}

echo "=== Restore Status ===" velero restore get $RESTORE

echo "" echo "=== Restore Phase ===" velero restore get $RESTORE -o jsonpath='{.status.phase}'

echo "" echo "=== Restore Errors ===" velero restore describe $RESTORE | grep -A20 "Errors"

echo "" echo "=== Restore Warnings ===" velero restore describe $RESTORE | grep -A20 "Warnings"

echo "" echo "=== Backup Status ===" BACKUP=$(velero restore get $RESTORE -o jsonpath='{.spec.backupName}') velero backup get $BACKUP

echo "" echo "=== Backup Details ===" velero backup describe $BACKUP --details | head -30

echo "" echo "=== Velero Server Logs ===" kubectl logs -n velero deploy/velero --tail=20 | grep -i $RESTORE

echo "" echo "=== Storage Locations ===" velero backup-location get

echo "" echo "=== Recommendations ===" echo "1. Verify backup is completed and not expired" echo "2. Check existing resources for conflicts" echo "3. Create target namespaces if needed" echo "4. Verify storage classes exist" echo "5. Install missing CRDs before restore" echo "6. Check restore hooks are valid" echo "7. Use --existing-resource-policy=update for conflicts" EOF

chmod +x /usr/local/bin/check-velero-restore.sh

# Usage: /usr/local/bin/check-velero-restore.sh restore-xxx ```

Velero Restore Checklist

Verify the Fix

```bash # After fixing Velero restore issues

# 1. Create restore velero restore create --from-backup my-backup // Restore created

# 2. Check restore phase velero restore get restore-xxx // Phase: Completed

# 3. Verify resources restored kubectl get all -n my-namespace // Resources created

# 4. Check PVC binding kubectl get pvc -n my-namespace // PVCs Bound

# 5. Verify pods running kubectl get pods -n my-namespace // Pods Running

# 6. Check restore logs velero restore logs restore-xxx // No errors ```

Related Issues

• [Fix Velero Backup Failed](/articles/fix-velero-backup-failed)
• [Fix Kubernetes Deployment Not Working](/articles/fix-kubernetes-deployment-not-working)
• [Fix Kubernetes PVC Pending State](/articles/fix-kubernetes-pvc-pending-state)