Fix Kubernetes VPA Not Working

Your Vertical Pod Autoscaler (VPA) is installed, but pods aren getting resource recommendations or updates. VPA should automatically optimize CPU and memory requests based on actual usage, but when it stops working, pods either run with incorrect resource settings or VPA recommendations aren applied.

Understanding VPA Components

VPA has three components: Recommender (analyzes metrics and generates recommendations), Updater (evicts pods to apply new recommendations), and Admission Controller (sets resources on new pods based on recommendations).

Each component can have issues: Recommender can't access metrics, Updater can't evict pods, or Admission Controller isn intercepting pod creation.

Diagnosis Commands

Check VPA components:

```bash # Check VPA components are running kubectl get pods -n kube-system | grep vpa

# Check VPA recommender kubectl get deployment vpa-recommender -n kube-system kubectl logs -n kube-system deployment/vpa-recommender

# Check VPA updater kubectl get deployment vpa-updater -n kube-system kubectl logs -n kube-system deployment/vpa-updater

# Check VPA admission controller kubectl get deployment vpa-admission-controller -n kube-system kubectl logs -n kube-system deployment/vpa-admission-controller ```

Check VPA resource:

```bash # Check VPA status kubectl get vpa vpa-name -n namespace

# Get detailed VPA information kubectl describe vpa vpa-name -n namespace

# Check recommendations kubectl get vpa vpa-name -n namespace -o yaml | grep -A 30 recommendation ```

Check metrics server:

# Verify metrics server is running (VPA needs it)
kubectl get pods -n kube-system -l k8s-app=metrics-server
kubectl top pods -n namespace

Common Solutions

Solution 1: Fix VPA Components Not Running

VPA components must be running:

```bash # Check component status kubectl get pods -n kube-system | grep vpa

# If missing, install VPA kubectl apply -f https://github.com/kubernetes/autoscaler/releases/latest/download/vertical-pod-autoscaler.yaml

# Or install from source git clone https://github.com/kubernetes/autoscaler.git cd autoscaler/vertical-pod-autoscaler ./hack/deploy-vpa.sh ```

Verify installation:

```bash # Check all VPA components kubectl get pods -n kube-system -l app=vpa-recommender kubectl get pods -n kube-system -l app=vpa-updater kubectl get pods -n kube-system -l app=vpa-admission-controller

# Check admission controller webhook kubectl get apiservices | grep vpa kubectl get mutatingwebhookconfigurations | grep vpa ```

Solution 2: Fix VPA Recommender Issues

The recommender analyzes metrics and generates recommendations:

```bash # Check recommender logs for errors kubectl logs -n kube-system deployment/vpa-recommender --tail=100

# Common errors: # - "failed to fetch metrics" - metrics server issue # - "no metrics available" - pod hasn't run long enough ```

Fix recommender configuration:

# Recommender deployment args
args:
  - --vpa-object-namespace=namespace  # Target namespace
  - --prometheus-address=http://prometheus:9090  # If using Prometheus
  - --storage=prometheus  # Use Prometheus for metrics storage

Check recommender can access metrics:

```bash # Verify metrics API is available kubectl get --raw "/apis/metrics.k8s.io/v1beta1/namespaces/default/pods" | jq .

# If using Prometheus, verify connection kubectl logs -n kube-system deployment/vpa-recommender | grep prometheus ```

Solution 3: Fix VPA Updater Issues

The updater evicts pods to apply recommendations:

```bash # Check updater logs kubectl logs -n kube-system deployment/vpa-updater --tail=100

# Common issues: # - Pods can't be evicted (PDB blocking) # - Updater in "Off" mode won't evict ```

Fix updater configuration:

# VPA with update mode
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: my-vpa
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-deployment
  updatePolicy:
    updateMode: "Auto"  # Options: Off, Initial, Recreate, Auto

UpdateMode options: - Off: Only recommendations, no updates - Initial: Only set on new pods - Recreate: Evict and recreate to apply - Auto: Automatically apply recommendations

Solution 4: Fix VPA Admission Controller Issues

The admission controller sets resources on new pods:

```bash # Check webhook configuration kubectl get mutatingwebhookconfigurations vpa-webhook

# Describe webhook kubectl describe mutatingwebhookconfigurations vpa-webhook

# Check admission controller logs kubectl logs -n kube-system deployment/vpa-admission-controller ```

Fix webhook configuration:

apiVersion: admissionregistration.k8s.io/v1
kind: MutatingWebhookConfiguration
metadata:
  name: vpa-webhook
webhooks:
  - name: vpa.k8s.io
    clientConfig:
      service:
        name: vpa-webhook-service
        namespace: kube-system
        path: "/validate"
    rules:
      - operations: ["CREATE", "UPDATE"]
        apiGroups: [""]
        apiVersions: ["v1"]
        resources: ["pods"]
    failurePolicy: Ignore  # Or Fail
    sideEffects: None

Solution 5: Fix Pod Disruption Budget Blocking Updates

VPA Updater can't evict pods protected by PDB:

# Check PDB settings
kubectl get pdb -n namespace
kubectl describe pdb pdb-name -n namespace

Adjust PDB to allow evictions:

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: my-pdb
spec:
  minAvailable: 1  # Allow some evictions
  selector:
    matchLabels:
      app: myapp

Or set VPA updateMode to "Initial" to avoid evictions:

spec:
  updatePolicy:
    updateMode: "Initial"  # Only applies on new pods

Solution 6: Fix Missing Metrics

VPA needs metrics to generate recommendations:

```bash # Check metrics server kubectl top pods -n namespace

# Pod must have run for some time (minimum 24 hours for accurate recommendations) kubectl get pods -n namespace -o jsonpath='{.items[*].metadata.creationTimestamp}' ```

Pods need time for VPA to gather metrics:

```bash # VPA needs at least 8 hours of metrics for basic recommendations # 24+ hours for stable recommendations

# Check if pod has metrics kubectl get vpa vpa-name -n namespace -o yaml | grep -A 10 "containerRecommendations" ```

Solution 7: Fix Container Names Mismatch

VPA must match container names exactly:

```bash # Check VPA container policy kubectl get vpa vpa-name -n namespace -o yaml | grep -A 20 "containerPolicies"

# Check deployment container names kubectl get deployment deployment-name -n namespace -o jsonpath='{.spec.template.spec.containers[*].name}' ```

Fix container name references:

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: my-vpa
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-deployment
  resourcePolicy:
    containerPolicies:
    - containerName: "main-container"  # Must match deployment container name
      minAllowed:
        cpu: "50m"
        memory: "64Mi"
      maxAllowed:
        cpu: "2"
        memory: "4Gi"

Solution 8: Check VPA Recommendation Status

VPA provides different recommendation states:

```bash # Check recommendation status kubectl get vpa vpa-name -n namespace -o yaml | grep -A 5 "status"

# Check recommendation message kubectl describe vpa vpa-name -n namespace ```

Status messages: - "No metrics available yet" - Pod hasn't run long enough - "No historical data" - Metrics server issue - "Recommendation provided" - Working correctly

Solution 9: Fix Resource Policy Constraints

VPA recommendations might be constrained:

# Check min/max constraints
resourcePolicy:
  containerPolicies:
  - containerName: "*"
    minAllowed:
      cpu: "100m"  # Minimum recommendation
      memory: "128Mi"
    maxAllowed:
      cpu: "500m"  # Maximum recommendation
      memory: "512Mi"
    controlledResources: ["cpu", "memory"]
    controlledValues: RequestsAndLimits

If recommendation exceeds maxAllowed, VPA won't apply it:

# Compare recommendation with maxAllowed
kubectl get vpa vpa-name -n namespace -o yaml | grep -B 2 -A 10 "maxAllowed"
kubectl get vpa vpa-name -n namespace -o yaml | grep -A 10 "target"

Solution 10: Check Mode Off

VPA in "Off" mode only provides recommendations:

```yaml # Check update mode kubectl get vpa vpa-name -n namespace -o jsonpath='{.spec.updatePolicy.updateMode}'

# If "Off", change to apply recommendations spec: updatePolicy: updateMode: "Auto" ```

Verification

After fixing VPA issues:

```bash # Check VPA components are running kubectl get pods -n kube-system | grep vpa

# Check VPA has recommendations kubectl get vpa vpa-name -n namespace -o yaml | grep -A 20 recommendation

# Check pod resources after restart kubectl get pods -n namespace -o jsonpath='{.items[*].spec.containers[*].resources}'

# Monitor VPA behavior kubectl describe vpa vpa-name -n namespace -w ```

VPA Update Modes Comparison

VPA Not Working Causes Summary

Prevention Best Practices

Install all VPA components (recommender, updater, admission controller). Ensure metrics server is running and accessible. Set appropriate updateMode based on application requirements. Configure minAllowed and maxAllowed for realistic bounds. Allow sufficient time for metrics collection before expecting recommendations. Use "Off" mode initially to test recommendations before enabling updates. Monitor VPA recommendations regularly.

VPA not working usually traces back to component installation or metrics access. Check that all three components are running, metrics server is accessible, and the VPA resource configuration matches your deployment.