What's Actually Happening

A PersistentVolumeClaim (PVC) requests storage from the cluster. When PVC is Pending, Kubernetes cannot find a matching PersistentVolume (PV) or cannot dynamically provision one. The pod waiting for this PVC cannot start.

The Error You'll See

```bash $ kubectl get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE my-pvc Pending standard 5m

$ kubectl describe pvc my-pvc ... Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal WaitForFirstConsumer 5m persistentvolume-controller waiting for first consumer to be created before binding ```

Or with provisioning failure:

bash
Events:
  Warning  ProvisioningFailed    5m    persistentvolume-controller  storageclass.storage.k8s.io "fast" not found

Why This Happens

  1. 1.No matching PV - No available PV meets PVC requirements
  2. 2.StorageClass missing - Referenced StorageClass doesn't exist
  3. 3.Provisioner not available - CSI driver or provisioner not running
  4. 4.Capacity too large - Requested size exceeds available capacity
  5. 5.Access mode mismatch - PVC wants RWX but PV only RWO
  6. 6.WaitForFirstConsumer - Volume waits for pod to schedule

Step 1: Check PVC Details

bash
kubectl describe pvc my-pvc

Look at: - StorageClass - Which class is requested - Access Modes - RWO, RWX, ROX - Capacity - Size requested - Events for failure reason

Step 2: Check StorageClass Exists

bash
kubectl get storageclass

Shows available classes:

bash
NAME       PROVISIONER            RECLAIMPOLICY   VOLUMEBINDINGMODE      AGE
standard   kubernetes.io/aws-ebs  Delete          Immediate              10d
fast       kubernetes.io/aws-ebs  Delete          WaitForFirstConsumer   10d

If PVC references a missing class:

yaml
storageClassName: fast-ssd  # This doesn't exist

Either create the StorageClass or change PVC:

yaml
storageClassName: standard  # Use existing class

Step 3: Check Available PersistentVolumes

bash
kubectl get pv

Shows available volumes:

bash
NAME       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS
pv-001     10Gi       RWO            Retain           Available           standard
pv-002     5Gi        RWO            Retain           Available           standard

If STATUS is not Available or capacity/access don't match, PVC cannot bind.

Step 4: Check PVC Requirements Match PV

PVC requirements:

bash
kubectl get pvc my-pvc -o yaml | grep -A15 "spec:"

Shows:

yaml
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi
  storageClassName: standard

PV must have: - Same or larger capacity - Same access mode - Same StorageClass (or no class if PVC has none)

Step 5: Check Dynamic Provisioner

If using dynamic provisioning:

bash
kubectl get pods -n kube-system | grep provisioner

For CSI drivers:

bash
kubectl get csinodes
kubectl get csidrivers

If provisioner pod not running:

bash
kubectl logs -n kube-system csi-driver-xxx
kubectl describe pod -n kube-system csi-driver-xxx

Step 6: Check WaitForFirstConsumer Mode

Some StorageClasses use WaitForFirstConsumer binding:

yaml
volumeBindingMode: WaitForFirstConsumer

This delays binding until pod that uses PVC is scheduled.

Check if pod exists:

bash
kubectl get pods -o wide
kubectl describe pod my-pod

Pod might be Pending for other reasons (node selector, resources).

Step 7: Create Matching PV Manually

If no PV available, create one:

yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-for-my-pvc
spec:
  capacity:
    storage: 20Gi
  accessModes:
  - ReadWriteOnce
  storageClassName: standard
  hostPath:  # For testing only
    path: /mnt/data
  # Or for cloud:
  # awsElasticBlockStore:
  #   volumeID: vol-xxx
  #   fsType: ext4
bash
kubectl apply -f pv.yaml

Step 8: Fix PVC Size

If requested size too large:

yaml
resources:
  requests:
    storage: 100Gi  # Too large for available PVs

Reduce to available size:

yaml
resources:
  requests:
    storage: 10Gi  # Matches available PV

Step 9: Check Cloud Provider Limits

For cloud storage (AWS EBS, GCE PD):

```bash # AWS EBS limits per region # Maximum size: 16TiB # Some types have min size (io1: min 4GiB)

kubectl describe pvc my-pvc | grep -i "cannot create" ```

Verify the Fix

After fixing:

```bash kubectl get pvc # STATUS should be Bound

NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS my-pvc Bound pv-12345 20Gi RWO standard

# Pod using PVC should start kubectl get pods ```

Prevention Tips

When creating PVCs:

```yaml # Use realistic size resources: requests: storage: 10Gi # Match available capacity

# Use existing StorageClass kubectl get storageclass storageClassName: standard

# Check access mode support accessModes: - ReadWriteOnce # Most common, widely supported

# For dynamic provisioning, verify provisioner runs kubectl get pods -n kube-system | grep provisioner ```