What's Actually Happening

CrashLoopBackOff is a Kubernetes state where a pod's container keeps crashing, Kubernetes restarts it, it crashes again, and after repeated failures, Kubernetes backs off (waits longer between restarts) but continues trying.

The Error You'll See

bash
$ kubectl get pods
NAME                    READY   STATUS             RESTARTS   AGE
my-app-abc123d-x1y2z    0/1     CrashLoopBackOff   5          10m

The STATUS column shows CrashLoopBackOff and RESTARTS keeps increasing.

Why This Happens

  1. 1.Application crash - Container process exits with non-zero code
  2. 2.Missing dependencies - App needs config, secrets, or files that don't exist
  3. 3.Health check failing - Liveness probe kills container repeatedly
  4. 4.Resource limits - Container hits memory limit and gets OOMKilled
  5. 5.Startup timeout - Readiness probe never succeeds, container killed
  6. 6.Command errors - Wrong command or arguments in container spec

Step 1: Describe the Pod

bash
kubectl describe pod my-app-abc123d-x1y2z

Look at the Events section and container state:

``` Containers: my-app: State: Waiting Reason: CrashLoopBackOff Last State: Terminated Reason: Error Exit Code: 1 Started: Mon, 03 Apr 2026 10:00:00 Finished: Mon, 03 Apr 2026 10:00:05

Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 10m default-scheduler Successfully assigned... Normal Pulled 9m (x4 over 10m) kubelet Container image pulled Normal Created 9m (x4 over 10m) kubelet Created container Normal Started 9m (x4 over 10m) kubelet Started container Warning BackOff 2m (x6 over 9m) kubelet Back-off restarting failed container ```

Step 2: Check Container Logs

```bash # Get logs from current attempt kubectl logs my-app-abc123d-x1y2z

# Get logs from previous container instance (crashed one) kubectl logs my-app-abc123d-x1y2z --previous ```

Look for: - Application error messages - Stack traces - Configuration errors - "Failed to", "Error", "Exception" keywords

Step 3: Check Exit Code Meaning

From describe output, note Exit Code:

Exit CodeMeaning
0Graceful exit (not usually crash)
1Application error
137SIGKILL (OOM killed - memory limit)
139SIGSEGV (segmentation fault)
143SIGTERM (graceful termination signal)

For exit code 137, check memory:

bash
kubectl describe pod my-app-abc123d-x1y2z | grep -A5 "Limits:"

Step 4: Check Liveness Probe

If liveness probe is configured wrong, it kills healthy containers:

bash
kubectl get pod my-app-abc123d-x1y2z -o yaml | grep -A10 "livenessProbe"

Example problematic config:

yaml
livenessProbe:
  httpGet:
    path: /health
    port: 8080
  initialDelaySeconds: 0  # Too short, app not ready
  periodSeconds: 1        # Too frequent

Fix by increasing initialDelaySeconds:

yaml
livenessProbe:
  httpGet:
    path: /health
    port: 8080
  initialDelaySeconds: 30  # Wait for app to start
  periodSeconds: 10
  failureThreshold: 3

Step 5: Check Resource Limits

bash
kubectl describe pod my-app-abc123d-x1y2z | grep -A10 "Containers:" | grep -A5 "Limits:"

If memory limit is too low:

bash
Limits:
  memory: 128Mi  # Too low for your app

Increase in deployment:

yaml
resources:
  limits:
    memory: 512Mi
  requests:
    memory: 256Mi

Step 6: Check Container Command

Wrong command in pod spec causes immediate crash:

bash
kubectl get pod my-app-abc123d-x1y2z -o yaml | grep -A5 "command:"

If command is wrong:

```yaml # Wrong: command: ["./app"] # File doesn't exist or wrong path

# Correct: command: ["python", "app.py"] # Or use entrypoint from image: # Don't specify command if image has correct ENTRYPOINT ```

Step 7: Check ConfigMaps and Secrets

If app needs config that's missing:

```bash # Check if configmap exists kubectl get configmap app-config

# Check if mounted correctly kubectl describe pod my-app-abc123d-x1y2z | grep -A10 "Mounts:" ```

Step 8: Debug with Interactive Shell

If logs don't help, run container interactively:

```bash # Run a debug pod with same image kubectl run debug --image=my-app-image:latest --rm -it --restart=Never -- sh

# Or use ephemeral container (Kubernetes v1.23+) kubectl debug my-app-abc123d-x1y2z -it --image=busybox ```

This lets you explore the container environment.

Step 9: Check for Image Issues

```bash # Verify image exists and runs docker run --rm my-app-image:latest

# Check image entrypoint docker inspect my-app-image:latest | grep -A10 "Entrypoint" ```

Verify the Fix

After fixing and redeploying:

```bash kubectl rollout restart deployment/my-app kubectl get pods -w

# Watch pod status change from Pending -> Running # No CrashLoopBackOff, RESTARTS stays at 0 ```

Prevention Tips

When deploying new apps:

```bash # Test container locally first docker run my-app-image:latest

# Set reasonable resource limits resources: limits: memory: 256Mi requests: memory: 128Mi

# Use proper liveness probe timing livenessProbe: initialDelaySeconds: 30 # App startup time periodSeconds: 10 ```