What's Actually Happening

Linkerd proxy sidecar is not injected into pods. Pods start without the Linkerd proxy, preventing service mesh functionality.

The Error You'll See

No sidecar in pod:

```bash $ kubectl get pods

NAME READY STATUS RESTARTS myapp-abc123 1/1 Running 0 # Expected 2/2 with Linkerd sidecar ```

Injection annotation missing:

```bash $ kubectl describe pod myapp-abc123 | grep -i linkerd

# No Linkerd annotations found ```

Admission webhook error:

```bash $ kubectl describe pod myapp-abc123

Warning FailedCreate kubelet Error creating: Internal error occurred: failed calling webhook "linkerd-proxy-injector.linkerd.io": Post https://linkerd-proxy-injector.linkerd.svc:443?timeout=30s: connection refused ```

Why This Happens

  1. 1.Missing annotation - Namespace not annotated for injection
  2. 2.Webhook unavailable - Proxy injector service down
  3. 3.Injection disabled - Pod annotation disables injection
  4. 4.Label selector mismatch - Resource doesn't match injection selector
  5. 5.Certificate issues - Webhook TLS certificate expired
  6. 6.Namespace excluded - Linkerd excludes certain namespaces

Step 1: Check Namespace Annotation

```bash # Check namespace annotation: kubectl get namespace myapp -o yaml | grep -A 2 annotations

# Required annotation for injection: annotations: linkerd.io/inject: enabled

# Add annotation if missing: kubectl annotate namespace myapp linkerd.io/inject=enabled

# Check all namespaces: kubectl get namespaces -o json | jq '.items[] | {name: .metadata.name, inject: .metadata.annotations["linkerd.io/inject"]}'

# Verify annotation applied: kubectl describe namespace myapp | grep -i inject ```

Step 2: Check Pod Injection Status

```bash # Check pod annotations: kubectl get pod myapp-abc123 -o yaml | grep -A 5 annotations

# Injection-related annotations: # linkerd.io/inject: enabled (or disabled) # linkerd.io/proxy-version: stable-2.14.0

# Check if proxy container exists: kubectl get pod myapp-abc123 -o jsonpath='{.spec.containers[*].name}'

# Should show: myapp linkerd-proxy

# Check proxy status annotation: kubectl get pod myapp-abc123 -o jsonpath='{.metadata.annotations.linkerd\.io/proxy-version}'

# Check proxy container details: kubectl get pod myapp-abc123 -o jsonpath='{.spec.containers[?(@.name=="linkerd-proxy")]}' ```

Step 3: Check Proxy Injector Service

```bash # Check Linkerd control plane: linkerd check

# Check proxy injector: kubectl get pods -n linkerd -l linkerd.io/control-plane-component=proxy-injector

# Check service: kubectl get svc -n linkerd linkerd-proxy-injector

# Check endpoints: kubectl get endpoints -n linkerd linkerd-proxy-injector

# Check logs: kubectl logs -n linkerd -l linkerd.io/control-plane-component=proxy-injector

# Restart proxy injector: kubectl rollout restart deployment -n linkerd linkerd-proxy-injector ```

Step 4: Check Admission Webhook

```bash # Check webhook configuration: kubectl get mutatingwebhookconfigurations linkerd-proxy-injector-webhook-config -o yaml

# Check webhook service reference: # service: # name: linkerd-proxy-injector # namespace: linkerd # path: /

# Check webhook rules: # rules: # - apiGroups: [""] # apiVersions: ["v1"] # operations: ["CREATE"] # resources: ["pods"]

# Check webhook failure policy: # failurePolicy: Ignore (or Fail)

# Check CA bundle: kubectl get mutatingwebhookconfigurations linkerd-proxy-injector-webhook-config -o jsonpath='{.webhooks[0].clientConfig.caBundle}' | base64 -d | openssl x509 -text -noout

# Test webhook: kubectl run test --image=nginx --dry-run=server ```

Step 5: Check Certificate

```bash # Check webhook certificate: kubectl get secret -n linkerd linkerd-proxy-injector-k8s-tls -o yaml

# Decode certificate: kubectl get secret -n linkerd linkerd-proxy-injector-k8s-tls -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -text -noout

# Check expiry: kubectl get secret -n linkerd linkerd-proxy-injector-k8s-tls -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -noout -dates

# Regenerate certificates: linkerd install --ignore-cluster | kubectl apply -f -

# Or use cert-manager: linkerd install --set identityTrustAnchorsPEM="$(cat ca.crt)" | kubectl apply -f - ```

Step 6: Check Injection Config

```bash # Check proxy injector config: kubectl get configmap -n linkerd linkerd-config -o yaml

# Check proxy configuration: kubectl get configmap -n linkerd linkerd-config -o jsonpath='{.data.proxy}'

# Check injector configuration: kubectl get configmap -n linkerd linkerd-config -o jsonpath='{.data.values}' | jq .proxyInjector

# Check namespace selector in config: # namespaceSelector: # matchExpressions: # - key: linkerd.io/inject # operator: NotIn # values: # - disabled

# Update config if needed: kubectl edit configmap -n linkerd linkerd-config ```

Step 7: Check Pod Annotations Override

```bash # Pod can override namespace annotation:

# Disable injection for specific pod: annotations: linkerd.io/inject: disabled

# Check pod annotation: kubectl get pod myapp-abc123 -o jsonpath='{.metadata.annotations.linkerd\.io/inject}'

# If disabled, remove or change annotation: kubectl annotate pod myapp-abc123 linkerd.io/inject=enabled --overwrite

# Or delete and recreate pod: kubectl delete pod myapp-abc123 # Deployment will recreate with injection

# Check deployment template: kubectl get deployment myapp -o yaml | grep -A 10 spec.template.metadata.annotations ```

Step 8: Check HostNetwork

```bash # Pods with hostNetwork: true are not injected by default

# Check if pod uses hostNetwork: kubectl get pod myapp-abc123 -o jsonpath='{.spec.hostNetwork}'

# If true, injection is skipped

# To inject hostNetwork pods, add annotation: annotations: linkerd.io/inject: enabled

# And configure proxy: annotations: config.linkerd.io/skip-inbound-ports: "22,80,443" ```

Step 9: Check Linkerd Control Plane Health

```bash # Run Linkerd diagnostics: linkerd check

# Check specific components: linkerd check --proxy

# Check control plane: kubectl get all -n linkerd

# Check identity service: kubectl logs -n linkerd -l linkerd.io/control-plane-component=identity

# Check destination service: kubectl logs -n linkerd -l linkerd.io/control-plane-component=destination

# Check public API: kubectl logs -n linkerd -l linkerd.io/control-plane-component=public-api

# Full diagnostics: linkerd diagnostics control-plane ```

Step 10: Force Re-injection

```bash # Option 1: Delete pods to trigger reinjection: kubectl rollout restart deployment/myapp

# Option 2: Use linkerd inject command: kubectl get deployment myapp -o yaml | linkerd inject - | kubectl apply -f -

# Option 3: Manual injection for debugging: kubectl get deployment myapp -o yaml > deployment.yaml linkerd inject deployment.yaml > injected-deployment.yaml kubectl apply -f injected-deployment.yaml

# Verify injection: kubectl get pods -l app=myapp # Should show 2/2 containers

# Check proxy logs: kubectl logs myapp-abc123 -c linkerd-proxy

# Test connectivity: kubectl exec myapp-abc123 -c myapp -- curl http://localhost:4191/ready ```

Linkerd Injection Checklist

CheckCommandExpected
Namespace annotationkubectl get nsinject=enabled
Proxy injector podkubectl get podsRunning
Webhook configget mutatingwebhookExists
Certificateopenssl x509Valid
Pod annotationkubectl get podinject=enabled
Proxy containerget pod -o jsonlinkerd-proxy

Verify the Fix

```bash # After fixing injection

# 1. Restart deployment kubectl rollout restart deployment/myapp

# 2. Check new pods kubectl get pods -l app=myapp // 2/2 Running

# 3. Verify proxy kubectl get pod myapp-abc123 -o jsonpath='{.spec.containers[*].name}' // myapp linkerd-proxy

# 4. Check proxy version kubectl get pod myapp-abc123 -o jsonpath='{.metadata.annotations.linkerd\.io/proxy-version}' // stable-2.x.x

# 5. Test connectivity linkerd check --proxy // All checks pass

# 6. Verify traffic linkerd stat -n myapp deploy // Requests flowing ```

  • [Fix Istio Proxy Sidecar Not Ready](/articles/fix-istio-proxy-sidecar-not-ready)
  • [Fix Consul Connect Sidecar Not Proxying](/articles/fix-consul-connect-sidecar-not-proxying)
  • [Fix Envoy Proxy Upstream Connection Refused](/articles/fix-envoy-proxy-upstream-connection-refused)