What's Actually Happening
Linkerd proxy sidecar is not injected into pods. Pods start without the Linkerd proxy, preventing service mesh functionality.
The Error You'll See
No sidecar in pod:
```bash $ kubectl get pods
NAME READY STATUS RESTARTS myapp-abc123 1/1 Running 0 # Expected 2/2 with Linkerd sidecar ```
Injection annotation missing:
```bash $ kubectl describe pod myapp-abc123 | grep -i linkerd
# No Linkerd annotations found ```
Admission webhook error:
```bash $ kubectl describe pod myapp-abc123
Warning FailedCreate kubelet Error creating: Internal error occurred: failed calling webhook "linkerd-proxy-injector.linkerd.io": Post https://linkerd-proxy-injector.linkerd.svc:443?timeout=30s: connection refused ```
Why This Happens
- 1.Missing annotation - Namespace not annotated for injection
- 2.Webhook unavailable - Proxy injector service down
- 3.Injection disabled - Pod annotation disables injection
- 4.Label selector mismatch - Resource doesn't match injection selector
- 5.Certificate issues - Webhook TLS certificate expired
- 6.Namespace excluded - Linkerd excludes certain namespaces
Step 1: Check Namespace Annotation
```bash # Check namespace annotation: kubectl get namespace myapp -o yaml | grep -A 2 annotations
# Required annotation for injection: annotations: linkerd.io/inject: enabled
# Add annotation if missing: kubectl annotate namespace myapp linkerd.io/inject=enabled
# Check all namespaces: kubectl get namespaces -o json | jq '.items[] | {name: .metadata.name, inject: .metadata.annotations["linkerd.io/inject"]}'
# Verify annotation applied: kubectl describe namespace myapp | grep -i inject ```
Step 2: Check Pod Injection Status
```bash # Check pod annotations: kubectl get pod myapp-abc123 -o yaml | grep -A 5 annotations
# Injection-related annotations: # linkerd.io/inject: enabled (or disabled) # linkerd.io/proxy-version: stable-2.14.0
# Check if proxy container exists: kubectl get pod myapp-abc123 -o jsonpath='{.spec.containers[*].name}'
# Should show: myapp linkerd-proxy
# Check proxy status annotation: kubectl get pod myapp-abc123 -o jsonpath='{.metadata.annotations.linkerd\.io/proxy-version}'
# Check proxy container details: kubectl get pod myapp-abc123 -o jsonpath='{.spec.containers[?(@.name=="linkerd-proxy")]}' ```
Step 3: Check Proxy Injector Service
```bash # Check Linkerd control plane: linkerd check
# Check proxy injector: kubectl get pods -n linkerd -l linkerd.io/control-plane-component=proxy-injector
# Check service: kubectl get svc -n linkerd linkerd-proxy-injector
# Check endpoints: kubectl get endpoints -n linkerd linkerd-proxy-injector
# Check logs: kubectl logs -n linkerd -l linkerd.io/control-plane-component=proxy-injector
# Restart proxy injector: kubectl rollout restart deployment -n linkerd linkerd-proxy-injector ```
Step 4: Check Admission Webhook
```bash # Check webhook configuration: kubectl get mutatingwebhookconfigurations linkerd-proxy-injector-webhook-config -o yaml
# Check webhook service reference: # service: # name: linkerd-proxy-injector # namespace: linkerd # path: /
# Check webhook rules: # rules: # - apiGroups: [""] # apiVersions: ["v1"] # operations: ["CREATE"] # resources: ["pods"]
# Check webhook failure policy: # failurePolicy: Ignore (or Fail)
# Check CA bundle: kubectl get mutatingwebhookconfigurations linkerd-proxy-injector-webhook-config -o jsonpath='{.webhooks[0].clientConfig.caBundle}' | base64 -d | openssl x509 -text -noout
# Test webhook: kubectl run test --image=nginx --dry-run=server ```
Step 5: Check Certificate
```bash # Check webhook certificate: kubectl get secret -n linkerd linkerd-proxy-injector-k8s-tls -o yaml
# Decode certificate: kubectl get secret -n linkerd linkerd-proxy-injector-k8s-tls -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -text -noout
# Check expiry: kubectl get secret -n linkerd linkerd-proxy-injector-k8s-tls -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -noout -dates
# Regenerate certificates: linkerd install --ignore-cluster | kubectl apply -f -
# Or use cert-manager: linkerd install --set identityTrustAnchorsPEM="$(cat ca.crt)" | kubectl apply -f - ```
Step 6: Check Injection Config
```bash # Check proxy injector config: kubectl get configmap -n linkerd linkerd-config -o yaml
# Check proxy configuration: kubectl get configmap -n linkerd linkerd-config -o jsonpath='{.data.proxy}'
# Check injector configuration: kubectl get configmap -n linkerd linkerd-config -o jsonpath='{.data.values}' | jq .proxyInjector
# Check namespace selector in config: # namespaceSelector: # matchExpressions: # - key: linkerd.io/inject # operator: NotIn # values: # - disabled
# Update config if needed: kubectl edit configmap -n linkerd linkerd-config ```
Step 7: Check Pod Annotations Override
```bash # Pod can override namespace annotation:
# Disable injection for specific pod: annotations: linkerd.io/inject: disabled
# Check pod annotation: kubectl get pod myapp-abc123 -o jsonpath='{.metadata.annotations.linkerd\.io/inject}'
# If disabled, remove or change annotation: kubectl annotate pod myapp-abc123 linkerd.io/inject=enabled --overwrite
# Or delete and recreate pod: kubectl delete pod myapp-abc123 # Deployment will recreate with injection
# Check deployment template: kubectl get deployment myapp -o yaml | grep -A 10 spec.template.metadata.annotations ```
Step 8: Check HostNetwork
```bash # Pods with hostNetwork: true are not injected by default
# Check if pod uses hostNetwork: kubectl get pod myapp-abc123 -o jsonpath='{.spec.hostNetwork}'
# If true, injection is skipped
# To inject hostNetwork pods, add annotation: annotations: linkerd.io/inject: enabled
# And configure proxy: annotations: config.linkerd.io/skip-inbound-ports: "22,80,443" ```
Step 9: Check Linkerd Control Plane Health
```bash # Run Linkerd diagnostics: linkerd check
# Check specific components: linkerd check --proxy
# Check control plane: kubectl get all -n linkerd
# Check identity service: kubectl logs -n linkerd -l linkerd.io/control-plane-component=identity
# Check destination service: kubectl logs -n linkerd -l linkerd.io/control-plane-component=destination
# Check public API: kubectl logs -n linkerd -l linkerd.io/control-plane-component=public-api
# Full diagnostics: linkerd diagnostics control-plane ```
Step 10: Force Re-injection
```bash # Option 1: Delete pods to trigger reinjection: kubectl rollout restart deployment/myapp
# Option 2: Use linkerd inject command: kubectl get deployment myapp -o yaml | linkerd inject - | kubectl apply -f -
# Option 3: Manual injection for debugging: kubectl get deployment myapp -o yaml > deployment.yaml linkerd inject deployment.yaml > injected-deployment.yaml kubectl apply -f injected-deployment.yaml
# Verify injection: kubectl get pods -l app=myapp # Should show 2/2 containers
# Check proxy logs: kubectl logs myapp-abc123 -c linkerd-proxy
# Test connectivity: kubectl exec myapp-abc123 -c myapp -- curl http://localhost:4191/ready ```
Linkerd Injection Checklist
| Check | Command | Expected |
|---|---|---|
| Namespace annotation | kubectl get ns | inject=enabled |
| Proxy injector pod | kubectl get pods | Running |
| Webhook config | get mutatingwebhook | Exists |
| Certificate | openssl x509 | Valid |
| Pod annotation | kubectl get pod | inject=enabled |
| Proxy container | get pod -o json | linkerd-proxy |
Verify the Fix
```bash # After fixing injection
# 1. Restart deployment kubectl rollout restart deployment/myapp
# 2. Check new pods kubectl get pods -l app=myapp // 2/2 Running
# 3. Verify proxy kubectl get pod myapp-abc123 -o jsonpath='{.spec.containers[*].name}' // myapp linkerd-proxy
# 4. Check proxy version kubectl get pod myapp-abc123 -o jsonpath='{.metadata.annotations.linkerd\.io/proxy-version}' // stable-2.x.x
# 5. Test connectivity linkerd check --proxy // All checks pass
# 6. Verify traffic linkerd stat -n myapp deploy // Requests flowing ```
Related Issues
- [Fix Istio Proxy Sidecar Not Ready](/articles/fix-istio-proxy-sidecar-not-ready)
- [Fix Consul Connect Sidecar Not Proxying](/articles/fix-consul-connect-sidecar-not-proxying)
- [Fix Envoy Proxy Upstream Connection Refused](/articles/fix-envoy-proxy-upstream-connection-refused)