What's Actually Happening
Traefik reverse proxy fails to route requests to backend services. Requests receive 404, 502, or timeout errors instead of reaching target services.
The Error You'll See
```bash $ curl http://traefik-server/app
404 page not found ```
Traefik logs:
time="2024-01-01T10:00:00Z" level=error msg="service \"app\" not found"
time="2024-01-01T10:00:00Z" level=error msg="no routers matching request"Backend unreachable:
```bash $ curl http://traefik-server/app
502 Bad Gateway ```
Dashboard empty:
Traefik dashboard shows:
- Routers: 0
- Services: 0
- Middlewares: 0Why This Happens
- 1.Router not configured - Missing routing rules
- 2.Service not discovered - Backend not registered
- 3.Docker provider issues - Container labels missing
- 4.Kubernetes Ingress issues - Ingress resource wrong
- 5.Health check failing - Backend marked unhealthy
- 6.TLS certificate issues - HTTPS routing blocked
Step 1: Check Traefik Status
```bash # Check Traefik process: docker ps | grep traefik
# Or systemd: systemctl status traefik
# Check Traefik logs: docker logs traefik
# Or: journalctl -u traefik -f
# Check Traefik version: docker exec traefik traefik version
# Check Traefik dashboard: curl http://localhost:8080/api/overview
# Check routers: curl http://localhost:8080/api/http/routers
# Check services: curl http://localhost:8080/api/http/services
# Check providers: curl http://localhost:8080/api/providers ```
Step 2: Check Traefik Configuration
```bash # Check static configuration: docker exec traefik cat /etc/traefik/traefik.yml
# Or: cat /etc/traefik/traefik.yml
# Check providers config: # Docker provider: providers: docker: endpoint: "unix:///var/run/docker.sock" exposedByDefault: false
# Kubernetes provider: providers: kubernetesIngress: endpoint: "https://kubernetes.api"
# File provider: providers: file: filename: /etc/traefik/dynamic.yml
# Check dynamic configuration: cat /etc/traefik/dynamic.yml
# Check entrypoints: curl http://localhost:8080/api/entrypoints ```
Step 3: Check Router Configuration
```bash # Check routers via API: curl http://localhost:8080/api/http/routers | jq .
# Check specific router: curl http://localhost:8080/api/http/routers/app@docker | jq .
# Expected router config:
{
"service": "app",
"rule": "Host(app.example.com)",
"entryPoints": ["web"],
"status": "enabled"
}
# Docker labels for router: docker inspect app-container | grep -A10 Labels
# Required labels:
traefik.enable=true
traefik.http.routers.app.rule=Host(app.example.com)
traefik.http.routers.app.entrypoints=web
# Check rule syntax:
# Common rules:
# Host(example.com)
# PathPrefix(/app)
# Host(example.com) && PathPrefix(/api)
# Check rule matching: curl -H "Host: app.example.com" http://traefik-server/ ```
Step 4: Check Service Discovery
```bash # Check services via API: curl http://localhost:8080/api/http/services | jq .
# Check specific service: curl http://localhost:8080/api/http/services/app@docker | jq .
# Docker labels for service: traefik.http.services.app.loadbalancer.server.port=8080
# Check backend servers: curl http://localhost:8080/api/http/services/app@docker | jq .loadBalancer.servers
# Should show backend IPs/ports
# Kubernetes service discovery: kubectl get svc -n traefik
# Check service endpoints: kubectl get endpoints app-service -n myapp
# Check Ingress: kubectl get ingress -n myapp
# Check Ingress annotations: kubectl get ingress app-ingress -n myapp -o yaml | grep annotations ```
Step 5: Check Docker Provider
```bash # Check Docker provider status: curl http://localhost:8080/api/providers/docker
# Check Docker connectivity: docker exec traefik ls -la /var/run/docker.sock
# Test Docker API from Traefik: docker exec traefik curl http://localhost:2375/containers/json
# Check Docker network: docker network ls
# Ensure containers on same network: docker network inspect traefik-net
# Check container labels: docker inspect app | jq .[0].Config.Labels
# Required labels:
{
"traefik.enable": "true",
"traefik.http.routers.app.rule": "Host(app.example.com)",
"traefik.http.services.app.loadbalancer.server.port": "8080"
}
# Add missing labels: docker update --label-add traefik.enable=true app ```
Step 6: Check Kubernetes Ingress
```bash # Check Ingress resource: kubectl get ingress -A
# Check Ingress details: kubectl describe ingress app-ingress -n myapp
# Check Ingress class: kubectl get ingressclass
# Traefik IngressClass: cat << 'EOF' | kubectl apply -f - apiVersion: networking.k8s.io/v1 kind: IngressClass metadata: name: traefik spec: controller: traefik.io/ingress-controller EOF
# Check Ingress annotations: annotations: kubernetes.io/ingress.class: traefik traefik.ingress.kubernetes.io/router.entrypoints: web
# Check service backend: kubectl get svc app-service -n myapp
# Check endpoints: kubectl get endpoints app-service -n myapp
# Test service: kubectl exec -it test-pod -- curl http://app-service:8080 ```
Step 7: Check Health Checks
```bash # Traefik health check config: # Docker labels: traefik.http.services.app.loadbalancer.healthcheck.path=/health traefik.http.services.app.loadbalancer.healthcheck.interval=10s traefik.http.services.app.loadbalancer.healthcheck.timeout=5s
# Check service health: curl http://localhost:8080/api/http/services/app@docker | jq .loadBalancer.healthCheck
# Check backend health: curl http://localhost:8080/api/http/services/app@docker | jq .serverStatus
# Output: # {"http://10.0.0.1:8080": "UP", "http://10.0.0.2:8080": "DOWN"}
# Test backend health endpoint: curl http://backend:8080/health
# If health check failing: # Disable health check temporarily: traefik.http.services.app.loadbalancer.healthcheck.path=
# Or fix health endpoint on backend ```
Step 8: Check TLS Configuration
```bash # Check TLS routers: curl http://localhost:8080/api/http/routers | jq '.[] | select(.tls != null)'
# TLS router labels: traefik.http.routers.app.tls=true traefik.http.routers.app.tls.certresolver=myresolver
# Check certificate resolver: curl http://localhost:8080/api/tls/certificates
# Let's Encrypt config: certificatesResolvers: myresolver: acme: email: admin@example.com storage: acme.json httpChallenge: entryPoint: web
# Check ACME certificates: cat /etc/traefik/acme.json | jq .
# Test HTTPS: curl https://app.example.com
# Check TLS entrypoint: curl http://localhost:8080/api/entrypoints/websecure | jq . ```
Step 9: Check Middlewares
```bash # Check middlewares: curl http://localhost:8080/api/http/middlewares | jq .
# Check middleware attached to router: curl http://localhost:8080/api/http/routers/app@docker | jq .middlewares
# Middleware labels: traefik.http.routers.app.middlewares=auth,ratelimit
# Common middlewares: # auth - Basic auth # ratelimit - Rate limiting # stripPrefix - Path prefix removal # redirectScheme - HTTP to HTTPS redirect
# Check middleware config: traefik.http.middlewares.auth.basicauth.users=admin:$apr1$...
# Test middleware: curl -H "Host: app.example.com" -u admin:password http://traefik-server/
# Check middleware errors: docker logs traefik | grep middleware ```
Step 10: Traefik Verification Script
```bash # Create verification script: cat << 'EOF' > /usr/local/bin/check-traefik.sh #!/bin/bash
API="http://localhost:8080/api"
echo "=== Traefik Overview ===" curl -s $API/overview | jq .
echo "" echo "=== Entrypoints ===" curl -s $API/entrypoints | jq '.[] | {name, address}'
echo "" echo "=== HTTP Routers ===" curl -s $API/http/routers | jq '.[] | {name, rule, service, status}'
echo "" echo "=== HTTP Services ===" curl -s $API/http/services | jq '.[] | {name, loadBalancer.servers, serverStatus}'
echo "" echo "=== Providers ===" curl -s $API/providers | jq .
echo "" echo "=== Docker Containers ===" docker ps --filter label=traefik.enable=true --format "table {{.Names}}\t{{.Status}}"
echo "" echo "=== Test Routing ===" curl -s -H "Host: app.example.com" http://localhost/ | head -20
echo "" echo "=== Traefik Logs (last 20) ===" docker logs traefik --tail 20 2>&1 | grep -E "error|warn|router|service" EOF
chmod +x /usr/local/bin/check-traefik.sh
# Run: /usr/local/bin/check-traefik.sh
# Watch dashboard: watch -n 5 'curl -s http://localhost:8080/api/http/routers | jq length' ```
Traefik Routing Checklist
| Check | Command | Expected |
|---|---|---|
| Traefik running | docker ps | Container up |
| Routers configured | API routers | Routers listed |
| Services discovered | API services | Backends listed |
| Docker labels | docker inspect | Labels present |
| Rule matching | curl with Host | Routes correctly |
| Health status | API serverStatus | Backends UP |
Verify the Fix
```bash # After fixing routing
# 1. Check routers curl http://localhost:8080/api/http/routers // Routers listed with status enabled
# 2. Check services curl http://localhost:8080/api/http/services // Services with backends
# 3. Test routing curl -H "Host: app.example.com" http://traefik-server/ // Response from backend
# 4. Check dashboard # Open http://traefik:8080 // Shows routers and services
# 5. Check logs docker logs traefik --tail 50 // No errors
# 6. Test all routes curl http://app.example.com/api curl http://app.example.com/health // All working ```
Related Issues
- [Fix HAProxy Backend Health Check Failed](/articles/fix-haproxy-backend-health-check-failed)
- [Fix Nginx Upstream Timed Out](/articles/fix-nginx-upstream-timed-out)
- [Fix Load Balancer Backend Unhealthy](/articles/fix-load-balancer-backend-unhealthy)