Introduction Mutual TLS (mTLS) requires both server and client to present valid certificates. When mTLS fails, service-to-service communication breaks completely, with opaque error messages.

Symptoms - curl returns: "SSL certificate problem: unable to get local issuer certificate" - Envoy logs: "TLS error: 268435581:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED" - Client: "remote error: tls: bad certificate" - Service mesh mTLS status showing PERMISSIVE instead of STRICT - Certificate chain incomplete

Common Causes - Client certificate expired - CA certificate not in trust store - Certificate signed by wrong CA - Certificate SAN does not match service name - Certificate chain order incorrect

Step-by-Step Fix 1. **Verify client certificate': ```bash openssl x509 -in client.crt -text -noout | grep -E "Subject|Issuer|Not After" openssl verify -CAfile ca.crt client.crt ```

  1. 1.**Test mTLS connection':
  2. 2.```bash
  3. 3.curl -v --cert client.crt --key client.key --cacert ca.crt https://service.example.com
  4. 4.`
  5. 5.**Check certificate SAN':
  6. 6.```bash
  7. 7.openssl x509 -in server.crt -text -noout | grep -A1 "Subject Alternative Name"
  8. 8.`

Prevention - Automate certificate rotation with cert-manager or SPIFFE/SPIRE - Monitor certificate expiration for all mTLS certificates - Test mTLS connectivity in CI/CD pipeline - Use service mesh for automated mTLS management - Document certificate chain requirements