Fix Load Balancer SSL Handshake Failed

What's Actually Happening

Load balancer SSL/TLS handshake fails when clients or backends attempt to establish secure connections. Connections are rejected or terminated.

The Error You'll See

```bash $ curl https://example.com

curl: (35) SSL connect error ```

Load balancer logs:

SSL handshake failed: protocol version not supported
SSL handshake failed: certificate verify failed
SSL handshake failed: no shared cipher

Backend connection error:

```bash $ openssl s_client -connect backend:443

140736109142928:error:1408F10B:SSL routines:ssl3_get_record:wrong version number ```

Why This Happens

1. 1.Certificate not installed - SSL certificate missing or expired
2. 2.Protocol mismatch - Client/server protocol versions incompatible
3. 3.Cipher mismatch - No shared cipher suites
4. 4.SNI not configured - Server Name Indication missing
5. 5.Backend SSL issues - Backend SSL certificate invalid
6. 6.Certificate chain incomplete - Missing intermediate certificates

Step 1: Check SSL Certificate Status

```bash # Check certificate on load balancer: openssl s_client -connect lb-server:443 -servername example.com

# Check certificate details: openssl s_client -connect lb-server:443 -servername example.com 2>/dev/null | openssl x509 -noout -text

# Check certificate dates: openssl s_client -connect lb-server:443 -servername example.com 2>/dev/null | openssl x509 -noout -dates

# Check certificate chain: openssl s_client -connect lb-server:443 -servername example.com -showcerts

# Check local certificate file: openssl x509 -in /etc/ssl/certs/example.com.crt -noout -text

# Check certificate key: openssl rsa -in /etc/ssl/private/example.com.key -check

# Verify certificate matches key: openssl x509 -noout -modulus -in example.com.crt | openssl md5 openssl rsa -noout -modulus -in example.com.key | openssl md5 # Must match! ```

Step 2: Check Protocol Support

```bash # Check supported protocols: nmap --script ssl-enum-ciphers -p 443 lb-server

# Test specific protocol versions: openssl s_client -connect lb-server:443 -tls1_2 openssl s_client -connect lb-server:443 -tls1_3 openssl s_client -connect lb-server:443 -tls1 # TLS 1.0 (deprecated) openssl s_client -connect lb-server:443 -ssl3 # SSL 3.0 (deprecated)

# Check load balancer protocol config: # Nginx: grep -E "ssl_protocols|ssl_ciphers" /etc/nginx/nginx.conf

# HAProxy: grep -E "bind.*ssl" /etc/haproxy/haproxy.cfg

# AWS ALB: aws elbv2 describe-load-balancers --load-balancer-arns arn

# For Nginx, set protocols: ssl_protocols TLSv1.2 TLSv1.3;

# For HAProxy: bind :443 ssl crt /etc/ssl/certs/example.com.pem alpn h2,http/1.1 ```

Step 3: Check Cipher Suite Configuration

```bash # Check cipher suites: nmap --script ssl-enum-ciphers -p 443 lb-server

# Or openssl: openssl s_client -connect lb-server:443 -cipher 'HIGH:!aNULL'

# List available ciphers: openssl ciphers -v 'HIGH:!aNULL'

# Check cipher config in load balancer: # Nginx: grep ssl_ciphers /etc/nginx/nginx.conf ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256;

# HAProxy: bind :443 ssl crt /cert.pem ciphers ECDHE-RSA-AES128-GCM-SHA256

# Test specific cipher: openssl s_client -connect lb-server:443 -cipher ECDHE-RSA-AES128-GCM-SHA256

# If cipher test fails, update cipher config: ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384; ```

Step 4: Check SNI Configuration

```bash # SNI required for multiple certificates on same IP

# Test with SNI: openssl s_client -connect lb-server:443 -servername example.com

# Test without SNI: openssl s_client -connect lb-server:443

# Compare certificates returned: openssl s_client -connect lb-server:443 -servername example.com 2>/dev/null | openssl x509 -noout -subject openssl s_client -connect lb-server:443 2>/dev/null | openssl x509 -noout -subject

# Check SNI config: # Nginx - each server block handles SNI: server { listen 443 ssl; server_name example.com; ssl_certificate /etc/ssl/certs/example.com.crt; }

# HAProxy - use crt-list for SNI: bind :443 ssl crt-list /etc/haproxy/crt-list.txt

# crt-list.txt: /etc/ssl/certs/example.com.pem example.com /etc/ssl/certs/other.com.pem other.com

# AWS ALB - SNI automatic with multiple certificates: aws elbv2 add-listener-certificates ```

Step 5: Check Backend SSL Configuration

```bash # Check backend SSL: openssl s_client -connect backend-server:443

# Check backend certificate: openssl s_client -connect backend-server:443 2>/dev/null | openssl x509 -noout -dates

# Test backend handshake: curl -k https://backend-server/

# Check load balancer backend config: # Nginx upstream SSL: upstream backend { server backend-server:443; }

proxy_ssl_verify on; proxy_ssl_trusted_certificate /etc/ssl/certs/ca.pem;

# HAProxy backend SSL: backend mybackend server s1 backend-server:443 ssl verify required ca-file /etc/ssl/certs/ca.pem

# If backend SSL fails: # Check backend certificate: openssl s_client -connect backend-server:443 -showcerts

# Skip verification if backend uses self-signed: proxy_ssl_verify off; # Or HAProxy: server s1 backend-server:443 ssl verify none ```

Step 6: Check Certificate Chain

```bash # Check if chain complete: openssl s_client -connect lb-server:443 -servername example.com -showcerts

# Output should show: # Certificate chain # 0 s:CN=example.com # i:CN=Intermediate CA # 1 s:CN=Intermediate CA # i:CN=Root CA

# If chain incomplete: # Combine certificate files: cat example.com.crt intermediate.crt root.crt > fullchain.pem

# Or download missing intermediate: # From certificate issuer's website

# For Let's Encrypt: cat fullchain.pem privkey.pem > combined.pem

# Nginx: ssl_certificate /etc/ssl/certs/fullchain.pem; ssl_certificate_key /etc/ssl/certs/privkey.pem;

# HAProxy requires combined format: cat fullchain.pem privkey.pem > haproxy.pem # HAProxy pem format: cert + intermediate + key

# Verify chain: openssl verify -CAfile /etc/ssl/certs/ca.pem /etc/ssl/certs/example.com.crt ```

Step 7: Check Certificate Renewal

```bash # Check certificate expiry: openssl s_client -connect lb-server:443 -servername example.com 2>/dev/null | openssl x509 -noout -dates

# Output: # notBefore=Jan 1 00:00:00 2024 GMT # notAfter=Jan 1 00:00:00 2025 GMT

# If expired, renew: # Let's Encrypt certbot: certbot renew

# Force renewal: certbot renew --force-renewal

# Check certbot certificates: certbot certificates

# For custom certificates: # Generate new CSR: openssl req -new -newkey rsa:2048 -nodes -keyout example.com.key -out example.com.csr

# Submit CSR to CA # Install new certificate: cp new_certificate.crt /etc/ssl/certs/example.com.crt cp new_key.key /etc/ssl/private/example.com.key

# Restart load balancer: systemctl restart nginx systemctl restart haproxy ```

Step 8: Check SSL Session Cache

```bash # SSL session cache improves performance

# Nginx session cache: ssl_session_cache shared:SSL:10m; ssl_session_timeout 10m; ssl_session_tickets on;

# Check if session cache configured: grep ssl_session /etc/nginx/nginx.conf

# HAProxy session cache: # Automatic by default

# Test session resumption: openssl s_client -connect lb-server:443 -servername example.com -reconnect

# Output should show: # Reusing session

# If session cache disabled, handshake overhead increases: ssl_session_cache off; # Each connection full handshake

# Enable session cache: ssl_session_cache shared:SSL:50m; ```

Step 9: Debug SSL Handshake

```bash # Enable SSL debug logging: # Nginx: error_log /var/log/nginx/error.log debug;

# Check logs: tail -f /var/log/nginx/error.log | grep -i ssl

# HAProxy debug: # Add to global: debug

# Use strace: strace -e trace=network,openat openssl s_client -connect lb-server:443

# Use tcpdump: tcpdump -i any port 443 -w ssl.pcap

# Analyze with Wireshark: wireshark ssl.pcap

# Check SSL error details: openssl s_client -connect lb-server:443 -servername example.com -debug

# Check specific error: openssl s_client -connect lb-server:443 -servername example.com -msg ```

Step 10: SSL Handshake Verification Script

```bash # Create verification script: cat << 'EOF' > /usr/local/bin/check-ssl-handshake.sh #!/bin/bash

SERVER=$1 PORT=${2:-443}

echo "=== SSL Certificate ===" openssl s_client -connect $SERVER:$PORT -servername $SERVER 2>/dev/null | openssl x509 -noout -subject -dates

echo "" echo "=== Protocol Support ===" for proto in tls1_2 tls1_3; do result=$(openssl s_client -connect $SERVER:$PORT -$proto 2>&1 | grep -E "Protocol|error") echo "$proto: $result" done

echo "" echo "=== Cipher Suites ===" nmap --script ssl-enum-ciphers -p $PORT $SERVER 2>/dev/null | grep -A 50 "TLS"

echo "" echo "=== Certificate Chain ===" openssl s_client -connect $SERVER:$PORT -servername $SERVER -showcerts 2>/dev/null | grep -E "s:|i:"

echo "" echo "=== SNI Test ===" cert_with=$(openssl s_client -connect $SERVER:$PORT -servername $SERVER 2>/dev/null | openssl x509 -noout -subject) cert_without=$(openssl s_client -connect $SERVER:$PORT 2>/dev/null | openssl x509 -noout -subject) echo "With SNI: $cert_with" echo "Without SNI: $cert_without"

echo "" echo "=== Connection Test ===" curl -v https://$SERVER 2>&1 | grep -E "SSL|TLS|Connected" EOF

chmod +x /usr/local/bin/check-ssl-handshake.sh

# Usage: /usr/local/bin/check-ssl-handshake.sh example.com ```

SSL Handshake Checklist

Verify the Fix

```bash # After fixing SSL handshake

# 1. Test connection curl https://example.com // SSL handshake successful

# 2. Check certificate openssl s_client -connect example.com:443 -servername example.com // Certificate verified

# 3. Check protocol openssl s_client -connect example.com:443 -tls1_2 // Connected with TLS 1.2

# 4. Test from client openssl s_client -connect example.com:443 // No errors

# 5. Check load balancer logs tail -f /var/log/nginx/error.log // No SSL errors

# 6. Monitor SSL metrics watch -n 5 'openssl s_client -connect example.com:443 2>&1 | grep -E "Verify|Protocol"' ```

Related Issues

• [Fix SSL Certificate Expired](/articles/fix-ssl-certificate-expired)
• [Fix HTTPS Certificate Not Obtained](/articles/fix-caddy-https-certificate-not-obtained)
• [Fix Nginx SSL Configuration Error](/articles/fix-nginx-ssl-configuration-error)