The Problem
Your application loses Redis connections with timeout errors. Connections that should persist are dropped. Error messages include:
Redis::ConnectionError: Connection timed out
Connection refused
Read error on connectionApplications may need to reconnect frequently, causing latency spikes and failed requests.
Understanding Redis Timeouts
Redis has several timeout mechanisms:
- 1.Client idle timeout - Disconnects idle clients after
timeoutseconds - 2.TCP keepalive - Detects dead peers
- 3.Connection pool exhaustion - Application-level timeouts
- 4.Network timeouts - OS-level socket timeouts
Diagnosis Commands
Check Redis Timeout Configuration
redis-cli CONFIG GET timeoutOutput:
timeout 300Value is in seconds. 0 means never timeout.
Check TCP Keepalive
redis-cli CONFIG GET tcp-keepaliveOutput:
tcp-keepalive 300Value is in seconds. 0 means disabled.
List Connected Clients
redis-cli CLIENT LISTKey fields:
id=12345 addr=10.0.0.5:54321 fd=10 name=app-client age=3600 idle=30 flags=N db=0age- Connection duration in secondsidle- Seconds since last commandname- Client name (if set)
Find Idle Clients
redis-cli CLIENT LIST | awk -F' ' '{for(i=1;i<=NF;i++) if($i ~ /^idle=/) print $i}' | sort -t= -k2 -n | tail -20Check Client Counts
redis-cli INFO clientsKey metrics:
connected_clients:150
client_recent_max_input_buffer:2
client_recent_max_output_buffer:0
blocked_clients:0Common Timeout Scenarios
Scenario 1: Idle Clients Disconnected
Clients are disconnected after being idle for timeout seconds.
Symptoms: - Long-running applications lose connections - Errors after periods of inactivity
Diagnosis:
redis-cli CONFIG GET timeoutIf timeout is non-zero, idle clients are disconnected.
Solution:
# Disable idle timeout (for connection-pooled environments)
redis-cli CONFIG SET timeout 0Or ensure applications use connection pooling with reconnect logic.
Scenario 2: TCP Keepalive Disconnects
TCP keepalive sends probes to detect dead connections. If the network drops packets, connections get terminated.
Symptoms: - Connections dropped across network partitions - Intermittent disconnections
Diagnosis:
redis-cli CONFIG GET tcp-keepaliveSolution:
# Increase or disable TCP keepalive
redis-cli CONFIG SET tcp-keepalive 0
# Or increase to 5 minutes
redis-cli CONFIG SET tcp-keepalive 300Scenario 3: Client Connection Pool Issues
Application connection pools exhaust or don't validate connections.
Symptoms: - "Connection pool exhausted" errors - Borrowed connections are dead
Diagnosis:
Check from application side. The issue is usually application configuration, not Redis.
Solution:
Enable connection validation in your client:
```python # Python redis-py example import redis from redis.connection import ConnectionPool
pool = ConnectionPool( host='localhost', port=6379, max_connections=100, socket_timeout=5, socket_connect_timeout=5, retry_on_timeout=True )
r = redis.Redis(connection_pool=pool) ```
Scenario 4: Client Name Not Set
Cannot identify which applications have timeout issues.
Solution:
# Set client name for identification
redis-cli CLIENT SETNAME "my-app-server-1"In application code:
r.client_setname('my-app-worker-1')Scenario 5: Max Clients Reached
Redis rejects new connections when maxclients is reached.
Symptoms:
ERR max number of clients reachedDiagnosis:
redis-cli CONFIG GET maxclients
redis-cli INFO clients | grep connected_clientsSolution:
# Increase max clients
redis-cli CONFIG SET maxclients 10000Also check system file descriptor limits:
```bash # Check Redis process limits cat /proc/$(pgrep redis-server)/limits | grep "open files"
# Increase system-wide sudo sysctl -w fs.file-max=100000 ```
Configuration Best Practices
Redis Configuration
```conf # redis.conf
# Disable idle timeout for persistent connections timeout 0
# Enable TCP keepalive to detect dead peers tcp-keepalive 300
# Allow enough clients maxclients 10000
# Listen on proper interface bind 0.0.0.0 protected-mode yes ```
Application Configuration Patterns
#### Connection Pooling with Validation
```python import redis from redis.connection import ConnectionPool
pool = ConnectionPool( host='redis.example.com', port=6379, db=0, max_connections=50, socket_timeout=5, socket_connect_timeout=5, retry_on_timeout=True, health_check_interval=30 # Validate connections )
client = redis.Redis(connection_pool=pool) ```
#### Reconnection Logic
```python import redis import time
def get_redis_connection(max_retries=3, retry_delay=1): for attempt in range(max_retries): try: r = redis.Redis( host='localhost', port=6379, socket_timeout=5, socket_connect_timeout=5 ) r.ping() return r except redis.ConnectionError as e: if attempt < max_retries - 1: time.sleep(retry_delay * (attempt + 1)) else: raise ```
#### Keepalive Pattern for Long-Running Connections
```python import redis import threading import time
class RedisKeepalive: def __init__(self, redis_client, interval=30): self.redis = redis_client self.interval = interval self._stop = threading.Event() self._thread = threading.Thread(target=self._keepalive, daemon=True)
def start(self): self._thread.start()
def _keepalive(self): while not self._stop.is_set(): try: self.redis.ping() except redis.ConnectionError: # Connection lost, will reconnect on next use pass time.sleep(self.interval)
def stop(self): self._stop.set() self._thread.join() ```
Monitoring Client Connections
Track Client Counts Over Time
```bash #!/bin/bash # Monitor client connections
while true; do COUNT=$(redis-cli INFO clients | grep connected_clients | cut -d: -f2 | tr -d '\r') MAX=$(redis-cli CONFIG GET maxclients | tail -1) IDLE=$(redis-cli CLIENT LIST | grep -c 'idle=[0-9][0-9][0-9]')
echo "$(date): Clients=$COUNT Max=$Max Idle>100s=$IDLE" sleep 60 done ```
Find Long-Idle Connections
redis-cli CLIENT LIST | awk -F'[ =]' '{
for(i=1;i<=NF;i++) {
if($i=="idle") idle=$(i+1)
if($i=="addr") addr=$(i+1)
if($i=="name") name=$(i+1)
}
if(idle>300) print "Long idle:", addr, name, "idle:", idle, "seconds"
}'Kill Idle Connections
```bash # Kill connections idle for more than 5 minutes redis-cli CLIENT LIST | while read line; do IDLE=$(echo "$line" | grep -oP 'idle=\K[0-9]+') ID=$(echo "$line" | grep -oP 'id=\K[0-9]+')
if [ -n "$IDLE" ] && [ "$IDLE" -gt 300 ]; then echo "Killing client $ID (idle for ${IDLE}s)" redis-cli CLIENT KILL ID $ID fi done ```
Verification
After making configuration changes:
```bash # Verify timeout is disabled redis-cli CONFIG GET timeout
# Verify keepalive redis-cli CONFIG GET tcp-keepalive
# Test connection persistence redis-cli CLIENT SETNAME test-connection sleep 400 redis-cli CLIENT LIST | grep test-connection # Should still be connected ```
Make Changes Persistent
redis-cli CONFIG REWRITEThis writes the current configuration to redis.conf.