The Problem

Redis fails to perform background saves. BGSAVE and BGREWRITEAOF commands return fork errors, and you see messages in logs like:

bash
Background saving error: Cannot fork: fork() failed: Cannot allocate memory

Or:

bash
BGREWRITEAOF: Can't fork: Cannot allocate memory (errno=12)

Checking persistence status shows failures:

bash
redis-cli INFO persistence
# rdb_last_bgsave_status:err
# aof_last_bgrewrite_status:err

The result: Redis cannot persist data. Writes continue but no snapshots are created, risking data loss on restart.

Why Forking Fails

Redis uses fork() to create child processes for persistence. The child inherits the parent's memory through copy-on-write. Fork failures occur when:

  1. 1.Insufficient memory - System cannot reserve memory for fork
  2. 2.Overcommit disabled - Linux refuses fork when it cannot guarantee memory
  3. 3.Memory fragmentation - Available memory exists but not contiguous
  4. 4.RSS exceeds limits - Redis memory usage too large to fork
  5. 5.THP enabled - Transparent Huge Pages slow down fork
  6. 6.Process limits - System max processes or memory limits reached

Understanding the Memory Problem

When Redis forks, the kernel must theoretically reserve the same amount of memory the parent uses. With copy-on-write, most pages are shared, but the kernel still needs to account for potential writes.

For example: - Redis uses 8GB of memory - Fork requires kernel to potentially reserve 8GB more - Total commitment: 16GB - If system has 12GB, fork fails

Even though actual memory usage stays near 8GB (copy-on-write shares pages), the kernel's accounting causes rejection.

Diagnosis Steps

Check Persistence Status

```bash # Check recent persistence results redis-cli INFO persistence

# Look for: # rdb_last_bgsave_status:err or ok # aof_last_bgrewrite_status:err or ok # rdb_last_bgsave_time_sec: -1 means failure ```

Check Redis Memory Usage

```bash # View memory statistics redis-cli INFO memory

# Key metrics: # used_memory: bytes allocated # used_memory_rss: bytes actually in RAM # mem_fragmentation_ratio: rss/used_memory ```

Check System Memory

```bash # Available memory free -h

# Memory committed cat /proc/meminfo | grep -i commit

# CommitLimit: maximum commitment # Committed_AS: current commitment ```

Check Overcommit Settings

```bash # Check overcommit configuration cat /proc/sys/vm/overcommit_memory

# Values: # 0 - heuristic overcommit (may reject legitimate forks) # 1 - always overcommit (Redis recommended) # 2 - never overcommit (strict, may cause fork failures) ```

Check THP Status

```bash # Transparent Huge Pages status cat /sys/kernel/mm/transparent_hugepage/enabled

# [always] means THP is on - problematic for Redis # [never] means THP is off - recommended # [madvise] means selective - may still cause issues ```

Check Process Limits

```bash # Redis user limits ulimit -a

# Check max processes ulimit -u

# System-wide limits cat /proc/sys/kernel/pid_max cat /proc/sys/vm/max_map_count ```

Solutions

Solution 1: Enable Memory Overcommit

This is the primary fix for Redis fork issues:

```bash # Enable overcommit (recommended for Redis) sudo sysctl vm.overcommit_memory=1

# Make permanent echo "vm.overcommit_memory=1" | sudo tee -a /etc/sysctl.conf

# Apply immediately sudo sysctl -p ```

What this does: Allows kernel to overcommit memory, accepting fork requests even when theoretical commitment exceeds physical memory.

Solution 2: Disable Transparent Huge Pages

THP causes latency spikes during fork and can prevent successful fork:

```bash # Disable THP temporarily sudo echo never > /sys/kernel/mm/transparent_hugepage/enabled

# Make permanent via rc.local or systemd sudo tee -a /etc/rc.local << EOF echo never > /sys/kernel/mm/transparent_hugepage/enabled EOF

# Or via systemd service sudo tee /etc/systemd/system/disable-thp.service << EOF [Unit] Description=Disable Transparent Huge Pages

[Service] Type=oneshot ExecStart=/bin/sh -c "echo never > /sys/kernel/mm/transparent_hugepage/enabled"

[Install] WantedBy=multi-user.target EOF

sudo systemctl enable disable-thp sudo systemctl start disable-thp ```

Solution 3: Reduce Redis Memory Usage

If overcommit alone doesn't help, reduce Redis memory:

```bash # Set memory limit redis-cli CONFIG SET maxmemory 4gb

# Set eviction policy redis-cli CONFIG SET maxmemory-policy allkeys-lru

# Trigger immediate cleanup redis-cli MEMORY PURGE

# In redis.conf: maxmemory 4gb maxmemory-policy allkeys-lru ```

Solution 4: Use Diskless Persistence

Skip the fork entirely by sending data directly to replicas:

```bash # Enable diskless replication redis-cli CONFIG SET repl-diskless-sync yes

# Delay to allow multiple replicas to join redis-cli CONFIG SET repl-diskless-sync-delay 5

# For persistence, consider disabling RDB if replicas handle it redis-cli CONFIG SET save "" ```

Solution 5: Increase Swap Space

Swap provides extra "memory" for fork accounting:

```bash # Check current swap sudo swapon --show

# Add swap file (e.g., 4GB) sudo fallocate -l 4G /swapfile sudo chmod 600 /swapfile sudo mkswap /swapfile sudo swapon /swapfile

# Make permanent in /etc/fstab /swapfile none swap sw 0 0 ```

Solution 6: Reduce Save Frequency

If fork occasionally succeeds, reduce persistence frequency:

```bash # Less frequent saves redis-cli CONFIG SET save "900 1 3600 10"

# Or disable automatic save, rely on manual redis-cli CONFIG SET save ""

# Manual save when load is low redis-cli BGSAVE ```

Solution 7: Check for Memory Leaks

If RSS grows over time, investigate:

```bash # Monitor RSS growth redis-cli INFO memory | grep used_memory_rss

# Find large keys redis-cli --bigkeys

# Check for fragmentation redis-cli INFO memory | grep mem_fragmentation_ratio

# If ratio > 1.5, enable active defrag redis-cli CONFIG SET activedefrag yes ```

Solution 8: Add Physical Memory

If all else fails, the system genuinely needs more RAM:

```bash # Estimate memory needed: # Redis memory + 50% for fork + overhead

# For 8GB Redis: # Need at least 12GB system memory # Recommended: 16GB or more ```

Configuration for Fork Safety

```ini # /etc/redis/redis.conf

# Memory management maxmemory 4gb maxmemory-policy allkeys-lru

# Active defragmentation activedefrag yes active-defrag-ignore-bytes 100mb active-defrag-threshold-lower 10 active-defrag-threshold-upper 100

# Persistence save 900 1 save 300 10 save 60 10000

# Diskless replication (if using replicas) repl-diskless-sync yes repl-diskless-sync-delay 5 ```

bash
# /etc/sysctl.conf additions
vm.overcommit_memory=1
bash
# Disable THP (in rc.local or systemd)
echo never > /sys/kernel/mm/transparent_hugepage/enabled

Monitoring Script

```bash #!/bin/bash # fork_check.sh

# Check persistence status PERSIST_STATUS=$(redis-cli INFO persistence)

RDB_STATUS=$(echo "$PERSIST_STATUS" | grep rdb_last_bgsave_status | cut -d: -f2 | tr -d '\r') AOF_STATUS=$(echo "$PERSIST_STATUS" | grep aof_last_bgrewrite_status | cut -d: -f2 | tr -d '\r')

if [ "$RDB_STATUS" != "ok" ]; then echo "CRITICAL: RDB persistence failing" fi

if [ "$AOF_STATUS" != "ok" ]; then echo "CRITICAL: AOF persistence failing" fi

# Check memory RSS=$(redis-cli INFO memory | grep used_memory_rss | cut -d: -f2 | tr -d '\r') COMMIT=$(cat /proc/meminfo | grep Committed_AS | awk '{print $2}') LIMIT=$(cat /proc/meminfo | grep CommitLimit | awk '{print $2}')

PERCENT=$((COMMIT * 100 / LIMIT))

if [ $PERCENT -gt 90 ]; then echo "WARNING: Memory commitment at ${PERCENT}%" fi

# Check THP THP=$(cat /sys/kernel/mm/transparent_hugepage/enabled | grep -o '[.*]') if [ "$THP" != "[never]" ]; then echo "WARNING: Transparent Huge Pages enabled: $THP" fi

# Check overcommit OVERCOMMIT=$(cat /proc/sys/vm/overcommit_memory) if [ "$OVERCOMMIT" != "1" ]; then echo "WARNING: overcommit_memory is $OVERCOMMIT (should be 1)" fi ```

Prevention Checklist

  • [ ] Set vm.overcommit_memory=1
  • [ ] Disable Transparent Huge Pages
  • [ ] Configure appropriate maxmemory
  • [ ] Monitor RSS vs committed memory
  • [ ] Use diskless replication if possible
  • [ ] Keep system memory > Redis memory * 1.5
  • [ ] Enable active defragmentation
  • [ ] Test BGSAVE after configuration changes
  • [Redis Out of Memory](./fix-redis-out-of-memory)
  • [Redis Persistence Failed](./fix-redis-persistence-failed)
  • [Redis Memory Fragmentation High](./fix-redis-memory-fragmentation-high)