The Problem

Redis fails to perform background saves with the error:

bash
Background saving error: fork: Cannot allocate memory

or

bash
Can't save in background: fork: Resource temporarily unavailable

Despite having plenty of free memory, the fork() system call fails. This is a common issue on Linux systems running Redis with large datasets.

Understanding the Fork Problem

  1. 1.When Redis calls fork() for BGSAVE:
  2. 2.A child process is created with copy-on-write semantics
  3. 3.Memory is shared between parent and child
  4. 4.Linux must reserve space for worst-case (all pages written)
  5. 5.Without overcommit, this requires 2x memory

The fork doesn't actually use 2x memory, but Linux accounting thinks it might.

Diagnosis Commands

Check Fork Error in Logs

bash
grep -i "fork\|cannot allocate\|background saving" /var/log/redis/redis-server.log

Check Overcommit Settings

bash
cat /proc/sys/vm/overcommit_memory
cat /proc/sys/vm/overcommit_ratio

Values for overcommit_memory: - 0 - Heuristic overcommit (default, problematic for Redis) - 1 - Always overcommit (recommended for Redis) - 2 - Never overcommit (strict, requires careful sizing)

Check Memory State

bash
cat /proc/meminfo | grep -E "MemTotal|MemFree|MemAvailable|CommitLimit|Committed_AS"
free -h

Check Redis Memory Usage

bash
redis-cli INFO memory | grep -E "used_memory|used_memory_rss"

Check Process Limits

bash
prlimit --pid=$(pgrep redis-server) | grep -E "AS|MEM"
ulimit -a

Solutions

Solution 1: Enable Overcommit Memory

The primary fix for most fork errors:

```bash # Check current setting cat /proc/sys/vm/overcommit_memory

# Enable overcommit (recommended for Redis) sudo sysctl vm.overcommit_memory=1

# Make permanent echo "vm.overcommit_memory=1" | sudo tee -a /etc/sysctl.conf sudo sysctl -p ```

Why this works: With overcommit_memory=1, Linux allows fork() to succeed even if the theoretical memory requirement exceeds available memory. The copy-on-write mechanism means the actual memory usage won't double.

Solution 2: Disable Transparent Huge Pages

Transparent Huge Pages (THP) causes latency spikes during fork:

```bash # Check current setting cat /sys/kernel/mm/transparent_hugepage/enabled

# Disable THP sudo echo never | sudo tee /sys/kernel/mm/transparent_hugepage/enabled sudo echo never | sudo tee /sys/kernel/mm/transparent_hugepage/defrag

# Make permanent (add to rc.local or systemd unit) # For systemd, create: /etc/systemd/system/disable-thp.service ```

Create systemd service:

```bash sudo tee /etc/systemd/system/disable-thp.service <<'EOF' [Unit] Description=Disable Transparent Huge Pages (THP)

[Service] Type=oneshot ExecStart=/bin/sh -c 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' ExecStart=/bin/sh -c 'echo never > /sys/kernel/mm/transparent_hugepage/defrag'

[Install] WantedBy=multi-user.target EOF

sudo systemctl enable disable-thp sudo systemctl start disable-thp ```

Solution 3: Add Swap Space

Without swap, fork may fail even with overcommit:

```bash # Check swap free -h | grep Swap

# Add swap (2GB example) sudo fallocate -l 2G /swapfile sudo chmod 600 /swapfile sudo mkswap /swapfile sudo swapon /swapfile

# Make permanent echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab ```

Redis recommends swap equal to or greater than memory used.

Solution 4: Increase Resource Limits

Check and increase limits:

```bash # Check current limits for Redis process prlimit --pid=$(pgrep redis-server)

# Increase via systemd service file sudo systemctl edit redis-server ```

Add:

ini
[Service]
LimitNOFILE=65535
LimitNPROC=65535
LimitMEMLOCK=infinity
LimitAS=infinity

Then restart:

bash
sudo systemctl daemon-reload
sudo systemctl restart redis-server

Solution 5: Reduce Memory Fragmentation

High fragmentation increases fork memory requirements:

```bash # Check fragmentation redis-cli INFO memory | grep mem_fragmentation_ratio

# Enable active defrag if ratio > 1.5 redis-cli CONFIG SET activedefrag yes ```

Solution 6: Use AOF Instead

For systems where fork consistently fails:

bash
# Switch to AOF-only persistence
redis-cli CONFIG SET save ""
redis-cli CONFIG SET appendonly yes
redis-cli CONFIG SET appendfsync everysec

AOF uses fork for rewrite but less frequently than RDB snapshots.

Complete Fix Procedure

Step 1: Immediate Fix

```bash # Enable overcommit sudo sysctl vm.overcommit_memory=1

# Disable THP sudo echo never | sudo tee /sys/kernel/mm/transparent_hugepage/enabled

# Trigger save to test redis-cli BGSAVE ```

Step 2: Verify Fix

```bash # Check save status redis-cli INFO persistence | grep rdb_last_bgsave_status

# Should show: ok ```

Step 3: Make Permanent

Add to /etc/sysctl.conf:

conf
vm.overcommit_memory=1

Create THP disable service (shown above).

Step 4: Configure Redis

Update redis.conf:

```conf # Memory management maxmemory 4gb maxmemory-policy allkeys-lru

# Persistence tuning save 900 1 save 300 10 save 60 10000 stop-writes-on-bgsave-error yes ```

Special Cases

Docker Containers

Redis in Docker inherits host kernel settings but may need privileged mode:

```bash # Run with needed capabilities docker run --cap-add=SYS_ADMIN redis

# Or set host settings (run on host) echo 1 > /proc/sys/vm/overcommit_memory ```

Kubernetes

Use init container or privileged pod:

yaml
initContainers:
- name: sysctl
  image: busybox
  command: ['sh', '-c', 'sysctl -w vm.overcommit_memory=1']
  securityContext:
    privileged: true

Very Large Memory Systems

For Redis using > 50% of system memory:

```bash # Ensure sufficient swap sudo fallocate -l 16G /swapfile sudo chmod 600 /swapfile sudo mkswap /swapfile sudo swapon /swapfile

# Reduce swappiness (don't swap aggressively) sudo sysctl vm.swappiness=1 ```

Monitoring Fork Success

Check Fork Statistics

bash
redis-cli INFO stats | grep -E "total_forks|forked_child_errors"

Monitor Script

```bash #!/bin/bash # Monitor fork success

while true; do FORKS=$(redis-cli INFO stats | grep total_forks | cut -d: -f2 | tr -d '\r') ERRORS=$(redis-cli INFO stats | grep forked_child_errors | cut -d: -f2 | tr -d '\r')

if [ "$ERRORS" -gt 0 ]; then echo "$(date): WARNING - $ERRORS fork errors" redis-cli BGSAVE fi

sleep 60 done ```

Alert on Fork Failure

bash
# In monitoring system, alert on:
redis-cli INFO persistence | grep rdb_last_bgsave_status | grep -c "err"

Verification

After applying fixes:

```bash # Force BGSAVE redis-cli BGSAVE

# Monitor watch -n 1 'redis-cli INFO persistence | grep -E "rdb_bgsave_in_progress|rdb_last_bgsave_status"'

# Should complete successfully # rdb_last_bgsave_status:ok ```