What's Actually Happening

Redis persists data to disk using RDB snapshots or AOF log. When persistence fails, you risk data loss on restart. Common causes include disk issues, permission problems, and fork failures.

The Error You'll See

In Redis logs:

bash
Background saving error: Permission denied
# Or
BGSAVE failed: fork: Cannot allocate memory
# Or
Write error saving DB on disk: No space left on device

Why This Happens

  1. 1.Disk full - No space to write RDB or AOF file
  2. 2.Permission denied - Redis can't write to data directory
  3. 3.Fork failure - Can't fork process for background save
  4. 4.Memory overcommit - Linux overcommit settings prevent fork
  5. 5.Large dataset - Dataset too large to save within timeout

Step 1: Check Redis Persistence Status

bash
redis-cli info persistence

Output:

bash
rdb_last_bgsave_status:err
rdb_last_bgsave_time_sec:-1
rdb_last_bgsave_error:Permission denied
aof_last_bgrewrite_status:ok
aof_enabled:1

Look at rdb_last_bgsave_status and aof_last_bgrewrite_status.

Step 2: Check Disk Space

```bash # Check disk usage df -h /var/lib/redis

# Check inodes df -i /var/lib/redis ```

If disk full:

```bash # Free space by removing old files rm /var/log/redis/old-logs

# Or move Redis data to larger disk # Update redis.conf: dir /new/path ```

Step 3: Check Directory Permissions

```bash # Check Redis data directory ls -la /var/lib/redis

# Should be owned by redis user # drwxr-xr-x redis redis /var/lib/redis ```

Fix permissions:

bash
sudo chown -R redis:redis /var/lib/redis
sudo chmod 755 /var/lib/redis

Step 4: Test Manual Save

bash
redis-cli bgsave

Check result:

bash
redis-cli lastsave

If it fails, check logs:

bash
tail -f /var/log/redis/redis-server.log

Step 5: Fix Fork Issues

If fork fails with "Cannot allocate memory":

```bash # Check overcommit settings cat /proc/sys/vm/overcommit_memory

# 0 = heuristic (default) # 1 = always overcommit # 2 = don't overcommit ```

Fix:

```bash # Allow overcommit (recommended for Redis) echo 1 > /proc/sys/vm/overcommit_memory

# Persist echo "vm.overcommit_memory = 1" >> /etc/sysctl.conf sysctl -p ```

Step 6: Disable Transparent Huge Pages

THP can cause fork latency:

```bash # Check THP status cat /sys/kernel/mm/transparent_hugepage/enabled

# Disable THP echo never > /sys/kernel/mm/transparent_hugepage/enabled echo never > /sys/kernel/mm/transparent_hugepage/defrag ```

Persist in /etc/rc.local:

bash
echo never > /sys/kernel/mm/transparent_hugepage/enabled

Step 7: Check RDB Configuration

bash
redis-cli config get save
# 1) "save"
# 2) "900 1 300 10 60 10000"

Format: save <seconds> <changes>

To disable RDB (AOF only):

bash
redis-cli config set save ""

To reconfigure:

bash
redis-cli config set save "900 1 300 10 60 10000"

Step 8: Check AOF Configuration

```bash redis-cli config get appendonly # 1) "appendonly" # 2) "yes"

redis-cli config get appendfsync # 1) "appendfsync" # 2) "everysec" ```

If AOF file corrupted:

```bash # Check AOF file redis-check-aof /var/lib/redis/appendonly.aof

# Fix corrupted AOF redis-check-aof --fix /var/lib/redis/appendonly.aof ```

Step 9: Check RDB File

bash
# Check RDB file integrity
redis-check-rdb /var/lib/redis/dump.rdb

Step 10: Handle Large Dataset Saves

If save times out:

```bash # Check save time redis-cli info persistence | grep rdb_last_bgsave_time_sec

# If consistently high, increase stop-writes-on-bgsave-error redis-cli config set stop-writes-on-bgsave-error no # WARNING: This allows writes even if save fails ```

Better solution: Optimize dataset or use faster disk.

Verify the Fix

```bash # Trigger save redis-cli bgsave

# Check status redis-cli info persistence | grep rdb_last_bgsave_status # rdb_last_bgsave_status:ok

# Check file exists and has recent timestamp ls -la /var/lib/redis/dump.rdb ```

Prevention Tips

```bash # Monitor persistence status # Prometheus: redis_rdb_last_bgsave_status != 1

# Set up disk space alerts # Alert when disk > 80% full

# Regular backup of RDB/AOF files cp /var/lib/redis/dump.rdb /backup/redis-$(date +%Y%m%d).rdb

# Configure both RDB and AOF for redundancy save 900 1 appendonly yes ```