Introduction

Web frameworks like Flask and Django can use pickle-based session backends to serialize user session data. When the server restarts during a write, the disk fills up, or a race condition occurs between read and write, the pickle file becomes truncated or corrupted. The next request to load that session fails with UnpicklingError or EOFError, often taking down the entire request.

Symptoms

  • EOFError: Ran out of input when loading session data
  • _pickle.UnpicklingError: pickle data was truncated
  • Flask/Django 500 errors for all users with active sessions
  • django.core.exceptions.SuspiciousOperation when session decode fails
  • Error spikes after server deployment or restart
bash
Traceback (most recent call last):
  File "flask/sessions.py", line 93, in open_session
    return self.load_session(session_id)
  File "flask/sessions.py", line 101, in load_session
    data = pickle.loads(raw_data)
EOFError: Ran out of input

Common Causes

  • Server killed (SIGKILL) during session write
  • Disk full during session serialization
  • Concurrent requests reading/writing same session file
  • Session backend migration leaving stale pickle data
  • Redis session data corrupted during RDB snapshot

Step-by-Step Fix

  1. 1.Identify corrupted sessions:
  2. 2.```python
  3. 3.import pickle
  4. 4.import os

corrupted = [] session_dir = "/var/lib/app/sessions"

for fname in os.listdir(session_dir): filepath = os.path.join(session_dir, fname) try: with open(filepath, 'rb') as f: data = f.read() if len(data) == 0: corrupted.append((fname, "empty")) continue pickle.loads(data) except (EOFError, pickle.UnpicklingError) as e: corrupted.append((fname, str(e)))

print(f"Found {len(corrupted)} corrupted sessions") for fname, reason in corrupted[:5]: print(f" {fname}: {reason}") ```

  1. 1.Handle corrupted sessions gracefully:
  2. 2.```python
  3. 3.from flask import Flask, session
  4. 4.import pickle

app = Flask(__name__)

@app.before_request def safe_session_load(): try: raw = load_session_data() if raw: session_data = pickle.loads(raw) except (EOFError, pickle.UnpicklingError): # Corrupted session - start fresh session.clear() app.logger.warning("Corrupted session detected, starting fresh") ```

  1. 1.Migrate from pickle to JSON sessions:
  2. 2.```python
  3. 3.# Flask - switch to secure cookie sessions (JSON-based)
  4. 4.from flask import Flask
  5. 5.from itsdangerous import URLSafeTimedSerializer

app = Flask(__name__) app.config['SESSION_TYPE'] = 'filesystem' app.secret_key = 'your-secret-key'

# Django - switch to JSON serializer # settings.py SESSION_SERIALIZER = 'django.contrib.sessions.serializers.JSONSerializer' ```

  1. 1.Clear corrupted sessions from storage:
  2. 2.```bash
  3. 3.# File-based sessions
  4. 4.find /var/lib/app/sessions -name "*.pkl" -empty -delete
  5. 5.find /var/lib/app/sessions -name "*.pkl" -size 0 -delete

# Redis sessions redis-cli KEYS "session:*" | while read key; do size=$(redis-cli STRLEN "$key") if [ "$size" -lt 10 ]; then redis-cli DEL "$key" echo "Deleted empty session: $key" fi done ```

Prevention

  • Use JSON serialization instead of pickle for sessions (safer and more portable)
  • Implement atomic session writes:
  • ```python
  • import tempfile
  • import os

def save_session_atomic(session_id, data): """Write session atomically.""" tmp_fd, tmp_path = tempfile.mkstemp(dir=session_dir) try: with os.fdopen(tmp_fd, 'wb') as tmp: pickle.dump(data, tmp, protocol=5) os.replace(tmp_path, os.path.join(session_dir, session_id)) except Exception: os.unlink(tmp_path) raise ``` - Add session validation middleware that catches errors before they propagate - Monitor session file sizes and alert on empty or unusually small sessions - Use signed cookie sessions to avoid server-side storage issues entirely