Introduction

RabbitMQ monitors available disk space and blocks all publishing connections when free disk space drops below the configured disk_free_limit. This safety mechanism prevents disk exhaustion that could corrupt the message store. However, it also means that no new messages can be published until disk space is freed or the limit is adjusted.

Symptoms

  • Producers receive ChannelClosedException: connection blocked when publishing
  • RabbitMQ management UI shows Resource alarm: disk space warning banner
  • rabbitmqctl status shows disk_free_alarm: true
  • Queue depths grow as messages are consumed but not replaced by new publications
  • Error message: NOT_ALLOWED - connection blocked so that resource alarm can be cleared

Common Causes

  • Disk free limit set too high relative to available disk capacity
  • Persistent queues accumulating messages faster than consumers can process them
  • Lazy queues not configured, causing all messages to be stored on disk
  • Log files growing unbounded, consuming space needed by RabbitMQ data directory
  • Disk not sized to handle peak message retention period

Step-by-Step Fix

  1. 1.Confirm the disk alarm state and current disk usage: Check how far below the threshold the disk has fallen.
  2. 2.```bash
  3. 3.rabbitmqctl status | grep -A3 "disk_free"
  4. 4.df -h /var/lib/rabbitmq
  5. 5.`
  6. 6.Purge non-critical queues to free disk space: Remove message backlog from low-priority queues.
  7. 7.```bash
  8. 8.rabbitmqctl purge_queue low-priority-events
  9. 9.rabbitmqctl purge_queue staging-notifications
  10. 10.`
  11. 11.Delete unused queues and exchanges: Clean up stale resources.
  12. 12.```bash
  13. 13.rabbitmqctl list_queues name messages durable auto_delete
  14. 14.# Delete queues with auto_delete that have no consumers
  15. 15.rabbitmqctl delete_queue temp-test-queue
  16. 16.`
  17. 17.Adjust disk free limit if appropriately sized: Set the limit based on actual disk capacity.
  18. 18.```properties
  19. 19.# rabbitmq.conf - set to 2GB or 50MB per GB of RAM, whichever is higher
  20. 20.disk_free_limit.absolute = 2GB
  21. 21.`
  22. 22.Clear disk alarm and verify publishers resume: Confirm the alarm is cleared.
  23. 23.```bash
  24. 24.rabbitmqctl status | grep disk_free_alarm
  25. 25.# Should show: disk_free_alarm: false
  26. 26.`

Prevention

  • Set disk_free_limit.absolute to at least 2GB for production brokers
  • Enable lazy queues for high-volume, slow-consumption scenarios to reduce disk pressure
  • Configure message TTL and queue length limits to prevent unbounded message accumulation
  • Monitor disk usage trends and alert before reaching the alarm threshold
  • Use separate disk volumes for RabbitMQ data and system logs to prevent log growth from triggering alarms
  • Implement automated queue management policies that purge or expire old messages based on age