What's Actually Happening

Kafka consumer group fails to rebalance when consumers join or leave. Partitions are not reassigned, leading to unbalanced consumption.

The Error You'll See

bash
WARN [Consumer instanceId=...] Rebalance failed.

Or:

bash
ERROR [Consumer instanceId=...] Failed to join group.

Why This Happens

  1. 1.Session timeout too short
  2. 2.Coordinator unavailable
  3. 3.Static membership issues
  4. 4.Max poll interval exceeded
  5. 5.Network partition
  6. 6.Consumer stuck

Step 1: Check Consumer Group Status

bash
kafka-consumer-groups --bootstrap-server localhost:9092 --list
kafka-consumer-groups --bootstrap-server localhost:9092 --describe --group mygroup

Step 2: Check Coordinator

bash
kafka-broker-api-versions --bootstrap-server localhost:9092
kafka-log-dirs --bootstrap-server localhost:9092 --describe

Step 3: Check Session Timeout

properties
# Consumer config
session.timeout.ms=10000
heartbeat.interval.ms=3000
max.poll.interval.ms=300000

Step 4: Check Group State

bash
kafka-consumer-groups --bootstrap-server localhost:9092 --describe --group mygroup --state
kafka-consumer-groups --bootstrap-server localhost:9092 --describe --group mygroup --members --verbose

Step 5: Reset Consumer Group

bash
kafka-consumer-groups --bootstrap-server localhost:9092 --group mygroup --reset-offsets --to-earliest --execute --topic mytopic

Step 6: Delete Consumer Group

bash
kafka-consumer-groups --bootstrap-server localhost:9092 --delete --group mygroup

Step 7: Check Static Membership

properties
group.instance.id=consumer-1

Step 8: Check Broker Logs

bash
tail -f /var/log/kafka/server.log | grep -i "rebalance\|coordinator"

Step 9: Monitor Consumer Lag

bash
kafka-consumer-groups --bootstrap-server localhost:9092 --describe --group mygroup | grep -i lag

Step 10: Verify Rebalance

bash
kafka-consumer-groups --bootstrap-server localhost:9092 --describe --group mygroup
  • [Fix Kafka Producer Retries Exceeded](/articles/fix-kafka-producer-retries-exceeded)
  • [Fix Kafka Broker Not Starting](/articles/fix-kafka-broker-not-starting)