Introduction
Spring Retry provides declarative retry logic through @Retryable annotations, allowing operations to be retried on transient failures. When retry is configured without a backoff policy, retries happen immediately in rapid succession, overwhelming the failing service and exhausting all retry attempts within milliseconds. The ExhaustedRetryException is thrown when all retry attempts have been consumed without success, and without proper fallback handling, this exception propagates to the caller. The fix involves configuring exponential backoff, setting appropriate retry limits, and providing fallback methods.
Symptoms
org.springframework.retry.ExhaustedRetryException: Retry exhausted after 3 attempts
at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:356)
at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:180)
at com.example.service.PaymentService.processPayment(PaymentService.java:45)Application logs show rapid-fire retries:
10:23:45.100 ERROR PaymentService - Payment failed: Connection refused
10:23:45.101 ERROR PaymentService - Payment failed: Connection refused <-- retry 1, 1ms later
10:23:45.101 ERROR PaymentService - Payment failed: Connection refused <-- retry 2, 0ms later
10:23:45.102 ERROR PaymentService - Retry exhausted after 3 attempts <-- all retries in 2msCommon Causes
- No backoff configured: Default Spring Retry retries immediately with no delay
- Max attempts too low: Default 3 attempts may not be enough for genuinely transient failures
- Retrying on non-retryable exceptions: Retrying on
IllegalArgumentExceptionwhich will never succeed - No fallback method: When retries are exhausted, the exception propagates without a graceful degradation path
- Retry on synchronous blocking calls: Retry blocks the calling thread during backoff delays
- State not reset between retries: Retry reuses stale state that caused the original failure
Step-by-Step Fix
Step 1: Configure exponential backoff
```java import org.springframework.retry.annotation.Backoff; import org.springframework.retry.annotation.Recover; import org.springframework.retry.annotation.Retryable;
@Service public class PaymentService {
@Retryable( value = {PaymentGatewayException.class, TimeoutException.class}, maxAttempts = 5, backoff = @Backoff( delay = 1000, // Initial delay: 1 second maxDelay = 30000, // Maximum delay: 30 seconds multiplier = 2.0, // Exponential: 1s, 2s, 4s, 8s, 16s random = true // Add jitter to prevent thundering herd ) ) public PaymentResult processPayment(PaymentRequest request) { return paymentGateway.charge(request); }
@Recover public PaymentResult recoverPayment(PaymentGatewayException e, PaymentRequest request) { log.warn("Payment failed after all retries for order {}: {}", request.getOrderId(), e.getMessage()); // Queue for later processing retryQueue.add(request); return PaymentResult.pending(request.getOrderId()); } } ```
With this configuration, retries happen at 1s, 2s, 4s, 8s, 16s -- giving the downstream service time to recover.
Step 2: Use RetryTemplate for programmatic control
```java @Configuration @EnableRetry public class RetryConfig {
@Bean public RetryTemplate retryTemplate() { RetryTemplate template = RetryTemplate.builder() .maxAttempts(5) .exponentialBackoff(1000, 2.0, 30000, true) .retryOn(PaymentGatewayException.class) .retryOn(TimeoutException.class) .notRetryOn(InvalidPaymentException.class) .withListener(new RetryListener() { @Override public <T, E extends Throwable> boolean open(RetryContext context, RetryCallback<T, E> callback) { log.info("Starting retry for {}", context.getAttribute("context.name")); return true; }
@Override public <T, E extends Throwable> void onError(RetryContext context, RetryCallback<T, E> callback, Throwable throwable) { log.warn("Retry attempt {} failed: {}", context.getRetryCount(), throwable.getMessage()); }
@Override public <T, E extends Throwable> void close(RetryContext context, RetryCallback<T, E> callback, Throwable throwable) { if (throwable != null) { log.error("Retry exhausted after {} attempts", context.getRetryCount()); } } }) .build();
return template; } } ```
Step 3: Add circuit breaker for persistent failures
```java @Bean public RetryTemplate retryTemplateWithCircuitBreaker() { return RetryTemplate.builder() .maxAttempts(5) .exponentialBackoff(1000, 2.0, 30000, true) .retryOn(PaymentGatewayException.class) .traversingCauses() .build(); }
// Use with Resilience4j circuit breaker @Retryable(retryFor = {PaymentGatewayException.class}) @CircuitBreaker(name = "paymentGateway", fallbackMethod = "fallbackPayment") public PaymentResult processPayment(PaymentRequest request) { return paymentGateway.charge(request); }
public PaymentResult fallbackPayment(PaymentRequest request, Throwable t) { log.error("Circuit breaker open for payment gateway, queuing payment for later"); return PaymentResult.queued(request.getOrderId()); } ```
Prevention
- Always configure exponential backoff with
@Backoff(delay = 1000, multiplier = 2.0) - Set
maxAttemptsto 5 for most transient failures - Use
@Recovermethods to provide graceful degradation when retries are exhausted - Exclude non-retryable exceptions with
notRetryOn(e.g., validation errors) - Add a
RetryListenerto log retry attempts for monitoring - Combine retry with circuit breaker pattern for persistent failures
- Monitor retry rates in production -- a sudden spike indicates downstream degradation