Introduction

Tomcat processes HTTP requests using a thread pool. When all threads are busy and the accept queue (backlog) is full, new connections are rejected and clients receive connection refused errors. The default maxThreads of 200 is adequate for fast request-response cycles but becomes a bottleneck when requests involve slow database queries, external API calls, or large file processing. Unlike thread-per-request blocking I/O, Tomcat can use async request processing (Servlet 3.0+) to free threads while waiting for I/O, dramatically increasing throughput for I/O-bound workloads.

Symptoms

bash
# Client receives:
curl: (7) Failed to connect to server port 8080: Connection refused

Or in Tomcat logs:

bash
org.apache.tomcat.util.threads.ThreadPoolExecutor - All threads are busy, queue is full
java.util.concurrent.RejectedExecutionException: Task rejected

Or thread dump shows all threads in WAITING:

bash
"http-nio-8080-exec-199" #200 RUNNABLE
"http-nio-8080-exec-200" #201 RUNNABLE
# No idle threads available for new requests

Common Causes

  • maxThreads too low: Default 200 insufficient for concurrent traffic
  • Slow requests block threads: Long database queries or API calls hold threads
  • acceptCount too small: Default 100 connection backlog fills up quickly
  • Connection leak: HTTP connections not closed, threads stuck
  • No async processing: I/O-bound requests waste threads waiting
  • Thread pool starvation: All threads waiting on same slow dependency

Step-by-Step Fix

Step 1: Configure Tomcat thread pool

yaml
# Spring Boot application.yml
server:
  tomcat:
    threads:
      max: 500           # Increased from default 200
      min-spare: 50      # Pre-warmed threads
    max-connections: 10000  # Max simultaneous connections (NIO)
    accept-count: 200    # Queue size when all threads busy
    connection-timeout: 20000  # 20 seconds
    keep-alive-timeout: 30000   # 30 seconds for keep-alive
    max-keep-alive-requests: 100

Step 2: Use async request processing

```java @RestController public class ReportController {

@GetMapping("/reports/{id}") public CompletableFuture<ReportDTO> getReport(@PathVariable Long id) { // Returns immediately, freeing the Tomcat thread return CompletableFuture.supplyAsync(() -> { // This runs on ForkJoinPool, not Tomcat thread pool return reportService.generateReport(id); }); }

// With timeout @GetMapping("/exports/{id}") public DeferredResult<ExportResult> export(@PathVariable Long id) { DeferredResult<ExportResult> result = new DeferredResult<>(30000L); // 30s timeout

exportService.startExport(id, exportResult -> { result.setResult(exportResult); // Called when export completes });

result.onTimeout(() -> { result.setErrorResult( ResponseEntity.status(HttpStatus.GATEWAY_TIMEOUT) .body(ExportResult.timeout()) ); });

return result; } } ```

Step 3: Monitor thread pool health

```java @Component public class TomcatMetrics {

private final TomcatMetricsConfig config;

@EventListener(ApplicationStartedEvent.class) public void logTomcatConfig() { // Log current thread pool configuration log.info("Tomcat maxThreads: {}", config.getMaxThreads()); log.info("Tomcat maxConnections: {}", config.getMaxConnections()); } }

// Actuator endpoint shows thread pool metrics: // /actuator/metrics/tomcat.threads.current // /actuator/metrics/tomcat.threads.busy ```

Prevention

  • Configure maxThreads based on expected concurrent requests and request duration
  • Use async request processing for I/O-bound operations
  • Set connection-timeout to release threads from slow clients
  • Monitor Tomcat thread pool metrics with Micrometer
  • Add circuit breakers to prevent threads waiting on failing services
  • Use a load balancer to distribute traffic across multiple Tomcat instances
  • Profile slow endpoints and optimize them to reduce thread hold time