Introduction

Tomcat processes incoming HTTP requests using a fixed-size thread pool. When all threads are busy processing requests and the accept queue (configured by acceptCount) is also full, Tomcat refuses new TCP connections, causing clients to see Connection refused or connection timeout errors. This error typically occurs under load when slow request handlers hold threads for extended periods, gradually consuming all available threads and creating a cascading failure where even healthy requests cannot be processed.

Symptoms

Clients receive connection errors:

bash
curl: (7) Failed to connect to myapp.example.com port 8080: Connection refused

Or connection timeouts:

bash
java.net.ConnectException: Connection timed out (Connection timed out)

Tomcat access log shows requests queuing:

bash
10.0.1.50 - - [15/Mar/2024:14:23:01 +0000] "POST /api/reports HTTP/1.1" 200 45231 127843
                                                                                   ^ ms - request waited 127 seconds in queue

Tomcat thread pool metrics via JMX:

bash
currentThreadCount: 200
currentThreadsBusy: 200    <-- All threads busy
maxThreads: 200

Common Causes

  • maxThreads too low for traffic: Default 200 threads may be insufficient for high-traffic applications
  • Slow request handlers: Database queries, external API calls, or file processing holding threads for seconds
  • Deadlocked threads: Database deadlock or application-level deadlock permanently consumes threads
  • acceptCount too small: Default 100-connection queue fills quickly under burst traffic
  • Thread starvation from sync I/O: Blocking I/O operations (file reads, network calls) tie up threads
  • Memory pressure causing GC pauses: Long GC pauses make threads appear stuck

Step-by-Step Fix

Step 1: Tune Tomcat connector configuration

yaml
# application.yml (Spring Boot)
server:
  tomcat:
    threads:
      max: 400          # Increase from default 200
      min-spare: 50     # Minimum idle threads
    max-connections: 10000  # Max TCP connections (not threads)
    accept-count: 200   # Queue size when all threads busy
    connection-timeout: 20000  # 20 seconds

Or in server.xml:

xml
<Connector port="8080" protocol="HTTP/1.1"
           maxThreads="400"
           minSpareThreads="50"
           maxConnections="10000"
           acceptCount="200"
           connectionTimeout="20000"
           compression="on"
           compressibleMimeType="text/html,text/xml,text/plain,application/json"/>

Step 2: Identify slow request handlers

Enable Tomcat request duration logging:

```java @Component public class RequestTimingFilter implements Filter {

@Override public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException { long start = System.currentTimeMillis(); try { chain.doFilter(request, response); } finally { long duration = System.currentTimeMillis() - start; HttpServletRequest httpReq = (HttpServletRequest) request; if (duration > 5000) { log.warn("Slow request: {} {} took {}ms", httpReq.getMethod(), httpReq.getRequestURI(), duration); } } } } ```

Step 3: Offload long-running work to async processing

```java @RestController public class ReportController {

@PostMapping("/api/reports") public CompletableFuture<ReportResponse> generateReport(@RequestBody ReportRequest request) { return CompletableFuture.supplyAsync(() -> { // This runs on ForkJoinPool, not Tomcat threads return reportService.generate(request); }); } } ```

Or use Spring's async servlet processing:

```java @PostMapping("/api/reports") public DeferredResult<ReportResponse> generateReport(@RequestBody ReportRequest request) { DeferredResult<ReportResponse> deferredResult = new DeferredResult<>(60000L);

// Release Tomcat thread immediately asyncExecutor.execute(() -> { try { ReportResponse response = reportService.generate(request); deferredResult.setResult(response); } catch (Exception e) { deferredResult.setErrorResult(e); } });

return deferredResult; } ```

Step 4: Monitor thread pool in production

```java @Component public class TomcatThreadMonitor {

private final TomcatThreadPoolMXBean threadPool;

@Scheduled(fixedRate = 30000) public void checkThreadPool() { int busy = threadPool.getCurrentThreadsBusy(); int max = threadPool.getMaxThreads(); double utilization = (double) busy / max * 100;

log.info("Tomcat thread pool: {}/{} busy ({:.1f}%)", busy, max, utilization);

if (utilization > 80) { log.warn("Thread pool utilization above 80% - consider scaling or optimizing"); } } } ```

Prevention

  • Set maxThreads based on your application's concurrency profile, not the default 200
  • Set acceptCount to handle traffic bursts without dropping connections
  • Enable request timing logging to identify slow endpoints
  • Use async processing for requests that take more than 1 second
  • Monitor thread pool utilization and alert at 80% utilization
  • Set connection-timeout to prevent slow clients from holding threads indefinitely
  • Use server.tomcat.max-connections to control the total number of TCP connections