Introduction

The Go http.Client uses a http.Transport that maintains a pool of idle persistent connections for reuse. Under high load, the default settings may not maintain enough idle connections, causing new connections to be created for every request. This increases latency, exhausts file descriptors, and can cause dial timeouts when the server's listen backlog is full.

Symptoms

  • dial tcp: connect: connection refused under load
  • net/http: request canceled while waiting for connection
  • High latency for HTTP requests during traffic spikes
  • TIME_WAIT connections accumulating in netstat
  • Server-side: accept: too many open files
go
// Default transport settings may be insufficient:
// MaxIdleConns: 100
// MaxIdleConnsPerHost: 2  // Only 2 idle connections per host!
// MaxConnsPerHost: 0       // Unlimited (can overwhelm server)
// IdleConnTimeout: 90s

Common Causes

  • MaxIdleConnsPerHost default of 2 is too low for high-throughput services
  • Creating a new http.Client per request instead of reusing
  • Not reading response bodies, preventing connection reuse
  • Server closing idle connections before Go reuses them
  • No connection limits allowing unbounded connection creation

Step-by-Step Fix

  1. 1.Configure transport for high-throughput scenarios:
  2. 2.```go
  3. 3.transport := &http.Transport{
  4. 4.MaxIdleConns: 100,
  5. 5.MaxIdleConnsPerHost: 20, // Increase from default of 2
  6. 6.MaxConnsPerHost: 50, // Prevent unlimited connections
  7. 7.IdleConnTimeout: 90 * time.Second,
  8. 8.TLSHandshakeTimeout: 10 * time.Second,
  9. 9.ExpectContinueTimeout: 1 * time.Second,
  10. 10.}

client := &http.Client{ Transport: transport, Timeout: 30 * time.Second, } ```

  1. 1.Share a single http.Client across the application:
  2. 2.```go
  3. 3.// WRONG - new client per request, no connection reuse
  4. 4.func handler(w http.ResponseWriter, r *http.Request) {
  5. 5.client := &http.Client{Timeout: 10 * time.Second} // New transport each time!
  6. 6.resp, err := client.Get("https://api.example.com/data")
  7. 7.// ...
  8. 8.}

// CORRECT - global shared client with tuned transport var apiClient = &http.Client{ Transport: &http.Transport{ MaxIdleConns: 200, MaxIdleConnsPerHost: 40, MaxConnsPerHost: 80, }, Timeout: 30 * time.Second, }

func handler(w http.ResponseWriter, r *http.Request) { resp, err := apiClient.Get("https://api.example.com/data") // Connections are reused from the pool } ```

  1. 1.Monitor connection pool metrics:
  2. 2.```go
  3. 3.import "net/http"

func LogTransportMetrics(transport *http.Transport) { stats := transport.Clone().IdleConnStats() log.Printf( "Connections: idle=%d, active=%d, total=%d, closed=%d, wait=%d", stats.IdleConns, stats.TotalConns - stats.IdleConns, stats.TotalConns, stats.ReusedConns, stats.WaitCount, ) }

// Call periodically go func() { ticker := time.NewTicker(30 * time.Second) for range ticker.C { LogTransportMetrics(transport) } }() ```

  1. 1.Handle connection pool saturation gracefully:
  2. 2.```go
  3. 3.transport := &http.Transport{
  4. 4.MaxConnsPerHost: 50, // Cap total connections per host
  5. 5.}

// When pool is saturated, requests queue up until timeout // Set appropriate timeout to fail fast client := &http.Client{ Transport: transport, Timeout: 10 * time.Second, // Fail rather than queue indefinitely } ```

Prevention

  • Always use a shared http.Client with tuned http.Transport
  • Set MaxIdleConnsPerHost to match expected concurrent requests to a single host
  • Set MaxConnsPerHost to prevent overwhelming downstream services
  • Monitor IdleConns metrics in production dashboards
  • Use connection pooling libraries like github.com/hashic8/go-retryablehttp for resilience
  • Set server-side keep-alive timeout higher than client's IdleConnTimeout
  • Use httptrace to measure connection reuse rates