What's Actually Happening
Your Go application cancels a context to signal goroutines to stop, but they continue running. The cancellation signal doesn't propagate through the context chain, causing goroutine leaks and resource exhaustion.
The Error You'll See
Goroutines don't stop:
```go func main() { ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) defer cancel()
go worker(ctx)
time.Sleep(10 * time.Second) // Worker still running after context canceled! }
func worker(ctx context.Context) { for { select { default: doWork() // Never checks ctx.Done() } } } ```
Memory leak from goroutines:
```bash $ curl http://localhost:6060/debug/pprof/goroutine?debug=1
goroutine profile: total 5000 # Too many goroutines!
100 @ 0x1234 0x5678 # 0x1234 main.worker+0x50 /app/main.go:20 # 0x5678 runtime.gopark+0x100
# Goroutine leak detected ```
Context deadline exceeded but operations continue:
```bash $ go test -v
--- FAIL: TestWorker (10.00s) main_test.go:50: context deadline exceeded main_test.go:51: expected worker to stop, but still running FAIL ```
Why This Happens
- 1.Not checking ctx.Done() - Goroutine never selects on done channel
- 2.Wrong context passed - Using parent context instead of derived
- 3.Blocking operations - Code blocked before reaching select
- 4.Context not derived - Using background context without cancel
- 5.Goroutine copied context - Context value lost after goroutine spawn
- 6.HTTP client ignoring context - Using http.Get instead of Request with context
- 7.Database query not using context - SQL query without ctx parameter
- 8.Channel send blocks - Select blocked on channel send, not ctx.Done()
Step 1: Identify Where Context is Not Checked
```go // BAD: Never checks context func worker(ctx context.Context) { for { processData() // Runs forever } }
// BAD: Checks too late func worker(ctx context.Context) { for { data := readFromDB() // Blocking, no context if ctx.Err() != nil { // Never reached return } processData(data) } }
// BAD: Blocking send blocks select func worker(ctx context.Context, results chan<- Result) { for { select { case results <- doWork(): // Blocks if channel full! // ctx.Done() never checked } } }
// GOOD: Check context in select func worker(ctx context.Context) { for { select { case <-ctx.Done(): log.Println("Context canceled:", ctx.Err()) return default: if err := processData(ctx); err != nil { return } } } }
// GOOD: Non-blocking check before work func worker(ctx context.Context) { for { if ctx.Err() != nil { return } processData() } } ```
Step 2: Fix Context Chain Propagation
```go package main
import ( "context" "fmt" "time" )
// BAD: Using wrong context in chain func processData(ctx context.Context, data []byte) error { // Creating new context breaks the chain! ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) defer cancel()
return saveToDatabase(ctx, data) }
// GOOD: Derive from parent context func processData(ctx context.Context, data []byte) error { // Derive from parent to maintain cancellation chain ctx, cancel := context.WithTimeout(ctx, 5*time.Second) defer cancel()
return saveToDatabase(ctx, data) }
// Example: Proper context chain func main() { // Root context ctx := context.Background()
// User request timeout ctx, cancel := context.WithTimeout(ctx, 30*time.Second) defer cancel()
// Add trace ID ctx = context.WithValue(ctx, "traceID", "abc123")
// Pass to handler if err := handleRequest(ctx); err != nil { log.Fatal(err) } }
func handleRequest(ctx context.Context) error { // Derive with additional timeout ctx, cancel := context.WithTimeout(ctx, 10*time.Second) defer cancel()
// Pass to workers return processInParallel(ctx) }
func processInParallel(ctx context.Context) error { results := make(chan error, 3)
// All workers share same context for i := 0; i < 3; i++ { go func(id int) { results <- doWork(ctx, id) }(i) }
// Wait for all or cancellation for i := 0; i < 3; i++ { select { case err := <-results: if err != nil { return err } case <-ctx.Done(): return ctx.Err() } }
return nil } ```
Step 3: Handle Blocking Operations with Context
```go package main
import ( "context" "database/sql" "net/http" "time" )
// BAD: Blocking operation ignores context func fetchData(ctx context.Context, url string) ([]byte, error) { resp, err := http.Get(url) // No context support! if err != nil { return nil, err } defer resp.Body.Close() return io.ReadAll(resp.Body) }
// GOOD: Use context-aware HTTP client func fetchData(ctx context.Context, url string) ([]byte, error) { req, err := http.NewRequestWithContext(ctx, "GET", url, nil) if err != nil { return nil, err }
resp, err := http.DefaultClient.Do(req) if err != nil { return nil, err } defer resp.Body.Close()
return io.ReadAll(resp.Body) }
// GOOD: Database with context func queryDatabase(ctx context.Context, db *sql.DB) error { // Use context-aware query rows, err := db.QueryContext(ctx, "SELECT * FROM users") if err != nil { return err } defer rows.Close()
for rows.Next() { if ctx.Err() != nil { return ctx.Err() } // Process row } return nil }
// GOOD: Channel operations with context func processChannel(ctx context.Context, input <-chan Data, output chan<- Result) { for { select { case <-ctx.Done(): // Drain input channel to prevent goroutine leak go func() { for range input {} }() return
case data, ok := <-input: if !ok { return }
// Non-blocking send with context check select { case output <- process(data): case <-ctx.Done(): return } } } }
// GOOD: Long-running operation with periodic checks func longProcess(ctx context.Context) error { for i := 0; i < 1000000; i++ { // Check every 100 iterations if i%100 == 0 { select { case <-ctx.Done(): return ctx.Err() default: } }
// Do work doStep(i) } return nil } ```
Step 4: Fix HTTP Server Context Propagation
```go package main
import ( "context" "net/http" "time" )
// BAD: Handler doesn't respect request context func handler(w http.ResponseWriter, r *http.Request) { // Ignoring r.Context()! result := slowOperation() // Not cancelable w.Write(result) }
// GOOD: Use request context func handler(w http.ResponseWriter, r *http.Request) { ctx := r.Context()
result, err := slowOperationWithContext(ctx) if err != nil { if ctx.Err() == context.Canceled { // Client disconnected log.Println("Request canceled by client") return } http.Error(w, err.Error(), http.StatusInternalServerError) return }
w.Write(result) }
// GOOD: Graceful shutdown with context func main() { server := &http.Server{ Addr: ":8080", Handler: http.HandlerFunc(handler), }
// Start server in goroutine go func() { if err := server.ListenAndServe(); err != http.ErrServerClosed { log.Fatal(err) } }()
// Wait for interrupt signal quit := make(chan os.Signal, 1) signal.Notify(quit, os.Interrupt, syscall.SIGTERM) <-quit
// Create shutdown context ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second) defer cancel()
// Shutdown propagates cancellation to all handlers if err := server.Shutdown(ctx); err != nil { log.Fatal("Server shutdown error:", err) }
log.Println("Server stopped") }
// GOOD: Propagate context through middleware func contextMiddleware(next http.Handler) http.Handler { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { // Add timeout to context ctx, cancel := context.WithTimeout(r.Context(), 30*time.Second) defer cancel()
// Add values to context ctx = context.WithValue(ctx, "requestID", generateID())
// Call next handler with updated context next.ServeHTTP(w, r.WithContext(ctx)) }) } ```
Step 5: Debug Context Propagation Issues
```go package main
import ( "context" "log" "runtime" "runtime/pprof" "time" )
// Debug wrapper for context func debugContext(ctx context.Context, name string) context.Context { ctx, cancel := context.WithCancel(ctx)
go func() { <-ctx.Done() log.Printf("[%s] Context canceled: %v", name, ctx.Err()) }()
// Store cancel for debugging return context.WithValue(ctx, "cancel", cancel) }
// Check goroutine leaks func checkGoroutineLeaks() { before := runtime.NumGoroutine()
// Run your code here runTests()
time.Sleep(100 * time.Millisecond) // Let goroutines finish
after := runtime.NumGoroutine() if after > before { log.Printf("Potential goroutine leak: before=%d, after=%d", before, after)
// Print goroutine stack traces buf := make([]byte, 1<<20) n := runtime.Stack(buf, true) log.Printf("Goroutine dump:\n%s", buf[:n]) } }
// Use pprof to find leaks func startPprof() { go func() { log.Println(http.ListenAndServe("localhost:6060", nil)) }() }
// Debug context tree func printContextTree(ctx context.Context, indent int) { prefix := "" for i := 0; i < indent; i++ { prefix += " " }
log.Printf("%sContext type: %T", prefix, ctx) log.Printf("%sContext err: %v", prefix, ctx.Err())
// Check for deadline if deadline, ok := ctx.Deadline(); ok { log.Printf("%sDeadline: %v", prefix, deadline) }
// Check for values // Note: Standard library doesn't expose parent, but can use custom wrapper }
// Example test for context propagation func TestContextCancellation(t *testing.T) { ctx, cancel := context.WithCancel(context.Background())
done := make(chan struct{}) go func() { defer close(done) worker(ctx) }()
// Give worker time to start time.Sleep(100 * time.Millisecond)
// Cancel context cancel()
// Wait for worker to finish select { case <-done: // Good, worker stopped case <-time.After(time.Second): t.Error("Worker did not stop after context cancellation") } } ```
Step 6: Handle Context in Concurrent Patterns
```go package main
import ( "context" "sync" "time" )
// BAD: Goroutine leak when context canceled func processConcurrent(ctx context.Context, items []Item) error { var wg sync.WaitGroup
for _, item := range items { wg.Add(1) go func(i Item) { defer wg.Done() processItem(i) // Runs even after context canceled }(item) }
wg.Wait() return nil }
// GOOD: Early return on context cancellation func processConcurrent(ctx context.Context, items []Item) error { ctx, cancel := context.WithCancel(ctx) defer cancel()
var wg sync.WaitGroup errCh := make(chan error, len(items))
for _, item := range items { wg.Add(1) go func(i Item) { defer wg.Done()
// Check context before work if ctx.Err() != nil { return }
if err := processItemWithContext(ctx, i); err != nil { select { case errCh <- err: cancel() // Cancel all other workers default: } } }(item) }
// Wait in goroutine go func() { wg.Wait() close(errCh) }()
select { case err := <-errCh: return err case <-ctx.Done(): return ctx.Err() } }
// GOOD: Worker pool with context func workerPool(ctx context.Context, jobs <-chan Job, results chan<- Result, workers int) error { var wg sync.WaitGroup wg.Add(workers)
for i := 0; i < workers; i++ { go func() { defer wg.Done() for { select { case <-ctx.Done(): return case job, ok := <-jobs: if !ok { return }
select { case results <- processJob(ctx, job): case <-ctx.Done(): return } } } }() }
// Wait for workers go func() { wg.Wait() close(results) }()
return nil }
// GOOD: Pipeline with context propagation func pipeline(ctx context.Context, input <-chan Data) <-chan Result { output := make(chan Result)
go func() { defer close(output)
for { select { case <-ctx.Done(): return case data, ok := <-input: if !ok { return }
result := transform(ctx, data)
select { case output <- result: case <-ctx.Done(): return } } } }()
return output } ```
Step 7: Use Context with External Dependencies
```go package main
import ( "context" "database/sql" "github.com/redis/go-redis/v9" "go.mongodb.org/mongo-driver/mongo" "go.mongodb.org/mongo-driver/mongo/options" "time" )
// GOOD: Redis with context func redisWithCtx(ctx context.Context) error { rdb := redis.NewClient(&redis.Options{ Addr: "localhost:6379", })
// All operations take context val, err := rdb.Get(ctx, "key").Result() if err != nil { return err }
// With timeout ctx, cancel := context.WithTimeout(ctx, 5*time.Second) defer cancel()
return rdb.Set(ctx, "key", "value", 0).Err() }
// GOOD: MongoDB with context func mongoWithCtx(ctx context.Context) error { ctx, cancel := context.WithTimeout(ctx, 10*time.Second) defer cancel()
client, err := mongo.Connect(ctx, options.Client().ApplyURI("mongodb://localhost:27017")) if err != nil { return err } defer client.Disconnect(ctx)
collection := client.Database("test").Collection("users")
// Find with context cursor, err := collection.Find(ctx, bson.M{}) if err != nil { return err } defer cursor.Close(ctx)
for cursor.Next(ctx) { // Process document }
return cursor.Err() }
// GOOD: gRPC with context func grpcWithCtx(ctx context.Context, client pb.UserServiceClient) error { // Set timeout on context ctx, cancel := context.WithTimeout(ctx, 5*time.Second) defer cancel()
// Call with context resp, err := client.GetUser(ctx, &pb.GetUserRequest{Id: "123"}) if err != nil { return err }
log.Printf("User: %v", resp) return nil }
// GOOD: Kafka with context func kafkaConsumer(ctx context.Context, r *kafka.Reader) error { for { select { case <-ctx.Done(): return ctx.Err() default: // Read with context msg, err := r.ReadMessage(ctx) if err != nil { if ctx.Err() != nil { return ctx.Err() } return err }
// Process message processMessage(ctx, msg) } } } ```
Step 8: Implement Context Timeout Patterns
```go package main
import ( "context" "errors" "time" )
// GOOD: Timeout with fallback func fetchWithTimeout(ctx context.Context, url string) ([]byte, error) { ctx, cancel := context.WithTimeout(ctx, 5*time.Second) defer cancel()
result, err := fetchData(ctx, url) if errors.Is(err, context.DeadlineExceeded) { // Fallback to cache return getFromCache(url) } return result, err }
// GOOD: Timeout different operations differently func complexOperation(ctx context.Context) error { // Auth: 2 seconds authCtx, authCancel := context.WithTimeout(ctx, 2*time.Second) defer authCancel()
if err := authenticate(authCtx); err != nil { return err }
// DB: 5 seconds dbCtx, dbCancel := context.WithTimeout(ctx, 5*time.Second) defer dbCancel()
data, err := queryDatabase(dbCtx) if err != nil { return err }
// External API: 3 seconds apiCtx, apiCancel := context.WithTimeout(ctx, 3*time.Second) defer apiCancel()
return callExternalAPI(apiCtx, data) }
// GOOD: Propagate deadline if shorter func withDeadlineFromParent(ctx context.Context, timeout time.Duration) (context.Context, context.CancelFunc) { if deadline, ok := ctx.Deadline(); ok { // If parent has sooner deadline, use it if time.Until(deadline) < timeout { return context.WithDeadline(ctx, deadline) } } return context.WithTimeout(ctx, timeout) }
// GOOD: Circuit breaker with context func withCircuitBreaker(ctx context.Context, name string, fn func(context.Context) error) error { cb := getCircuitBreaker(name)
if !cb.Ready() { return errors.New("circuit breaker open") }
start := time.Now() err := fn(ctx) duration := time.Since(start)
if err != nil { cb.RecordFailure(duration) } else { cb.RecordSuccess(duration) }
return err } ```
Step 9: Test Context Propagation
```go package main
import ( "context" "testing" "time" )
// Test context cancellation propagates func TestContextCancellation(t *testing.T) { ctx, cancel := context.WithCancel(context.Background())
// Start worker done := make(chan struct{}) go func() { defer close(done) worker(ctx) }()
// Wait for worker to start time.Sleep(100 * time.Millisecond)
// Cancel context cancel()
// Verify worker stops select { case <-done: // Success case <-time.After(time.Second): t.Error("Worker did not stop after context cancellation") } }
// Test deadline propagation func TestDeadlinePropagation(t *testing.T) { ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second) defer cancel()
start := time.Now() err := slowOperation(ctx) duration := time.Since(start)
if err != context.DeadlineExceeded { t.Errorf("expected DeadlineExceeded, got %v", err) }
if duration > 3*time.Second { t.Errorf("operation took too long: %v", duration) } }
// Test context value propagation func TestContextValuePropagation(t *testing.T) { ctx := context.Background() ctx = context.WithValue(ctx, "traceID", "abc123") ctx = context.WithValue(ctx, "userID", "user456")
// Derive new context ctx, cancel := context.WithTimeout(ctx, time.Second) defer cancel()
// Values should propagate if traceID, ok := ctx.Value("traceID").(string); !ok || traceID != "abc123" { t.Error("traceID not propagated") }
if userID, ok := ctx.Value("userID").(string); !ok || userID != "user456" { t.Error("userID not propagated") } }
// Benchmark context operations func BenchmarkContextCancel(b *testing.B) { for i := 0; i < b.N; i++ { ctx, cancel := context.WithCancel(context.Background()) cancel() _ = ctx } }
func BenchmarkContextSelect(b *testing.B) { ctx, cancel := context.WithCancel(context.Background()) defer cancel()
b.ResetTimer() for i := 0; i < b.N; i++ { select { case <-ctx.Done(): default: } } } ```
Step 10: Implement Production Context Management
```go package main
import ( "context" "log" "os" "os/signal" "runtime/debug" "syscall" "time" )
// Context manager for application type ContextManager struct { rootCtx context.Context rootCancel context.CancelFunc }
func NewContextManager() *ContextManager { ctx, cancel := context.WithCancel(context.Background())
// Handle signals go func() { sigCh := make(chan os.Signal, 1) signal.Notify(sigCh, os.Interrupt, syscall.SIGTERM) <-sigCh log.Println("Received shutdown signal") cancel() }()
return &ContextManager{ rootCtx: ctx, rootCancel: cancel, } }
func (cm *ContextManager) Context() context.Context { return cm.rootCtx }
func (cm *ContextManager) Shutdown() { cm.rootCancel() }
func (cm *ContextManager) WithTimeout(timeout time.Duration) (context.Context, context.CancelFunc) { return context.WithTimeout(cm.rootCtx, timeout) }
func (cm *ContextManager) WithValue(key, value interface{}) context.Context { return context.WithValue(cm.rootCtx, key, value) }
// Request context with all necessary values type RequestContext struct { TraceID string UserID string SpanID string Deadline time.Time }
func NewRequestContext(parent context.Context, req *http.Request) (context.Context, context.CancelFunc) { // Add timeout ctx, cancel := context.WithTimeout(parent, 30*time.Second)
// Add trace info traceID := req.Header.Get("X-Trace-ID") if traceID == "" { traceID = generateTraceID() } ctx = context.WithValue(ctx, "traceID", traceID)
// Add user info userID := req.Header.Get("X-User-ID") ctx = context.WithValue(ctx, "userID", userID)
// Add span for tracing spanID := generateSpanID() ctx = context.WithValue(ctx, "spanID", spanID)
return ctx, cancel }
// Helper to extract context values func GetTraceID(ctx context.Context) string { if v := ctx.Value("traceID"); v != nil { return v.(string) } return "" }
func GetUserID(ctx context.Context) string { if v := ctx.Value("userID"); v != nil { return v.(string) } return "" }
// Best practices summary: // 1. Always pass context as first parameter // 2. Don't store context in structs // 3. Derive from parent context, never create new chain // 4. Always call cancel function // 5. Check ctx.Done() in loops // 6. Use context-aware libraries // 7. Set appropriate timeouts // 8. Propagate context through all layers // 9. Test cancellation behavior // 10. Monitor goroutine leaks
// Example complete application: func main() { ctxMgr := NewContextManager() defer ctxMgr.Shutdown()
// Run application with context if err := run(ctxMgr.Context()); err != nil { log.Fatal(err) } }
func run(ctx context.Context) error { // Create HTTP server with context-aware handlers server := &http.Server{ Addr: ":8080", Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { // Derive request context from app context ctx, cancel := NewRequestContext(ctx, r) defer cancel()
// Handle with context handleRequest(ctx, w, r) }), }
// Run server in goroutine errCh := make(chan error, 1) go func() { if err := server.ListenAndServe(); err != http.ErrServerClosed { errCh <- err } }()
// Wait for shutdown or error select { case <-ctx.Done(): // Graceful shutdown shutdownCtx, cancel := context.WithTimeout(context.Background(), 30*time.Second) defer cancel() return server.Shutdown(shutdownCtx) case err := <-errCh: return err } } ```
Go Context Propagation Checklist
| Check | Pattern | Expected |
|---|---|---|
| Context checked | select <-ctx.Done() | In every loop |
| Context passed | func(ctx, ...) | First parameter |
| Context derived | WithTimeout/Cancel | From parent |
| Cancel called | defer cancel() | Always |
| HTTP uses context | RequestWithContext | Client.Do |
| DB uses context | QueryContext | All queries |
| Goroutines check | ctx.Done() case | In select |
Verify the Fix
```bash # After fixing context propagation:
# 1. Run tests go test -v -race ./... # All tests pass, no race conditions
# 2. Check for goroutine leaks go test -run TestGoroutineLeak # No leaks detected
# 3. Run with pprof go run main.go & curl http://localhost:6060/debug/pprof/goroutine?debug=1 # Reasonable number of goroutines
# 4. Load test ab -n 1000 -c 100 http://localhost:8080/ # Requests properly canceled
# 5. Check metrics # Request latency should match context timeout # Goroutine count should stay stable
# Compare before/after: # Before: Goroutine leak, requests hang, no cancellation # After: Clean shutdown, context propagates, goroutines stop ```
Related Issues
- [Fix Go Goroutine Leak](/articles/fix-go-goroutine-leak)
- [Fix Go Channel Deadlock](/articles/fix-go-channel-deadlock)
- [Fix Go HTTP Client Timeout](/articles/fix-go-http-client-timeout)