Introduction
Go memory leaks occur when memory that is no longer needed remains allocated and reachable, preventing the garbage collector from reclaiming it. Unlike languages without garbage collection, Go doesn't have traditional memory leaks where memory is lost; instead, Go has "goroutine leaks" and "unintentional references" where objects remain reachable through global variables, unclosed channels, stuck goroutines, or forgotten timers. Common causes include goroutines blocked on channel sends/receives waiting forever, background goroutines without exit conditions, tickers and timers not stopped, response bodies not closed, connections not released to pool, large objects stored in global cache without eviction, goroutines holding references to large objects preventing GC, and slice/array retention where small slice references large underlying array. The fix requires using profiling tools (pprof), understanding escape analysis, identifying root references keeping objects alive, and implementing proper resource cleanup patterns. This guide provides production-proven techniques for detecting and fixing memory leaks in Go applications.
Symptoms
- RSS memory grows continuously over time
- GC runs more frequently but reclaims little memory
- Heap size increases monotonically
- Goroutine count grows without bound
- Application slows down as GC overhead increases
- OOM killed in containerized deployments
- Memory profile shows growing heap objects
- Runtime.MemStats shows increasing HeapAlloc
- Application works initially but degrades over hours/days
- pprof heap profile shows accumulation of specific type
Common Causes
- Goroutine blocked forever on channel operation
- Background goroutine without exit/timeout
- time.Ticker or time.Timer not stopped
- HTTP response body not closed
- Database connection not released
- Global cache growing without bounds
- Slice referencing large underlying array
- Closure capturing large objects
- sync.WaitGroup not decremented
- Context without cancellation
- Finalizer preventing GC
Step-by-Step Fix
### 1. Diagnose memory issues
Check memory statistics:
```go // Add memory stats monitoring to application package main
import ( "fmt" "runtime" "time" )
func printMemStats() { var m runtime.MemStats runtime.ReadMemStats(&m)
fmt.Printf("Alloc = %v MiB", bToMb(m.Alloc)) fmt.Printf("\tTotalAlloc = %v MiB", bToMb(m.TotalAlloc)) fmt.Printf("\tSys = %v MiB", bToMb(m.Sys)) fmt.Printf("\tNumGC = %v\n", m.NumGC)
fmt.Printf("HeapAlloc = %v MiB", bToMb(m.HeapAlloc)) fmt.Printf("\tHeapSys = %v MiB", bToMb(m.HeapSys)) fmt.Printf("\tHeapObjects = %v\n", m.HeapObjects)
fmt.Printf("NumGoroutine = %v\n", runtime.NumGoroutine()) }
func bToMb(b uint64) uint64 { return b / 1024 / 1024 }
func main() { // Print memory stats every 5 seconds go func() { for range time.Tick(5 * time.Second) { printMemStats() } }()
// ... rest of application } ```
Enable pprof profiling:
```go // Add pprof endpoint to HTTP server package main
import ( "net/http" _ "net/http/pprof" // Import for side effects )
func main() { // pprof endpoints available at /debug/pprof/ // - /debug/pprof/heap // - /debug/pprof/goroutine // - /debug/pprof/allocs // - /debug/pprof/block // - /debug/pprof/mutex
go func() { http.ListenAndServe("localhost:6060", nil) }()
// ... rest of application }
// Access via browser or curl: // curl http://localhost:6060/debug/pprof/heap > heap.prof // curl http://localhost:6060/debug/pprof/goroutine > goroutine.prof ```
Analyze heap profile:
```bash # Capture heap profile go tool pprof http://localhost:6060/debug/pprof/heap
# Or save to file first curl http://localhost:6060/debug/pprof/heap > heap.prof go tool pprof heap.prof
# In pprof interactive mode: # top - Show top memory consumers # top10 - Show top 10 # list FunctionName - Show specific function # web - Generate visual graph (requires graphviz) # svg - Export as SVG
# Example output: # (pprof) top # Showing nodes accounting for 50MB, 90% of 55MB total # flat flat% sum% cum cum% # 30MB 54.55% 54.55% 30MB 54.55% main.processData # 20MB 36.36% 90.91% 20MB 36.36% main.cacheResults
# Compare profiles over time go tool pprof -base heap_old.prof heap_new.prof
# Show allocations by type go tool pprof -alloc_space http://localhost:6060/debug/pprof/heap
# Show inuse_space (current memory) vs alloc_space (total allocated) go tool pprof -inuse_space http://localhost:6060/debug/pprof/heap go tool pprof -alloc_space http://localhost:6060/debug/pprof/heap ```
Analyze goroutine profile:
```bash # Capture goroutine profile curl http://localhost:6060/debug/pprof/goroutine > goroutine.prof go tool pprof goroutine.prof
# Show goroutine stack traces (pprof) list main.worker (pprof) web
# Or get raw goroutine dump curl http://localhost:6060/debug/pprof/goroutine?debug=2
# Count goroutines by state curl http://localhost:6060/debug/pprof/goroutine?debug=1 | \ grep -E "goroutine [0-9]+" | wc -l
# Analyze blocked goroutines # Look for goroutines stuck in: # - chan send (blocked forever) # - chan receive (blocked forever) # - select with no default ```
### 2. Fix goroutine leaks
Goroutine blocked on channel:
```go // BAD: Goroutine blocks forever if channel not consumed func worker() { done := make(chan bool) go func() { // This goroutine blocks forever if done is never sent <-done }() // Missing: done <- true }
// BAD: Goroutine blocked on send with no receiver func producer() { ch := make(chan int) // Unbuffered channel go func() { ch <- 42 // Blocks forever if no receiver }() // Missing: <-ch }
// CORRECT: Use buffered channel or ensure receiver exists func workerFixed() { done := make(chan bool, 1) // Buffered go func() { <-done }() done <- true // Non-blocking send }
// CORRECT: Use context for cancellation func workerWithCtx(ctx context.Context) { go func() { select { case <-done: // Do work case <-ctx.Done(): // Exit on cancellation return } }() }
// CORRECT: Use select with default for non-blocking func producerFixed() { ch := make(chan int, 1) // Or use select select { case ch <- 42: // Sent successfully default: // No receiver, skip or handle differently } } ```
Goroutine without exit condition:
```go // BAD: Infinite loop without exit func startServer() { go func() { for { // Runs forever, no way to stop doWork() } }() }
// CORRECT: Use context for graceful shutdown func startServerWithCtx(ctx context.Context) { go func() { for { select { case <-ctx.Done(): // Graceful shutdown cleanup() return default: doWork() } } }() }
// Usage: ctx, cancel := context.WithCancel(context.Background()) startServerWithCtx(ctx)
// Later, to stop: cancel() // All goroutines receive ctx.Done() signal ```
Ticker not stopped:
```go // BAD: Ticker continues running func startPeriodicTask() { ticker := time.NewTicker(time.Second) go func() { for range ticker.C { doWork() } // Missing: ticker.Stop() }() }
// CORRECT: Stop ticker when done func startPeriodicTaskFixed(ctx context.Context) { ticker := time.NewTicker(time.Second) go func() { defer ticker.Stop() // Ensure ticker is stopped for { select { case <-ctx.Done(): return case <-ticker.C: doWork() } } }() }
// CORRECT: Use time.After for one-shot timers func doWithTimeout() error { select { case result := <-doWork(): return result case <-time.After(30 * time.Second): return errors.New("timeout") } } ```
### 3. Fix resource leaks
HTTP response body not closed:
```go // BAD: Response body not closed - memory leak! func fetchData(url string) ([]byte, error) { resp, err := http.Get(url) if err != nil { return nil, err } // Missing: resp.Body.Close() return io.ReadAll(resp.Body) }
// CORRECT: Always close response body func fetchDataFixed(url string) ([]byte, error) { resp, err := http.Get(url) if err != nil { return nil, err } defer resp.Body.Close() // Ensure body is closed return io.ReadAll(resp.Body) }
// CORRECT: Handle partial reads func fetchDataRobust(url string) ([]byte, error) { resp, err := http.Get(url) if err != nil { return nil, err } defer resp.Body.Close()
body, err := io.ReadAll(resp.Body) if err != nil { return nil, err } return body, nil }
// For streaming responses, close on function exit func streamData(url string) error { resp, err := http.Get(url) if err != nil { return err } defer resp.Body.Close()
// Stream processing decoder := json.NewDecoder(resp.Body) for { var item Item if err := decoder.Decode(&item); err == io.EOF { break } else if err != nil { return err } process(item) } return nil } ```
Database connection not released:
```go // BAD: Rows not closed - connection leak func queryUsers(db *sql.DB) ([]User, error) { rows, err := db.Query("SELECT * FROM users") if err != nil { return nil, err } // Missing: rows.Close()
var users []User for rows.Next() { var u User if err := rows.Scan(&u); err != nil { return nil, err } users = append(users, u) } return users, rows.Err() }
// CORRECT: Always close rows func queryUsersFixed(db *sql.DB) ([]User, error) { rows, err := db.Query("SELECT * FROM users") if err != nil { return nil, err } defer rows.Close() // Release connection back to pool
var users []User for rows.Next() { var u User if err := rows.Scan(&u); err != nil { return nil, err } users = append(users, u) } return users, rows.Err() }
// For single row queries func getUser(db *sql.DB, id int) (*User, error) { row := db.QueryRow("SELECT * FROM users WHERE id = ?", id) // No need to close QueryRow, but handle error properly var u User if err := row.Scan(&u.ID, &u.Name); err == sql.ErrNoRows { return nil, nil } else if err != nil { return nil, err } return &u, nil }
// Transactions - ensure commit or rollback func transfer(db *sql.DB, from, to, amount int) error { tx, err := db.Begin() if err != nil { return err }
// Ensure transaction is completed var committed bool defer func() { if !committed { tx.Rollback() // Rollback on panic or error } }()
// ... transaction logic
if err := tx.Commit(); err != nil { return err } committed = true return nil } ```
### 4. Fix cache and global variable leaks
Unbounded cache growth:
```go // BAD: Cache grows forever var cache = make(map[string][]byte)
func cacheResult(key string, data []byte) { cache[key] = data // Never removed, grows forever }
// CORRECT: Use LRU cache with size limit import "github.com/hashicorp/golang-lru"
var cache *lru.Cache
func init() { cache, _ = lru.New(1000) // Max 1000 items }
func cacheResult(key string, data []byte) { cache.Add(key, data) // Evicts oldest when full }
// CORRECT: Use sync.Map with cleanup for concurrent access type Cache struct { mu sync.RWMutex data map[string]*CacheItem }
type CacheItem struct { Value []byte ExpiresAt time.Time }
func (c *Cache) Get(key string) ([]byte, bool) { c.mu.RLock() defer c.mu.RUnlock()
item, ok := c.data[key] if !ok || time.Now().After(item.ExpiresAt) { return nil, false } return item.Value, true }
func (c *Cache) Set(key string, value []byte, ttl time.Duration) { c.mu.Lock() defer c.mu.Unlock()
c.data[key] = &CacheItem{ Value: value, ExpiresAt: time.Now().Add(ttl), } }
// Periodic cleanup func (c *Cache) StartCleanup(ctx context.Context, interval time.Duration) { go func() { ticker := time.NewTicker(interval) defer ticker.Stop()
for { select { case <-ctx.Done(): return case <-ticker.C: c.cleanup() } } }() }
func (c *Cache) cleanup() { c.mu.Lock() defer c.mu.Unlock()
now := time.Now() for key, item := range c.data { if now.After(item.ExpiresAt) { delete(c.data, key) } } } ```
Global variable retaining references:
```go // BAD: Global slice retains all data var allData [][]byte
func processData(data []byte) { // Copy to avoid retaining reference to caller's buffer copied := make([]byte, len(data)) copy(copied, data) allData = append(allData, copied) // Grows forever }
// CORRECT: Process and discard, don't retain func processDataFixed(data []byte) error { result := analyze(data) // Don't store data after processing return saveResult(result) }
// CORRECT: Use ring buffer for limited history import "container/ring"
var recentData *ring.Ring var dataMu sync.Mutex
func init() { recentData = ring.New(100) // Keep only last 100 items }
func storeRecent(data []byte) { dataMu.Lock() defer dataMu.Unlock()
copied := make([]byte, len(data)) copy(copied, data) recentData.Value = copied recentData = recentData.Next() // Advance ring } ```
### 5. Fix slice and array retention
Slice referencing large array:
```go // BAD: Small slice retains reference to large array func getFirstLine(file []byte) []byte { // This returns a slice that references the entire file // The whole file stays in memory even if we only need first line return bytes.SplitN(file, []byte("\n"), 2)[0] }
// CORRECT: Copy the slice to release reference to large array func getFirstLineFixed(file []byte) []byte { line, _, _ := bytes.Cut(file, []byte("\n")) // Copy to new allocation result := make([]byte, len(line)) copy(result, line) return result }
// Or use strings for text data func getFirstLineString(content string) string { // strings.Split creates new strings, no retention issue lines := strings.SplitN(content, "\n", 2) return lines[0] } ```
Preallocate correctly:
```go // BAD: Growing slice causes multiple allocations func collectResults(items []int) []int { var results []int // Starts with capacity 0 for _, item := range items { results = append(results, item) // Reallocates many times } return results }
// CORRECT: Preallocate with known size func collectResultsFixed(items []int) []int { results := make([]int, 0, len(items)) // Preallocate capacity for _, item := range items { results = append(results, item) } return results }
// For unknown size, use reasonable initial capacity func collectUnknown() []string { results := make([]string, 0, 100) // Guess initial capacity for item := range stream() { results = append(results, item) } return results } ```
### 6. Use sync.Pool for memory reuse
Basic sync.Pool usage:
```go // Pool for reusable byte buffers var bufferPool = sync.Pool{ New: func() interface{} { return make([]byte, 0, 4096) // 4KB buffer }, }
func processWithPool(data []byte) []byte { // Get buffer from pool buf := bufferPool.Get().([]byte) buf = buf[:0] // Reset length, keep capacity
// Return buffer to pool when done defer bufferPool.Put(buf)
// Use buffer buf = append(buf, data...) result := transform(buf)
// Copy result before returning (pool may reuse buffer) resultCopy := make([]byte, len(result)) copy(resultCopy, result) return resultCopy }
// For io.Writer operations func writeWithPool(w io.Writer, data []byte) error { buf := bufferPool.Get().([]byte) buf = buf[:0] defer bufferPool.Put(buf)
buf = append(buf, data...) _, err := w.Write(buf) return err } ```
Pool for structs:
```go // Pool for expensive-to-allocate structs type QueryResult struct { Rows [][]string Cols []string Count int }
var resultPool = sync.Pool{ New: func() interface{} { return &QueryResult{ Rows: make([][]string, 0, 100), Cols: make([]string, 0, 10), } }, }
func executeQuery(query string) (*QueryResult, error) { result := resultPool.Get().(*QueryResult) // Reset state result.Rows = result.Rows[:0] result.Cols = result.Cols[:0] result.Count = 0
// ... execute query ...
return result, nil }
// Caller must return to pool after use func useQuery() error { result, err := executeQuery("SELECT * FROM users") if err != nil { return err } defer resultPool.Put(result) // Return to pool
// Use result processResult(result) return nil } ```
### 7. Tune garbage collector
Adjust GC behavior:
```go // Set GOGC environment variable (default is 100) // Lower = more frequent GC, less memory // Higher = less frequent GC, more memory
// In code (use sparingly, prefer env var) import _ "runtime"
// Set via environment: // export GOGC=50 // More aggressive GC (50% of default) // export GOGC=200 // Less aggressive GC (2x of default)
// Check current setting func getGOGC() int { var debug struct { gogc int } // Use runtime/debug.ReadGCPercent() instead return debug.ReadGCPercent() }
// Programmatically adjust (runtime/debug package) import "runtime/debug"
func init() { // Set GC percentile (same as GOGC env var) old := debug.SetGCPercent(50) log.Printf("Changed GOGC from %d to 50", old) } ```
GC logging and monitoring:
```go // Enable GC logging // Run with: GODEBUG=gctrace=1 ./app
// Output format: // gc 1 @0.5s 0%: 0.10+41+0.021 ms clock, 0.40+26/44/59+0.085 ms cpu, 4->4->2 MB, 5 MB goal, 4 P
// Parse GC logs // gc N: GC number // @Ts: Time since start // 0%: CPU percentage // 0.10+41+0.021: wall-clock time (stop-the-world + assist + background) // 4->4->2: heap before, during, after GC // 5 MB goal: next GC target
// Monitor GC in application import ( "runtime" "runtime/debug" )
func monitorGC() { var stats runtime.MemStats var gcStats debug.GCStats
for range time.Tick(time.Minute) { runtime.ReadMemStats(&stats) debug.ReadGCStats(&gcStats)
log.Printf( "Heap: %d MB, GC: %d, LastGC: %v, Pause: %v", stats.HeapAlloc/1024/1024, stats.NumGC, gcStats.LastGC, gcStats.PauseTotal, ) } } ```
Set memory limits (Go 1.16+):
```go // Set soft memory limit // GC will run more aggressively when approaching limit
import "runtime/debug"
func init() { // Set 1GB soft limit debug.SetMemoryLimit(1024 * 1024 * 1024) }
// For containerized deployments // Set based on container memory limit // Leave room for non-heap memory (stack, code, etc.)
// Example: 2GB container -> 1.5GB heap limit // debug.SetMemoryLimit(int64(float64(containerLimit) * 0.75)) ```
Prevention
- Always close resources (response.Body, rows, files) with defer
- Use context.WithCancel for goroutine cancellation
- Stop tickers and timers when done
- Use bounded caches with eviction policies
- Copy small slices from large arrays to release references
- Use sync.Pool for frequently allocated objects
- Enable GODEBUG=gctrace=1 in production for GC monitoring
- Set memory limits appropriate for container size
- Profile memory regularly with pprof
- Test with load for extended periods to catch slow leaks
Related Errors
- **panic: runtime error: out of memory**: Heap exhausted
- **fatal error: too many goroutines**: Goroutine leak
- **context deadline exceeded**: Timeout from stuck operation
- **connection pool exhausted**: Resources not released
- **channel send on closed channel**: Incorrect channel lifecycle