Introduction

The defer statement schedules function calls to run when the enclosing function returns, not when the enclosing block ends. When defer is used inside a loop or conditional, resources (files, connections, locks) are not released until the entire function returns. In long-running functions or loops processing large datasets, this causes file descriptor exhaustion, connection pool depletion, and memory leaks.

Symptoms

  • too many open files after processing many files
  • dial tcp: too many open connections in database operations
  • Memory usage grows linearly with loop iterations
  • defer stack grows and delays resource cleanup
  • Works for small inputs, fails for large datasets
go
// WRONG - all files stay open until function returns
func processFiles(files []string) error {
    for _, f := range files {
        file, err := os.Open(f)
        if err != nil { return err }
        defer file.Close()  // NOT closed until ALL files processed!
        data, _ := io.ReadAll(file)
        process(data)
    }
    return nil
}
// Processing 10,000 files = 10,000 open file descriptors simultaneously

Common Causes

  • defer inside a loop (file handles, HTTP responses, DB connections)
  • Early return paths that skip resource cleanup
  • Deferring in a function that runs for a long time
  • Multiple resources deferred in sequence, all held until function end
  • Mixing defer with explicit cleanup in same function

Step-by-Step Fix

  1. 1.Use anonymous function for per-iteration cleanup:
  2. 2.```go
  3. 3.// CORRECT - file closed at end of each iteration
  4. 4.func processFiles(files []string) error {
  5. 5.for _, f := range files {
  6. 6.err := func() error {
  7. 7.file, err := os.Open(f)
  8. 8.if err != nil { return err }
  9. 9.defer file.Close() // Closed when anonymous function returns

data, err := io.ReadAll(file) if err != nil { return err } return process(data) }() if err != nil { return err } } return nil } ```

  1. 1.Explicit close instead of defer in loops:
  2. 2.```go
  3. 3.// CORRECT - explicit close, no defer needed
  4. 4.func processFiles(files []string) error {
  5. 5.for _, f := range files {
  6. 6.file, err := os.Open(f)
  7. 7.if err != nil { return err }

data, err := io.ReadAll(file) file.Close() // Explicit close immediately after reading if err != nil { return err }

process(data) } return nil } ```

  1. 1.Handle conditional return paths:
  2. 2.```go
  3. 3.// WRONG - response body leaked on error path
  4. 4.func fetchAndProcess(url string) error {
  5. 5.resp, err := http.Get(url)
  6. 6.if err != nil { return err }
  7. 7.// If next line errors, resp.Body never closed

data, err := io.ReadAll(resp.Body) if err != nil { return err } // LEAK: body not closed

defer resp.Body.Close() // Too late - we already might have returned return process(data) }

// CORRECT - defer immediately after error check func fetchAndProcess(url string) error { resp, err := http.Get(url) if err != nil { return err } defer resp.Body.Close() // Immediately deferred

data, err := io.ReadAll(resp.Body) if err != nil { return err } // Body closed by defer

return process(data) } ```

  1. 1.Process files with filepath.Walk and proper cleanup:
  2. 2.```go
  3. 3.func processDir(root string) error {
  4. 4.return filepath.Walk(root, func(path string, info os.FileInfo, err error) error {
  5. 5.if err != nil { return err }
  6. 6.if info.IsDir() { return nil }

// Process each file in its own scope return processSingleFile(path) }) }

func processSingleFile(path string) error { f, err := os.Open(path) if err != nil { return err } defer f.Close() // Safe - function is called once per file

data, err := io.ReadAll(f) if err != nil { return err } return handle(data) } ```

Prevention

  • Never use defer inside loops - use anonymous functions or explicit close
  • Use filepath.Walk or os.ReadDir with per-file function calls
  • Run go vet which detects some defer-in-loop patterns
  • Monitor file descriptor count: lsof -p <pid> | wc -l
  • Use ulimit -n to set lower limits during testing to catch leaks early
  • In HTTP handlers, always defer resp.Body.Close() right after the nil check