Introduction

When a Go server's net.Listener is closed during graceful shutdown, the accept loop calling listener.Accept() returns an error. If this error is not handled correctly -- for example, by logging it as a fatal error or calling log.Fatal in the accept loop -- the application panics during what should be a clean shutdown sequence. This is a common issue in custom TCP/HTTP servers where the shutdown logic closes the listener but the accept gorine does not recognize the "use of closed network connection" error as a normal shutdown signal.

Symptoms

During shutdown:

``` 2024/03/15 10:23:45 accept tcp [::]:8080: use of closed network connection fatal error: accept loop failed

goroutine 17 [running]: main.(*Server).Start.func1() /app/server.go:45 +0x1a8 ```

Or with a panic:

``` panic: accept tcp [::]:8080: use of closed network connection

goroutine 17 [running]: main.(*Server).acceptLoop(0xc0000a2000) /app/server.go:52 +0x156 ```

The application does not exit cleanly and requires kill -9 to terminate.

Common Causes

  • Accept loop calls log.Fatal on any error: Not distinguishing between accept errors and normal close
  • Closing listener before signaling accept loop to stop: Race condition between listener.Close() and Accept()
  • No context-based shutdown signal: Accept goroutine has no way to know the listener was intentionally closed
  • Panic in defer during shutdown: Cleanup code panics because resources are already nil
  • Multiple goroutines closing the same listener: Duplicate close calls cause double-close errors

Step-by-Step Fix

Step 1: Recognize the closed listener error as normal

```go import ( "errors" "log" "net" "strings" )

func (s *Server) acceptLoop(listener net.Listener) { for { conn, err := listener.Accept() if err != nil { // Check if this is a normal shutdown error if isClosedListenerError(err) { log.Println("Listener closed, stopping accept loop") return } // Real error - log and continue accepting log.Printf("Accept error: %v", err) continue } go s.handleConnection(conn) } }

func isClosedListenerError(err error) bool { if err == nil { return false } // Go 1.16+ errors.Is approach if errors.Is(err, net.ErrClosed) { return true } // Fallback for older Go versions return strings.Contains(err.Error(), "use of closed network connection") } ```

Step 2: Use context for coordinated shutdown

```go func (s *Server) Start(ctx context.Context) error { listener, err := net.Listen("tcp", s.addr) if err != nil { return err } s.listener = listener

errCh := make(chan error, 1)

go func() { for { conn, err := listener.Accept() if err != nil { select { case <-ctx.Done(): return // Normal shutdown default: if !errors.Is(err, net.ErrClosed) { log.Printf("Accept error: %v", err) } return } } go s.handleConnection(conn) } }()

// Wait for context cancellation or accept error select { case <-ctx.Done(): return s.shutdown() case err := <-errCh: return err } }

func (s *Server) shutdown() error { if s.listener != nil { return s.listener.Close() } return nil } ```

Step 3: Graceful shutdown with in-flight connection tracking

```go import "sync"

type Server struct { listener net.Listener mu sync.Mutex conns map[net.Conn]struct{} closing bool }

func (s *Server) Start(ctx context.Context) error { listener, err := net.Listen("tcp", s.addr) if err != nil { return err } s.listener = listener s.conns = make(map[net.Conn]struct{})

for { conn, err := listener.Accept() if err != nil { if errors.Is(err, net.ErrClosed) { return nil } continue }

s.mu.Lock() if s.closing { s.mu.Unlock() conn.Close() continue } s.conns[conn] = struct{}{} s.mu.Unlock()

go func() { s.handleConnection(conn) s.mu.Lock() delete(s.conns, conn) s.mu.Unlock() }() } }

func (s *Server) Shutdown(ctx context.Context) error { s.mu.Lock() s.closing = true s.listener.Close() // This will cause Accept to return net.ErrClosed s.mu.Unlock()

// Wait for in-flight connections to finish deadline := time.After(15 * time.Second) for { select { case <-deadline: // Force close remaining connections s.mu.Lock() for conn := range s.conns { conn.Close() } s.mu.Unlock() return errors.New("shutdown timeout exceeded") default: s.mu.Lock() if len(s.conns) == 0 { s.mu.Unlock() return nil } s.mu.Unlock() time.Sleep(100 * time.Millisecond) } } } ```

Prevention

  • Always check errors.Is(err, net.ErrClosed) in accept loops
  • Use context-based shutdown coordination rather than relying on error strings
  • Track in-flight connections and wait for them to complete during shutdown
  • Never call log.Fatal in background goroutines -- use channel-based error reporting
  • Add a shutdown test that starts the server, sends requests, and verifies clean shutdown
  • Use go test -race to detect race conditions during shutdown sequencing