Introduction

A panic in a gRPC handler crashes the entire Go process because gRPC does not recover from panics by default. Unlike HTTP handlers where recover() can catch panics, gRPC unary and stream handlers need explicit panic recovery middleware (interceptors) to catch panics, log the stack trace, and return a proper gRPC error to the client instead of crashing the server. Without recovery, a single nil pointer dereference in any handler takes down the entire service, affecting all concurrent clients.

Symptoms

``` panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1234567]

goroutine 42 [running]: main.(*server).GetUser(0x0, 0xc000123456, 0xc000234567) /app/server.go:45 +0x17 # Entire process exits -- all clients disconnected ```

Common Causes

  • Nil pointer dereference: Database query returns nil, handler does not check
  • Uninitialized dependencies: Server struct fields not set before serving
  • Type assertion panic: Type assertion without comma-ok idiom
  • Slice index out of range: Accessing slice with invalid index
  • Nil map write: Writing to uninitialized map
  • Panic in middleware: Interceptor or wrapper panics before handler

Step-by-Step Fix

Step 1: Implement panic recovery interceptor

```go import ( "context" "log" "runtime/debug"

"google.golang.org/grpc" "google.golang.org/grpc/codes" "google.golang.org/grpc/status" )

func PanicRecoveryInterceptor( ctx context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler, ) (interface{}, error) { defer func() { if r := recover(); r != nil { log.Printf("Panic recovered in %s: %v\n%s", info.FullMethod, r, debug.Stack())

// Return gRPC error instead of crashing // Do not expose panic details to client _ = r // r is used in log above } }()

return handler(ctx, req) }

func PanicRecoveryStreamInterceptor( srv interface{}, ss grpc.ServerStream, info *grpc.StreamServerInfo, handler grpc.StreamHandler, ) error { defer func() { if r := recover(); r != nil { log.Printf("Panic recovered in stream %s: %v\n%s", info.FullMethod, r, debug.Stack()) } }()

return handler(srv, ss) } ```

Step 2: Register interceptors with server

```go func NewGRPCServer() *grpc.Server { return grpc.NewServer( grpc.UnaryInterceptor(PanicRecoveryInterceptor), grpc.StreamInterceptor(PanicRecoveryStreamInterceptor), // Chain with other interceptors // grpc.ChainUnaryInterceptor(loggingInterceptor, PanicRecoveryInterceptor), ) }

func main() { s := NewGRPCServer() pb.RegisterUserServiceServer(s, &userServer{})

lis, _ := net.Listen("tcp", ":50051") s.Serve(lis) } ```

Step 3: Add contextual panic recovery in handlers

```go func (s *userServer) GetUser(ctx context.Context, req *pb.GetUserRequest) (*pb.UserResponse, error) { // Even with interceptor-level recovery, add handler-level safety defer func() { if r := recover(); r != nil { log.Printf("Handler panic: %v", r) } }()

// Safe database access user, err := s.db.GetUserByID(req.GetId()) if err != nil { if errors.Is(err, sql.ErrNoRows) { return nil, status.Errorf(codes.NotFound, "user not found: %d", req.GetId()) } return nil, status.Errorf(codes.Internal, "database error: %v", err) }

// Safe nil check if user == nil { return nil, status.Error(codes.NotFound, "user not found") }

return &pb.UserResponse{ Id: user.ID, Name: user.Name, Email: user.Email, }, nil } ```

Prevention

  • Always add panic recovery interceptors to gRPC servers in production
  • Use grpc.ChainUnaryInterceptor to combine recovery with logging and auth interceptors
  • Check all pointer returns from database queries and external calls before use
  • Use comma-ok idiom for type assertions: v, ok := x.(Type)
  • Initialize all dependencies before registering handlers
  • Add integration tests that trigger error paths to verify graceful degradation
  • Monitor panic rates with structured logging and alerting