Introduction

filepath.Walk() traverses directory trees but stops entirely when it encounters a permission error on any file or directory. Returning the error from the WalkFunc callback causes Walk to abort the entire traversal, which is problematic when scanning large directory trees that contain system directories, symlink targets, or files owned by other users. The fix requires handling permission errors in the callback and returning nil to continue the walk, while using filepath.SkipDir for directories that cannot be read.

Symptoms

bash
stat /private/var/folders/xyz: permission denied
# Walk returns the error and stops traversing

Or:

bash
lstat /System/Volumes/Data: operation not permitted
# Entire walk aborted on first permission error

Common Causes

  • WalkFunc returns errors: Any non-nil error from callback stops the walk
  • System directories without permission: macOS SIP, Linux root-only directories
  • Symlinks to inaccessible targets: Symlink points to directory without read permission
  • Running as non-root: User lacks permission for some directories in tree
  • Docker volume mounts: Mounted volumes with different ownership
  • Walk does not distinguish error types: All errors treated equally

Step-by-Step Fix

Step 1: Handle permission errors in WalkFunc

```go import ( "errors" "io/fs" "log" "os" "path/filepath" )

func walkWithPermissionHandling(root string) error { return filepath.Walk(root, func(path string, info os.FileInfo, err error) error { // Handle errors from stat/lstat if err != nil { if errors.Is(err, os.ErrPermission) { if info.IsDir() { log.Printf("Skipping inaccessible directory: %s", path) return filepath.SkipDir } log.Printf("Skipping inaccessible file: %s", path) return nil } return err // Return other errors to abort }

// Process file log.Printf("Found: %s (%d bytes)", path, info.Size()) return nil }) } ```

Step 2: Use filepath.WalkDir (Go 1.16+)

```go func walkDirWithPermissionHandling(root string) error { return filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error { if err != nil { if errors.Is(err, os.ErrPermission) { if d.IsDir() { log.Printf("Skipping directory (permission denied): %s", path) return filepath.SkipDir } return nil // Skip file, continue walk } return err }

// d.Type() is more efficient than os.FileInfo if d.Type().IsRegular() { processFile(path) }

return nil }) } ```

Step 3: Build a resilient file scanner

```go type FileScanner struct { Files []string Skipped []string Errors []error }

func (s *FileScanner) Walk(root string) error { return filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error { if err != nil { s.Skipped = append(s.Skipped, path) if errors.Is(err, os.ErrPermission) { if d != nil && d.IsDir() { return filepath.SkipDir } return nil } s.Errors = append(s.Errors, err) return err // Abort on non-permission errors }

if d.Type().IsRegular() { s.Files = append(s.Files, path) }

return nil }) }

// Usage scanner := &FileScanner{} if err := scanner.Walk("/"); err != nil { log.Printf("Walk failed: %v", err) } log.Printf("Found %d files, skipped %d paths, %d errors", len(scanner.Files), len(scanner.Skipped), len(scanner.Errors)) ```

Prevention

  • Use filepath.WalkDir instead of filepath.Walk (more efficient, Go 1.16+)
  • Always check the error parameter in WalkFunc before accessing info/DirEntry
  • Return filepath.SkipDir for inaccessible directories, nil for inaccessible files
  • Log skipped paths for audit purposes
  • Never ignore errors silently -- at minimum log permission denials
  • Use os.Stat to check permissions before walking if you know which dirs may be restricted
  • Test walk behavior on different OS platforms where permission models differ