Introduction

filepath.Walk traverses a directory tree and calls a visitor function for each file. When it encounters a directory with permission denied (e.g., owned by root, no read permission), the default behavior depends on how the error is handled in the visitor function. Returning the error aborts the walk entirely. Swallowing the error silently skips the directory and all its contents. Either behavior can cause issues in production.

Symptoms

  • Walk completes but misses files in restricted directories
  • Walk aborts entirely on first permission error
  • No logging of skipped directories
  • Inconsistent behavior between local development and production
  • filepath.Walk returns error for directories with no execute permission

```go // WRONG - aborts entire walk on first permission error err := filepath.Walk("/data", func(path string, info os.FileInfo, err error) error { if err != nil { return err // Walk stops here! } process(path) return nil })

// WRONG - silently skips, no logging err := filepath.Walk("/data", func(path string, info os.FileInfo, err error) error { if err != nil { return nil // Silently skips this directory and all children } process(path) return nil }) ```

Common Causes

  • Walk starting from root / or a high-level directory
  • Production user lacks permissions for some subdirectories
  • Docker volume mounts with different ownership
  • Symlinks pointing to restricted directories
  • Mixed permission modes across directory tree

Step-by-Step Fix

  1. 1.Log permission errors and continue walking:
  2. 2.```go
  3. 3.err := filepath.Walk("/data", func(path string, info os.FileInfo, err error) error {
  4. 4.if err != nil {
  5. 5.if os.IsPermission(err) {
  6. 6.log.Printf("WARN: skipping %s: %v", path, err)
  7. 7.return filepath.SkipDir // Skip this dir, continue with siblings
  8. 8.}
  9. 9.return err // Other errors abort
  10. 10.}
  11. 11.process(path)
  12. 12.return nil
  13. 13.})
  14. 14.`
  15. 15.Use fs.WalkDir (Go 1.16+) for better performance:
  16. 16.```go
  17. 17.err := filepath.WalkDir("/data", func(path string, d fs.DirEntry, err error) error {
  18. 18.if err != nil {
  19. 19.if os.IsPermission(err) {
  20. 20.log.Printf("WARN: permission denied: %s", path)
  21. 21.return filepath.SkipDir
  22. 22.}
  23. 23.return err
  24. 24.}

if d.IsDir() { return nil // Continue into directory }

// Process file info, err := d.Info() if err != nil { log.Printf("WARN: cannot stat %s: %v", path, err) return nil }

if info.Size() > 0 { processFile(path) } return nil }) ```

  1. 1.Skip known restricted directories upfront:
  2. 2.```go
  3. 3.skipDirs := map[string]bool{
  4. 4."/proc": true,
  5. 5."/sys": true,
  6. 6."/dev": true,
  7. 7.}

err := filepath.WalkDir("/", func(path string, d fs.DirEntry, err error) error { if err != nil { if os.IsPermission(err) { return filepath.SkipDir } return err }

if d.IsDir() { if skipDirs[path] { return filepath.SkipDir } } return nil }) ```

  1. 1.Collect skipped paths for reporting:
  2. 2.```go
  3. 3.type Walker struct {
  4. 4.Skipped []string
  5. 5.mu sync.Mutex
  6. 6.}

func (w *Walker) Walk(root string) error { return filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error { if err != nil { if os.IsPermission(err) { w.mu.Lock() w.Skipped = append(w.Skipped, path) w.mu.Unlock() return filepath.SkipDir } return err } return nil }) }

// After walk walker := &Walker{} if err := walker.Walk("/data"); err != nil { log.Fatalf("Walk failed: %v", err) } if len(walker.Skipped) > 0 { log.Printf("Skipped %d directories due to permissions", len(walker.Skipped)) } ```

Prevention

  • Always handle the err parameter in walk visitor functions
  • Use os.IsPermission(err) to distinguish permission errors from other failures
  • Return filepath.SkipDir to skip a directory but continue the walk
  • Log all skipped paths for audit and debugging
  • Run walk operations with appropriate user permissions
  • Use fs.WalkDir instead of filepath.Walk for better performance (lazy stat calls)
  • Test walk behavior with restricted directories in CI