Introduction

Log4j2's RollingFileAppender rotates log files based on size or time triggers. During rotation, it renames the active log file and creates a new one. If another process (log shipper, backup agent, NFS client) holds a lock on the active file, the rename fails. This causes log data loss, application errors, or the appender to stop writing entirely.

Symptoms

  • WARN Unable to rename file app-2024-01-15.log to app-2024-01-15.log.1
  • ERROR Unable to move file /var/log/app.log to /var/log/app.log.1
  • Log file stops growing after rotation failure
  • java.io.IOException: The process cannot access the file because it is being used by another process (Windows)
  • Log rotation creates 0-byte files
bash
2024-01-15 10:30:00,123 main WARN RollingFileManager (/var/log/app/app.log)
java.nio.file.FileSystemException: /var/log/app/app.log -> /var/log/app/app-2024-01-15.log:
The process cannot access the file because another process has locked a portion of it
    at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:95)

Common Causes

  • Logstash/Filebeat tailing the active log file with a lock
  • Windows process holding file handle
  • NFS mount with file locking enabled
  • Backup software scanning log directory during rotation
  • Multiple application instances writing to the same log file

Step-by-Step Fix

  1. 1.Use DirectWriteRolloverStrategy to avoid file locking:
  2. 2.```xml
  3. 3.<?xml version="1.0" encoding="UTF-8"?>
  4. 4.<Configuration>
  5. 5.<Appenders>
  6. 6.<!-- DirectWriteRolloverStrategy writes directly to dated files -->
  7. 7.<!-- No rename operation = no file lock issues -->
  8. 8.<RollingFile name="RollingFile"
  9. 9.filePattern="/var/log/app/app-%d{yyyy-MM-dd-HH}-%i.log.gz">
  10. 10.<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
  11. 11.<Policies>
  12. 12.<SizeBasedTriggeringPolicy size="100 MB"/>
  13. 13.<TimeBasedTriggeringPolicy interval="1"/>
  14. 14.</Policies>
  15. 15.<DirectWriteRolloverStrategy maxFiles="100"/>
  16. 16.</RollingFile>
  17. 17.</Appenders>
  18. 18.<Loggers>
  19. 19.<Root level="info">
  20. 20.<AppenderRef ref="RollingFile"/>
  21. 21.</Root>
  22. 22.</Loggers>
  23. 23.</Configuration>
  24. 24.`
  25. 25.Configure file locking behavior:
  26. 26.```xml
  27. 27.<RollingFile name="RollingFile"
  28. 28.fileName="/var/log/app/app.log"
  29. 29.filePattern="/var/log/app/app-%d{yyyy-MM-dd}-%i.log.gz"
  30. 30.locking="false"> <!-- Disable file locking -->
  31. 31.<PatternLayout pattern="%d %p %c{1} - %m%n"/>
  32. 32.<Policies>
  33. 33.<SizeBasedTriggeringPolicy size="50MB"/>
  34. 34.</Policies>
  35. 35.<DefaultRolloverStrategy max="30"/>
  36. 36.</RollingFile>
  37. 37.`
  38. 38.Configure log shippers to not lock files:
  39. 39.```yaml
  40. 40.# Filebeat configuration - do not hold file locks
  41. 41.filebeat.inputs:
  42. 42.- type: log
  43. 43.enabled: true
  44. 44.paths:
  45. 45.- /var/log/app/*.log
  46. 46.close_inactive: 1m # Release file handle after 1min inactivity
  47. 47.close_eof: true # Close file at EOF
  48. 48.close_renamed: true # Close when file is renamed
  49. 49.close_removed: true # Close when file is removed
  50. 50.clean_removed: true # Remove registry info for removed files
  51. 51.`
  52. 52.Handle Windows file locking specifically:
  53. 53.```xml
  54. 54.<!-- On Windows, use immediateFlush to reduce lock time -->
  55. 55.<RollingFile name="RollingFile"
  56. 56.fileName="C:/logs/app.log"
  57. 57.filePattern="C:/logs/app-%d{yyyy-MM-dd}-%i.log.gz"
  58. 58.immediateFlush="true">
  59. 59.<PatternLayout pattern="%d %m%n"/>
  60. 60.<Policies>
  61. 61.<TimeBasedTriggeringPolicy interval="1"/>
  62. 62.</Policies>
  63. 63.</RollingFile>
  64. 64.`
  65. 65.Use syslog or socket appender as alternative:
  66. 66.```xml
  67. 67.<!-- Avoid file locking entirely by sending logs over network -->
  68. 68.<Syslog name="Syslog"
  69. 69.host="log-server.example.com"
  70. 70.port="514"
  71. 71.protocol="TCP"
  72. 72.facility="LOCAL0"
  73. 73.format="RFC5424"/>
  74. 74.`

Prevention

  • Use DirectWriteRolloverStrategy for environments with active log shippers
  • Configure log shippers (Filebeat, Fluentd) with close_* settings
  • Avoid NFS-mounted log directories in production
  • On Windows, ensure no other process scans the log directory during rotation
  • Monitor log rotation failures with a custom StatusListener:
  • ```java
  • StatusLogger.getLogger().registerListener(new StatusListener() {
  • @Override
  • public void log(StatusData data) {
  • if (data.getLevel().isMoreSpecificThan(Level.WARN)) {
  • // Alert on rotation failures
  • alertService.warn("Log rotation issue: " + data.getFormattedMessage());
  • }
  • }
  • });
  • `