Introduction

Let's Encrypt validates domain ownership by placing a challenge file in the webroot directory and requesting it via HTTP. If the challenge file is not accessible -- due to incorrect webroot path, .htaccess rules blocking access, or the .well-known/acme-challenge/ directory not existing -- the renewal fails and the SSL certificate eventually expires.

Symptoms

  • Let's Encrypt renewal fails with Challenge failed for domain
  • Certbot logs show Invalid response from http://domain/.well-known/acme-challenge/
  • SSL certificate expires because automatic renewal did not complete
  • Website shows SSL certificate expired warning in browser
  • Error message: urn:ietf:params:acme:error:unauthorized :: The key authorization file from the server did not match this challenge

Common Causes

  • Webroot path configured incorrectly in certbot renewal configuration
  • .htaccess file redirecting or blocking access to .well-known/acme-challenge/
  • .well-known directory does not exist or has incorrect permissions
  • Nginx/Apache configuration not serving files from the .well-known directory
  • CDN or reverse proxy not passing through the ACME challenge requests

Step-by-Step Fix

  1. 1.Check the certbot renewal configuration: Verify the webroot path.
  2. 2.```bash
  3. 3.cat /etc/letsencrypt/renewal/example.com.conf
  4. 4.# Check: authenticator = webroot
  5. 5.# Check: webroot_path = /var/www/html
  6. 6.`
  7. 7.Verify the challenge directory exists and is accessible: Create it if needed.
  8. 8.```bash
  9. 9.mkdir -p /var/www/html/.well-known/acme-challenge
  10. 10.chmod 755 /var/www/html/.well-known/acme-challenge
  11. 11.# Test accessibility
  12. 12.echo "test" > /var/www/html/.well-known/acme-challenge/test
  13. 13.curl http://example.com/.well-known/acme-challenge/test
  14. 14.rm /var/www/html/.well-known/acme-challenge/test
  15. 15.`
  16. 16.Fix .htaccess to allow ACME challenge access: Ensure challenge files are not blocked.
  17. 17.```apache
  18. 18.# Add to .htaccess in the webroot
  19. 19.<Directory "/.well-known/acme-challenge">
  20. 20.AllowOverride None
  21. 21.Require all granted
  22. 22.</Directory>
  23. 23.# Or add before any rewrite rules:
  24. 24.RewriteRule ^\.well-known/acme-challenge/ - [L]
  25. 25.`
  26. 26.Configure Nginx to serve challenge files: Add a location block.
  27. 27.```nginx
  28. 28.location /.well-known/acme-challenge/ {
  29. 29.root /var/www/html;
  30. 30.allow all;
  31. 31.}
  32. 32.`
  33. 33.Retry the certificate renewal: Run certbot manually.
  34. 34.```bash
  35. 35.certbot renew --force-renewal --dry-run
  36. 36.# If dry-run succeeds, run the actual renewal
  37. 37.certbot renew --force-renewal
  38. 38.`

Prevention

  • Test certbot renewal with --dry-run after any web server configuration change
  • Ensure the .well-known/acme-challenge/ directory is permanently created and accessible
  • Exclude the ACME challenge path from any redirect or rewrite rules
  • Monitor certificate expiration dates and alert 14 days before expiry
  • Configure automatic renewal via cron: 0 3 * * * certbot renew --quiet
  • Use webroot authentication consistently rather than standalone mode for production servers