Introduction

Cloudflare's rate limiting feature blocks requests that exceed a configured threshold within a time window. While effective against brute force attacks and scraping, overly aggressive rate limiting rules can block legitimate API consumers, webhook deliveries, CI/CD pipelines, and automated monitoring systems that make frequent requests from the same IP address. The blocked request returns HTTP 429 (Too Many Requests) with a Cloudflare challenge page.

Symptoms

  • Legitimate API requests return HTTP 429 with Cloudflare rate limit page
  • Webhook deliveries from trusted services (GitHub, Stripe) are blocked
  • CI/CD pipeline fails intermittently with rate limit errors
  • curl to API endpoint works occasionally but fails under load
  • Cloudflare dashboard shows rate limit events for legitimate user IPs

Common Causes

  • Rate limit threshold too low for the API's normal usage pattern
  • Rate limit applied per-IP without considering shared corporate NAT IPs
  • API endpoint not excluded from rate limiting rules
  • Webhook source IPs not whitelisted in rate limit bypass rules
  • Rate limit window too short, not accounting for burst traffic patterns

Step-by-Step Fix

  1. 1.Identify which rate limit rule is blocking the traffic:
  2. 2.- In Cloudflare dashboard: Security > WAF > Rate limiting rules
  3. 3.- Check the rule that is triggering for the affected endpoint
  4. 4.- Note the threshold, window, and match criteria
  5. 5.Check rate limit event logs:
  6. 6.```bash
  7. 7.# Via API, check rate limit events
  8. 8.curl -s "https://api.cloudflare.com/client/v4/zones/ZONE_ID/rate_limits" \
  9. 9.-H "Authorization: Bearer API_TOKEN"
  10. 10.`
  11. 11.Increase the rate limit threshold for the API endpoint:
  12. 12.```bash
  13. 13.curl -X PUT "https://api.cloudflare.com/client/v4/zones/ZONE_ID/rate_limits/RULE_ID" \
  14. 14.-H "Authorization: Bearer API_TOKEN" \
  15. 15.-H "Content-Type: application/json" \
  16. 16.--data '{
  17. 17."threshold": 1000,
  18. 18."period": 60,
  19. 19."match": {
  20. 20."request": {
  21. 21."url": "*api.example.com/*"
  22. 22.}
  23. 23.},
  24. 24."action": {
  25. 25."mode": "simulate",
  26. 26."timeout": 60
  27. 27.}
  28. 28.}'
  29. 29.`
  30. 30.Whitelist trusted IP ranges:
  31. 31.```bash
  32. 32.# Create a WAF rule to skip rate limiting for known webhook sources
  33. 33.curl -X POST "https://api.cloudflare.com/client/v4/zones/ZONE_ID/firewall/rules" \
  34. 34.-H "Authorization: Bearer API_TOKEN" \
  35. 35.-H "Content-Type: application/json" \
  36. 36.--data '{
  37. 37."action": "skip",
  38. 38."expression": "(ip.src in {192.168.1.0/24 10.0.0.0/8}) and (http.request.uri.path contains \"/api/\")",
  39. 39."description": "Skip rate limiting for trusted API consumers"
  40. 40.}'
  41. 41.`
  42. 42.Use API tokens for identification instead of IP-based limiting:
  43. 43.Configure rate limiting based on API keys or authentication tokens rather than source IP, which is more accurate for multi-tenant environments.
  44. 44.Monitor rate limit hits after adjustment:
  45. 45.```bash
  46. 46.# Check analytics for rate limit events
  47. 47.curl -s "https://api.cloudflare.com/client/v4/zones/ZONE_ID/analytics/dashboard" \
  48. 48.-H "Authorization: Bearer API_TOKEN"
  49. 49.`

Prevention

  • Set rate limit thresholds based on actual API usage patterns, not arbitrary values
  • Use Cloudflare's simulate mode to test rate limits before enforcing them
  • Maintain a whitelist of known webhook source IPs and trusted API consumers
  • Implement application-level rate limiting in addition to Cloudflare's edge limiting
  • Monitor rate limit 429 responses as part of API health monitoring