Introduction

DNS split-horizon (also called split-brain DNS) serves different DNS responses based on the source of the query. Internal users receive private IP addresses for services, while external users receive public IPs. When the two views are not kept in sync - for example, a new service is added to the external DNS but not the internal zone, or vice versa - users on one side cannot access services that exist on the other side.

Symptoms

  • Internal users cannot reach a service that works from external networks
  • dig app.example.com returns different IPs from internal vs external DNS
  • Service added externally but internal DNS still points to an old decommissioned server
  • VPN users cannot access internal services because they are using external DNS
  • Internal DNS returns NXDOMAIN for a domain that resolves externally

Common Causes

  • Separate DNS zones for internal and external not updated together
  • Internal zone file manually maintained while external uses a DNS provider API
  • New subdomain added to external DNS provider but not to internal BIND/PowerDNS
  • VPN clients receiving external DNS resolver instead of internal DNS
  • DNS forwarding rules misconfigured, sending internal queries to external resolvers

Step-by-Step Fix

  1. 1.Compare internal vs external DNS responses:
  2. 2.```bash
  3. 3.# Query internal DNS
  4. 4.dig app.example.com @internal-dns +short
  5. 5.# Query external DNS
  6. 6.dig app.example.com @8.8.8.8 +short
  7. 7.# Compare the results
  8. 8.`
  9. 9.Check which DNS server the client is using:
  10. 10.```bash
  11. 11.cat /etc/resolv.conf
  12. 12.# Or on Windows:
  13. 13.ipconfig /all | grep "DNS Servers"
  14. 14.`
  15. 15.Identify missing records in the internal zone:
  16. 16.```bash
  17. 17.# List all records from external DNS
  18. 18.dig example.com AXFR @external-ns 2>/dev/null || \
  19. 19.dig example.com ANY +noall +answer @8.8.8.8

# Compare with internal zone dig example.com ANY +noall +answer @internal-dns ```

  1. 1.Add missing records to the internal DNS zone:
  2. 2.```bash
  3. 3.# For BIND, edit the zone file:
  4. 4.sudo nano /etc/bind/zones/example.com.internal.zone
  5. 5.# Add the missing record:
  6. 6.app IN A 10.0.1.50
  7. 7.# Reload BIND:
  8. 8.sudo rndc reload example.com
  9. 9.`
  10. 10.For PowerDNS, add the record via the API:
  11. 11.```bash
  12. 12.curl -X PATCH "http://localhost:8081/api/v1/servers/localhost/zones/example.com" \
  13. 13.-H "X-API-Key: secret" \
  14. 14.-d '{
  15. 15."rrsets": [{
  16. 16."name": "app.example.com.",
  17. 17."type": "A",
  18. 18."ttl": 300,
  19. 19."changetype": "REPLACE",
  20. 20."records": [{"content": "10.0.1.50", "disabled": false}]
  21. 21.}]
  22. 22.}'
  23. 23.`
  24. 24.Configure conditional forwarding for domains you do not host internally:
  25. 25.```bash
  26. 26.# In BIND named.conf:
  27. 27.zone "partner-company.com" {
  28. 28.type forward;
  29. 29.forwarders { 8.8.8.8; 8.8.4.4; };
  30. 30.};
  31. 31.`

Prevention

  • Use a single DNS management system that can serve both internal and external views
  • Automate DNS record synchronization between internal and external zones
  • Implement DNS change management processes that update both views simultaneously
  • Monitor DNS resolution from both internal and external networks
  • Document the split-horizon DNS architecture and record ownership in runbooks