Introduction

GitLab runners using the Docker executor with network_mode = "host" share the host's network namespace. While this provides direct network access, it can cause DNS resolution failures because the Docker container inherits the host's /etc/resolv.conf, which may not be compatible with the container's DNS resolver or may point to Docker's internal DNS that is not available in host mode.

Symptoms

  • CI/CD jobs fail with Could not resolve host or DNS lookup failed
  • Jobs work with bridge network mode but fail with host network mode
  • ping by IP address works but ping by hostname fails
  • Container cannot reach internal services by hostname
  • Error message: curl: (6) Could not resolve host: registry.internal.example.com

Common Causes

  • Host's /etc/resolv.conf points to Docker's internal DNS (127.0.0.11) which is not available in host mode
  • Host DNS configuration uses systemd-resolved with stub resolver on 127.0.0.53
  • Container does not have access to the host's DNS resolver in host network mode
  • DNS search domains not inherited correctly in host mode
  • Host firewall blocking DNS queries from container processes

Step-by-Step Fix

  1. 1.Check the DNS configuration inside the container: Identify the resolver.
  2. 2.```yaml
  3. 3.# .gitlab-ci.yml
  4. 4.test-dns:
  5. 5.image: alpine
  6. 6.network_mode: host
  7. 7.script:
  8. 8.- cat /etc/resolv.conf
  9. 9.- nslookup registry.internal.example.com
  10. 10.- ping -c 1 8.8.8.8 # Test IP connectivity
  11. 11.`
  12. 12.Configure the runner to use a specific DNS server: Override the DNS settings.
  13. 13.```toml
  14. 14.# /etc/gitlab-runner/config.toml
  15. 15.[[runners]]
  16. 16.executor = "docker"
  17. 17.[runners.docker]
  18. 18.network_mode = "host"
  19. 19.dns = ["8.8.8.8", "1.1.1.1", "10.0.0.2"] # Add your internal DNS
  20. 20.`
  21. 21.Fix the host's resolv.conf for compatibility: Ensure the host DNS config works.
  22. 22.```bash
  23. 23.# Check current resolv.conf
  24. 24.cat /etc/resolv.conf
  25. 25.# If it points to 127.0.0.53 (systemd-resolved), add real DNS servers
  26. 26.echo "nameserver 10.0.0.2" > /etc/resolv.conf
  27. 27.echo "nameserver 8.8.8.8" >> /etc/resolv.conf
  28. 28.`
  29. 29.Use bridge mode with explicit port mapping as an alternative: Avoid host mode entirely.
  30. 30.```toml
  31. 31.# /etc/gitlab-runner/config.toml
  32. 32.[[runners]]
  33. 33.executor = "docker"
  34. 34.[runners.docker]
  35. 35.network_mode = "bridge"
  36. 36.# Map required ports explicitly
  37. 37.extra_hosts = ["registry.internal.example.com:10.0.0.5"]
  38. 38.`
  39. 39.Verify DNS resolution works in CI/CD jobs: Test the fix.
  40. 40.```yaml
  41. 41.test-dns:
  42. 42.image: alpine
  43. 43.script:
  44. 44.- nslookup registry.internal.example.com || exit 1
  45. 45.- curl -s https://registry.internal.example.com || exit 1
  46. 46.`

Prevention

  • Test DNS resolution in runner configuration before deploying to production
  • Use bridge network mode with explicit port mapping when possible
  • Configure explicit DNS servers in the runner config rather than relying on host defaults
  • Document network requirements for each CI/CD job in the pipeline configuration
  • Monitor DNS resolution failure rates in CI/CD pipeline metrics
  • Use extra_hosts in Docker runner config for services that do not have DNS entries