Introduction

Google Cloud DNS provides managed DNS infrastructure integrated with Google Cloud Platform. It supports public zones, private zones for GCP networks, and DNSSEC signing. Google's global infrastructure offers high availability, but GCP-specific features like private zones, DNS peering, and forwarding policies create unique troubleshooting scenarios. Understanding how Cloud DNS interacts with VPC networking and Compute Engine is essential for effective diagnosis.

Symptoms

  • Cloud DNS changes not propagating
  • Private zone records not resolving in VPC
  • DNSSEC validation failures for Cloud DNS zones
  • DNS forwarding rules not working
  • DNS peering not functioning between VPCs
  • Records visible in GCP console but not in queries
  • External queries failing while internal work

Common Causes

  • Private zone not visible to expected VPC networks
  • DNS forwarding configuration issues
  • DNSSEC misconfiguration or chain issues
  • Record creation API errors not visible in console
  • Zone visibility settings incorrect
  • VPC DNS server configuration conflicts
  • Cloud DNS quotas exceeded

Step-by-Step Fix

  1. 1.Verify Cloud DNS zone configuration and visibility.

```bash # List Cloud DNS managed zones: gcloud dns managed-zones list

# Get specific zone details: gcloud dns managed-zones describe ZONE_NAME

# Check zone visibility: # visibility: public - visible to internet # visibility: private - only visible to specified VPCs

# For private zones, check network associations: gcloud dns managed-zones describe PRIVATE_ZONE \ --format="value(privateVisibilityConfig.networks)"

# If VPC not listed, private zone won't resolve there

# Test zone resolution from outside: dig @ns-cloud-a1.googledomains.com example.com A +short

# For private zone, external query should fail: dig @ns-cloud-a1.googledomains.com internal.example.com A # Should return REFUSED or no answer for private zone ```

  1. 1.Check Cloud DNS authoritative nameservers.

```bash # Google Cloud DNS nameservers follow pattern: # ns-cloud-a1.googledomains.com through ns-cloud-e4.googledomains.com

# Get zone nameservers: gcloud dns managed-zones describe ZONE_NAME --format="value(nameServers)"

# Test each nameserver: for ns in $(gcloud dns managed-zones describe ZONE_NAME --format="value(nameServers)"); do echo "Testing $ns:" dig @$ns example.com A +short | head -1 done

# All nameservers should return same records

# Compare authoritative with resolver: echo "Authoritative:" dig @ns-cloud-a1.googledomains.com example.com A +short echo "Public resolver:" dig @8.8.8.8 example.com A +short ```

  1. 1.Debug private zone visibility in VPC.

```bash # Private zones only resolve within associated VPCs

# Check zone associations: gcloud dns managed-zones describe INTERNAL_ZONE \ --format="yaml(privateVisibilityConfig)"

# Should list VPC networks

# Test from GCE instance in VPC: # SSH to instance gcloud compute ssh INSTANCE_NAME --zone=ZONE

# On instance: dig internal.example.com A +short # Should return records from private zone

# Check instance's DNS configuration: cat /etc/resolv.conf # Should show Google's internal DNS: 169.254.169.254

# Test internal DNS resolver: dig @169.254.169.254 internal.example.com A +short # Google's metadata server provides DNS in VPC

# If private zone not resolving: # 1. VPC not associated with zone # 2. Instance in wrong VPC # 3. Zone visibility settings misconfigured ```

  1. 1.Resolve DNS forwarding policy issues.

```bash # DNS forwarding allows VPC to resolve external nameservers

# Check forwarding zones: gcloud dns managed-zones list --filter="type=forwarding"

# Describe forwarding zone: gcloud dns managed-zones describe FORWARD_ZONE \ --format="yaml(forwardingConfig)"

# Check target nameservers: gcloud dns managed-zones describe FORWARD_ZONE \ --format="value(forwardingConfig.targetNameServers)"

# Test forwarding from VPC instance: gcloud compute ssh INSTANCE_NAME dig forwarded-domain.com A @169.254.169.254

# Common forwarding issues: # - Target nameserver IPs incorrect # - Firewall blocking DNS to target # - Forwarding config not applied to correct VPC

# Verify target reachable from VPC: ping TARGET_NAMESERVER_IP nc -vz TARGET_NAMESERVER_IP 53 ```

  1. 1.Fix DNS peering between VPC networks.

```bash # DNS peering allows one VPC to use another's DNS configuration

# Check peering zones: gcloud dns managed-zones list --filter="type=peering"

# Describe peering zone: gcloud dns managed-zones describe PEERING_ZONE \ --format="yaml(peeringConfig)"

# Verify: # - Target VPC network name # - Target project (if cross-project)

# Check VPC network peering: gcloud compute networks peerings list --network=SOURCE_VPC

# DNS peering requires: # - VPC network peering established # - DNS peering zone configured # - Target VPC has DNS configuration to peer with

# Test peering resolution: # From instance in source VPC: gcloud compute ssh INSTANCE_IN_SOURCE_VPC dig peered-domain.example.com A @169.254.169.254

# If fails: # - Check VPC peering status is "ACTIVE" # - Verify DNS peering zone exists # - Check target project permissions ```

  1. 1.Troubleshoot Cloud DNS record updates.

```bash # Check current records: gcloud dns record-sets list --zone=ZONE_NAME

# Get specific record: gcloud dns record-sets describe example.com --zone=ZONE_NAME --type=A

# Create/update records: gcloud dns record-sets create example.com --zone=ZONE_NAME \ --type=A --ttl=300 --rrdatas="192.0.2.1"

gcloud dns record-sets update example.com --zone=ZONE_NAME \ --type=A --ttl=300 --rrdatas="192.0.2.2"

# Check change status: gcloud dns changes list --zone=ZONE_NAME

# Get specific change: gcloud dns changes describe CHANGE_ID --zone=ZONE_NAME

# Status: pending -> done

# Cloud DNS changes propagate within seconds # But resolver TTL adds delay

# Force immediate verification: dig @ns-cloud-a1.googledomains.com example.com A +short ```

  1. 1.Debug Cloud DNS DNSSEC issues.

```bash # Check zone DNSSEC status: gcloud dns managed-zones describe ZONE_NAME \ --format="value(dnssecConfig.state)"

# States: off, on, transferring

# Check DNSSEC configuration: gcloud dns managed-zones describe ZONE_NAME \ --format="yaml(dnssecConfig)"

# Verify DNSKEY: dig @ns-cloud-a1.googledomains.com example.com DNSKEY +dnssec

# Check DS at parent: dig @a.gtld-servers.net example.com DS +short

# Common DNSSEC issues in Cloud DNS: # - DS not configured at registrar # - DNSSEC turned on but DS missing # - Algorithm mismatch

# Activate DNSSEC: gcloud dns managed-zones update ZONE_NAME --dnssec-state=on

# Get DS record to configure at registrar: gcloud dns dnskeys describe KEY_ID --zone=ZONE_NAME \ --format="value(dsRecord)"

# Submit DS to registrar ```

  1. 1.Check Cloud DNS quotas and limits.

```bash # View DNS quotas: gcloud compute project-info describe --format="yaml(quotas)"

# Key quotas: # - Managed zones per project # - Record sets per zone # - DNS queries per second

# Check current usage: gcloud dns managed-zones list --format="value(name)" | wc -l # Number of zones

gcloud dns record-sets list --zone=ZONE_NAME | wc -l # Records in zone

# If quota exceeded: # - Request quota increase in GCP Console # - IAM & Admin -> Quotas -> Request higher quota

# Common quota errors: # - "Quota exceeded for quota metric 'Managed zones'" # - "Too many record sets in zone" ```

  1. 1.Use Cloud DNS logging for troubleshooting.

```bash # Enable Cloud DNS logging: gcloud dns managed-zones update ZONE_NAME --enable-logging

# View logs in Cloud Logging: gcloud logging read "resource.type=dns_query" --limit=50

# Filter for specific zone: gcloud logging read "resource.type=dns_query AND labels.zone_name=ZONE_NAME" --limit=50

# Log fields: # - query_name: Domain queried # - query_type: Record type # - source_ip: Who queried # - rcode: Response code (NOERROR, NXDOMAIN, etc.) # - answer_count: Number of records returned

# Look for patterns: # - Unexpected NXDOMAIN responses # - Query source patterns (DDoS detection) # - Response time issues ```

  1. 1.Verify zone delegation at registrar.

```bash # Check registrar nameserver settings match Cloud DNS

# Get Cloud DNS nameservers: gcloud dns managed-zones describe ZONE_NAME --format="value(nameServers)"

# Check TLD servers for delegation: dig @a.gtld-servers.net example.com NS +short

# Compare outputs - must match!

# If mismatch: # Update registrar nameservers to Cloud DNS values

# Verify delegation after update: for ns in $(dig @a.gtld-servers.net example.com NS +short); do echo "Testing $ns:" dig @$ns example.com SOA +short done

# Check for glue records if needed: dig @a.gtld-servers.net example.com NS +additional ```

Verification

Complete Google Cloud DNS verification:

```bash # 1. Check zone configuration echo "=== Zone Configuration ===" gcloud dns managed-zones describe ZONE_NAME \ --format="yaml(name,visibility,nameServers,dnssecConfig.state)"

# 2. Test all nameservers echo -e "\n=== Nameserver Test ===" for ns in $(gcloud dns managed-zones describe ZONE_NAME --format="value(nameServers)"); do echo -n "$ns: " dig @$ns example.com A +short | head -1 done

# 3. Check record sets echo -e "\n=== Records ===" gcloud dns record-sets list --zone=ZONE_NAME --format="table(name,type,ttl,rrdatas)"

# 4. Verify public resolution echo -e "\n=== Public Resolver ===" dig @8.8.8.8 example.com A +short

# 5. Private zone test (if applicable) echo -e "\n=== Private Zone Test ===" # SSH to VPC instance: gcloud compute ssh INSTANCE_NAME --command="dig internal.example.com A @169.254.169.254 +short"

# 6. Check DNSSEC (if enabled) echo -e "\n=== DNSSEC ===" dig @ns-cloud-a1.googledomains.com example.com DNSKEY +short dig @ns-cloud-a1.googledomains.com example.com A +dnssec | grep RRSIG

# 7. Check changes status echo -e "\n=== Recent Changes ===" gcloud dns changes list --zone=ZONE_NAME --limit=5 ```

Google Cloud DNS CLI Quick Reference

```bash # List zones: gcloud dns managed-zones list

# Create public zone: gcloud dns managed-zones create ZONE_NAME \ --dns-name="example.com." \ --description="Public zone"

# Create private zone: gcloud dns managed-zones create INTERNAL_ZONE \ --dns-name="internal.example.com." \ --visibility=private \ --networks="https://www.googleapis.com/compute/v1/projects/PROJECT/global/networks/VPC_NAME"

# Add record: gcloud dns record-sets create www.example.com --zone=ZONE_NAME \ --type=A --ttl=300 --rrdatas="192.0.2.1"

# Update record: gcloud dns record-sets update www.example.com --zone=ZONE_NAME \ --type=A --ttl=300 --rrdatas="192.0.2.2"

# Delete record: gcloud dns record-sets delete www.example.com --zone=ZONE_NAME --type=A

# Enable DNSSEC: gcloud dns managed-zones update ZONE_NAME --dnssec-state=on

# Create forwarding zone: gcloud dns managed-zones create FORWARD_ZONE \ --dns-name="external.com." \ --type=forwarding \ --target-name-servers="192.0.2.1,192.0.2.2" \ --visibility=private \ --networks="VPC_URL"

# Create peering zone: gcloud dns managed-zones create PEER_ZONE \ --dns-name="peer.example.com." \ --type=peering \ --target-network="TARGET_VPC_URL" \ --visibility=private \ --networks="SOURCE_VPC_URL" ```

Google Cloud DNS integrates closely with GCP networking. Always account for zone visibility, VPC associations, and the 169.254.169.254 metadata DNS server when troubleshooting private zones.