Fix Consul Template Not Rendering

What's Actually Happening

Consul Template fails to render templates from Consul KV or service data. Output files are not generated or updated.

The Error You'll See

```bash $ consul-template -template "template.ctmpl:output.txt"

2024/01/01 12:00:00 [ERR] (runner) error parsing template: template: :1: unexpected EOF ```

Connection error:

2024/01/01 12:00:00 [ERR] (cli) error connecting to Consul: connection refused

Permission error:

2024/01/01 12:00:00 [ERR] (runner) error rendering: permission denied

Query error:

2024/01/01 12:00:00 [ERR] (runner) error rendering: key "config/app" not found

Why This Happens

1. 1.Consul unreachable - Cannot connect to Consul server
2. 2.Template syntax error - Invalid template syntax
3. 3.Key not found - KV path doesn't exist
4. 4.Permission denied - Cannot write output file
5. 5.ACL token missing - Authentication required
6. 6.Service not found - Referenced service doesn't exist

Step 1: Check Consul Template Status

```bash # Check if running: ps aux | grep consul-template

# Check service: systemctl status consul-template

# Check logs: journalctl -u consul-template -f

# Run manually: consul-template -config=/etc/consul-template/config.hcl

# Verbose mode: consul-template -config=/etc/consul-template/config.hcl -log-level=debug

# Dry run: consul-template -template "template.ctmpl:output.txt" -dry

# Check version: consul-template -version

# Check configuration: cat /etc/consul-template/config.hcl

# Check template file: cat /etc/consul-template/templates/template.ctmpl ```

Step 2: Check Consul Connectivity

```bash # Test Consul connectivity: consul members

# Check Consul agent: consul info

# Check Consul KV: consul kv get -recurse

# Check Consul services: consul catalog services

# Check Consul health: curl http://localhost:8500/v1/agent/self

# Test DNS: dig @localhost -p 8600 consul.service.consul

# Check Consul address: grep "address" /etc/consul-template/config.hcl

# Set Consul address: consul { address = "127.0.0.1:8500" }

# Or via environment: export CONSUL_HTTP_ADDR=127.0.0.1:8500

# Check for TLS: consul { address = "127.0.0.1:8501" ssl { enabled = true verify = false } }

# Test with curl: curl http://localhost:8500/v1/kv/config/app ```

Step 3: Fix Template Syntax

```bash # Common template syntax:

# Key lookup: {{ key "config/app/database" }}

# Service lookup: {{ range service "web" }} {{ .Address }}:{{ .Port }} {{ end }}

# Service with tags: {{ range service "web" "tag:production" }} {{ .Address }} {{ end }}

# Key with default: {{ keyOrDefault "config/app/port" "8080" }}

# Service by tag: {{ range $tag, $services := service "web" | byTag }} {{ $tag }}: {{ range $services }} {{ .Address }} {{ end }} {{ end }}

# Check template syntax: # Common errors:

# 1. Unclosed braces: # Wrong: {{ key "path" # Right: {{ key "path" }}

# 2. Wrong function name: # Wrong: {{ getkey "path" }} # Right: {{ key "path" }}

# 3. Missing range end: # Wrong: {{ range service "web" }} # Right: {{ range service "web" }}...{{ end }}

# 4. Incorrect service query: # Wrong: {{ service web }} # Right: {{ service "web" }}

# Test template: consul-template -template "test.ctmpl:-" -dry ```

Step 4: Check KV Path

```bash # Check if key exists: consul kv get config/app/database

# List all keys: consul kv get -recurse config/

# Check key value: consul kv get config/app/port

# Set test key: consul kv put config/test "value"

# Check in template: {{ key "config/test" }}

# If key missing: # Option 1: Create key consul kv put config/app/database "mysql"

# Option 2: Use default in template: {{ keyOrDefault "config/app/database" "localhost" }}

# Check key path format: # Consul uses / as separator # Wrong: config.app.database # Right: config/app/database

# Check for namespace (Consul Enterprise): consul kv get -namespace=default config/app ```

Step 5: Check Service Discovery

```bash # Check service exists: consul catalog services

# Check service instances: consul catalog nodes -service=web

# Check service health: curl http://localhost:8500/v1/health/service/web

# Check via DNS: dig @localhost -p 8600 web.service.consul

# Service template: {{ range service "web" }} server {{ .Name }} {{ .Address }}:{{ ..Port }}; {{ end }}

# Filter by health status: {{ range service "web" }} {{ if eq .Status "passing" }} server {{ .Name }} {{ .Address }}:{{ .Port }}; {{ end }} {{ end }}

# Service with tag: consul services -tag=production

# Template with tag: {{ range service "web" "production" }} {{ .Address }} {{ end }}

# Check for service registration: consul services ```

Step 6: Fix ACL Configuration

```bash # Check if ACL enabled: consul acl policy list

# Check Consul ACL token: consul acl token list

# Get token: consul acl token read -id=<token-id>

# In consul-template config: consul { token = "your-token-here" }

# Or via environment: export CONSUL_HTTP_TOKEN=your-token

# Check token permissions: consul acl token read -self

# Create token: consul acl policy create -name "consul-template" \ -rules='key_prefix "config/" { policy = "read" }'

consul acl token create -description="Consul Template" \ -policy-name="consul-template"

# Policy for services: service_prefix "" { policy = "read" }

# Policy for KV: key_prefix "config/" { policy = "read" }

# Check for ACL errors: consul-template -log-level=debug ```

Step 7: Check File Permissions

```bash # Check output directory: ls -la /etc/nginx/conf.d/

# Check output file: ls -la /etc/nginx/conf.d/upstream.conf

# Check consul-template user: ps aux | grep consul-template

# Fix permissions: chown consul-template:consul-template /etc/nginx/conf.d/ chmod 755 /etc/nginx/conf.d/

# Check template file: ls -la /etc/consul-template/templates/

# Fix template permissions: chmod 644 /etc/consul-template/templates/*.ctmpl

# Check config permissions: ls -la /etc/consul-template/ chmod 644 /etc/consul-template/config.hcl

# SELinux context: ls -laZ /etc/nginx/conf.d/

# Restore context: restorecon -Rv /etc/nginx/conf.d/ ```

Step 8: Check Configuration

```bash # Main configuration file: cat /etc/consul-template/config.hcl

# Common configuration: consul { address = "127.0.0.1:8500" retry { enabled = true attempts = 12 backoff = "250ms" max_backoff = "1m" } }

template { source = "/etc/consul-template/templates/app.ctmpl" destination = "/etc/app/config.conf" command = "systemctl reload app" perms = 0644 backup = true }

# Multiple templates: template { source = "template1.ctmpl" destination = "/etc/app/config1.conf" }

template { source = "template2.ctmpl" destination = "/etc/app/config2.conf" }

# Reload signal: reload_signal = "SIGHUP"

# Kill signal: kill_signal = "SIGINT"

# Check config syntax: consul-template -config=/etc/consul-template/config.hcl -dry

# Check multiple config files: ls /etc/consul-template/config.d/ ```

Step 9: Fix Common Issues

```bash # Template not updating:

# 1. Check if watching for changes: consul-template -log-level=debug

# 2. Check block query: # In template: {{ range ls "config" }} {{ .Key }}={{ .Value }} {{ end }}

# Output file not created:

# 1. Check directory exists: mkdir -p /etc/app/

# 2. Check permissions: chmod 755 /etc/app/

# Command not running:

# 1. Check command syntax: command = "/usr/bin/systemctl reload nginx"

# 2. Check command exists: which systemctl

# 3. Check command permissions: # May need sudo or specific user

# Memory issues:

# Set max stale: max_stale = "10m"

# Set wait time: wait { min = "5s" max = "10s" }

# Exec mode: exec { command = "/usr/local/bin/my-app" splay = "5m" } ```

Step 10: Consul Template Verification Script

```bash # Create verification script: cat << 'EOF' > /usr/local/bin/check-consul-template.sh #!/bin/bash

echo "=== Consul Template Process ===" ps aux | grep consul-template | grep -v grep || echo "No consul-template process"

echo "" echo "=== Service Status ===" systemctl status consul-template 2>/dev/null | head -5 || echo "Service not configured"

echo "" echo "=== Configuration ===" cat /etc/consul-template/config.hcl 2>/dev/null | head -20 || echo "No configuration file"

echo "" echo "=== Template Files ===" ls -la /etc/consul-template/templates/ 2>/dev/null || echo "No templates directory"

echo "" echo "=== Consul Connectivity ===" consul members 2>/dev/null || echo "Cannot connect to Consul"

echo "" echo "=== Consul KV ===" consul kv get -recurse 2>/dev/null | head -20 || echo "Cannot read KV"

echo "" echo "=== Consul Services ===" consul catalog services 2>/dev/null | head -20 || echo "Cannot list services"

echo "" echo "=== Test Template ===" for tmpl in /etc/consul-template/templates/*.ctmpl; do if [ -f "$tmpl" ]; then echo "Testing: $tmpl" consul-template -template "$tmpl:-" -dry 2>&1 | head -5 fi done

echo "" echo "=== Output Files ===" grep -h "destination" /etc/consul-template/config.hcl 2>/dev/null | while read line; do dest=$(echo $line | grep -oP '(?<=destination = ")[^"]+') if [ -n "$dest" ] && [ -f "$dest" ]; then echo "File: $dest" ls -la "$dest" fi done

echo "" echo "=== Recent Logs ===" journalctl -u consul-template --no-pager -n 10 2>/dev/null || echo "No systemd logs"

echo "" echo "=== Recommendations ===" echo "1. Verify Consul is running and accessible" echo "2. Check template syntax with -dry flag" echo "3. Ensure KV paths exist in Consul" echo "4. Verify services are registered" echo "5. Check ACL token has required permissions" echo "6. Verify output directory permissions" echo "7. Enable debug logging for troubleshooting" EOF

chmod +x /usr/local/bin/check-consul-template.sh

# Usage: /usr/local/bin/check-consul-template.sh ```

Consul Template Rendering Checklist

Verify the Fix

```bash # After fixing Consul Template

# 1. Test template consul-template -template "test.ctmpl:-" -dry // Template renders

# 2. Run consul-template consul-template -config=/etc/consul-template/config.hcl // Running without errors

# 3. Check output cat /etc/app/config.conf // Content rendered correctly

# 4. Verify updates consul kv put config/test "newvalue" // Output file updated

# 5. Check service systemctl status consul-template // Active running

# 6. Monitor logs journalctl -u consul-template -f // No errors ```

Related Issues

• [Fix Consul Service Not Registering](/articles/fix-consul-service-not-registering)
• [Fix Consul DNS Resolution Wrong](/articles/fix-consul-dns-resolution-wrong)
• [Fix Vault Token Renewal Failed](/articles/fix-vault-token-renewal-failed)