Introduction

Template variables (also called dashboard variables) make Grafana dashboards dynamic and reusable. They allow users to filter data by hostname, application, region, or other dimensions. When variables fail, dashboards become static or show errors. Common issues include query errors for variable options, refresh failures, and problems with chained variables that depend on each other.

Symptoms

  • Variable dropdown shows "No options" or empty list
  • Error: "Failed to fetch variable values" or "Query error"
  • Variable values are outdated or don't reflect current data
  • Chained variables show wrong options based on parent selection
  • Variable query returns duplicate or incorrect values
  • Dashboard panels show errors when variable values change
  • Error: "Variable depends on itself" or circular dependency

Common Causes

  • Variable query has syntax error or datasource is unavailable
  • Variable query time range doesn't include data with labels
  • Datasource doesn't support the query type used for variable
  • Chained variable refers to a variable that hasn't loaded yet
  • Variable refresh settings are too aggressive causing rate limiting
  • Query returns too many values causing UI slowdown
  • Variable names contain special characters that break interpolation

Step-by-Step Fix

Basic Variable Query Issues

  1. 1.Test variable query directly in datasource:
  2. 2.```bash
  3. 3.# For Prometheus label values query
  4. 4.curl -s "http://prometheus:9090/api/v1/label/job/values"

# For Prometheus label names curl -s "http://prometheus:9090/api/v1/labels" ```

  1. 1.Check variable query syntax in dashboard settings:
  2. 2.- Navigate to Dashboard settings > Variables
  3. 3.- Click on the failing variable
  4. 4.- Check Query options > Query field
  5. 5.Fix Prometheus variable queries:
  6. 6.```promql
  7. 7.# Correct format for label values
  8. 8.label_values(metric_name, label_name)

# Or use Prometheus query result query_result(metric_name) ```

  1. 1.Fix InfluxDB variable queries:
  2. 2.```sql
  3. 3.# Show tag values
  4. 4.SHOW TAG VALUES FROM "measurement" WITH KEY = "tag_name"

# Show measurements SHOW MEASUREMENTS ```

  1. 1.Fix MySQL/PostgreSQL variable queries:
  2. 2.```sql
  3. 3.SELECT DISTINCT column_name FROM table_name ORDER BY column_name
  4. 4.`

Datasource Configuration

  1. 1.Verify datasource supports variable queries:
  2. 2.- Check datasource documentation for supported query formats
  3. 3.- Some datasources require specific query syntax for variables
  4. 4.Test datasource connectivity:
  5. 5.```bash
  6. 6.curl -s -u admin:password http://localhost:3000/api/datasources/proxy/$datasource_id/api/v1/labels
  7. 7.`
  8. 8.Check datasource is configured correctly:
  9. 9.- Navigate to Configuration > Data sources
  10. 10.- Click "Test" to verify connectivity
  11. 11.- Ensure datasource returns expected data

Time Range Issues

  1. 1.Variable queries may be affected by dashboard time range:
  2. 2.- Check if variable query is scoped to time range
  3. 3.- Try expanding dashboard time range
  4. 4.- Verify labels/values exist in the selected time range
  5. 5.For time-independent variables:
  6. 6.- Set "Refresh" to "On Dashboard Load"
  7. 7.- Not "On Time Range Change" to avoid time scoping

Chained Variable Issues

  1. 1.Check variable dependencies:
  2. 2.- Navigate to Dashboard settings > Variables
  3. 3.- List variables in dependency order
  4. 4.- Ensure parent variables load before dependent ones
  5. 5.Fix circular dependencies:
  6. 6.`
  7. 7.# Error: Variable $var1 depends on $var2, which depends on $var1
  8. 8.# Solution: Break the chain by making one variable independent
  9. 9.`
  10. 10.Use correct syntax for variable reference in query:
  11. 11.```promql
  12. 12.# Correct: use $variable or ${variable}
  13. 13.label_values(metric{job="$job"}, instance)

# Wrong: missing quotes for string matching label_values(metric{job=$job}, instance) ```

  1. 1.Handle chained variable refresh:
  2. 2.- Set parent variable refresh: "On Dashboard Load"
  3. 3.- Set dependent variable refresh: "On Variable Change"
  4. 4.- This ensures parent loads before dependent queries

Variable Value Issues

  1. 1.Handle duplicate values:
  2. 2.```promql
  3. 3.# Use sort and dedup
  4. 4.label_values(up, job)
  5. 5.`
  6. 6.- Or enable "Sort" and "Multi-value" options in variable settings
  7. 7.Handle too many values:
  8. 8.- Enable "Include All option"
  9. 9.- Set max values with regex filter:
  10. 10.`
  11. 11.Regex: ^(prod|staging).* # Filter to specific pattern
  12. 12.`
  13. 13.Use regex to filter variable options:
  14. 14.`
  15. 15.# In variable settings > Regex filter
  16. 16.Regex: /.*-prod-.*/ # Only show production instances
  17. 17.`

Variable Refresh Issues

  1. 1.Configure appropriate refresh mode:
  2. 2.- On Dashboard Load: Good for static values
  3. 3.- On Time Range Change: For time-dependent values
  4. 4.- Manually: For rarely-changing values
  5. 5.- On Variable Change: For dependent variables
  6. 6.Handle rate limiting for frequent refreshes:
  7. 7.- Reduce refresh frequency
  8. 8.- Increase datasource rate limit
  9. 9.- Use caching if available

Variable Interpolation Issues

  1. 1.Check variable name for special characters:
  2. 2.- Use alphanumeric and underscore only
  3. 3.- Avoid hyphens, spaces, and special characters
  4. 4.- Example: var_host not var-host
  5. 5.Verify variable interpolation in panel queries:
  6. 6.```promql
  7. 7.# Test in panel query
  8. 8.rate(http_requests_total{host="$hostname"}[5m])

# For multi-select rate(http_requests_total{host=~"$hostname"}[5m]) ```

  1. 1.Use correct match operators:
  2. 2.`
  3. 3.$var - exact match, single value
  4. 4.~$var - regex match, multi-value
  5. 5.${var} - alternative syntax
  6. 6.`

Verification

  1. 1.Verify variable loads options:
  2. 2.- Open dashboard
  3. 3.- Click variable dropdown
  4. 4.- Confirm options are listed
  5. 5.Test variable interpolation:
  6. 6.- Select a variable value
  7. 7.- Verify dashboard panels update
  8. 8.- Check panel queries use selected value
  9. 9.Test chained variables:
  10. 10.- Change parent variable value
  11. 11.- Verify dependent variable options update
  12. 12.- Confirm panels reflect both selections
  13. 13.Check variable in panel edit mode:
  14. 14.- Edit a panel that uses the variable
  15. 15.- Verify query shows interpolated value
  16. 16.- Confirm query preview returns data