Your Grafana dashboard is stuck loading, shows blank panels, or displays cryptic error messages. This can be frustrating when you need critical visibility into your systems. Let's systematically diagnose and fix dashboard issues.

Understanding the Problem

Dashboard issues typically manifest in several ways:

Browser console errors: `` Error: Panel plugin not found: grafana-piechart-panel

bash
Failed to load dashboard: Unexpected token < in JSON at position 0

Panel errors: `` Panel data state changed to: Error

bash
Cannot read property 'data' of undefined

General UI errors: `` Dashboard not found

bash
Failed to save dashboard

Initial Diagnosis

Start by checking browser developer tools and Grafana logs:

```bash # Open browser DevTools (F12) and check the Console tab for JavaScript errors # Check the Network tab for failed requests (red entries)

# Check Grafana server logs journalctl -u grafana-server -f | grep -i "error|dashboard|panel"

# For Kubernetes kubectl logs -l app=grafana -n monitoring -f | grep -i "error|dashboard"

# Check Grafana log file tail -f /var/log/grafana/grafana.log ```

Check the dashboard's JSON model:

```bash # Get dashboard by UID curl -s http://admin:password@localhost:3000/api/dashboards/uid/dashboard-uid | jq '.' > dashboard.json

# Check for JSON parsing errors jq '.' dashboard.json ```

Common Cause 1: Missing Plugin Dependencies

If your dashboard uses custom panels or data sources, missing plugins will cause failures.

Error pattern: `` Error: Panel plugin not found: grafana-piechart-panel

Diagnosis:

```bash # List installed plugins grafana-cli plugins ls-all

# Check via API curl -s http://admin:password@localhost:3000/api/plugins | jq '.[] | {id: .id, enabled: .enabled}'

# Check for plugin errors in logs grep -i "plugin" /var/log/grafana/grafana.log | tail -20 ```

Solution:

Install missing plugins:

```bash # Install the required plugin grafana-cli plugins install grafana-piechart-panel

# Restart Grafana systemctl restart grafana-server

# For Kubernetes, you may need to update your deployment kubectl set env deployment/grafana GF_INSTALL_PLUGINS=grafana-piechart-panel -n monitoring

# Or specify plugins in grafana.ini # [plugins] # allow_loading_unsigned_plugins = grafana-piechart-panel ```

Common Cause 2: Dashboard JSON Corruption

Sometimes the dashboard JSON model gets corrupted, especially after failed saves or migrations.

Error pattern: `` Failed to load dashboard: Unexpected token < in JSON at position 0

Diagnosis:

```bash # Export dashboard and check for JSON errors curl -s http://admin:password@localhost:3000/api/dashboards/uid/your-dashboard-uid > dashboard.json

# Validate JSON jq '.' dashboard.json # If this fails, the JSON is corrupted

# Check database for the dashboard sqlite3 /var/lib/grafana/grafana.db "SELECT data FROM dashboard WHERE uid='your-dashboard-uid';" | jq '.' > dashboard_db.json diff dashboard.json dashboard_db.json ```

Solution:

Restore from a backup or fix the JSON manually:

```bash # If you have a previous version in version control curl -X POST http://admin:password@localhost:3000/api/dashboards/db \ -H "Content-Type: application/json" \ -d @backup-dashboard.json

# Or reset to a clean dashboard # First, get the dashboard ID DASHBOARD_ID=$(curl -s http://admin:password@localhost:3000/api/search?query=dashboard-name | jq '.[0].id')

# Delete the corrupted dashboard curl -X DELETE http://admin:password@localhost:3000/api/dashboards/uid/your-dashboard-uid

# Re-import from backup curl -X POST http://admin:password@localhost:3000/api/dashboards/import \ -H "Content-Type: application/json" \ -d @dashboard-backup.json ```

Common Cause 3: Datasource Issues

If the datasource referenced by the dashboard is broken or renamed, panels won't load.

Error pattern: `` Datasource not found

bash
Unknown datasource name: Prometheus

Diagnosis:

```bash # List all datasources curl -s http://admin:password@localhost:3000/api/datasources | jq '.[] | {id: .id, name: .name, type: .type}'

# Check datasource references in dashboard curl -s http://admin:password@localhost:3000/api/dashboards/uid/your-dashboard-uid | \ jq '.dashboard.panels[].targets[]?.datasource'

# Test datasource connectivity curl -s http://admin:password@localhost:3000/api/datasources/1/health | jq '.' ```

Solution:

Fix datasource references:

```bash # Option 1: Update the datasource name to match curl -X PATCH http://admin:password@localhost:3000/api/datasources/1 \ -H "Content-Type: application/json" \ -d '{"name": "Prometheus"}'

# Option 2: Update dashboard to use correct datasource # Export dashboard, edit JSON, re-import curl -s http://admin:password@localhost:3000/api/dashboards/uid/your-dashboard-uid > dashboard.json

# Edit the file to fix datasource references sed -i 's/"OldPrometheus"/"Prometheus"/g' dashboard.json

# Re-import curl -X POST http://admin:password@localhost:3000/api/dashboards/db \ -H "Content-Type: application/json" \ --data-binary @dashboard.json ```

Common Cause 4: Query Performance Issues

Complex dashboards with many panels or expensive queries can timeout or fail to load.

Error pattern: `` Query execution was interrupted

bash
context deadline exceeded

Diagnosis:

```bash # Check query performance in browser DevTools Network tab # Look for slow /api/ds/query requests

# Enable query logging in Grafana # Add to grafana.ini [log] filters = tsdb:debug

# Check for slow queries in logs grep -i "slow|timeout" /var/log/grafana/grafana.log | tail -20

# Monitor Grafana metrics curl -s http://localhost:3000/metrics | grep -E "grafana_http_request_duration|grafana_api_response" ```

Solution:

Optimize queries and dashboard:

```bash # Increase Grafana timeout # Edit /etc/grafana/grafana.ini [dataproxy] timeout = 60 dialTimeout = 30

[unified_alerting] execute_alerts = true evaluation_timeout = 30s

# Restart Grafana systemctl restart grafana-server ```

Optimize the dashboard itself:

```javascript // Reduce refresh rate // Change from 5s to 30s or 1m

// Use query variables to reduce queries // Instead of 10 separate queries, use one with group by

// Disable auto-refresh for complex dashboards // Set refresh to "off" and manually refresh when needed ```

Common Cause 5: Browser Issues

Sometimes the problem is in the browser itself.

Diagnosis: - Clear browser cache and cookies - Try in an incognito/private window - Try a different browser - Check for browser extensions blocking content

Solution:

```bash # Clear Grafana's browser data # Chrome: Settings > Privacy > Clear browsing data > select "grafana" domain

# Or force refresh in Grafana # Add ?refresh=1s to the dashboard URL to force a refresh

# Reset user preferences via API curl -X DELETE http://admin:password@localhost:3000/api/user/preferences ```

Common Cause 6: Permission Issues

Users without proper permissions may see partial or no dashboards.

Error pattern: `` You do not have permission to view this dashboard

Diagnosis:

```bash # Check current user permissions curl -s http://admin:password@localhost:3000/api/user | jq '.isGrafanaAdmin, .isDisabled'

# Check team memberships curl -s http://admin:password@localhost:3000/api/teams | jq '.[] | {id: .id, name: .name}'

# Check dashboard permissions curl -s http://admin:password@localhost:3000/api/dashboards/uid/dashboard-uid | jq '.meta.permissions' ```

Solution:

```bash # Grant access via API curl -X POST http://admin:password@localhost:3000/api/dashboards/uid/dashboard-uid/permissions \ -H "Content-Type: application/json" \ -d '{ "items": [ { "role": "Viewer", "permission": 1 }, { "role": "Editor", "permission": 2 } ] }'

# Add user to team curl -X POST http://admin:password@localhost:3000/api/teams/1/members \ -H "Content-Type: application/json" \ -d '{"userId": 5}' ```

Common Cause 7: Grafana Database Issues

The underlying Grafana database (SQLite, MySQL, or PostgreSQL) can cause problems.

Error pattern: `` Error 1062: Duplicate entry for key 'dashboard'

bash
database disk image is malformed

Diagnosis:

```bash # Check database connection curl -s http://admin:password@localhost:3000/api/admin/stats | jq '.'

# For SQLite sqlite3 /var/lib/grafana/grafana.db "PRAGMA integrity_check;"

# Check database size ls -lh /var/lib/grafana/grafana.db

# For MySQL/PostgreSQL mysql -u grafana -p -e "SELECT COUNT(*) FROM dashboard;" psql -U grafana -d grafana -c "SELECT COUNT(*) FROM dashboard;" ```

Solution:

```bash # For SQLite corruption # Stop Grafana systemctl stop grafana-server

# Backup database cp /var/lib/grafana/grafana.db /var/lib/grafana/grafana.db.backup

# Attempt repair sqlite3 /var/lib/grafana/grafana.db ".recover" | sqlite3 grafana_fixed.db mv grafana_fixed.db /var/lib/grafana/grafana.db

# Start Grafana systemctl start grafana-server

# For MySQL/PostgreSQL issues # Check logs for specific database errors # Run database-specific repair commands ```

Common Cause 8: Template Variables Issues

Broken template variables can prevent the entire dashboard from loading.

Error pattern: `` Template variable service init failed

bash
Error updating options: cannot fetch values

Diagnosis:

```bash # Check dashboard variables curl -s http://admin:password@localhost:3000/api/dashboards/uid/dashboard-uid | \ jq '.dashboard.templating.list'

# Test variable query directly curl -s "http://admin:password@localhost:3000/api/datasources/proxy/1/api/v1/label/__name__/values" | jq '.' ```

Solution:

bash
# Fix variable definition via API
curl -X POST http://admin:password@localhost:3000/api/dashboards/db \
  -H "Content-Type: application/json" \
  -d '{
    "dashboard": {
      "uid": "dashboard-uid",
      "templating": {
        "list": [
          {
            "name": "instance",
            "type": "query",
            "datasource": "Prometheus",
            "query": "label_values(up, instance)",
            "refresh": 1
          }
        ]
      }
    },
    "overwrite": true
  }'

Verification

After fixing the issue, verify everything works:

```bash # Test dashboard loading curl -s http://admin:password@localhost:3000/api/dashboards/uid/dashboard-uid | jq '.meta.isFolder, .dashboard.title'

# Test panel queries curl -X POST http://admin:password@localhost:3000/api/ds/query \ -H "Content-Type: application/json" \ -d '{ "queries": [ { "datasourceId": 1, "refId": "A", "rawSql": "SELECT 1" } ] }' | jq '.results'

# Check Grafana health curl -s http://admin:password@localhost:3000/api/health | jq '.' ```

Prevention

Implement these practices to prevent future dashboard issues:

```bash # 1. Regular dashboard backups #!/bin/bash # backup-dashboards.sh GRAFANA_URL="http://localhost:3000" API_KEY="your-api-key"

# Get all dashboards DASHBOARDS=$(curl -s -H "Authorization: Bearer $API_KEY" "$GRAFANA_URL/api/search?type=dash-db" | jq -r '.[].uid')

for UID in $DASHBOARDS; do curl -s -H "Authorization: Bearer $API_KEY" \ "$GRAFANA_URL/api/dashboards/uid/$UID" > "dashboards/$UID.json" done

# 2. Monitor dashboard loading times # Add to Prometheus scrape config - job_name: 'grafana' static_configs: - targets: ['grafana:3000']

# 3. Set up alerts for dashboard issues groups: - name: grafana_dashboards rules: - alert: GrafanaDashboardLoadError expr: increase(grafana_http_request_duration_seconds_count{status="500"}[5m]) > 5 for: 5m labels: severity: warning annotations: summary: "Grafana dashboard errors detected" ```

Dashboard issues usually stem from plugin dependencies, datasource problems, or JSON corruption. Start with browser DevTools to identify the specific error, then work through the relevant solutions.