Introduction

Grafana requires a database to store dashboards, users, and configuration. By default, it uses SQLite, but production deployments often use PostgreSQL or MySQL. Database errors can prevent Grafana from starting, cause dashboard data loss, or result in failed migrations during upgrades.

Symptoms

  • Grafana fails to start with "database connection failed" error
  • Error: "Failed to initialize orm" or "migration failed"
  • Dashboards or users are missing after restart
  • Logs show "database is locked" (SQLite) or "connection refused" (PostgreSQL/MySQL)
  • Grafana API returns 500 errors when saving dashboards
  • Migration errors appear during Grafana upgrade

Common Causes

  • SQLite database file is locked by another process or corrupted
  • PostgreSQL/MySQL connection string is incorrect or credentials expired
  • Database user lacks required permissions
  • Database migrations failed to complete during upgrade
  • Disk space exhausted on database storage
  • Network connectivity issues to remote database servers
  • Database schema version mismatch between Grafana and database

Step-by-Step Fix

SQLite Database Issues

  1. 1.Check if SQLite database is locked or corrupted:
  2. 2.```bash
  3. 3.# Stop Grafana
  4. 4.systemctl stop grafana-server

# Check database integrity sqlite3 /var/lib/grafana/grafana.db "PRAGMA integrity_check;"

# Check for lock files ls -la /var/lib/grafana/grafana.db* ```

  1. 1.Remove stale lock files:
  2. 2.```bash
  3. 3.rm -f /var/lib/grafana/grafana.db-shm
  4. 4.rm -f /var/lib/grafana/grafana.db-wal
  5. 5.`
  6. 6.If database is corrupted, restore from backup:
  7. 7.```bash
  8. 8.# Backup current database
  9. 9.cp /var/lib/grafana/grafana.db /var/lib/grafana/grafana.db.corrupted

# Restore from backup cp /path/to/backup/grafana.db /var/lib/grafana/grafana.db chown grafana:grafana /var/lib/grafana/grafana.db ```

PostgreSQL Connection Issues

  1. 1.Verify PostgreSQL connectivity:
  2. 2.```bash
  3. 3.psql -h postgres-server -U grafana -d grafana
  4. 4.`
  5. 5.Check Grafana database configuration:
  6. 6.```ini
  7. 7.# In grafana.ini
  8. 8.[database]
  9. 9.type = postgres
  10. 10.host = postgres-server:5432
  11. 11.name = grafana
  12. 12.user = grafana
  13. 13.password = your-password
  14. 14.ssl_mode = disable
  15. 15.`
  16. 16.Verify PostgreSQL user permissions:
  17. 17.```sql
  18. 18.-- Connect as postgres superuser
  19. 19.psql -U postgres

-- Grant required permissions GRANT ALL PRIVILEGES ON DATABASE grafana TO grafana; GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO grafana; GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA public TO grafana; ```

MySQL Connection Issues

  1. 1.Verify MySQL connectivity:
  2. 2.```bash
  3. 3.mysql -h mysql-server -u grafana -p grafana
  4. 4.`
  5. 5.Check Grafana MySQL configuration:
  6. 6.```ini
  7. 7.# In grafana.ini
  8. 8.[database]
  9. 9.type = mysql
  10. 10.host = mysql-server:3306
  11. 11.name = grafana
  12. 12.user = grafana
  13. 13.password = your-password
  14. 14.`
  15. 15.Grant required MySQL permissions:
  16. 16.```sql
  17. 17.GRANT ALL PRIVILEGES ON grafana.* TO 'grafana'@'%';
  18. 18.FLUSH PRIVILEGES;
  19. 19.`

Migration Failures

  1. 1.Check current database migration status:
  2. 2.```bash
  3. 3.# SQLite
  4. 4.sqlite3 /var/lib/grafana/grafana.db "SELECT * FROM migration_log ORDER BY id DESC LIMIT 10;"

# PostgreSQL/MySQL psql -U grafana -d grafana -c "SELECT * FROM migration_log ORDER BY id DESC LIMIT 10;" ```

  1. 1.If migrations are stuck, manually run migrations:
  2. 2.```bash
  3. 3.grafana-cli admin data-migration
  4. 4.`
  5. 5.For failed upgrades, check if database schema version matches:
  6. 6.```bash
  7. 7.sqlite3 /var/lib/grafana/grafana.db "SELECT * FROM migration_log WHERE migration_id LIKE '%clean%';"
  8. 8.`

Disk Space and Permissions

  1. 1.Check disk space:
  2. 2.```bash
  3. 3.df -h /var/lib/grafana
  4. 4.df -h /var/lib/postgresql # For PostgreSQL
  5. 5.`
  6. 6.Verify file permissions:
  7. 7.```bash
  8. 8.chown -R grafana:grafana /var/lib/grafana
  9. 9.chmod 750 /var/lib/grafana
  10. 10.chmod 640 /var/lib/grafana/grafana.db
  11. 11.`

Verification

  1. 1.Verify database connection:
  2. 2.```bash
  3. 3.curl http://localhost:3000/api/health
  4. 4.`
  5. 5.Check Grafana logs for database errors:
  6. 6.```bash
  7. 7.journalctl -u grafana-server -n 50 | grep -i "database|migration|orm"
  8. 8.`
  9. 9.Verify dashboard data is accessible by logging in and viewing existing dashboards.