Prometheus Retention Period Config Ignored Disk Still Filling
Prometheus continues to consume disk space beyond the configured retention period because size-based retention is not set or compaction is failing.
Open guideCategory archive
Published troubleshooting guides for prometheus issues.
57 guides available
Prometheus continues to consume disk space beyond the configured retention period because size-based retention is not set or compaction is failing.
Open guidePrometheus Kubernetes service discovery is throttled by the API server, causing stale target lists and missed scrapes.
Open guidePrometheus write-ahead log becomes corrupted after an unclean shutdown, preventing the TSDB from starting.
Open guideUnbounded label values in metrics create millions of time series, exhausting Prometheus memory and disk resources.
Open guideOverly broad relabeling rule with action drop or keep accidentally filters out all metrics from a scrape target.
Open guidePrometheus federation scrape times out when querying a slow upstream Prometheus instance via remote read API.
Open guideAlertmanager fails to deliver alert notifications to configured webhook endpoints, causing missed alerts.
Open guideRecording rule evaluation exceeds the configured timeout, causing stale or missing precomputed metrics.
Open guidePrometheus scrape targets show as DOWN because the exporter process is not accepting connections on the expected port.
Open guidePrometheus TSDB cannot write new samples because the head chunk disk has reached capacity.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideDeep dive into Prometheus issue resolution with production-tested solutions.
Open guideReduce time series cardinality with label optimization and recording rules.
Open guideRecover from WAL corruption with checkpoint tuning and remote write buffering.
Open guide