Metric Histogram Buckets
Histogram bucket boundaries must be chosen so the SLO threshold falls inside a bucket, so the p99 computed from bucket counts is accurate at the latency that matters rather than only at the median.
Definition
Histogram bucket boundaries are the configurable cut points that aggregate latency observations into count buckets; the choice matters because percentile estimates from buckets are accurate only when the SLO threshold falls well inside a bucket, and a bucket boundary at the SLO threshold makes the p99 a guess.
Context and scope
Applied in Prometheus histograms, OpenTelemetry, and any metrics system that computes percentiles from pre-aggregated bucket counts.
Common misconceptions
- More buckets mean more accuracy — they mean more accuracy at more points; each bucket adds a time series, so the cost is linear and the accuracy gain is logarithmic
- Exponential buckets are always right — they are right for latency; for distributions with a spike at a specific value, linear buckets around the spike are better
Anti-patterns
- Putting a bucket boundary exactly at the SLO threshold, so the p99 jumps between buckets and the alert flaps
- Using so few buckets that the p99 is a coarse estimate, making the SLO burn rate a guess
Related concepts
Authoritative source
What it is
Histogram bucket boundaries are the configurable cut points that aggregate latency observations into count buckets; the choice matters because percentile estimates from buckets are accurate only when the SLO threshold falls well inside a bucket, and a bucket boundary at the SLO threshold makes the p99 a guess. Applied in Prometheus histograms, OpenTelemetry, and any metrics system that computes percentiles from pre-aggregated bucket counts. The goal is not to collect all data but to be able to answer the question you did not think to ask in advance — which requires high-cardinality labels and exemplars that link traces to the metric that triggered them.
How it works
Three signals compose: metrics aggregate over time, traces preserve causality across boundaries, and logs carry the unstructured detail. Sampling is adaptive — head sampling for known-interesting paths, tail sampling to retain slow or error traces at a fixed budget. Cardinality is controlled at the source, not the query, because a high-cardinality label explodes storage and makes dashboards unusable.
When to invest and pitfalls
Invest when mean time to detect matters — in production, at scale, with paying users. The pitfall is instrumenting everything and alerting on nothing: dashboards full of lines that no one reads, alerts that fire on noise and are then muted, leaving the real outage undetected. Symptom-based alerts on user-visible degradation outperform cause-based alerts on internal thresholds.