Dashboard Latency Budget
A dashboard latency budget caps the query time behind a dashboard panel, so the operator sees data in seconds and the panel does not hang on a query that scans a week of data at every refresh.
Definition
A dashboard latency budget is the practice of setting a maximum query execution time for each dashboard panel and designing the query and pre-aggregation to meet it, so that the dashboard loads in a few seconds and a slow query degrades to a stale-but-visible state rather than a spinning placeholder.
Context and scope
Applied in Grafana, observability platforms, and any dashboard that operators use under time pressure during an incident.
Common misconceptions
- A fast dashboard is just a cached dashboard — caching helps; the budget is met by pre-aggregation, sampling, and query design, not caching alone
- The budget is the query timeout — the timeout is the upper bound; the budget is the target, met by design, with the timeout as the safety net
Anti-patterns
- Building a dashboard that issues a full-scan query on every refresh, so the panel spins for 30 seconds and the operator gives up
- Setting the timeout so high the dashboard is technically never slow but practically unusable
Related concepts
Authoritative source
https://grafana.com/blog/2020/06/01/how-to-build-dashboard-observability/
What it is
A dashboard latency budget is the practice of setting a maximum query execution time for each dashboard panel and designing the query and pre-aggregation to meet it, so that the dashboard loads in a few seconds and a slow query degrades to a stale-but-visible state rather than a spinning placeholder. Applied in Grafana, observability platforms, and any dashboard that operators use under time pressure during an incident. The goal is not to collect all data but to be able to answer the question you did not think to ask in advance — which requires high-cardinality labels and exemplars that link traces to the metric that triggered them.
How it works
Three signals compose: metrics aggregate over time, traces preserve causality across boundaries, and logs carry the unstructured detail. Sampling is adaptive — head sampling for known-interesting paths, tail sampling to retain slow or error traces at a fixed budget. Cardinality is controlled at the source, not the query, because a high-cardinality label explodes storage and makes dashboards unusable.
When to invest and pitfalls
Invest when mean time to detect matters — in production, at scale, with paying users. The pitfall is instrumenting everything and alerting on nothing: dashboards full of lines that no one reads, alerts that fire on noise and are then muted, leaving the real outage undetected. Symptom-based alerts on user-visible degradation outperform cause-based alerts on internal thresholds.