Metric Relabeling Rules
Metric relabeling rules rewrite label values at ingest — bucketing, dropping, renaming — so the stored metric has controlled cardinality and the labels the dashboard will query by.
Definition
Metric relabeling rules are the ingest-time transformations that rewrite, bucket, or drop label values before the metric is stored, so that a high-cardinality label like instance_id is bucketed into a zone, a noisy label is dropped, and the stored series count stays bounded and queryable.
Context and scope
Applied in Prometheus relabeling, OpenTelemetry processors, and any metrics pipeline where the source labels do not match the storage budget.
Common misconceptions
- Relabeling is just label editing — it is label editing with a rule engine at ingest; the value is in doing it once at the source rather than per query
- Dropping labels is always safe — it is safe for cardinality; it loses a dimension the operator may need, so drop with intent
Anti-patterns
- Bucketing a label so coarsely that the dimension is useless, e.g., bucketing latency into fast and slow
- Relabeling without testing against the existing dashboards, so a rename breaks every dashboard query
Related concepts
Authoritative source
https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config
What it is
Metric relabeling rules are the ingest-time transformations that rewrite, bucket, or drop label values before the metric is stored, so that a high-cardinality label like instance_id is bucketed into a zone, a noisy label is dropped, and the stored series count stays bounded and queryable. Applied in Prometheus relabeling, OpenTelemetry processors, and any metrics pipeline where the source labels do not match the storage budget. The goal is not to collect all data but to be able to answer the question you did not think to ask in advance — which requires high-cardinality labels and exemplars that link traces to the metric that triggered them.
How it works
Three signals compose: metrics aggregate over time, traces preserve causality across boundaries, and logs carry the unstructured detail. Sampling is adaptive — head sampling for known-interesting paths, tail sampling to retain slow or error traces at a fixed budget. Cardinality is controlled at the source, not the query, because a high-cardinality label explodes storage and makes dashboards unusable.
When to invest and pitfalls
Invest when mean time to detect matters — in production, at scale, with paying users. The pitfall is instrumenting everything and alerting on nothing: dashboards full of lines that no one reads, alerts that fire on noise and are then muted, leaving the real outage undetected. Symptom-based alerts on user-visible degradation outperform cause-based alerts on internal thresholds.