Log Aggregation Design
Log aggregation design collects logs from every instance into a queryable store with retention and search, so an operator finds a log across a fleet without SSH to each machine.
Definition
Log aggregation design is the architecture that ships logs from every service instance through a collector to a central store with indexing, retention, and search, so that an operator can query across the fleet — by service, time, correlation ID — without accessing each machine, with volume controlled by sampling and level filtering.
Context and scope
Applied in any multi-instance deployment where per-machine logs are not queryable and the volume is too high for ad-hoc collection.
Common misconceptions
- Log aggregation is just structured logging — structured logging is the emit format; aggregation is the collection, storage, and search, which is separate
- Centralizing logs always helps — it helps queryability; it adds a pipeline failure mode where a log shipper bug drops logs silently
Anti-patterns
- Shipping every log at INFO level, overwhelming the store and making search slow and expensive
- Aggregating without retention limits, so the store grows without bound and the query cost rises monthly
Related concepts
Authoritative source
What it is
Log aggregation design is the architecture that ships logs from every service instance through a collector to a central store with indexing, retention, and search, so that an operator can query across the fleet — by service, time, correlation ID — without accessing each machine, with volume controlled by sampling and level filtering. Applied in any multi-instance deployment where per-machine logs are not queryable and the volume is too high for ad-hoc collection. The goal is not to collect all data but to be able to answer the question you did not think to ask in advance — which requires high-cardinality labels and exemplars that link traces to the metric that triggered them.
How it works
Three signals compose: metrics aggregate over time, traces preserve causality across boundaries, and logs carry the unstructured detail. Sampling is adaptive — head sampling for known-interesting paths, tail sampling to retain slow or error traces at a fixed budget. Cardinality is controlled at the source, not the query, because a high-cardinality label explodes storage and makes dashboards unusable.
When to invest and pitfalls
Invest when mean time to detect matters — in production, at scale, with paying users. The pitfall is instrumenting everything and alerting on nothing: dashboards full of lines that no one reads, alerts that fire on noise and are then muted, leaving the real outage undetected. Symptom-based alerts on user-visible degradation outperform cause-based alerts on internal thresholds.