Structured Logging
Structured logging emits machine-parseable events with consistent fields — level, message, ids, context — replacing grep patterns with indexed queries across all services.
Definition
Structured logging writes each event as a record with named fields — timestamp, severity, message template, request and trace ids, and arbitrary structured context — usually JSON. Parsing moves from regex-on-text to schema-aware indexing: fields become query dimensions, and cross-service correlation rides on standardized id fields rather than message formats.
Context and scope
Adopted fleet-wide with JSON formatters or key-value formats, OpenTelemetry log bridges, and centralized log platforms. The prerequisite for logs-as-data pipelines.
Common misconceptions
- Structured logging means JSON — structure is the fields; JSON is one encoding. Key-value and logfmt deliver the same queryability
- Pretty logs in production are fine — human-formatted production logs fragment fields and defeat ingestion pipelines; prettiness belongs in development
- Every field should be indexed — indexing costs; select the fields queries actually filter on and let the rest ride along
Anti-patterns
- Free-text messages with embedded ids and parsing them back out at query time
- Logging entire object dumps per event and blowing ingestion volume and cost
- Mixing field names across services — requestId versus request_id versus trace id — and losing correlation queries
Related concepts
Related examples
Authoritative source
What it is
A log line read by a human and a log record read by a machine are different artifacts. Structured logging commits to the second: events carry typed fields, and queries operate on the fields instead of string fragments.
How it works
Applications log through a framework that serializes events with stable field names. Severity, message templates, and exception stacks become fields; trace and span ids bridge logs to traces. Ingestion parses once at write time, indexes selected fields, and leaves the full record searchable.
When to adopt
Immediately in any multi-service system: correlation queries, alerting on log-derived metrics, and retention policies all require structure. Migration paths include sidecar parsers for legacy text logs.
Pitfalls
PII rides the same pipeline — redact at emission, before ingestion. Volume discipline: debug-level dumps in production turn structured logs into structured floods. Field schema governance is the ongoing cost — drift between teams quietly reintroduces the grep era.