OpenTelemetry Collector Pipeline
The OTel collector is a configurable telemetry router — receivers, processors, exporters — that decouples emission from backends, handling batching, enrichment, sampling, and multi-backend fan-out.
Definition
The collector is a vendor-neutral pipeline component: receivers accept telemetry in various protocols, processors transform it (batching, attribute enrichment, redaction, tail sampling, filtering), and exporters ship it to backends. Deployed as agent or gateway, it isolates applications from backend changes and centralizes cross-cutting telemetry policy.
Context and scope
Deployed as sidecar/daemon per node, or as a gateway tier per cluster/region. The standard place for tail sampling, PII redaction, cost-based filtering, and multi-backend fan-out.
Common misconceptions
- The collector is required to use OpenTelemetry — SDKs export directly to backends; the collector adds routing, processing, and lifecycle decoupling
- Processing belongs in applications — cross-cutting policy — redaction, sampling, cost control — belongs in one place, not forty services
- One collector per environment is enough — agent and gateway tiers serve different jobs: node-local low-latency shipping versus cluster-wide heavy processing
Anti-patterns
- Running expensive processors like tail sampling in every node agent and multiplying cost and complexity
- Skipping backpressure configuration and having collector queues spill or drop during backend outages
- Sending unredacted telemetry through a gateway and discovering PII in the backend instead of in the pipeline where it can be dropped
Related concepts
Related examples
Authoritative source
What it is
Applications emit; backends consume; the collector sits between and owns everything that is neither emission nor storage. That middle layer is where protocol translation, batching, enrichment, filtering, sampling, fan-out, and retry live once instead of in forty services.
How it works
The pipeline composes receivers (OTLP, Jaeger, syslog), processors (batch, attributes, k8s-metadata, tail sampling, filter), and exporters (OTLP to vendors, Prometheus). Queue and retry settings bound memory and survive backend outages. Gateway tiers aggregate from agents, apply expensive policy once, and fan out to multiple backends.
When to deploy
Always as the decoupling layer for backend migrations and vendor evaluation. Deploy as a gateway when tail sampling, cost-based filtering, or centralized redaction are needed; as an agent when node-local shipping latency and resilience matter.
Pitfalls
The collector becomes a critical component: monitor its own RED metrics and queue depths, or telemetry outages masquerade as application silence. Version skew between SDK protocols and collector receivers needs management. Overloaded processors in agents — the tail-sampling-in-every-node mistake — turn a router into a bottleneck.