Wire Distributed Tracing with OpenTelemetry

Auto-instrument HTTP and database calls with the OpenTelemetry SDK, configure a batch exporter to a collector, and verify a trace spans three services.

~40 minopentelemetrytracingobservability

Task

Make a request that crosses three services produce a single trace with one span per service in the collector.

When to use this

A request fails somewhere between the gateway, the orders service, and the payments service, but each log line stands alone. The team wants one trace id that ties every span together so they can find the slow hop.

Prerequisites

Node.js 20 LTS in every service
An OpenTelemetry collector reachable from every service
A shared service name for each process

Steps

1. Install the SDK and the auto-instrumentation package that patches http and pg.

pnpm add @opentelemetry/sdk-node @opentelemetry/auto-instrumentations-node

Expected output: The packages are in dependencies and the SDK can start before the app code loads.

2. Start the SDK in a tracing file that loads before the application entry point.

import { NodeSDK } from "@opentelemetry/sdk-node";
new NodeSDK({
  serviceName: "orders",
  traceExporter: new OTLPTraceExporter({ url: "http://collector:4318/v1/traces" }),
  instrumentations: [getNodeAutoInstrumentations()],
}).start();

Expected output: The SDK patches http and pg before the app starts, so every outbound call becomes a child span of the request span.

3. Send a request through the gateway and confirm the collector receives one trace with three service spans.

curl http://gateway/checkout && tail -f /var/log/collector/trace.json

Expected output: The collector log shows one trace id with spans from gateway, orders, and payments, linked by parent ids.

Troubleshooting

Problem

The trace breaks at a service boundary

Solution

The service is not propagating the trace context; ensure the auto-instrumentation patches the http client it uses, or inject the context manually.

Problem

The collector receives no traces

Solution

Check the exporter URL and protocol; the OTLP HTTP exporter expects /v1/traces, the gRPC exporter expects port 4317.

Limitations

Auto-instrumentation covers popular libraries but not custom RPC layers; those need manual spans.
A high sampling rate at traffic peak can overwhelm the collector; sample head-based or tail-based to control volume.

One trace, three services

The SDK patches http and pg before the app starts, so every outbound call becomes a child span of the incoming request span. The trace context propagates over the wire, so the gateway, orders, and payment spans share one trace id.

import { NodeSDK } from '@opentelemetry/sdk-node';
new NodeSDK({
  serviceName: 'orders',
  traceExporter: new OTLPTraceExporter({ url: 'http://collector:4318/v1/traces' }),
  instrumentations: [getNodeAutoInstrumentations()],
}).start();

Load before the app

The tracing file must load before the application code, so the patches are in place before any library is imported. Use –require or the NODE_OPTIONS environment variable to load it first.

node --require ./tracing.js dist/index.js

Expected outcome

A request through the gateway produces one trace in the collector with spans from all three services, linked by parent ids. The slow hop is the span with the longest duration, and a failure shows exactly which service threw.