Add Structured JSON Logging

Replace ad-hoc console output with structured JSON logs that carry request ids, status, latency, and redacted headers for reliable production debugging.

~20 minobservabilityloggingnodejs

Task

Emit one JSON object per request with a stable schema, correlation id, status, duration, and no secrets.

When to use this

Production incidents require correlating a user report with server logs. Unstructured messages are hard to query, and logs that include authorization headers leak credentials.

Prerequisites

A Node.js HTTP service
A log collector that accepts JSON lines
Agreement on field names for request id, route, status, and duration

Steps

1. Configure a JSON logger with redaction for authorization and cookie headers, and a base field for the service name.

import pino from 'pino';

export const logger = pino({
  level: process.env.LOG_LEVEL ?? 'info',
  base: { service: 'orders-api' },
  redact: {
    paths: ['req.headers.authorization', 'req.headers.cookie'],
    censor: '[REDACTED]',
  },
});

Expected output: Log lines are single JSON objects and sensitive headers are replaced with [REDACTED].

2. Generate or accept a request id, attach it to the request, and log a completion event with status and duration.

import { randomUUID } from 'node:crypto';

app.use((req, res, next) => {
  req.id = req.header('x-request-id') ?? randomUUID();
  res.setHeader('x-request-id', req.id);
  const start = process.hrtime.bigint();
  res.on('finish', () => {
    const durationMs = Number(process.hrtime.bigint() - start) / 1e6;
    logger.info({
      requestId: req.id,
      method: req.method,
      route: req.route?.path ?? req.path,
      status: res.statusCode,
      durationMs: Math.round(durationMs),
    }, 'request completed');
  });
  next();
});

Expected output: Each completed request emits one JSON line with requestId, route, status, and durationMs.

3. Verify the output is valid JSON and that the same request id appears in the response header and the log.

curl -s -D - http://localhost:3000/health && tail -n 1 app.log | jq .

Expected output: The header and JSON log share a request id; jq parses the line without error.

Troubleshooting

Problem

Logs contain passwords or tokens from request bodies

Solution

Do not log request bodies by default. Redact known sensitive fields and review the logger configuration.

Problem

Request ids change between services

Solution

Accept an upstream x-request-id only after validating its format, and propagate it on outbound calls.

Limitations

Structured logs still need retention and access controls; they are not an audit trail by themselves.
High-cardinality fields such as raw URLs can increase storage cost and leak query data.

Log events, not sentences

A structured log is an event with fields, not a sentence that has to be parsed with a regular expression. requestId, route, status, and durationMs can be indexed and aggregated. Keep field names stable; renaming durationMs breaks dashboards and alerts even if the value is still present under another key.

{"level":30,"service":"orders-api","requestId":"6f1c...","method":"GET","route":"/orders/:id","status":200,"durationMs":12,"msg":"request completed"}

Redact at the source

Configure redaction in the logger, not only in the call sites. A new developer may log req.headers without thinking. The logger should remove authorization, cookies, and other sensitive fields deterministically. Never log request bodies, passwords, or tokens, even at debug level in production.

Control volume

Log one completion event per request at info level. Put detailed diagnostics at debug level and sample them under high traffic. High-cardinality fields such as full URLs, user input, and raw error messages can explode storage and expose personal data. Route errors through a handler that logs the stack server-side and returns a generic message to the client.

Verify the output

Expected outcome: every completed request produces one parseable JSON line whose requestId matches the x-request-id response header, and no line contains an authorization or cookie value. Confirm this with jq and a manual search for [REDACTED]. If either check fails, fix the logger before deploying.