头部与尾部采样

概念定义

你存不下每条 trace，但 trace 开始时又无法知道它是否有意思。头部采样早猜、便宜；尾部采样晚判、带着全貌。

工作机制

收集层按 trace id 聚合持有 span 直到完成或超时；策略评估整条：时长阈值、错误标志、属性匹配。保留率按策略分别设定——错误 100%、超 1 秒 100%、其余 5%。组合决策时策略评估顺序有讲究。

使用场景

凡“错误与离群 trace 必须存在于样本”的场景：生产诊断、用 trace 校验 SLI、exemplar 的链接目标。成本下限优先、完整性次要时用头部采样。

常见误区

尾部采样不是省存储而是省摄取成本与隐私暴露。它要求源侧一个不丢：到不了采样层的 span 任何策略都救不回。按连接而非 trace id 哈希的负载均衡会拆散 trace、腐蚀决策；收集器变成扩容单位——它处理的是未采样流，容量要按它观察的生产流量配；长寿命 trace 会活过缓冲区，截止冲刷策略决定它们的命运。健康 trace 保留率为零会丢掉“让异常可解释”的基线。

代码示例

import { NodeSDK } from '@opentelemetry/sdk-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';
import { trace, context, SpanStatusCode } from '@opentelemetry/api';

const sdk = new NodeSDK({
  serviceName: 'orders-service',
  traceExporter: new OTLPTraceExporter({
    url: process.env.OTLP_ENDPOINT ?? 'http://collector:4318/v1/traces',
  }),
  instrumentations: [getNodeAutoInstrumentations()],
});

await sdk.start();
const tracer = trace.getTracer('orders', '1.0.0');

async function processOrder(orderId: string): Promise<void> {
  return tracer.startActiveSpan('order.process', async (span) => {
    span.setAttribute('order.id', orderId);
    try {
      await tracer.startActiveSpan('order.validate', async (child) => {
        await new Promise((r) => setTimeout(r, 10));
        child.setAttribute('validation.rules', 12);
        child.end();
      });

      await tracer.startActiveSpan('order.charge', async (child) => {
        await new Promise((r) => setTimeout(r, 25));
        child.recordException(new Error('card declined simulated'));
        child.setStatus({ code: SpanStatusCode.ERROR });
        child.end();
      });

      span.setStatus({ code: SpanStatusCode.OK });
    } catch (err) {
      span.recordException(err as Error);
      span.setStatus({ code: SpanStatusCode.ERROR });
    } finally {
      span.end();
    }
  });
}

await processOrder('ord-7');
await sdk.shutdown();

use tracing::{debug, error, info, instrument, warn};
use tracing_subscriber::{fmt, layer::SubscriberExt, util::SubscriberInitExt, EnvFilter};

#[instrument(name = "order_flow", skip(payload))]
fn process_order(order_id: u64, payload: &str) -> u64 {
    debug!(bytes = payload.len(), "payload received");
    let total = payload.len() as u64 * order_id % 97;
    info!(total, "order total computed");
    total
}

fn init_tracing() {
    let filter = EnvFilter::try_from_default_env()
        .unwrap_or_else(|_| EnvFilter::new("info,order_service=debug"));

    let is_prod = std::env::var("APP_ENV")
        .map(|v| v == "production")
        .unwrap_or(false);

    let registry = tracing_subscriber::registry().with(filter);

    if is_prod {
        registry
            .with(fmt::layer().json().flatten_event(true))
            .init();
    } else {
        registry
            .with(fmt::layer().pretty().with_target(false))
            .init();
    }
}

fn main() {
    init_tracing();

    let span = tracing::info_span!("startup", version = env!("CARGO_PKG_VERSION"));
    let _guard = span.enter();

    info!("service starting");
    warn!("cache warmup skipped");

    let total = process_order(42, "line-item;quantity=3");
    error!(total, "downstream settlement slow");

    drop(_guard);
    info!("shutdown complete");
}