Micro-Batching

Micro-batching groups arriving items into small batches over a short window, amortizing per-request overhead — network round trips, transaction commits — at the cost of window latency.

performance

Definition

Micro-batching collects items for a fixed interval or size threshold, then processes the group in one operation. Per-item fixed costs — RPC setup, transaction begin and commit, index lookups amortized over the batch — drop dramatically; each item gains the window duration in latency. It is the throughput-latency dial at the heart of stream processors and write paths.

Context and scope

Applied in Spark Structured Streaming intervals, Kafka producer batch configuration, database bulk inserts, and logging pipelines aggregating appends.

Common misconceptions

Batching always improves throughput — past the point where batches saturate downstream capacity, larger batches only add latency
Micro-batching removes the need for backpressure — queues between producers and batchers grow without backpressure signaling and memory limits
Batch size is the only knob — the trigger policy matters: size, time, or count, plus what happens on flush failure and partial retry

Anti-patterns

Using fixed long windows for low-traffic streams where every item waits the full window for a near-empty batch
Retrying whole batches on single-item failure and repeatedly poisoning the group
Batching without item-level result attribution, losing per-item failure reporting and retry semantics

Related concepts

Authoritative source

https://spark.apache.org/docs/latest/streaming-programming-guide.html

What it is

Per-operation overhead dominates small writes: one network round trip, one commit, one fsync. Grouping N items into one operation divides those costs by N. The price is simple: each item waits for the window to close.

How it works

A collector accumulates items and flushes on size threshold, timer expiry, or explicit request. Kafka producers batch by partition with linger time; database drivers batch inserts into multi-row statements; stream engines close processing windows every N milliseconds. Failure handling splits: whole-batch retry, per-item segregation of failures, or dead-letter the group.

When to use it

High-rate write paths — logs, metrics, telemetry ingestion — and stream analytics where per-event overhead outweighs freshness. Choose windows by latency budget: sub-second windows behave nearly like pure streaming.

Pitfalls

Low-traffic streams on fixed windows mean every item pays full window latency for tiny batches — size triggers with a time cap behave better. Batch amplification on failure: one bad item can block a batch repeatedly; extract and dead-letter it. Downstream limits — statement size, payload caps — must shape the batch bound.