阅读中文版本

Batch Network Calls Without Losing Latency

Coalesce many small requests into fewer batched calls using a short scheduling window to cut round-trip overhead.

const pending = [];
let scheduled = false;
function enqueue(item) {
  pending.push(item);
  if (scheduled) return;
  scheduled = true;
  setTimeout(flush, 10);
}

async function flush() {
  const batch = pending.splice(0, pending.length);
  scheduled = false;
  const res = await api.batch(batch);
  // resolve each caller from the batched response
}

What this recipe does

Implement request coalescing with a small time window to reduce chatty network traffic.

A client makes thousands of tiny requests, and round-trip overhead dominates the total time.

Prerequisites

• A client and a batching-capable endpoint
• Control over the request layer
• Latency measurement

Procedure

1. Coalesce requests within a small window and batch them.

const pending = [];
let scheduled = false;
function enqueue(item) {
  pending.push(item);
  if (scheduled) return;
  scheduled = true;
  setTimeout(flush, 10);
}

Expected result: Many small requests are grouped into one batched call every 10 ms.

2. Flush in a single batched call.

async function flush() {
  const batch = pending.splice(0, pending.length);
  scheduled = false;
  const res = await api.batch(batch);
  // resolve each caller from the batched response
}

Expected result: One network round trip serves all coalesced requests.

How to confirm it worked

Run the commands above and confirm every expected result in order. If a step does not produce the stated output, use the troubleshooting notes before continuing. A passing run shows the number of network calls drops dramatically while end-to-end latency stays within the added window.

Troubleshooting

Latency-sensitive calls delayed — Keep the window tiny or let callers opt out of batching.

Batch too large — Cap batch size and flush early when the cap is reached.

Limitations

• Adds a small delay to every batched call.
• Requires the endpoint to accept batches.
• Error handling must map per-item failures.