阅读中文版本

Build a gRPC Service Mesh Client

gRPC Service Mesh Client — a complete walkthrough of inter-service rpc, covering channel resolver, load balancer, retry policy, and testing.

grpc_service_mesh/
├── src/index.ts
├── src/channel-resolver.ts
├── src/load-balancer.ts
├── tests/grpc_service_mesh.test.ts
└── package.json

// Channel resolver: inter-service rpc core
export function channelresolver(input: ChannelresolverInput): ChannelresolverResult {
  // Build the channel resolver that anchors inter-service rpc and defines the core data shape.
  return processChannelresolver(input);
}

// Load balancer — handles inter-service rpc Add the load balancer so
export class Loadbalancer {
  constructor(private opts: LoadbalancerOptions) {}
  async run(ctx: Context): Promise<LoadbalancerOutput> {
    return this.execute(ctx);
  }
}

// Retry policy: inter-service rpc stage
export const retrypolicy = (next: Next) => async (req: Request) => {
  const out = await transform(req);
  return next(out);
};

// Circuit breaker for inter-service rpc
export async function circuitbreaker(stream: AsyncIterable<Chunk>): Promise<Aggregate> {
  let acc = init();
  for await (const c of stream) acc = merge(acc, c);
  return acc;
}

// Intercept chain: inter-service rpc finalization
export function interceptchain(state: State): Result {
  validate(state);
  return finalize(state);
}

Project overview

This project builds a inter-service rpc system whose reason for existing is adding retry, load balancing, and circuit breaking to gRPC calls without modifying service code. The temptation in inter-service rpc is to bolt the interesting part onto a working prototype and discover the hard cases in production; this build inverts that order and treats a retry storm where every upstream is slow and retries multiply load exactly when the system is least able to handle it as the first thing designed, not the last.

You will implement Channel resolver, Load balancer, Retry policy, Circuit breaker, and Intercept chain, each as a module with one responsibility and a test that proves it in isolation. The inter-service rpc pipeline is then the wiring that composes them, and the integration test is where the composition is shown to survive a retry storm where every upstream is slow and retries multiply load exactly when the system is least able to handle it.

The walkthrough is written for a developer who has used a inter-service rpc system but has not had to make one honest. It favors explicit data shapes over clever abstractions, and it names the failure each design choice prevents rather than presenting the choice as a style preference.

Each phase ends with something observable — a channel resolver that returns the right shape, a load balancer that rejects bad input, an integration test that goes green — so you can stop at any point and still have a inter-service rpc system that works for the cases covered so far.

What you will build

The finished inter-service rpc system exposes five capabilities, each with a focused test:

• Channel resolver
• Load balancer
• Retry policy
• Circuit breaker
• Intercept chain

Channel resolver is the foundation that every later stage assumes; Load balancer is where inter-service rpc correctness lives; Retry policy handles a retry storm where every upstream is slow and retries multiply load exactly when the system is least able to handle it; Circuit breaker makes the system observable; and Intercept chain is the graceful-degradation layer that keeps overload from becoming corruption.

The build keeps the inter-service rpc logic in ordinary modules you can read in one sitting, and pulls in dependencies only for the parts that are genuinely tedious to hand-roll. That keeps the diff between the tutorial and a real codebase small, and keeps the failure modes visible in code rather than hidden behind a framework.

Architecture and data model

The system is organized around Channel resolver, Load balancer, and Retry policy, with Circuit breaker tying them together. Data flows as a set of upstream addresses resolved dynamically, with per-method policies applied through client interceptors.

Each component is a module with a pure interface where practical: the same input produces the same output, and side effects are pushed to the edges. That discipline is what makes the phases independently testable, because a test for Retry policy does not need to stand up Channel resolver and Load balancer to exercise the interesting branch.

The data model is deliberately explicit. A set of upstream addresses resolved dynamically, with per-method policies applied through client interceptors Keeping the shape written down — in a schema, a type, or a fixture — means a change to the shape is a visible diff rather than a silent runtime shift, and it means the integration test can assert against a concrete shape instead of guessing.

Configuration and code are kept apart: the configuration says what inter-service rpc should do, the modules say how. A reviewer can change a threshold or a policy without reading the implementation, which keeps the risky changes small and lets the diff stand as the review.

The failure path through Retry policy is designed before the happy path is finished, because a retry storm where every upstream is slow and retries multiply load exactly when the system is least able to handle it is the case that distinguishes a inter-service rpc system from a demo. Every component declares what it does when its inputs are wrong, and the integration test asserts each of those declarations.

Prerequisites and dependencies

Have these ready before you install anything, so the inter-service rpc setup is not blocked by a missing tool:

• Node.js 20 LTS or later
• Familiarity with inter-service rpc fundamentals
• A local environment to run the test suite

The runtime dependencies are chosen to keep the inter-service rpc datapath legible:

• @grpc/grpc-js (^1.12.0) — gRPC client runtime
• nice-grpc (^2.0.0) — Typed client wrappers

# setup for grpc-service-mesh
pnpm init
pnpm add @grpc/grpc-js nice-grpc
pnpm add -D typescript tsx vitest

Versions are pinned to a major range so the inter-service rpc install is reproducible; commit the lockfile. The channel resolver reads its operational parameters from the environment and validates them at startup, so a missing value fails the boot rather than the first inter-service rpc request.

Directory structure

Keep each inter-service rpc component in its own file so it can be tested without starting the others:

grpc_service_mesh/
├── src/index.ts
├── src/channel-resolver.ts
├── src/load-balancer.ts
├── tests/grpc_service_mesh.test.ts
└── package.json

Phase 1: Channel resolver

Build the channel resolver that anchors inter-service rpc and defines the core data shape.

// Channel resolver: inter-service rpc core
export function channelresolver(input: ChannelresolverInput): ChannelresolverResult {
  // Build the channel resolver that anchors inter-service rpc and defines the core data shape.
  return processChannelresolver(input);
}

After this phase the channel resolver works in isolation and is covered by a unit test. The channel resolver defines the data shape that every later stage assumes, so getting it right first means the later phases compile against a stable contract.

Treat the channel resolver as the single source of truth for the inter-service rpc data shape. Validate inputs at the boundary and reject early, so a malformed value never reaches Load balancer or Retry policy where its effect would be harder to localize.

Phase 2: Load balancer

Add the load balancer so inter-service rpc can adding retry, load balancing, correctly.

// Load balancer — handles inter-service rpc Add the load balancer so
export class Loadbalancer {
  constructor(private opts: LoadbalancerOptions) {}
  async run(ctx: Context): Promise<LoadbalancerOutput> {
    return this.execute(ctx);
  }
}

After this phase the load balancer works in isolation and is covered by a unit test. This is where inter-service rpc correctness lives: the load balancer is what makes adding retry, load balancing, and circuit breaking to gRPC calls without modifying service code true, and its test is the proof.

Keep the load balancer pure where you can. A function that takes an input and returns an output is trivial to test and impossible to mock wrong; a class with hidden state is the opposite. Push side effects to the caller and keep the load balancer about the transformation.

Phase 3: Retry policy

Wire the retry policy to handle a retry storm where without losing data.

// Retry policy: inter-service rpc stage
export const retrypolicy = (next: Next) => async (req: Request) => {
  const out = await transform(req);
  return next(out);
};

After this phase the retry policy works in isolation and is covered by a unit test. The retry policy is the difference between a inter-service rpc system that works on the happy path and one that survives a retry storm where every upstream is slow and retries multiply load exactly when the system is least able to handle it.

Design the failure before the success. Decide what retry policy does when a retry storm where every upstream is slow and retries multiply load exactly when the system is least able to handle it: reject, retry, shed, or degrade — and write that test first. A inter-service rpc system whose failure behavior is unspecified will pick one at random under pressure, and it is usually the wrong one.

Phase 4: Circuit breaker

Integrate the circuit breaker to make inter-service rpc observable and testable end to end.

// Circuit breaker for inter-service rpc
export async function circuitbreaker(stream: AsyncIterable<Chunk>): Promise<Aggregate> {
  let acc = init();
  for await (const c of stream) acc = merge(acc, c);
  return acc;
}

After this phase the circuit breaker works in isolation and is covered by a unit test. Observability is not a finishing touch; it is how you know the inter-service rpc system is doing what it claims in production.

Emit one structured event per meaningful action, with enough context to localize a failure without a second round-trip. The circuit breaker is what turns “something is slow” into “the load balancer is slow for inputs matching this shape,” which is the difference between an alert and an investigation.

Phase 5: Intercept chain

Finalize the intercept chain so the inter-service rpc system degrades gracefully under overload.

// Intercept chain: inter-service rpc finalization
export function interceptchain(state: State): Result {
  validate(state);
  return finalize(state);
}

After this phase the intercept chain works in isolation and is covered by a unit test. The intercept chain is what keeps overload from becoming corruption: when the inter-service rpc system is asked for more than it can do, it degrades in a defined way instead of failing in an undefined one.

Pick a degradation strategy for intercept chain — shed, queue, or backpressure — and make it explicit in code and in a test. Leaving the behavior to chance is how a inter-service rpc traffic spike becomes a data loss incident.

Testing

Each phase has a focused test: channel resolver, load balancer, and retry policy are exercised in isolation, then an integration test drives the full inter-service rpc pipeline and asserts the a retry storm where case is handled.

Start with unit tests that pin each component’s contract: the channel resolver returns the documented shape, the load balancer transforms correctly, and the retry policy fails in the documented way. These tests are fast and they localize regressions to the component that broke.

Then add an integration test that composes the components and drives a realistic inter-service rpc scenario end to end. Assert the happy path first, then deliberately trigger a retry storm where every upstream is slow and retries multiply load exactly when the system is least able to handle it and assert the system degrades rather than corrupts. Because the components are pure where practical, the integration test stays deterministic rather than flaky.

A inter-service rpc suite must also prove the negative: bad input is rejected at the boundary, overload is shed rather than queued indefinitely, and a partial failure does not leave the system in an inconsistent state. These are the tests that catch the regressions a happy-path suite never would.

The entire inter-service rpc suite runs with no external service; every dependency is either in-process or stubbed, so every case is reproducible in CI. That is what lets the retry policy failure-mode assertions live as tests rather than as a manual checklist that drifts out of date.

# run the inter-service rpc test suite
pnpm vitest run

Deployment and operations

Run at least two inter-service rpc instances behind a load balancer and share state in Redis. Set resource limits so inter-service rpc overload surfaces as backpressure rather than failure.

Run at least two stateless instances behind a load balancer, and move any shared state into a shared store so the inter-service rpc behavior is global rather than per instance. Size limits so the system sheds load before it runs out of memory, because a inter-service rpc system that OOMs under load fails in a way that is hard to reason about.

Drain on deploy: stop accepting new work, let in-flight inter-service rpc operations finish, then exit. Because the system holds no state that cannot be reconstructed, a rollout is a matter of cycling instances one at a time while watching the error rate.

When a dependency is unavailable, decide explicitly whether the inter-service rpc system fails closed or degrades. Leaving that choice to chance is how a dependency outage becomes a silent data loss, so make the choice visible in code and in a test.

Security considerations

Validate all inter-service rpc inputs at the boundary, reject oversized payloads before buffering them, and keep secrets in the environment rather than the repository.

The inter-service rpc system is a choke point, so it is the right place to reject malformed input before it reaches load balancer or retry policy. Validate the shape and the size at the edge, and let the inner components assume what they receive is well-formed.

Never log secrets or full payloads; log enough to localize a failure without recording data that should not be on disk. Treat any external call as untrusted and validate its response before using it, because a inter-service rpc system that trusts its inputs blindly is one malformed response away from corruption.

Keep dependencies pinned and run an audit in CI. A transitive dependency with a known vulnerability is still a vulnerability in your inter-service rpc system, and the lockfile is what makes the audit meaningful.

Observability

Emit one structured event per inter-service rpc operation with the inputs’ shape, the outcome, and a correlation id. Track error rate and latency per component and alert on a step change rather than a single spike.

The correlation id is the thread that ties a user-visible symptom to the inter-service rpc decision and to the component log entry for the same operation. Without it, an investigation starts from a timestamp and a guess.

Watch the retry policy failure rate separately from the overall error rate. A jump in retry policy failures usually means a retry storm where every upstream is slow and retries multiply load exactly when the system is least able to handle it, not a bug in load balancer, and the two have different owners and different fixes.

Chart the share of inter-service rpc operations that shed, queue, or succeed. A change in the mix points at the cause faster than an aggregate latency can, because each class has a different root.

Troubleshooting

A retry storm where every upstream is slow and retries multiply load exactly when the system is least able to handle it. Reproduce with a minimal fixture, then add a guard test so the inter-service rpc system rejects the condition loudly.

Performance regresses under inter-service rpc load. Profile with the sampling analyzer, locate the hot path in load balancer, and add a targeted benchmark to lock the regression.

Keep the inter-service rpc dashboards separate from dependency dashboards. When an operation fails, the correlation id lets you decide in seconds whether the fault is channel resolver, load balancer, or retry policy, instead of guessing from a timeout.

Limitations and next steps

• Single-region; multi-region replication is out of scope.
• The channel resolver is in-process and not horizontally shared by default.
• No auth layer; assume deployment behind an edge that enforces identity.

The inter-service rpc system is single-region by design; extending to multi-region means making channel resolver shareable across regions, which is a real change to the consistency model rather than a config flag. The retry policy is in-process, which is correct for one instance but must move to a shared store before scaling out.

There is no authentication or authorization layer; the system assumes it is deployed behind an edge that enforces identity. Adding one is straightforward but out of scope for this build, and doing it here would distract from the inter-service rpc correctness that is the point.

The natural next step, once the inter-service rpc pipeline is trusted in production, is to add a short-TTL cache for idempotent reads and a richer circuit breaker that exports histograms rather than counters. Both are additive and neither changes the contracts this build establishes.