阅读中文版本

Build a Mini Package Manager with Semver Resolution

A toy package manager that resolves semver ranges into a deterministic lockfile, fetches tarballs from a local registry, and installs verified packages into node_modules.

mini-pkg/
├── bin/mini-pkg.js
├── src/
│   ├── semver.js
│   ├── resolver.js
│   ├── lockfile.js
│   ├── fetcher.js
│   └── installer.js
├── registry/
│   └── index.json
├── test/
│   ├── semver.test.js
│   └── resolver.test.js
└── package.json

function parseRange(range) {
  return range.split('||').map((alt) =>
    alt.split(' ').map((comp) => {
      const m = /^(>=|<=|>|<|=|~|\^)?(\d+|x|\*)(?:\.(\d+|x|\*))?(?:\.(\d+|x|\*))?$/.exec(comp.trim());
      if (!m) throw new Error('invalid comparator: ' + comp);
      return { op: m[1] || '=', major: m[2], minor: m[3], patch: m[4] };
    }),
  );
}

function resolve(manifest, registry) {
  const constraints = new Map();
  const queue = [{ name: manifest.name, deps: manifest.dependencies, path: [] }];
  while (queue.length) {
    const item = queue.shift();
    for (const [name, range] of Object.entries(item.deps ?? {})) {
      const entry = constraints.get(name) ?? [];
      entry.push({ range, requester: item.name, path: item.path.concat(name) });
      constraints.set(name, entry);
    }
  }
  return pickVersions(constraints, registry);
}

async function fetchAndVerify(pkg) {
  const tarball = await fs.readFile(registryTarball(pkg));
  const digest = crypto.createHash('sha256').update(tarball).digest('hex');
  if (digest !== pkg.integrity) {
    throw new Error('integrity mismatch for ' + pkg.name + '@' + pkg.version);
  }
  return tarball;
}

for (const pkg of lockfile.packages) {
  const dest = path.join(root, 'node_modules', pkg.name);
  await fs.mkdir(dest, { recursive: true });
  await tar.extract({ file: cachePath(pkg), cwd: dest, strip: 1 });
}

Project overview

npm resolves thousands of packages in milliseconds and everyone treats it as ambient infrastructure. Building a miniature forces the interesting questions into the open: how does a range become one version, why do lockfiles exist, what makes an install reproducible, and what does integrity verification actually protect?

The scope is a local registry, real semver semantics, deterministic resolution, and a lockfile-driven installer — small enough to finish, complete enough that every design decision has a visible consequence.

The finished tool delivers five capabilities:

• Semver range parsing and satisfaction for caret, tilde, exact, wildcard, hyphen, and OR-set ranges, with prerelease semantics.
• Deterministic resolution producing the same lockfile for the same manifest regardless of run order.
• Conflict detection when two ranges on one package have no intersecting version, with a path explaining both requesters.
• Tarball fetch with integrity: every tarball is SHA-256 verified against the lockfile before extraction.
• A hoisted or nested install layout, honoring the lockfile exactly.

The architecture is four modules in a line: a resolver walks the dependency graph breadth-first, collecting version ranges per package and choosing the highest version satisfying every range seen; a lockfile module serializes the resolved graph deterministically; a fetcher downloads and verifies tarballs against recorded digests; and an installer materializes the tree from the lockfile alone — which is what makes installs reproducible.

Setup

Verify the prerequisites before starting: Node.js 20 LTS or later, an understanding of semantic versioning ranges and lockfiles, and familiarity with tar archives and streaming extraction.

The dependency footprint is one runtime package plus the test runner:

Create the project and install:

mkdir mini-pkg && cd mini-pkg
npm init -y
npm install tar@^7.0.0
npm install -D vitest@^2.1.0

Configure the environment in package.json — the bin entry, the test script, and the registry location via the MINI_PKG_REGISTRY environment variable defaulting to ./registry:

{
  "name": "mini-pkg",
  "bin": { "mini-pkg": "bin/mini-pkg.js" },
  "scripts": { "test": "vitest run" }
}

The directory structure mirrors the pipeline: semver.js and resolver.js are pure logic with no I/O, lockfile.js and fetcher.js own their file access, and the registry is a plain directory of tarballs with an index.json you fully control.

Phase 1: Semver engine

The semver module parses versions and ranges — caret, tilde, hyphen, comparator sets — and implements satisfaction checks with prerelease semantics.

// src/semver.js
export function parseRange(range) {
  return range.split('||').map((alt) =>
    alt.split(' ').map((comp) => {
      const m = /^(>=|<=|>|<|=|~|\^)?(\d+|x|\*)(?:\.(\d+|x|\*))?(?:\.(\d+|x|\*))?$/.exec(comp.trim());
      if (!m) throw new Error('invalid comparator: ' + comp);
      return { op: m[1] || '=', major: m[2], minor: m[3], patch: m[4] };
    }),
  );
}

export function satisfies(version, range) {
  const v = parseVersion(version);
  return parseRange(range).some((alt) => alt.every((c) => comparatorHolds(v, c)));
}

The expansion rules are the phase’s real content. Caret ^1.2.3 allows changes that do not modify the leftmost nonzero digit: >=1.2.3 <2.0.0, with ^0.2.3 meaning >=0.2.3 <0.3.0 because the zero major makes minor the stabilizer. Tilde ~1.2.3 allows patch-level changes: >=1.2.3 <1.3.0. Wildcards — 1.x, *, 1.2.x — are ranges, not versions, and expand accordingly. OR-sets split on || and satisfy if any alternative holds.

The subtlety that separates a correct engine from a nearly-correct one is prerelease exclusion: a plain range like >=1.0.0 must reject 1.5.0-beta even though the version sorts above the floor, because opting into prereleases requires the range itself to carry a prerelease with the same tag — the rule npm adopted after years of beta builds appearing in production installs. The test table pins it explicitly.

Acceptance for this phase: semver tests pass the caret and tilde expansion table, x-range wildcards, hyphen ranges, comparator OR sets, and prerelease exclusion from plain ranges.

Phase 2: Deterministic resolver

The resolver walks the graph collecting constraints, picks the highest version satisfying all ranges for each package, and detects unsatisfiable conflicts with the requester chain.

// src/resolver.js
export function resolve(manifest, registry) {
  const constraints = new Map();
  const queue = [{ name: manifest.name, deps: manifest.dependencies, path: [] }];
  const seen = new Set();

  while (queue.length) {
    const item = queue.shift();
    for (const [name, range] of Object.entries(item.deps ?? {})) {
      const entry = constraints.get(name) ?? [];
      entry.push({ range, requester: item.name, path: item.path.concat(name) });
      constraints.set(name, entry);

      if (!seen.has(name)) {
        seen.add(name);
        const version = highestSatisfying(registry, name, [range]);
        const manifestFor = registry.manifest(name, version);
        queue.push({ name, deps: manifestFor.dependencies, path: item.path.concat(name) });
      }
    }
  }
  return pickVersions(constraints, registry);
}

function pickVersions(constraints, registry) {
  const resolved = {};
  for (const name of [...constraints.keys()].sort()) {
    const ranges = constraints.get(name).map((c) => c.range);
    const version = highestSatisfying(registry, name, ranges);
    if (!version) {
      const requesters = constraints.get(name).map((c) => `${c.requester} -> ${c.range}`);
      throw new Error(`conflict on ${name}: no version satisfies\n  ` + requesters.join('\n  '));
    }
    resolved[name] = { version, requesters: constraints.get(name) };
  }
  return resolved;
}

The resolver does not pick versions per package in isolation: it collects every range that anything in the graph requests first, then selects the highest version satisfying the intersection. This two-phase shape is what makes conflicts detectable — when two requesters ask for disjoint ranges (left@^1.0.0 and right@^2.0.0 on the same package), the failure names both chains, which is the single most valuable debugging feature a resolver can have.

The breadth-first walk with a seen set terminates on cyclic graphs — a package already expanded contributes its constraints again through a second requester, but is never re-walked. And determinism is enforced, not assumed: everything is sorted before serialization, and the test resolves the same fixture twice and compares lockfiles byte for byte — insertion order leaking into output is the classic resolver bug, and this is the test that catches it.

Acceptance for this phase: resolving a fixture graph twice yields identical lockfiles, and a planted conflict reports both requesters and the two ranges that cannot intersect.

Phase 3: Lockfile and integrity

The lockfile module serializes the resolved graph sorted by name with exact versions and tarball digests, and the fetcher verifies every tarball against its recorded SHA-256.

// src/fetcher.js
import { createHash } from 'node:crypto';
import { readFile } from 'node:fs/promises';

export async function fetchAndVerify(pkg) {
  const tarball = await readFile(registryTarball(pkg));
  const digest = createHash('sha256').update(tarball).digest('hex');
  if (digest !== pkg.integrity) {
    throw new Error('integrity mismatch for ' + pkg.name + '@' + pkg.version +
      ' expected ' + pkg.integrity + ' got ' + digest);
  }
  return tarball;
}

// src/lockfile.js
export function serialize(resolved, registry) {
  const packages = Object.keys(resolved).sort().map((name) => {
    const { version } = resolved[name];
    return {
      name, version,
      integrity: registry.digest(name, version),
      dependencies: sortedDeps(registry.manifest(name, version).dependencies),
    };
  });
  return JSON.stringify({ lockfileVersion: 1, packages }, null, 2) + '\n';
}

The lockfile is the contract: it pins exact versions and integrity digests, and the installer reads only the lockfile, never the manifest. That one-way dependency is what turns an install into a reproduction — same lockfile, same bytes on disk, regardless of what the registry looks like today.

Integrity verification happens before a single entry is extracted: the tarball is hashed whole, compared to the lockfile digest, and only then passed to extraction, because an integrity error after partial extraction is a corrupted tree pretending to be an install. The tamper test flips one byte in a registry tarball and requires the failure to name the package — the error message carrying both the expected and actual digest is what makes the failure diagnosable rather than merely loud.

Acceptance for this phase: the lockfile is byte-stable across runs, and tampering with one registry tarball byte makes install fail naming the package before any extraction happens.

Phase 4: Install layout

The installer materializes the node_modules tree from the lockfile, supporting hoisted and nested layouts, extracting verified tarballs with the tar stream API.

// src/installer.js
import { mkdir } from 'node:fs/promises';
import path from 'node:path';
import tar from 'tar';

export async function install(root, lockfile, { layout = 'hoisted' } = {}) {
  for (const pkg of lockfile.packages) {
    const dest = layout === 'nested' && needsNesting(pkg, lockfile)
      ? nestedPath(root, pkg)
      : path.join(root, 'node_modules', pkg.name);
    await mkdir(dest, { recursive: true });
    await tar.extract({ file: cachePath(pkg), cwd: dest, strip: 1 });
  }
}

The hoisted layout places every package flat under one node_modules, which is what npm did for a decade: it maximizes sharing and breaks when two requesters need incompatible versions of the same package. The nested layout creates per-parent node_modules directories for exactly the conflicting packages, which is always correct and duplicates nothing that does not conflict — and the choice is a flag because both layouts are legitimate teaching material and real-world behavior.

The strip: 1 on extraction handles the tarball convention of a single top-level package/ directory. And the installer never re-runs resolution: it reads the lockfile, fetches each tarball through the verified path, and extracts — the test deletes the tree, installs from the lockfile alone into a fresh directory, and compares both trees byte for byte. If the installer had peeked at ranges again, drift would be possible and the test would fail.

Acceptance for this phase: a fixture project installs and its main file resolves through require, a conflicting pair nests correctly in the nested layout, and a fresh directory installed from the lockfile alone matches the first byte for byte.

Design trade-offs

Highest-satisfying over first-found or lowest-satisfying. Lowest-satisfying maximizes stability; highest-satisfying matches ecosystem convention and gets bug fixes by default. The choice is visible in the resolver tests either way, and the conflict error is identical under both.

Constraint collection over greedy selection. Greedy selection — picking a version when first encountered — resolves faster and produces different results depending on traversal order, which breaks determinism and makes conflicts order-dependent. Collecting first, choosing second, costs one pass and buys both properties.

Whole-file hashing over streamed incremental hashing. Streamed hashing avoids buffering large tarballs; whole-file hashing is simpler and the local registry has no large tarballs. The security property is identical.

Lockfile-only installs over re-resolution. Re-resolving on every install is simpler and gives up reproducibility — the property the entire tool exists to demonstrate. The lockfile is the product; the manifest is its input.

Testing strategy

Run npx vitest. Semver tests run the caret, tilde, wildcard, hyphen, and OR-set tables plus prerelease rules — table-driven, so each rule is one row with a version, a range, and an expected boolean. Resolver tests resolve fixture graphs twice for byte-identical lockfiles and assert the conflict error carries both requester paths, which is the test that keeps the debugging feature honest.

Fetcher tests tamper with a tarball byte and require the integrity failure to name the package with both digests. Installer tests install a fixture project into temp directories from the lockfile only and compare trees — including the nested layout for a version-conflict pair, where a hoisting bug would silently install the wrong version and only the tree comparison catches it. The registry fixtures live in the repository, so every failure mode — conflict, tamper, layout — is reproducible on demand and in CI.

The fixture registry itself is worth building deliberately rather than generating: a dozen packages with hand-authored manifests where the interesting shapes live — a diamond dependency (two requesters, one shared transitive target), a version conflict pair, a cycle (A depends on B depends on A), and a package with no dependencies at all. Each fixture exists to make one resolver behavior observable, and the fixture names document the test matrix the way the chart fixtures document the bug inventory.

Troubleshooting playbook

Lockfiles differ between runs. Two resolutions of the same manifest produce different bytes. The cause is insertion-order leakage: a Map iterated in insertion order, or dependency keys serialized in manifest order rather than sorted. The fix is sorting packages and every dependency key before serialization, and the double-resolve test is the tripwire.

Conflict error names no requesters. A conflict surfaces as “no version satisfies” with no explanation of who asked. The cause is constraint collection without provenance — the ranges arrive, the requesters do not. The fix is recording the requester path at collection time, before version selection, so the failure can print both chains; retrofitting provenance after selection is impossible because the information is gone.

Prerelease versions satisfy plain ranges. 1.5.0-beta installs for a range of ^1.2.0. The cause is satisfaction implemented as pure ordering, ignoring the opt-in rule. The fix is the comparator guard: a comparator without a prerelease component rejects prerelease versions unless the range carries the same tag — the exact rule the phase 1 test table pins.

Extending the project

The registry abstraction is the growth surface: a directory-and-index registry is the whole implementation today, and an HTTP registry is a fetcher swap that the lockfile format already supports because digests are transport-independent. Peer dependencies are a resolver feature with real design surface — they change the constraint model from per-package to per-context, and the known limitations name them as unmodeled rather than approximated. Optional dependencies, platform variants, and an audit command over the lockfile’s integrity fields are all lockfile-driven features that never touch resolution.

For deployment, use it as a teaching artifact and a testbed for resolution experiments: the local registry is a directory you fully control, which makes deterministic resolution, conflicts, and integrity failures reproducible on demand. Point the registry path at a checked-in fixture tree for CI, and never point it at anything production-critical. Two practices from real package management are worth importing into how you use the tool. First, lockfile diffs belong in code review: a resolution change is a dependency change, and the lockfile’s sorted, exact-version format makes the diff readable — a version bump is one line, a conflict resolution is a handful, and a transitive dependency silently jumping a major version is a line a reviewer can catch before it ships. Reviewing lockfile diffs is the cheapest supply-chain control a team can adopt, and this tool’s deterministic serialization is what makes the diffs meaningful rather than noise.

Second, integrity digests are a supply-chain control, not a formality: the tamper test demonstrates the property, and the operational version of the same idea is pinning and verifying — a registry whose tarball for [email protected] no longer matches the recorded digest is a registry that has been republished, accidentally or otherwise, and the install refusing to proceed is the correct behavior. The mini registry makes this experiment safe to run: swap a byte in a fixture tarball, watch the failure name the package, and restore it, all in a directory you own. That muscle memory is what operators need when a real registry advisory lands and the question is whether anything in the tree changed.