Data Contracts
A data contract is an enforceable agreement between a data producer and its consumers covering schema, semantics, quality, freshness, and change management.
Definition
A data contract specifies what a data producer will deliver and what consumers can rely on: field names and types, meaning and units, key uniqueness, null rules, freshness, volume expectations, ownership, and the process for breaking changes. It is versioned and tested in CI rather than documented informally.
Context and scope
Data pipelines fail socially more often than technically: a producer renames a column, changes a timezone, or stops emitting an event, and downstream dashboards break without warning. Contracts make those dependencies explicit and reviewable.
Common misconceptions
- A data contract is just a JSON Schema attached to an event.
- Contracts are only necessary for external partners, not for internal teams.
Anti-patterns
- Publishing a contract that is never validated in the producer's deployment pipeline.
- Treating a breaking schema change as a consumer problem after it ships.
Related concepts
Related examples
Authoritative source
How it works
A contract has three layers. The structural layer defines schema and types. The semantic layer defines meaning: what revenue includes, which currency, whether timestamps are UTC, and what a null means. The operational layer defines service levels: freshness, completeness, uniqueness, and incident contacts. A good contract also states how changes are announced and how much notice consumers get.
Enforcement turns a document into a guarantee. The producer validates output against the contract before publishing. CI runs compatibility checks against the previous version. The platform monitors freshness and volume, and alerts the owner when a promise is missed. Consumers can then build against a stable interface instead of reverse-engineering files.
When to use it
Use contracts for datasets that cross team boundaries, feed financial or regulatory reporting, or power shared dashboards. Start with the highest-impact datasets rather than every table. A contract is most valuable where a silent change would be expensive and where a producer can realistically be held to the promise.
Trade-offs
Contracts add review and coordination overhead, and they can slow a producer that needs to move quickly. Over-specifying fields makes evolution painful; under-specifying leaves consumers exposed. The contract must be owned by someone with authority to enforce it, otherwise it becomes stale documentation. Schema-only contracts miss the semantic failures that cause most incidents.
Example and misconception
An events team promises order_created with order_id unique, amount in minor currency units, UTC timestamps, and delivery within five minutes. A producer change that switches to decimal units fails the contract test before deployment, and the consumer never sees corrupted revenue.
The misconception is that a schema registry subscription is a contract. A registry checks structure; it does not promise freshness, meaning, or that anyone will respond when data stops.