Lakehouse
A lakehouse adds warehouse-style tables, transactions, and governance on top of cheap object storage, so one copy serves BI, streaming, and machine learning.
Definition
A lakehouse is an architecture that stores data as open files in object storage while a table format and catalog provide ACID transactions, schema enforcement, time travel, and efficient metadata. It aims to combine the low cost and openness of a data lake with the reliability and query performance of a warehouse.
Context and scope
The lakehouse pattern became practical as open table formats such as Delta Lake, Apache Iceberg, and Apache Hudi matured, and as cloud query engines learned to use their metadata for planning.
Common misconceptions
- A lakehouse is just a data lake with a SQL engine, with no transactional guarantees.
- Open table formats eliminate the need for catalog governance and data modeling.
Anti-patterns
- Writing many tiny files without compaction, then blaming the table format for slow queries.
- Letting every team write directly to the same table without contracts or ownership.
Related concepts
Related examples
Authoritative source
How it works
A lakehouse keeps the data files, usually Parquet, in object storage. A table format adds a metadata layer with a transaction log or manifest list. Writers commit atomically by publishing a new metadata version; readers resolve a consistent snapshot from it. That enables ACID behavior, schema enforcement, partition evolution, row-level deletes, and time travel without copying the whole dataset.
A catalog supplies names, locations, and access control. Query engines read the metadata to prune files by partition and statistics, then scan only the required columns. Background jobs compact small files and rewrite manifests so planning does not degrade as ingestion continues.
When to use it
Use a lakehouse when one copy of data must serve SQL analytics, streaming ingestion, and machine learning, and when open formats or avoiding vendor lock-in matter. It is a strong default for new analytical platforms on cloud object storage. Keep a separate operational database for transactional writes; the lakehouse is not a replacement for a low-latency OLTP store.
Trade-offs
The lakehouse reduces duplication and storage cost, but it shifts complexity into table maintenance: compaction, snapshot expiration, file sizing, and catalog consistency. Query latency can be higher than a tuned warehouse for small, highly concurrent queries. Governance is not automatic; without ownership and contracts, the lakehouse becomes a fast lake full of untrusted tables.
Example and misconception
A stream writes events every minute into an Iceberg table partitioned by date. Analysts query the latest snapshot, while a nightly job compacts files and expires old snapshots. A reader never sees a half-written batch because commits are atomic.
The common misconception is that open files alone make a lakehouse. Without a table format, catalog, and maintenance discipline, you still have a lake. The format provides the mechanics; modeling, quality checks, and ownership provide the trust.