Text Embeddings
A text embedding maps text to a dense vector that encodes meaning, letting machines compare and cluster language by proximity in vector space.
Definition
A text embedding is a fixed-length numeric vector produced by a model from an input text, positioned so that semantically similar texts lie near each other under a distance metric.
Context and scope
It is the representation layer under semantic search, clustering, recommendation, and retrieval-augmented generation.
Common misconceptions
- embedding dimensions are interchangeable between models
- similarity scores are calibrated probabilities
Anti-patterns
- comparing embeddings produced by different models
- assuming a similarity threshold generalises across datasets
Related concepts
Related examples
Authoritative source
Definition
A text embedding is a fixed-length numeric vector produced by a model from an input text, positioned so that semantically similar texts lie near each other under a distance metric.
It is the representation layer under semantic search, clustering, recommendation, and retrieval-augmented generation. Compared to simpler alternatives, this concept fits best when the trade-off it represents is the one the system actually cares about.
When to use it
A common use case is applying it at the boundary where the cost of getting it wrong is highest, and the scenario usually involves a concrete guarantee the system must uphold. Unlike an ad hoc approach, it makes that guarantee explicit, reviewable, and testable.
How it works
An embedding model fixes the vector space, so vectors from two different models are not comparable and a corpus must be re-embedded whenever the model changes.
Cosine similarity is commonly used on normalised vectors, and the resulting score is a relative ranking signal rather than a calibrated probability.
Trade-offs and alternatives
Domain and language coverage vary, so effectiveness depends on choosing a model trained for the content’s language and vocabulary. The main drawback is the extra complexity it introduces, so the benefit should clearly outweigh that cost before a team adopts it.
Common misconceptions
A persistent misconception is that embedding dimensions are interchangeable between models. Another is that similarity scores are calibrated probabilities. For example, an engineer who assumes otherwise may design around a guarantee that does not actually exist, which only becomes obvious under load or failure.
Anti-patterns to avoid
Two recurring anti-patterns are to comparing embeddings produced by different models and to assuming a similarity threshold generalises across datasets. Both tend to surface as hard-to-debug production issues rather than obvious test failures. Related concepts include Tokenization and Subword Units, The Context Window, Prompt Injection.