Embeddings
Embeddings map discrete items to dense vectors so that geometric distance in the vector space reflects semantic similarity.
Definition
An embedding is a learned dense vector representation of an item — a word, sentence, image, or user — such that items with similar meaning lie close together under a chosen metric, usually cosine similarity or dot product. A model trained on co-occurrence or contrastive objectives places related items near each other, turning semantic search, clustering, and recommendation into nearest-neighbour problems over vectors.
Context and scope
Used whenever text, images, or behaviour must be compared by meaning rather than exact match: semantic search, deduplication, retrieval-augmented generation, and recommendations.
Common misconceptions
- Higher-dimensional embeddings are always better — beyond a point, extra dimensions add cost and noise; task fit matters more than raw size
- Cosine similarity is universally correct — cosine ignores magnitude and suits normalised text, while dot product or Euclidean may fit other tasks
- An embedding model can be swapped silently — vectors from different models live in incompatible spaces; mixing them corrupts all distances
Anti-patterns
- Recomputing embeddings on every request instead of caching them by content hash
- Comparing embeddings produced by two different models without re-embedding the whole corpus
- Treating the raw vector as human-readable output and exposing it in user interfaces
Related concepts
Authoritative source
What it is
An embedding model converts an item into a fixed-length array of floats. Training shapes the space so that distance encodes meaning: synonyms cluster, unrelated concepts separate, and arithmetic on vectors can capture relations. Because the output is numeric, once items are embedded a whole toolkit of geometric algorithms — clustering, classification, and nearest-neighbour search — becomes available.
Why it matters
Keyword search fails on paraphrase and multilingual text. Embeddings let a query about “reduce cloud spend” match a document about “trimming infrastructure cost” with no shared terms. This is the substrate for semantic retrieval, and it is why retrieval-augmented generation can ground a language model in private documents it was never trained on.
How to use it well
Choose the model for the task and the language mix, keep one model per index, and store vectors next to the source identifiers. Normalise vectors when you intend to rank by cosine similarity, cache embeddings by content hash, and re-embed the entire corpus whenever the model changes. Pair the vector index with metadata filters so results stay relevant and lawful.