阅读中文版本

Full-Text Search Vectors

Full-text search turns documents into normalised lexeme vectors so queries match words and stems rather than raw substrings, ranked by relevance.

Definition

Full-text search is the process of parsing text into lexemes, discarding stop words, applying stemming, and indexing the result so that language-aware queries can be ranked by relevance.

It is used for site search, article lookup, and any feature where users type words rather than exact patterns. Compared to simpler alternatives, this concept fits best when the trade-off it represents is the one the system actually cares about.

When to use it

A common use case is applying it at the boundary where the cost of getting it wrong is highest, and the scenario usually involves a concrete guarantee the system must uphold. Unlike an ad hoc approach, it makes that guarantee explicit, reviewable, and testable.

How it works

A stored search vector is a precomputed, indexed representation of the searchable columns, and it must be regenerated whenever those columns change or queries go stale.

Weighting lets titles count more than body text, and the ranking functions combine term frequency, document frequency, and normalisation into a relevance score.

Trade-offs and alternatives

Configuration is language-specific, so the wrong dictionary produces poor stemming; choose the language explicitly rather than relying on a default that may not fit the content. The main drawback is the extra complexity it introduces, so the benefit should clearly outweigh that cost before a team adopts it.

Common misconceptions

A persistent misconception is that full-text search is the same as using LIKE with wildcards. Another is that ranking always reflects what the user considers most relevant. For example, an engineer who assumes otherwise may design around a guarantee that does not actually exist, which only becomes obvious under load or failure.

Anti-patterns to avoid

Two recurring anti-patterns are to storing documents without a maintained search vector and to ranking by a single factor instead of a weighted combination. Both tend to surface as hard-to-debug production issues rather than obvious test failures. Related concepts include Advisory Locks, Cost-Based Query Optimization, Database Connection Multiplexing.