Derives synonyms from how words actually behave inside documents: how often they co-occur, how closely they sit, and whether they appear in titles or anchors, then combines those signals into a single decision.
Patent Overview
- Inventor
- Steven D. Baker
- Assignee
- Google LLC
- Filed
- 2007-09-05
- Granted
- 2011-02-15
- Application Number
- US 11/899,937
The Challenge
Thesauri Are Brittle. The Web Is The Better Corpus.
Building synonyms from a static thesaurus produces a vocabulary that lags real usage, misses domain-specific language, and ignores how often two words actually substitute for each other in the wild. The web already encodes that information at massive scale. The question is how to read it cleanly. Naive approaches that grab any word pair appearing together produce floods of false positives because document-level co-occurrence is noisy, distance between words matters, and structural cues like title or anchor placement carry far more signal than bulk co-occurrence ever will.
- Bag-Of-Words Co-Occurrence Is Too Loose — Two words appearing in the same document often have nothing to do with each other. "car" and "insurance" co-occur constantly but are not synonyms. Document-level co-occurrence on its own produces noise that overwhelms the real signal.
- Distance In Text Encodes Meaning — Words that appear within the same sentence or phrase are far more likely to be substitutable than words that appear in different paragraphs. Distance matters as much as presence, and a naive frequency count throws that signal away.
- Structural Signals Are Wasted — Titles and anchor text are densely meaningful. A static thesaurus throws away the signal that a word appearing in a title also appears in body text in the same document, even though that title-body pairing is a strong synonym indicator.
- Phrase Boundaries Get Lost In Free Text — Real documents are written in sentences, paragraphs, and headings. Treating the entire document as a bag-of-words discards the phrase structure that delineates which word pairs are meaningfully related and which are accidentally adjacent.
- Single-Signal Approaches Over-Fire On Collocations — Approaches that rely on a single statistic such as raw co-occurrence count produce many phrases that are collocates ("strong" + "coffee") rather than synonyms. A robust system needs orthogonal signals that catch different failure modes.
Innovation
Co-Occurrence Plus Closeness Plus Structure
The system reads document collections and scores word pairs on three orthogonal signals: how often they co-occur, how close they sit when they do, and whether they appear in title or anchor positions. A pair that passes all three is treated as a strong synonym candidate. Single-signal failures (collocations, off-topic mentions, title-only matches) are filtered out because they fail one or more of the orthogonal gates.
- Index The Document Collection — Process the corpus into a form where word positions, sentence boundaries, and structural roles (title, anchor, body) are preserved. Without this preserved structure, none of the downstream signals can be computed.
- Count Pair Co-Occurrence — For every word pair in the document collection, increment a co-occurrence counter each time both words appear in the same document. This is the baseline frequency signal that other signals modulate.
- Score Closeness — For each occurrence, compute a closeness score that measures whether the two words appear within the same sentence or phrase. Pairs with high closeness are weighted more in the final score. Closeness can be defined as a token-distance threshold, a same-sentence boolean, or a softer kernel function.
- Read Title-Body Correlations — When a word appears in a title or anchor and a candidate appears in the same document's body, that title-body correlation lifts the pair's synonym probability. This signal captures the case where an author has used two labels for the same concept across structural roles.
- Read Anchor-Body Correlations — Anchor text linking to a document plus body text in the destination document also acts as a label-body pair. Different anchors used by different linking pages to refer to the same document are strong synonym candidates because they reflect the linking community's varying labels for the same concept.
- Combine The Signals — Co-occurrence frequency, closeness scores, and structural correlations are combined into a single synonym score per pair. Pairs above threshold are emitted as candidates. The combination function is the load-bearing piece because it determines how single-signal failures are filtered.
- Feed The Live Engine — Validated synonym pairs are written to the runtime lookup the search engine consults at query time, broadening matching when the substitution preserves intent. The output of this offline pipeline is what the runtime sees as a synonym table.
Documents Are Their Own Synonym Dictionary
The patent's central claim is that documents already contain everything needed to discover synonyms, provided you read them structurally instead of as bags of words. Authors signal synonymy constantly: by using two labels for the same concept across title and body, by anchoring different terms to the same destination, by placing related terms inside the same sentence. The system makes that signal extractable.
Three Orthogonal Reads
Co-occurrence, closeness, and structural role are three independent ways to read the same document collection. A real synonym pair satisfies all three; a false positive fails at least one.
- Frequency Read — Reads the corpus as a bag of words. Cheap to compute, noisy on its own, but the baseline signal that the other reads modulate.
- Proximity Read — Reads phrase and sentence structure. Catches the closeness that frequency-only approaches miss, and rejects pairs that happen to share documents without ever appearing near each other.
- Structural Read — Reads titles, headings, and anchor text as label positions. A pair that surfaces in label-body correspondences is more likely to be synonymous than a pair that lives only in body text.
The web is already a structured thesaurus. The system's job is to read it that way.
<\/section>Technical Foundation
The Three Orthogonal Signals
The strength of the patent is in combining three signals that each catch a different failure mode of the others. The combination is what makes the synonym table reliable enough to ship into production retrieval.
- Co-Occurrence Frequency — How often a word pair appears together across the corpus. Strong on its own only for very high-frequency pairs and produces many false positives elsewhere because it cannot distinguish synonymy from generic co-mention.
- Closeness Score — Whether the pair tends to appear in the same sentence or phrase. Catches the looseness of document-level co-occurrence by adding a positional constraint.
- Title And Anchor Correlation — Words appearing in titles or anchors paired with body-text occurrences of the candidate. Catches synonym pairs that are explicitly used as labels of the same concept across the corpus.
- Combined Synonym Score — The output value that triggers promotion. Combines the three input signals via a weighted function. The exact weights are tunable per language and per topical domain.
Quality Metrics
- Co-Occurrence Count — The raw frequency baseline. High values for high-frequency pairs ("the", "is") provide no information; the metric is most useful when normalized by individual term frequency.
cooc(A, B) = |D : A in D AND B in D| - Closeness — Lower values mean tighter pairing. Pairs that consistently appear in the same sentence or phrase produce closeness scores below a configurable threshold and are treated as proximity-correlated.
close(A, B) = avg(min |pos(A) - pos(B)|) per D - Title-Body Correlation — Captures the case where authors use one label in the headline and another in body text. Pairs with high values are strong candidates because the author has explicitly signaled equivalence across structural roles.
P(B in body | A in title)
Key Insight: Each signal alone is noisy. Co-occurrence overfires on collocations. Closeness misses synonyms used in different sections of long documents. Title correlations are sparse. Combining all three yields a sharper signal than any single one delivers. The patent's contribution is the combination function more than any individual metric.
<\/section>The Process
The Mining Pipeline
The full pipeline runs offline against the document index and writes to the synonym lookup that the runtime engine consults. Each stage is independent and can be re-run when the underlying corpus changes.
- Snapshot The Document Index — Take a snapshot of the indexed documents with their per-token positions and structural role annotations (title, anchor, body, heading).
- Enumerate Candidate Pairs — Generate the set of word pairs to evaluate. The candidate space can be all word pairs in vocabulary or filtered to pairs that pass a frequency floor to keep the computation tractable.
- Compute Per-Signal Scores — For each candidate pair, compute co-occurrence frequency, closeness, and title-anchor correlation independently. The signals can be computed in parallel since they read the same corpus snapshot.
- Apply Combination Function — Combine the per-signal scores into a single synonym score using the configured weights. Pairs below the combined threshold are dropped.
- Cross-Check Against Known False Positives — Compare the surviving pairs against a list of known collocations or anti-synonyms to catch obvious failures that slipped through. This step is optional but cheap insurance.
- Write To Synonym Lookup — Promoted pairs are written to the synonym table consulted by the runtime engine. The lookup is keyed by phrase so per-query retrieval is constant-time.
Quality Control
Quality Control
Combining Signals To Filter Failures
Single-signal approaches each fail in characteristic ways. The combination filter catches those failures because a true synonym pair satisfies all three signals while a false positive fails at least one.
- Frequency Floor — A minimum co-occurrence count is required before the pair is even scored on closeness or structure. This drops infinitely rare pairs whose other signals would be statistically unreliable.
- Closeness Threshold — Pairs must appear within a configured token distance in a substantial fraction of their co-occurrences. Pairs that co-occur in documents but never near each other fail this gate.
- Structural Gate — Title-body or anchor-body correlation must exceed a minimum value. Pairs that live only in body text without ever appearing in a label role are weakened.
- Combined Score Threshold — Even after passing each individual gate, the combined synonym score must exceed a final threshold. This catches edge cases that scrape past individual gates but lack overall strength.
- Anti-Collocation Filter — A small curated list of known collocations ("strong coffee", "heavy traffic") is checked against the candidate output. Pairs on this list are dropped regardless of score because they are known systematic failures of the underlying signals.
What This Means for SEO
What This Means for SEO
Document-based synonym generation is the structural counterpart to the session-based synonym mining of US7636714. It tells you which signals from the document body itself the engine reads to decide that two terms refer to the same thing, and almost all of those signals are under your control as an author.
- Use Synonyms Inside The Same Sentence Or Phrase — When you need a piece of content to rank for variants, the most efficient place to put them is inside the same sentence or paragraph as the primary term, not scattered across the document. Closeness is one of the three load-bearing signals and the easiest to influence directly.
- Title-Body Alignment Is A Synonym Signal — A term used in your title or H1 that also appears in your body text reinforces that you are treating both forms as labels of the same concept. This is one of the three signals the patent emphasizes and is rarely mentioned in standard SEO guidance.
- Anchor Text Variants Are Read As Synonym Evidence — Internal and external anchors using different surface forms for the same target page train the synonym graph. Diversifying anchors (without keyword stuffing) is reading like signal, not noise, and is one of the few ways to feed the synonym pipeline from outside the page itself.
- Don't Rely On Document-Level Co-Occurrence Alone — A common content strategy is to drop a term anywhere in a long document to "cover" the variant. The patent shows that this signal alone is noisy. To register as a synonym pair, you need closeness or title-anchor reinforcement, not just presence.
- Headings Are Title-Like For The Section Below Them — The title-body signal extends to headings that govern the subsection. A term in an H2 that appears in the paragraphs immediately below participates in the same kind of label-body correlation the patent describes for page titles.
- Glossary And Definition Sections Are Synonym Goldmines — A definition section that pairs a term with its variants in the same sentence ("X, also known as Y, is...") is one of the cleanest synonym signals you can offer. Sites with strong glossaries feed the synonym pipeline disproportionately.
- Long-Distance Mentions Don't Count The Same — Mentioning a variant once at the top and once at the bottom of a 3,000-word page contributes far less to the synonym signal than two mentions in adjacent paragraphs. Concentrate variant coverage in the section most likely to be the answer passage.
- Anti-Collocation Awareness Matters For Niche Terms — If your topic includes a known collocation (e.g., "machine learning"), the system may resist treating one half as a synonym of a related concept. Counter this with explicit pairing in titles and definitions so the signal reaches the structural gate, not just the frequency gate.