Treats user-authored lists across the web as a giant correlation signal for which items belong together as members of the same concept, turning bulleted lists into a categorization graph.
Patent Overview
- Inventor
- Steven D. Baker
- Assignee
- Google LLC
- Filed
- 2007-07-09
- Granted
- 2011-10-11
- Application Number
- US 11/775,495
The Challenge
How Do You Find What Belongs Together?
Search engines need to know that "London" and "Paris" are both European capitals, that "Python" and "Ruby" are both programming languages, that "Notion" and "Obsidian" are both note-taking apps. Static taxonomies are incomplete and lag real categorization. The web already has the answer, scattered across millions of hand-authored lists. The challenge is to extract that signal cleanly without drowning in the noise of generic co-occurrence.
- Taxonomies Are Incomplete — Curated category lists miss long-tail concepts and lag updates. "Best note-taking apps" had no answer in any taxonomy when the category emerged, and most editorial taxonomies still under-cover emerging tool ecosystems.
- Co-Occurrence In Prose Is Too Loose — Words that appear in the same article often have nothing categorical in common. A document about travel might mention "London" and "sushi" with no shared category, and bag-of-words mining cannot tell the difference.
- Lists Are An Untapped Signal — When humans author a list, every member shares a category that the list itself names. That structural cue is far more reliable than free-text co-occurrence because the author has explicitly grouped the items as peers.
- Cross-Document Aggregation Is Where The Signal Lives — Any single list is too small to draw conclusions from. The signal emerges when the same pair of items appears in many independently authored lists across the web.
- Generic Items Pollute Naive Aggregation — Items that appear in many lists for many different reasons ("home", "search", "about") would dominate raw co-occurrence counts. The system needs to normalize so that exclusive pairing scores higher than generic frequency.
Innovation
Mine The Lists, Not The Prose
The system identifies lists in documents (bulleted, numbered, comma-separated tables), enumerates the items in each list, and counts how often each pair of items co-occurs in the same list across the web. Strongly co-occurring pairs are marked as members of the same correlated-pair set. The structural restriction to lists is what makes the signal clean enough to ship.
- Detect Lists In Documents — Parse documents for explicit list structures: HTML ul and ol elements, comma- and semicolon-delimited series in prose, structured tables, and definition lists. Each detected list is a candidate signal source.
- Extract Items — For each detected list, extract its individual items as candidate concepts. Items can be single terms or short phrases. Tokenization and entity recognition help when the items are noun phrases.
- Pair And Count — For every pair of items that co-occur in any list, increment a co-occurrence counter. The counter is keyed by the unordered pair so that order does not double-count.
- Score Correlation — Convert raw co-occurrence counts into a normalized correlation score that accounts for how often each item appears across all lists. PMI (pointwise mutual information) or Jaccard similarity are typical scoring functions.
- Apply Strength Threshold — Pairs whose correlation score exceeds a configured threshold are promoted. The threshold tunes how aggressive the categorization graph is; higher thresholds yield smaller but cleaner groups.
- Emit Correlated-Pair Lists — Pairs above the threshold become entries in a correlated-pair list that downstream systems can consult for entity grouping, related searches, topical clusters, and recommendation surfaces.
Lists Are Latent Category Annotations
Every hand-authored list on the web is an implicit categorization. The page author has decided that these items belong together under whatever theme the list serves. Aggregating that signal across the web yields a categorization graph that no manual taxonomy could match, both for coverage and for currency.
Structure Over Frequency
It is not how often two items appear in the same document that matters. It is how often they appear in the same list. The list boundary is the key piece of structure that distinguishes peer items from coincidental co-mentions.
- Bulleted And Numbered Lists — HTML lists are the cleanest signal because the boundaries are explicit. The system can confidently say item A and item B were authored as peers under a single category implied by the list.
- Comma And Semicolon Series — Prose like "Paris, London, Berlin, and Rome" is also a list. Parsing recognizes these inline enumerations and treats them as authored lists too. They are noisier than HTML lists but contribute valuable coverage.
- Structured Tables — Tables with item columns also count as lists. Each column or each row can be an implicit category, and the items in it are peers under that category.
Categorical knowledge emerges from the structural shape of the web, not from explicit category labels.
<\/section>Technical Foundation
What The System Measures
The correlation between two items reflects how often they are treated as peers, normalized by how often each appears anywhere. The normalization is what prevents generic items from dominating the output.
- List Co-Occurrence Count — Number of distinct lists across the corpus that contain both items. Raw input to the correlation score. Higher counts are necessary but not sufficient for a strong pair.
- Item Frequency — Number of lists containing each item separately. Used as the denominator in normalization to discount items that appear in everything.
- Correlation Score — A normalized score (typical forms: PMI, Jaccard over list-membership) that rewards exclusive pairing over generic frequency. The score is the value compared to the threshold for promotion.
- List Boundary Definition — The rule that decides what counts as a list. Tight boundaries (only HTML ul/ol) produce cleaner but smaller signal; loose boundaries (including inline comma series) produce more signal but more noise.
Quality Metrics
- PMI (Pointwise Mutual Information) — Positive values indicate the pair is over-represented in lists relative to chance. Higher values are stronger evidence of category co-membership.
PMI(A, B) = log( P(A, B in same list) / (P(A) * P(B)) ) - Jaccard Over List Membership — A simple symmetric measure that ranges from 0 to 1. Useful when PMI is too sensitive to low-frequency items.
J(A, B) = |lists(A) ∩ lists(B)| / |lists(A) ∪ lists(B)|
Key Insight: Lists are the closest thing to ground-truth categorization that the web offers at scale. Every author's choice of list members is a vote for category membership. Aggregating those votes produces a categorization graph that taxonomies cannot match for coverage or currency.
<\/section>The Process
The Mining Pipeline
End to end, the pipeline reads documents, extracts lists, counts pairs, scores correlations, and emits a graph of category-correlated pairs that downstream systems consume.
- Document Crawl Snapshot — Take a snapshot of the document index with structural annotations preserved (ul, ol, table, paragraph).
- List Extraction — For each document, run the list detector. Extract HTML lists, inline comma series, and table columns as candidate lists.
- Item Tokenization — For each extracted list, tokenize its items into normalized form. Run optional entity recognition to canonicalize items to known entity IDs when possible.
- Pair Counting — For each list, enumerate all pairs and increment the global co-occurrence counter. Keep per-item totals as well.
- Score Computation — After all lists are processed, compute the correlation score for each pair using PMI or Jaccard over the accumulated counts.
- Threshold And Emit — Pairs with score above the configured threshold are written to the correlated-pair output. The output feeds entity grouping, related-search generation, and topical clustering.
Quality Control
Quality Control
Filtering Out Generic Co-Membership
Without normalization, generic items would dominate the output and the correlated-pair graph would be useless. Each control catches a different mode of generic co-membership.
- Per-Item Frequency Cap — Items that appear in more than a configured fraction of all lists are excluded entirely. This filters out "home", "search", "about", and other UI elements that appear in nav lists everywhere.
- Minimum Co-Occurrence Count — Pairs that co-occur in fewer than N lists are dropped regardless of correlation score. The minimum count ensures statistical reliability.
- Normalized Score Required — Raw co-occurrence is never used directly. Only normalized scores (PMI, Jaccard) drive the threshold check so that frequency does not overwhelm exclusivity.
- List Size Cap — Lists longer than a configured maximum (e.g., 50 items) are downweighted because they tend to be link directories rather than genuine peer-group lists. Their co-occurrence counts are scaled down.
What This Means for SEO
What This Means for SEO
The list co-occurrence signal is one of Google's quieter inputs to topical clustering and entity grouping. It is also one of the easiest to act on intentionally because the structural cue (a list) is fully under your control.
- Build Comprehensive Lists On Topic Pages — A well-constructed list of peer entities (e.g., "best CRM tools in 2026" with 10-15 actual tools named) is a high-signal asset. Every co-mention reinforces the engine's understanding that the named entities belong to the same category, and that your page is authoritative on the category.
- Be On Other People's Lists — Being included in third-party listicles and roundups is more than a link. Each list membership feeds the correlation graph that links you to your category peers. This is a structural argument for digital PR aimed at listicle-style content.
- Avoid Mixing Categories Within A List — Lists that mix item categories ("top productivity tools and SEO platforms") dilute the correlation signal. Each list should have a single category so that every co-mention pair is meaningful.
- Use Proper HTML List Markup — Use ul, ol, or dl markup for real lists. Plain paragraph text with line breaks does not parse as a list and the items in it will not feed the correlation signal cleanly.
- Tables Are Lists Too — Comparison tables and feature matrices count as structured lists. Each column header is a category and the row entries are peers under that category. Build comparison content with proper table markup.
- Position In The List Doesn't Matter — The pairing is order-agnostic. Being item 1 or item 15 in a list contributes equally to the co-occurrence count. Focus on being on the list rather than gaming list order for this signal.
- Generic Items Are Filtered Out — Avoid generic items like "see also" or "home" in your real category lists. They contribute to per-item frequency that lowers the correlation score for the actual category members.
- List Size Sweet Spot Is Roughly 5 To 20 — Shorter lists give weak statistical evidence. Longer lists are downweighted as link directories. The 5-to-20 range produces the strongest per-pair correlation contribution.