Table of contents :

BM25 and What It Means for Product Search

100-days-of-weaviate-027

Table of contents :

Product search is often described as a relevance problem, but relevance is not one thing. A shopper searching for black leather boots expects matching products to contain those words, while a shopper searching for waterproof hiking shoes may expect semantic matches even when the exact wording differs. BM25 is one of the most useful foundations for the first category: lexical search based on the terms that appear in product data.

What BM25 does

BM25, short for Best Matching 25, is a ranking function used by search engines to score how well a document matches a text query. In a WooCommerce catalog, the document is usually a product or a product representation containing fields such as the product name, description, SKU, category, brand, and attributes.

BM25 gives a higher score when:

  • A query term appears in the product.
  • The term is relatively rare across the catalog.
  • The product contains a strong proportion of the query terms.
  • The matching field is not excessively long compared with other documents.

The algorithm is more useful than a simple count of matching words because it accounts for term frequency, inverse document frequency, and document length.

The main BM25 factors

Term frequency

Term frequency measures how often a query term occurs in a document. Repeated mentions can indicate stronger relevance, but BM25 deliberately limits the benefit of repetition. This prevents a product description stuffed with the word boots from dominating every other result.

Inverse document frequency

Inverse document frequency, commonly called IDF, gives more weight to terms that appear in fewer documents. A rare term such as a specific model code is usually more useful for ranking than a common term such as product or sale.

This is particularly valuable for WooCommerce catalogs with model numbers, collection names, technical standards, and brand-specific terminology. A search for GX-500 should strongly favor products containing that exact identifier.

Document length normalization

BM25 normalizes for document length. Without normalization, long descriptions would have more opportunities to match query terms simply because they contain more words. The normalization factor helps prevent a verbose product description from automatically outranking a concise product with a better match.

In practice, this can produce unexpected results when product pages contain long SEO content, shipping information, specification tables, or repeated attribute labels. The indexed text should represent searchable product information rather than every piece of page content.

BM25 parameters: k1 and b

Two parameters control the behavior of BM25:

  • k1 controls how quickly the benefit of repeated terms reaches a ceiling. A higher value allows term frequency to have more influence.
  • b controls document length normalization. A value near 1 applies stronger normalization, while a value near 0 largely removes it.

Many search systems use sensible defaults, so agencies should not tune these values immediately. First inspect the indexed fields, tokenization, analyzers, and query behavior. Poor field selection or inconsistent product data is usually a bigger problem than the default BM25 parameters.

Why BM25 matters for WooCommerce search

WooCommerce product catalogs contain many exact terms that semantic search alone may handle inconsistently. These include SKUs, MPNs, dimensions, fabric names, compatibility codes, certifications, and product series. BM25 is strong when the shopper uses the same vocabulary as the catalog.

For example, a catalog might contain:

  • Product name: Alpine Waterproof Hiking Boot
  • Brand: North Ridge
  • Attributes: black, leather, waterproof, men's, size 10
  • SKU: NR-ALP-10-BLK

A query for waterproof black leather boot can receive a high lexical score because several query terms occur in the product. A query for NR-ALP-10-BLK can benefit from the rarity of the SKU, provided the SKU is indexed and tokenized in a way that preserves useful matches.

BM25 in Weaviate

Weaviate supports BM25 keyword search through its keyword search capabilities. A BM25 query searches the text properties configured for the query and returns objects ranked by lexical relevance. The exact query syntax depends on whether the project uses GraphQL or the current client APIs, but the important design decision is the same: specify which product properties should contribute to the lexical score.

A conceptual GraphQL query looks like this:

{ Get { Product( bm25: { query: "black leather boots", properties: ["name^3", "description", "brand^2", "attributes"] } ) { name brand _additional { score } } } }

The caret notation in this example expresses a property boost in systems and query configurations that support it. Verify the supported syntax and client version in the Weaviate version used by the project before deploying it. The key principle is to give product names and brands more influence than long descriptions when that matches the store's merchandising goals.

Choosing fields for a product index

Do not treat the entire WooCommerce page as one undifferentiated text field. Create a deliberate searchable representation. A practical product object may include:

  • name for the product title.
  • shortDescription for concise selling points.
  • description for detailed product information.
  • brand and categories for controlled catalog vocabulary.
  • attributes for color, material, compatibility, size, and other structured values.
  • sku and mpn for exact identifiers.

Agencies should decide whether each field belongs in lexical search, semantic search, filtering, or more than one. A size such as 10 is often better handled as a structured filter than as a free-text relevance signal. A brand can be both a searchable field and a filter. An SKU may need exact or prefix matching in addition to BM25.

BM25 versus vector search

BM25 and vector search solve different parts of the search problem:

  • BM25: strong for exact terms, rare identifiers, product names, and catalog vocabulary.
  • Vector search: strong for meaning, paraphrases, and queries whose wording differs from the product content.
  • Hybrid search: combines both signals to cover exact and semantic intent.

Consider the query phone case for rainy cycling. A vector search may identify products described as weather-resistant bike cases even if the exact phrase does not appear. A BM25 search may favor a product explicitly containing rainproof cycling phone case. Hybrid search can use both signals, provided the result fusion and weighting are tested against real customer queries.

Filtering should remain separate from relevance

BM25 ranks candidates; it does not replace catalog filters. A search for black running shoes may be ranked lexically, while filters enforce constraints such as size, stock status, price range, gender, or shipping region.

In a Weaviate implementation, combine the text search with filters where appropriate. For example, restrict results to products that are published and in stock before ranking them. This prevents an unavailable product with an excellent textual match from appearing above purchasable products.

Common BM25 problems in WooCommerce projects

Inconsistent product vocabulary

If one product uses trainer, another uses sneaker, and a third uses running shoe, BM25 cannot infer that these terms are related unless the index uses suitable analysis or the application also uses semantic search, synonyms, redirects, or curated query expansion.

SKUs split into unhelpful tokens

Identifiers containing hyphens, slashes, or mixed letters and numbers can be analyzed differently depending on the tokenizer. Test searches for the full SKU, partial SKU, and individual segments. If exact identifier lookup is important, store a normalized identifier field and support a dedicated exact-match path.

Long descriptions overpowering useful fields

Repeated marketing copy, compatibility lists, and technical documentation can create noisy matches. Keep high-value fields distinct, boost them where supported, and consider indexing a concise search text specifically for retrieval.

Ignoring empty or poorly maintained metadata

BM25 cannot rank information that is missing. If brand, material, or compatibility data is important to search, improve catalog enrichment and synchronization rather than trying to compensate entirely through ranking parameters.

A practical implementation workflow

  1. Export a representative set of WooCommerce products, including names, descriptions, attributes, categories, brands, SKUs, and stock status.
  2. Define which fields are searchable text and which are filters or exact-match fields.
  3. Index a controlled product representation in Weaviate and verify tokenization with real catalog terms.
  4. Build a BM25 baseline using common shopper queries.
  5. Add vector or hybrid search for paraphrases and discovery-oriented queries.
  6. Evaluate results using query sets grouped by intent, such as product name, SKU, category, attribute, and natural-language discovery.
  7. Track zero-result searches, reformulations, clicks, add-to-cart events, and purchases.
  8. Adjust field weighting, synonyms, preprocessing, or query routing based on evidence rather than isolated examples.

How agencies should evaluate BM25 relevance

Use a labeled query set instead of relying on a few manual searches. For each query, record which products are relevant, which are acceptable alternatives, and which should not appear. Then compare ranking changes after modifying field weights, tokenization, filters, or hybrid settings.

Useful test cases include exact SKU searches, misspellings, brand-plus-category queries, attribute combinations, broad category searches, and queries using terminology that differs from the catalog. Review both the top result and the first page. A ranking change that improves one query but harms a high-volume query may not be a worthwhile production change.

BM25 is a dependable lexical foundation for WooCommerce product search. It rewards meaningful term matches, handles rare catalog vocabulary well, and provides an interpretable baseline for measuring more advanced retrieval strategies. When paired with clean product fields, structured filters, and vector search for semantic intent, it gives agencies a practical path to search that is both testable and useful to shoppers.

Trending posts
You might also like