219 words
1 minute
Practical BM25 - Part 1: How Shards Affect Relevance Scoring in Elasticsearch

BM25 Demystified#

Here is a list of where to look for to understand the mathematical foundations of BM25:

  • Stephen Robertson & Hugo Zaragoza, The Probabilistic Relevance Framework: BM25 and Beyond (2009): The definitive monograph by the creators of BM25. It derives the formula from the 2-Poisson model and explains why the saturation curve and document length normalization take their exact functional forms.
  • Christopher Manning, Prabhakar Raghavan, & Hinrich Schütze, Introduction to Information Retrieval (Chapter 11): Covers the Binary Independence Model and transitions directly into Okapi BM25 weighting with clean, formal notation.

Background#

Okapi BM25 has been the default similarity algorithm, which is what’s used to score results as they relate to a query, in Elasticsearch since version 5.0. The rest of this post focuses on the practical usage of BM25, including covering the parameters that are available and what affects scoring.

Local Elasticsearch Setup for BM25

Run a single-node container with security and TLS disabled so we can send unauthenticated HTTP requests directly from our terminal:

Terminal window
docker run -d \
--name es-bm25 \
-p 9200:9200 \
-e "discovery.type=single-node" \
-e "xpack.security.enabled=false" \
-e "ES_JAVA_OPTS=-Xms1g -Xmx1g" \
docker.elastic.co/elasticsearch/elasticsearch:8.17.4

Verify the cluster is ready before creating our test index:

Terminal window
curl -s "http://localhost:9200/_cluster/health?pretty"

Understanding How Shards Affect Scoring#

(To be continued)

Practical BM25 - Part 1: How Shards Affect Relevance Scoring in Elasticsearch
https://blogs.openml.io/posts/bm25/
Author
OpenML Blogs
Published at
2026-11-17
License
CC BY-NC-SA 4.0