How Elasticsearch works

The Problem Statement

Interviewer: "Your team runs a product catalog with 50 million items. Users type partial, misspelled queries like 'runnng shoes nike' and expect relevant results in under 200ms. Your current SQL database can only do LIKE queries with full table scans. Walk me through how Elasticsearch solves this, from how it indexes a document to how it executes a search query across a distributed cluster."

This question tests three things: whether you understand the inverted index data structure that makes full-text search fast, whether you know how Elasticsearch distributes data and queries across shards, and whether you can explain how relevance scoring works so the user sees the best results first.

Most candidates say "Elasticsearch is fast because it uses an index." That is too vague. The interesting parts are how the analysis pipeline transforms "Running" into "run" at both index and query time, how the scatter-gather execution model fans out queries to shards and merges results, and how BM25 scoring ranks documents by relevance rather than just matching or not matching.

I find this question reveals a lot about a candidate's depth. Everyone has heard of Elasticsearch. Fewer can explain why a document is not searchable the instant you index it, or why deep pagination is expensive, or what happens during a segment merge.

Clarifying the Scenario

You: "Good question. Let me clarify a few things before diving in."

You: "When you say 'how Elasticsearch solves this,' should I focus on the core data structures (inverted index, analysis pipeline) or the distributed architecture (shards, replicas, coordinating nodes)? Or both?"

Interviewer: "Both. I want to understand the full picture from ingestion to query."

You: "Got it. Should I cover relevance scoring? Understanding BM25 is important for explaining why some results rank higher than others."

Interviewer: "Yes, cover BM25 at a conceptual level. I do not need the full math."

You: "One more thing: should I cover near-real-time indexing? There is an important nuance about when a document becomes searchable after you write it."

Interviewer: "Definitely. That comes up in production all the time."

You: "OK. I will structure this in five parts: the inverted index and analysis pipeline, how shards and replicas distribute data, the scatter-gather query execution model, BM25 relevance scoring, and the near-real-time indexing mechanics including segment lifecycle."

My Approach

I break this into five parts:

1. The inverted index and analysis pipeline: How text becomes searchable tokens, and how Elasticsearch looks up documents by term in O(1)
2. Shards and replicas: How Elasticsearch partitions data horizontally and replicates for fault tolerance
3. Scatter-gather query execution: How a search query fans out to shards and results are merged at the coordinating node
4. BM25 relevance scoring: How Elasticsearch ranks results by term rarity, frequency, and document length
5. Near-real-time indexing and segment lifecycle: Why documents are not instantly searchable, and how Lucene segments get created, refreshed, and merged

The mental model I use: think of Elasticsearch as a library's catalog system, but distributed across multiple buildings. The inverted index is the card catalog (look up a word, find all books containing it). Shards are different buildings, each holding a portion of the collection. When you search, a librarian (coordinating node) sends your query to every building, each building's local catalog returns its best matches, and the librarian merges everything into a single ranked list.

The analysis pipeline is the set of rules for how words get filed in the catalog. "Running" is filed under "run" so you can find it regardless of tense or capitalization.

The Architecture

Here is the full picture of an Elasticsearch cluster, from document ingestion through query execution:

Here is how a document flows through the system:

1. A client sends an index request (PUT /products/_doc/123) to any node. That node acts as the coordinating node for this request.

2. The coordinating node routes the document to the correct primary shard using hash(_id) % number_of_primary_shards. This is why the number of primary shards is fixed at index creation: changing it would invalidate the routing.

3. The primary shard indexes the document. It runs the analysis pipeline (tokenize, lowercase, stem), writes to an in-memory buffer, and writes the operation to the transaction log (translog) for durability.

4. The primary replicates to all replica shards. The write is not acknowledged to the client until all in-sync replicas confirm.

5. After a refresh (default 1 second), the in-memory buffer is written as a new Lucene segment, making the document searchable.

For queries, the coordinating node fans out to one copy of each shard (primary or replica), collects results, merges and re-ranks them, then fetches full documents for the top results. The coordinating node does not hold any data itself; any node can act as coordinator.

I always emphasize this in interviews: the architecture separates the write path (route to primary, replicate to replicas) from the read path (scatter to any shard copy, gather at coordinator). Understanding this split explains most of Elasticsearch's performance characteristics.

The Inverted Index and Analysis Pipeline

The inverted index is the core data structure that makes full-text search fast. Instead of scanning every document to find which ones contain your search term (O(n)), the inverted index lets you look up a term and instantly get the list of documents containing it (O(1) hash lookup + O(k) to read the posting list where k is the number of matching documents).

Here is how it works:

Documents indexed:
  Doc 1: "Nike Running Shoes - Lightweight"
  Doc 2: "Adidas Running Sneakers"  
  Doc 3: "Nike Basketball Shoes"

Analysis pipeline transforms each document:
  1. Tokenizer: split on whitespace/punctuation
  2. Lowercase filter: "Nike" → "nike"
  3. Stemmer: "Running" → "run", "Shoes" → "shoe", "Lightweight" → "lightweight"

Inverted index (after analysis):
  "nike"        → [Doc 1 (pos:0), Doc 3 (pos:0)]
  "run"         → [Doc 1 (pos:1), Doc 2 (pos:1)]
  "shoe"        → [Doc 1 (pos:2), Doc 3 (pos:2)]
  "lightweight" → [Doc 1 (pos:3)]
  "adidas"      → [Doc 2 (pos:0)]
  "sneaker"     → [Doc 2 (pos:2)]
  "basketball"  → [Doc 3 (pos:1)]

Each entry in the posting list stores the document ID, the position of the term within the document (for phrase queries), and the term frequency (for scoring). This is what makes phrase queries like "running shoes" work: Elasticsearch checks that "run" appears at position N and "shoe" appears at position N+1 in the same document.

The analysis pipeline is critical because it runs at both index time and query time. When a user searches for "runnng shoes nike," the query analyzer:

1. Tokenizes: ["runnng", "shoes", "nike"]
2. Lowercases: ["runnng", "shoes", "nike"]
3. Stems: ["runnng", "shoe", "nike"]
4. Fuzzy matching (if enabled): "runnng" matches "run" with edit distance 1

Because the same stemmer runs at both index and query time, "running" in the document and "shoes" in the query both become "shoe" and match correctly.

This symmetry between index-time and query-time analysis is one of the most elegant parts of the design. It means you never have to worry about case sensitivity or verb tenses when searching. The pipeline handles it automatically.