Elasticsearch

Database Fundamentals named indexing in general terms — a B-tree index speeds up equality and range lookups on a known column. It does nothing for "find every document containing this word," which is a different problem entirely. Elasticsearch is the building block a design reaches for once search, not lookup, is the actual requirement.

The inverted index

A normal database index maps a column value to the rows that have it. An inverted index does the opposite direction: it maps each term to the list of documents containing it, so a search for a word is a direct lookup against that list rather than scanning every row for a substring match — the reason LIKE '%term%' against a relational column can't use a B-tree index at all, while the same search against Elasticsearch is close to instant regardless of how much text exists.

Sharding and replicas

An Elasticsearch index — its unit of data, distinct from a database index — is split into shards, the same partitioning idea covered generally in Sharding, and each shard is itself a self-contained search engine instance underneath. Replica shards exist for the two usual reasons: availability if a node holding a primary shard goes down, and extra read throughput, since a search can be served by any replica of the shard it needs.

Near-real-time, not real-time

A write to Elasticsearch doesn't become searchable the instant it's written — it becomes visible after the next refresh, which happens on a short configured interval (commonly around one second) rather than per-write, because rebuilding the in-memory search structures on every single write would be far too costly at any real volume. This is a deliberate trade — a small, bounded delay between "written" and "searchable" in exchange for write throughput — and it's worth naming explicitly rather than assuming search results are always instantly current.

Relevance scoring, briefly

A search doesn't just return documents containing a term, it ranks them — the other half of Elasticsearch beyond the inverted-index lookup itself. The default ranking algorithm, BM25, scores a document higher the rarer a matched term is across the whole index (a hit on "the" is worth much less than a hit on a specific product name) and the more densely that term appears in the matched document, with diminishing returns for repetition so keyword-stuffing doesn't game the score. Naming that ranking is a real, tunable step — not just "the search returns matches" — is worth doing whenever a design's requirements care about result quality, not only result correctness.

Aggregations: analytics, not just search

Elasticsearch's other major real-world use case looks nothing like a search bar: aggregations compute metrics — counts, averages, min/max, histograms bucketed by time or by a field's value — across the documents matching a query, at speed, because the same inverted index and distributed-shard structure built for search also makes "how many of these happened per hour, broken down by category" a fast query rather than a slow scan. This is why Elasticsearch (and the broader Elastic stack) shows up as heavily in log analytics and dashboarding designs as it does in classic full-text search ones — the underlying engine is the same, the use case is different.

Not a primary datastore

The pattern that avoids this: a primary database stays the source of truth, and Elasticsearch is kept in sync as a derived, rebuildable search index — commonly via change-data-capture, streaming row changes through something like Kafka into an indexing consumer, though a simpler design might just dual-write on the same request path at the cost of the two stores briefly disagreeing if one write succeeds and the other fails.

Because the index is rebuildable from the primary, losing it entirely is an inconvenience — reindex from source — rather than data loss, which is exactly the property that justifies trading some consistency for search speed in the first place.