Why an Index Is Not Used: the Estimate, Not the Index

An index that the planner refuses to use is almost never a problem with the index. It is a problem with the number the planner expected the index to return.

How does PostgreSQL decide whether to use an index?

Before choosing a path, PostgreSQL does not sample the table. It reads the statistics collected by ANALYZE — a most-common-values list, a histogram, and a null fraction per column — and multiplies its way to an estimated row count. Every cost that follows is derived from that number.

Once the estimate exists, the cost model compares access paths. Reading pages in physical order is charged seq_page_cost, which defaults to 1.0. Reading a page at a random offset is charged random_page_cost, which defaults to 4.0. An index scan pays that higher rate for each heap page it has to visit, because the index gives it row locations in index order, not in table order.

So the arithmetic is straightforward. Below some number of rows, the index wins. Above it, the sequential scan wins, because touching most of the table in random order costs more than reading all of it in physical order.

The estimate is what decides

That means an index is skipped in two quite different situations, and they need different responses.

The first: the estimate is correct and the index genuinely is the wrong choice. A query returning 40% of a table should not use an index, and forcing it to will make things slower. This case is the planner working.

The second: the estimate is wrong. The planner expected 800,000 rows, the query returns 12, and the sequential scan it chose reads the whole table to find them. The index was fine the entire time.

You can tell the two apart in one step, because EXPLAIN ANALYZE reports the actuals alongside the estimates:

check-the-estimate.sql

EXPLAIN ANALYZE

SELECT id FROM orders WHERE status = 'refunded';

Seq Scan on orders (cost=0.00..48221.00 rows=812443 width=8)

(actual time=0.031..612.884 rows=12 loops=1)

Filter: (status = 'refunded'::text)

Rows Removed by Filter: 1198788

rows=812443 against rows=12 is the whole diagnosis. Nothing about the index definition needed to be examined. Comparing estimated rows against actual rows at every node, starting from the leaves, localizes most plan problems the same way.

What makes the estimate better?

Three things, in roughly this order of practical impact.

Statistics freshness. A summary describing last month’s distribution answers last month’s question. A table that recently gained a large batch of rows with a new value distribution will mislead the planner until ANALYZE runs again.

Resolution. default_statistics_target controls how many buckets the histogram gets. Raising it for a specific column costs more ANALYZE time and buys a better estimate on skewed data.

Correlated predicates. The planner assumes independence between columns. WHERE country = 'JP' AND city = 'Osaka' multiplies two selectivities as if the second were unrelated to the first, and lands far below reality. Extended statistics exist for this case.

Limits

The cost constants above are defaults, and defaults describe hardware assumptions rather than facts. random_page_cost = 4.0 encodes a seek penalty that does not apply to NVMe storage, which is why lowering it on modern hardware often makes the planner willing to use indexes it previously rejected. That is a real fix, but it is a change to a global assumption, so it belongs in a considered configuration review rather than in the middle of debugging one query.

This article also describes only the choice between an index scan and a sequential scan. Bitmap heap scans sit between the two — collecting row locations before reading the heap in physical order — and the point at which the planner switches to one is governed by the same estimate.