Connection Pool Sizing: Why More Connections Make It Slower

Raising the pool size is the standard response to database timeouts, and it usually makes them worse. The reason is that the pool does not control how much work the server can do. It controls where the queue forms.

Why does a bigger pool not increase throughput?

A database server executes a limited number of requests genuinely in parallel. That limit comes from CPU cores for compute-bound work and from the storage layer’s ability to service concurrent I/O for the rest. It is a property of the machine.

If that limit is sixteen and the pool allows two hundred concurrent connections, the server does not do two hundred things at once. It does sixteen and interleaves the rest. Total throughput is unchanged; the additional connections contribute waiting, not work.

Little’s Law states this precisely: for a system in steady state, concurrency equals throughput multiplied by latency. Fix throughput at the server’s capacity, raise concurrency, and latency rises to match. The extra connections buy queueing delay and nothing else.

Why waiting inside PostgreSQL is expensive

Every PostgreSQL connection is a separate operating system process. That has consequences an idle connection in a pool does not have.

Each backend holds its own memory, and work_mem is allocated per sort or hash operation per connection — a query with three such nodes across two hundred connections can commit six hundred times work_mem in the worst case. Each backend participates in scheduling and in the shared structures the server uses to coordinate, so more of them means more contention on those structures.

So two hundred connections mostly idle is not free, and two hundred connections mostly active is actively harmful. This is also why the argument here is specific to PostgreSQL: an engine that serves a connection with a thread rather than a process pays a different price for an idle one, and the sizing conclusion does not carry over unchanged.

Reducing the number of server-side backends without reducing application concurrency is what a connection proxy is for, and how much reduction it delivers depends on which PgBouncer pooling mode is configured.

Where the queue belongs

Given that the queue exists either way, it should form in the pool, in the application, where it is cheap and observable.

A pool of sixteen with a bounded acquisition timeout produces a clear signal: a request waited for a connection and failed after a stated interval. That is a metric with a name, and it degrades predictably under load.

A pool of two hundred produces a database whose latency rises for everyone, including requests that would have been fast, until something times out somewhere less specific. The failure is worse and the diagnosis is harder.

Attribution suffers too. Under saturation, a statement slowed by contention and a statement that is expensive on its own merits look alike from the outside, and separating them means reading what a plan reports about pages read and hit for that statement rather than watching the server average.

How many connections should the pool allow?

The frequently quoted starting point is roughly twice the core count, plus an allowance for concurrent disk I/O. It is a heuristic and should be treated as one: a place to begin, not a result.

The workload decides the other half of it. A request that issues one query per element of a collection holds its connection across all of them, so the same pool serves fewer requests per second than the individual query timings suggest.

The measurement that settles it is straightforward. Hold the workload constant and vary only the pool size. Throughput rises with pool size, then flattens. Past the flattening point, latency continues to climb while throughput does not. That inflection is the answer for that workload on that hardware, and it moves when either changes.

What matters more than the exact number is the shape: it is a curve with a plateau, not a line, and the whole class of “we increased the pool and it got slower” reports comes from being on the far side of the plateau.

Limits

This describes a single application talking to a single database. Multiple services, background workers, and migration jobs each with their own pool share the same server bound, and sizing each independently against the server’s total capacity is how instances end up with connection counts nobody chose. The server-side limit is the one that has to add up.