Thread pool pattern
Reuse a fixed set of worker threads to execute tasks from a shared queue, avoiding the overhead of creating and destroying threads per request.
Thread Pool Pattern: TL;DR
A thread pool separates task submission from task execution. A controlled set of reusable worker threads takes tasks from a shared queue, so callers do not create a new platform thread for every task. This limits concurrent work, amortizes worker creation, and gives the system a deliberate overload policy.
Mental model
Think of a small team of cooks and a ticket rail. The cooks are workers, the rail is the task queue, and incoming orders are tasks. When every cook is busy and the rail is full, the system must decide whether to wait, reject, run the task in the caller, or shed non-critical work.
In the example below, the pool uses a fixed worker count, a bounded ArrayBlockingQueue, and a configurable rejection policy.
Problem and Context
Your web server handles incoming requests by spawning a fresh thread for each one. Under light load this works fine. Under real traffic, it falls apart.
Three costs compound. First, each platform thread reserves a stack (often hundreds of kilobytes to about 1 MB, depending on the runtime and OS), so 10,000 threads can consume gigabytes of memory. Second, as the runnable thread count grows, the OS scheduler can spend more time context-switching than running useful work. Third, creating and destroying threads adds allocation and scheduling overhead. A single new Thread() call may take only microseconds on a given system, but repeated creation at high rates still adds up.
A common response is to stop creating a platform thread for every request and reuse a controlled set of workers instead. The pool makes that separation explicit.
Here is what changes when you apply the Thread Pool pattern.
Core idea
For a fixed-pool variant, pre-create a fixed number of worker threads. Tasks go into a shared blocking queue. Workers loop: take a task, execute it, and go back for the next one. Worker creation cost is paid when the workers are created, not for every task.
Participants and Structure
The main participants are:
ThreadPoolowns the workers, queue, lifecycle state, and submission policy.WorkerThreadwaits for work, runs one task, and then returns to the queue.BlockingQueuebuffers pending tasks and coordinates producers with workers.RejectionPolicydefines what happens when capacity is exhausted: fail, apply caller-side backpressure, or deliberately discard non-critical work.
When submit() is called, the task goes into the queue. Each WorkerThread loops, calling take() (which blocks if the queue is empty). The pool therefore separates the rate at which tasks arrive from the rate at which workers can execute them.
Implementation
Continue Reading with Premium
Unlock this article and every other in-depth system design guide on the platform with SDEpedia Premium.
Related Articles
Decouple producers from consumers using a shared bounded buffer so each side can work at its own pace without blocking or losing data.
The command pattern turns requests into objects so you can queue, log, and undo operations. Decouple sender from receiver by encapsulating actions.
The observer pattern decouples event producers from consumers. A subject notifies all registered observers of state changes without knowing who they are or what they do.