Producer consumer pattern
Decouple producers from consumers using a shared bounded buffer so each side can work at its own pace without blocking or losing data.
Introduction
The Producer-Consumer pattern separates creating work from processing work by passing items through a shared queue or buffer. Producers and consumers can run at different rates, while the buffer defines how much work can wait between them.
TL;DR / mental model: Think of a bounded queue as a waiting room. Producers put work into it, consumers take work out, and a full room applies back-pressure instead of allowing memory to grow without a limit.
Problem and Context
Your logging service writes log entries to a file. Every request handler calls logger.write() directly, which means the handler blocks until the disk write completes. Under normal load it takes 2 ms. Under burst traffic the disk saturates, writes take 200 ms, and your API response times collapse.
Two problems compound here. First, the producer (request handler) runs at network speed but the consumer (disk writer) runs at I/O speed. Coupling them means the fast side slows down to match the slow side. Second, if you add more request handler threads, they all contend on the same synchronized lock, making throughput worse, not better.
The usual fix is to put a buffer between the producer and consumer so they can run independently. The buffer does not make the disk faster; it makes the handoff explicit and gives the system a policy for bursts and overload.
Here is what changes when you apply the Producer-Consumer pattern.
Core idea
Insert a bounded buffer (often a blocking queue) between producers and consumers. Producers put messages into the buffer and return while capacity is available; when it is full, the configured policy may block, reject, drop, or spill work elsewhere. Consumers pull messages at their own pace. Neither side needs to know the other's implementation or speed.
When It Helps
Use the pattern when production and processing have separate lifecycles or rates, work can safely wait, and the system needs an explicit full-buffer policy. It is less useful for a synchronous operation where the caller must receive the result immediately.
Participants and Structure
Producer and Consumer never talk to each other. They only interact through the BoundedBuffer interface. put() blocks when the buffer is full (back-pressure), and take() blocks when the buffer is empty (no busy-waiting). The Message record carries data between the two sides. This decoupling is the entire point: producers and consumers can be added, removed, or scaled independently.
Idiomatic Example and Implementation Notes
Hand-rolled bounded buffer (wait/notify)
Understanding the low-level mechanics helps when reasoning about thread coordination. This implementation uses a circular array with synchronized, wait(), and notifyAll() for thread coordination. Application code should normally use a library queue unless it needs a custom policy.
Continue Reading with Premium
Unlock this article and every other in-depth system design guide on the platform with SDEpedia Premium.
Related Articles
Reuse a fixed set of worker threads to execute tasks from a shared queue, avoiding the overhead of creating and destroying threads per request.
The observer pattern decouples event producers from consumers. A subject notifies all registered observers of state changes without knowing who they are or what they do.
The command pattern turns requests into objects so you can queue, log, and undo operations. Decouple sender from receiver by encapsulating actions.