No Backpressure

Introduction

Backpressure is the feedback that limits work entering a component when the component cannot keep up. The anti-pattern is allowing producers to continue at full speed while an in-memory buffer grows without a clear bound or overload policy.

Mental model: a queue is a shock absorber, not an extra processor. It can smooth a finite burst, but it cannot make a producer rate greater than consumer capacity sustainable; when the buffer fills, the system must slow, reject, shed, or persist work deliberately.

TL;DR

• Backpressure is the mechanism by which a slow consumer signals to a fast producer to slow down. Without it, producers keep producing and queues grow unbounded.
• An unbounded in-memory queue can become a delayed crash: memory fills up gradually, then the process may OOM at the worst possible time (peak traffic, critical path).
• The three responses to backpressure: slow down (rate-limit the producer), shed load (drop or reject excess work), scale up (add consumers). Each has a different cost and latency implication.
• The most dangerous anti-pattern is silently swallowing backpressure: a queue that "just grows" looks healthy until it doesn't.

What It Is

Before: an unbounded work buffer

It's 2:47 a.m. on Black Friday. Your notification service receives events from Kafka and sends emails. During normal operation, events arrive at 500/second and you process 500/second. Perfect balance. Your monitoring dashboards are green. Queue depth hovers near zero.

Then traffic spikes. Events arrive at 5,000/second. Your email provider rate-limits you to 600/second. Your in-memory processing queue starts growing: 4,400 events per second being added, 600 per second being removed.

After 10 minutes: 4400 × 600 = 2,640,000 pending events. Your notification service's heap is at 8GB. It OOMs and crashes. On restart, it picks up where Kafka left off, still at 5,000 events/second. It crashes again in 10 minutes. Your on-call engineer wakes up to a PagerDuty storm: the service has restarted 6 times in the last hour.

The queue was the problem. It had no maximum size, so it silently absorbed every message the producer sent. No alert fired, no error logged, no metric crossed a threshold. The system looked healthy right up until the moment it died.

Meanwhile, customers received zero emails for a 10-minute window, then could receive 4.4 million emails in a flood once you scaled the consumer. Users could get 50 "your order was placed" emails. Support tickets spiked. The operational impact was visible to customers.

The root problem was no backpressure: the consumer had no mechanism to signal "slow down" to the producer, and the queue had no bound. The queue was the silent failure point, dutifully accepting every message without raising a single alarm.

Why It Happens

Backpressure is upstream-visible consumer slowness. When your consumer falls behind, what happens upstream? Without backpressure, the answer is "nothing." The producer keeps producing, the queue keeps growing, and the system looks healthy right up until it OOMs.

Teams end up here for a few individually-reasonable reasons:

"We'll scale the consumers before the queue gets too big." Manual scaling doesn't work at 3 a.m. when traffic spikes. By the time your on-call engineer wakes up, the queue has 4 million messages and the process is dead. Auto-scaling helps, but it takes 1-3 minutes to spin up new consumers. Without a bounded queue, the process may OOM before scaling completes.

"In-memory queues are faster than bounded ones." True, but an unbounded in-memory queue is a delayed crash. You're trading latency now for an OOM later. The queue absorbs all excess work invisibly, hiding the rate mismatch until it's catastrophic.

"Our consumers are fast enough." They are, until they aren't. A downstream dependency slows down, a database connection pool fills up, or traffic doubles on a holiday. A system can run for months with balanced rates and still fail on the first sustained traffic spike. Backpressure is insurance for the moment "fast enough" stops being true.

"We haven't had a problem yet." Survivorship bias. The absence of an OOM doesn't mean the queue is bounded. It means you haven't had a large enough traffic spike yet. Check your queue's maximum observed depth. If it's been growing month over month, you're trending toward a crash.

"The queue library doesn't have a size limit option." Many in-memory queue implementations default to unbounded. If you never set a max size, that queue may not provide useful backpressure. The default is the anti-pattern.

Real-world examples

This pattern shows up everywhere queues exist:

• HTTP request queues in web servers. Tomcat, Netty, and Node.js all have configurable request queue sizes. The default is often unbounded or very large. During a traffic spike, the server accepts thousands of connections it can't process, eats memory, and eventually crashes.
• Kafka consumer applications. The consumer reads messages into an in-memory buffer for processing. If the processing pipeline is slower than the read rate, the buffer grows until OOM. Kafka's max.poll.records and pause()/resume() are the backpressure mechanisms.
• Thread pool task queues. A ThreadPoolExecutor with an unbounded LinkedBlockingQueue will accept infinite tasks. When the pool is saturated, tasks pile up in memory. Use a bounded queue with a rejection policy (CallerRunsPolicy or AbortPolicy).
• gRPC streaming. A server-streaming RPC with a fast producer and slow consumer fills gRPC's internal buffers. Without flow control, the server allocates memory for every buffered message.

The common thread: every system that moves data between components at different speeds needs a bounded buffer and a strategy for when that buffer fills. If you grep your codebase for "new Queue()" or "new LinkedList()" used as a work buffer, check whether there's a size limit. If there isn't, you've found the anti-pattern.

How to Detect It

In Kafka, the key metric is consumer lag: the number of messages produced but not yet consumed. Alert when consumer lag increases over a 5-minute window. A lag-based alert can catch a rate mismatch 10-15 minutes before an OOM, depending on the buffer and alert thresholds.

Monitoring queries to set up now

# Prometheus: alert when consumer lag is increasing
rate(kafka_consumer_lag[5m]) > 0

# Prometheus: alert when process memory exceeds 80% of limit
process_resident_memory_bytes / container_memory_limit_bytes > 0.8

# Grafana: dashboard panel for queue depth over time
kafka_consumer_lag{consumer_group="notification-service"}

If you're using an in-memory queue without Kafka, instrument it yourself. Export a gauge metric for queue.size and alert when it exceeds 50% of your heap budget. The earlier you catch the growth, the more options you have.

The Fix

After: bounded buffering with an explicit overload policy

The three responses to backpressure are: slow down the producer, shed excess load, or scale consumers. In practice, most production systems combine two or three of these.

The important insight is that safe local buffering requires a bound and an explicit full-queue policy. Without a bound, an in-process queue can absorb everything while the producer never slows down and no load is shed. A durable broker can hold backlog outside the process heap, but consumer lag still needs a capacity and overload policy.

Fix 1: Bounded queues with blocking

Set a maximum queue size. When the queue is full, the producer blocks until space is available. This propagates pressure upstream, which is exactly what you want when the producer is within your control.

In Kafka, this is equivalent to using pause() on the consumer when your processing buffer is full, and resume() when space is available. The consumer stops fetching new messages until it can handle them. For HTTP-based producers, return 503 Service Unavailable with a Retry-After header.

class BoundedQueue<T> {
  private queue: T[] = [];
  private maxSize: number;

  constructor(maxSize: number) {
    this.maxSize = maxSize;
  }

  async enqueue(item: T): Promise<void> {
    while (this.queue.length >= this.maxSize) {
      await sleep(10); // backpressure: wait until space available
    }
    this.queue.push(item);
  }

  dequeue(): T | undefined {
    return this.queue.shift();
  }
}

Trade-off: Blocking the producer increases upstream latency. If your producer is an API server, users see slower responses. This is acceptable when "slow" is better than "crash."

The polling loop above is illustrative. A production queue should use a condition variable, semaphore, or library primitive so a full queue does not create a busy-wait loop.

Fix 2: Load shedding (drop or reject)

When the queue is full, reject the incoming item with a 429 Too Many Requests or drop it deterministically (e.g., drop oldest). This is the right response when you can't slow the producer down (external clients, third-party webhooks).

async function enqueue(item: Event): Promise<void> {
  if (queue.size >= MAX_QUEUE_SIZE) {
    metrics.increment('events.dropped');
    return; // shed load: drop the event
  }
  queue.push(item);
}

Trade-off: You lose data. For notification emails, losing a few during a spike is acceptable. For financial transactions, it's not. Choose shedding only when losing some messages is safer than losing all of them.

When shedding, always emit a metric (events.dropped) so you know it's happening. Silent shedding is better than silent queuing, but visible shedding is best.

Fix 3: Auto-scale consumers

Monitor queue depth and auto-scale consumer instances when lag exceeds a threshold. This is the elastic response, effective but slow (30-120 seconds for new containers). Always combine with bounded queues so the system survives the scaling delay.

The auto-scaling loop: consumer lag exceeds threshold, orchestrator spins up new consumer pods, new pods join the consumer group, Kafka rebalances partitions across consumers, throughput increases, lag decreases. The entire cycle takes 1-3 minutes. During that window, the bounded queue and load shedding policy keep the system alive.

// Auto-scaling rule (Kubernetes HPA config concept)
const scalingConfig = {
  metric: 'kafka_consumer_lag',
  threshold: 10_000,        // scale when lag > 10K messages
  scaleUpCooldown: '60s',
  maxReplicas: 20,
};

Trade-off: Infrastructure cost. You pay for the headroom to absorb spikes. But it's cheaper than an OOM crash during peak traffic.

The bottom line: long-running services need bounded in-process buffers and an explicit policy for a full queue. Blocking, shedding, scaling, or durable handoff are all valid choices; an unbounded buffer that "just grows" hides a rate mismatch and is usually unsafe.

Which fix to use?

Severity and Blast Radius

No backpressure is a high severity anti-pattern. When it triggers, the system may have no graceful degradation because it has not decided how to handle excess work.

The failure can be gradual and still end abruptly: memory grows with backlog until the process reaches its limit, while latency and lag increase beforehand. This makes it dangerous when monitoring watches only errors and misses queue depth, consumer lag, or heap trend.

The blast radius depends on what the queue feeds. If the overloaded queue is in your notification service, you lose emails for a window. If it's in your payment processing pipeline, you lose transactions. If it's in your API gateway's request queue, the entire service goes down.

Recovery can be fast once you add bounds (sometimes minutes for a configuration change), but the OOM crash can cascade. If the crashed service holds database connections, those connections may not release cleanly. The database can hit its connection limit, preventing other services from connecting too.

Trade-offs and Legitimate Exceptions

• Batch processing with finite input. If you're reading from a file with a known record count, an unbounded in-memory queue won't OOM because the input itself is bounded. The queue will drain.
• Development and testing. Unbounded queues simplify local development. You don't want backpressure slowing your test suite. Just don't let the dev config leak into production.
• Short-lived processes. A CLI tool that processes a CSV and exits doesn't need queue bounds. The process lifetime is the bound. If the input is finite and small relative to memory, the queue drains before it becomes a problem.
• Consumer is provably faster. If your consumer processes faster than any producer can emit (hardware-limited, not software-limited), backpressure adds complexity without value. But "provably" is doing a lot of work in that sentence. Network hiccups, GC pauses, and dependency slowdowns can all temporarily slow a "provably fast" consumer.

The key question to ask is: "What happens to this queue if the consumer stops for 5 minutes?" If the answer is "the process runs out of memory," you need a bound or durable handoff. If the answer is "the queue drains when the consumer resumes and the total data fits in memory," an unbounded queue may be acceptable for that finite, controlled workload.

30-Second Explanation

No backpressure lets a fast producer keep feeding a slower consumer until an in-memory queue consumes the process's resources. Bound the buffer, measure lag and depth, and choose what happens when it fills: block, reject or shed work, scale consumers, or hand off to durable storage.

5-Minute Explanation

First compare producer rate, consumer rate, and burst duration. A queue can absorb a finite burst, but a sustained positive difference grows without bound. Set a memory or message limit, alert on queue depth/consumer lag and heap trend, and define the full-queue behavior for each class of work.

Use blocking for controllable producers when preserving every item matters; use 429 or deterministic load shedding for best-effort work; scale consumers for bursts when capacity can catch up; and use a durable broker or workflow when work must survive process failure. Autoscaling is not a substitute for a policy when the long-term arrival rate exceeds capacity.

Common Mistakes and Misconceptions

• "Autoscaling is backpressure." Autoscaling adds capacity, but it has startup delay and cannot fix a sustained rate mismatch by itself.
• "A bounded queue means no data is lost." A full queue still requires a choice: block, reject, drop, or persist the item for later retry.
• "Kafka lag cannot cause an OOM." Kafka can keep the backlog outside the process, but an unbounded consumer buffer can still exhaust application memory while reading ahead.
• "Blocking is always safer." Blocking can tie up request threads, connections, and upstream resources; use rejection or shedding when latency must remain bounded.
• "One global queue policy fits every event." Preserve critical work and shed low-value work by priority when the business allows it.
• "A queue only needs a max length." Set the bound in bytes or estimated work as well when messages have highly variable sizes or processing costs.

Test Your Understanding

Recap

1. Without backpressure, a fast producer fills an unbounded queue until the process OOMs.
2. The failure can be delayed and quiet. The queue may grow until it hits a heap or broker limit at the worst possible time.
3. Teams end up here because the defaults are unbounded and the system looks healthy until it crashes.
4. The three responses to consumer lag: slow down the producer (block), shed load (drop), or scale the consumer (add workers). Combine two or three.
5. Bounded queues make backpressure visible: when the queue is full, you have to make a decision. Unbounded queues defer that decision until resource pressure or a crash.
6. Kafka consumer lag is the observability signal: alert when lag is increasing, auto-scale when it exceeds a time threshold.
7. Real-world examples include HTTP request queues, Kafka consumer buffers, thread pool task queues, and gRPC streaming buffers.
8. The most dangerous variant is silent swallowing: a queue that grows without any metric alerting on it.

Related Concepts

• Backpressure — The mechanics of propagating consumer capacity upstream.
• Message Queues — Delivery semantics, consumer lag, and durable buffering.
• Rate Limiting — Controls admission before work reaches a saturated component.
• Bulkhead Pattern — Contains resource exhaustion so one overloaded path does not consume every pool.
• Circuit Breaker — Stops calls to an unhealthy dependency while it recovers.