How WebSockets work (and when to use them)

The Problem Statement

Interviewer: "You are building a live chat feature for your application. Users need to see messages from other participants within 200ms of them being sent. Walk me through how WebSockets work, how you would scale them to a million concurrent connections, and when you would pick a simpler alternative like Server-Sent Events instead."

This question tests three things: whether you understand the WebSocket protocol at the handshake and frame level, whether you can design a horizontally scalable real-time system (not just "open a WebSocket"), and whether you have the judgment to choose the right tool. Many candidates default to WebSockets for everything real-time without considering SSE or long polling.

I find that the strongest candidates explain the upgrade handshake, mention the frame format briefly, and then spend most of their time on the hard part: scaling persistent connections across multiple server instances. That is where the interview differentiates.

WebSocket (RFC 6455, standardized in 2011) is one of those protocols that sounds simple but has deep operational implications. The protocol itself is straightforward. The engineering challenge is everything that surrounds it: connection lifecycle management, horizontal scaling, graceful degradation, and knowing when a simpler tool is the right choice.

Clarifying the Scenario

You: "Before I jump into the protocol, let me scope the problem."

You: "When you say live chat, are we talking about a group chat (like Slack channels with hundreds of participants) or 1-on-1 messaging (like WhatsApp)? The fan-out pattern is different."

Interviewer: "Group chat, similar to a Slack channel. Assume up to 500 people in a channel."

You: "Got it. And when you say a million concurrent connections, do you mean a million users connected via WebSocket at the same time across all channels?"

Interviewer: "Yes. Assume 1M concurrent WebSocket connections to your backend."

You: "One more question: do messages need to be persisted, or is this fire-and-forget like a live stream chat?"

Interviewer: "Persisted. Users who reconnect should see messages they missed."

You: "OK, I will structure my answer in three parts: first, how the WebSocket protocol works at the handshake and frame level. Second, how to scale WebSocket connections horizontally using a pub/sub layer for cross-server fan-out. Third, a decision framework for when to use WebSockets versus SSE versus long polling."

My Approach

I break this into four parts:

1. The upgrade handshake: How an HTTP connection becomes a WebSocket connection via the 101 Switching Protocols response, and why this matters for proxies and load balancers.
2. The frame protocol: How WebSocket frames carry data bidirectionally, including opcodes, masking, ping/pong heartbeats, and the close handshake.
3. Scaling horizontally: The fundamental challenge that a WebSocket connection is stateful and pinned to one server, and how pub/sub (Redis, NATS, Kafka) solves cross-server fan-out.
4. Decision matrix: When WebSockets are the right choice versus SSE or long polling, based on directionality, connection count, and operational complexity.

The core insight is that WebSocket itself is simple. The hard part is building reliable infrastructure around it: connection management, heartbeats, reconnection, state synchronization across servers, and graceful degradation when connections drop.

Here is a quick comparison of the raw protocol overhead that explains why WebSockets win for high-frequency real-time messaging:

At 1,000 messages per second, HTTP polling generates 200-800KB/s of pure header overhead. WebSockets generate 2-14KB/s. That is a 50-100x reduction in bandwidth for the same message throughput.

The Architecture

Here is the full picture of a WebSocket-based chat system at scale. The key architectural element is the pub/sub layer that bridges messages between server instances, because a user in Channel A might be connected to Server 1 while another user in the same channel is connected to Server 2.

Walk through what happens when Alice sends a message:

1. Alice's browser sends a WebSocket frame containing the message to Server 1.
2. Server 1 writes the message to the persistent store (Cassandra/DynamoDB) and the recent-messages cache (Redis sorted set).
3. Server 1 publishes the message to channel:chat-1 on Redis Pub/Sub.
4. All three servers are subscribed to channel:chat-1. Each server receives the message.
5. Server 1 pushes the message to all locally connected users in that channel (Alice gets her own message back as confirmation, and Carol sees it).
6. Server 2 pushes it to Bob. Server 3 pushes it to Dave.
7. Total latency: typically 10-50ms from send to delivery across all participants.

The load balancer must be L4 (TCP-level), not L7 (HTTP-level), because WebSocket connections are long-lived TCP connections. An L7 load balancer would need to understand the WebSocket upgrade and maintain the connection, which adds complexity. Most teams use a network load balancer (AWS NLB, HAProxy in TCP mode) that simply forwards TCP connections.

Let me give you the capacity math. Each WebSocket server running on a standard cloud instance (8 vCPU, 32GB RAM) can handle roughly 200K-500K concurrent connections, depending on message rate and payload size. For 1M connections, you need 2-5 servers minimum. The bottleneck is almost always memory, not CPU, because each connection holds buffer state even when idle. A message rate of 1,000 messages/second across all connections uses less than 5% CPU on a modern server, but the connection state alone from 500K connections consumes 10-25GB of RAM.

The Upgrade Handshake and Frame Protocol

The WebSocket lifecycle starts as a normal HTTP request and then "upgrades" to a persistent bidirectional connection. Understanding this handshake is important because it determines how WebSockets interact with proxies, CDNs, and load balancers.

The handshake in detail

The client sends a regular HTTP GET with two special headers: Upgrade: websocket and Connection: Upgrade. It also sends a Sec-WebSocket-Key, which is a random base64-encoded 16-byte nonce. The server concatenates this key with a fixed GUID (258EAFA5-E914-47DA-95CA-C5AB0DC85B11), takes the SHA-1 hash, and returns the base64 of that hash as Sec-WebSocket-Accept.

This handshake has a specific purpose: it proves that the server actually understands the WebSocket protocol and is not just a regular HTTP server echoing back requests. The fixed GUID prevents caching proxies from caching the upgrade response and serving it to non-WebSocket clients.

After the 101 response, the HTTP connection is "upgraded." The TCP socket is now a WebSocket connection, and both sides communicate using WebSocket frames, not HTTP.

Frame format

Every WebSocket message is wrapped in a frame with this structure:

Two important details here. First, all client-to-server frames must be masked with a random 4-byte key. This prevents cache poisoning attacks where a malicious client could send data that looks like a valid HTTP response to a caching proxy. Second, messages can be fragmented across multiple frames using the FIN bit, which is useful for streaming large payloads without buffering the entire message in memory.