How Zoom video conferencing works

The Problem Statement

Interviewer: "Millions of people use Zoom for video calls every day. Walk me through how a Zoom call actually works, from the moment you click 'Join Meeting' to live video appearing on your screen. How does it scale to meetings with hundreds of participants?"

This question tests four things: your understanding of real-time media transport (why UDP and not TCP), your knowledge of conferencing topologies (P2P vs MCU vs SFU), your reasoning about bandwidth constraints when participants have wildly different network conditions, and whether you can explain how the system degrades gracefully under packet loss rather than falling apart.

Most candidates say "it uses WebRTC" and stop there. That is like saying "the car uses an engine." The interviewer wants to know how the SFU decides which streams to forward, how simulcast handles a participant on 4G sitting next to someone on gigabit fiber, and what happens when 15% of packets get dropped on a congested network.

I find this question fascinating because video conferencing sits at the intersection of networking, codec design, and distributed systems. The latency budget is brutally tight: anything above 300ms round-trip feels like talking over a satellite phone. Every design decision is driven by that constraint.

Clarifying the Scenario

You: "Great question. Before I dive in, let me scope this."

You: "When you say 'how Zoom works,' should I focus on the media transport layer (how video packets move between participants) or the full stack including signaling, authentication, and the control plane?"

Interviewer: "Focus on the media side. Assume the user is already authenticated and in the meeting."

You: "Got it. Should I cover both small calls (2-5 people) and large meetings (100+ participants)? The architecture is different for each."

Interviewer: "Start with a typical 10-person meeting, then explain how it scales to larger meetings."

You: "One more thing. Should I cover end-to-end encryption? Zoom's E2EE story is interesting because it conflicts with the SFU architecture."

Interviewer: "Mention it, but do not spend too long on it."

You: "OK. I will structure my answer in four parts: the topology choice (why SFU beats P2P and MCU), how media actually flows through the SFU, how simulcast handles bandwidth differences between participants, and how the system handles packet loss without TCP retransmission."

My Approach

I break this into five parts:

1. Topology selection: Why Zoom uses SFU (Selective Forwarding Unit) instead of P2P or MCU, and the trade-offs of that choice
2. Media transport: How video and audio packets flow from sender to SFU to receivers using DTLS-SRTP over UDP
3. Simulcast and bandwidth adaptation: How senders transmit multiple quality layers and the SFU selects which layer to forward to each receiver
4. Packet loss recovery: How FEC (Forward Error Correction), NACKs, and codec concealment keep the call intelligible even at 15% packet loss
5. Large meeting optimization: Active speaker detection, stream count reduction, and how Zoom handles 100+ participants without melting everyone's CPU

The key insight I always emphasize: video conferencing is a real-time system with a hard latency budget. Every design choice (UDP over TCP, SFU over MCU, simulcast over transcoding) is driven by the need to keep round-trip latency below 300ms. Miss that budget, and the call feels broken.

The Architecture

Here is the full architecture of a Zoom meeting with 10 participants:

The critical path is simple: each participant encodes their video into multiple quality layers (simulcast), sends all layers to the SFU over encrypted UDP (DTLS-SRTP), and the SFU forwards the appropriate layer to each receiver based on their available download bandwidth. The SFU never decodes or re-encodes the video. It is a smart packet router, not a media processor.

The signaling path (SDP, ICE) happens before media flows. It uses WebSockets or HTTPS to exchange capabilities and discover the best network path. Once ICE succeeds and the DTLS handshake completes, media flows directly over UDP between clients and the SFU.

SFU Architecture and Stream Selection

The SFU is the heart of Zoom's media plane. Understanding why SFU wins over P2P and MCU is the most important part of this answer.

Let me break down why each topology fails or succeeds.

P2P works for 2-person calls. For N participants, each person sends N-1 streams and receives N-1 streams. With 10 participants, each person uploads 9 streams. Upload bandwidth scales as O(N), which kills mobile clients. A phone on 4G with 5 Mbps upload cannot send 9 concurrent video streams.

MCU solves the bandwidth problem. Each participant sends 1 stream to a central server. The MCU decodes all streams, composites them into a single grid view, re-encodes, and sends one mixed stream back. Upload: 1 stream. Download: 1 stream. Perfect bandwidth efficiency. But the MCU must decode and re-encode every frame in real-time. For a 10-person meeting at 720p/30fps, that is 10 decode operations and 10 re-encode operations per frame, 30 times per second. The CPU cost is enormous. Worse, the decode-mix-encode pipeline adds 100-200ms of latency, which pushes the total round-trip above the 300ms budget.

SFU is the sweet spot. Each participant sends 1 stream (or a few simulcast layers) to the SFU. The SFU forwards relevant streams to each receiver without decoding them. Upload: 1 stream. Download: up to N-1 streams, but in practice fewer because only visible participants get high-quality streams. The SFU's CPU cost is negligible (packet forwarding, not video processing). Latency addition: ~5-20ms (just routing delay). The trade-off: receivers must decode multiple streams, which requires more client CPU. But modern phones and laptops handle 4-9 decode operations easily.