Networking

TL;DR

• Networking is the connective tissue of every distributed system. Every service you draw on a whiteboard needs to talk to other services — and how they talk determines your latency, reliability, and scalability ceiling.
• Three layers matter for interviews: IP handles addressing and routing, TCP/UDP handle reliable (or fast) delivery, and application protocols (HTTP, WebSockets, gRPC) define how your services exchange data.
• TCP is your default. Use UDP only when you can tolerate packet loss and need minimal latency (live video, gaming). QUIC is the modern upgrade path — mention it to impress, default to TCP.
• REST is your default API. Reach for GraphQL when flexible client queries matter, gRPC when internal service throughput is critical. For real-time push, SSE covers most use cases; WebSockets when you need bidirectional; WebRTC only for peer-to-peer audio/video.
• Load balancers distribute traffic and detect failures. L7 for HTTP traffic, L4 for WebSocket or raw TCP. Client-side load balancing for internal microservices. Always mention health checks.
• Networks fail. Retries with exponential backoff, idempotency keys, and circuit breakers are how you survive it — and every senior interviewer expects you to know them.

The Problem It Solves

Your microservices architecture looks perfect on the whiteboard. Five services, clean arrows, a database behind each one. You deploy to production and everything works — for about three hours.

Then the order service tries to call the payment service. The payment service is overloaded and takes 30 seconds to respond. The order service holds the connection open, eating one of its 200 threads.

More orders come in, each blocking on the payment service. Within four minutes, the order service has exhausted its thread pool — and now the API gateway can't reach the order service either. Users see 503s across the board.

Nothing crashed. No server died. The network between two services just got slow — and that slowness cascaded through your entire system because nobody thought about how services actually communicate.

This pattern is common in almost every first-attempt system design. Engineers draw boxes and arrows but treat the arrows as magic — instant, reliable, free. They're not.

Every arrow is a network call with latency, a protocol with overhead, and a connection that can fail. Understanding networking transforms those arrows from handwaving into deliberate engineering decisions.

The arrows between your services carry your system's entire communication burden — getting them right is the difference between a resilient architecture and a house of cards.

What Is It?

Networking is how independent machines exchange data across physical and virtual connections. In system design, it's the set of protocols, patterns, and infrastructure that determine how your services discover each other, communicate, handle failures, and scale.

Analogy: Think of a large hospital. Doctors, nurses, pharmacists, and lab technicians are all specialists (services) who need to coordinate patient care. They don't all stand in the same room shouting — they use pagers (UDP), phone calls (TCP), written orders on clipboards (HTTP), and real-time intercoms during surgery (WebSockets).

The pager is fast but you might miss a page, while the phone call guarantees you reach someone but takes time to dial. The clipboard order creates a paper trail but is slow. During surgery, you need immediate two-way communication — nothing else will do.

Networking in system design is about choosing the right communication channel for each interaction, understanding the cost of each choice, and designing for what happens when the channel breaks.

For your interview: know these three layers and what each one gives you. The network layer handles addressing (IP), the transport layer handles reliability (TCP) or speed (UDP), and the application layer handles your business logic protocol (HTTP, WebSockets, gRPC). Everything else is implementation detail you can skip unless asked.

How It Works

Let's trace a single web request end-to-end. When you type example.com into your browser, a carefully orchestrated sequence of protocol interactions unfolds across all three layers. Understanding this flow is the foundation for every networking decision in system design.

Here's what happened across those layers:

1. DNS resolution (Application Layer) — Your browser translates example.com into an IP address like 93.184.216.34. This lookup usually takes 1–50ms depending on caching.
2. TCP handshake (Transport Layer) — A three-way handshake (SYN → SYN-ACK → ACK) establishes a reliable, ordered byte stream. One round trip of latency before any data flows.
3. TLS handshake (Transport/Application) — For HTTPS, another 1–2 round trips to negotiate encryption. TLS 1.3 reduces this to one round trip; 0-RTT resumption eliminates it for returning visitors.
4. HTTP request/response (Application Layer) — Your browser sends GET / HTTP/1.1 with headers; the server returns 200 OK with the page content.
5. TCP teardown — A four-way handshake (FIN → ACK → FIN → ACK) closes the connection cleanly.

// The entire sequence above in a single line of application code:
const response = await fetch('https://example.com');
// Underneath: DNS lookup + TCP connect + TLS negotiate + HTTP transfer + TCP close
// Total latency: DNS (1-50ms) + TCP RTT (1-100ms) + TLS (1-100ms) + server processing

The key observation: one conceptual "request" involves many round trips at lower layers. The higher you go in the stack, the more convenient the abstraction — but also the more latency you're paying. This tension between convenience and performance surfaces in every protocol decision you'll make.

Every protocol decision you make in an interview carries this overhead. The question isn't just "what data do I send?" — it's "how many round trips does it cost and can I afford them?"

Key Components

The Networking Stack

While the full OSI model has 7 layers, only three consistently appear in system design interviews. Let's go through each one and understand what it gives us as application developers.

Network Layer — IP

The Internet Protocol (IP) handles two things: addressing (where is the destination?) and routing (how do packets get there?). Every machine on a network gets an IP address — either assigned by DHCP when it boots or configured statically.

Public IPs are routable across the internet. The backbone infrastructure knows that addresses starting with 17.x.x.x belong to Apple, and routes packets accordingly. Private IPs (like 10.0.0.x or 192.168.x.x) only work within a local network and require Network Address Translation (NAT) to reach the public internet.

For your interview: IP is plumbing. You almost never need to discuss it explicitly. The one exception is when you're designing for multi-region deployments — then you'll need to talk about IP-based routing, Anycast (multiple servers sharing one IP for geographic routing), and how DNS maps domain names to different IPs in different regions.

Transport Layer — TCP, UDP, and QUIC

This is where things get interesting for system design. The transport layer determines the reliability and performance characteristics of your communication.

The next section breaks down each protocol in detail, but here's the one-liner: TCP guarantees delivery and ordering at the cost of latency. UDP sacrifices both for speed. QUIC gives you TCP's reliability with UDP's performance — but it's still gaining adoption.

Application Layer — Where You Live

Everything above the transport layer is the application layer — HTTP, WebSockets, gRPC, DNS, and every custom protocol you might design. This is where 90% of your interview decisions happen.

The application layer runs in user space, meaning you control it entirely. Transport and below run in the kernel — fast, but inflexible. This distinction matters: changing your HTTP serialization format is a deploy, but changing TCP congestion control requires a kernel update across your fleet.

Most of your design decisions live at the application layer. The transport and network layers are infrastructure choices you make once and rarely revisit. For your interview: spend your time on application-layer decisions — that's where you have control and where interviewers expect depth.

Transport Protocols Deep Dive

For most system design interviews, the real choice is between TCP and UDP. QUIC is increasingly relevant but still supplementary knowledge. Let me walk through each one.

TCP — The Reliable Workhorse

Transmission Control Protocol (TCP) is a connection-oriented, reliable, ordered byte stream protocol. It guarantees that data arrives in the order it was sent, retransmitting anything lost along the way.

The connection is called a "stream" — a stateful, ordered channel between client and server. Two messages sent on the same stream arrive in the same order. TCP handles acknowledgement, retransmission, flow control (don't overwhelm the receiver), and congestion control (don't overwhelm the network).

Key characteristics:

• Connection-oriented: Three-way handshake before data flows
• Reliable delivery: Every byte acknowledged; lost packets retransmitted
• Ordering guaranteed: Bytes arrive in the order sent
• Flow control: Receiver advertises how much data it can handle
• Congestion control: Sender adapts rate to avoid network overload

TCP is the default for almost everything. If you're not sure which transport protocol to use, use TCP. Interviewers expect it as the baseline and won't ask you to justify it.

UDP — Speed Over Safety

User Datagram Protocol (UDP) is a connectionless, best-effort protocol. No handshake, no acknowledgements, no ordering. You fire packets into the void and hope they arrive.

What you get for that lack of guarantees is speed. UDP adds only 8 bytes of header (vs TCP's 20–60 bytes) and has zero connection setup overhead. The first byte of real data can be on the wire immediately.

Key characteristics:

• Connectionless: No handshake, no state, no teardown
• Best-effort delivery: Packets can be lost, duplicated, or reordered
• No flow/congestion control: Sender can blast at any rate
• Minimal overhead: 8-byte header, no ACK traffic

So why would anyone use a protocol that doesn't guarantee delivery? Because for some applications, getting data fast is more important than getting every packet.

When UDP wins:

• Live video/audio streaming — a dropped frame is invisible; a retransmitted frame arrives too late to display
• Online gaming — knowing where a player was 200ms ago is useless; you want their position now
• DNS lookups — small, stateless queries where retrying from scratch is faster than TCP handshake + retry
• Telemetry/metrics collection — losing 0.1% of data points doesn't affect aggregates

A reasonable default is: default to TCP in interviews. When you reach for UDP, you should be able to say exactly why packet loss is acceptable in your use case. If you can't articulate that in one sentence, stick with TCP.

QUIC — The Modern Compromise

QUIC is a transport protocol built by Google on top of UDP that provides TCP-like reliability with significant performance improvements. HTTP/3 runs on QUIC. It's gaining adoption rapidly — Chrome, Firefox, and Safari all support it, and Cloudflare and Google serve significant traffic over QUIC.

What QUIC fixes:

• Zero-RTT connection establishment — Returning clients can send data immediately, no handshake wait
• No head-of-line blocking — In TCP, one lost packet blocks all streams. QUIC multiplexes independent streams, so a lost packet only stalls the affected stream
• Built-in encryption — TLS 1.3 is mandatory and integrated into the handshake, reducing total round trips
• Connection migration — When your phone switches from Wi-Fi to cellular, the connection survives because QUIC identifies connections by ID, not by IP:port

For interviews, think of QUIC as "better TCP." Mention it when discussing mobile-first designs or global services — your interviewer will be impressed. But don't build your entire design around it; TCP is the safe, universal default.

Choosing Your Transport Protocol

Here's the honest decision framework. Most of the time this decision is obvious — the hard part is knowing the exceptions.

The bottom line: TCP until you have a specific reason for UDP. QUIC if you want bonus points and your clients support it.

Application Layer Protocols

The application layer is where most of your interview design decisions live. These protocols define how your services exchange data — and each one carries its own set of trade-offs around performance, flexibility, and complexity.

HTTP/HTTPS — The Web's Foundation

Hypertext Transfer Protocol (HTTP) is a stateless, request-response protocol. The client sends a request, the server sends a response, and neither remembers the other. Every web page, every API call, every image download — HTTP.

HTTP is stateless by design — and that's a feature, not a limitation. Stateless services are dramatically easier to scale: any server can handle any request because no server needs to remember previous interactions. Move session state to Redis, auth tokens to JWTs, and keep your HTTP servers as pure functions of (request → response).

Key concepts you should know:

HTTPS wraps HTTP in TLS encryption. For any production system, HTTPS is non-negotiable — your interviewer assumes it. Don't burn interview time explaining that you'll use HTTPS; just use it.