Collaborative Docs

What is a collaborative document editing system?

Google Docs lets multiple people edit the same paragraph at once while seeing each other's cursors move in real time. The engineering challenge is not the UI. It is that two users can type at the same cursor position within the same millisecond, and both keystrokes must survive without either one disappearing. This question tests real-time communication protocol design, distributed state merge algorithms (OT vs CRDTs), and the append-only operation log patterns that let a document survive server crashes mid-session.

TL;DR

Keep a durable document and operation log behind a WebSocket collaboration tier. Each operation carries enough revision or CRDT identity information for the chosen merge algorithm. A server-controlled OT design serializes and transforms stale operations; a CRDT design uses stable element IDs and deterministic merge rules. The final architecture below chooses a CRDT library while retaining server revisions for durability and recovery.

Persist operations append-only, materialize periodic snapshots for bounded load time, and use Redis Pub/Sub only for cross-instance fan-out and ephemeral presence. Reconnects must resynchronize from a durable revision or CRDT state; a Pub/Sub message is not the source of truth. Route collaborators for a document consistently enough to manage connections, but make every server recoverable from storage.

Scope and assumptions

The following are illustrative interview assumptions and boundaries:

• Plain-text editing is the primary model. Rich media, tables, and embedded objects need a tree-structured document model and are outside the main path.
• Up to 10 million daily active users, 500,000 active documents, up to 100 editors per document, and approximately 1 million operations per second are planning inputs. Validate the distribution of hot documents and editor activity before sizing.
• A server acknowledgement target of under 100ms and broadcast target of under 500ms p99 are used for the collaboration contract. The user should receive local optimistic feedback immediately, but durable acknowledgement is not the same as peer broadcast.
• Operation history is retained for 90 days and compressed snapshots are retained longer. Presence is ephemeral and may be reconstructed after reconnect.
• The article compares OT and CRDTs. Pick one algorithm for a concrete deployment; do not mix numeric-position transforms and CRDT element semantics without an explicit adapter.

Functional Requirements

Core Requirements

1. Multiple users can edit the same document simultaneously with their changes visible to all collaborators.
2. Changes from all editors appear in every connected client within 500ms.
3. Concurrent edits never overwrite or lose each other's work.
4. Documents are persisted durably and can be reopened after the session ends (or after a server crash).

Below the Line (out of scope)

• Rich media editing (images, tables, embedded spreadsheets) - focus on plain text editing.
• Version history and rollback - the operation log makes this possible, but building the UI and diffing logic is a separate concern.
• Spell check, grammar suggestions, and AI writing assistance.

The hardest part in scope: Merging concurrent edits without data loss. When two users type at the same position at revision 42, both operations claim to be "against revision 42." Applying them naively in arrival order destroys the second user's intent. The merge algorithms are covered in a dedicated deep dive.

Rich media editing is below the line because it changes the data model fundamentally. Text is a linear sequence of characters. Images and tables are embedded objects with their own dimensions and layout properties. Adding them requires a tree-structured document model instead of a flat string, which is a substantial project separate from the concurrency problem. An extension could model the document as a tree of blocks (similar to Notion's block model) and apply CRDT semantics at the block level.

Version history and rollback is below the line but becomes a read-path extension once a persistent operation log exists. Every operation is recorded with a user ID and timestamp. A rollback query can reconstruct the document at revision N, but production use still needs permissions, conflict handling, retention, and a user-facing diff/restore policy.

Non-Functional Requirements

Core Requirements

• Consistency: Zero data loss. Every committed operation survives a server crash. Concurrent edits must both appear in the final document with no operation silently discarded.
• Latency: Server acknowledges each operation in under 100ms. The operation broadcasts to all collaborators within 500ms (p99).
• Scale: 10M DAU, up to 500K concurrently active documents. Support up to 100 simultaneous editors per document. Peak system-wide operation rate: approximately 1M ops/sec during business hours.
• Availability: 99.99% uptime for document serving. A user should never lose their work due to an infrastructure failure mid-session.
• Storage: Retain the full operation history for 90 days. Keep compressed snapshots indefinitely.

Below the Line

• Sub-50ms operation propagation (WebSocket pub/sub achieves 100-200ms; sub-50ms requires region-local servers and is outside scope)
• Real-time spell-check or grammar feedback

Write pattern: Collaborative documents are write-intensive in a way most systems are not. During an active session with 10 users typing at normal speed (40 WPM), a single document generates roughly 40 operations per second. At 500K concurrently active documents with an average of 3 active users each, peak write load hits approximately 60K ops/sec. The storage layer must handle this without becoming a bottleneck on the hot path.

The 100ms server-acknowledge target is deliberately generous. It lets us use a standard relational database for the operation log on the hot path rather than forcing a specialized write buffer. Above 100ms, users perceive their own keystrokes as laggy, which breaks the feeling of local responsiveness.

The 99.99% availability target drives the replication strategy. We need at least two replicas of the operation log, and the WebSocket serving layer must restart sessions transparently on instance failure.

30-second answer / outline

1. Store documents, an append-only operation log, and periodic snapshots in durable storage.
2. Use WebSockets for low-latency operation delivery and a per-document collaboration owner/route for connection management.
3. Choose one merge algorithm: OT transforms stale numeric-position operations at a serialization point; CRDTs use stable element IDs and deterministic merges. The final architecture uses a CRDT library.
4. Publish committed operations through Redis Pub/Sub to other collaboration instances, but recover from the durable log after disconnects or crashes.
5. Keep cursors and heartbeats in ephemeral presence state, snapshot every bounded operation count, and monitor lag, replay windows, hot documents, and tombstone growth.

5-minute explanation

Start with the user-visible contract: local edits should feel immediate, peers should see committed changes within the broadcast budget, and concurrent edits must converge without data loss. HTTP polling wastes work and adds bounded staleness, so a WebSocket carries operations, acknowledgements, cursor updates, and heartbeats.

The collaboration server serializes or merges operations according to the chosen algorithm and appends the result to durable storage. OT uses a client revision and transforms a stale position against committed operations. A CRDT uses globally unique element IDs, parent references, and tombstones so operations can be merged in different arrival orders. Both approaches need a clear wire protocol and validation.

The operation log is the durable source of truth. Snapshots make document load practical by limiting replay to a recent tail. Redis Pub/Sub only distributes messages between instances; a reconnecting client asks for missing operations or a fresh CRDT state. Presence is separate ephemeral state and may disappear without corrupting the document.

Scale comes from routing connections, isolating hot documents, batching or compressing transport where appropriate, and keeping snapshot/compaction work off the keystroke path. Reliability comes from atomic persistence, idempotent operation IDs, gap detection, replay, and explicit backpressure. The deep dives compare merge and storage choices in detail.

45-minute interview approach

This is a time-boxed plan for answering the design question, not a claim that the article should be read in 45 minutes.

• 0–5 minutes — Clarify the contract: Confirm plain text versus rich text, offline editing, maximum editors/document, ordering guarantees, version history, cursor semantics, and acceptable propagation delay.
• 5–10 minutes — Establish scale: Use the illustrative DAU, active-document, editor, operation-rate, retention, and latency assumptions. Ask whether traffic is dominated by a few hot documents.
• 10–15 minutes — Define APIs and invariants: Walk through create/list/open, WebSocket sync, operation acknowledgement/broadcast, cursor presence, revision/operation IDs, and reconnect behavior.
• 15–22 minutes — Draw the real-time path: Show clients, load balancing/routing, collaboration servers, durable operation store, and cross-instance fan-out. Explain why polling fails.
• 22–30 minutes — Deep dive on consistency: Compare OT and CRDTs with the same concurrent-insert example, choose one for the deployment, and state how gaps, duplicate operations, and offline edits are handled.
• 30–35 minutes — Deep dive on persistence: Show append-only operations, snapshots, atomic snapshot pointers, retention, compaction, and load/recovery paths.
• 35–41 minutes — Reliability, security, and operations: Cover crash recovery, backpressure, hot-document isolation, authorization, document access checks, encrypted transport/storage, presence expiry, and monitoring.
• 41–45 minutes — Trade-offs and close: Discuss rich-text extensions, version history, P2P, Redis failure, multi-region routing, recap the source of truth, and invite follow-ups.

Core Entities

• Document: The container. Carries a document ID, title, owner user ID, created timestamp, and a pointer to the latest materialized snapshot revision.
• Operation: One atomic edit event. Carries an op ID, document ID, authoring user ID, the client revision it was written against, the server-assigned revision after serialization, op type (insert or delete), position, and content.
• Snapshot: A materialized full-text copy of the document at a specific revision. Used on load to avoid replaying thousands of operations from scratch.
• Session: A live editing session. Carries session ID, document ID, user ID, WebSocket connection ID, current cursor position, and last-heartbeat timestamp.
• User: Account entity. Carries user ID, display name, and a color used for cursor rendering in the editor.

The full schema (indexes, foreign keys, partition strategy) is deferred to the data model deep dive. These five entities drive the API and High-Level Design.

API Design

Group endpoints by the functional requirement they satisfy.

FR 1 and FR 4 - Create, list, and open documents:

POST /documents
Body: { title }
Response: { doc_id, title, created_at }

A POST because this creates a resource. The response gives back the doc_id the client uses for all subsequent calls.

GET /documents/{doc_id}
Response: { doc_id, title, content, revision, owner_id }

The revision field in the response is critical. The client uses it to stamp every outgoing operation with the document revision it was written against. Without it, the server cannot detect concurrent edits.

GET /documents
Response: { documents: [...], next_cursor: "..." }

Cursor-based pagination over the user's document list. Users with hundreds of documents need paginated results.

FR 2 and FR 3 - Real-time editing and conflict-free merges:

The naive approach is a REST endpoint:

POST /documents/{doc_id}/operations
Body: { revision, op_type, position, content }
Response: { server_revision }

This fails the 500ms latency requirement immediately. Every collaborator would need to poll GET /documents/{doc_id}/operations?since=revision to pick up others' changes. With 100 editors polling every 100ms, that is 1,000 HTTP requests per second for a single document. Worse, polling introduces up to 100ms of additional latency per poll cycle.

The evolved shape uses a persistent WebSocket connection:

WebSocket: wss://collab.example.com/documents/{doc_id}

Client connects → Server sends: { type: "sync", content, revision }

Client sends:
{ type: "op", revision: 42, op_type: "insert", position: 15, content: "hello" }

Server sends to client (acknowledgment):
{ type: "op_ack", server_revision: 43 }

Server broadcasts to all other clients on this document:
{ type: "op_broadcast", user_id: "u-123", server_revision: 43,
  op_type: "insert", position: 15, content: "hello" }

The client sends its local revision in every operation. The server assigns a monotonically increasing server_revision and broadcasts to all other connected clients. This is the entire real-time editing contract.

FR - Live presence (bonus, out of scope for NFRs but cheap to add):

Client sends over existing WebSocket:
{ type: "cursor", position: 47 }

Server broadcasts to other clients:
{ type: "cursor_update", user_id: "u-123", position: 47 }

Presence piggybacks on the same WebSocket connection, so it does not require a second client transport. Add it when presence is part of the product contract; keep it logically separate from durable document state.

High-Level Design

Critical flows

Follow four flows: document load and reconnect; live operation delivery; conflict resolution and convergence; and snapshot/presence maintenance. The durable operation log is the recovery boundary, while WebSockets and Redis are delivery mechanisms around it.

1. Users can create and open documents

The document load path: fetch the latest snapshot, replay any operations applied after that snapshot, and return the reconstructed content.

Components:

• Client: Web browser running the editor UI.
• API Server: Handles document CRUD, serves document content on load.
• Document DB (PostgreSQL): Stores documents, operation log, and snapshots.

Request walkthrough: