Document Vault

TL;DR

Separate document metadata from document bytes. Store nodes, version metadata, and access-control records in PostgreSQL; store document content, deltas, and snapshot anchors in object storage. Record edits as immutable versions using compressed deltas plus periodic full snapshots so reconstruction is bounded. Materialize effective permissions for fast reads, while preserving explicit document overrides. Index searchable content asynchronously through a durable change stream, and enforce authorization before returning search results.

The design assumes text-first documents and leaves real-time co-editing, large embedded binaries, comments, and export below the line. The key invariants are: a confirmed save has durable content and metadata, restore appends a new version, permission changes cannot overwrite explicit overrides, and the search index is derived rather than authoritative.

Scope and assumptions

• The service handles authenticated users creating text-oriented documents and folders, versioning them, sharing nodes, and searching content. Authentication and identity/group management are upstream concerns.
• The scale figures in this article are illustrative planning assumptions: 100M users, 1B documents, 50KB average current content, a 10:1 read/write ratio, and a 300-500ms user-facing latency budget.
• PostgreSQL, object storage, Redis, Kafka, and Elasticsearch represent logical roles. Equivalent managed or self-hosted systems can fill those roles.
• There is one metadata authority for a document at a time. Cross-region replication and backups satisfy the stated single-region-failure durability goal; active-active multi-primary editing is not required.
• Document bodies are immutable by version. Object-store URLs are short-lived delivery capabilities issued only after an authorization check.

What is a document management system?

A document management system lets users create, organize, and retrieve documents in a folder hierarchy, with version history and per-node access control. The interesting engineering challenge isn't the CRUD; it's storing 1,000 edits without consuming 50MB per document, enforcing permission inheritance across a deep folder tree without an O(depth) query on every read, and making 1 billion documents searchable under 500ms.

The useful framing is to surface the version-storage and ACL-inheritance trade-offs early because they drive different designs depending on which constraint is prioritized. The combination of storage efficiency, hierarchical access control, and full-text indexing makes this a broad system-design question.

---

Functional Requirements

Core Requirements

1. Users can create, read, update, and delete documents organized in folders.
2. The system maintains a version history for every document and supports restoring to any previous version.
3. Users can share documents and folders with specific people, controlling read, write, and manage permissions.
4. Users can search documents by title and full-text content.

Below the Line (out of scope)

• Real-time collaborative editing (concurrent multi-user editing using OT or CRDT)
• Large binary attachments embedded in documents (images, PDFs, videos)
• Comment threads on specific document sections
• Document templates and publishing or export to PDF

The hardest parts in scope: Version history storage and hierarchical access control. Storing 1,000 edits naively at 50KB per copy consumes 50MB per document. Permission inheritance across a deep folder tree requires careful design to avoid a slow tree walk on every read request.

Real-time collaborative editing is below the line because it requires Operational Transformation or CRDT-based merge logic plus a WebSocket broadcast layer. To add it, introduce a Collaboration Service that serializes concurrent operations and broadcasts change deltas to all connected clients over WebSocket, sitting alongside the Document Service rather than inside it. The server resolves conflicts by serializing concurrent ops against a shared document state, so two users editing the same paragraph see a merged result rather than a last-write-wins overwrite.

Large binary attachments are below the line because they introduce a separate upload flow (chunked multipart, virus scanning, CDN delivery) without changing document metadata or versioning semantics. To add them, store the binary in S3, embed an attachment_id reference link in the document body, and lazy-load the binary on the client. The document itself stays lightweight text; the attachment reference is just a URL pointer.

Comment threads are below the line because they introduce a separate content type with its own read patterns, including threading, reactions, and notifications. To add them, store comments as a separate entity anchored to a (document_id, anchor_offset) tuple, using the same ACL table to control visibility.

Document templates and export are below the line because they are rendering concerns that don't affect the storage or access control design. To add export, a background renderer consumes the latest document version and produces a PDF, storing it in S3 as a derived artifact.

---

Non-Functional Requirements

Core Requirements

• Scale assumption: 100M registered users, 1B documents total.
• Storage estimate: Average document 50KB; 50TB for text content. Version history adds 2-3x storage overhead with delta encoding, bringing the total to roughly 100-150TB.
• Write-latency target: Document save acknowledges in under 500ms p99.
• Read-latency targets: Document load completes in under 300ms p99. Search results return in under 500ms p99.
• Availability target: 99.9% uptime. Consistency over availability for document writes: a confirmed save must never be silently lost.
• Durability target: Document content and version history must survive any single-region failure.

Below the Line

• Sub-100ms global read latency via CDN edge caching (achievable with aggressive caching but not a core NFR here)
• Exactly-once change event delivery to the search indexer (at-least-once with idempotent indexing is sufficient)

Read/write ratio: 10:1. Documents are read far more than written. This opens the door for a Redis cache in front of hot metadata reads and justifies async search index updates rather than synchronous writes on every save.

The 500ms write latency budget is generous enough to absorb a synchronous write to PostgreSQL plus an async delta computation. It is not generous enough to also synchronously update the search index, so search replication goes through an async pipeline with a few seconds of lag.

Storage math drives a key design decision: 1B documents at 50KB each is 50TB for content alone. With version history averaging 10 deltas per document at 1-2KB per delta, that adds another 10-20TB. Delta encoding is not optional at this scale.

Run this math early in the interview. It anchors the capacity assumptions and makes the case for delta encoding before the storage design is chosen.

---

30-second answer / outline

“I would keep metadata and content separate: PostgreSQL stores the folder tree, current-version pointers, immutable version rows, and ACLs; object storage holds the document bytes. A save writes a compressed delta and periodic snapshot anchors, so restore can replay at most a bounded number of deltas. Effective permissions make reads a single indexed lookup, with overrides protected from inheritance. A committed change emits an idempotent event to an asynchronous search indexer, but authorization remains in the document service. The main trade-offs are write-time delta computation and permission propagation versus storage efficiency and read latency.”

5-minute explanation

1. Start with the boundary: the API and database own metadata and authorization; object storage owns large content blobs. This keeps database rows small and lets clients fetch authorized content through short-lived presigned URLs.
2. Walk the save path: write the new content to object storage, compute a delta against the parent version, insert the immutable version and current pointer transactionally, and publish a change event after commit. Snapshot anchors bound reconstruction; restore creates a new forward version rather than mutating history.
3. Walk the read path: authorize the node using materialized effective permissions, return metadata and a content capability, and reconstruct from the nearest anchor only when the requested version is not already a snapshot.
4. Add sharing and search: propagate folder grants asynchronously while honoring document-level overrides. Consume committed changes to build a derived full-text index, but apply an authorization filter and, for sensitive results, a final service-side check.
5. Close with trade-offs: full snapshots simplify reads but multiply storage; tree walks simplify writes but hurt tail latency; synchronous indexing improves freshness but couples writes to search availability.

45-minute interview approach

This is an interview plan for presenting the design, not a claim that the article should take 45 minutes to read.

• 0-5 minutes — Clarify scope: confirm text versus binary content, collaboration expectations, sharing semantics, version retention, search freshness, and the illustrative scale assumptions.
• 5-10 minutes — Requirements and capacity: state the read/write ratio, content-storage calculation, latency targets, availability, durability, and the consistency boundary between confirmed saves and search results.
• 10-17 minutes — API and data model: sketch node, version, ACL, and search interfaces; identify immutable version IDs, parent pointers, current-version pointers, and idempotency keys.
• 17-25 minutes — Baseline and evolution: draw metadata in PostgreSQL and content in object storage, then evolve full snapshots into deltas with periodic anchors. Walk create, save, read, and restore.
• 25-33 minutes — Authorization and search: compare explicit ACLs, ancestor traversal, and materialized permissions; explain overrides, revocation safety, asynchronous indexing, and search-result filtering.
• 33-40 minutes — Reliability, security, and operations: cover write-before-confirm ordering, durable change delivery, retries, orphan cleanup, regional recovery, presigned-URL expiry, ACL propagation lag, and search rebuilds.
• 40-45 minutes — Trade-offs and follow-ups: discuss collaboration, group principals, multi-region writes, retention, hot-document caching, and what would change if search freshness or global latency became stricter.

---

Core Entities

• Document: The primary content item. Contains document_id, title, owner_id, parent_folder_id, created_at, and updated_at. The content body lives in object storage; the database row holds only metadata and a pointer to the latest version.
• Folder: A container node in the hierarchy. Shares the same nodes table as documents, distinguished by a node_type discriminator column. Folders can nest arbitrarily deep.
• Version: A recorded revision of a document. Contains version_id, document_id, created_by, created_at, parent_version_id, and an S3 key pointing to the compressed delta (or a periodic full snapshot anchor).
• AclEntry: A permission record linking a principal (user or group) to a node (document or folder). Contains node_id, principal_id, permission (read, write, manage), and an is_override flag that stops permission propagation from overwriting this entry.

Full schema, index design, and delta format are covered in the deep dives. These four entities are sufficient to drive the API and high-level architecture.

---

API Design

One endpoint group per functional requirement, evolved where the naive shape breaks down.

FR 1 - CRUD on documents and folders: