Dropbox

TL;DR

Treat file synchronization as a metadata-coordination problem plus a bulk-byte-delivery problem. The client splits a file into illustrative 4MB SHA-256 chunks, sends a manifest, uploads only missing chunks directly to object storage through presigned URLs, and confirms an immutable file version in PostgreSQL. A change feed notifies other devices; each device downloads only chunks absent from its local cache. Offline concurrent edits become visible conflict copies, and reference-counted chunk metadata plus background garbage collection provides deduplication without deleting chunks still referenced by versions.

The design keeps the application tier off the byte path. The important invariants are that confirmation happens only after all referenced chunks are durably present, version manifests are immutable, conflict detection never silently discards an upload, and notification loss can be repaired from an authoritative cursor-based change feed.

Scope and assumptions

• The service covers authenticated personal and shared folders, file upload, sync to connected devices, version restore, and collaborator sharing. Public links, real-time co-editing, advanced team policy, and media previews remain below the line.
• The capacity figures are illustrative planning assumptions: 500M users, 100M DAU, 5 changes per DAU per day, 5GB maximum file size, 4MB example chunks, and a 5-second online-device sync target.
• PostgreSQL, object storage, Redis, and Kafka are logical roles. The design assumes a metadata authority per file and immutable chunk objects; multi-primary conflict-free metadata writes are not required.
• Redis Pub/Sub can provide low-latency local fan-out, but a durable outbox or log plus the cursor-based change feed is needed if notification loss must be recoverable.
• Hashes identify stored bytes for deduplication; if client-side encryption uses different keys per user, cross-user deduplication is intentionally reduced or lost.

What is Dropbox?

Dropbox is a cloud file-sync service: upload a file on one device and it appears on every other device you own within seconds. Three hard problems hide behind that simplicity: avoiding re-upload of unchanged file content on every edit, preserving both versions when two devices edit the same file offline, and deduplicating 500 petabytes of data where most bytes are identical across users. Content-addressed chunking is the architectural mechanism that solves all three at once.

This makes the question useful for discussing content addressing, chunked storage, conflict resolution in an eventually consistent system, hierarchical metadata, and notification delivery. Lead with the chunking insight because sync, deduplication, and versioning all depend on it.

Functional Requirements

Core Requirements

1. Users can upload files from any device.
2. Files automatically sync to all of a user's connected devices after an upload.
3. Users can share folders and collaborate with other users.
4. Users can view and restore previous versions of files.

Below the Line (out of scope)

• Public file sharing via link
• Real-time collaborative editing (that's Google Docs territory)
• Advanced team permissions, audit logs, and compliance reporting
• Media preview and thumbnail generation

The hardest part in scope: Efficiently syncing large files without re-uploading them in full on every edit. The mechanism behind this, content-addressed chunking with client-side delta computation, is the architectural keystone of the entire system.

Public link sharing is below the line because it introduces its own access-control and abuse-prevention surface without changing the upload or sync architecture. A later extension could generate a short-lived signed token for the file, store it in Redis with a configurable TTL, and serve downloads via a CDN edge with token validation. The file data path is identical to authenticated download.

Real-time collaborative editing is below the line because it requires operational transformation or CRDT logic at the application layer. Dropbox is a sync system, not an editor. Integrating collaborative editing would effectively mean building Google Docs alongside Dropbox.

Advanced team permissions are below the line because they introduce a role and policy evaluation tree that sits on top of, but does not change, the underlying file storage and sync architecture. A later extension could replace the simple folder_members table with a full RBAC model and evaluate policies at the API Gateway level on every request.

Media thumbnails are below the line because they are a read-side enhancement. A later extension could trigger an asynchronous job on each confirmed upload that generates thumbnails and stores them in S3 under a predictable key, then serve them via CDN with a separate GET endpoint.

Non-Functional Requirements

Core Requirements

• Durability target: Files must not be lost. An 11-nines durability target for chunk storage is an illustrative assumption; the actual guarantee depends on the selected object-store contract and replication policy.
• Availability target: 99.99% uptime. Availability over consistency: a sync delay of a few seconds is acceptable; losing a file or a confirmed upload is not.
• Consistency target: Online devices receive sync notifications within 5 seconds of a change. Offline devices sync on reconnect.
• Latency target: Upload begins streaming within 1 second of a file change being detected. Chunk uploads are parallel with no queuing at the application tier.
• Scale assumption: 500M users, 100M DAU. Average 1GB stored per user = 500 PB total. 5 file changes per DAU per day = 500M changes/day = ~5,800 writes/second at average load, peaking at 3x (~17,000/second) during business hours. Max file size: 5GB.

Below the Line

• Sub-100ms sync latency via edge routing
• HIPAA-compliant encryption key management per user

Read/write ratio: Roughly 5:1 reads to writes. Users download and open files far more often than they upload them. Reads are further amplified by multi-device sync: a single upload triggers downloads on all of a user's other devices. Write throughput is the harder engineering constraint at 17,000 peak writes/second; read throughput is handled by CDN caching of S3 chunks.

Use 5 seconds as the end-to-end sync target for online devices in this example. A filesystem event propagates in under 100ms, chunk hashing takes 1-2 seconds for a typical file, and the network upload of only changed chunks takes the bulk of that budget. The window is bounded by client-side hashing time as much as by server-side architecture, so the target should be validated with client and network measurements.

30-second answer / outline

“I would split files into content-addressed chunks and separate the control plane from the byte path. The client hashes illustrative 4MB chunks, declares a manifest, uploads only missing chunks directly to object storage, and confirms an immutable version in PostgreSQL. A durable change feed and low-latency fan-out notify other devices, which fetch only chunks they do not have. If a device uploads after another device changed the file, create a conflict copy instead of rejecting or overwriting it. Reference counts and background garbage collection protect shared chunks, while owner-scoped metadata indexes handle folder listings and sync cursors.”

5-minute explanation

1. Start with the bottleneck: full-file proxy uploads waste bandwidth when only a small part changed and make the application tier carry bulk bytes.
2. Evolve the upload path: hash chunks on the client, send a manifest, receive missing hashes and presigned upload URLs, upload chunks directly to object storage, then confirm the manifest in a transaction.
3. Explain sync: commit metadata before publishing a change; notify the owner's devices and collaborators; let each device compare manifests and download missing chunks in parallel.
4. Handle offline edits: compare the uploader's last processed change token with the current file version. A stale token creates a separately named conflict copy, preserving both manifests and letting the user resolve the conflict.
5. Close on storage and metadata: immutable chunk objects are shared by hash, version manifests hold references, reference counts feed garbage collection, and owner-scoped indexes support folder listing and cursor-based change feeds.

45-minute interview approach

This is an interview plan for the design question, not a claim that the article should take 45 minutes to read.

• 0-5 minutes — Clarify scope: confirm authenticated sync, devices, shared folders, file-size limits, offline behavior, version retention, and whether public links or co-editing are excluded.
• 5-10 minutes — Requirements and capacity: state the illustrative user/change assumptions, 5-second online sync target, durability/availability goals, read/write skew, and chunk-size trade-off.
• 10-17 minutes — APIs and entities: sketch manifest initiation, direct chunk upload, confirm, change-feed polling, sharing, version listing, restore, and the File/Chunk/Folder/FileVersion relationships.
• 17-25 minutes — Baseline and evolution: draw full-file upload first, quantify why it fails, then evolve to client-side chunking, presigned URLs, manifest confirmation, and immutable version pointers.
• 25-33 minutes — Critical flows: walk upload/confirm, device sync, collaborator fan-out, restore, and the ordering between metadata commits and notifications.
• 33-40 minutes — Deep dives: compare chunking strategies, conflict policies, reference counting/GC, metadata indexes, cache/CDN choices, encryption, and regional recovery.
• 40-45 minutes — Failure modes and follow-ups: cover retries, partial uploads, lost notifications, stale cursors, GC races, conflict-copy cleanup, hot shared folders, and what changes for public links or real-time collaboration.

Core Entities

• File: Metadata record for a file: file_id, name, owner_id, folder_id, size, content_hash (SHA-256 of the full file), created_at, updated_at, current_version_id.
• Chunk: A fixed-size (4MB) block of file data identified by its SHA-256 hash. Multiple files may reference the same chunk if they share content. Chunk data lives in S3; the registry lives in PostgreSQL.
• Folder: A container for files and sub-folders. Has folder_id, name, parent_folder_id, owner_id, and a materialised path string encoding ancestor folder IDs.
• FileVersion: Every confirmed upload creates a new FileVersion record linking to the chunk manifest at that point in time. Restoring a version is a metadata operation, not a data movement.
• User: Account with user_id, device list, and storage quota. Schema details are deferred to deep dive 4.

Schema details, indexes, and partition key design are covered in the metadata storage deep dive. The five entities above are sufficient to drive the API design and High-Level Design.

API Design

Start with one endpoint per functional requirement, then evolve where the naive shape breaks down.

FR 1: Upload a file (naive):

POST /files
Body: { name, folder_id, size, file_bytes }
Response: { file_id }

FR 1: Download a file:

GET /files/{file_id}
Response: { file_id, name, size, download_url }

These two endpoints satisfy the upload requirement on paper, but POST /files with raw bytes means the Upload Service must receive every byte of every file. For a 5GB file at scale, that is a bandwidth bottleneck that cannot be horizontally scaled away (you would need to buffer the entire file before writing to S3). The upload endpoint must be redesigned as a three-step protocol: declare the manifest, upload chunks directly to S3, confirm.

FR 1 (evolved): Declare chunk manifest:

POST /files
Body: { name, folder_id, size, content_hash, chunks: [{ index, chunk_hash }] }
Response: { file_id, missing_chunks: [chunk_hash], upload_urls: { chunk_hash: presigned_url } }

FR 1 (evolved): Upload a chunk directly to S3:

PUT {presigned_url}
Body: raw chunk bytes
Response: HTTP 200

FR 1 (evolved): Confirm upload complete:

PUT /files/{file_id}/confirm
Body: { chunks: [{ index, chunk_hash }], device_last_sync_token }
Response: { file_id, version_id }

Why three steps instead of one: The manifest step lets the server identify which chunks are already stored before any bytes transfer. The pre-signed URL step offloads bulk data transfer from the Upload Service directly to S3, so the Upload Service only handles lightweight metadata coordination regardless of file size. The confirm step is the authoritative write: metadata and version records are only committed once all chunks are durably in S3.

device_last_sync_token in confirm: The timestamp of the last sync event the uploading device processed. The server compares it to file.updated_at to detect whether another device modified the file while this device was offline. If so, a conflict copy is created rather than overwriting. Covered in deep dive 2.

FR 2: Poll for sync changes (long-poll):

GET /sync/changes?cursor={timestamp}&timeout=30
Response: { changes: [{ file_id, event_type, version_id }], new_cursor }

The cursor is the updated_at timestamp of the last delivered change. Long-poll with a 30-second timeout means the connection hangs open until an event fires or the timeout expires, then the client immediately re-connects. This avoids a persistent WebSocket per device while still delivering changes within seconds.

FR 3: List folder contents:

GET /folders/{folder_id}/contents
Response: { files: [...], folders: [...] }

FR 3: Share a folder:

POST /folders/{folder_id}/members
Body: { user_id, permission: "view" | "edit" }
Response: { share_id }

FR 4: List file versions:

GET /files/{file_id}/versions
Response: { versions: [{ version_id, size, created_at, created_by_device }] }

FR 4: Restore a version:

POST /files/{file_id}/restore/{version_id}
Response: { file_id, version_id }

Restore is a POST, not a PUT, because it creates a new version record pointing to the historical chunk manifest. No file bytes move; the metadata pointer changes. The response triggers a FileChangedEvent that syncs all connected devices.

High-Level Design

1. Users can upload files

Start simple: the client detects a file change, reads the file, and POSTs the raw bytes to the Upload Service, which writes them to S3 and records metadata in PostgreSQL.

Components:

• Client Sync Engine: Monitors the filesystem for changes and POSTs the full file bytes to the Upload Service.
• Upload Service: Receives file bytes, writes them to S3 under a generated key, records metadata in PostgreSQL.
• Object Store (S3): Stores file data, one object per upload.
• Metadata DB (PostgreSQL): Stores file records: file_id, name, owner_id, folder_id, size, s3_key.

Request walkthrough:

1. Client detects a modified file via filesystem watcher.
2. Client reads the entire file.
3. Client POSTs the raw bytes to POST /files.
4. Upload Service writes the bytes to S3 under a generated key.
5. Upload Service inserts file metadata into PostgreSQL and returns file_id.

This works for small files. It breaks immediately on the scale NFR: editing one paragraph of a 1GB file re-uploads 1GB. At 100M DAU making 5 edits per day on 50MB average files, that is 25 PB of daily upload traffic, most of it unchanged bytes. The Upload Service becomes a bandwidth bottleneck routing every byte through itself, and two users uploading the same Node.js installer store two full copies with zero sharing.

The naive path is useful as an interview starting point because its failure mode motivates each subsequent design decision.

Evolving the design: content-addressed chunking

The key insight is that most of a file's content does not change between edits. If each block of bytes is identified by its SHA-256 hash, the server can answer "which blocks are new?" without reading any file content (just check whether the hash is in the registry). The client then uploads only the missing blocks, in parallel, directly to S3.

Components (evolved):

• Client Sync Engine: Splits modified files into 4MB chunks, computes SHA-256 per chunk, and orchestrates a three-step upload: declare the manifest, upload only missing chunks directly to S3, confirm.
• Upload Service: Receives the chunk manifest, queries the Chunk Registry for missing hashes, returns pre-signed S3 PUT URLs for missing chunks, and writes metadata on confirm. Never handles raw chunk bytes.
• Chunk Registry (PostgreSQL): Maps chunk_hash → s3_key with a reference count. O(1) existence check per hash.
• Object Store (S3): Stores immutable content-addressed chunk data keyed by SHA-256 hash. Shared across users and versions.
• Metadata DB (PostgreSQL): Stores file and folder records, version history, and the chunk-to-file mapping.

Request walkthrough (evolved):