Design Instagram

TL;DR

Split the system into two asynchronous pipelines. The upload path stores post metadata, lets the client upload image bytes directly to object storage, and uses workers to create multiple resolutions. The feed path publishes a post only after media is ready, then uses hybrid fan-out: push regular-author posts into per-user Redis feeds, while fetch-and-merge handles high-follower authors at read time. Cassandra tables serve profile queries and post hydration, and a CDN with signed URLs serves processed photos close to users.

Scope and assumptions

• The design covers photo upload, follow/unfollow, a reverse-chronological home feed, and a user's profile grid. Likes, comments, discovery, Stories, Reels, and direct messages are below the line.
• The baseline is illustrative: 500M daily active users, 100M photo uploads per day, 10 feed loads per active user per day, 12 posts per feed page, and an average of 300 follows. Real traffic is bursty and should be measured by geography, client, and account type.
• The influencer threshold (50K followers), feed cap (800 post IDs), activity TTLs, signed-URL TTL, CDN cache policy, and media resolutions are tunable choices. The raw feed-ID payload for 500M users at 800 IDs and 8 bytes per ID is about 3.2 TB before Redis metadata, replication, and overhead.
• Post IDs are assumed to be time-sortable for the examples. If strict global chronological ordering is required, store created_at and a tie-breaker in the feed entry; a distributed ID alone is not a universal ordering guarantee.
• Object-storage durability, CDN latency, and provider behavior are targets or assumptions for capacity planning. Verify the selected provider's durability, egress, signed-URL, invalidation, and lifecycle semantics before committing to them.

The calculations in this article use approximate decimal units and are meant to show how to reason about the workload, not to claim a production capacity or product guarantee.

What is Instagram?

Instagram is a photo-sharing social network where users upload images, follow other accounts, and scroll through a personalized feed of photos from people they follow. The apparent core is simple: upload a photo, show it to followers. The hard part is everything underneath.

Photos need resizing into multiple resolutions before delivery, durable storage at petabyte scale, and global serving under an illustrative 100ms target. The feed must merge photos from hundreds of followed accounts for 500 million daily users without touching a database on every scroll. Separate the upload write path from the fan-out read path: they have different workload shapes, failure modes, and scaling controls. That separation is the central architectural decision.

Functional Requirements

Core Requirements

1. Users can upload a photo with a caption.
2. Users can follow and unfollow other users.
3. Users can view their home feed: reverse-chronological photos from accounts they follow.
4. Users can view a profile page: the photo grid of all posts by a specific user.

Below the Line (out of scope)

• Engagement features (likes, comments, reactions) and content discovery (Explore, search)
• Stories (24-hour ephemeral photos)
• Reels (short-form video)
• Direct messages

The hardest part in scope: Generating the home feed. At the illustrative 500M DAU and 10 refreshes per day, assembling a feed on demand by querying across all followed users is not viable. Pre-computing feeds and delivering photos globally via a CDN under the illustrative 100ms target is the axis on which the design turns.

Engagement features (likes, comments, reactions, Explore, and search) are below the line because they do not change the upload or feed delivery paths. A likes extension could use a post_likes table keyed by (post_id, user_id) and cache the count in Redis per post. Search would require a separate search index such as Elasticsearch consuming post creation events from Kafka for full-text caption indexing.

Stories are below the line because they introduce a separate ephemeral storage lifecycle and a dedicated stories feed that does not interact with the home feed pipeline. A possible extension could store story metadata with a 24-hour TTL and reuse the same S3/object-storage and CDN path for media delivery.

Reels introduce video transcoding, converting the upload pipeline into a multi-step encoding job. A possible extension would add a video transcoder and Adaptive Bitrate (ABR) manifest generation alongside the image resizing workers.

Direct messages require a separate real-time messaging system. A possible extension would use WebSocket connections through a dedicated chat service backed by a Cassandra message store, entirely separate from the feed and media systems.

Non-Functional Requirements

Core Requirements

• Availability: 99.99% uptime. Availability over consistency for feeds: a feed missing the last 30 seconds of posts is acceptable; a failed feed load is not.
• Durability: Photos are never silently dropped or corrupted. The baseline assumes an object-store durability target commonly expressed as 11 nines; the selected provider and replication policy must be verified.
• Latency: Home feed loads under 200ms p99. Photo delivery (the image bytes) targets under 100ms from major geographies when cached. Upload acknowledgment targets under 1 second; media processing remains asynchronous.
• Scale: 2B registered users, 500M DAU. Approximately 100M photos uploaded per day (~1,160 uploads per second, peaking at ~3,500 per second during events).
• Read throughput: Each active user loads their feed ~10 times per day. That is 5B feed loads per day, ~58K per second peaking at ~175K per second. Each feed load fetches 12 photos.

Below the Line

• Sub-10ms photo delivery via CDN edge-node pre-warming
• Real-time like-count consistency in feed

Read/write ratio: For every 1 photo uploaded, expect roughly 600 photo views (100M uploads per day vs 5B feed loads at 12 photos each). But the more important number is write amplification on the feed cache. With an average of 300 follows per active user, each photo upload triggers up to 300 feed cache writes. That is 30B feed-cache updates per 100M uploads per day. This fan-out multiplier, not the raw upload rate, drives the infrastructure decisions in this article.

The design targets 200ms p99 for feed loads and accepts eventual consistency on the feed: a user missing the last 30 seconds of posts is preferable to a failed page load. That target rules out assembling the feed on the read path by querying the database for each followed user. The 1-second upload acknowledgment budget requires decoupling media processing from the upload response, and the 99.99% availability target requires redundancy in the hot read path.

30-Second Answer / Outline

1. Clarify photo-only scope, feed ordering, privacy, media sizes, follower distribution, geography, and the freshness/latency trade-off.
2. Use a two-phase upload: create post metadata and a pre-signed object-storage URL, then let the client upload bytes directly and confirm completion.
3. Process media asynchronously into immutable variants and publish PostPublishedEvent only after the variants are ready.
4. Store the follow graph in both directions, push regular-author post IDs into Redis feed caches, and merge high-follower authors on read.
5. Keep Cassandra tables for the two post access patterns—by user for profiles and by ID for hydration—and serve photo bytes through a signed-URL CDN.
6. Close with idempotent events, inactive-user suppression, deletion behavior, cache misses, and observability.

5-Minute Explanation

The upload request creates a post row in pending state and returns a short-lived pre-signed object-storage URL. The client sends the large binary directly to object storage and calls the confirmation endpoint. The Post Service publishes a durable MediaUploadedEvent; workers fetch the raw object, generate thumbnail/medium/full variants, store them, update the denormalized post rows, and publish PostPublishedEvent only when the media is ready.

The feed is a derived view. For regular authors, feed workers read the reverse follow adjacency list and add the post ID to each active follower's Redis sorted set. For high-follower authors, workers skip the large push and the Feed Service fetches a small recent set at read time, then merges it with the pre-computed IDs. The feed service hydrates IDs from the post-by-ID table and signs CDN URLs only after access checks.

Profile reads use the post-by-user table with a cursor. A CDN caches immutable processed variants near users; signed URLs and deletion invalidation limit unauthorized or stale access. Kafka's at-least-once delivery is safe when media processing, feed writes, deletion, and reconciliation are idempotent.

45-Minute Interview Approach

This is a discussion plan for the design question, not a claim that the article should take 45 minutes to read.

• 0-5 minutes — Clarify the contract: Confirm photo-only scope, feed semantics, privacy, ordering, freshness, upload acknowledgment, geography, and follower skew.
• 5-10 minutes — Estimate the workload: Use the illustrative 500M DAU, 100M uploads/day, 5B feed loads/day, 12 posts/page, and 300-follow average; separate peak factors from averages.
• 10-16 minutes — Draw the upload path: Show API Gateway, Post Service, post metadata, pre-signed object storage, confirmation, and the media event.
• 16-22 minutes — Add media processing: Cover raw versus processed objects, worker retries, idempotent keys, status transitions, dead letters, and the ready-event ordering invariant.
• 22-30 minutes — Solve the feed: Start with profile reads, then compare fan-out on read and fan-out on write. Introduce hybrid fan-out, activity gating, Redis sorted sets, cursor pagination, and influencer merge.
• 30-35 minutes — Add storage and delivery: Explain the dual-access-pattern Cassandra schema, post hydration, signed CDN URLs, origin shield, cache policy, and deletion.
• 35-41 minutes — Cover reliability and operations: Discuss Kafka replay, dual-write reconciliation, follow-cache invalidation, hot-key/influencer skew, regional failure, metrics, and backpressure.
• 41-45 minutes — Close with trade-offs: Re-state eventual feed consistency, the tunable influencer threshold, object-storage/CDN assumptions, and the features kept below the line.

Core Entities

• Post: The core content entity. Carries a post_id, user_id, caption, media_keys (the S3 object keys for each processed resolution), media_status, and created_at. The media_keys are populated asynchronously after processing completes.
• User: An account with a profile, follower_count, and following_count. The follower_count field drives the influencer threshold check in the fan-out strategy.
• Follow: A directed edge from follower to followee. The follow graph is the input to every home feed generation and fan-out operation in the system.
• Feed (derived): A pre-computed ordered list of post IDs cached per user. Not a stored entity.

The full schema, index strategy, and partition keys are deferred to the deep dives. The four entities above are sufficient to drive the API design and High-Level Design.

API Design

Use a two-phase upload rather than a multipart form POST to the app server so binary image bytes stay off the application fleet at the illustrative 3,500 uploads-per-second peak. Proxying image bytes through the application tier couples bandwidth and connection usage to request handling and is a common scaling mistake.

Upload a photo:

POST /posts
Body: { caption, media_type }
Response: { post_id, upload_url }

Acknowledge upload complete:

PUT /posts/{post_id}/media
Body: { upload_confirmed: true }
Response: 202 Accepted

Get home feed:

GET /feed/home
Query: { cursor?, limit? }
Response: { posts: [...], next_cursor }

Get profile posts:

GET /users/{user_id}/posts
Query: { cursor?, limit? }
Response: { posts: [...], next_cursor }

Follow a user:

POST /users/{user_id}/follows
Response: 201 Created

Unfollow a user:

DELETE /users/{user_id}/follows
Response: 204 No Content

Two-phase upload: Photo uploads use a two-phase pattern. The first POST /posts generates a pre-signed S3 URL and a post_id without touching media storage. The client uploads directly to S3 using the signed URL. The second PUT /posts/{id}/media signals the server that the upload is complete, triggering the async processing pipeline. This keeps large binary transfers off the application servers entirely.

Cursor pagination: All feed endpoints use cursor-based pagination rather than offset. A user's feed changes while they scroll as new posts arrive. Offset pagination skips or repeats posts when items are inserted at the top. A cursor encodes the last-seen post_id, and every subsequent page begins strictly after that ID.

High-Level Design

1. Users can upload a photo

The write path: client requests a pre-signed URL, uploads image bytes directly to S3, then confirms the upload. The Post Service never touches the image bytes.

Components:

• Client: Mobile or web app initiating the two-phase upload flow.
• Post Service: Validates the request, generates a post_id, issues a pre-signed S3 URL, inserts the post row with media_status = pending, and publishes a MediaUploadedEvent on confirmation.
• Object Storage (S3): Receives the raw binary upload directly from the client.
• Post DB: Stores the post metadata row. Media keys are populated asynchronously after processing.

Request walkthrough:

1. Client sends POST /posts with caption and media type.
2. Post Service generates a post_id and inserts { post_id, user_id, caption, media_status: "pending", created_at } into Post DB.
3. Post Service generates a pre-signed S3 URL valid for 5 minutes and returns { post_id, upload_url }.
4. Client uploads image bytes directly to S3 using the pre-signed URL.
5. Client sends PUT /posts/{post_id}/media to confirm the upload is complete.
6. Post Service publishes MediaUploadedEvent { post_id, s3_raw_key } to Kafka.
7. Post Service returns 202 Accepted.

The media processing pipeline that resizes and optimizes the uploaded image is deferred to deep dive 1. Only the upload and acknowledgment path is shown here.

2. Users can view a profile page

Treat the profile page as the simpler read case before tackling the home-feed merging problem. A database index on (user_id, post_id) is the key structure for the profile access pattern; the feed requires additional derived state.

Components:

• Post Service (updated): Serves profile page reads. Queries the Post DB using the user_id plus a cursor.
• Post DB (updated): Index on (user_id, post_id) enables efficient per-user queries. Since post_id encodes creation time (Snowflake; covered in deep dive 4), this index gives chronological order without a separate timestamp index.

Request walkthrough:

1. Client sends GET /users/{user_id}/posts?limit=12.
2. Post Service queries: SELECT * FROM posts WHERE user_id = ? AND post_id < cursor ORDER BY post_id DESC LIMIT 12.
3. Post Service returns the post list with a cursor encoding the last post_id.