News Feed

What is a social media news feed?

A news feed is the personalized, scrollable homepage showing posts from people a user follows. The real challenge is not storing posts; it is delivering a feed in under 200ms when a single celebrity post must reach 10 million followers. This design uses caching, message queues, sharding, and a hybrid fan-out strategy to balance write amplification against read latency.

TL;DR

Use hybrid fan-out. The Post Service stores the post, writes it through to a post cache, and publishes a post.created event. Fan-out Workers push the post ID and timestamp into Redis sorted-set feeds for regular authors, while celebrity posts go only to the author's timeline and are pulled when a follower reads.

The Feed Service merges the pre-built feed with celebrity timelines, removes duplicates, hydrates post IDs from the Post Cache or Post DB, and returns a cursor-paginated response. Kafka is at-least-once, so writes and consumers must be idempotent. Eventual consistency is intentional: a delay of a few seconds is acceptable, but cache loss must be recoverable by rebuilding from the social graph and Post DB.

Scope and Assumptions

This design assumes:

• The core product is a home feed of posts from followed accounts; post creation and paginated feed reads are the primary APIs.
• Ranking is represented by a stored score or a simple recency-based order. Candidate retrieval and ranking-model internals, ads, Stories, and other content surfaces are extension points rather than part of the core design.
• The illustrative workload is 500M daily active users, 10M posts per day, 200–500 followees per user, and up to 10M followers for a celebrity account.
• Feed freshness is eventual: regular posts should usually appear within 2–5 seconds, and a brief stale response is preferable to a failed read during cache or dependency degradation.
• Media bytes are uploaded and served by a separate media system; this API stores media references and focuses on feed metadata and delivery.

Functional Requirements

Core Requirements

1. Users see a personalized, ranked feed of posts from friends and accounts they follow.
2. New posts appear in followers' feeds within seconds of publishing.
3. The feed supports infinite scroll (paginated reads).

Below the Line (out of scope)

• Feed recommendation ML model internals
• Ad insertion logic
• Stories and ephemeral content (separate architecture)

If ranking ML model internals were in scope, introduce a candidate retrieval and ranking service layer sitting between feed storage and the client: retrieve the top 500 candidate post_ids from the raw chronological feed and pass them to the ranking model, which applies signals like engagement rate, relationship strength, and recency to return the final ordered set. The ML model itself is a separate offline training pipeline that publishes scoring parameters to a feature store the ranking service reads at request time.

Ad insertion is out of scope because it requires its own auction pipeline and targeting model. The integration point is a slot injection layer that periodically splices ad slots between organic posts in the ranked feed before serializing the response, a background concern that sits downstream of everything in this design.

Stories live on a separate fan-out and storage architecture because they have short TTLs (24 hours), a different content format, and different engagement mechanics. They would not share the feed cache or fan-out worker design covered here.

The hardest part in scope: Deciding how to fan-out a post from a user with 10 million followers. Fan-out-on-write (write to each follower's feed at post time) creates 10 million writes per post. Fan-out-on-read (compute each feed fresh at read time) creates N database lookups per page load. Neither works at scale in isolation; the entire system design pivots on getting this trade-off right.

Non-Functional Requirements

Core Requirements

• Latency: Feed load under 200ms p99 end to end.
• Scale: 500M DAU; each user follows 200 to 500 accounts on average; celebrity accounts have up to 10M followers.
• Writes: 10M new posts per day (approximately 115 posts per second on average; 5x at peak).
• Availability: 99.99% uptime (about 52 minutes of downtime per year).
• Consistency: Eventual. Seeing a post 2 to 5 seconds late is acceptable; feed staleness for minutes is not.

Below the Line

• Exactly-once delivery guarantees for feed updates (at-least-once with idempotent writes is sufficient)
• Cross-device feed position synchronization (cursor managed per device)

Read/write amplification: 10M new posts per day sounds modest, but fan-out is the multiplier that changes the math entirely. A user with 500 followers creates 500 feed-write operations per post. A celebrity with 10M followers creates 10M operations per post.

At peak post rates, fan-out writes can hit 500M feed updates per second across the system, a 4,000x amplification over the raw post write rate. Every design decision in this article traces back to controlling that amplification.

The read-to-write ratio shapes which trade-off to optimize for: feeds are read roughly 100 times for every post written, so read latency is the primary cost and write throughput is the secondary cost. The system can spend more write latency through asynchronous fan-out to buy lower read latency with a pre-materialized Redis feed.

30-Second Answer

1. Store posts in the Post DB and seed a separate post-content cache on write.
2. Publish post.created to Kafka and return without waiting for follower updates.
3. For regular authors, workers fetch followers and ZADD the post ID into each follower's Redis feed. For celebrity authors, write a timeline once and skip the million-follower fan-out.
4. On a feed read, page the caller's sorted set, pull recent entries from followed celebrity timelines, merge and deduplicate them, then batch-hydrate post content.
5. Use cursor pagination, at-least-once events with idempotent writes, cache rebuilds, and backpressure. The key trade-off is bounded write amplification without making every read scan all followees.

5-Minute Explanation

Start with the source of truth: the Post DB stores post metadata and content references, while the social graph stores follow edges. A feed entry is derived state containing only a post_id and ordering score; storing full post objects in every user's feed would multiply memory use and make edits difficult to propagate.

The write path is asynchronous. After the Post Service commits the post, it publishes post.created. Workers partition and consume those events, read the author's followers, and push the post ID into regular followers' Redis sorted sets. A celebrity with 10M followers bypasses this step: the worker updates the author's timeline once, and the Feed Service pulls that timeline for followers at read time.

The read path reads a cursor page from the caller's feed, pulls a bounded recent window from each followed celebrity, merges by score, deduplicates, and hydrates content with a pipelined Post Cache lookup. Cache misses fall back to the Post DB and can be repopulated. This keeps the common read path fast while accepting a few seconds of eventual-consistency lag.

The hard parts are not the CRUD endpoints. They are controlling celebrity write amplification, making replay safe, rebuilding derived feeds after cache loss, and preventing a slow Redis, Post DB, or Kafka dependency from turning into a user-visible outage.

45-Minute Interview Approach

Use this agenda to answer the design question and keep the discussion focused on the highest-leverage decisions:

• 0–5 minutes — Clarify the product: Confirm whether the feed is chronological or ranked, freshness expectations, pagination semantics, post edits/deletes, privacy, and whether media or ranking internals are in scope.
• 5–10 minutes — Establish scale: Calculate posts per second, average followee/follower counts, celebrity fan-out, read/write skew, the 200ms p99 target, and the acceptable consistency model.
• 10–15 minutes — Define entities and APIs: Introduce User, Follow, Post, FeedEntry, POST /v1/posts, and cursor-based GET /v1/feed.
• 15–22 minutes — Draw the baseline: Show synchronous fan-out and quantify why a 10M-follower post is the bottleneck. Keep the baseline short; its purpose is to motivate the hybrid design.
• 22–30 minutes — Explain the write path: Add Kafka, idempotent fan-out workers, Redis sorted sets, trimming, celebrity detection, and the distinction between a user feed and an author timeline.
• 30–36 minutes — Explain the read path: Page the pre-built feed, pull celebrity timelines in parallel, merge and deduplicate, then hydrate post IDs from the Post Cache with a database fallback.
• 36–41 minutes — Choose deep dives: Prioritize celebrity handling, returning users with stale or missing feeds, cursor correctness, cache rebuilds, and queue backpressure. Ranking ML and ads can wait unless the prompt makes them core.
• 41–45 minutes — Close operationally: Cover failure modes, authorization/privacy, deletion propagation, metrics, trade-offs, and the recap. State the key invariant: the feed cache is derived state, not the source of truth.

Core Entities

• User: An account on the platform. Has a list of accounts they follow stored in the social graph.
• Post: Content published by a user (text, image references, video references). The atomic unit of the news feed.
• Follow: A directed edge in the social graph from follower to followee. Used to determine whose posts appear in a user's feed.
• FeedEntry: A materialized mapping of (user_id, post_id, score) representing a pre-computed feed item stored in the user's feed cache.

Full schema and indexing strategy are deferred to the deep dives. These four entities are enough to drive the API and High-Level Design.

API Design

FR 1 and FR 2: Create a post and publish it to followers' feeds:

# Create a new post; triggers async fan-out to followers
POST /v1/posts
Body: { content: "...", media_urls: [], created_at: "2026-03-29T12:00:00Z" }
Response: { post_id: "p_abc123", created_at: "2026-03-29T12:00:00Z" }

POST because this is a state-creating operation. Media uploads are handled separately via a pre-signed URL flow; this endpoint accepts media references, not raw bytes. The fan-out happens asynchronously after the response is returned, so the caller does not wait for all followers' feeds to be updated.

FR 3: Read the paginated news feed:

# Fetch next page of the caller's feed
GET /v1/feed?cursor=eyJ0c...&limit=20
Response: {
  posts: [
    { post_id: "p_abc", author_id: "u_xyz", content: "...", created_at: "...", like_count: 312 },
    ...
  ],
  next_cursor: "eyJ0c...",
  has_more: true
}

Use cursor-based pagination rather than offset-based. Offset pagination on a mutable, ranked feed skips or repeats posts when new content is inserted ahead of the current offset. The cursor encodes a timestamp and post_id so the feed resumes deterministically after new posts arrive. Limit defaults to 20 and caps at 50.

High-Level Design

Critical flows

The critical flows are post creation and asynchronous fan-out, feed reads with celebrity merging, and cache-miss or returning-user recovery. The numbered designs below build from a simple baseline to the hybrid architecture.

1. Basic post creation and feed write

The naive write path: a user posts, the Post Service saves the content, then synchronously writes the post_id into every follower's feed table before returning.

This is simple to reason about and works for small accounts. It fails for celebrity accounts, but quantifying that failure motivates the design that follows.

Components:

• Client: Web or mobile app sending POST /v1/posts.
• Post Service: Validates and stores the post in the Post DB. Queries the social graph for the poster's follower list, then writes one row per follower into the Feed DB.
• Post DB: Stores full post content (PostgreSQL). The source of truth for all post data.
• Social Graph DB: Stores follower relationships as a directed adjacency list. Read-heavy: queried on every post to enumerate followers.
• Feed DB: A per-user feed table storing (user_id, post_id, timestamp) rows. Feed reads query this table.

Request walkthrough:

1. Client sends POST /v1/posts with post content.
2. Post Service validates the request and inserts the post into Post DB.
3. Post Service queries Social Graph DB: give me all followers for user X.
4. Post Service loops through the follower list and writes (follower_id, post_id, timestamp) into Feed DB for each follower.
5. Post Service returns the new post_id to the client.