Twitter / X
Walk through a complete Twitter design, from a bare-bones tweet service to a hybrid fan-out architecture serving home timelines to 200M DAU in under 300ms.
What is Twitter / X?
Twitter is a social network where users post short messages and see a personalized feed from the accounts they follow. The interesting engineering challenge is not storing tweets; it is the fan-out problem. When a celebrity with 50 million followers posts a tweet, the system must update 50 million timelines nearly instantly while serving hundreds of millions of users refreshing their feeds. No single fan-out strategy works across the full follower distribution, making Twitter a rich test of trade-off thinking between write amplification and read amplification. A useful framing is: "the hard problem is not tweets, it is fan-out."
TL;DR
Write each tweet to a canonical store and publish a NewTweetEvent to Kafka. Use a hybrid timeline strategy: fan out normal authors’ tweets into Redis home-timeline sorted sets, but merge celebrity tweets at read time so one post cannot trigger tens of millions of writes. Store the follow graph in both directions, hydrate tweet IDs from a Redis content cache with a replica fallback, and use cursor pagination with time-sortable IDs.
The central trade-off is write amplification versus read amplification. Kafka, idempotent fan-out workers, bounded timeline caches, and an explicit celebrity threshold keep the two paths independently scalable while allowing timelines to be eventually consistent.
Scope and assumptions
These are illustrative planning assumptions for the design; the threshold and capacity numbers should be tuned from measured traffic:
- About 500 million registered users, 200 million daily active users, roughly five tweets per active user per day, 35,000 tweet writes/second at peak, and 140,000 home-timeline reads/second at peak.
- Tweets contain up to 280 characters. Home timelines are reverse-chronological, use cursor pagination, and may be eventually consistent within the stated availability/latency targets.
- Normal authors use fan-out on write; authors above an illustrative follower threshold use fan-out on read. Timeline caches are bounded, and inactive users may be materialized lazily when they return.
- The core system owns tweets, follows, profile timelines, home timelines, and content hydration. Likes, retweets, media, search, notifications, moderation workflows, and recommendation ranking are outside the primary design.
- Authentication is assumed at the gateway. Protected-account, block, mute, and policy filters must be enforced if those product features are added; they are not hidden consistency guarantees of the base timeline.
Functional Requirements
Core Requirements
- Users can post a tweet (up to 280 characters of text).
- Users can follow and unfollow other users.
- Users can view their home timeline: reverse-chronological tweets from users they follow.
- Users can view a profile timeline: all tweets posted by a specific user.
Below the Line (out of scope)
- Like, retweet, and quote tweet interactions
- Full-text search for tweets and users
- Media attachments (images and videos)
- Notifications and push alerts
The hardest part in scope: Generating the home timeline for 200M daily active users, where the fan-out ratio ranges from 1 (a new account followed by nobody) to 100M+ (a celebrity). No single strategy satisfies both ends of this distribution.
Likes and retweets are below the line because they do not change the core write or timeline delivery paths. To add them, store a tweet_likes table keyed by (tweet_id, user_id) and cache a like counter per tweet in Redis. Retweets can create a new tweet row with a retweet_of reference and follow the same fan-out path as an original tweet.
Search is below the line because it requires a separate indexing pipeline that does not interact with the timeline design. To add it, emit every new tweet to a Kafka topic and consume it into an Elasticsearch index. Full-text tweet search does not fit the key-value access patterns of the timeline service.
Media is below the line because it converts the write path into a two-phase upload without changing the fan-out logic. To add it, the client uploads directly to S3 via a pre-signed URL and includes the returned object key in the POST body. The tweet row stores the key; a CDN serves the bytes.
Notifications are below the line because they form a separate outbound delivery system that reads from tweet events but does not affect the read path. To add them, consume tweet creation events from Kafka and dispatch push notifications via APNs and FCM per follower.
Non-Functional Requirements
Core Requirements
- Availability: 99.99% uptime. Availability over consistency for home timelines: a tweet visible to some followers before others is acceptable; a failed timeline load is not.
- Latency: Home timeline loads under 300ms p99. Profile timeline loads under 200ms p99. Tweet creation completes under 500ms.
- Scale: 500M registered users, 200M DAU. Each active user posts ~5 tweets per day on average.
- Write throughput: ~11,600 tweet writes per second on average (200M × 5 / 86,400), peaking at ~35K per second during events.
- Read throughput: ~46,000 home timeline reads per second on average (200M DAU × 20 refreshes/day / 86,400), peaking at ~140K per second.
Below the Line
- Sub-50ms timeline latency via CDN edge caching
- Real-time guarantee on notification delivery
Fan-out ratio: For every tweet posted by a user with 1,000 followers, 1,000 timeline cache entries need to be updated. With an average of ~200 follows per active user, the effective write amplification on the timeline cache peaks at approximately 11,600 × 200 = 2.3M cache writes per second. This number, not the raw tweet write rate, drives the infrastructure decisions in this article.
Write this number down early. The 2.3M cache writes per second, rather than the raw tweet rate, shapes the storage and caching decisions.
The 300ms latency target for home timelines rules out assembling the feed on the read path by querying the database across all followed accounts in real time. Pre-computation is required. The 99.99% availability target means a single Redis node is not acceptable for the timeline cache, and the primary tweet database cannot be in the read path for every timeline load.
30-second answer / outline
- Authenticate and validate the tweet, generate a time-sortable ID, write the canonical row, and publish a durable
NewTweetEvent. - Maintain both directions of the follow graph so the system can find an author’s followers for fan-out and a viewer’s followees for celebrity reads.
- Fan out normal-author tweet IDs asynchronously into per-user Redis sorted sets; skip mass fan-out for celebrity authors and merge their recent tweets at read time.
- Serve home timelines with
ZREVRANGE, batch-hydrate tweet content from Redis, and fall back to a read replica. Serve profile timelines from an indexed user/tweet access path. - Make fan-out idempotent, replayable, and bounded; tolerate eventual consistency while protecting the canonical tweet store, follow graph, and authorization checks.
5-minute explanation
Start with the fan-out arithmetic. Building a home timeline by querying every followed account on demand creates hundreds of reads per request and multiplies at peak timeline traffic. Fan-out on write makes reads cheap, but a celebrity with tens of millions of followers turns one tweet into an unbounded burst of cache writes. The correct design is therefore hybrid rather than ideological.
The Tweet Service validates text, generates a globally unique time-sortable ID, writes the canonical tweet, and emits an event. Fan-out workers consume Kafka and add the tweet ID to each normal follower’s Redis sorted set, trimming the list to a bounded history. A follower threshold is an operational tuning parameter: below it, pay the write cost once; above it, defer the cost and merge the author’s recent tweets when a viewer loads the feed.
The Follow Service maintains forward and reverse adjacency lists. The Timeline Service reads a precomputed home list, merges any celebrity candidates, orders by tweet ID, applies pagination, and hydrates content with a batched cache lookup. Profile timelines are simpler because all tweets for one author can be read from a user-partitioned index. Redis absorbs the hot read load; the tweet primary is for canonical writes and replicas are fallbacks for content misses.
Reliability comes from Kafka replay, idempotent ZADD, cache rebuilds, and a clear freshness contract. A tweet can appear in one follower’s home timeline before another’s, but the canonical row must be durable and deleted/protected content must be filtered consistently. The hardest deep dives are celebrity handling, follow-graph access, tweet ID generation, hydration, and sharding the tweet store.
45-minute interview approach
Spend the most time on the fan-out decision and the read-path merge; defer secondary social features unless the interviewer asks for them.
- 0–5 minutes — Clarify the contract: Confirm tweet size, timeline ordering, follow semantics, celebrity behavior, consistency, pagination, protected users, deletion, media/search scope, and latency/availability targets.
- 5–10 minutes — Establish scale: Calculate tweet writes, timeline reads, average follow count, fan-out amplification, timeline cache size, celebrity burst size, and content hydration load.
- 10–15 minutes — Define entities and APIs: Walk through
User,Tweet,Follow,Timeline, Snowflake-style IDs, cursor semantics, and post/follow/timeline endpoints. - 15–22 minutes — Draw canonical writes: Show the gateway, Tweet Service, ID generator, Tweet DB, Kafka event, transactional/outbox consideration, and the profile-timeline access path.
- 22–31 minutes — Deep dive on home timelines: Compare fan-out on write and read, choose the hybrid threshold, show follow-graph directions, worker queues, Redis sorted sets, trimming, and celebrity merging.
- 31–36 minutes — Add hydration and pagination: Show Redis tweet caching, replica fallback, cursor construction, cache misses, ordering, and stale/deleted content behavior.
- 36–41 minutes — Reliability, security, and operations: Cover duplicate events, Kafka lag, Redis loss, graph repair, replica lag, authorization, rate limits, privacy, and hot-user monitoring.
- 41–45 minutes — Trade-offs and close: Compare data stores, ID schemes, cache structures, threshold choices, and multi-region options; recap write amplification versus read amplification and invite follow-ups.
Core Entities
- Tweet: A 280-character message. Carries a
tweet_id,user_id,text, andcreated_at. The schema also supports a nullableretweet_ofreference for the retweet feature we've deferred. - User: An account with a profile and follower and following counts. The
follower_countfield drives the celebrity threshold check in the fan-out deep dive. - Follow: A directed relationship from a follower to a followee. The follow graph is the input to every home timeline generation and fan-out operation in the system.
- Timeline (derived): A pre-computed ordered list of tweet IDs cached per user, not a stored entity. It is the most performance-critical data structure in the design.
The full schema, indexes, and partition keys are deferred to the data model deep dive. The four entities above are sufficient to drive the API design and the High-Level Design.
API Design
Post a tweet:
POST /tweets
Body: { text }
Response: { tweet_id, created_at }
Get home timeline:
GET /timelines/home
Query: { cursor?, limit? }
Response: { tweets: [...], next_cursor }
Get profile timeline:
GET /users/{user_id}/tweets
Query: { cursor?, limit? }
Response: { tweets: [...], next_cursor }
Follow a user:
POST /users/{user_id}/follows
Response: 201 Created
Unfollow a user:
DELETE /users/{user_id}/follows/{followee_id}
Response: 204 No Content
Cursor pagination: All timeline endpoints use cursor-based pagination, not offset-based. Offset pagination breaks when new tweets arrive between page loads: inserting one tweet at position 0 shifts every offset by 1, causing items to be skipped or duplicated across pages. A cursor encodes the last-seen tweet_id, and the next page begins strictly after that ID.
Authentication is not shown in the endpoint bodies but it is assumed to be present. In practice, an API gateway validates a session token and injects the viewer_id into every downstream request. The follow and post endpoints require authentication; the profile timeline endpoint is public.
High-Level Design
The critical flows are: persist and publish a tweet, update or defer follower timelines based on author fan-out, read/merge a home timeline, and hydrate the resulting IDs without putting the primary database on the hot read path.
1. Users can post a tweet
The write path: client submits a tweet, the Tweet Service validates it, generates a tweet_id, and writes it to the database.
Components:
- Client: Web or mobile interface sending
POST /tweets. - Tweet Service: Validates that text is 280 characters or fewer, generates a tweet_id (black box for now, covered in the deep dives), and inserts the row.
- Tweet DB: Stores the canonical tweet record. Indexed on
user_idfor profile timeline queries.
Request walkthrough:
- Client sends
POST /tweetswith the text body. - Tweet Service validates the length constraint.
- Tweet Service generates a tweet_id.
- Tweet Service inserts
{ tweet_id, user_id, text, created_at }into the Tweet DB. - Tweet Service returns
{ tweet_id, created_at }to the client.
The write path only. Fan-out to follower timelines is deferred to requirement 3, once the follow graph exists.
2. Users can view a profile timeline
The profile timeline is ordered tweets from a single user. A database index on (user_id, tweet_id) is sufficient for this access pattern, so it is the simple read case before the harder home timeline in requirement 4.
Components:
- Timeline Service: Handles all read requests. Queries the Tweet DB for profile timelines.
- Tweet DB (updated): The index on
(user_id, tweet_id)makes profile timeline queries fast. Because tweet_id encodes the timestamp (covered in deep dive 2), this index also sorts by time.
Request walkthrough:
- Client sends
GET /users/{user_id}/tweets?limit=20. - Timeline Service queries the Tweet DB:
SELECT * FROM tweets WHERE user_id = ? ORDER BY tweet_id DESC LIMIT 20. - Timeline Service returns the tweet list with a cursor pointing to the last tweet_id.
Profile timeline is a single-account read. Home timeline is more complex because it requires aggregating tweets across many accounts.
3. Users can follow and unfollow other users
The follow graph drives every home timeline. It answers two questions: "who do I follow?" (for reading my home timeline) and "who follows me?" (for fan-out when I post). Both access patterns need to be fast.
Components:
- Follow Service: Handles
POSTandDELETEon follow relationships. Updates both the forward and reverse indices. - Follow Store: Stores the follow graph as two adjacency lists:
follower_id → [followee_ids]andfollowee_id → [follower_ids]. Both directions are required.
Request walkthrough:
- Client sends
POST /users/{followee_id}/follows. - Follow Service writes
(follower_id, followee_id)to the Follow Store in both the forward and reverse direction. - Follow Service returns 201 Created.
Maintaining both adjacency directions in the Follow Store doubles write cost on follow and unfollow but makes every read O(1) per user. The alternative, computing one direction from the other on the fly, is a full table scan. At 100B follow edges in the graph, that is not viable.
4. Users can view a home timeline
This is the hard requirement. A user's home timeline is the merged, reverse-chronological feed of tweets from every account they follow. Assembling this at read time for a user following 500 people against a live database would mean 500 queries per request. At 140K timeline requests per second, that is 70M database queries per second. A pre-computed feed is required; the naive cost should be made explicit before introducing the solution.
Components:
- Fan-out Worker: An async worker that consumes new tweet events and pushes tweet_ids into each follower's timeline cache.
- Kafka: A durable message queue decoupling tweet writes from fan-out. The Tweet Service publishes a
NewTweetEventon every write. The Fan-out Worker consumes it. - Redis Timeline Cache (new): Stores per-user sorted sets. Key:
home_timeline:{user_id}. Score: tweet creation timestamp. Value: tweet_id. Capped at 800 entries per user. - Timeline Service (updated): On a home timeline read, fetches tweet_ids from Redis and hydrates them into full tweet objects via the Tweet DB.
Request walkthrough (write path):
- Client sends
POST /tweets. - Tweet Service inserts into Tweet DB and publishes
NewTweetEvent { tweet_id, author_id }to Kafka. - Fan-out Worker reads the event, fetches the author's follower list from Follow Store.
- Fan-out Worker calls
ZADD home_timeline:{follower_id} {timestamp} {tweet_id}for each follower. - Fan-out Worker trims each list to 800 entries.
Request walkthrough (read path):
- Client sends
GET /timelines/home. - Timeline Service calls
ZREVRANGE home_timeline:{user_id} 0 19on Redis. - Timeline Service batch-fetches full tweet objects for the returned tweet_ids from Tweet DB (or a tweet cache).
- Timeline Service returns the assembled tweet list.
This is the High-Level Design: tweets write through Kafka to pre-computed Redis timelines; home timeline reads serve entirely from Redis. The fan-out worker is the component that collapses under celebrity-scale writes, which we address in deep dive 1.
The fan-out worker is shown as a simple loop over all followers. A user with 50 million followers makes this loop catastrophically slow, which is why the fan-out deep dive introduces a hybrid strategy.
Potential Deep Dives
1. How do we generate home timelines at scale?
Three constraints define this problem:
- Home timeline reads must complete in under 300ms p99.
- A celebrity tweet must not stall the fan-out pipeline for all other users.
- The fan-out write rate must stay manageable: our average is 2.3M timeline cache writes per second across all accounts.
2. How do we generate unique, time-sortable tweet IDs?
Three constraints drive the design:
- Tweet IDs must be globally unique across all servers and regions with no central coordination.
- Tweet IDs should sort chronologically so that
ORDER BY tweet_id DESCgives the timeline order. - Generation must be fast enough not to add latency to the tweet write path.
3. How do we hydrate tweet content at read time?
Context: When the Timeline Service retrieves a home timeline from Redis, it gets a list of up to 20 tweet_ids. It must fetch the full tweet content (text, author display name, like count) for each. At 140K timeline reads per second with 20 tweet_ids each, the service needs to handle approximately 2.8 million tweet-content fetches per second. A direct primary database read for each is not viable.
4. How do we store and query the follow graph at scale?
Context: The follow graph is enormous. At 500M users with an average of 200 follows per active user, the graph has ~100 billion edges. The fan-out worker reads the reverse direction (followers of author X) on every tweet write. The Timeline Service reads the forward direction (celebrities user Y follows) on every home timeline load. Both reads must complete in milliseconds.
5. How do we model and scale the tweet table?
Context: Core Entities identified four fields for a tweet (tweet_id, user_id, text, created_at). The two dominant access patterns are very different: profile timeline reads filter by user_id and sort by tweet_id; tweet content hydration looks up by tweet_id directly. At 1B tweets per day, the table grows by roughly 300GB per day of raw text data alone. After one year, that is over 100TB. How you physically store and shard this table determines whether both access patterns stay fast at that scale.
Final Architecture
The read/write split into Tweet Service and Timeline Service lets each scale independently. Redis absorbs the vast majority of both timeline and tweet-content reads. Kafka decouples the tweet write path from the fan-out pipeline so a celebrity post cannot block other users' tweet delivery. Walk through the final diagram by tying each component back to a constraint from the requirements.
Reliability, security, and operations
Reliability. The Tweet DB is the canonical source for tweet content; use a transactional outbox or a durable publish/reconciliation path so a committed tweet cannot silently lose its Kafka event. Fan-out workers consume at least once and make ZADD plus timeline trimming replay-safe. Redis timelines and content caches are derived state that can be rebuilt from Kafka, the follow graph, and the tweet store. During fan-out lag, serve an older timeline or merge a bounded recent window; do not make a celebrity post block all authors. Replicate the follow graph, repair dual-write divergence, and keep read-replica lag visible because stale hydration can otherwise look like missing content.
Security. Authenticate every write and follow operation at the gateway, derive author_id and viewer_id from trusted credentials, and authorize reads for protected accounts. Validate text length, encoding, URLs, and payload size; escape rendered text to prevent injection; and rate-limit posting, following, and timeline abuse. Encrypt transport and storage, avoid logging private tweet bodies or session tokens, isolate user data by access policy, and audit moderation, deletion, and privileged timeline access. If block, mute, or protected-account features are added, apply those filters at read time and ensure fan-out does not leak IDs to unauthorized viewers.
Operations. Monitor tweet-write p99, outbox/publish failures, Kafka lag and fan-out queue age, followers-per-tweet, celebrity deferrals, timeline freshness, Redis hit rate/memory/latency, content hydration misses, replica lag, graph read/write errors, and per-user hot-spot skew. Test duplicate events, worker loss, Redis failover, Tweet DB failover, a 50-million-follower post, cache rebuild, follow-graph divergence, deletion propagation, and a protected-account read. Keep a backfill tool for reconstructing a user timeline and a kill switch for runaway fan-out.
Trade-offs and alternatives
- Fan-out on write versus fan-out on read: Write fan-out makes home reads predictable but amplifies writes by follower count. Read fan-out avoids celebrity bursts but makes latency and query work grow with followee count. The hybrid policy matches the long-tail distribution.
- Redis sorted sets versus lists or a database query: Sorted sets provide ordered range reads and idempotent member updates. Lists are cheaper when ordering is immutable but awkward for duplicate suppression and backfills; database assembly is simpler but cannot meet the home-timeline latency target at this scale.
- Snowflake-style IDs versus UUIDs: Time-sortable IDs support chronological ordering and compact cursors without a central allocator. UUIDs simplify independent generation and are harder to guess, but require a separate timestamp/order field and consume more index space.
- Cassandra versus relational storage for the follow graph: Cassandra wide rows fit adjacency-list access and scale horizontally. A relational graph is easier to transact and query flexibly, but large follower lists and high fan-out may require sharding and careful read replicas.
- Cache content versus hydrate from replicas: A tweet cache protects replicas and reduces p99, but introduces invalidation and memory cost. Replica reads are simpler and fresher within replication lag, but a viral tweet can create a cache-miss storm.
- Single-region versus multi-region writes: Single-region canonical writes simplify ordering and follow consistency. Multi-region serving improves latency and availability but needs conflict, ID, deletion, and fan-out routing policies.
Follow-up questions
- How is the celebrity threshold chosen? Measure follower count, post rate, fan-out worker capacity, cache write latency, and acceptable read merge cost; make the threshold dynamic and observable rather than a universal constant.
- What happens when a celebrity posts? Store the tweet canonically and skip mass fan-out; at read time fetch recent tweets from celebrity followees and merge them with the precomputed timeline.
- How do you handle a new follow? Persist both graph directions, then backfill a bounded recent history into the follower’s timeline asynchronously or merge it on the first read.
- How are blocks, mutes, protected accounts, and deletes enforced? Apply authorization and policy filters during timeline assembly, remove or tombstone derived IDs, and propagate invalidations; never rely only on a cached fan-out result.
- What if Redis is lost? Rebuild timelines from the follow graph and recent tweet ranges, or serve a degraded recent feed while background reconstruction catches up.
- How do you prevent duplicate fan-out? Use the tweet ID as the sorted-set member and make workers safe to retry; add an event ID/outbox key for the DB-to-Kafka boundary.
- How would media, search, or recommendations be added? Publish tweet events to separate consumers for object storage/CDN, a search index, or an offline/online ranking pipeline without putting those workloads in the timeline critical path.
Common mistakes
- Choosing pure fan-out on write or pure fan-out on read without calculating the celebrity and long-tail costs.
- Building a home timeline with one database query per followed account on every request.
- Sending a celebrity tweet to tens of millions of synchronous cache writes or letting it block normal fan-out work.
- Putting tweet content hydration on the primary database or omitting a batch
MGET/replica fallback. - Using offset pagination, non-sortable IDs, or an unbounded per-user timeline cache.
- Ignoring the DB-to-Kafka dual-write gap, at-least-once duplicates, cache rebuilds, replica lag, and follow-graph divergence.
- Treating authentication as sufficient authorization for protected content, or forgetting rate limiting, text safety, deletion, and policy filters.
Interview Cheat Sheet
- State the fan-out problem in your first breath: when someone with 50 million followers posts a tweet, the system must update 50 million timelines. Everything downstream is an answer to this one constraint.
- Fan-out on read is too slow at scale: a user following 500 accounts triggers 500 DB queries per timeline load, and latency grows linearly with follow count.
- Pure fan-out on write breaks for celebrities: a single tweet creates 50 million Redis writes and stalls the fan-out pipeline for all other users queued behind it.
- The hybrid strategy splits at an illustrative follower threshold: write fan-out for normal users, live read fan-out for celebrity tweets at timeline load time. Tune the threshold from measured capacity and latency.
- Store pre-computed home timelines as Redis sorted sets: key is
home_timeline:{user_id}, score is creation timestamp, value is tweet_id. Cap each list at 800 entries. - At 200M DAU storing 800 tweet_ids per timeline at 8 bytes, the full timeline cache totals roughly 1.2TB. Plan for Redis Cluster from the start.
- Use Snowflake IDs for tweets: 64-bit integers encoding 41 bits of timestamp, 10 bits of machine ID, 12 bits of sequence counter per millisecond.
- Snowflake IDs are time-sortable, so
ORDER BY tweet_id DESCreplacesORDER BY created_at DESC. No secondary timestamp index is needed for chronological timeline queries. - Use cursor-based pagination for all timeline endpoints. Offset pagination breaks when new tweets arrive between page loads.
- Cache tweet content (full tweet objects) in Redis keyed by tweet_id with a 24-hour TTL. The Timeline Service hits the Redis tweet cache with a batch MGET before falling back to a read replica, never the primary.
- The primary tweet database handles writes only. Read replicas absorb all tweet content hydration on cache miss. Keep the primary out of the read path entirely.
- Cassandra is a natural fit for the follow graph: partition by followee_id maps directly to an adjacency list lookup, and wide-row reads return an entire follower list in one operation.
- Maintain both directions of the follow graph (follows_by_follower and follows_by_followee) in Cassandra. The fan-out worker uses the reverse index; the timeline service uses the forward index for celebrity lookups.
- Skip fan-out for users inactive for 30+ days. Check a Redis key set on login with a 30-day TTL. Reconstruct their timeline from the follow graph and tweet DB on their next login.
- Fan-out workers must be idempotent: a duplicate
ZADDwith an already-present member is a no-op in a Redis sorted set. Kafka at-least-once delivery is safe.
Test Your Understanding
-
Why is pure fan-out on write unsafe for celebrities?
One post can require millions of cache writes and monopolize the fan-out workers, delaying everyone else.
-
Why is pure fan-out on read unsafe for home timelines?
A viewer following hundreds of accounts would trigger hundreds of reads and a large merge on every page load.
-
What does the hybrid strategy do?
It precomputes normal authors into follower timelines and merges high-follower authors’ recent tweets at read time.
-
Why are Snowflake-style IDs useful here?
They are compact, globally unique, and time-sortable, which simplifies ordering and cursor pagination.
-
What is the source of truth for a tweet?
The canonical Tweet DB row; Redis timelines and caches are derived data that can be replayed or rebuilt.
Recap
Twitter’s core scale problem is fan-out. Keep tweets canonical and events durable, maintain both follow-graph directions, use write fan-out for normal authors and read-time merging for celebrities, then serve bounded Redis timelines with batched content hydration and replica fallback. Idempotency, authorization, deletion, lag, cache rebuilds, and operational controls make the hybrid design safe to run.
Related concepts
Fan-out on write/read; social-graph adjacency lists; Redis sorted sets; Kafka event delivery and outbox patterns; Snowflake IDs; cursor pagination; cache-aside and stampede control; Cassandra wide rows; read replicas; hot-key isolation; eventual consistency; content authorization.