Trending Topics

What is a trending articles system?

A trending articles system tracks the K most-shared articles over a configurable sliding time window, such as the last hour, 24 hours, or 7 days. The hidden engineering challenge is time-bounded counting: share events that fall outside the window must stop contributing to an article's score, which means counts age out continuously rather than only accumulating.

This is harder than a generic Top-K because the answer changes every second as old events drop off. It is a useful interview question because the tempting "count stuff and sort" approach fails once the sliding window must continuously exclude old events. The design gets pushed toward bucket-level granularity, approximate counting, and precomputation.

TL;DR

Accept share events into Kafka, aggregate them into sharded one-minute Redis buckets, and run a background job every 30 seconds to union the buckets for each window and category. Store the resulting Top-K lists as precomputed Redis keys and optionally cache them at the CDN, so reads never perform a sliding-window aggregation.

The design handles viral articles with hot-key sharding, uses a Count-Min Sketch as a bounded candidate gate when the article population is large, and exposes computed_at so the deliberate freshness bound is visible. At-least-once delivery, replay, bucket TTLs, and stale-result fallback are part of the operational contract.

Scope and assumptions

These are illustrative planning assumptions for the design; tune them with measured traffic and workload shape:

• Up to 100,000 share events per second at peak, roughly 170,000 trending reads per second, and 500 million daily active users. A viral article may account for a large fraction of writes.
• Supported windows are the last hour, 24 hours, and seven days, represented by one-minute buckets; results may be up to 30 seconds stale and K is at most 100.
• Results can be global or filtered by a bounded category set. At-least-once event delivery is acceptable when duplicate-share semantics and any deduplication policy are explicit.
• Kafka is the durable replay boundary, Redis holds bucket and snapshot state, and a separate article service owns titles and canonical URLs. The trending service does not own article content.
• The primary design covers share ingestion, time-bounded counting, category filtering, ranked reads, and operations. Content serving, personalized ranking, notifications, and real-time push are outside the primary scope.

Functional Requirements

Core Requirements

1. Users can share an article, generating a share event that increments the article's trending score.
2. The system exposes the top-K trending articles for configurable sliding windows (last 1 hour, 24 hours, 7 days). K is configurable, defaulting to 10, with a maximum of 100.
3. Trending results can be filtered by category (tech, sports, politics).

Below the Line (out of scope)

• Article content storage and serving
• User notifications when a shared article enters the trending list
• Trending people, hashtags, or topics (articles only)
• Real-time push of trending updates to all connected clients

The hardest part in scope: Maintaining a correct leaderboard across a sliding time window at 100,000 share events per second. Counts must age out continuously as the window moves, which rules out naive counters and requires a bucket-based aggregation strategy combined with precomputed results.

Article content storage is below the line because it solves a completely different problem. To add it, store article metadata in a relational database with a full-text search index, separate from the counter pipeline here.

User notifications are below the line because they introduce a fan-out write pattern orthogonal to the counter design. To add them, publish a Kafka event when an article first enters the top-K and consume it in a dedicated notification service that fans out to interested users asynchronously.

Real-time push of trending updates is below the line because it adds WebSocket fan-out complexity without changing the storage design. To add it, use Server-Sent Events: when the precomputed top-K changes, the Read Service publishes a diff event that SSE connections consume, keeping the push path entirely separate from the read path.

Non-Functional Requirements

Core Requirements

• Write throughput: 100,000 share events per second at peak, during viral events when a single article dominates a window. This rules out any design that routes all writes through a single storage key.
• Read throughput: 500M DAU with 30 page loads per day each, producing roughly 170,000 trending reads per second. This demands precomputed results, not live aggregation.
• Read latency: Trending list returns in under 50ms p99. A real-time ZUNIONSTORE across 60 Redis sorted sets on every request is too slow at this read rate.
• Write latency: Share event acknowledged in under 100ms. The ingestion path must stay thin and delegate aggregation work to an async pipeline.
• Freshness: Top-K results are at most 30 seconds stale. This is the key tolerance that enables precomputation and CDN caching.
• Availability: 99.99% uptime. Availability over consistency: a slightly stale trending list is always preferable to an error response.

Below the Line

• Sub-5ms global read latency via CDN edge (achievable but not a core NFR in this design)
• Exactly-once share counting (at-least-once with deduplication is sufficient)

Read/write ratio: At steady state, reads outpace writes roughly 50:1. During viral spikes, write rate briefly spikes 10x. The system must handle both extremes independently: the write path must absorb bursts without affecting read latency, and the read path at 170,000 requests/second requires precomputed answers rather than live computation.

The 30-second freshness tolerance is the most consequential NFR in this design. It gives us permission to compute the top-K in a background job and cache the result, rather than aggregating on every read. Without this tolerance, the architecture would require a much more complex real-time aggregation pipeline.

The 100,000 writes per second for a single viral article is the worst-case write multiplier. Any design that routes all writes for one article to a single Redis key will saturate a Redis node, since Redis serializes all operations on a single key on one thread.

30-second answer / outline

1. Validate and publish share events to Kafka, acknowledging them once durably accepted rather than waiting for the leaderboard update.
2. Consume the stream into one-minute Redis sorted-set buckets, sharding hot article updates and writing category-specific buckets when filtering is required.
3. Every 30 seconds, union the relevant bucket shards for each window and category, then write a small precomputed result key containing the ranked article IDs and scores.
4. Serve GET /trending with one ZREVRANGE or a cached JSON response, including computed_at so clients can see freshness.
5. Use TTLs, replay, idempotent compute jobs, stale snapshots, and metrics for lag and hot-key skew so failures affect freshness before availability.

5-minute explanation

Start with the sliding-window constraint. A counter that only increments cannot answer “last hour” because shares older than an hour must stop contributing. Storing one-minute buckets gives the system a small, explicit set of inputs for every window: a one-hour result unions about 60 buckets, while a seven-day result uses a coarser or hierarchical tier if the one-minute history is too expensive.

The write path is asynchronous. The Share Ingestion Service validates the article and user context, publishes to Kafka, and returns 202 Accepted. Stream processors consume partitions and update sharded bucket leaderboards. For a viral article, routing all ZINCRBY calls to one key would create a hot serialization point, so updates are spread across deterministic shards and combined later. A Count-Min Sketch can gate which low-frequency articles are promoted into sorted sets when the corpus is large.

The read path never unions buckets for every request. A Trending Compute Job periodically reads the active buckets, performs the expensive ZUNIONSTORE work once per window/category, and writes a compact result key. The Read Service returns that snapshot; a CDN can cache public, non-personalized responses for the same 30-second freshness budget. Category and window must be part of every cache and Redis key.

Correctness is intentionally eventual: the result is a ranking snapshot, not a transactional counter. Kafka replay rebuilds missing buckets, bucket TTLs remove old storage, and the previous result remains available if computation or Redis has a short outage. The system should state whether counts are exact, deduplicated, or approximate rather than implying stronger guarantees than the pipeline provides.

45-minute interview approach

The sliding-window and viral-hot-key paths deserve the most discussion; article metadata and notifications can stay below the line.

• 0–5 minutes — Clarify the contract: Confirm what counts as a share, duplicate semantics, windows, K, category cardinality, freshness, exactness, content ownership, and whether results are public or personalized.
• 5–10 minutes — Establish scale: Calculate peak writes, reads, write/read ratio, bucket count, per-article skew, Redis memory, compute frequency, and CDN/origin load.
• 10–15 minutes — Define entities and APIs: Walk through ShareEvent, TrendingBucket, TrendingResult, Article, POST /shares, GET /trending, computed_at, and cursor/key validation.
• 15–22 minutes — Draw ingestion: Start with the database GROUP BY baseline, show its failure mode, then add the gateway, Kafka, stream processors, producer acknowledgement, and replay boundary.
• 22–31 minutes — Deep dive on windows and hot keys: Prioritize bucket expiry, ZUNIONSTORE, deterministic shard routing, category fan-out, CMS gating, and the cost of a viral article.
• 31–36 minutes — Add the read path: Draw the compute job, precomputed result keys, Read Service, CDN, key naming, and the 30-second freshness signal.
• 36–41 minutes — Reliability, security, and operations: Cover Redis/Kafka failure, stale results, duplicates, replay, rate limits, authorization, privacy, lag, and hot-key monitoring.
• 41–45 minutes — Trade-offs and close: Compare exact sorted sets, sketches, batch, stream, query-time aggregation, and CDN caching; recap why time buckets and precomputation are the core invariants.

Core Entities

• ShareEvent: A record that a specific user shared a specific article at a timestamp. Contains article_id, user_id, and shared_at. This is the raw event that drives all downstream counting.
• Article: An external content item identified by article_id, with a category field used for filtered trending queries. This service does not own article content or metadata beyond the category.
• TrendingBucket: An aggregated score for an article within a specific one-minute time bucket. Contains bucket_ts, article_id, category, and share_count. This is the fundamental unit of the sliding window implementation.
• TrendingResult: A precomputed cached list of the top-K articles for a given window and optional category. Refreshed every 30 seconds by the background Trending Compute Job.

Full schema, bucket key design, and Redis data structures are covered in the deep dives. These four entities are sufficient to drive the API and high-level design.

API Design

Three functional requirements drive the API shape.

FR 1 - Record a share event:

POST /articles/{article_id}/shares
Authorization: Bearer <token>

Response: 202 Accepted

202 Accepted over 200 OK because the write is asynchronous: the ingestion service accepts the event to Kafka and returns immediately. A 200 would incorrectly imply the count was atomically updated, which it is not.

FR 2 - Get the top-K trending articles:

GET /trending?window=1h&k=10&category=tech&cursor=<opaque_cursor>

Response: {
  "articles": [
    { "article_id": "a1b2c3", "title": "...", "share_count": 45231, "rank": 1 }
  ],
  "window": "1h",
  "computed_at": "2026-03-29T00:00:00Z",
  "next_cursor": "..."
}

The computed_at field is the explicit freshness signal to clients. It tells product teams that results are up to 30 seconds stale by design, preventing them from building features that depend on exact real-time counts.

Cursor-based pagination handles large K values. Even at K=100, the result set is small, but the cursor enables incremental loading in mobile UIs. The category filter is optional; when omitted, the response covers all categories. When specified, the Read Service fetches a category-specific precomputed result at no extra aggregation cost.

FR 3 - Trending articles filtered by category:

This uses the same GET /trending endpoint shown above with the category query parameter. No separate endpoint is needed because category filtering is resolved at precompute time, not at query time.

High-Level Design

The critical flows are separate: accept and buffer shares, update expiring bucket state, precompute window/category snapshots, and serve those snapshots without doing aggregation on the read path.

1. Naive approach: share events directly to the database

The simplest design writes every share to a share_events table and queries it for the trending list at read time.

Components:

• Client: Sends POST /articles/{id}/shares for writes and GET /trending for reads.
• Share Service: Accepts requests, inserts share rows, and runs aggregate queries for the trending list.
• Database: Stores all share events. Top-K query uses GROUP BY article_id ORDER BY count DESC LIMIT K with a time range filter.

Request walkthrough:

1. Client sends POST /articles/42/shares.
2. Share Service extracts user_id from the auth token.
3. Service inserts (article_id=42, user_id=891, shared_at=NOW()) into the share_events table.
4. Client receives 202 Accepted.
5. Client calls GET /trending?window=1h.
6. Share Service runs SELECT article_id, COUNT(*) FROM share_events WHERE shared_at > NOW() - INTERVAL '1 hour' GROUP BY article_id ORDER BY count DESC LIMIT 10 against the database.
7. Client receives the top-10 list.