← Reddit Interview InsightsSoftware Engineer
I started with the ingestion pipeline and that part felt okay, events flowing into a queue, consumers aggregating views, likes, shares.
Start by clarifying requirements: define 'hot' (e.g., upvotes, comments, shares), the configurable time window, scale, and latency needs. Then propose a hybrid architecture: stream ingestion for real-time updates, a time-decayed scoring model, and a two-tier storage (fast in-memory for recent windows, batch for longer windows). Finally, discuss top-K maintenance using heaps or approximate algorithms and tradeoffs between real-time and batch processing.
Pro tip: Emphasize that 'hot' is subjective and changes over time; propose a configurable scoring function and mention that you'd A/B test different decay parameters to optimize engagement. Also, highlight the importance of idempotent ingestion and handling late data.
Ask about scale (posts per second, users), time windows (1 hour, 1 day), latency requirements (real-time vs near-real-time), and what signals define 'hot' (upvotes, comments, shares, etc.). Define a configurable scoring function that combines these signals.
Propose a scalable ingestion layer (e.g., Kafka) to collect user interactions (votes, comments) in real-time. Ensure events are partitioned by post ID for ordered processing and idempotency to handle duplicates.
Aggregate signals per post using a stream processor (e.g., Flink, Spark Streaming) that applies a time-decay function (e.g., exponential decay) to older interactions. Maintain a running score for each post within the active window.
Use a min-heap or a count-min sketch with heap for approximate top-K per time window. Store scores in a fast in-memory store (e.g., Redis) for real-time queries, and periodically persist to a durable store (e.g., Cassandra) for batch processing and longer windows.
Discuss tradeoffs: real-time gives low latency but may be noisy; batch gives accuracy but higher latency. Propose a lambda architecture: speed layer for real-time top-K, batch layer for recomputation and correction, serving layer merges results. Also, consider windowing and late data handling.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.