The news feed question is really "when do you do the work?"
Fan-out on write feels fast until a celebrity posts to 90 million followers. The senior answer isn't picking a side, it's knowing when to switch.
Design a news feed. First real decision, before data models?
When you do the work. At write time, or at read time.
What's the difference?
A feed is a merge of posts from everyone you follow. You can build that merge when someone posts, or when someone opens the app.
Start with write time.
Fan-out on write. The moment I post, it's pushed into every follower's ready-made feed.
So reads are what?
Instant. Your feed is already assembled, just sitting there.
Perfect for a read-heavy app. Ship it?
Now let a celebrity post to 90 million followers.
And?
That one post becomes 90 million feed writes.
Oof.
It takes minutes to spread. The pipeline stalls. Followers see it at wildly different times.
So flip it. Build at read time.
Fan-out on read. Store each post once. Assemble the feed when someone opens the app.
Now the celebrity is easy.
One write. Trivial.
But?
You've moved the cost onto every read. And reads are the common, latency-sensitive path.
So the average user pays for the celebrity's problem.
Exactly. Which is why neither pure approach is right.
So what do you pick?
You refuse to pick. That's the senior answer: go hybrid.
Hybrid how?
Fan-out on write for normal accounts. For the giant accounts, don't push. Merge their posts in at read time.
you post -> push to followers (write)
celebrity posts -> store once, pull (read)
open app -> precomputed feed + recent celebrity posts How do you decide who's "giant"?
A follower-count threshold. Tuned, not sacred. Above it, that account switches to pull.
Merging at read time, though. Won't the feed be stale or out of order?
It can be. And that's fine. A feed doesn't need strong consistency.
Why not?
A few seconds of staleness harms nobody. So you're free to cache and precompute hard.
The whiteboard line?
It's a question of when you pay for the merge. Write for the many, read for the few, eventually consistent throughout.
↑ answer it in your head first ↑
Traps
- ⚠ Committing to one fan-out strategy for everyone. Fan-out on write breaks for celebrities; fan-out on read breaks for the average user's latency.
- ⚠ Forgetting the write amplification. One celebrity post can mean tens of millions of feed writes if you fan out on write blindly.
- ⚠ Treating the feed as needing strong consistency. A timeline is fine being eventually consistent and slightly stale.