← Netflix Interview Insights

Netflix·Machine Learning Engineer·Technical Phone Screen·Senior

Senior
May 2026

Summary

Netflix ML engineer interview, technical phone screen focused on transformer internals. Just one question but it went pretty deep pretty fast.

Questions Asked (1)

Q1

In scaled dot-product attention, scores are computed as QK^T divided by the square root of d_k. Why is that scaling factor necessary, and what happens to training if you leave it out?

Technical Trade-offsAlgorithms & Data Structures
Author's notes

I knew the 'variance grows with d_k' part well enough, dot product of two d_k-dimensional vectors with unit variance entries gives you variance d_k, so you scale by sqrt(d_k) to bring it back to 1.

Create a free account to read the full note