Author's notes
I knew the 'variance grows with d_k' part well enough, dot product of two d_k-dimensional vectors with unit variance entries gives you variance d_k, so you scale by sqrt(d_k) to bring it back to 1. The rest of the author's notes on Machine Learning Engineer interview at Netflix, Technical Phone Screen round, covers how they worked through the question, what the panel pushed back on, and what they would do differently.
Create a free account to read the full note