← Microsoft Interview InsightsMachine Learning Engineer
This one took me a minute to get the log-probability part right.
Start by defining the beam search algorithm and how it extends greedy decoding by maintaining k hypotheses. Then walk through the iterative process: expanding each hypothesis, scoring with log-probabilities, selecting top k, and handling EOS. Finally, discuss termination conditions and trade-offs.
Pro tip: Emphasize that beam search with k=1 is equivalent to greedy decoding, and mention that length normalization is often needed to avoid favoring shorter sequences. Also, note that beam search can be parallelized across hypotheses for efficiency.
Begin with a single hypothesis containing the start-of-sequence token, with a cumulative log-probability of 0. Set up a list to hold the current beam of size up to k.
For each hypothesis in the current beam, compute the next-token probabilities using the model. Take the top k token candidates per hypothesis (or all if vocabulary is small) to form new extensions.
For each extension, add the log-probability of the new token to the hypothesis's cumulative score. Collect all extensions, sort by score, and keep the top k hypotheses as the new beam.
When a hypothesis generates the EOS token, mark it as complete and move it to a separate list of finished hypotheses. It no longer participates in expansion but remains for final selection.
Stop when all hypotheses in the beam are complete or a maximum length is reached. Return the completed hypothesis with the highest score, optionally applying length normalization.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.