← Microsoft Interview Insights
This one has three moving parts and you need all three to get credit.
Start by clarifying the requirements: packing strategy (e.g., greedy first-fit), handling of sequences longer than target length, and whether to include padding. Then outline the algorithm: iterate through pairs, place prompt+answer into bins, and construct attention and loss masks that respect sequence boundaries. Finally, discuss trade-offs like efficiency vs. simplicity and edge cases.
Pro tip: Emphasize that loss masks should only be active on answer tokens (not prompt tokens) and that attention masks must prevent cross-contamination between packed sequences—this shows you understand the nuances of SFT training.
Ask about packing strategy (e.g., greedy, best-fit), handling of sequences longer than target length (truncate or skip), and whether padding is allowed. Confirm that loss is computed only on answer tokens.
Propose a greedy approach: iterate through pairs, maintain current packed sequence and remaining capacity, and start a new sequence when the next pair doesn't fit. Discuss alternatives like sorting by length for better packing efficiency.
Explain that attention masks must be block-diagonal to prevent tokens from attending to other sequences in the same packed sequence. For causal LM, also ensure causal masking within each block.
Set loss mask to 1 only for answer tokens and 0 for prompt tokens and padding. This ensures the model is trained only on generating answers.
Address sequences longer than target (truncate or skip), empty sequences, and padding. Discuss trade-offs: packing improves efficiency but complicates masking; greedy is simple but may waste space.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.