← OpenAI Interview InsightsMachine Learning Engineer

OpenAI·Machine Learning Engineer·Technical Phone Screen·Senior

Senior
Jun 2026

Summary

OpenAI ML engineer interview, got a pretty theoretical question about nearest neighbor search that pushed into implementation territory fast.

Questions Asked (1)

Q1

If you switched from L2 to L1 distance, how would you implement a 1-nearest-neighbor classifier using a neural network?

Algorithms & Data StructuresTechnical Trade-offs
Author's notes

Tripped me up a bit because the neural network framing felt like a weird wrapper around what's basically a distance computation problem.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying that a 1-NN classifier with L1 distance can be implemented as a neural network by computing pairwise L1 distances between the query and support set, then selecting the nearest neighbor via a softmin or hard argmin. Explain how to handle the non-differentiability of L1 and the discrete selection, possibly using a differentiable relaxation for training or a hard selection for inference.

Pro tip: Mention that L1 distance is more robust to outliers than L2, but requires careful handling of non-differentiability at zero; use a smooth approximation like the Huber loss or subgradients if training the network end-to-end.

1. Clarify the problem and assumptions

Confirm that the classifier is 1-NN with L1 distance, and that the neural network will either compute distances or be trained to approximate the selection. State whether the support set is fixed or variable.

2. Design the distance computation layer

Use a neural network layer to compute pairwise L1 distances between the query and each support example. This can be done by subtracting and taking absolute values, then summing over dimensions.

3. Implement the nearest neighbor selection

For hard 1-NN, use an argmin over distances to pick the nearest neighbor's label. For a differentiable version, use a softmin (e.g., negative distances passed through a softmax with low temperature) to approximate the selection.

4. Address differentiability and training

If training the network end-to-end, handle the non-differentiability of L1 at zero and the argmin by using subgradients or smooth approximations. Alternatively, train the feature extractor separately and use the network only for inference.

5. Discuss trade-offs and extensions

Compare L1 vs L2 in terms of robustness, computational cost, and gradient behavior. Mention possible extensions like k-NN or using learned distance metrics.

Key Points to Mention

  • L1 distance computation: sum of absolute differences.
  • Non-differentiability of L1 at zero and how to handle it (subgradients, smooth approximations).
  • Softmin/softmax with temperature for differentiable nearest neighbor selection.
  • Trade-offs: L1 robustness to outliers vs L2 sensitivity; computational complexity.
  • Implementation details: pairwise distance matrix, broadcasting, and efficient computation.
  • Relation to metric learning and prototypical networks.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.