← Instacart Interview Insights

Instacart·Machine Learning Engineer·Technical Phone Screen·Senior

Senior
May 2026

Summary

ML concepts interview at Instacart for an MLE role. One meaty theory question that turned into a back-and-forth grilling session. The interviewer clearly wanted more than a surface-level answer and kept pushing until I got to the nuance.

Questions Asked (1)

Q1

With MSE as the loss function, do batch gradient descent and SGD converge to the same point?

Technical Trade-offsAlgorithms & Data Structures
Author's notes

Said 'yes' pretty confidently at first because MSE is convex and I figured both optimizers are just finding the minimum.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Clarify that with MSE loss, batch gradient descent and SGD converge to the same global minimum for convex models (e.g., linear regression), but may converge to different local minima or points for non-convex models (e.g., neural networks). Emphasize that the loss surface and optimization dynamics determine convergence, not just the loss function.

Pro tip: Mention that in practice, SGD's noise can help escape shallow local minima and sometimes find better solutions, but it never guarantees the same point as batch GD. Also note that with a decaying learning rate, SGD can converge to a local minimum, but not necessarily the same as batch GD.

1. Define the optimization problem

State that MSE loss defines a loss surface, and both batch GD and SGD are iterative methods to minimize it. The convergence point depends on the convexity of the loss function with respect to the model parameters.

2. Analyze convex case

For convex models like linear regression, MSE is convex, so both batch GD and SGD (with appropriate learning rate schedule) converge to the same global minimum.

3. Analyze non-convex case

For non-convex models like neural networks, MSE is non-convex. Batch GD follows the full gradient and may converge to a local minimum; SGD's stochasticity can lead to different local minima or even saddle points.

4. Discuss convergence conditions

Mention that SGD requires a decreasing learning rate to converge to a minimum, while batch GD can converge with a fixed learning rate if it's small enough. Even then, the points may differ due to different optimization paths.

5. Summarize practical implications

Conclude that they do not always converge to the same point; only in convex settings with proper schedules. In deep learning, differences are expected and can be beneficial.

Key Points to Mention

  • Convexity of MSE loss for linear models vs. non-convexity for neural networks
  • Batch GD uses full gradient; SGD uses noisy gradient estimates
  • Learning rate schedules: SGD needs decay to converge; batch GD can use fixed rate
  • Local minima and saddle points in non-convex optimization
  • SGD's noise can act as regularization and help escape shallow minima
  • In practice, exact convergence point is less important than generalization performance

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.