← Meta Interview InsightsMachine Learning Engineer

Meta·Machine Learning Engineer·Technical Phone Screen·Senior

Senior
Jun 2026

Summary

Meta ML engineer screen that was basically one meaty coding problem: implement linear regression from scratch using the closed-form solution. They let you use AI to look up the gradient derivation if you blanked, which is a weird flex but actually made it more interesting since you still had to verify the result numerically.

Questions Asked (1)

Q1

Given a set of (x, y) points, implement linear regression by hand using the closed-form solution that minimizes mean squared error. Derive the gradient of MSE with respect to the weights, set it to zero, write out the closed-form expression, then implement it in code. You can use AI assistance to derive the gradient if needed, but verify your result numerically against gradient descent on a small synthetic dataset.

Algorithms & Data StructuresTechnical Trade-offs
Author's notes

This is the kind of question that sounds easy until you're actually staring at it live.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clearly stating the linear regression model and MSE loss, then derive the gradient with respect to weights, set it to zero, and solve for the closed-form solution. Implement the solution in code, and validate it by comparing against gradient descent on a small synthetic dataset, discussing any numerical considerations.

Pro tip: Emphasize the importance of numerical stability and verification: mention using np.linalg.lstsq or adding a small ridge term to avoid singular matrices, and always validate closed-form solutions against iterative methods to catch implementation errors.

1. Define Model and Loss

State the linear model y = Xw + b (or include bias in X) and define MSE loss as (1/n) * ||y - Xw||^2. Clarify assumptions and notation.

2. Derive Gradient

Compute the gradient of MSE with respect to w: ∇_w MSE = -(2/n) * X^T (y - Xw). Show steps or use matrix calculus identities.

3. Set Gradient to Zero and Solve

Set gradient to zero: X^T X w = X^T y, leading to the normal equations. Solve for w: w = (X^T X)^{-1} X^T y, discussing when inverse exists.

4. Implement in Code

Write a function that computes w using the closed-form expression, handling bias by augmenting X with a column of ones. Use numpy for matrix operations.

5. Validate Numerically

Generate a small synthetic dataset, run gradient descent to minimize MSE, and compare the resulting weights and loss to the closed-form solution. Discuss tolerance and convergence.

Key Points to Mention

  • Matrix calculus derivation of the gradient, including the chain rule and properties of transpose.
  • The normal equations and conditions for invertibility (e.g., full rank, positive definiteness of X^T X).
  • Numerical stability: using pseudo-inverse or lstsq instead of explicit inverse, and regularization if needed.
  • Implementation details: augmenting X with a bias column, vectorization for efficiency.
  • Validation methodology: synthetic data generation, gradient descent hyperparameters (learning rate, iterations), and comparison metrics.
  • Trade-offs: closed-form vs. iterative methods (e.g., scalability, memory, and when to use each).

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.