← Waymo Interview InsightsMachine Learning Engineer

Waymo·Machine Learning Engineer·Technical Phone Screen·Senior

Senior
Jun 2026

Summary

Waymo ML Engineer interview had me coding up logistic regression from scratch using only NumPy, then defending the math and design choices out loud. Pretty rigorous for a coding screen, they clearly wanted to see if you actually understood what's happening under the hood.

Questions Asked (6)

Q1

Implement binary logistic regression from scratch using only NumPy, including forward pass, loss computation, gradient derivation, and a fit method with gradient descent.

Algorithms & Data StructuresTechnical Trade-offs
Author's notes

This one took the full session.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the problem setup and assumptions (binary labels, sigmoid activation, cross-entropy loss). Then implement the class step by step: initialize weights, define forward pass, compute loss, derive gradients, and update weights via gradient descent. Finally, test on a small synthetic dataset and discuss convergence and regularization.

Pro tip: Vectorize all operations to avoid loops, and numerically stabilize the sigmoid and log computations (e.g., clip probabilities or use log-sum-exp trick). Also, mention that you would add L2 regularization and monitor the loss curve to ensure proper learning.

1. Clarify requirements and assumptions

Confirm that the model is for binary classification, uses a sigmoid activation, and is trained with gradient descent on cross-entropy loss. Ask about any constraints (e.g., no bias term, specific initialization).

2. Implement forward pass and loss

Define the linear combination z = X @ w + b, apply sigmoid to get probabilities, and compute binary cross-entropy loss. Ensure numerical stability by clipping probabilities or using a stable log implementation.

3. Derive gradients and implement backward pass

Derive the gradients of the loss with respect to weights and bias: dw = (1/m) * X.T @ (y_hat - y), db = (1/m) * sum(y_hat - y). Implement these vectorized operations.

4. Implement fit method with gradient descent

Initialize weights (e.g., zeros or small random), then iterate for a number of epochs: compute forward pass, loss, gradients, and update weights using learning rate. Optionally include L2 regularization.

5. Test and validate

Generate a small synthetic dataset (e.g., two Gaussian blobs) and verify that the model learns to separate classes. Plot the loss curve and decision boundary to confirm convergence.

Key Points to Mention

  • Vectorization: use NumPy matrix operations to avoid explicit loops for efficiency.
  • Numerical stability: clip probabilities or use log-sum-exp to prevent log(0) errors.
  • Gradient derivation: show that the gradient of cross-entropy with sigmoid simplifies to (y_hat - y).
  • Regularization: add L2 penalty to the loss and its gradient to prevent overfitting.
  • Learning rate and convergence: discuss how to choose learning rate and monitor loss for convergence.
  • Bias term: include a bias term (intercept) unless specified otherwise, and handle it separately in updates.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q2

Add predict_proba and predict methods to your logistic regression class, with a configurable classification threshold.

Algorithms & Data Structures
Author's notes

Straightforward after the hard part was done.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the interface: predict_proba returns class probabilities (or positive class probability), and predict applies a configurable threshold to those probabilities to output class labels. Then implement both methods using the existing model parameters, ensuring numerical stability and vectorized operations. Finally, discuss edge cases like threshold tuning and probability calibration.

Pro tip: Mention that the threshold should be a constructor parameter or a method argument, and that predict should call predict_proba internally to avoid code duplication. Also, note that for binary classification, predict_proba typically returns a 2D array with probabilities for both classes.

1. Define method signatures and threshold handling

Decide whether the threshold is a class attribute (set in __init__) or a parameter to predict. Ensure predict_proba returns probabilities in a consistent format (e.g., 1D array for positive class or 2D array for both classes).

2. Implement predict_proba using the logistic function

Compute the linear combination of weights and features, then apply the sigmoid function to get probabilities. Use vectorized operations (e.g., NumPy) for efficiency and numerical stability (e.g., clip large values).

3. Implement predict using the threshold

Call predict_proba to get probabilities, then compare them against the threshold to produce binary labels (0 or 1). Ensure the threshold is applied consistently and handle edge cases like probabilities exactly equal to the threshold.

4. Validate with tests and discuss edge cases

Test with simple data to verify outputs, and discuss how to handle multi-class scenarios (if applicable) or imbalanced classes. Mention that threshold tuning can be done via cross-validation.

Key Points to Mention

  • Sigmoid function and its role in logistic regression
  • Vectorization for efficiency (avoid loops)
  • Numerical stability (e.g., clipping to avoid overflow)
  • Configurable threshold as a hyperparameter
  • Return format of predict_proba (1D vs 2D array)
  • Threshold tuning and its impact on precision/recall

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q3

How would you add L2 regularization to your implementation, and what changes does it require in the gradient update?

Technical Trade-offsAlgorithms & Data Structures
Author's notes

I knew the penalty term goes in the loss but blanked briefly on whether to regularize the bias term.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by defining L2 regularization as adding a penalty term (λ/2)||w||² to the loss function, then explain how this modifies the gradient update to include λw. Emphasize that this is a straightforward change: compute the gradient of the penalty and add it to the existing gradient, and note that biases are typically not regularized.

Pro tip: Mention that L2 regularization is equivalent to weight decay only for plain SGD; for adaptive optimizers like Adam, you should use decoupled weight decay (AdamW) to avoid unintended interactions. This shows depth and awareness of modern best practices.

1. Define L2 Regularization

Explain that L2 regularization adds a penalty term (λ/2) * sum of squared weights to the loss function, where λ is the regularization hyperparameter.

2. Modify the Loss Function

Show the new total loss: L_total = L_original + (λ/2) * ||w||². Clarify that this penalty is added only for weights, not biases.

3. Compute the Gradient

Derive the gradient of the penalty term: ∇_w (λ/2 ||w||²) = λw. The gradient for biases remains unchanged.

4. Update the Gradient Descent Step

Incorporate the penalty gradient into the weight update: w := w - η(∇_w L_original + λw), where η is the learning rate. This can be rewritten as w := (1 - ηλ)w - η∇_w L_original, showing a multiplicative decay of weights.

5. Discuss Implementation Considerations

Mention that in practice, you add λw to the gradient before the optimizer step, and note that for adaptive optimizers, decoupled weight decay (AdamW) is preferred.

Key Points to Mention

  • L2 regularization adds a penalty proportional to the squared magnitude of weights to the loss function.
  • The gradient of the L2 penalty is λw, which is added to the original gradient.
  • Biases are typically not regularized because they do not contribute to overfitting in the same way.
  • The update rule becomes w := w - η(∇L + λw), which includes a weight decay factor (1 - ηλ).
  • For adaptive optimizers like Adam, L2 regularization via gradient addition is not equivalent to weight decay; use AdamW for decoupled weight decay.
  • L2 regularization helps prevent overfitting by discouraging large weights, improving generalization.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q4

Why do we use cross-entropy loss for classification instead of mean squared error?

Technical Trade-offs
Author's notes

Talked about MSE producing flat gradients when predictions are confident and wrong, which kills learning.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by explaining the probabilistic interpretation: cross-entropy is the negative log-likelihood of the correct label under a softmax model, while MSE assumes Gaussian noise. Then discuss the practical optimization benefits: cross-entropy provides stronger gradients and avoids the vanishing gradient problem when using sigmoid/softmax outputs. Finally, mention that cross-entropy is the proper scoring rule for classification, aligning with maximum likelihood estimation.

Pro tip: Emphasize that the choice is not just about performance but about matching the loss to the underlying data distribution—this shows deeper understanding. Also, note that in practice, cross-entropy leads to faster convergence and better calibration, which is crucial for safety-critical applications like autonomous driving.

1. Probabilistic Foundation

Explain that cross-entropy arises from maximum likelihood estimation for categorical distributions, while MSE corresponds to Gaussian assumptions. This sets the theoretical basis.

2. Gradient Behavior

Discuss how cross-entropy with softmax yields gradients proportional to the error (prediction - target), avoiding the saturation issues of MSE with sigmoid/softmax. This leads to faster and more stable training.

3. Optimization Landscape

Mention that cross-entropy is convex when combined with softmax for linear models, while MSE is non-convex, making optimization easier. For deep networks, it still provides better-conditioned gradients.

4. Practical Implications

Highlight that cross-entropy penalizes confident wrong predictions more heavily, which is desirable for classification. Also, it naturally handles multi-class problems and outputs calibrated probabilities.

5. Contextual Relevance

Tie back to the role: in autonomous driving, well-calibrated probabilities and robust training are critical for safety. Cross-entropy helps achieve that.

Key Points to Mention

  • Cross-entropy is the negative log-likelihood for categorical distributions, aligning with maximum likelihood estimation.
  • MSE assumes Gaussian noise, which is inappropriate for discrete class labels.
  • With softmax output, cross-entropy gradients are (prediction - target), avoiding vanishing gradients.
  • Cross-entropy penalizes confident misclassifications more, leading to better decision boundaries.
  • MSE can lead to slow convergence and poor calibration in classification tasks.
  • Cross-entropy is a proper scoring rule, encouraging honest probabilistic predictions.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q5

How does learning rate affect convergence in gradient descent for logistic regression, and what are the failure modes?

Technical Trade-offsAlgorithms & Data Structures
Author's notes

Too high and you overshoot or diverge, too low and it takes forever.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by defining the role of learning rate as the step size in gradient descent, then explain how it controls convergence speed and stability. Discuss the trade-offs between small and large learning rates, and describe the failure modes such as slow convergence, oscillation, and divergence. Finally, mention practical strategies like learning rate schedules or adaptive methods to mitigate these issues.

Pro tip: Relate the discussion to real-world ML systems: for example, in autonomous driving, a poorly tuned learning rate can lead to models that either take too long to train or fail to converge, impacting safety-critical performance. Mentioning adaptive optimizers like Adam or learning rate warmup shows you understand production-grade training.

1. Define Learning Rate and Its Role

Explain that the learning rate (α) determines the size of steps taken during gradient descent to minimize the logistic regression loss function. It directly influences how quickly or slowly the model converges to the optimal parameters.

2. Describe Convergence Behavior

Discuss how a well-chosen learning rate leads to smooth and efficient convergence. Contrast with too small (slow convergence, may get stuck in plateaus) and too large (overshooting, oscillation, or divergence) learning rates.

3. Identify Failure Modes

Detail specific failure modes: slow convergence, oscillation around the minimum, divergence (loss increases), and potential numerical issues like overflow. Explain how these manifest in logistic regression (e.g., weights exploding).

4. Mitigation Strategies

Propose solutions such as learning rate schedules (step decay, exponential decay), adaptive methods (Adam, RMSprop), or grid search for hyperparameter tuning. Mention that logistic regression is convex, so with a proper learning rate, global convergence is guaranteed.

5. Connect to Practical ML Engineering

Tie the answer to real-world implications, such as training time, resource usage, and model performance. Emphasize the importance of monitoring loss curves and using techniques like early stopping.

Key Points to Mention

  • Learning rate as a hyperparameter controlling step size in gradient descent.
  • Effect on convergence: too small → slow; too large → divergence or oscillation.
  • Failure modes: slow convergence, oscillation, divergence, numerical instability.
  • Logistic regression loss is convex, so proper learning rate ensures global minimum.
  • Adaptive learning rate methods (Adam, RMSprop) and schedules (decay, warmup).
  • Practical monitoring: loss curves, early stopping, and hyperparameter tuning.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q6

How would you implement a numerically stable sigmoid function to avoid overflow issues?

Algorithms & Data StructuresTechnical Trade-offs
Author's notes

Clipping the logits before passing through sigmoid was my first answer.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by explaining the numerical instability of the naive sigmoid implementation for large negative inputs, then present a piecewise stable version that avoids overflow by using different algebraic forms for positive and negative inputs. Finally, discuss the trade-offs and practical considerations for deployment in ML systems.

Pro tip: Mention that many deep learning frameworks already implement a stable sigmoid, but understanding the underlying math is crucial for custom layers or when debugging numerical issues in production. Also, note that the same technique applies to softmax and log-sum-exp.

1. Identify the overflow problem

Explain that computing exp(-x) for large negative x causes overflow, leading to inf and then 0/0 = NaN. Show the naive formula: 1 / (1 + exp(-x)).

2. Derive a stable piecewise formula

For x >= 0, use 1 / (1 + exp(-x)); for x < 0, use exp(x) / (1 + exp(x)). This avoids overflow because the exponent is always non-positive.

3. Implement with vectorization and branching

Show how to implement this efficiently in NumPy or PyTorch using np.where or torch.where to avoid Python loops, ensuring both branches are computed safely.

4. Discuss trade-offs and alternatives

Mention that the piecewise approach adds a branch but is numerically stable; alternatively, use a library function like scipy.special.expit. Also note that for very large |x|, the output saturates to 0 or 1, which is fine.

5. Relate to broader ML context

Connect to other functions like softmax and log-sum-exp that use similar stabilization tricks, and emphasize the importance of numerical stability in production ML systems.

Key Points to Mention

  • Overflow in exp(-x) for large negative x leads to inf and NaN.
  • Piecewise definition: for x >= 0 use 1/(1+exp(-x)), for x < 0 use exp(x)/(1+exp(x)).
  • Vectorized implementation using np.where or torch.where to avoid loops.
  • Trade-off: added branch vs. numerical stability; library functions like scipy.special.expit are available.
  • Saturation for large |x| is acceptable and expected.
  • Similar stabilization techniques apply to softmax and log-sum-exp.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.