← Upstart Interview Insights

Upstart·Machine Learning Engineer·Technical Phone Screen·Senior

Senior
May 2026

Summary

Interviewed for an ML Engineer role at Upstart and got hit with a pretty deep theory question on logistic regression loss. Not a vibe check, they wanted you to actually derive things from scratch.

Questions Asked (1)

Q1

What is the loss function used in logistic regression? Can you write out the binary cross-entropy expression, derive it from first principles using maximum likelihood estimation, and explain why it's preferred over mean squared error for classification tasks?

Technical Trade-offsAlgorithms & Data Structures
Author's notes

I started with the formula and then worked backward to MLE, which felt cleaner than going the other direction.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clearly stating that logistic regression uses binary cross-entropy (log loss), then write the expression and derive it from maximum likelihood estimation. Finally, compare it to mean squared error, highlighting why cross-entropy is preferred for classification due to its convexity and probabilistic interpretation.

Pro tip: Emphasize that cross-entropy is not just a choice but a natural consequence of assuming a Bernoulli distribution for the target and using maximum likelihood; this shows deep understanding beyond memorization.

1. State the loss function

Identify binary cross-entropy as the loss function for logistic regression and write its mathematical expression.

2. Derive from maximum likelihood

Assume the target follows a Bernoulli distribution, write the likelihood of the parameters given the data, take the negative log to obtain the cross-entropy loss.

3. Explain why cross-entropy over MSE

Discuss that cross-entropy is convex for logistic regression, leading to efficient optimization, while MSE is non-convex and suffers from vanishing gradients when predictions are saturated.

4. Summarize and connect to practice

Conclude by reiterating the probabilistic foundation and practical benefits, and mention that cross-entropy is standard in modern machine learning frameworks.

Key Points to Mention

  • Binary cross-entropy expression: L(y, p) = -[y log(p) + (1-y) log(1-p)]
  • Maximum likelihood estimation: likelihood = ∏ p_i^{y_i} (1-p_i)^{1-y_i}, negative log-likelihood gives cross-entropy
  • Convexity: cross-entropy is convex in parameters for logistic regression, ensuring global optimum
  • MSE non-convexity: MSE leads to non-convex loss surface and slow convergence due to gradient vanishing when sigmoid saturates
  • Probabilistic interpretation: cross-entropy aligns with Bernoulli distribution and maximum likelihood
  • Practical preference: cross-entropy is standard in logistic regression and neural networks for classification

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.