I started with the formula and then worked backward to MLE, which felt cleaner than going the other direction.
Start by clearly stating that logistic regression uses binary cross-entropy (log loss), then write the expression and derive it from maximum likelihood estimation. Finally, compare it to mean squared error, highlighting why cross-entropy is preferred for classification due to its convexity and probabilistic interpretation.
Pro tip: Emphasize that cross-entropy is not just a choice but a natural consequence of assuming a Bernoulli distribution for the target and using maximum likelihood; this shows deep understanding beyond memorization.
Identify binary cross-entropy as the loss function for logistic regression and write its mathematical expression.
Assume the target follows a Bernoulli distribution, write the likelihood of the parameters given the data, take the negative log to obtain the cross-entropy loss.
Discuss that cross-entropy is convex for logistic regression, leading to efficient optimization, while MSE is non-convex and suffers from vanishing gradients when predictions are saturated.
Conclude by reiterating the probabilistic foundation and practical benefits, and mention that cross-entropy is standard in modern machine learning frameworks.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.