← Amazon Interview InsightsMachine Learning Engineer
The 'what is logistic regression' part is easy enough, but the 'why log' follow-up is where people trip.
Start by defining logistic regression as a linear model for binary classification that estimates probabilities using the logistic (sigmoid) function. Then explain that the log transformation (log-odds or logit) linearizes the relationship between the input features and the target, enabling the use of a linear model while ensuring outputs are valid probabilities. Finally, connect this to the optimization process (e.g., maximum likelihood) and practical benefits like interpretability and numerical stability.
Pro tip: Emphasize that the log-odds transformation is not just a mathematical convenience but also aligns with the assumption that features have a multiplicative effect on the odds, which is often more realistic than additive effects on probabilities. Also, mention that logistic regression outputs well-calibrated probabilities, which is crucial for decision-making in many applications.
Explain that logistic regression is a statistical model used for binary classification that predicts the probability of an instance belonging to a particular class. It uses a linear combination of input features passed through a sigmoid function to output a probability between 0 and 1.
Discuss that a linear model directly on probabilities would be inappropriate because probabilities are bounded between 0 and 1, while linear combinations can take any real value. The log-odds (logit) transformation maps probabilities from [0,1] to the entire real line, making it suitable for linear modeling.
Introduce the logit function: log(p/(1-p)), which is the inverse of the sigmoid. Explain that this transformation linearizes the relationship between features and the log-odds of the outcome, allowing for a linear decision boundary in the feature space.
Mention that logistic regression is typically fit by maximizing the likelihood (or minimizing log loss), which is equivalent to minimizing the negative log-likelihood. The coefficients represent the change in log-odds per unit change in the feature, which can be exponentiated to odds ratios for interpretability.
Highlight that the log transformation ensures numerical stability, provides interpretable coefficients, and allows for efficient optimization. However, note that logistic regression assumes a linear relationship between features and log-odds, which may not always hold.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.