I started with MSE and MAE which was fine, but then they kept asking me to go deeper on when you'd actually pick one over the other in practice.
Start by stating that the choice of loss function depends on the data distribution and the desired robustness to outliers. Then compare MSE, MAE, and Huber loss in terms of their mathematical properties and practical implications. Finally, discuss how these properties affect optimization and model performance.
Pro tip: Mention that in practice, the loss function should be aligned with the evaluation metric and business objective; for example, if the metric is MAE, using MSE for training might lead to suboptimal performance. Also, note that Huber loss is a good default when you suspect outliers but still want smooth gradients.
Clarify that the choice depends on the presence of outliers, noise distribution, and whether you need probabilistic outputs. For UiPath, consider if the regression task involves robotic process automation data which may have outliers.
List MSE (L2), MAE (L1), and Huber loss as the main options. Briefly define each: MSE is squared error, MAE is absolute error, Huber is a combination that is quadratic for small errors and linear for large errors.
Explain that MSE is sensitive to outliers because errors are squared, MAE is robust as it treats all errors linearly, and Huber is robust but less extreme than MAE for large errors.
MSE has smooth gradients that are proportional to error, which can lead to large updates for outliers. MAE has constant gradient magnitude (except at zero) which can cause issues near the optimum. Huber has smooth gradients and transitions to linear for large errors, combining benefits.
MSE assumes Gaussian noise, MAE assumes Laplacian noise, and Huber assumes a mixture or is a compromise. Mention that the choice should reflect the believed noise distribution.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.