← AppLovin Interview Insights

AppLovin·Machine Learning Engineer·Technical Phone Screen·Senior

Senior
May 2026

Summary

Phone screen for an MLE role at Applovin, pretty much all ML theory. Four questions back to back, all loss functions and imbalanced data stuff. Not a lot of back-and-forth, more like a knowledge check.

Questions Asked (4)

Q1

How do you handle imbalanced classification datasets?

Technical Trade-offsAlgorithms & Data Structures
Author's notes

Ran through the usual toolkit: resampling, class weights, threshold tuning, switching to precision-recall instead of accuracy.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by acknowledging that imbalanced datasets are common in real-world ML, especially in ad-tech. Then outline a systematic approach: first diagnose the imbalance and its impact, then choose appropriate techniques (data-level, algorithm-level, or hybrid), and finally evaluate with the right metrics. Emphasize that the choice depends on the problem context and business goals.

Pro tip: Mention that in production, you often care more about the ranking of predictions than the absolute threshold, so techniques like focal loss or adjusting class weights can be more effective than resampling, which may introduce bias. Also, always validate on a holdout set that reflects the real-world distribution.

1. Diagnose the imbalance

Quantify the class distribution and assess how it affects model performance. Check if the imbalance is severe (e.g., 1:1000) and whether it causes the model to ignore the minority class.

2. Choose the right metric

Select evaluation metrics that are robust to imbalance, such as precision-recall AUC, F1-score, or Matthews correlation coefficient, instead of accuracy.

3. Apply data-level techniques

Consider resampling methods like oversampling the minority class (e.g., SMOTE) or undersampling the majority class, but be aware of their limitations and potential to introduce noise or lose information.

4. Apply algorithm-level techniques

Use class weights, cost-sensitive learning, or specialized loss functions (e.g., focal loss) to make the model pay more attention to the minority class.

5. Validate and iterate

Evaluate the chosen approach on a validation set that mirrors the real-world distribution, and iterate on the combination of techniques to optimize the business metric.

Key Points to Mention

  • Class imbalance is common in ad-tech (e.g., click-through rate prediction, fraud detection).
  • Accuracy is misleading; use precision, recall, F1, PR-AUC, or ROC-AUC.
  • Resampling techniques: oversampling (SMOTE, ADASYN), undersampling, and hybrid methods.
  • Algorithm-level approaches: class weights, cost-sensitive learning, focal loss, and threshold tuning.
  • Ensemble methods like balanced bagging or boosting can help.
  • Always validate on a holdout set that reflects the real-world distribution and consider business impact.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q2

Compare MAE, MSE, and Huber loss. When would you choose each?

Technical Trade-offsAlgorithms & Data Structures
Author's notes

Felt pretty solid here.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by defining each loss function mathematically and highlighting their sensitivity to outliers. Then compare their properties (e.g., gradient behavior, robustness) and discuss practical scenarios for choosing each, especially in the context of ad-tech and recommendation systems at Applovin.

Pro tip: Mention that in production systems like Applovin's, the choice often depends on the noise level in the data and the business metric—e.g., if outliers are due to bot traffic, Huber loss can provide robustness without completely ignoring them.

1. Define the loss functions

Briefly state the formulas: MAE is the mean of absolute errors, MSE is the mean of squared errors, and Huber is a combination of both with a threshold delta.

2. Compare sensitivity to outliers

Explain that MSE heavily penalizes large errors (due to squaring), MAE is robust but has a constant gradient, and Huber behaves like MSE for small errors and like MAE for large errors.

3. Discuss optimization and convergence

Note that MSE is smooth and easy to optimize with gradient descent, MAE has a discontinuous gradient at zero (subgradient methods needed), and Huber is smooth and differentiable.

4. Relate to business context

Tie the choice to Applovin's use cases: e.g., MSE for well-behaved data, MAE for noisy data with outliers, Huber for a balance when outliers exist but you still want smooth optimization.

5. Summarize with a decision rule

Conclude with a simple guideline: use MSE when outliers are not a concern and you want fast convergence; use MAE when outliers dominate and you want robustness; use Huber when you need a compromise.

Key Points to Mention

  • Mathematical definitions: MAE = (1/n)Σ|y - ŷ|, MSE = (1/n)Σ(y - ŷ)², Huber = 0.5(y - ŷ)² if |y - ŷ| ≤ δ else δ|y - ŷ| - 0.5δ².
  • Outlier sensitivity: MSE is highly sensitive, MAE is robust, Huber is robust to outliers but still smooth.
  • Gradient behavior: MSE has gradients proportional to error, MAE has constant gradient (except at 0), Huber transitions from linear to quadratic.
  • Optimization: MSE and Huber are differentiable everywhere (Huber at δ), MAE is not differentiable at 0.
  • Use cases: MSE for Gaussian noise, MAE for Laplacian noise, Huber for mixed noise or when outliers are present but not dominant.
  • Applovin context: In ad-tech, outliers can come from fraudulent clicks or rare events; Huber can mitigate their impact while maintaining stable training.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q3

If you need a model to penalize over-predictions more heavily than under-predictions (or the other way around), how would you design the loss function?

Technical Trade-offsAlgorithms & Data Structures
Author's notes

This one tripped me up more than I expected.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the business context and the cost asymmetry between over- and under-predictions. Then propose a weighted loss function (e.g., asymmetric MSE or quantile loss) that encodes the desired penalty, and discuss how to tune the weight and evaluate the trade-off.

Pro tip: Mention that asymmetric loss is equivalent to quantile regression at a specific quantile, which provides a principled way to set the penalty ratio. Also, emphasize the importance of validating the loss with business metrics, not just statistical ones.

1. Clarify the problem and cost asymmetry

Ask about the specific application and quantify the relative cost of over- vs under-prediction. This ensures the loss function aligns with business objectives.

2. Choose a base loss and introduce asymmetry

Select a standard loss (e.g., MSE, MAE) and modify it with a weight parameter that penalizes one direction more. For example, asymmetric MSE: L(y, ŷ) = α * (y - ŷ)^2 if y < ŷ else (1-α) * (y - ŷ)^2.

3. Consider quantile loss as a principled alternative

Explain that quantile loss (pinball loss) naturally penalizes over- and under-predictions differently based on the quantile τ. Setting τ > 0.5 penalizes under-predictions more, while τ < 0.5 penalizes over-predictions more.

4. Tune the penalty parameter

Use cross-validation or a validation set to tune the weight α or quantile τ based on the desired trade-off. Monitor both the loss and relevant business metrics.

5. Evaluate and iterate

Assess the model's performance using metrics that reflect the asymmetric costs (e.g., asymmetric error, profit curves). Iterate on the loss design if needed.

Key Points to Mention

  • Asymmetric loss functions: weighted MSE, weighted MAE, quantile (pinball) loss
  • Quantile regression interpretation: choosing τ to reflect the penalty ratio
  • Tuning the penalty parameter via cross-validation
  • Business metric alignment: ensuring the loss reflects real-world costs
  • Potential issues: gradient scaling, optimization stability, and calibration
  • Alternatives: custom loss with smooth approximations for differentiability

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q4

Explain quantile regression and derive the pinball loss. How does it handle asymmetric error penalization?

Algorithms & Data StructuresTechnical Trade-offs
Author's notes

Genuinely the hardest one.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by defining quantile regression as estimating conditional quantiles of the target, then derive the pinball loss by minimizing the asymmetric absolute error. Explain how the loss weights positive and negative errors differently based on the target quantile, enabling asymmetric penalization.

Pro tip: Connect quantile regression to real-world applications like predicting delivery times or financial risk, where asymmetric costs are common, to show practical insight.

1. Define Quantile Regression

Explain that quantile regression estimates the conditional quantile of the response variable given predictors, unlike mean regression. Mention that it is robust to outliers and heteroscedasticity.

2. Introduce the Pinball Loss

State that the pinball loss (or quantile loss) is the objective function minimized to obtain quantile estimates. For a quantile τ, it is defined as L(y, ŷ) = (τ - 1_{y < ŷ}) * (y - ŷ).

3. Derive the Pinball Loss

Show that minimizing the expected pinball loss yields the τ-th quantile. Derive by setting the derivative of the expected loss with respect to ŷ to zero, leading to P(y < ŷ) = τ.

4. Explain Asymmetric Penalization

Describe how the loss penalizes overestimation and underestimation differently: for τ > 0.5, underestimation is penalized more; for τ < 0.5, overestimation is penalized more. This asymmetry aligns with the desired quantile.

5. Discuss Handling of Asymmetric Errors

Conclude that this asymmetric penalization allows quantile regression to model the entire conditional distribution and provide prediction intervals, useful when errors have different costs.

Key Points to Mention

  • Quantile regression estimates conditional quantiles, not just the mean.
  • Pinball loss is defined as ρ_τ(y - ŷ) = max(τ(y - ŷ), (τ - 1)(y - ŷ)).
  • Derivation: minimize expected loss; derivative yields P(y < ŷ) = τ.
  • Asymmetric penalization: weights depend on τ, penalizing over- and under-prediction differently.
  • Use cases: robust regression, prediction intervals, asymmetric cost scenarios.
  • Comparison to mean regression: mean regression minimizes squared error, sensitive to outliers.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.