← ByteDance Interview InsightsMachine Learning Engineer

ByteDance·Machine Learning Engineer·Technical Phone Screen·Senior

Senior
Jun 2026

Summary

Bytedance ML engineer interview that went pretty deep into loss functions, way deeper than I expected for what I thought would be a surface-level screen. The KL divergence portion especially caught me off guard in terms of how much they wanted.

Questions Asked (2)

Q1

Walk through the common loss functions (MSE, MAE, Huber, cross-entropy, hinge, focal) and explain when you'd use each one.

Technical Trade-offsAlgorithms & Data Structures
Author's notes

Started fine with MSE vs MAE, talked about outlier sensitivity and when you'd prefer one over the other.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Structure your answer by grouping loss functions by task type (regression vs. classification), then for each loss explain its mathematical intuition, sensitivity to outliers, and typical use cases. Emphasize trade-offs and practical considerations like dataset noise and business impact.

Pro tip: Mention that loss function choice should align with the evaluation metric and business objective—e.g., using MAE when outliers are noise but MSE when large errors are costly—and note that Huber combines the best of both.

1. Categorize by task

Briefly separate regression losses (MSE, MAE, Huber) from classification losses (cross-entropy, hinge, focal) to set a clear structure.

2. Explain regression losses

For MSE, MAE, and Huber, describe their formulas, sensitivity to outliers, and when to use each (e.g., MSE for Gaussian noise, MAE for robustness, Huber as a compromise).

3. Explain classification losses

For cross-entropy, hinge, and focal, explain their probabilistic or margin-based nature, typical use cases (e.g., cross-entropy for probabilistic outputs, hinge for SVMs, focal for class imbalance).

4. Highlight trade-offs and practical tips

Discuss how loss choice affects optimization, convergence, and final model behavior, and mention practical considerations like class imbalance, noisy labels, and evaluation metrics.

5. Conclude with a summary

Summarize the key decision factors (outliers, imbalance, probabilistic needs) and emphasize aligning loss with business goals.

Key Points to Mention

  • MSE penalizes large errors heavily and assumes Gaussian noise; MAE is robust to outliers but has non-smooth gradients; Huber combines both.
  • Cross-entropy is standard for probabilistic classification and works well with softmax; hinge is for maximum-margin classifiers like SVMs.
  • Focal loss down-weights easy examples to address class imbalance, commonly used in object detection.
  • Loss function should match the evaluation metric and business objective (e.g., MAE for median-like predictions, MSE for mean-like).
  • Considerations like noisy labels, outliers, and class imbalance influence loss selection.
  • Practical tip: start with a standard loss, then adjust based on validation performance and error analysis.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q2

Define KL divergence formally, explain why it's asymmetric, and describe its relationship to cross-entropy. When would you use it over a simpler loss?

Technical Trade-offsAlgorithms & Data Structures
Author's notes

This is where things got interesting.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start with the formal definition of KL divergence, then explain its asymmetry through the lens of directionality and support mismatch. Connect it to cross-entropy by decomposing cross-entropy into entropy plus KL divergence, and finally discuss when KL is preferable to simpler losses like MSE, emphasizing probabilistic modeling and distribution matching.

Pro tip: Mention that KL divergence is not a true distance metric (no symmetry, no triangle inequality) and that its asymmetry has practical implications: forward KL (MLE) is mass-covering, while reverse KL is mode-seeking. This shows depth beyond textbook definitions.

1. Formal Definition

Define KL divergence between distributions P and Q as KL(P||Q) = E_P[log(P(x)/Q(x))] = ∫ p(x) log(p(x)/q(x)) dx (or sum for discrete). Emphasize it's the expected log-likelihood ratio under P.

2. Asymmetry Explanation

Explain that KL(P||Q) ≠ KL(Q||P) because the expectation is taken under different distributions. Highlight the consequences: forward KL penalizes Q for assigning low probability where P has mass (zero-avoiding), while reverse KL penalizes Q for assigning mass where P has none (zero-forcing).

3. Relationship to Cross-Entropy

Show that cross-entropy H(P,Q) = H(P) + KL(P||Q). Since H(P) is constant w.r.t. Q, minimizing KL(P||Q) is equivalent to minimizing cross-entropy. This explains why cross-entropy is used as a loss in classification.

4. When to Use KL Over Simpler Losses

Discuss scenarios: when modeling probability distributions (e.g., variational inference, generative models), when the output is a distribution (e.g., softmax), or when you care about relative probabilities rather than absolute errors. Contrast with MSE which assumes Gaussian noise and penalizes squared differences.

5. Practical Considerations

Mention numerical stability (KL can be infinite if Q has zero where P has mass), the need for smoothing, and the choice between forward and reverse KL depending on the application (e.g., forward for density estimation, reverse for variational inference).

Key Points to Mention

  • KL divergence is non-negative and zero iff P=Q almost everywhere.
  • Asymmetry arises because expectation is under P vs Q; forward KL is mass-covering, reverse KL is mode-seeking.
  • Cross-entropy = entropy + KL divergence; minimizing KL is equivalent to minimizing cross-entropy when entropy of P is fixed.
  • KL is preferred when outputs are probability distributions (e.g., classification, language modeling) or when matching distributions (e.g., VAEs, knowledge distillation).
  • MSE is simpler but assumes Gaussian likelihood and penalizes absolute differences, not distributional mismatch.
  • Practical issues: KL can be undefined/infinite if support mismatch; use smoothing or alternative divergences (e.g., Jensen-Shannon).

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.