← Airbnb Interview InsightsMachine Learning Engineer

Airbnb·Machine Learning Engineer·Onsite - System Design / Architecture·Senior

SeniorPrefer not to say
May 2026

Summary

Round 3 of the Airbnb MLE loop was a full ML system design session centered on retrieval and ranking, and it ran long. They clearly care a lot about marketplace ranking depth here, more than any other company I've interviewed with.

Questions Asked (2)

Q1

Design a user embedding and two-stage semantic retrieval and ranking system to surface listings a user is most likely to book, with the goal of maximizing booking conversion.

System DesignTechnical Trade-offs
Author's notes

This was the whole round basically.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the problem scope and success metrics, then walk through the two-stage architecture: candidate generation via user and listing embeddings, followed by a ranking model. Emphasize how each stage contributes to booking conversion and discuss trade-offs in embedding design, retrieval efficiency, and ranking features.

Pro tip: Highlight the importance of negative sampling and position bias in training the ranking model, and mention how you would incorporate business constraints like availability and price sensitivity into the retrieval and ranking stages.

1. Clarify requirements and metrics

Define the goal (maximize booking conversion), identify key metrics (e.g., booking rate, NDCG), and discuss constraints like latency, scalability, and data availability.

2. Design user and listing embeddings

Explain how to create embeddings for users and listings using collaborative filtering, two-tower models, or content-based features, and how to train them with booking data.

3. Build candidate generation (retrieval) stage

Describe using approximate nearest neighbor search (e.g., FAISS) on embeddings to efficiently retrieve a set of candidate listings for each user, balancing recall and latency.

4. Design ranking model

Outline a ranking model (e.g., gradient boosted trees or neural network) that takes user, listing, and context features to predict booking probability, and discuss training with implicit feedback and handling position bias.

5. Evaluate and iterate

Propose offline evaluation (e.g., holdout, counterfactual) and online A/B testing to measure conversion lift, and discuss how to incorporate feedback loops and business rules.

Key Points to Mention

  • Two-tower architecture for learning user and listing embeddings jointly
  • Approximate nearest neighbor search for scalable retrieval
  • Negative sampling and handling implicit feedback in ranking
  • Feature engineering for ranking: user history, listing attributes, context
  • Position bias and debiasing techniques in ranking
  • Online evaluation with A/B tests and business metric alignment

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q2

How would you handle position bias in your training data for a ranking model?

A/B Testing & ExperimentationTechnical Trade-offsProduct Analytics & Metrics
Author's notes

This is where the round got interesting.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by explaining what position bias is in ranking data and why it matters for Airbnb's search ranking. Then outline a structured approach: diagnose the bias, apply mitigation techniques (e.g., inverse propensity weighting, position-aware models), and validate with offline and online experiments. Emphasize trade-offs and the importance of unbiased evaluation metrics.

Pro tip: Mention that position bias can be addressed both in data (e.g., reweighting) and in model architecture (e.g., position as a feature during training but not at inference), and that the choice depends on whether you have exploration data or not. Also highlight the need to monitor for feedback loops in production.

1. Diagnose and quantify position bias

Explain how to detect position bias by analyzing click-through rates across positions and comparing with unbiased data (e.g., from randomized experiments). Use metrics like examination propensity or position-based CTR curves.

2. Choose mitigation strategy

Discuss options such as inverse propensity weighting (IPW), position-aware training (e.g., adding position as a feature only during training), or using randomized exploration data. Consider trade-offs like variance vs. bias and data requirements.

3. Implement and train model

Describe how to incorporate the chosen technique into the training pipeline, ensuring that the model learns unbiased relevance signals. For IPW, compute propensity scores from position data; for position-aware models, use position features only during training.

4. Evaluate offline with unbiased metrics

Use evaluation methods robust to position bias, such as counterfactual evaluation or metrics computed on randomized data. Avoid using biased click data directly for evaluation.

5. Validate online and monitor

Run A/B tests to measure the impact on business metrics and monitor for feedback loops. Continuously check for bias drift and adjust the approach as needed.

Key Points to Mention

  • Inverse propensity weighting (IPW) and its challenges (propensity estimation, variance)
  • Position-aware models (e.g., position as a feature during training only)
  • Use of randomized exploration data to debias
  • Evaluation with unbiased metrics (e.g., from interleaving or counterfactual methods)
  • Trade-offs between bias and variance, and data requirements
  • Monitoring for feedback loops and bias drift in production

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.