← DoorDash Interview InsightsMachine Learning Engineer

DoorDash·Machine Learning Engineer·Onsite - System Design / Architecture·Senior

SeniorPrefer not to say
Jun 2026Remote

Summary

DoorDash ML engineer system design round focused entirely on building a restaurant recommendation system. The emphasis was less on model internals and more on the infrastructure side, which I wasn't fully expecting.

Questions Asked (6)

Q1

Design a restaurant recommendation system, focusing on the deployment and serving infrastructure rather than model internals.

System DesignTechnical Trade-offs
Author's notes

I spent the first few minutes going too deep into model architecture and they redirected me pretty quickly.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the scope: focus on serving infrastructure for a restaurant recommendation system at DoorDash scale, including low-latency inference, feature serving, and model deployment. Structure your answer around the end-to-end request flow, from user request to recommendation response, highlighting key components and trade-offs.

Pro tip: Emphasize how you would handle real-time feature updates and model versioning to ensure freshness and reliability, and discuss monitoring and fallback strategies to maintain service during failures.

1. Clarify Requirements and Constraints

Ask about scale (QPS, number of users/restaurants), latency SLAs, freshness requirements, and availability targets to frame the design.

2. High-Level Architecture

Outline the main components: API gateway, recommendation service, feature store, model server, and data pipelines, and how they interact.

3. Model Serving and Deployment

Discuss how models are deployed (e.g., containerized microservices, serverless), versioning, canary releases, and A/B testing infrastructure.

4. Feature Serving and Real-time Updates

Explain how features are computed and served with low latency, including online/offline consistency and real-time updates.

5. Scalability, Reliability, and Monitoring

Cover scaling strategies (horizontal scaling, caching), failure handling (fallbacks, circuit breakers), and monitoring/alerting for model performance and system health.

Key Points to Mention

  • Low-latency inference: use of caching, pre-computed recommendations, and efficient model serving (e.g., TensorFlow Serving, ONNX Runtime).
  • Feature store: online feature serving with low latency and consistency with offline training (e.g., Feast, Tecton).
  • Model versioning and deployment: canary releases, shadow mode, and A/B testing to safely roll out new models.
  • Scalability: horizontal scaling of services, load balancing, and autoscaling based on traffic patterns.
  • Reliability: fallback to popular or rule-based recommendations if model service fails, circuit breakers, and graceful degradation.
  • Monitoring: tracking prediction latency, error rates, model drift, and business metrics (e.g., click-through rate) with alerting.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q2

What are the functional and non-functional requirements for a restaurant recommendation system, including latency targets, data freshness, and scale?

System DesignProduct Analytics & Metrics
Author's notes

Pretty standard scoping question to open with.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the product scope and user context, then systematically separate functional requirements (what the system does) from non-functional requirements (how well it does it). For non-functional aspects, anchor latency, freshness, and scale targets to concrete user scenarios and business constraints, and justify each with reasoning.

Pro tip: Tie every non-functional target to a user-facing metric or business outcome (e.g., 'p95 latency under 200ms to keep conversion high') and mention trade-offs like freshness vs. cost—this shows product sense beyond pure ML.

1. Clarify scope and assumptions

Ask clarifying questions about the recommendation surface (homepage, search, post-order), user base size, and business goals to ground your requirements.

2. Define functional requirements

List core capabilities: personalized restaurant ranking, filtering by availability/distance, handling cold-start users, and incorporating real-time signals like popular dishes.

3. Define non-functional requirements

Specify latency (e.g., p95 < 200ms for serving), data freshness (e.g., menu updates within minutes, real-time order signals), and scale (e.g., millions of users, thousands of restaurants).

4. Justify targets with trade-offs

Explain why each target matters (e.g., latency impacts conversion) and discuss trade-offs like freshness vs. cost or model complexity vs. inference speed.

5. Summarize and prioritize

Conclude by prioritizing requirements based on business impact and feasibility, and note how you would validate them (e.g., A/B tests, load testing).

Key Points to Mention

  • Latency targets: p95 < 200ms for online serving, with separate budgets for feature fetching and model inference
  • Data freshness: near-real-time updates for restaurant availability and order signals (e.g., <5 min), daily batch for historical features
  • Scale: handle millions of daily active users and tens of thousands of restaurants, with horizontal scaling and caching
  • Functional requirements: personalized ranking, cold-start handling, diversity, and business rules (e.g., sponsored placements)
  • Non-functional: availability (99.9%), fault tolerance, and model monitoring for drift
  • Trade-offs: freshness vs. cost, latency vs. model complexity, and personalization vs. exploration

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q3

How would you distinguish between long-term static features and short-term real-time features in this system, and where would you store each?

System DesignData ModelingTechnical Trade-offs
Author's notes

This was the question I felt best about.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by defining the distinction based on update frequency, latency requirements, and access patterns, then map each feature type to appropriate storage solutions. Emphasize how this separation supports real-time inference and model training at DoorDash scale.

Pro tip: Highlight the trade-offs between consistency and latency, and mention how you'd handle feature freshness and backfilling for real-time features to avoid training-serving skew.

1. Define feature characteristics

Clarify what makes a feature long-term static (e.g., user demographics, store attributes) vs. short-term real-time (e.g., recent orders, current location, session activity).

2. Map to storage solutions

Propose appropriate storage for each: static features in a data warehouse or key-value store with batch updates; real-time features in a low-latency store like Redis or a feature store with streaming ingestion.

3. Address access patterns and latency

Explain how online inference requires millisecond reads for real-time features, while static features can be cached or precomputed; discuss batch vs. streaming pipelines.

4. Discuss consistency and freshness

Cover how to keep real-time features fresh (e.g., via Kafka streams) and ensure static features are updated periodically without impacting serving.

5. Consider operational trade-offs

Mention cost, scalability, and complexity: real-time stores are expensive but necessary for low latency; static stores are cheaper but may have stale data.

Key Points to Mention

  • Feature store architecture (e.g., Feast, Tecton) to manage both types
  • Batch vs. streaming processing (e.g., Spark for static, Flink/Kafka for real-time)
  • Low-latency storage (Redis, DynamoDB) for real-time features
  • Data warehouse (Snowflake, BigQuery) or object storage for static features
  • Training-serving skew and feature freshness guarantees
  • Caching strategies and precomputation for static features

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q4

How do you ensure consistency between online serving and offline training in an ML system like this?

System DesignTechnical Trade-offs
Author's notes

Training-serving skew.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by defining consistency as ensuring that features and model behavior are identical between offline training and online serving. Then walk through the key mechanisms: a shared feature store, point-in-time correct joins, and continuous monitoring with automated retraining. Emphasize trade-offs like latency vs. freshness and how you'd validate consistency in production.

Pro tip: Highlight the importance of logging online features and predictions to detect drift and enable replay-based debugging—this shows you think about production observability beyond just training pipelines.

1. Define consistency and its dimensions

Clarify that consistency means identical feature computation, data distributions, and model behavior across offline and online environments. Mention temporal consistency (point-in-time correctness) and schema consistency.

2. Use a shared feature store

Explain how a feature store (e.g., Feast, Tecton) provides a single definition of features for both training and serving, with offline and online stores kept in sync via materialization jobs.

3. Ensure point-in-time correct joins

Describe how training data must be generated using only data available at prediction time, avoiding label leakage. Use event timestamps and as-of joins to simulate online serving conditions.

4. Monitor and validate in production

Discuss logging online features and predictions, comparing distributions to training data, and setting up alerts for drift. Use shadow deployment or A/B tests to validate model behavior.

5. Automate retraining and backfills

Explain how to periodically retrain models on fresh data and backfill features to correct inconsistencies. Use CI/CD for model deployment with consistency checks as gates.

Key Points to Mention

  • Feature store with unified offline/online transformations
  • Point-in-time correct joins to prevent label leakage
  • Logging and monitoring of online features and predictions
  • Data drift detection and automated retraining
  • Trade-offs between latency, freshness, and consistency
  • Shadow deployment and A/B testing for validation

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q5

How would you set up logging, monitoring, and A/B testing for this recommendation system?

A/B Testing & ExperimentationSystem Design
Author's notes

I covered impression and click logging, model score distributions, and a basic experiment assignment layer.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the system's goals and constraints, then structure your answer around three pillars: logging (what data to capture and how), monitoring (metrics, alerts, and dashboards), and A/B testing (experiment design, metrics, and analysis). Emphasize how these components work together to enable rapid iteration and ensure the recommendation system's health and effectiveness.

Pro tip: Highlight the importance of guardrail metrics and long-term holdout groups to catch regressions and measure long-term impact, showing you understand the trade-offs in production ML systems.

1. Clarify Requirements and Goals

Ask about the system's scale, latency requirements, and key business objectives (e.g., conversion, user satisfaction) to tailor your approach.

2. Design Logging

Specify what to log: user interactions, recommendations shown, model predictions, features, and outcomes. Discuss logging infrastructure (e.g., Kafka, S3) and ensure data quality and privacy.

3. Set Up Monitoring

Define operational metrics (latency, error rates, throughput) and model performance metrics (CTR, coverage, diversity). Describe alerting and dashboards for real-time visibility.

4. Plan A/B Testing

Explain experiment design: randomization unit, sample size, duration, and success metrics. Include guardrail metrics and analysis methods (e.g., sequential testing).

5. Integrate and Iterate

Show how logging feeds monitoring and A/B testing, and how insights drive model improvements. Mention feedback loops and automated retraining.

Key Points to Mention

  • Logging user interactions, impressions, clicks, and model features/predictions for offline analysis and debugging.
  • Monitoring key metrics: latency, error rates, CTR, conversion rate, and business KPIs; set up alerts for anomalies.
  • A/B testing with proper randomization, sample size calculation, and guardrail metrics to avoid negative impacts.
  • Use of long-term holdout groups to measure long-term effects and detect degradation.
  • Data privacy and compliance considerations (e.g., PII handling, GDPR).
  • Integration with existing infrastructure (e.g., DoorDash's experimentation platform) and automation for retraining.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q6

How would you handle cold start for new users and new restaurants, and what fallback strategies would you use?

System DesignAdaptability & Ambiguity
Author's notes

Ended on this one.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the two-sided cold start problem: new users lack interaction history and new restaurants lack engagement data. Then propose a two-pronged solution: for users, leverage onboarding signals and contextual features; for restaurants, use content-based features and exploration. Finally, describe fallback strategies that gracefully degrade to popularity or heuristic-based recommendations.

Pro tip: Emphasize that cold start is not just a modeling problem but a product and data problem—suggest instrumenting the onboarding flow to collect explicit preferences and using that data to bootstrap personalization.

1. Clarify the problem and constraints

Define what 'cold start' means for users and restaurants, and identify available signals (e.g., onboarding survey, restaurant metadata, location, time).

2. User cold start strategy

Use onboarding preferences, contextual features (location, time, device), and demographic data to build an initial profile; consider bandit-based exploration to learn quickly.

3. Restaurant cold start strategy

Leverage content-based features (cuisine, price, ratings, menu items) and similarity to existing restaurants; use exploration to gather engagement data.

4. Fallback strategies

Design a hierarchy of fallbacks: personalized model → segment-based popularity → global popularity → editorial picks, ensuring graceful degradation.

5. Evaluation and iteration

Define metrics for cold-start performance (e.g., CTR, conversion, exploration regret) and set up A/B tests to validate and improve strategies.

Key Points to Mention

  • Two-sided cold start: users and restaurants have different data availability and solutions.
  • Exploration-exploitation trade-off: use multi-armed bandits or Thompson sampling to learn about new entities.
  • Content-based filtering and feature engineering: use metadata and similarity to bridge the gap.
  • Onboarding flow design: collect explicit preferences to bootstrap user profiles.
  • Fallback hierarchy: ensure system robustness with popularity-based or heuristic fallbacks.
  • Evaluation metrics: measure both short-term engagement and long-term learning (e.g., regret).

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.