← DoorDash Interview InsightsMachine Learning Engineer
This is a beast of a question because it sounds like an ML design problem but they explicitly want you off the model and onto the plumbing.
Start by clarifying requirements and scale, then design a high-level architecture covering data ingestion, feature serving, candidate generation, ranking, and ad serving. Focus on backend infrastructure components like low-latency feature stores, real-time bidding, and scalable serving, while discussing trade-offs between latency, cost, and relevance.
Pro tip: Emphasize how you would handle cold-start and real-time updates in the feature store, as this is a common pain point in ads systems and shows depth beyond basic design.
Ask about scale (QPS, number of users, ads), latency requirements (e.g., <100ms), and key metrics (CTR, conversion). This ensures the design meets business needs.
Outline the main components: data pipeline (batch and stream), feature store, candidate generation, ranking service, ad server, and feedback loop. Sketch how they interact.
Pick 2-3 components to detail, such as the feature store (for low-latency serving) and the ad server (for real-time bidding and pacing). Discuss technologies and trade-offs.
Explain how to scale horizontally, handle failures (e.g., fallback to popular ads), and ensure low latency via caching, sharding, and async processing.
Summarize key trade-offs (e.g., latency vs. model complexity, cost vs. relevance) and how you would monitor and iterate on the system.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Start by clarifying the business goal: pacing ensures advertisers spend their budget smoothly over the campaign window, balancing delivery and ROI. Then propose a control system that forecasts spend, detects deviations, and adjusts bidding or delivery in real-time, using ML to predict future spend and optimize pacing decisions.
Pro tip: Emphasize the trade-off between pacing accuracy and delivery performance—overly strict pacing can hurt ad relevance and user experience. Mention how you'd simulate or A/B test pacing strategies to measure impact on advertiser ROI and platform revenue.
Ask about campaign duration, budget type (daily vs. lifetime), advertiser goals, and tolerance for overspend/underspend. Understand data availability and latency requirements.
Propose a closed-loop system: monitor spend, compare to target, and adjust bidding or throttling. Use a PID controller or model predictive control for smooth adjustments.
Use time-series models to predict future spend and detect anomalies (e.g., sudden traffic spikes). Incorporate features like time of day, day of week, and competitor activity.
Balance pacing with delivery (e.g., avoid throttling too much). Handle cold start, budget changes, and campaign pauses. Consider multi-objective optimization.
Define metrics (e.g., pacing error, ROI, fill rate) and set up A/B tests or simulations. Continuously retrain models and tune controller parameters.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Exactly-once is always a trap in distributed systems and I said that out loud, which maybe wasn't the smoothest move.
Start by clarifying the business context: DoorDash bills advertisers for clicks and impressions, so exactly-once billing means each valid event is counted once and only once. Then propose an end-to-end pipeline that uses idempotent event ingestion, deduplication with unique event IDs, and a transactional billing layer that reconciles counts before charging. Emphasize trade-offs between latency, cost, and accuracy, and how you'd handle late or duplicate events.
Pro tip: Mention that exactly-once billing is often achieved through at-least-once delivery plus idempotent processing and a reconciliation step, rather than trying to enforce exactly-once end-to-end. Also highlight the importance of a dead-letter queue and audit trail for disputed charges.
Ask about event volume, latency requirements, billing frequency, and tolerance for discrepancies. Confirm that 'exactly-once' applies to billing, not necessarily to event processing.
Use a unique event ID generated at the source (e.g., client or server) and store events in a durable log like Kafka. Ensure producers can retry without creating duplicates by using idempotent producers.
Process events through a stream processor (e.g., Flink) that deduplicates based on event ID within a time window, validates against fraud or invalid traffic, and emits only unique billable events.
Aggregate billable events per advertiser and campaign, then write to a billing database using transactions with unique constraints to prevent double-counting. Use a reconciliation job to compare raw event counts with billed counts.
Set up monitoring for duplicates, late events, and discrepancies. Maintain an audit log of all billing decisions and provide a way to correct errors without double-billing.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Went with a two-stage approach: a fast approximate retrieval step using an inverted index on targeting criteria (geo, category, audience segment) to get a candidate set, then a second pass with embedding similarity for more personalized ranking.
Start by clarifying the requirements: what 'candidate ads' means (e.g., ads eligible for a user-context pair), latency/throughput targets, and scale. Then outline a multi-stage retrieval system: offline candidate generation, online filtering/ranking, and serving with caching and sharding. Emphasize trade-offs between recall and latency, and how you'd measure and optimize each stage.
Pro tip: Show awareness of the two-tower model architecture for candidate generation and the importance of precomputing embeddings and using approximate nearest neighbor (ANN) search. Also mention that you'd monitor p99 latency and have fallbacks to avoid cascading failures.
Ask about scale (QPS, number of users/ads), latency SLA (e.g., <100ms), throughput, and what 'candidate ads' entails (e.g., ads eligible for auction). Confirm if personalization is needed and if context includes time, location, etc.
Propose a multi-stage pipeline: candidate generation (retrieve hundreds from millions), filtering (remove ineligible), ranking (score and select top K). Mention offline/online split: precompute user/ad embeddings offline, serve via ANN lookup online.
Describe using two-tower models to embed users and ads into a shared space, then ANN search (e.g., FAISS, ScaNN) for fast retrieval. Alternatively, use inverted indices or rule-based retrieval for cold-start. Emphasize precomputation and caching.
Discuss sharding by user ID or geo, in-memory databases (Redis), and caching frequent queries. Use async I/O, connection pooling, and load balancing. Consider edge caching for popular contexts.
Define metrics: p50/p99 latency, QPS, recall@K, CTR. Set up A/B testing for model updates. Implement fallbacks (e.g., popular ads) if retrieval fails. Continuously optimize by pruning candidates and tuning ANN parameters.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Start by defining the core non-functional requirements for an ads serving system, such as low latency, high throughput, scalability, reliability, and cost-efficiency. Then, explain how these requirements inherently conflict—for example, low latency vs. high accuracy, or scalability vs. cost—and discuss strategies to balance them in a real-world ML-driven ads system.
Pro tip: Frame the trade-offs in the context of DoorDash's three-sided marketplace (consumers, merchants, Dashers) and emphasize that ad relevance must be balanced with user experience and platform revenue goals.
List the critical NFRs for an ads serving system: latency (e.g., <100ms), throughput (QPS), scalability, reliability/availability, and cost-efficiency.
Briefly justify each NFR: low latency for real-time bidding, high throughput for peak traffic, scalability for growth, reliability for revenue continuity, and cost control for profitability.
Discuss conflicts: low latency vs. model complexity/accuracy, high throughput vs. consistency, scalability vs. cost, and reliability vs. latency (e.g., retries).
Suggest techniques like model distillation, caching, approximate nearest neighbors, graceful degradation, and auto-scaling to balance competing NFRs.
Highlight how ML decisions (feature engineering, model selection, serving infrastructure) directly impact these NFRs and require cross-functional collaboration.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.