← Meta Interview InsightsMachine Learning Engineer
This is the kind of question where I started confident and slowly realized I was glossing over the hard parts.
Frame your answer around three distinct phases: feature engineering (defining the signal and its variants), online serving (how to compute and cache the feature at low latency), and training integrity (preventing label leakage with strict temporal splits). Ground every decision in real-world constraints like latency budgets, data freshness, and storage costs that Meta-scale systems face.
Pro tip: Explicitly distinguish between author-level and post-level engagement signals and discuss how you'd handle cold-start (new authors or posts with no history) — interviewers at Meta love seeing candidates think about edge cases and graceful degradation rather than just the happy path.
Identify the two distinct signals: (a) user–author affinity — did the user like posts from this author in the last 6 months, and (b) user–post affinity — did the user previously interact with this specific post. Decide on raw counts, binary flags, and normalized/decayed versions (e.g., exponential time-decay to weight recent likes more heavily).
Precompute user–author like counts over a rolling 6-month window and store them in a low-latency key-value store (e.g., ZippyDB or a Redis-like cache) keyed by (user_id, author_id). For the post-level signal, maintain a user–post interaction log and look up at serving time; consider bucketing counts (0, 1–5, 6–20, 20+) to reduce cardinality and improve generalization.
At inference time, perform a point lookup in the precomputed store using the (user_id, author_id) or (user_id, post_id) key, targeting sub-10ms p99 latency. Discuss a fallback strategy for cache misses — either a default value (0 or global mean) or a lightweight real-time aggregation from a stream processor like Flink.
Use strict temporal splits: the feature value must be computed using only interactions that occurred before the label timestamp (i.e., the moment the post was shown). Implement point-in-time correct feature generation by replaying the historical interaction log up to each training example's request_time, ensuring the 6-month window is anchored to that moment and not to the dataset creation date.
Measure offline AUC lift from adding the feature and run an ablation to confirm it adds signal beyond existing engagement features. Post-launch, monitor feature coverage (% of requests with a non-null value), distribution drift over time, and whether the feature's SHAP importance remains stable — a sudden spike could indicate a data pipeline bug or leakage.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Spent probably too long on the embedding angle when the interviewer seemed more interested in the interaction feature idea.
Frame your answer around a principled decision tree: start by understanding the feature's distribution and relationship to the target, then choose the integration strategy (raw, embedded, or interaction) based on that analysis. Explicitly address the risks to existing features like multicollinearity, feature importance dilution, and training instability, showing you think holistically about the model ecosystem rather than just the new feature in isolation.
Pro tip: At Meta scale, mention shadow scoring or a staged rollout where the new feature is logged and evaluated offline before being introduced to the live model — this signals you understand production ML risk management and not just modeling theory.
Analyze the feature's distribution (skewness, sparsity, cardinality), its correlation with the target, and its relationship to existing features. This diagnostic step determines which integration strategy is appropriate and surfaces potential risks upfront.
Decide between raw numeric (for dense, well-scaled continuous features), embedding (for high-cardinality categoricals or sparse IDs), or interaction terms (when the feature's signal is conditional on another feature). Justify the choice with the feature's statistical properties and the model architecture in use.
Evaluate multicollinearity with existing features, potential for feature importance redistribution, and whether the new feature could dominate gradients and destabilize training. Run a correlation matrix and variance inflation factor (VIF) analysis as a baseline check.
Run controlled offline experiments comparing model performance with and without the new feature using held-out data, checking both aggregate metrics (AUC, NDCG) and slice-level performance to ensure no regression on key user segments.
Deploy via shadow scoring or a small A/B test to monitor live metric impact, feature value distributions in production, and any training-serving skew before full rollout. Define clear rollback criteria in advance.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Start by anchoring the answer to the product goal the feature is meant to achieve, then map that goal to measurable offline proxy metrics before discussing how to slice the analysis for robustness. Demonstrate that you understand the gap between offline metrics and real-world impact, and show awareness of potential pitfalls like metric gaming or Simpson's paradox in slicing.
Pro tip: At Meta scale, interviewers love when candidates proactively mention that a single aggregate metric can mask harm to specific user cohorts — explicitly calling out fairness slices (e.g., new vs. returning users, mobile vs. desktop, geographic regions) signals senior-level thinking and awareness of responsible ML practices.
Restate what the feature is trying to optimize — engagement, relevance, retention, or something else — so your metric choices are clearly motivated. This shows structured thinking and avoids proposing metrics in a vacuum.
Identify the core offline metrics that serve as proxies for the business goal, such as AUC-ROC, NDCG, precision@k, or calibration error depending on the task type. Explain why each metric is a valid proxy and acknowledge its limitations.
Propose guardrail metrics to ensure the feature doesn't degrade other important dimensions — for example, latency, coverage, or diversity — even if the primary metric improves. This demonstrates awareness that optimizing one metric can harm others.
Describe how you would segment the analysis across meaningful dimensions: user cohorts (new vs. power users), content types, device/platform, geographic regions, and time windows. Explain that slicing catches cases where aggregate gains hide losses for specific subgroups.
Discuss how you would sanity-check that your offline metrics historically correlate with online A/B test outcomes, and outline the plan to move from offline validation to a live experiment. Mention shadow scoring or replay evaluation as intermediate steps if applicable.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Structure your answer by first acknowledging that offline-online metric gaps are common and expected, then systematically walk through the possible root causes across data, model, and experimentation dimensions. Demonstrate a methodical debugging process that shows both technical depth and product intuition, concluding with how you'd prioritize investigations and make a ship/no-ship decision.
Pro tip: Mentioning 'surrogate metric misalignment' — where your offline metric (e.g., AUC, NDCG) doesn't faithfully proxy the true business metric (e.g., long-term engagement, revenue) — signals senior-level thinking and is a very common real-world culprit at companies like Meta.
First rule out instrumentation and experimental design issues: check for sample ratio mismatch (SRM), ensure proper randomization, verify that logging is consistent between control and treatment, and confirm the experiment ran long enough to avoid novelty effects or insufficient power.
Investigate whether the offline metric is a faithful proxy for the online business metric — misalignment here is a primary culprit. Check if the offline evaluation dataset is representative of live traffic distribution, and whether label quality or feedback loops differ between offline and online settings.
Confirm the model is actually being served correctly by checking prediction distributions, feature value distributions, and model version in production logs. Look for training-serving skew, where features computed offline differ from how they're computed at serving time.
Break down the online results by user segments, surfaces, or content types to see if gains exist in subpopulations but are diluted in aggregate. Also check if the primary metric is moving but secondary or guardrail metrics are offsetting it, masking real impact.
Summarize the most likely root cause with supporting evidence and propose a concrete action: fix the offline metric, iterate on the model, extend the experiment, or redesign the evaluation protocol. Communicate the tradeoffs clearly to stakeholders before making a ship/no-ship recommendation.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.