Credit Karma·Machine Learning Engineer·Onsite - System Design / Architecture
- How would you design an ML system for Credit Karma's marketplace to maximize revenue?
- Should you use a single model across the entire funnel or separate models for each stage like click, application, and approval?
- How do you handle conversion labels that don't arrive until days or weeks after the original impression?
- Conversion rates in this setting can be extremely low, around 0.1 to 0.5 percent. How do you deal with that kind of label sparsity?
- If you use negative sampling during training, how do you correct for that bias at inference time?
- You're training on multiple objectives like CTR, conversion, and approval rate. How do you combine these into a single ranking score at the end?
- What is the actual objective function you'd use for the final ranking step?
- When does it make sense to use a two-stage retrieval plus reranking setup versus just scoring all candidates directly?
- Walk through the pros and cons of two-stage versus single-stage ranking architectures.
- Under what conditions would you choose a two-stage architecture over a single-stage one?
“This was the anchor question that everything else branched off of.”