← Expedia Interview Insights

Expedia·Machine Learning Engineer·Technical Phone Screen·Intermediate

Intermediate
May 2026

Summary

First round for an MLE role at Expedia, basically a notebook-based case study centered on LLM fundamentals. The framework was pre-built so you're just filling in the gaps, but those gaps are non-trivial if you haven't thought about tokenization in a while.

Questions Asked (2)

Q1

Implement a tokenizer from scratch inside a provided notebook, including word-to-token and token-to-word conversion mappings.

Algorithms & Data StructuresTechnical Trade-offs
Author's notes

The framework was already there which sounds like a relief until you realize it just narrows what you can hide behind.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the tokenization strategy (e.g., word-level, subword) and the expected handling of unknown words. Then implement a class with fit, encode, and decode methods, ensuring the vocabulary includes special tokens like <unk> and <pad>. Finally, test with sample text to verify round-trip conversion.

Pro tip: Mention that in production, you'd use a battle-tested tokenizer like Hugging Face's, but implementing from scratch demonstrates understanding of the underlying mechanics and helps debug issues.

1. Clarify requirements

Ask about the tokenization level (word, character, subword), handling of unknown words, and whether special tokens are needed. Confirm the expected input/output format.

2. Design the tokenizer class

Outline a class with methods: fit (build vocabulary), encode (text to token IDs), and decode (token IDs to text). Include mappings word2idx and idx2word.

3. Implement vocabulary building

In fit, tokenize the corpus, count frequencies, and create the vocabulary. Add special tokens like <pad> and <unk> with reserved indices.

4. Implement encode and decode

In encode, split text into tokens, map each to its ID (or <unk> if unseen). In decode, map IDs back to tokens and join them.

5. Test and validate

Run a sample sentence through encode and decode to ensure round-trip consistency. Check edge cases like empty strings and unknown words.

Key Points to Mention

  • Choice of tokenization granularity (word-level vs. subword) and its trade-offs
  • Handling of out-of-vocabulary words with an <unk> token
  • Inclusion of special tokens (<pad>, <unk>, <bos>, <eos>) and their reserved indices
  • Efficiency considerations: using dictionaries for O(1) lookups and handling large vocabularies
  • Round-trip consistency: ensuring decode(encode(text)) recovers the original text (except for unknown words)
  • Potential for extending to subword tokenization (e.g., BPE) if needed

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q2

Write an optimizer and a loss function for training, then verify the loss decreases over iterations.

Technical Trade-offsAlgorithms & Data Structures
Author's notes

Pretty mechanical once you're in it.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the problem setup: choose a simple model (e.g., linear regression) and a standard loss (e.g., MSE). Implement the loss and its gradient, select an optimizer (e.g., SGD or Adam), and run a training loop while logging loss values. Finally, verify that the loss decreases over iterations and discuss potential issues like learning rate selection.

Pro tip: Mention that you would first test with a tiny dataset and a simple model to ensure the loss decreases, then scale up. Also, highlight the importance of monitoring loss curves and being ready to adjust hyperparameters like learning rate if the loss doesn't decrease.

1. Clarify the problem and choose components

Define the model, loss function, and optimizer. For simplicity, suggest linear regression with MSE loss and SGD optimizer, but mention that the choice depends on the task.

2. Implement loss and gradient

Write the loss function and its gradient with respect to model parameters. Ensure numerical stability and vectorization for efficiency.

3. Set up optimizer and training loop

Initialize parameters, choose learning rate, and implement the update rule. Run the training loop for a fixed number of iterations, recording loss at each step.

4. Verify loss decreases

Plot or print the loss over iterations. Check for a decreasing trend. If not, debug by checking gradients, learning rate, or data preprocessing.

5. Discuss trade-offs and improvements

Talk about optimizer choices (SGD vs Adam), loss function alternatives, and techniques like learning rate scheduling or early stopping.

Key Points to Mention

  • Choice of loss function (e.g., MSE for regression, cross-entropy for classification) and its properties.
  • Optimizer selection (SGD, Adam, etc.) and hyperparameters like learning rate, momentum.
  • Gradient computation (analytical or automatic differentiation) and checking for correctness.
  • Training loop structure: forward pass, loss computation, backward pass, parameter update.
  • Monitoring loss over iterations and interpreting the curve (e.g., convergence, oscillation).
  • Common pitfalls: exploding/vanishing gradients, poor initialization, inappropriate learning rate.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.