← Lila Sciences Interview Insights

Lila Sciences·Machine Learning Engineer·Onsite - System Design / Architecture·Senior

SeniorPrefer not to say
May 2026

Summary

System design round at Lila for an ML Engineer role. The whole thing was a single deep-dive question about using LLMs with reinforcement learning to optimize DNA sequences. Dense topic, and I felt like I was holding on for dear life through some parts of it.

Questions Asked (1)

Q1

Design a system that uses an LLM combined with reinforcement learning to optimize DNA sequences toward a target objective like binding affinity or expression level. Walk through tokenization, model choice, reward design, the RL loop, evaluation metrics, and how you'd prevent reward hacking.

System DesignTechnical Trade-offsAlgorithms & Data Structures
Author's notes

This question is basically six questions glued together.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by framing the problem as a sequential decision-making task where an LLM generates DNA sequences and RL fine-tunes them toward a target objective. Walk through each component systematically, emphasizing trade-offs and safeguards, and conclude with evaluation and anti-reward-hacking strategies.

Pro tip: Emphasize that reward hacking is inevitable in biological optimization, so design the reward to be robust and include multiple validation layers. Also, mention that using a pretrained genomic LLM (e.g., DNABERT, Nucleotide Transformer) can save compute and improve sample efficiency.

1. Tokenization and Model Choice

Discuss tokenization strategies for DNA (e.g., k-mers, BPE, or single nucleotides) and choose an LLM pretrained on genomic data. Consider trade-offs between sequence length, vocabulary size, and model capacity.

2. Reward Design

Define a reward function that combines the target objective (e.g., predicted binding affinity from a surrogate model) with penalties for undesired properties (e.g., off-target effects, sequence validity). Use a surrogate model trained on experimental data.

3. RL Loop

Outline the RL loop: the LLM acts as the policy, generating sequences; the reward model scores them; and an RL algorithm (e.g., PPO, REINFORCE) updates the policy. Discuss exploration strategies and handling of sparse rewards.

4. Evaluation Metrics

Specify metrics such as predicted binding affinity, expression level, sequence diversity, and validity. Include wet-lab validation as the ultimate metric and discuss how to measure generalization.

5. Preventing Reward Hacking

Describe techniques like reward shaping, adversarial validation, using an ensemble of reward models, and incorporating uncertainty estimates. Also, mention the importance of periodic retraining with new experimental data.

Key Points to Mention

  • Tokenization: k-mers vs. BPE vs. single nucleotides; impact on model performance and sequence length.
  • Model choice: pretrained genomic LLMs (e.g., DNABERT, Nucleotide Transformer) and fine-tuning with RL.
  • Reward design: surrogate model for binding affinity, penalties for off-target effects, and multi-objective optimization.
  • RL algorithm: PPO or REINFORCE with KL penalty to prevent divergence from pretrained model.
  • Evaluation: in silico metrics (predicted affinity, diversity) and wet-lab validation; use of held-out test sets.
  • Reward hacking prevention: ensemble reward models, uncertainty quantification, adversarial examples, and iterative retraining.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.