← Lila Sciences Interview Insights

Lila Sciences·Software Engineer·Technical Phone Screen·Senior

Senior
Apr 2026

Summary

Technical screen for a research engineer role at Lila Sciences focused almost entirely on LLM-driven scientific automation. The questions were dense and pretty niche, definitely not a standard ML interview.

Questions Asked (2)

Q1

How would you use LLMs to drive automation in experimental or atomic-scale manipulation tasks, and how do you address the fact that LLMs have no real physical intuition?

System DesignTechnical Trade-offsAdaptability & Ambiguity
Author's notes

This is where I got a bit turned around.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Frame LLMs as high-level planners and interfaces that translate natural language goals into structured action sequences, while relying on physics simulators and real-time feedback for grounding. Emphasize a closed-loop system where LLMs propose, simulators verify, and robotic controllers execute, with human oversight for safety and ambiguity.

Pro tip: Acknowledge that physical intuition is not the LLM's job—its strength is rapid hypothesis generation and tool orchestration. Show you'd measure success by task success rate and sample efficiency, not by expecting the LLM to 'understand' physics.

1. Define the task and constraints

Clarify the experimental goal, available instruments (e.g., STM, optical tweezers), and safety limits. Identify what decisions can be automated versus what requires human judgment.

2. Design the LLM's role as a planner

Use the LLM to parse high-level objectives into subgoals, select tools, and generate candidate action sequences. Constrain outputs to a formal action language or API to avoid unsafe or invalid commands.

3. Ground with simulation and feedback

Integrate a physics simulator or digital twin to validate proposed actions before execution. Use real-time sensor data to detect deviations and trigger replanning or human intervention.

4. Implement closed-loop control with human oversight

Combine LLM planning with low-level controllers and safety checks. Allow humans to approve high-risk steps and provide corrections that feed back into the LLM's context.

5. Evaluate and iterate

Measure task success, efficiency, and safety. Use failures to refine prompts, add constraints, or improve the simulator, treating the LLM as one component in a larger system.

Key Points to Mention

  • LLMs as high-level planners, not low-level controllers
  • Tool use and API integration for atomic-scale instruments
  • Physics simulators and digital twins for grounding
  • Closed-loop feedback with real-time sensor data
  • Human-in-the-loop for safety and ambiguity
  • Sample efficiency and transfer learning from simulation to reality

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q2

How would you design a benchmark to evaluate whether an LLM agent reasons correctly about crystal structure operations, including logical rigor and physical validity?

A/B Testing & ExperimentationTechnical Trade-offsProduct Analytics & Metrics
Author's notes

Caught me a bit flat-footed because I hadn't thought much about evaluation design for this specific domain.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by defining the specific reasoning capabilities you want to measure, such as symmetry operations, lattice transformations, and space group logic. Then propose a benchmark that combines synthetic tasks with known ground truth, expert validation, and automated checks for logical consistency and physical plausibility. Emphasize iterative refinement and the use of metrics that balance correctness, rigor, and validity.

Pro tip: Include adversarial examples that are physically plausible but logically inconsistent to test whether the agent can detect subtle errors, and report performance on both seen and unseen crystal systems to demonstrate generalization.

1. Define scope and tasks

Identify the core crystal structure operations (e.g., symmetry detection, coordinate transformations, space group identification) and the reasoning types (deductive, spatial, causal) to evaluate.

2. Design task suite with ground truth

Create a diverse set of problems with verified correct answers, including synthetic and real-world cases, and ensure they cover varying complexity and edge cases.

3. Establish evaluation metrics

Define metrics for logical rigor (e.g., step-by-step validation, consistency checks) and physical validity (e.g., adherence to crystallographic rules, energy minimization).

4. Implement automated and expert evaluation

Combine automated solvers for objective checks with expert human review for nuanced validity, and use inter-rater reliability to ensure robustness.

5. Validate and iterate

Pilot the benchmark on baseline models, analyze failure modes, and refine tasks and metrics to improve discriminative power and fairness.

Key Points to Mention

  • Use of crystallographic databases (e.g., ICSD, COD) for realistic structures and ground truth.
  • Incorporation of symmetry operations and space group theory as core reasoning elements.
  • Design of metrics that separate logical correctness from physical plausibility.
  • Adversarial and counterfactual examples to test robustness and detect hallucinations.
  • Cross-validation with domain experts to ensure tasks are unambiguous and physically meaningful.
  • Reporting of both quantitative scores and qualitative error analysis for actionable insights.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.