← American Express Interview Insights

American Express·AI Engineer·Technical Phone Screen·Senior

Senior
May 2026

Summary

Technical screen for an AI Engineer role at Amex, pretty much one big conceptual question about how LLMs, RAG, and agents relate to each other. Felt like a solid conversation but also the kind of question where you can ramble forever and not actually say anything useful.

Questions Asked (1)

Q1

Walk through the differences between a base LLM, a RAG system, and an autonomous agent. How do they relate to each other, when would you use each, and how do they compose?

System DesignTechnical Trade-offsAdaptability & Ambiguity
Author's notes

This question looks clean on paper but it's actually easy to fumble because the three concepts bleed into each other and you can lose the thread fast.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by defining each architecture in terms of its core components and control flow, then contrast them along axes like complexity, cost, latency, and reliability. Use a concrete example (e.g., a customer support assistant) to illustrate when each is appropriate and how they can be layered together.

Pro tip: Emphasize that these are not mutually exclusive; in production, you often start with a base LLM, add RAG for grounding, and then introduce agentic loops only where dynamic tool use is essential. This shows you understand incremental complexity and risk management.

1. Define each architecture

Briefly explain what a base LLM, RAG system, and autonomous agent are, focusing on their core components and how they process input to output.

2. Compare along key dimensions

Contrast them on dimensions such as knowledge freshness, factual accuracy, cost, latency, complexity, and reliability to highlight trade-offs.

3. Map to use cases

Give examples of when to use each: base LLM for creative tasks, RAG for knowledge-intensive Q&A, and agents for multi-step tasks requiring tool use.

4. Explain composition

Describe how they can be combined: e.g., an agent can use RAG as a tool, or a RAG system can be built on top of a base LLM with fine-tuning.

5. Tie to business context

Relate to American Express by discussing how these architectures could support customer service, fraud detection, or personalized recommendations, emphasizing reliability and compliance.

Key Points to Mention

  • Base LLM: pre-trained, no external knowledge, may hallucinate, best for general language tasks.
  • RAG: augments LLM with retrieved documents, improves factual accuracy and freshness, but adds retrieval latency and complexity.
  • Autonomous agent: uses LLM for planning and decision-making, can call tools/APIs, handle multi-step tasks, but risks error propagation and higher cost.
  • Composition: agents can use RAG as a tool; RAG can be built on fine-tuned LLMs; base LLM is the foundation for both.
  • Trade-offs: RAG reduces hallucination but not all; agents offer flexibility but need guardrails and monitoring.
  • American Express context: focus on reliability, compliance, and customer experience when choosing architecture.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.