← American Express Interview Insights

American Express·AI Engineer·Onsite - System Design / Architecture·Senior

Senior
Jun 2026

Summary

30-minute system design round at Amex for an AI Engineer role. The whole thing was a single RAG design case, which sounds focused until you realize how many layers they actually want you to cover in half an hour.

Questions Asked (1)

Q1

Design a Retrieval-Augmented Generation (RAG) system over a corporate document corpus. Walk through ingestion, retrieval, generation, evaluation, and operational concerns.

System DesignTechnical Trade-offsAdaptability & Ambiguity
Author's notes

They gave 30 minutes for this and I spent way too long on chunking strategy upfront.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying requirements and constraints (e.g., document types, scale, latency, compliance) to show adaptability. Then walk through the RAG pipeline stages—ingestion, retrieval, generation, evaluation, and operations—highlighting trade-offs and design choices at each stage. Emphasize how your design addresses American Express’s needs for accuracy, security, and scalability.

Pro tip: Anchor your answer in a concrete example or two (e.g., handling PDFs with tables, or mitigating hallucinations in financial Q&A) to demonstrate practical experience and maturity. Also, proactively mention compliance and data privacy considerations, which are critical in financial services.

1. Clarify Requirements and Constraints

Ask about document types, corpus size, update frequency, latency/throughput needs, and compliance requirements. This shows you can adapt to ambiguity and tailor the design.

2. Design Ingestion Pipeline

Cover document parsing (e.g., PDF, HTML), chunking strategies (fixed-size, semantic), embedding model selection, and metadata extraction. Discuss handling of tables, images, and incremental updates.

3. Design Retrieval and Generation

Explain hybrid retrieval (dense + sparse), re-ranking, and query understanding. For generation, discuss prompt engineering, context window management, and model choice (e.g., fine-tuned vs. API-based).

4. Plan Evaluation and Monitoring

Define offline metrics (retrieval recall, answer faithfulness) and online metrics (user feedback, latency). Describe A/B testing and continuous evaluation to catch drift.

5. Address Operational Concerns

Discuss scalability (vector DB sharding, caching), cost management, security (access control, encryption), and compliance (audit logs, data retention). Mention CI/CD for model updates.

Key Points to Mention

  • Chunking strategies and their impact on retrieval quality
  • Hybrid retrieval combining dense and sparse methods
  • Re-ranking to improve precision
  • Hallucination mitigation via grounding and citation
  • Evaluation metrics: retrieval recall, answer faithfulness, latency
  • Security and compliance: access control, encryption, audit logs

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.