← American Express Interview Insights

American Express·AI Engineer·Technical Phone Screen·Senior

Senior
Mar 2026

Summary

Technical interview at American Express for an AI Engineer role, focused almost entirely on how agents extend LLM capabilities through tooling. Pretty deep session, they clearly wanted someone who'd actually built this stuff rather than just read about it.

Questions Asked (7)

Q1

How does tool use and function calling work in AI agents? Walk through schema design, how the agent routes to the right tool, and how errors get handled.

System DesignTechnical Trade-offsAPI & Integrations
Author's notes

I started with schema design and talked about strongly typed inputs, required vs optional fields, and keeping descriptions tight so the model doesn't hallucinate parameters.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Structure your answer around the three pillars the interviewer asked about: schema design, routing, and error handling. For each, explain the core mechanism, then discuss trade-offs and how you would implement it in a production system, ideally with a concrete example like a financial assistant. Emphasize reliability, observability, and security, which are critical in a regulated environment like American Express.

Pro tip: Mention that tool schemas should be treated as contracts with versioning and backward compatibility, and that error handling must include idempotency and retries with exponential backoff to avoid duplicate transactions—this shows you understand production-grade AI systems in finance.

1. Define tool schemas as structured contracts

Explain that each tool is described by a JSON Schema (or similar) specifying name, description, parameters, types, and required fields. Emphasize clear, unambiguous descriptions and strict typing to guide the model.

2. Route user intent to the right tool

Describe how the agent uses the LLM to parse the user's request and select a tool, often via function calling or a router prompt. Mention techniques like embedding-based retrieval for large tool sets and fallback to clarification when confidence is low.

3. Execute and validate tool calls

Cover the execution step: the agent generates a structured call, which is validated against the schema before invocation. Discuss sandboxing, permission checks, and logging for auditability.

4. Handle errors gracefully

Explain error categories (validation, execution, timeout) and strategies: retries with backoff, fallback tools, user clarification, and circuit breakers. Stress idempotency and transactional integrity for financial operations.

5. Monitor, evaluate, and iterate

Highlight the need for observability: logging tool calls, success rates, latency, and error patterns. Use this data to refine schemas, routing prompts, and error handling, and to detect drift or misuse.

Key Points to Mention

  • JSON Schema for tool definitions with clear descriptions and strict types
  • Function calling vs. prompt-based routing and when to use each
  • Handling ambiguous user requests with clarification or disambiguation
  • Error handling: retries, idempotency, fallbacks, and user-friendly messages
  • Security: input validation, permission scopes, and audit logging
  • Observability: tracing tool calls, metrics, and continuous evaluation

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q2

How would you design a plug-in or MCP-style integration layer for an agent system?

System DesignAPI & IntegrationsTechnical Trade-offs
Author's notes

Talked about standardized interfaces, manifest-style declarations for what each plugin exposes, and versioning.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the requirements and constraints of the agent system, then propose a modular, secure, and scalable integration layer inspired by MCP and plug-in architectures. Walk through the design from interface definition to runtime execution, emphasizing trade-offs and alignment with American Express's regulatory and security needs.

Pro tip: Highlight how you would enforce security and compliance (e.g., OAuth, audit logs) without sacrificing extensibility, and mention the importance of versioning and backward compatibility to avoid breaking existing integrations.

1. Clarify Requirements and Constraints

Ask about the agent system's architecture, expected scale, latency requirements, and security/compliance needs (e.g., PCI DSS, GDPR). This ensures the design is grounded in real constraints.

2. Define the Integration Contract

Specify a standard interface (e.g., OpenAPI, gRPC, or MCP-style manifest) that plug-ins must implement, including authentication, input/output schemas, and error handling. This promotes consistency and interoperability.

3. Design the Runtime and Lifecycle Management

Describe how plug-ins are discovered, loaded, executed, and monitored. Consider isolation (e.g., containers, sandboxes), resource limits, and hot-swapping for zero-downtime updates.

4. Address Security, Governance, and Observability

Explain how you would enforce authentication/authorization, audit logging, rate limiting, and compliance checks. Include monitoring for performance and failures.

5. Discuss Trade-offs and Extensibility

Compare design choices (e.g., centralized vs. decentralized registry, synchronous vs. asynchronous invocation) and explain how the design supports future extensions without major refactoring.

Key Points to Mention

  • Standardized interface (e.g., MCP manifest, OpenAPI) for plug-in discovery and invocation
  • Security mechanisms: OAuth 2.0, mTLS, RBAC, audit trails, and data encryption
  • Isolation and resource management: sandboxing, containers, and quotas to prevent interference
  • Versioning and backward compatibility to manage plug-in evolution
  • Observability: logging, metrics, tracing, and health checks for plug-ins
  • Scalability and performance considerations: caching, async processing, and load balancing

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q3

When does it make sense to give an agent the ability to write and execute code as a tool?

Technical Trade-offsSystem Design
Author's notes

This one I actually liked.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by framing the decision as a trade-off between flexibility and risk, then outline the specific conditions where code execution adds value (e.g., dynamic data manipulation, complex calculations, or interacting with APIs). Emphasize that you would only grant this capability when the benefits outweigh the risks, and describe how to mitigate those risks through sandboxing, monitoring, and human oversight.

Pro tip: Mention that in regulated industries like finance, you must also consider compliance and auditability—so you'd log all code executions and ensure they are reviewable. This shows you understand the broader enterprise context.

1. Identify the need

Determine if the task requires dynamic, open-ended computation that cannot be pre-defined with static tools. Examples include data analysis, generating visualizations, or interacting with external systems.

2. Assess the risks

Evaluate potential security, safety, and reliability risks such as code injection, resource exhaustion, or unintended side effects. Consider the agent's autonomy and the sensitivity of the environment.

3. Define constraints and safeguards

Specify how you would sandbox execution (e.g., isolated containers, limited permissions), monitor behavior, and enforce time/resource limits. Include human-in-the-loop for high-risk actions.

4. Evaluate trade-offs

Weigh the benefits (flexibility, reduced engineering effort) against costs (latency, complexity, security overhead). Decide if the use case justifies the added capability.

5. Decide and iterate

Make a recommendation based on the analysis, and propose a phased rollout with monitoring to validate safety and effectiveness before scaling.

Key Points to Mention

  • Dynamic task requirements that cannot be anticipated at design time
  • Security risks like arbitrary code execution and data exfiltration
  • Sandboxing techniques (e.g., Docker, gVisor, WebAssembly) and permission scoping
  • Resource limits and monitoring to prevent abuse or runaway processes
  • Compliance and auditability requirements in regulated industries
  • Alternative approaches (e.g., pre-built tools, API integrations) and when they suffice

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q4

How does retrieval fit into the capability stack of an agent, and how is it different from just using a bigger context window?

System DesignTechnical Trade-offs
Author's notes

Made the case that retrieval is about precision and cost, not just fitting more text in.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by defining retrieval as a modular component that augments an agent's knowledge and reasoning, then contrast it with simply increasing context window size. Emphasize that retrieval offers dynamic, up-to-date, and cost-effective access to information, while larger contexts are static, expensive, and limited by model constraints. Conclude by discussing trade-offs and when each approach is appropriate.

Pro tip: Frame retrieval as a tool in the agent's toolkit that enables selective attention and reduces hallucination, and mention that at scale, retrieval often outperforms larger contexts in both cost and accuracy—showing you understand production constraints.

1. Define the capability stack

Outline the layers of an agent's capabilities: perception, memory, reasoning, action, and knowledge. Position retrieval as part of the knowledge/memory layer that feeds into reasoning.

2. Explain retrieval's role

Describe how retrieval dynamically fetches relevant information from external sources (e.g., vector databases, APIs) to ground the agent's responses, enabling up-to-date and domain-specific knowledge.

3. Contrast with larger context windows

Highlight that larger context windows are static, limited by token limits, and incur high computational cost and latency. Retrieval is dynamic, scalable, and cost-efficient, allowing access to vast knowledge bases.

4. Discuss trade-offs and integration

Acknowledge that retrieval adds complexity (e.g., indexing, latency) but can be combined with context windows for hybrid approaches. Emphasize that retrieval enables selective focus and reduces noise.

5. Relate to real-world scenarios

Give examples like customer support agents needing real-time policy updates, where retrieval ensures accuracy without retraining or massive contexts.

Key Points to Mention

  • Retrieval augments agent memory and knowledge, enabling dynamic access to external data.
  • Larger context windows are limited by token limits, cost, and latency, and cannot scale to web-scale knowledge.
  • Retrieval reduces hallucination by grounding responses in retrieved evidence.
  • Retrieval allows for up-to-date information without model retraining.
  • Hybrid approaches can combine retrieval with context windows for best results.
  • Retrieval introduces challenges like indexing, relevance ranking, and latency that must be managed.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q5

Explain the planner/executor pattern in agentic systems and when you'd use it.

System DesignAdaptability & Ambiguity
Author's notes

Described a two-stage setup where one component decomposes a goal into subtasks and another executes them, possibly in parallel or conditionally.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Define the planner/executor pattern as a separation of concerns where a planner decomposes a high-level goal into a sequence of steps, and an executor carries out each step, often with feedback loops. Then explain when to use it: complex, multi-step tasks requiring dynamic adaptation, and contrast with simpler reactive agents. Finally, tie it to American Express use cases like fraud detection or customer service automation.

Pro tip: Emphasize that the pattern shines when tasks are too complex for a single LLM call but too dynamic for a fixed workflow, and mention how you'd handle failure modes like planner errors or executor drift.

1. Define the pattern

Explain that the planner breaks down a goal into sub-tasks, and the executor performs each sub-task, often using tools or APIs. Highlight the iterative loop where the planner can re-plan based on executor feedback.

2. Contrast with alternatives

Compare to monolithic LLM calls (limited by context and reasoning) and fixed workflows (inflexible). Show when planner/executor is preferable: tasks needing dynamic decomposition and adaptation.

3. Describe when to use it

List scenarios: multi-step reasoning, tool use, long-horizon tasks, and environments with uncertainty. Give examples like travel booking, fraud investigation, or customer support ticket resolution.

4. Discuss implementation considerations

Mention design choices: planner granularity, executor capabilities (tools, APIs), feedback mechanisms, and error handling. Address trade-offs like latency, cost, and complexity.

5. Relate to American Express

Connect to AmEx context: e.g., automating dispute resolution, personalized financial advice, or real-time fraud detection where multi-step reasoning and tool use are essential.

Key Points to Mention

  • Separation of concerns: planning vs. execution
  • Iterative feedback loop between planner and executor
  • Suitable for complex, multi-step tasks with uncertainty
  • Contrast with single-agent and fixed-workflow approaches
  • Implementation: tool use, memory, error recovery
  • Use cases in finance: fraud detection, customer service, risk assessment

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q6

How does an agent discover and select which tools to use at runtime, especially when the tool set is large?

System DesignAlgorithms & Data Structures
Author's notes

Blanked for a second on this one.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by framing the problem as a retrieval and ranking challenge: the agent must efficiently narrow a large tool set to a few relevant candidates, then select the best one(s) based on context and constraints. Describe a layered architecture that combines semantic search, metadata filtering, and learned policies, and discuss trade-offs between accuracy, latency, and cost.

Pro tip: Emphasize that tool discovery should be dynamic and context-aware, not a static list; mention that you would evaluate the selection mechanism with offline metrics (e.g., recall@k) and online A/B tests to ensure it improves task success without excessive latency.

1. Clarify requirements and constraints

Ask about the scale of the tool set, latency budgets, and whether tools have structured metadata (e.g., descriptions, parameters, cost). This shapes the discovery and selection strategy.

2. Index tools for efficient retrieval

Embed tool descriptions and metadata into a vector store, and maintain a keyword index for hybrid search. This enables fast candidate generation even with thousands of tools.

3. Retrieve and filter candidates

Use the agent's current goal and context to query the index, applying hard filters (e.g., permissions, cost) and soft ranking (e.g., semantic similarity) to produce a shortlist of relevant tools.

4. Select and invoke tools

Use a learned policy or LLM-based reasoning to pick the best tool(s) from the shortlist, considering past success, tool compatibility, and execution order. Optionally, allow the agent to compose multiple tools.

5. Monitor and improve

Log tool selection decisions and outcomes, then use feedback to fine-tune retrieval and ranking models. Continuously update the tool index as new tools are added.

Key Points to Mention

  • Semantic search with embeddings for tool descriptions
  • Hybrid retrieval combining dense and sparse (keyword) methods
  • Metadata filtering (e.g., permissions, cost, latency) to prune candidates
  • Learned ranking or LLM-based selection from a shortlist
  • Caching and pre-computation to reduce latency
  • Evaluation metrics: recall@k, task success rate, and cost per task

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q7

How do you decide whether a new capability should be added as an external tool versus baked into the model through fine-tuning or training?

Technical Trade-offsProduct StrategySystem Design
Author's notes

This was the most interesting question of the session.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by framing the decision as a trade-off between cost, latency, flexibility, and control, then walk through a structured evaluation process. Emphasize that in most enterprise settings, external tools are preferred for dynamic or frequently changing capabilities, while fine-tuning is reserved for stable, high-volume tasks where latency and cost savings justify the investment.

Pro tip: Always quantify the trade-offs: estimate the cost of fine-tuning (data labeling, compute, maintenance) versus the cost of tool calls (API fees, latency, error handling) and show that you consider total cost of ownership. Mention that at American Express, regulatory and audit requirements often favor external tools for traceability and rapid updates.

1. Define the capability and its requirements

Clearly specify what the new capability is, how often it will be used, and its performance requirements (e.g., latency, accuracy, explainability).

2. Assess data availability and stability

Determine if you have enough high-quality, labeled data for fine-tuning and whether the task is stable over time. If data is scarce or the task changes frequently, an external tool is more suitable.

3. Evaluate cost and latency trade-offs

Compare the total cost of ownership: fine-tuning requires upfront data labeling, compute, and ongoing maintenance, while external tools incur per-call costs and network latency. Choose based on expected usage volume and latency sensitivity.

4. Consider control, compliance, and maintainability

External tools offer easier updates, auditability, and separation of concerns, which are critical in regulated industries. Fine-tuning gives more control but can be harder to update and audit.

5. Prototype and measure

When uncertain, build a quick prototype for both approaches and measure performance, cost, and developer velocity. Let empirical data drive the final decision.

Key Points to Mention

  • Trade-off between flexibility and performance: external tools are easier to update but may add latency; fine-tuning can reduce latency but is rigid.
  • Data requirements: fine-tuning needs large, high-quality labeled datasets, while external tools can work with zero or few examples.
  • Cost considerations: fine-tuning has high upfront and maintenance costs; external tools have variable per-call costs.
  • Latency and throughput: fine-tuning can be faster at inference if the model is smaller, but external tools may introduce network overhead.
  • Compliance and auditability: external tools provide clear separation and logging, which is important for regulated industries like finance.
  • Hybrid approaches: sometimes a combination works best, e.g., fine-tune for common cases and fall back to tools for edge cases.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.