← Sesame Interview Insights

Sesame·Software Engineer·Onsite - System Design / Architecture·Senior

Senior
Apr 2026

Summary

System design round at Sesame for a software engineer role. The whole thing was one big question about building a voice chatbot from scratch, and it went pretty deep into areas I hadn't fully prepped for.

Questions Asked (1)

Q1

Design a voice chatbot end-to-end, covering speech-to-text, intent and dialog understanding, state management, response generation, text-to-speech, and audio streaming back to the user. Also address latency budgets per stage, barge-in and interruption handling, scaling for concurrent calls, and observability.

System DesignTechnical Trade-offsAPI & Integrations
Author's notes

This is one of those questions where you think you know the space until you actually have to draw it out.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying requirements and constraints (e.g., expected concurrency, latency targets, supported languages). Then walk through the end-to-end pipeline stage by stage, highlighting key design decisions, trade-offs, and how you'd handle latency, interruptions, scaling, and observability. Finally, summarize with a cohesive architecture and discuss potential bottlenecks or future improvements.

Pro tip: Emphasize the importance of end-to-end latency budgeting and how you'd measure and optimize each stage; real-world voice bots live or die by responsiveness, so showing you can reason about milliseconds and user-perceived latency will set you apart.

1. Clarify Requirements and Scope

Ask about expected scale (concurrent calls), latency targets, languages, use cases, and any constraints (e.g., on-prem vs cloud). This ensures your design is tailored and shows you think before diving in.

2. High-Level Architecture

Sketch the main components: audio ingestion, STT, NLU/dialog management, response generation, TTS, and audio streaming back. Explain how they interact and the data flow.

3. Deep Dive into Critical Stages

For each stage, discuss technology choices (e.g., streaming STT, intent classifiers, state management with Redis, streaming TTS), latency budgets, and trade-offs (accuracy vs speed, cost vs performance).

4. Handle Interruptions and Barge-in

Describe how to detect user speech during bot output, cancel ongoing TTS/audio streaming, and update dialog state. Mention techniques like VAD, echo cancellation, and state rollback.

5. Scaling and Observability

Explain how to scale each component (e.g., horizontal scaling, load balancing, async processing) and what metrics/logs/traces to collect (latency per stage, error rates, concurrency, user satisfaction).

Key Points to Mention

  • Latency budgets per stage (e.g., STT <300ms, NLU <100ms, TTS <200ms) and end-to-end target (<1s).
  • Streaming vs batch processing for STT and TTS to minimize latency.
  • State management: session state, dialog history, and context persistence (e.g., Redis, DynamoDB).
  • Barge-in handling: voice activity detection (VAD), echo cancellation, and cancellation of ongoing TTS/audio.
  • Scaling strategies: stateless services, horizontal scaling, message queues, and load balancing for concurrent calls.
  • Observability: distributed tracing, per-stage latency metrics, error logging, and user feedback loops.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.