← Mithril Interview Insights

Mithril·Software Engineer·Onsite - System Design / Architecture·Senior

SeniorPrefer not to say
May 2026

Summary

System design round at Mithril for a software engineer role. The prompt was a beast: design a GPU resource manager spanning on-prem and multiple public clouds, covering basically everything from telemetry to cost-aware scheduling. Big scope, not a lot of hand-holding.

Questions Asked (1)

Q1

Design a system that manages and allocates GPU resources across a hybrid cloud environment, covering on-prem infrastructure and multiple public cloud providers. Your design should address GPU inventory tracking and telemetry, multi-tenant scheduling with quota enforcement, job queueing and preemption, fair-share and priority-based scheduling, fault tolerance, and cost-aware placement decisions.

System DesignTechnical Trade-offsData Modeling
Author's notes

The scope of this thing is genuinely absurd.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying requirements and constraints, then propose a high-level architecture that separates control plane (scheduling, quota, inventory) from data plane (GPU nodes, telemetry agents). Walk through each required area—inventory, scheduling, fault tolerance, cost—explaining trade-offs and how components interact, and finish with a brief discussion of scaling and failure scenarios.

Pro tip: Emphasize that the scheduler must be pluggable and policy-driven, allowing different strategies (fair-share, priority, cost-aware) to be swapped without rearchitecting; this shows you understand extensibility and real-world evolution.

1. Clarify Requirements and Constraints

Ask about scale (number of GPUs, jobs, tenants), latency requirements, budget constraints, and whether preemption is allowed. Establish what 'hybrid cloud' means here (e.g., on-prem + AWS + GCP) and any compliance or data locality issues.

2. Design High-Level Architecture

Propose a control plane with a global scheduler, quota manager, and inventory service, and a data plane with agents on each GPU node reporting telemetry. Use a message queue for job submission and a database for state.

3. Detail Core Components

Explain how inventory tracking works (heartbeats, labels), how scheduling policies are enforced (fair-share, priority, preemption), and how cost-aware placement decides between on-prem and cloud based on price, availability, and data transfer costs.

4. Address Fault Tolerance and Scalability

Describe how the system handles node failures, network partitions, and scheduler outages. Discuss horizontal scaling of the control plane and use of leader election for high availability.

5. Discuss Trade-offs and Extensions

Summarize key trade-offs (e.g., centralized vs. decentralized scheduling, strict vs. soft quotas) and mention possible extensions like gang scheduling, GPU sharing, or spot instance integration.

Key Points to Mention

  • GPU inventory tracking via periodic heartbeats and labels (e.g., GPU model, memory, health) stored in a distributed key-value store.
  • Multi-tenant scheduling with hierarchical quotas and fair-share algorithms (e.g., weighted fair queuing) to prevent resource starvation.
  • Job queueing with priority levels and preemption policies (e.g., preempt lower-priority jobs when higher-priority jobs arrive).
  • Cost-aware placement using real-time pricing from cloud providers, considering data transfer costs and on-prem capacity.
  • Fault tolerance through replication of scheduler state, health checks, and automatic rescheduling of failed jobs.
  • Telemetry pipeline for GPU utilization, temperature, and errors to inform scheduling and auto-scaling decisions.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.