← Nuro Interview Insights

Nuro·Machine Learning Engineer·Onsite - System Design / Architecture·Senior

Senior
Apr 2026

Summary

Nuro ML engineer interview hit me with a pretty gnarly concurrency design question. Not what I expected going in, but in hindsight it makes sense for a robotics company where things run in parallel constantly.

Questions Asked (1)

Q1

Design a thread-safe in-memory cache that wraps an expensive compute function, where if two threads request the same missing key simultaneously, only one should run the computation while the other waits for the result. Walk through your locking strategy, how you'd signal waiting threads, and how you'd handle exceptions from the compute function.

System DesignTechnical Trade-offsAlgorithms & Data Structures
Author's notes

The core mechanic they wanted was a per-key in-flight marker plus a condition variable, not just slapping a global mutex around everything.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying requirements and constraints, then propose a design using a concurrent map with per-key locking (e.g., striped locks or a ConcurrentHashMap of futures) to ensure only one computation per key. Walk through the locking strategy, how waiting threads are signaled (e.g., via CompletableFuture or condition variables), and how exceptions are propagated and handled. Finally, discuss trade-offs and potential optimizations.

Pro tip: Mention that you would use a ConcurrentHashMap with computeIfAbsent to atomically check-and-insert a placeholder future, which elegantly handles the single-computation guarantee without explicit locks. Also, highlight the importance of cleaning up failed computations to avoid caching exceptions permanently.

1. Clarify requirements and constraints

Ask about expected concurrency level, cache size limits, eviction policies, and whether the compute function is idempotent. This shows you consider practical aspects before diving into design.

2. Design the core data structure and locking

Propose a ConcurrentHashMap that maps keys to futures or locks. Use per-key locking (e.g., striped locks or computeIfAbsent) to ensure only one thread computes per key, while others wait on the future.

3. Explain signaling and waiting mechanism

Describe how waiting threads block on the future (e.g., CompletableFuture.get()) and are signaled upon completion. Mention that this avoids busy-waiting and handles timeouts if needed.

4. Handle exceptions and cleanup

Detail how exceptions from the compute function are propagated to all waiting threads and how the cache entry is removed or marked as failed to allow retries. Emphasize not caching exceptions permanently.

5. Discuss trade-offs and optimizations

Talk about lock granularity, memory overhead, eviction policies (e.g., LRU), and potential deadlocks. Mention alternatives like using a single lock for simplicity vs. per-key locks for scalability.

Key Points to Mention

  • Use of ConcurrentHashMap with computeIfAbsent to atomically insert a placeholder (e.g., CompletableFuture) for the key.
  • Per-key locking or striped locks to reduce contention and allow concurrent computations for different keys.
  • Signaling waiting threads via futures (CompletableFuture) or condition variables, avoiding busy-waiting.
  • Exception handling: propagate exceptions to all waiting threads and remove the failed entry to allow retries.
  • Consideration of cache eviction policies (e.g., LRU) and size limits to prevent memory leaks.
  • Thread safety of the compute function itself and potential reentrancy issues.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.