← Expedia Interview InsightsSoftware Engineer
This one sprawled in ways I didn't anticipate.
Start by clarifying requirements and constraints, then walk through the synchronous flow step-by-step: validate order state, call the payment gateway, and update order/payment/inventory atomically. Emphasize how you handle failures, idempotency, and latency while ensuring the API returns only after full resolution.
Pro tip: Design for idempotency and partial failures from the start—use idempotency keys for the refund and a saga or compensation pattern for atomicity across services. Also, set a strict timeout budget for the gateway call and have a fallback to avoid blocking the client indefinitely.
Ask about expected latency, consistency guarantees, failure handling, and whether the refund must be synchronous with the cancellation. Confirm the payment gateway's capabilities (e.g., idempotency, timeouts).
Check that the order is in a cancellable state (e.g., not shipped, not already cancelled) and that the refund amount is correct. Ensure the user has permission to cancel.
Call the payment gateway synchronously with an idempotency key to avoid duplicate refunds. Handle gateway timeouts and errors gracefully, possibly with retries or a fallback.
Use a transaction or saga to update the order status to cancelled, record the refund, and restock inventory. Ensure all updates succeed or roll back to maintain consistency.
Return success only after all steps complete. If latency is a concern, discuss trade-offs like async processing with polling or webhooks, but adhere to the synchronous requirement.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Said 'rollback' reflexively and immediately knew it was wrong.
Acknowledge the dual-write problem and propose a solution that ensures eventual consistency, such as the Saga pattern with compensating transactions or transactional outbox. Emphasize idempotency, reconciliation, and monitoring to detect and recover from such failures.
Pro tip: Mention that you would design the system to make the database update idempotent and retryable, and that you'd use a reconciliation job to detect and fix inconsistencies, showing you think beyond the immediate failure.
Clearly state the problem: a partial failure where the refund succeeds at the gateway but the database update fails, leading to inconsistency.
Discuss approaches like Saga pattern (with compensating transactions), transactional outbox, or two-phase commit, weighing trade-offs between consistency and availability.
Explain how to make operations idempotent (e.g., using idempotency keys) and implement retries with exponential backoff to handle transient failures.
Describe a reconciliation process (e.g., scheduled job) to detect and resolve inconsistencies, and monitoring/alerting to catch failures early.
Address how to handle the user-facing side, such as notifying the user of the refund status and ensuring support teams can intervene if needed.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Explain that idempotency keys ensure a refund is processed exactly once even if the request is retried, by using a unique key per refund attempt and enforcing it at the API gateway or service layer. Then describe where enforcement happens: typically at the entry point (API gateway) and persisted in a database with a unique constraint, so duplicate requests are detected and the original response is returned.
Pro tip: Mention that idempotency keys should be stored with the response and a TTL, and that the key must be generated by the client, not the server, to guarantee uniqueness across retries.
Clarify that idempotency means multiple identical requests have the same effect as one, which prevents duplicate refunds when retries occur due to network issues or timeouts.
Describe that the client generates a unique key (e.g., UUID) for each refund request and includes it in the request header or body. The server uses this key to detect and ignore duplicate requests.
Specify that enforcement happens at the API gateway (for early rejection) and at the refund service/database layer (for persistence and atomicity). The key is stored with a unique constraint, and the response is cached.
Walk through a scenario: client sends refund request with key, server processes and stores key+response. If client retries with same key, server returns the stored response without reprocessing.
Mention handling of concurrent requests with the same key (using locks or atomic operations), key expiration, and ensuring keys are unique per refund attempt (not per refund).
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Start by clarifying the business context and requirements, then compare distributed transactions (2PC) and sagas with compensation on key dimensions like consistency, availability, complexity, and performance. Conclude with a clear recommendation, typically favoring sagas for microservices-based travel platforms like Expedia due to scalability and resilience, but acknowledge when 2PC might be appropriate.
Pro tip: Emphasize that in travel systems, partial failures are common and user experience matters; sagas allow for graceful degradation and compensating actions that can be designed to be idempotent and retryable, which is crucial for high-volume booking flows.
Ask about consistency needs, latency requirements, failure scenarios, and whether the system is distributed across microservices. This shows you understand the problem before jumping to solutions.
Describe how 2PC works with a coordinator, prepare/commit phases, and its guarantees (atomicity). Mention drawbacks: blocking, single point of failure, latency, and unsuitability for long-running or cross-service operations.
Describe saga pattern: a sequence of local transactions, each with a compensating action for rollback. Highlight benefits: eventual consistency, non-blocking, resilience, and suitability for microservices. Mention challenges: complexity of compensation logic, idempotency, and lack of isolation.
Contrast the two approaches on consistency (strong vs eventual), availability, performance, complexity, and failure handling. Relate to the specific use case of refund, order update, and inventory update.
Choose saga with compensation for Expedia's microservices architecture, justifying with scalability, resilience, and alignment with business needs. Acknowledge that 2PC might be used in limited scopes but is generally avoided.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Short answer: tight timeouts on the gateway call, circuit breaker to fail fast when the gateway is degraded, and a clear error response back to the caller rather than hanging.
Start by acknowledging the synchronous nature of the flow and the need to prevent payment gateway latency from cascading into user-facing failures. Then, propose a layered resilience strategy: timeouts to bound each call, circuit breakers to fail fast when the gateway is unhealthy, and fallbacks or retries with backoff for transient errors. Emphasize configuration, monitoring, and testing to ensure the solution is production-ready.
Pro tip: Mention that timeouts should be set slightly above the gateway's p99 latency and that circuit breakers should be tuned to trip on a combination of error rate and slow calls, not just failures. Also, highlight the importance of idempotency keys to safely retry payment requests without double-charging.
Set a connect timeout (e.g., 1s) and a read timeout (e.g., 3s) based on the gateway's performance SLAs. Ensure timeouts are shorter than the overall request deadline to avoid cascading delays.
Wrap the payment call in a circuit breaker that opens after a threshold of failures or slow calls (e.g., 50% failure rate over 20 requests). When open, fail fast and optionally fall back to a queued retry or alternate payment method.
For transient errors (e.g., network timeouts), retry with exponential backoff and jitter, but only if the operation is idempotent. Use idempotency keys to prevent duplicate charges.
Emit metrics for timeout occurrences, circuit breaker state changes, and retry counts. Set alerts on circuit breaker trips and high timeout rates to detect gateway degradation early.
Use chaos engineering to simulate gateway latency and failures, and load testing to validate timeout and circuit breaker settings. Continuously adjust thresholds based on production data.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Start by clarifying what synchronous and asynchronous cancellation mean in the context of distributed systems, then explain when a synchronous flow is preferable (e.g., immediate consistency, simpler error handling) and when it's not (e.g., scalability, fault tolerance). Discuss trade-offs in terms of latency, complexity, reliability, and user experience, and tie it back to real-world scenarios like booking systems at Expedia.
Pro tip: Acknowledge that the choice often depends on business requirements and system constraints; showing you can weigh trade-offs rather than advocating for one approach demonstrates senior-level thinking.
Briefly define synchronous and asynchronous cancellation flows, highlighting that synchronous blocks the caller until completion while asynchronous returns immediately and processes in the background.
Explain situations where synchronous is beneficial, such as when immediate confirmation is required, when the operation is quick and reliable, or when strong consistency is critical (e.g., canceling a booking to release inventory instantly).
Compare trade-offs: synchronous offers simplicity and immediate feedback but can lead to higher latency, reduced throughput, and cascading failures; asynchronous improves scalability and resilience but adds complexity, eventual consistency, and potential user confusion.
Connect to Expedia's domain: for example, canceling a hotel reservation might need synchronous to update availability, while canceling a flight might be asynchronous to handle downstream systems.
Summarize that the choice depends on factors like latency requirements, consistency needs, and system architecture, and mention that hybrid approaches (e.g., synchronous for user-facing, asynchronous for backend) are common.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.