← Amazon Interview InsightsMachine Learning Engineer

Amazon·Machine Learning Engineer·Technical Phone Screen·Senior

Senior
May 2026

Summary

Amazon ML engineer technical screen, basically one long coding question that asked me to build a neural network from scratch using only NumPy. No PyTorch, no autograd, just raw math and array ops. Harder than I expected to keep all the shapes straight under pressure.

Questions Asked (1)

Q1

Implement a feed-forward neural network from scratch using only NumPy, including forward pass, loss computation, backpropagation, and a mini-batch SGD training loop. Demonstrate it working on a small dataset.

Algorithms & Data StructuresTechnical Trade-offsSystem Design
Author's notes

This one took me the full session.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the network architecture (e.g., input size, hidden layers, activation functions) and the dataset. Then, walk through the implementation step by step: forward pass, loss computation, backpropagation, and SGD updates, explaining the matrix operations and gradient derivations. Finally, demonstrate training on a small dataset (e.g., XOR or MNIST subset) and discuss convergence and performance.

Pro tip: Emphasize vectorization and modular design: implement each component (layer, activation, loss) as reusable functions, and use NumPy broadcasting to avoid loops. This showcases production-quality code and deep understanding.

1. Define Architecture and Data

Specify the network structure (e.g., 2-layer MLP with ReLU and softmax) and choose a small dataset (e.g., XOR or Iris) to demonstrate. Explain input/output dimensions and parameter initialization.

2. Implement Forward Pass

Write vectorized code for each layer: linear transformation (Z = WX + b) followed by activation (ReLU, sigmoid, etc.). Cache intermediate values for backpropagation.

3. Compute Loss and Backpropagate

Calculate loss (e.g., cross-entropy) and derive gradients using chain rule. Implement backward pass to compute gradients for all parameters, ensuring correct shapes.

4. Mini-batch SGD Training Loop

Shuffle data, split into mini-batches, and for each batch: forward pass, loss, backward pass, and parameter update (W -= lr * dW). Track loss over epochs.

5. Demonstrate and Evaluate

Train on the small dataset, plot loss curve, and evaluate accuracy. Discuss hyperparameter choices (learning rate, batch size) and potential improvements.

Key Points to Mention

  • Vectorization: use NumPy operations to avoid explicit loops for efficiency.
  • Gradient derivation: explain chain rule for backpropagation, especially for softmax-cross-entropy.
  • Parameter initialization: use Xavier/He initialization to avoid vanishing/exploding gradients.
  • Mini-batch SGD: explain benefits (faster convergence, noise for generalization) and implementation details (shuffling, batch size).
  • Numerical stability: use log-sum-exp trick for softmax and avoid log(0) in cross-entropy.
  • Testing: verify gradients with numerical gradient checking and ensure loss decreases over epochs.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.