← Bank of America Interview Insights
Start by defining regression in simple terms, then explain that linear regression models the linear relationship between input features and a continuous target by fitting a line (or hyperplane) that minimizes prediction error. Emphasize that it captures the conditional mean of the target given the inputs, and mention its assumptions and limitations.
Pro tip: Connect the explanation to a practical example, such as predicting loan default risk or interest rates, to show how regression supports decision-making in banking. Also, briefly note that linear regression is a foundational model that underpins more complex techniques, demonstrating awareness of trade-offs.
Explain that regression is a supervised learning technique used to predict a continuous output variable based on one or more input variables.
State that linear regression assumes a linear relationship between inputs and output, and finds the best-fitting straight line (or hyperplane) by minimizing the sum of squared errors.
Clarify that it captures the conditional mean of the target given the inputs, i.e., the expected value of the output for given feature values.
Briefly note key assumptions like linearity, independence, homoscedasticity, and normality of errors, and that violations can affect validity.
Give a relevant example, such as predicting credit risk or housing prices, to show practical application in a banking/software engineering context.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
This is where I started to sweat a little.
Start by stating the classical assumptions clearly and concisely, then explain why each matters for OLS to be BLUE (Best Linear Unbiased Estimator). Finally, connect the assumptions to practical implications in a software engineering context, such as data quality checks or model validation.
Pro tip: Emphasize that while strict exogeneity is critical for causal inference, in practice many software engineers focus on prediction, where some assumptions (like normality) are less critical. Showing this nuance demonstrates maturity.
Briefly explain that OLS estimates coefficients by minimizing the sum of squared residuals, and the classical assumptions ensure desirable statistical properties.
Enumerate the assumptions: linearity in parameters, random sampling, no perfect multicollinearity, zero conditional mean (exogeneity), homoskedasticity, and no autocorrelation (for time series).
Describe how violating each assumption affects OLS: e.g., omitted variable bias from exogeneity violation, inefficient estimates from heteroskedasticity, etc.
Discuss how these assumptions translate into data preprocessing, feature engineering, and model diagnostics in a production environment.
Summarize that under these assumptions, OLS is BLUE, and note that normality is needed for inference but not for unbiasedness.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
They actually wanted the derivation, not just the formula.
Start by clearly stating the OLS objective: minimize the sum of squared residuals. Then derive the normal equations using matrix calculus, and solve for the coefficient vector to obtain the closed-form estimator. Finally, briefly mention the assumptions required for the estimator to be valid.
Pro tip: Emphasize the geometric interpretation: OLS projects the response vector onto the column space of the design matrix, which gives the normal equations directly. This shows deeper understanding and connects to linear algebra concepts valued in software engineering.
State the linear model y = Xβ + ε and define the OLS objective as minimizing the sum of squared residuals: S(β) = (y - Xβ)^T (y - Xβ).
Expand the quadratic form: S(β) = y^T y - 2β^T X^T y + β^T X^T X β. This sets up for differentiation.
Take the gradient with respect to β: ∂S/∂β = -2X^T y + 2X^T X β. Set it to zero to obtain the normal equations: X^T X β = X^T y.
Assuming X^T X is invertible, solve the normal equations: β_hat = (X^T X)^{-1} X^T y. This is the OLS closed-form estimator.
Mention key assumptions: linearity, exogeneity, no perfect multicollinearity, homoscedasticity, and no autocorrelation. Under these, OLS is BLUE (Best Linear Unbiased Estimator) by Gauss-Markov theorem.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Talked about variance inflation and how the coefficient estimates get unstable, signs can flip, and numerically the matrix inversion gets sketchy.
Start by defining multicollinearity and explaining its consequences in regression, such as unstable coefficient estimates and inflated standard errors. Then, discuss practical detection methods and remedies, emphasizing trade-offs and context-specific choices. Tailor your answer to a software engineering role by highlighting computational and implementation aspects.
Pro tip: Mention that multicollinearity doesn't affect predictive performance but harms interpretability, and in a banking context, interpretability is often critical for regulatory compliance. Also, suggest using domain knowledge to decide whether to drop or combine features.
Explain that multicollinearity occurs when two or more predictor variables in a regression model are highly correlated, making it difficult to isolate their individual effects.
Discuss how multicollinearity leads to unstable coefficient estimates, inflated standard errors, and reduced statistical power, which complicates interpretation and inference.
Mention techniques like correlation matrices, variance inflation factor (VIF), and condition number to identify multicollinearity.
List common solutions such as removing or combining variables, using regularization (ridge regression), principal component analysis (PCA), or collecting more data, and discuss their trade-offs.
Relate to the banking domain: emphasize that interpretability is crucial for regulatory and business decisions, so remedies should preserve interpretability when possible.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Start by defining L1 and L2 regularization mathematically, then contrast their effects on model coefficients and sparsity. Discuss practical implications such as feature selection, computational efficiency, and robustness, and conclude with when to prefer each based on problem characteristics and domain constraints.
Pro tip: In banking, where interpretability and regulatory compliance are critical, emphasize that L1's feature selection can simplify model explanations, but also mention that L2 is often preferred when all features are relevant and multicollinearity is a concern.
Clearly state that L1 adds the sum of absolute weights to the loss, while L2 adds the sum of squared weights. Mention that L1 promotes sparsity (many weights become exactly zero) and L2 promotes small but non-zero weights.
Discuss how L1 leads to feature selection and simpler models, while L2 distributes weight more evenly and handles correlated features better. Note that L1 is non-differentiable at zero, requiring subgradient methods, while L2 is smooth and easier to optimize.
Cover computational aspects: L1 can be solved with techniques like coordinate descent or LARS, L2 with gradient descent. Mention that L1 is robust to outliers in features, while L2 is sensitive but provides stability.
Prefer L1 when you need a sparse model for interpretability or when you suspect many features are irrelevant. Prefer L2 when all features contribute and you want to prevent overfitting without eliminating features, or when features are highly correlated.
Briefly note that Elastic Net combines L1 and L2, often used when there are multiple correlated features or when you want a balance between sparsity and stability.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.