Chapter 1 — Interview Bank

Every answer here follows the same four beats. Say the first two within 20 seconds; the last two are what turn a correct answer into a strong one.

Questions are stacked by difficulty. Each level assumes everything above it.

How questions are written here. Real interviewers rarely ask "define X". They describe a situation — a model, a chart, a stakeholder's claim — and watch whether you spot the concept underneath. So every question below opens with a scenario, then names the concept being tested, then gives the four beats. Practice by reading only the scenario and answering out loud before you look.

Level 1 — Screen

Q1 · "It's just linear regression"

You've joined a team that prices homes. The PM says: "Our model is just linear regression. What is it actually doing under the hood, and why is it a sensible first model?"

• Concept tested: what least squares optimizes.
• Intuition: it lays a ruler through the cloud of past sales so the vertical misses are as small as possible.
• Answer: it predicts price as a weighted sum of features plus an intercept, and picks the weights that minimize the sum of squared misses:

\min_{b,\,\mathbf w}

choose the intercept b and the weights \mathbf w that make the sum as small as possible

\sum_{i=1}^{n}

add up over every past sale, i = 1 to n

y_i

the actual sale price of home i

b

the intercept: the baseline price before any feature counts

\mathbf w^\top \mathbf x_i

home i's features times their weights, added up (^\top = transpose: turns the weight column into a row)

• Why it's a sensible first model: the loss is a bowl with one bottom, so the answer is exact and fast (Card 2); every weight is readable; and it's the baseline anything fancier must beat.
• Next they'll ask: "Why squared misses?" → Q6.

Q2 · The coaching program that "worked"

A regional manager says: "The 10 worst-performing stores last quarter all improved this quarter after we sent them coaching. Let's roll coaching out everywhere." How do you respond?

• Concept tested: regression to the mean — the effect that gave regression its name.
• Intuition: a terrible quarter is partly bad luck, and bad luck doesn't repeat.
• Answer: the worst stores would be expected to improve even with no coaching. Without a comparison group of similar stores that didn't get coaching, the improvement says nothing about the program.
• Why: with correlation r < 1 between quarters, the predicted next quarter in standard units is closer to average than this one:

\hat z_{\text{next}}

the predicted next-quarter result, in standard units (SDs from the average store)

z_{\text{now}}

this quarter's result in standard units: how many SDs from the average store

r

the correlation between one quarter's results and the next

• Next they'll ask: "Where does the word 'regression' come from?" → Galton noticed children of very tall parents were tall but closer to average, and called it "regression toward mediocrity".

Q3 · Correlation is not a slope

Marketing reports: "Ad spend and signups have a correlation of 0.8, so each extra dollar brings in 0.8 signups." What's wrong with that sentence?

• Concept tested: correlation vs regression slope.
• Intuition: correlation says how tightly two things move together; the slope says how far one moves per unit of the other.
• Answer: correlation is unit-free and symmetric; it can't be "signups per dollar". The slope rescales it by the two spreads:

w

the slope: extra signups per extra dollar of ad spend

r

the correlation between spend and signups (0.8 here), unit-free

s_y

the standard deviation of signups

s_x

the standard deviation of ad spend, in dollars

R^2

the share of the variation in the target that the line explains

• Why: a correlation of 0.8 is the same whether spend is in dollars or millions; the slope isn't.
• Next they'll ask: "Even with the right slope, can you say each dollar causes signups?" → Only without confounding (Q41).

Q4 · "Linear can't bend"

Price rises with size but flattens for very large homes. A teammate says: "Linear regression can't capture a curve, so we need a neural net." Do you agree?

• Concept tested: "linear" means linear in the weights, not in the features.
• Intuition: the model only needs the weights to be plain multipliers; the columns can be any shape you compute first.
• Answer: no. Add a curved feature and it's still linear regression:

\hat y

the predicted price

w_1

the weight on size (or on log size)

w_2

the weight on size squared: bends the line into a curve

• Why: the loss is still quadratic in the weights, so the same normal equation solves it.
• Next they'll ask: "Then what isn't linear?" → Weights inside a nonlinearity, such as b \cdot e^{w \cdot x}, or a neural network.

Q5 · "The residuals look perfect"

A junior analyst shows you that the residuals sum to exactly zero and are uncorrelated with every feature, and concludes the model's assumptions hold perfectly. What do you tell them?

• Concept tested: errors vs residuals.
• Intuition: OLS chose the line so that the residuals look like that; it's checking its own homework.
• Answer: those properties hold for every OLS fit with an intercept, whatever the truth. Residuals estimate the unseen errors but are forced to satisfy:

X^\top

the feature matrix transposed: rows become columns, one row per feature

\mathbf e

the residuals, actual minus fitted, one per row (bold = vector)

\mathbf y

all the actual target values stacked in a vector

X

the feature matrix: one row per home, a column of 1s plus one column per feature

\hat{\mathbf w}

the fitted weights (hat = estimated from data)

\boldsymbol\varepsilon

the true errors: misses from the true line, never observed

\boldsymbol\beta

the true weights, unknown

• Why: it's the normal equation itself (Card 2). Real checks look at patterns in residuals (curves, funnels, waves), not their correlation with x.
• Next they'll ask: "So how would you check exogeneity?" → Q59.

Q6 · Delivery times and storms

You predict delivery times. A few deliveries take 10\times longer because of storms. Ops complains that predictions for normal days are now too high. What's happening, and what would you change?

• Concept tested: what squared loss predicts, and its sensitivity to outliers.
• Intuition: squaring makes one huge miss count as much as a hundred small ones, so the line leans toward the storms.
• Answer: squared loss predicts the conditional mean, which the storms drag up. Absolute loss predicts the median, which ignores them; Huber loss sits between.

\arg\min_c

the value of c that makes what follows as small as possible

c

one constant guess, used for every delivery

\sum_i

add up over every delivery i

y_i

delivery i's actual time

\bar y

the average delivery time (bar = mean)

\text{median}(y)

the median delivery time: half the deliveries are faster, half slower

• Why: squared error is maximum likelihood under Gaussian noise; heavy-tailed noise breaks that assumption.
• Next they'll ask: "Other options?" → Model storms as a feature, or use quantile regression to predict the 90th percentile for promised delivery windows.

Q7 · Bedrooms hurt price?

Your price model has a coefficient of −$8k on "number of bedrooms", with square footage also in the model. The PM asks: "So bedrooms make a home worth less?"

• Concept tested: "holding the other features fixed".
• Intuition: at a fixed size, one more bedroom means every room is smaller.
• Answer: the coefficient is the change in price per extra bedroom at the same square footage. Bedrooms on their own still correlate positively with price, because bigger homes have more of them.
• Why: by Frisch–Waugh–Lovell, a coefficient uses only the part of a feature the other features don't explain (Q52).
• Next they'll ask: "What if the features are highly correlated?" → "Holding fixed" becomes unrealistic and the coefficient unstable (Q39).

Q8 · Probabilities of −0.1

Someone predicted loan default (0 or 1) with linear regression. Some predicted probabilities are −0.1 and 1.2. Is the model usable?

• Concept tested: the linear probability model vs logistic regression.
• Intuition: a straight line keeps going forever; probabilities have to stop at 0 and 1.
• Answer: for ranking or average effects it can be acceptable, but as probabilities it's broken. Logistic regression bends the line into the [0, 1] range.
• Why: squared loss also penalizes being "too right", so confidently-correct points far from the boundary pull the fit around.
• Next they'll ask: "When do people still use the linear version?" → Economists use it for easy-to-read average effects when probabilities stay in the middle range.

Q9 · Why one model trains instantly

An engineer asks: "Your regression trains instantly with one library call, but the logistic model on the same data needs an iterative solver and a tolerance setting. Why?"

• Concept tested: closed-form vs iterative solutions.
• Intuition: the quadratic formula versus guess-and-check.
• Answer: least squares has an exact formula; logistic regression doesn't, so it must be solved step by step.
• Why: quadratic loss → linear gradient → a system of linear equations, solvable exactly. The logistic loss has a nonlinear gradient.

\nabla L(\mathbf w)

the gradient: the vector of slopes of the loss, one per weight; points uphill

\mathbf w

the weights, one per feature (bold = vector)

(X^\top X)^{-1}

the matrix inverse of X^\top X: undoes it, the matrix version of dividing

• Next they'll ask: "Does lasso have a closed form?" → No: \lvert w \rvert has a kink at zero.

Q10 · Loss goes to NaN

Your gradient descent loss goes 3.2, 15.7, 210, NaN. A colleague's run on the same data barely moves after 10,000 steps. What's going on in each case?

• Concept tested: the learning rate.
• Intuition: step size on a foggy hill — giant steps leap over the valley and climb the other side; tiny steps take forever.
• Answer: your learning rate is too high (diverging); theirs is too low (crawling).
• Why: gradient descent on this bowl converges only when the step is smaller than a limit set by the steepest curvature:

\eta

the learning rate: the size of each gradient-descent step

\lambda_{\max}

the largest eigenvalue: how sharply the loss bowl curves in its steepest direction

n

the number of rows

• Next they'll ask: "How do you choose it in practice?" → Standardize features first, then a learning-rate range test, a decaying schedule, or an adaptive optimizer like Adam.

Q11 · Two billion rows

Your training data is 2 billion rows spread across disks. Full-batch gradient descent takes an hour per step. What do you change?

• Concept tested: batch vs mini-batch vs stochastic gradient descent.
• Intuition: before each step, you don't need to ask everyone — a focus group gives a good enough direction.
• Answer: use mini-batches of a few hundred to a few thousand rows. Each step is far cheaper and the direction is right on average.
• Why: a mini-batch gradient is an unbiased estimate of the full gradient; its noise shrinks with batch size, and batches map well onto GPUs.
• Next they'll ask: "Epoch vs iteration?" → An epoch is one full pass over the data; an iteration is one update on one batch.

Q12 · Two analysts, two estimators

Two analysts estimate average order value every week. Analyst A's method is exactly right on average over many weeks but swings wildly week to week. Analyst B's is always about 2% low but very stable. Which would you use?

• Concept tested: bias, variance, and an estimator's total error.
• Intuition: A is a thrower who averages on the bullseye with a shaky hand; B aims slightly off with a steady one.
• Answer: often B. What matters is total error, and a small bias can be worth a large cut in variance:

\text{MSE}(\hat\theta)

the estimator's total error: its average squared distance from the truth

\hat\theta

an analyst's weekly estimate of average order value (hat = estimated)

\theta

the true average order value

\mathbb E

the average over many repeated weeks (samples)

\text{Bias}(\hat\theta)

how far the estimates land from the truth on average: the aim error

\text{Var}(\hat\theta)

how much the estimate swings from week to week

• Why: A is unbiased (E[\hat\theta] = \theta) but its variance dominates; B trades a little bias for much less variance.
• Next they'll ask: "Where does this show up in regression?" → Ridge (Q53).

Q13 · Can I trust this coefficient?

A stakeholder hands you a regression saying "each extra year of tenure raises salary by $3,200, p < 0.001" and asks whether they can trust the number. What would you check?

• Concept tested: the assumptions, and what each one protects.
• Intuition: the model's own name is the checklist.
• Answer: L-I-N-E-A-R — linearity, independence, normality, equal variance, absence of perfect multicollinearity, regressors exogenous. For this claim, the big worry is R: something like role or performance drives both tenure and salary.
• Why: L, A and R keep the coefficient unbiased; I and E keep its p-value honest; N makes small-sample tests exact.
• Next they'll ask: "Which ones matter if I only want predictions?" → Mostly L, plus new data that looks like the training data (Q35).

Q14 · The mansion and the bargain

One mansion, four times larger than any other home, is in your training data; removing it changes the price-per-sq-ft coefficient by 30%. Another house sold $200k above the line, but removing it barely changes the fit. Explain both.

• Concept tested: outliers, leverage and influence.
• Intuition: an outlier is a weird y; leverage is a weird x — a long lever arm; influence needs a long lever and a push.
• Answer: the mansion has huge leverage and moves the line (influential). The bargain is an outlier in y but sits near the middle in x, so it can't tilt the line much.
• Why: influence is roughly residual size \times leverage, which Cook's distance combines (Card 13):

D_i

Cook's distance for home i: how far the fit moves if you delete it

e_i^2

home i's residual, squared: how badly the line misses it

p

the number of fitted parameters

s^2

the estimated noise variance

h_{ii}

home i's leverage: how unusual its x is, from 0 to 1

• Next they'll ask: "Do you delete the mansion?" → Not automatically: check it's real, restrict the model's valid range, or use a robust loss, and report results both ways.

Q15 · "A 97% chance it works"

A PM says: "The feature's p-value is 0.03, so there's a 97% chance it really works." Correct them.

• Concept tested: what a p-value is.
• Intuition: a p-value is how surprised you'd be by this data in a world where the effect is zero.
• Answer: it's the probability of a result at least this extreme if the effect were zero — not the probability that the effect is real.

p

the p-value (0.03 here)

P

"the probability that"

T

the test statistic you'd get on a fresh dataset where the effect is truly zero

t_{\text{obs}}

the test statistic you actually observed

H_0

the null hypothesis: the feature has no effect

• Why: it's computed assuming H_0, so it can't also be the probability of H_0. Getting P(\text{effect is real}) needs a prior (Bayes).
• Next they'll ask: "Significant but useless?" → With huge n, tiny effects get tiny p-values; look at the effect size and its interval.

Q16 · "Your model says $480k is impossible"

Your model values a 1,500 sq ft house at $412k, with a 95% interval of $377k–$447k. A seller lists at $480k and says: "Your model says my price is impossible." Is that right?

• Concept tested: confidence interval vs prediction interval.
• Intuition: $377k–$447k is where the average 1,500 sq ft house sells; one particular house varies much more.
• Answer: no. That's the confidence interval for the mean. The prediction interval for a single house is $326k–$498k, which includes $480k.
• Why: the prediction interval adds the house's own noise — the "1 +":

\hat y_0

the model's value for this house, $412k

t

the t-table cutoff for 95% (about 2 with plenty of data)

s

the estimated noise standard deviation: a typical miss

\mathbf x_0^\top (X^\top X)^{-1}\mathbf x_0

the line's own uncertainty at this house's features \mathbf x_0, in noise units; grows far from the average house

• Next they'll ask: "Which one shrinks to zero with more data?" → Only the CI; the PI never gets narrower than the noise.

Q17 · R^2 = 0.3

Your model of weekly customer spend has R^2 = 0.3. A director wants to scrap it. What do you say?

• Concept tested: what R^2 does and doesn't measure.
• Intuition: explaining 30% of something very noisy can be worth a lot.
• Answer: R^2 alone doesn't decide it. Compare held-out error against the current baseline, check whether the key coefficients are precise, and ask what decision the model supports.

• Why: human behavior is noisy; a model can rank customers well or estimate an effect precisely while R^2 stays low.
• Next they'll ask: "When is low R^2 a red flag?" → When the target is physical or mechanical and you expected near-deterministic behavior.

Q18 · 500 overlapping features

You have 500 candidate features for predicting churn, many overlapping ("logins in the last 7 days" and "logins in the last 14 days"). Leadership wants a short list of drivers. What do you fit?

• Concept tested: ridge vs lasso vs elastic net.
• Intuition: ridge turns every volume knob down a little; lasso switches most of them off.
• Answer: lasso gives a short list by setting most weights to exactly zero, but among overlapping features it picks one somewhat arbitrarily. Elastic net keeps correlated groups together while still selecting.

\lambda

the penalty strength: bigger means more shrinkage

\lVert\mathbf w\rVert_2^2

the squared L2 length of the weights: the sum of their squares

\lVert\mathbf w\rVert_1

the L1 length of the weights: the sum of their absolute values

\lambda_1

the strength of elastic net's L1 (lasso) part

\lambda_2

the strength of its L2 (ridge) part

• Why: the L1 constraint has corners on the axes, where weights are exactly zero (Card 11).
• Next they'll ask: "Are lasso's picks 'the drivers'?" → They're predictive, not causal, and can change between bootstrap samples; check stability.

Q19 · Should we raise prices 5%?

A regression of \log(\text{units sold}) on \log(\text{price}) gives a slope of -1.4. The PM asks whether to raise prices 5%.

• Concept tested: reading log–log coefficients as elasticities.
• Intuition: logs on both sides turn "units per dollar" into "percent per percent".
• Answer: a 5% price rise cuts units by about 6.6%, so revenue falls by about 1.9%. Demand is elastic (|slope| > 1); don't raise prices on revenue grounds.

\frac{d\ln q}{d\ln p}

how fast log units sold (q) changes as you nudge log price (p): the elasticity, a slope, not a fraction to divide

• Why: \text{revenue} = \text{price} \times \text{quantity}, so it rises with price only when elasticity is between -1 and 0.
• Next they'll ask: "Is -1.4 causal?" → Only if price changes weren't driven by demand (Q60).

Q20 · Explaining to the CFO

The CFO, with no statistics background, asks why she should trust a model saying a kitchen remodel adds $25k to a home's value.

• Concept tested: communicating a coefficient and its limits.
• Intuition: use her example, not the math.
• Answer: "Comparing homes that are otherwise similar in size, location and age, those with remodeled kitchens sold for about $25k more, give or take $X. That's a strong pattern, but homes that get remodeled may differ in ways we can't see, so treat it as an estimate of association, not a guarantee."
• Why: decision-makers need direction, size and certainty — and an honest note on causality.
• Next they'll ask: "How would you make it causal?" → Compare the same homes before and after remodeling, or find a natural experiment.

Level 2 — Onsite core

Q21 · Five points on the whiteboard

The interviewer writes five (size, price) pairs: (1000, 310), (1200, 360), (1500, 380), (1800, 500), (2100, 530). "No libraries. Derive the least-squares line and tell me what it predicts for 1,500 sq ft."

• Concept tested: deriving simple OLS (Card 1).
• Intuition: fix the intercept first and the data centers itself.
• Answer: \bar x = 1{,}520 and \bar y = 416, so:

w

the slope: $k of price per extra sq ft

S_{xy}

the co-movement sum: positive when bigger homes cost more

S_{xx}

the spread of the sizes around their average

\sum

add up over the five houses

x_i

house i's size in sq ft

\bar x

the average size, 1,520 sq ft (bar = mean)

y_i

house i's price in $k

\bar y

the average price, $416k

b

the intercept: where the line crosses size 0

\hat y(1500)

the line's predicted price, in $k, for a 1,500 sq ft house

• Why: \partial L/\partial b = 0 forces the line through (\bar x, \bar y); what's left is a slope through the centered cloud.
• Next they'll ask: "Force it through the origin?" → Q25.

Q22 · Now thirty features

"Same problem with 30 features. Derive the solution in matrix form. When does it fail to exist?"

• Concept tested: the normal equation (Card 2).
• Intuition: at the bottom of a bowl, the slope is zero in every direction at once.
• Answer:

L(\mathbf w)

the loss: the sum of squared misses, as a function of the weight vector

\mathbf w

the 31 weights (intercept plus 30 features) as one vector (bold = vector)

\mathbf y

all the actual prices stacked in a vector

X

the design matrix: one row per house, a column of 1s plus 30 feature columns

\lVert \mathbf y - X\mathbf w\rVert^2

the squared length of the residual vector: the sum of squared misses

\nabla L

the gradient: the vector of slopes of the loss, one per weight; points uphill

X^\top

X transposed: rows become columns

\hat{\mathbf w}

the fitted weights (hat = estimated from data)

It fails to be unique when X^\top X is singular: perfect multicollinearity (sq ft and sq m both included, or all dummies plus an intercept) or more features than rows.

• Why: the Hessian 2X^\top X is positive semi-definite, so any stationary point is a minimum; full column rank makes it unique.
• Next they'll ask: "Would you invert X^\top X in code?" → No: QR or SVD (lstsq), since forming X^\top X squares the condition number.

Q23 · Draw it

The interviewer draws y as an arrow and a plane spanned by two feature columns. "Show me where OLS's prediction lives on this picture, and why there."

• Concept tested: OLS as orthogonal projection.
• Intuition: the shadow of y on the floor of all possible predictions, with the light directly overhead.
• Answer: \hat y is the foot of the perpendicular from y to the plane; the residual is at right angles to every column.

\hat{\mathbf y}

OLS's prediction vector: the foot of the perpendicular on the plane

H

the hat matrix: projects \mathbf y onto the plane of possible predictions

(X^\top X)^{-1}

the matrix inverse of X^\top X: undoes it

• Why: the closest point on a flat surface is always straight below.
• Next they'll ask: "What does orthogonality give you for free?" → Residuals sum to zero, are uncorrelated with every feature, and the fit passes through the means.

Q24 · A leverage of 0.62

Your diagnostics flag h_{ii} = 0.62 for one row in a 50-row model with 4 parameters. What is that number, and should you worry?

• Concept tested: the hat matrix and leverage.
• Intuition: leverage measures how much a point's own y pulls its own prediction toward itself.
• Answer: it's the diagonal of H. Leverages average p/n = 0.08, and the usual flag is 2p/n = 0.16, so 0.62 is extreme: this row's x is far from the others, and 62% of its prediction comes from its own y.

\hat y_i

row i's prediction

h_{ii}

row i's leverage: the weight its own y_i gets in its own prediction

\sum_{j \ne i}

add up over every other row j

h_{ij}

how much row j's actual value pulls row i's prediction

y_j

row j's actual value

\sum_i

add up over every row i

p

the number of fitted parameters, intercept included

• Why: H is a projection with trace p, so leverages sum to the number of parameters.
• Next they'll ask: "Is it a problem?" → Only if its residual is also large — check Cook's distance (Q14).

Q25 · R^2 jumped from 0.6 to 0.95

A team fits machine energy use against runtime and forces the line through zero, "because zero runtime means zero energy". R^2 jumps from 0.6 to 0.95. Is the model better?

• Concept tested: regression without an intercept.
• Intuition: pinning the line to (0, 0) changes the yardstick, not just the fit.
• Answer: not necessarily. Without an intercept, software reports an uncentered R^2 that compares against predicting 0, not the mean, so it isn't comparable. The slope becomes:

\hat w

the fitted slope with no intercept: energy per unit of runtime

x_i

machine i's runtime

y_i

machine i's energy use

R^2_{\text{uncentered}}

the R^2 software reports with no intercept: compares against predicting 0, not the mean

• Why: with no column of 1s, \sum e = 0 is no longer enforced and \text{TSS} = \text{ESS} + \text{RSS} breaks.
• Next they'll ask: "When is no intercept justified?" → When theory demands it and you have data near zero; compare RMSE, not R^2.

Q26 · "Squared error feels arbitrary"

An ML engineer says: "Why squared error at all? It feels like an arbitrary choice." Give the probabilistic justification.

• Concept tested: least squares = maximum likelihood (Card 3).
• Intuition: Gaussian noise says big misses are exponentially unlikely — exactly what squaring penalizes.
• Answer: assume y = w^\top x + \varepsilon with Gaussian \varepsilon. The log-likelihood is a constant minus the RSS:

\log\mathcal L(\mathbf w, \sigma^2)

the log-likelihood: how probable the data are under weights \mathbf w and noise variance \sigma^2

n

the number of rows

\sigma^2

the variance of the Gaussian noise

\pi

the circle constant 3.14159…, part of the Gaussian's formula

\mathbf w^\top\mathbf x_i

the model's prediction for row i: its features times the weights, added up

• Why: maximizing it over w means minimizing squared error. Other noise models give other losses.
• Next they'll ask: "Laplace noise?" → Absolute error. "Is the MLE of \sigma^2 unbiased?" → No (Q51).

Q27 · The retrain that never finishes

Your feature count grew from 100 to 100,000, and the nightly closed-form refit went from seconds to never finishing. Why, and what do you switch to?

• Concept tested: the cost of OLS.
• Intuition: summarizing the data into a p \times p table is cheap per row; solving that table grows like p cubed.
• Answer:

O(np^2)

cost grows like n rows times p^2: building the p \times p table X^\top X

O(p^3)

cost grows like p cubed: solving the p \times p system

At p = 100{,}000, p^3 = 10^{15} operations. Switch to (mini-batch) gradient descent or an iterative solver, usually with regularization, and exploit sparsity if features are mostly zeros.

• Why: an iterative step costs O(\text{batch} \cdot p) and needs no inversion.
• Next they'll ask: "Will it give the same answer?" → Q30.

Q28 · Sharded data

Your data is 10 million rows \times 50 features, sharded across machines. A colleague says gradient descent is pointless because a closed form exists. Do you agree?

• Concept tested: when iterative methods beat the formula.
• Intuition: the formula needs the whole map; gradient descent only needs to feel the ground underfoot.
• Answer: with p = 50, the formula is actually fine — X^\top X is only 50 \times 50 and can be summed shard by shard, then solved once. Gradient descent wins when p is huge, data streams in, or the model has no closed form.
• Why: X^\top X and X^\top y are sums over rows, so they parallelize perfectly.
• Next they'll ask: "Then why learn gradient descent here?" → It's the same engine that trains logistic regression and every neural network.

Q29 · The zigzag

Gradient descent on (square feet, bedrooms) zigzags and needs a tiny learning rate. After a teammate standardizes both features it converges in 50 steps. Explain.

• Concept tested: feature scaling and conditioning.
• Intuition: a long, narrow valley makes you zigzag between the walls; a round bowl lets you walk straight down.
• Answer: square feet in the thousands and bedrooms in single digits make the loss surface extremely stretched. Standardizing makes curvature similar in every direction, so one learning rate fits all.

x_j

feature j (square feet or bedrooms)

\leftarrow

"replace with": overwrite the feature with its standardized version

\bar x_j

feature j's average (bar = mean)

s_j

feature j's standard deviation

\kappa

the condition number: the bowl's steepest curvature over its flattest

\lambda_{\max}

the largest eigenvalue: curvature in the steepest direction

\lambda_{\min}

the smallest eigenvalue: curvature in the flattest direction

• Why: the safe step size is set by the steepest direction, so a large condition number \kappa means painfully slow progress in the flat direction.
• Next they'll ask: "Does scaling change OLS predictions?" → No; it does change ridge and lasso (Q45).

Q30 · Stuck in a local minimum?

A colleague worries that gradient descent for linear regression might get stuck in a local minimum, like their neural networks do.

• Concept tested: convexity of least squares.
• Intuition: a bowl has only one bottom.
• Answer: it can't: every local minimum is the global one. With a small enough learning rate it reaches the same answer as the formula.

\nabla^2 L

the Hessian: how the slopes themselves change, i.e. the bowl's curvature in every direction

\succeq 0

positive semi-definite: curves up (or stays flat) in every direction, never down

• Why: a positive semi-definite Hessian everywhere means the loss is convex.
• Next they'll ask: "And neural networks?" → Non-convex: many minima and saddle points, no such guarantee.

Q31 · Prove it's unbiased

"On the board: prove the OLS estimator is unbiased. Then tell me which assumptions you used, and which you didn't need."

• Concept tested: unbiasedness and exactly what it rests on (Card 5).
• Intuition: estimate = truth + weighted noise, and the noise averages out.
• Answer:

\boldsymbol\beta

the true weights (bold = vector)

\boldsymbol\varepsilon

the true errors of every row, never observed

\mathbb E[\hat{\mathbf w}\mid X]

the fitted weights averaged over repeated samples, given (holding fixed) the features X

\mathbb E[\boldsymbol\varepsilon\mid X]

the average error at every feature value; zero means exogeneity

Used: linearity, full rank, exogeneity. Not needed: equal variance, independence, normality.

• Why: only the noise's mean enters bias; its spread and shape enter the variance and the tests.
• Next they'll ask: "Still unbiased under heteroscedasticity?" → Yes; only the standard errors break (Q37).

Q32 · 367 or 612?

Your 5-row, 2-parameter model has \text{RSS} = 1{,}837. One analyst reports the noise variance as 367, another as 612. Who's right?

• Concept tested: degrees of freedom and the unbiased noise estimate (Card 7).
• Intuition: each fitted parameter bends the line toward the data and hides a little of the noise.
• Answer: 612. Dividing by n = 5 underestimates; dividing by n - p = 3 is unbiased:

s^2

the unbiased estimate of the noise variance

\mathbb E

the average over many repeated samples

• Why: X^\top e = 0 imposes p constraints on the residuals, leaving n - p free.
• Next they'll ask: "Why does the sample variance divide by n - 1?" → Same proof with p = 1 (the mean).

Q33 · Which 200 homes to survey?

You can afford to survey 200 homes to estimate price per square foot. Do you sample at random, or deliberately pick very small and very large homes?

• Concept tested: what controls the precision of a slope.
• Intuition: a ruler balanced on two points far apart wobbles less than one on two points close together.
• Answer: spreading x widely gives a much more precise slope:

\text{Var}(\hat w)

how much the fitted price-per-sq-ft slope wobbles across samples

\text{Var}(x)

the spread of home sizes in your survey

The catch: with only extremes you can't check whether the relationship is linear in the middle, so keep some mid-sized homes.

• Why: \text{Var}(\hat w) = \sigma^2 (X^\top X)^{-1} (Card 6), and spread in x makes X^\top X large.
• Next they'll ask: "Two correlated features?" → Correlation shrinks the useful spread, inflating variance by the VIF (Q39).

Q34 · Significant together, insignificant alone

A model with 8 marketing-channel features has an overall F-test p-value below 0.001, yet no individual coefficient is significant. How is that possible?

• Concept tested: the F-test vs individual t-tests; multicollinearity.
• Intuition: together the channels clearly matter, but they move together so much the model can't tell which one deserves credit.
• Answer: the F-test asks whether all slopes are zero; each t-test asks about one slope holding the others fixed. Correlated channels make each individual slope imprecise.

F

the F-statistic: tests whether all slopes are zero at once

• Why: collinearity inflates individual variances (VIF) but doesn't reduce the variance the group explains together.
• Next they'll ask: "What do you do?" → Report the joint effect, combine channels, or use ridge.

Q35 · Ranking leads only

Your model is used only to rank sales leads by predicted value. A reviewer flags that residuals are heteroscedastic and non-normal. Does it matter?

• Concept tested: which assumptions matter for which goal.
• Intuition: aim, error bars and exact tests each cost more assumptions; ranking needs only the aim.
• Answer: barely. For ranking you need good predictions on similar data — mostly linearity and a stable data distribution. Heteroscedasticity and non-normality affect standard errors and intervals, which ranking doesn't use.
• Why: L, A, R → unbiased coefficients; + I, E → correct SEs and BLUE; + N → exact small-sample tests.
• Next they'll ask: "When would it matter?" → If you report intervals on the predicted value or p-values on drivers.

Q36 · Heavy tails with 50,000 rows

The residual Q-Q plot shows heavy tails; n = 50{,}000. A reviewer says the coefficients are therefore invalid. Respond.

• Concept tested: what normality does and doesn't affect.
• Intuition: normality is about the shape of the noise; bias is about its center.
• Answer: the coefficients are fine — unbiasedness never used normality, and with 50,000 rows the central limit theorem makes the t-tests approximately valid. What is affected: prediction intervals, which rely on the noise's actual shape at any n.
• Why: \hat w is a weighted average of the errors, so it's close to normal for large n even when \varepsilon isn't.
• Next they'll ask: "How would you fix the intervals?" → Empirical residual quantiles, quantile regression, or a transform of y.

Q37 · $10k vs $150k misses

In your price model, residuals spread about ±$10k for cheap homes but ±$150k for expensive ones. What is this, what does it break, and how do you fix it?

• Concept tested: heteroscedasticity.
• Intuition: the noise grows with the price — a funnel.
• Answer: coefficients stay unbiased, but standard errors are wrong and OLS is no longer the most efficient estimator. Detect with a scale-location plot or Breusch–Pagan; fix with log(price), weighted least squares, or robust standard errors:

\widehat{\text{Var}}_{\text{robust}}(\hat{\mathbf w})

the robust ("sandwich") estimate of how much the fitted weights wobble

e_i^2

row i's squared residual, standing in for its own noise variance

\mathbf x_i\mathbf x_i^\top

row i's feature vector times its transpose: a p \times p matrix

• Why: the classical formula assumes \text{Var}(\varepsilon) = \sigma^2 I; the sandwich replaces it with each row's own squared residual (Card 6).
• Next they'll ask: "Which robust version?" → HC3 in small samples.

Q38 · Three years of daily sales

A daily sales model over 3 years gives the promo coefficient p = 0.001. Plotted in time order, the residuals show long runs above and below zero. Should you trust the p-value?

• Concept tested: autocorrelated errors.
• Intuition: today's luck spills into tomorrow, so 1,000 days aren't 1,000 independent clues.
• Answer: no. The coefficient is still unbiased (no lagged sales in the model), but its standard error is too small. Check Durbin–Watson; use Newey–West standard errors or model the time structure.

DW

the Durbin–Watson statistic: about 2 means no autocorrelation

\sum_t

add up over the days in time order

e_t

day t's residual

e_{t-1}

the previous day's residual

\hat\rho_1

the estimated correlation between one day's residual and the next (lag 1)

• Why: the effective sample size is much smaller than the number of days.
• Next they'll ask: "What if yesterday's sales is a feature?" → Then autocorrelated errors make OLS biased (Drill 25).

Q39 · The flipped sign

Adding "total rooms" next to "square feet" flips the square-feet coefficient negative and doubles its standard error, while predictions barely change. What happened?

• Concept tested: multicollinearity.
• Intuition: two witnesses telling the same story — you can't tell who contributed what.
• Answer: rooms and square feet carry almost the same information, so the model can't separate their effects. Measure it with the VIF:

\text{VIF}_j

the variance inflation factor: how many times feature j's slope variance is inflated by overlap

R_j^2

the R^2 from regressing feature j (say, square feet) on the other features

\text{Var}(\hat w_j)

how much feature j's fitted weight wobbles across samples

\text{Var}(x_j)

the spread of feature j

• Why: R_j^2 from regressing one feature on the others is near 1, so X^\top X is nearly singular.
• Next they'll ask: "Fix?" → Drop or combine features, use ridge, or collect more varied data. If you only need predictions, you may not need to.

Q40 · The U-shaped residuals

Residuals vs fitted values form a U: you underpredict cheap and expensive homes and overpredict the middle. What's wrong?

• Concept tested: a violated linearity assumption.
• Intuition: the line misses systematically — the truth bends and the line doesn't.
• Answer: the mean of y isn't linear in the features you gave the model. Add log or squared terms, splines or interactions, or switch to a nonlinear model. A formal check is the Ramsey RESET test.
• Why: if the mean is right, residuals have no pattern against fitted values; a U means the curvature landed in the residuals.
• Next they'll ask: "Does this bias the coefficients?" → Yes — linearity is one of the assumptions unbiasedness needs.

Q41 · $9k per year of school

Regressing salary on years of education gives $9k per extra year. A critic says that number is overstated. Name the mechanism, give the formula, and the likely direction.

• Concept tested: omitted-variable bias (Card 9).
• Intuition: a hidden factor's credit gets assigned to whichever visible feature it travels with.
• Answer: unmeasured ability raises salary and is correlated with schooling, so education's coefficient absorbs some of ability's effect:

\mathbb E[\hat\beta_{\text{educ}}]

education's fitted coefficient, averaged over repeated samples

\beta_{\text{educ}}

the true salary effect of one more year of school

\gamma_{\text{ability}}

the true salary effect of ability, the omitted variable

\delta

the slope from regressing ability on education: how much ability rises per extra year

Both \gamma and \delta are positive, so the estimate is too high.

• Why: the short regression's slope picks up everything correlated with education that also moves salary.
• Next they'll ask: "How would you get the causal effect?" → An instrument (e.g. school-leaving-age reforms), twins or sibling comparisons, or a natural experiment.

Q42 · Self-reported income

In your credit model, income is self-reported and noisy, while default is recorded exactly. A colleague says noise is noise. Which noise worries you more, and why?

• Concept tested: measurement error in x vs in y (Card 10).
• Intuition: a shaky ruler on x smears points sideways and flattens the line; a shaky ruler on y just adds vertical scatter.
• Answer: noise in income (x). It biases income's coefficient toward zero; noise in y would only widen standard errors.

\text{plim}\,\hat\beta

what income's fitted coefficient settles to with unlimited data

\beta

the true effect of income on default

\text{Var}(x^*)

the spread of true incomes x^* (star = the true, unseen value)

\text{Var}(u)

the spread of the reporting noise u

• Why: noise in x enters the denominator \text{Var}(x) but not the numerator \text{Cov}(x, y).
• Next they'll ask: "How would you fix it?" → Worked problem 1 below.

Q43 · 20% blank debt fields

20% of loan applicants leave "monthly debt" blank. An analyst fills the blanks with the average. What can go wrong, and what would you do instead?

• Concept tested: missing-data mechanisms.
• Intuition: why a value is missing decides what's safe to do — people with large debts may be the ones leaving it blank.
• Answer: mean-filling shrinks variance, weakens correlations, and hides a signal if missingness itself predicts default. Check the mechanism: MCAR → dropping is fine; MAR → multiple imputation or inverse-probability weights; MNAR → model the missingness, and at minimum add a "was missing" flag.
• Why: dropping or naive filling is only unbiased when missingness is unrelated to the outcome.
• Next they'll ask: "How would you tell MAR from MNAR?" → You can't fully from the data alone; compare missing vs non-missing rows on observed variables and run sensitivity analyses.

Q44 · 40 exact zeros

With comparable penalty strength, lasso set 40 of 50 coefficients exactly to zero, while ridge set none. Explain geometrically.

• Concept tested: why L1 produces sparsity.
• Intuition: a diamond's corners stick out along the axes, so the growing error ellipse usually touches a corner first.
• Answer: lasso's constraint region has corners where some weights are zero; ridge's is a sphere with no corners. In the simplest case (orthonormal features) lasso soft-thresholds:

\hat w_j^{\text{lasso}}

the lasso weight for feature j

\text{sign}(\hat w_j)

the sign of the OLS weight: +1 if positive, −1 if negative

\hat w_j

the plain OLS weight for feature j

\lambda

the penalty strength

\hat w_j^{\text{ridge}}

the ridge weight for feature j

• Why: subtracting a fixed amount hits zero; dividing by a factor never does (Card 11).
• Next they'll ask: "Downside of lasso?" → Among correlated features it picks one arbitrarily, and it shrinks the survivors too.

Q45 · Same data, different units

A ridge model shrinks "annual income in $" almost to zero but barely touches "income in $M" in an otherwise identical model. Same data, different units. Why?

• Concept tested: standardizing before regularizing.
• Intuition: the penalty charges per unit of weight; a feature in small units needs a huge weight, so it gets overcharged.
• Answer: coefficients scale inversely with their feature's units, and the penalty treats every coefficient's size equally. Standardize features first so the penalty is fair.

x

the income feature

c

the unit-conversion factor (e.g. dollars to millions of dollars)

\hat w_{\text{OLS}}

the plain OLS weight on that feature

w

the weight the penalty charges for

• Why: OLS is unit-equivariant; penalized regression isn't.
• Next they'll ask: "Penalize the intercept?" → Never.

Q46 · Defending \lambda

"How did you choose \lambda, and how would you defend that choice to a skeptical reviewer?"

• Concept tested: tuning by cross-validation.
• Intuition: try a range of dial settings and keep the one that predicts unseen data best.
• Answer: k-fold cross-validation over a log-spaced grid, choosing the \lambda with the lowest CV error — or, for a simpler and more stable model, the largest \lambda within one standard error of the best. Show the CV curve.
• Why: \lambda trades bias for variance; cross-validation estimates the resulting test-error curve directly.
• Next they'll ask: "Then can you report the CV error as your test error?" → No — you selected on it. Use a held-out test set or nested cross-validation.

Q47 · 300 neighborhoods

Your price model needs "neighborhood" with 300 levels; 120 of them have fewer than 5 sales. How do you include it?

• Concept tested: high-cardinality categorical features.
• Intuition: each neighborhood deserves its own offset, but you can't trust an offset estimated from three sales.
• Answer: options are 299 dummies with regularization, grouping rare levels, out-of-fold target encoding, or a mixed-effects model that shrinks small neighborhoods toward the overall mean.

\hat\alpha_j^{\text{shrunk}}

neighborhood j's shrunken average price: the mixed-model estimate

n_j

the number of sales in neighborhood j

\sigma^2

the noise variance of a single sale

\tau^2

how much the true neighborhood averages vary from one another

\bar y_j

neighborhood j's own average sale price (bar = mean)

\bar y

the overall average price across all neighborhoods

• Why: that's partial pooling: neighborhoods with few sales borrow strength from the rest.
• Next they'll ask: "Why out-of-fold for target encoding?" → Otherwise each row's own price leaks into its feature.

Q48 · Size, pool and their product

Coefficients: size $150/sq ft, pool +$20k, size \times pool +$40/sq ft. What's an extra 100 sq ft worth for a home with a pool? And what does the $150 mean?

• Concept tested: interpreting interactions.
• Intuition: a pool makes extra space worth more.
• Answer: with a pool, 100 \times (150 + 40) = $19,000. The $150 is the value per sq ft for homes without a pool.

\frac{\partial \hat y}{\partial\,\text{size}}

how fast the predicted price changes as you nudge size, holding the rest fixed: a slope, not a fraction to divide

w_{\text{size}}

the size weight: $150 per sq ft for homes without a pool

w_{\text{size}\times\text{pool}}

the interaction weight: +$40 per sq ft when there's a pool

• Why: the size slope depends on pool, so each main effect is read at the other variable equal to zero — which is why centering matters.
• Next they'll ask: "Drop the pool main effect since its p-value is high?" → No; keep main effects whenever the interaction stays.

Q49 · 15 new features

Adding 15 features moved R^2 from 0.71 to 0.73, but adjusted R^2 fell from 0.70 to 0.69. Keep them?

• Concept tested: R^2 vs adjusted R^2.
• Intuition: the new features didn't pay their rent.
• Answer: probably not. R^2 can't fall when you add features, even pure noise; adjusted R^2 charges for each one:

R^2_{\text{adj}}

adjusted R^2: charges each feature rent, so it can fall

R^2

the plain share of variance explained, which never falls when features are added

• Why: the 0.02 gain is about what noise features buy by chance (Drill 12).
• Next they'll ask: "Better check?" → Cross-validated error with and without the features.

Level 3 — Senior

Q50 · "OLS is BLUE, so why ridge?"

A reviewer objects: "OLS is BLUE — the best linear unbiased estimator. Nothing can beat it. Why are you using ridge?"

• Concept tested: the Gauss–Markov theorem and its limits (Card 8).
• Intuition: among all unbiased straight-edge rules OLS wobbles least — but a slightly off-center rule with a much steadier hand can land closer on average.
• Answer: Gauss–Markov says OLS has the lowest variance among linear unbiased estimators, under linearity, full rank, exogeneity, equal variance and uncorrelated errors. Ridge is biased, so it isn't in that competition, and it can have lower total error.

\text{Var}(\tilde{\mathbf w})

how much the competitor's weights wobble (a matrix)

\tilde{\mathbf w}

any other linear unbiased estimator of the weights (tilde = an alternative)

\sigma^2

the noise variance

(X^\top X)^{-1}

the matrix inverse of X^\top X; times \sigma^2 it is OLS's variance

DD^\top

D times its transpose (rows become columns): extra variance, never negative

D

the competitor's recipe minus OLS's recipe

\text{Var}(\hat{\mathbf w}_{\text{OLS}})

how much the OLS weights wobble

\succeq

"at least as large as", for matrices: the difference is positive semi-definite

• Why: any linear unbiased competitor is "OLS + D" with DX = 0, which adds DD^\top \succeq 0 to the variance.
• Next they'll ask: "When does OLS stop being best even among unbiased estimators?" → Unequal or correlated errors; then GLS is BLUE.

Q51 · Intervals that are too narrow

Your pipeline estimates the noise variance by maximum likelihood and builds 95% prediction intervals. On small datasets, the intervals cover noticeably fewer than 95% of new points. Why?

• Concept tested: the MLE of \sigma^2 is biased.
• Intuition: the fit hugs the training data, so its leftover scatter understates real noise.
• Answer: the MLE divides RSS by n, which is biased low by a factor (n - p)/n. On top of that, normal quantiles ignore the uncertainty in \hat\sigma. Use s^2 = \text{RSS}/(n - p) and t quantiles.

\mathbb E[\hat\sigma^2_{\text{MLE}}]

the maximum-likelihood noise variance, averaged over repeated samples

n

the number of rows

p

the number of fitted parameters

• Why: residuals are a projection of the noise into n - p dimensions (Card 7).
• Next they'll ask: "Is the MLE at least consistent?" → Yes; the bias vanishes as n grows.

Q52 · One honest scatterplot

A stakeholder wants one 2D scatterplot that "shows the effect of marketing spend after controlling for seasonality and price". How do you make a plot whose slope is exactly the multiple-regression coefficient?

• Concept tested: the Frisch–Waugh–Lovell theorem.
• Intuition: strip from both sales and spend everything seasonality and price explain, then plot what's left.
• Answer: an added-variable (partial regression) plot. Regress sales on the controls and keep residuals; regress spend on the controls and keep residuals; plot one against the other. Its slope equals spend's coefficient in the full model.

\hat\beta_1

spend's coefficient in the full model

\tilde{\mathbf x}_1

marketing spend with seasonality and price regressed out: what's left (tilde = residualized)

\tilde{\mathbf y}

sales with seasonality and price regressed out

M_2

the residual-maker: subtracts whatever the controls can explain

\mathbf x_1

the marketing-spend column

\mathbf y

the outcome column (sales)

I

the identity matrix: leaves a vector unchanged

X_2

the control columns (seasonality and price)

X_2^\top

X_2 transposed: rows become columns

(X_2^\top X_2)^{-1}

the matrix inverse of X_2^\top X_2: undoes it

• Why: OLS only uses the part of x_1 orthogonal to the other columns.
• Next they'll ask: "Where else is this used?" → Fixed effects (demeaning) and double machine learning.

Q53 · 150 features, 200 rows

With 200 rows and 150 correlated features, OLS has a terrible test RMSE while ridge's is half as large. Explain using bias and variance.

• Concept tested: why a biased estimator can win.
• Intuition: OLS fits 150 knobs to 200 noisy points and memorizes noise; ridge holds the knobs back.
• Answer: OLS is unbiased but its variance explodes when X^\top X is nearly singular. Ridge adds a little bias and removes far more variance. A \lambda > 0 that beats OLS on MSE always exists (Hoerl and Kennard, 1970).

\hat{\mathbf w}_{\text{ridge}}

the ridge weights

X

the feature matrix: 200 rows, 150 feature columns

X^\top

X transposed: rows become columns

(X^\top X + \lambda I)^{-1}

the matrix inverse of X^\top X + \lambda I; it exists whenever \lambda > 0

\lambda

the penalty strength

d_j

the j-th singular value of X: how firmly the data pin down direction j

• Why: weakly determined directions (small singular values d_j) carry the most variance, and ridge shrinks exactly those the most (Card 11).
• Next they'll ask: "Why not just drop features?" → Hard selection is unstable with correlated features; ridge degrades gracefully.

Q54 · "That's just my prior"

A Bayesian colleague says: "Your ridge regression is just my model with a prior." Make that precise.

• Concept tested: penalties as priors.
• Intuition: a penalty is a prior belief that weights are small.
• Answer: with Gaussian noise and a Gaussian prior on the weights, the posterior mode (MAP) is the ridge solution; a Laplace prior gives lasso.

\mathbf w

the weights (bold = vector)

\sim

"is distributed as"

N(0, \tau^2 I)

the prior: a normal distribution centered at 0, with variance \tau^2 for each weight

\tau^2

the prior's variance: how big you believe the weights can be

\hat{\mathbf w}_{\text{MAP}}

the posterior mode: the single most probable weight vector

\arg\min_{\mathbf w}

the \mathbf w that makes what follows as small as possible

\lVert\mathbf y - X\mathbf w\rVert^2

the sum of squared misses

\lVert\mathbf w\rVert^2

the sum of squared weights

• Why: the negative log-prior is added to the negative log-likelihood.
• Next they'll ask: "What does full Bayes add?" → A whole posterior: credible intervals for weights and predictions.

Q55 · Email vs SMS

Marketing claims email and SMS have the same effect per dollar. The coefficients are 0.42 and 0.31, each significant on its own. How do you test whether they differ?

• Concept tested: testing a linear combination of coefficients.
• Intuition: test whether the difference is zero, remembering the two estimates wobble together.
• Answer: two separate significant results say nothing about the difference. Test it directly, using their covariance:

t

the t-statistic for the difference between the two effects

\hat w_1

email's fitted effect per dollar (0.42)

\hat w_2

SMS's fitted effect per dollar (0.31)

\widehat{\text{Var}}(\hat w_1)

the estimated variance (squared SE) of email's coefficient

\widehat{\text{Var}}(\hat w_2)

the estimated variance of SMS's coefficient

\widehat{\text{Cov}}(\hat w_1, \hat w_2)

how the two estimates wobble together; large when email and SMS spend move together

• Why: the covariance comes from \sigma^2 (X^\top X)^{-1}; if email and SMS spend move together, it's large.
• Next they'll ask: "A trick with plain regression output?" → Regress y on x_1 and (x_1 + x_2); the coefficient on x_1 is then w_1 - w_2, with its own SE.

Q56 · The duplicated join

A data-join bug duplicated every row. The coefficients didn't change, but every p-value got much smaller. Which results do you trust?

• Concept tested: independence and effective sample size.
• Intuition: photocopying your evidence doesn't make it stronger.
• Answer: trust the coefficients, not the p-values. Duplicating rows shrinks SEs by about \sqrt{2} with no new information.

\text{SE}(\hat w)

the coefficient's standard error

\propto

"is proportional to"

\sqrt n

the square root of the number of rows

n

the number of rows; n \to 2n means every row was copied

• Why: the formulas assume independent rows; copies are perfectly correlated.
• Next they'll ask: "Real-world version?" → Many rows per user or session: cluster standard errors by user.

Q57 · Dollars to thousands

After converting price from dollars to thousands of dollars, a teammate panics that coefficients changed. What changed, what didn't, and is the answer different for the lasso model?

• Concept tested: scale equivariance.
• Intuition: changing units doesn't change the world.
• Answer: dividing y by 1,000 divides every coefficient and the RMSE by 1,000; R^2, t and p-values don't change. Rescaling a feature by c divides its coefficient by c. Lasso and ridge are different: unless features are standardized, rescaling one changes how hard it's penalized, and the selected set can change.
• Why: OLS is scale-equivariant; penalties aren't.
• Next they'll ask: "Adding a constant to x?" → Only the intercept changes.

Q58 · 2.0 vs 5.0

One analyst regresses sales on ad spend and gets a slope of 2.0. Another regresses ad spend on sales, inverts the slope, and gets 5.0. Who's right?

• Concept tested: regression is not symmetric.
• Intuition: each regression minimizes misses in its own direction — vertical in one, horizontal in the other.
• Answer: neither inversion is valid. The two slopes multiply to r^2, here 2.0 \times (1/5.0) = 0.4, so r \approx 0.63:

b_{y\mid x}

the slope from regressing sales on ad spend (2.0)

b_{x\mid y}

the slope from regressing ad spend on sales (1/5.0)

r

the correlation between spend and sales

s_y

the standard deviation of sales

s_x

the standard deviation of ad spend

• Why: they agree only when the fit is perfect.
• Next they'll ask: "Any use for the reverse slope?" → It bounds a slope attenuated by measurement error in x (Worked problem 1).

Q59 · "Correlation with residuals is zero"

To prove that price isn't endogenous in a demand model, an analyst shows that the correlation between price and the residuals is 0.000. Convinced?

• Concept tested: exogeneity can't be tested with OLS residuals.
• Intuition: OLS chose the line precisely so residuals look uncorrelated with price — it's checking its own homework.
• Answer: no. X^\top e = 0 holds by construction for any data. Exogeneity is about the errors, which you never see.
• Why: it's the normal equation (Card 2).
• Next they'll ask: "Then how?" → Argue it with a causal diagram, test it with an instrument (Hausman), or randomize price.

Q60 · Higher prices, more sales

Regressing units sold on price from historical data gives a positive slope: higher prices, more sales. Is demand upward-sloping?

• Concept tested: endogeneity from simultaneity.
• Intuition: prices were raised because demand was high, so you're seeing the demand shifts, not the demand curve.
• Answer: almost certainly not. Price responds to demand shocks (holidays, trends), so \text{Cov}(\text{price}, \varepsilon) > 0 and the slope is biased upward. Causes of endogeneity: omitted variables, measurement error in x, simultaneity, selection.

\text{plim}\,\hat\beta

what the fitted price slope settles to with unlimited data

\beta

the true slope of the demand curve

\text{Cov}(\text{price}, \varepsilon)

how price moves with demand shocks \varepsilon; positive when prices rise in busy times

• Why: each cause makes \text{Cov}(x, \varepsilon) \ne 0, which breaks Card 5.
• Next they'll ask: "How would you estimate the real demand curve?" → A cost-side instrument that moves price but not demand (Q61), or randomized price tests.

Q61 · Tutoring vouchers

You want the effect of tutoring hours on test scores, but motivated students both tutor more and score higher. A district randomly mailed free-tutoring vouchers to some families. How do you use them?

• Concept tested: instrumental variables.
• Intuition: the voucher is a lever that moves tutoring hours but can only reach test scores through tutoring — a natural experiment.
• Answer: use the voucher as an instrument for hours (two-stage least squares). With one instrument:

\hat\beta_{\text{IV}}

the instrumental-variables estimate of tutoring's effect per hour

It needs relevance (vouchers change hours — testable) and exclusion (vouchers affect scores only via tutoring — argued, never tested).

• Why: random assignment makes the voucher independent of motivation, so the confounder drops out of both covariances (Drill 6).
• Next they'll ask: "Whose effect is this?" → The compliers' — students who tutor because of the voucher (the local average treatment effect). And with a weak first stage, IV drifts toward OLS (Drill 15).

Q62 · Trained on homes that sold

Your list-price recommendation model is trained only on homes that sold within 90 days. What's the risk, and how do you fix it?

• Concept tested: selection on the outcome.
• Intuition: you only see the listings that cleared; the overpriced ones vanished from your data.
• Answer: inclusion depends on price relative to value, so among included rows the errors depend on x and the slope is biased; the model also never learns what "too high" looks like. Fix by adding the unsold listings with a time-to-sale or survival model, a Heckman correction, or inverse probability weighting.
• Why: after selection, E[\varepsilon \mid x, \text{sold}] \ne 0.
• Next they'll ask: "When is selection harmless?" → When it depends only on x — e.g. training only on homes under 3,000 sq ft (Drill 4).

Q63 · Riskier overall, safer in every band

In a lending pilot, the treated group has a higher default rate overall than control, yet within every credit-score band the treated group defaults less. How?

• Concept tested: Simpson's paradox.
• Intuition: the pilot was mostly offered to low-score applicants, who default more regardless.
• Answer: the treated group is concentrated in risky bands, so pooling mixes the treatment effect with the score mix. Within bands — holding the confounder fixed — treatment helps.
• Why: pooled difference = within-band effect + a between-band composition term driven by how treatment was allocated.
• Next they'll ask: "Which answer is right?" → The within-band one, because score drove both treatment and default (a confounder). If score were caused by treatment, you wouldn't control for it.

Q64 · "Control for job title, to be safe"

Estimating a training program's effect on salary, a colleague adds "job title after the program" as a control "to be safe". Good idea?

• Concept tested: bad controls — mediators and colliders.
• Intuition: the program raises salary partly by getting people promoted; holding title fixed blocks that path.
• Answer: no. Job title is a mediator, so controlling for it removes the part of the effect that works through promotion. If title also depends on unobserved ability, it's a collider too, and conditioning on it creates bias (Drill 8: a true effect of 1 becomes 0).
• Why: only pre-treatment confounders belong as controls; post-treatment variables change the question.
• Next they'll ask: "How do you decide in general?" → Draw the causal diagram first.

Q65 · Tenure increases default risk?

In your credit model, more years at the current job increases predicted default risk. The business says that's nonsense. Investigate.

• Concept tested: diagnosing a "wrong" sign.
• Intuition: the coefficient is telling a story about something you didn't put in the model, or something you did.
• Answer: check in order: the one-variable relationship (is it negative on its own?); collinearity with age or income (VIF); confounding or selection (e.g. long-tenure applicants were approved under looser rules); suppressor effects; and finally the pipeline (units, joins, sign conventions).
• Why: each of these changes what "holding others fixed" means or biases the estimate.
• Next they'll ask: "If it's real but unintuitive, do you ship it?" → Only with a plausible explanation and stability across time and segments; regulators will ask.

Q66 · Drop until everything is significant

A feature has p = 0.06. Your manager wants it removed, then the model refit, then the next weakest removed, until every p-value is below 0.05.

• Concept tested: post-selection inference.
• Intuition: 0.05 is a convention, not a cliff — and if you keep dropping until things look significant, they will.
• Answer: push back. Stepwise selection by p-value makes the final p-values and intervals invalid and the coefficients biased away from zero. Decide on theory and effect size; use cross-validation or lasso for prediction; pre-specify the model for inference.
• Why: the reported p-values assume the model was fixed before seeing the data.
• Next they'll ask: "Then how do you get valid inference after selection?" → Sample splitting (select on one half, estimate on the other) or post-selection inference methods.

Q67 · 10,000 features, 1,000 patients

A dataset has 10,000 gene-expression features and 1,000 patients. A colleague runs OLS and gets a perfect training fit. What's going on, and what would you do?

• Concept tested: p > n.
• Intuition: more knobs than data points means infinitely many perfect fits.
• Answer: with p > n, X^\top X is singular and OLS has no unique solution; the perfect fit is pure memorization. Use lasso, ridge or elastic net, reduce dimension with PCA, or use domain-driven selection, and judge it on held-out data.

\text{rank}(X^\top X)

the number of independent directions in X^\top X; below p means no unique solution

n

the number of patients (1,000)

p

the number of features (10,000)

• Why: any of infinitely many weight vectors reproduces the training y exactly.
• Next they'll ask (research): "Which solution does gradient descent find, started from zero?" → The minimum-norm one, X^+ y — a form of implicit regularization, and part of why huge neural networks still generalize.

Q68 · 18k offline, 45k in production

Offline RMSE was $18k. After launch, production RMSE is $45k. What do you investigate?

• Concept tested: why validation can lie.
• Intuition: something about the test wasn't like the real world.
• Answer: in order: leakage (a feature that wasn't available at prediction time); train/serve skew (the feature computed differently in production); drift (the market moved since training); extrapolation (production inputs outside the training range); and a random split that let near-duplicate homes sit in both train and test.
• Why: validation error only estimates production error when the distribution and the feature pipeline are the same.
• Next they'll ask: "How do you catch it before launch?" → Time-based splits, a feature-availability audit, shadow deployment, and monitoring of inputs and residuals.

Q69 · A million models

You need the leave-one-out CV error for a million candidate linear models, each on 10,000 rows. Refitting each model 10,000 times is impossible. What do you do?

• Concept tested: the LOOCV shortcut (Card 13).
• Intuition: each point's left-out error is its residual, inflated by how hard it pulled the fit toward itself.
• Answer: fit each model once and use the leverages:

e_{(-i)}

row i's leave-one-out residual: its miss when the model is fit without it

e_i

row i's ordinary residual from the full fit

h_{ii}

row i's leverage

n

the number of rows (10,000)

\sum_{i=1}^{n}

add up over all n rows

• Why: removing a row is a rank-one update to X^\top X, which Sherman–Morrison solves in closed form.
• Next they'll ask: "Generalized cross-validation?" → Replace each h_{ii} with its average, p/n.

Q70 · 50 stores, 50,000 transactions

An experiment covers 50 stores, each with about 1,000 daily transactions, and the errors are clearly heteroscedastic. Leadership wants a 95% interval on the effect. How do you build it?

• Concept tested: valid standard errors when assumptions fail.
• Intuition: if you can't trust the formula, trust the data's own variability — at the level where things are actually independent.
• Answer: cluster-robust standard errors by store (transactions within a store share shocks), which also handle heteroscedasticity. With only 50 clusters, use a wild cluster bootstrap or analyze store-level aggregates.

\widehat{\text{Var}}_{\text{cluster}}(\hat{\mathbf w})

the cluster-robust estimate of how much the fitted weights wobble

\sum_{g=1}^{G}

add up over every store (cluster) g, from 1 to G = 50

X_g

store g's rows of the feature matrix

\mathbf e_g

store g's residuals as a vector

• Why: the effective sample size is closer to 50 stores than to 50,000 transactions (the Moulton factor, Card 16).
• Next they'll ask: "Bootstrap instead?" → Yes, but resample whole stores, not transactions.

Level 4 — Research and quant

These questions give you a data-generating process (DGP) and ask what a regression will estimate. There's one method, and every problem below uses it.

The recipe: Cov over Var, then expand.

1. Write what OLS converges to. One feature: b \to \text{Cov}(Y, X_{\text{obs}}) / \text{Var}(X_{\text{obs}}). Several: b \to \Sigma^{-1}c, with \Sigma the features' covariance matrix and c their covariances with Y.
2. Substitute the DGP and expand. Covariance is bilinear; independence kills cross terms.
3. Compare with the true coefficient. The gap is the bias.
4. Name the mechanism. Attenuation, omitted variable, selection, collider, simultaneity.
5. Say how you'd detect it, fix it, and whether it matters for the goal. Prediction and causal estimation want different things.

Worked problem 1 — Regression under measurement error

The classic form of a question reported from an Upstart data science interview, reconstructed from a candidate's memory; the real wording may differ.

DGP: Y = Z + X, with Z \sim N(0, 1) and X \sim N(0, 1) independent, so Y \sim N(0, 2). You don't observe X; you observe X' = X + u, with u \sim N(0, \sigma^2) independent of everything. You fit Y = a + bX' + \varepsilon. What is E[\hat b]? How does it compare with the true relationship, and which way is the bias?

Intuition first: noise in x stretches the cloud of points sideways but not upward, so the best-fitting line through it gets flatter.

Solution.

b

the OLS slope of Y on the noisy X', as the data grow

Y

the outcome, Z + X

X'

the noisy measurement you observe, X + u

X

the true feature, never observed

Z

a separate cause of Y, independent of X

u

the measurement noise added to X

\text{Cov}

covariance: how two things move together

\text{Var}

variance: how much something spreads around its mean

1 + \sigma^2

the spread of the measured X': the true variance 1 plus the noise variance \sigma^2

Every cross term (Z with X, Z with u, X with u) is zero by independence; the intercept a \to 0.

What to say out loud: the true effect is 1, the regression estimates 1/(1 + \sigma^2). It's biased toward zero — attenuation — by the reliability ratio \lambda = \text{signal} / (\text{signal} + \text{noise}). With \sigma^2 = 1, half the effect disappears. The sign never flips.

Z is a distractor: independent of X, it only adds noise to Y, lowering R^2 to 1 / (2(1 + \sigma^2)) without biasing anything.

Follow-ups that separate senior from research-level answers:

• "So is \hat b wrong?" Depends on the goal. To predict Y from the same noisy X' in production, 1/(1 + \sigma^2) is exactly the best linear predictor. It's biased only as an estimate of the structural effect of X.
• "What if the noise were in Y?" No bias, just wider standard errors.
• "Can you bound the truth without knowing \sigma^2?" Regress X' on Y and invert: \text{Var}(Y)/\text{Cov}(X', Y) = 2. With classical errors, 1/(1 + \sigma^2) \le \beta \le 2 — the Frisch bounds (Card 10).
• "Other regressors?" The error leaks: the noisy feature is attenuated more, and correlated features absorb its effect (Drill 7).
• "Always toward zero?" For classical error, yes. For Berkson error, no bias at all (Drill 3).
• "Does normality matter?" No; only independence and finite variances.

How to identify it in real data: it never shows in residual plots. Know how the feature was measured (self-reports, proxies, sensors); run a reliability study (two measurements: \text{Var}(X'_1 - X'_2) = 2\sigma^2); watch whether the coefficient grows with a cleaner measure or an instrument.

How to mitigate it:

1. Measure better, or average k repeated measurements — noise variance falls to \sigma^2/k.
2. Disattenuate: divide \hat b by \hat\lambda = (\text{Var}(X') - \hat\sigma^2) / \text{Var}(X').
3. Instrument with a second independent measurement X'': \hat b_{\text{IV}} = \text{Cov}(Y, X'') / \text{Cov}(X', X'') \to 1.
4. Errors-in-variables models: Deming regression or total least squares, SIMEX, structural equation models.
5. For pure prediction with the same noisy input, leave it alone.

In lending, the textbook case is self-reported income: noisy, so its coefficient is attenuated; verified income can serve as a cleaner measure or an instrument.

Worked problem 2 — Height and basketball performance

The second question from the same reported interview. It tests whether you can see selection behind a "null" result.

Setup: regressing \text{performance} = \alpha + \beta \cdot \text{height} + \varepsilon on NBA players gives \hat\beta \approx 0, yet height obviously matters. (1) What does \hat\beta \approx 0 mean? (2) Can the true causal effect still be non-zero? (3) How could omitted-variable bias explain it? (4) If true = estimated + bias, what drives the bias? (5) How should you think about P(\text{NBA} \mid \text{performance})?

Intuition first: everyone in the NBA got there by being tall or exceptionally skilled, so among them height and skill trade off — and leaving skill out lets that trade-off cancel height's real effect.

1. What \hat\beta \approx 0 means. Among NBA players, taller players don't have higher average performance. It's an association conditional on being in the league, not a statement about what height does. Check the interval too: NBA heights span a narrow range, so \text{Var}(\text{height}) is small and the \text{SE} = \sigma / \sqrt{n \cdot \text{Var}(x)} is large — "close to zero" may just mean "imprecise".

2. Can the true effect be non-zero? Yes. In the simulation the true effect is 0.5 everywhere, and the within-NBA slope still comes out near zero.

3. The omitted-variable explanation.

\alpha

the baseline performance

\beta

the true effect of height on performance (0.5 in the simulation)

\gamma

the effect of skill on performance (positive)

\mathbb{E}[\hat\beta]

the fitted height slope, averaged over repeated NBA samples

\delta

the slope of skill on height among NBA players (negative)

\varepsilon

everything else that moves performance

\delta is the slope of skill on height in the sample. Skill helps (\gamma > 0). Inside the NBA \delta < 0: a short player only makes it with exceptional skill. So \gamma\delta < 0 and can cancel \beta (Card 9).

4. What drives the bias term. With true = estimated + bias, the bias is -\gamma\delta: omitted skill and athleticism, negatively correlated with height among selected players. Also: position (centers and guards are judged on different stats), the choice of performance metric (rebounds favor height, assists don't), and range restriction. Adding a skill measure recovers the effect: in the simulation, the height slope returns to about 0.5.

5. P(\text{NBA} \mid \text{performance}) and the selection behind it. The sample exists only for players with high P(\text{NBA} \mid \text{height}, \text{skill}). Making the league is a common effect of height and skill, so restricting to NBA players conditions on a collider, creating a negative height–skill correlation out of nothing — Berkson's paradox. Height mostly works at the door: P(\text{NBA} \mid \text{height}) rises steeply with height (the extensive margin), while its effect on performance once inside (the intensive margin) looks small.

• Don't confuse P(\text{NBA} \mid \text{height}) with P(\text{height} \mid \text{NBA}): almost all NBA players are tall, but most tall people aren't in the NBA.
• If teams select largely on performance itself, you're selecting on (nearly) the outcome, which flattens slopes on its own (Drill 4).

How to fix it: analyze the pre-selection population (all college or draft-eligible players) and model P(\text{NBA} \mid \text{height}, \text{skill}) with logistic regression; control for skill; use a Heckman selection model if something affects selection but not performance; compare within positions; report bounds otherwise.

Remember: Cov over Var, then expand. Noise in x → attenuation by \lambda. A surprising null inside a selected group → ask what the selection depended on.

Level 4 drills

Thirty-four DGP problems: Drills 1–12 cover the core mechanisms, 13–34 extend them to causal designs, experiments, time series, models beyond OLS, and estimation theory. Cover the answer, run the recipe on paper, then check. Every answer was confirmed by simulating two million draws. All variables are N(0, 1) and independent unless stated.

Drill 1 · The variable you don't have

You're estimating how loan size affects losses, but you don't observe borrower wealth, which affects losses too and tends to be higher for bigger loans. The interviewer simplifies: "Let Y = X + Z, with X and Z each N(0, 1) and \text{Corr}(X, Z) = \rho. You regress Y on X alone. What does the slope converge to? Which way is it biased, and what determines the direction?"

Concept tested: omitted-variable bias.

• Answer: b \to 1 + \rho (1.5 when \rho = 0.5).
• Intuition: Z rides along with X, so X gets credit for Z's work.
• Why: \text{Cov}(X + Z, X) / \text{Var}(X) = 1 + \rho. Sign of bias = sign of Z's effect \times sign of \rho (Card 9).

b

the slope from regressing Y (losses) on X alone, as the data grow

X

loan size, the feature you keep

Z

borrower wealth, the omitted variable

\rho

the correlation between X and Z

Drill 2 · A noisy outcome

Your outcome is customer satisfaction from a survey — noisy, but the feature (wait time) is logged precisely. "Let Y = X + Z, but you observe Y' = Y + v, with v independent noise. Is the slope of Y' on X biased? What, if anything, gets worse?"

Concept tested: measurement error in y vs in x.

• Answer: b \to 1, unbiased; only the standard error grows.
• Intuition: vertical scatter doesn't tilt a line; it just makes it harder to pin down.
• Why: \text{Cov}(Y + v, X) = \text{Cov}(Y, X) because v is independent of X.

Drill 3 · The dose on the prescription

In a clinical dataset you record the prescribed dose X_r, but patients' actual intake scatters around it: X = X_r + u, with u independent of X_r. The outcome is Y = X + \varepsilon. "You regress Y on the recorded dose. You've just told me measurement error attenuates slopes. Is this slope attenuated too? Why or why not?"

Concept tested: Berkson error vs classical measurement error.

• Answer: b \to 1, unbiased.
• Intuition: the noise scatters around what you recorded, so it averages out at every recorded value instead of stretching the x-axis.
• Why: \text{Cov}(X_r + u + \varepsilon, X_r) / \text{Var}(X_r) = 1. In Worked problem 1 the noise was independent of the truth; here it's independent of the recording.

b

the slope of the outcome on the recorded dose

X_r

the recorded (prescribed) dose

u

how far actual intake strays from the prescription

\varepsilon

the outcome's own noise

\perp

"is independent of"

Drill 4 · Which rows you keep

A colleague fits a model only on "profitable" accounts. Another fits only on "large" accounts. The interviewer formalizes it: "Y = X + \varepsilon, both standard normal. (a) You keep only rows with Y > 0. (b) Separately, you keep only rows with X > 0. What happens to the slope in each case, and why are they different?"

Concept tested: selection on the outcome vs selection on a feature.

• Answer: (a) b \approx 0.53, biased toward zero. (b) b \to 1, unbiased.
• Intuition: cutting on y removes the unlucky low points mostly at small x, lifting the left end of the cloud; cutting on x just narrows the window you look through.
• Why: after truncating on Y, E[\varepsilon \mid X, \text{kept}] depends on X; after selecting on X, E[\varepsilon \mid X] = 0 still holds.

Drill 5 · "No relationship"

An analyst reports that a sensor reading X has no relationship with machine stress Y: the slope is zero and the correlation is zero. You know physically that Y = X^2, with X \sim N(0, 1). "Explain how both can be true, and how you'd have caught it."

Concept tested: zero correlation does not mean independence.

• Answer: b \to 0.
• Intuition: a symmetric U has no overall tilt.
• Why: \text{Cov}(X^2, X) = E[X^3] = 0 for a symmetric X. Zero slope and zero correlation don't mean independence; the residual plot shows the U.

b

the slope of stress on the sensor reading

X

the sensor reading, standard normal

\mathbb E

the average over the distribution of X

Drill 6 · The hidden push, and a lever

An unobserved factor U (say, local demand) raises both your ad spend X and your sales Y. A randomized budget shock Z also moves spend. "Let X = Z + U and Y = X + U, all inputs standard normal and independent. What does OLS of Y on X give? What does IV using Z give? Why does Z work?"

Concept tested: confounding and instrumental variables.

• Answer: OLS: b \to 1.5. IV using Z: b \to 1.
• Intuition: U pushes X and Y up together, so OLS mistakes U's push for X's effect; Z moves X without touching U.
• Why: OLS: \text{Cov}(X + U, X) / \text{Var}(X) = (2 + 1)/2. IV: \text{Cov}(Y, Z) / \text{Cov}(X, Z) = 1/1.

\hat b_{\text{OLS}}

the plain OLS slope of sales on ad spend

X

ad spend

U

unobserved local demand, which pushes both spend and sales up

\hat b_{\text{IV}}

the slope using the budget shock Z as an instrument

Y

sales

Z

the randomized budget shock: moves spend, not demand

Drill 7 · One noisy feature among two

Your credit model uses self-reported income (noisy) and verified assets (clean), which are correlated. "Let Y = X_1 + X_2 + \varepsilon, \text{Corr}(X_1, X_2) = 0.5, unit variances. You observe X_1' = X_1 + u, \text{Var}(u) = 1, and regress Y on X_1' and X_2. What are both coefficients? Does the noise only hurt X_1?"

Concept tested: measurement error spreading across correlated regressors.

• Answer: b_1 \to 0.43, b_2 \to 1.29.
• Intuition: the model trusts the noisy feature less and hands its credit to the clean feature that travels with it.
• Why: solve \Sigma b = c with \Sigma = [[2, 0.5], [0.5, 1]] and c = [1.5, 1.5]. The noisy feature is attenuated more than it would be alone (0.43 vs 0.5).

\Sigma

the covariance matrix of the two observed features, X_1' and X_2

\mathbf c

each feature's covariance with Y

\mathbf b

the two OLS coefficients, b_1 and b_2

\Sigma^{-1}

the matrix inverse of \Sigma: undoes it

Drill 8 · The control that erases the effect

Studying whether a feature X drives engagement Y, a colleague adds a "user health score" C as a control — but C is computed from both X and Y. "Let Y = X + \varepsilon and C = X + Y + v, all standard normal. Regress Y on X and C. What do you get for X's coefficient, and why?"

Concept tested: conditioning on a collider.

• Answer: b_X \to 0, b_C \to 0.5.
• Intuition: C already contains both X and Y; once you hold C fixed, a higher X implies a lower Y, which cancels X's real effect.
• Why: \Sigma = [[1, 2], [2, 6]], c = [1, 3]; solving gives (0, 0.5). Never control for something caused by both.

\Sigma

the covariance matrix of X and C

X

the feature you're studying

C

the "user health score", built from both X and Y: a collider

\mathbf c

the covariances of X and of C with Y

\mathbf b

the two coefficients, on X and on C

Drill 9 · Tall parents

"Parent and child heights are standardized and have correlation 0.5. A parent is 2 standard deviations above average. What's your best prediction for the child? Now regress parent on child — does the same thing happen in reverse? What does that tell you about causation?"

Concept tested: regression to the mean.

• Answer: predicted child: +1 SD.
• Intuition: part of an extreme value is luck, and luck doesn't inherit.
• Why: in standard units, b = r = 0.5. Regress parent on child and you also get 0.5 — so it's not a causal pull; it's what imperfect correlation does in both directions.

\hat z_{\text{child}}

the predicted child height, in standard units

r

the parent–child height correlation, 0.5

z_{\text{parent}}

the parent's height in standard units (SDs above average)

\hat z_{\text{parent}}

the predicted parent height, given the child's

z_{\text{child}}

the child's height in standard units

Drill 10 · Noise that grows with x

Spending varies far more among high-income customers. "Let Y = X + \lvert X \rvert \cdot \varepsilon with X, \varepsilon standard normal. Is the OLS slope biased? Is the classical standard error right? If not, by roughly how much is it off, and in which direction?"

Concept tested: heteroscedasticity and robust standard errors.

• Answer: b \to 1, unbiased, but the classical SE is too small by about \sqrt{3} \approx 1.7\times (simulation: 1.65\times).
• Intuition: the noisiest points are the high-leverage ones at large |x|, exactly where the classical formula assumes they're calm.
• Why: true \text{Var}(\hat b) \approx E[X^4]/n = 3/n; the classical formula assumes 1/n. Robust HC SEs fix it (Card 6).

\text{Var}_{\text{true}}(\hat b)

the slope's actual sampling variance

\hat b

the OLS slope of spending on income

X

income, standard normal

\text{Var}(\varepsilon\mid X)

the noise variance at a given X: here X^2, growing with \lvert X \rvert

n

the number of rows

\text{Var}_{\text{classical}}(\hat b)

the slope variance the classical formula assumes

Drill 11 · Shrinkage by hand

"Assume orthonormal features, X^\top X = I, so OLS gives \hat w = X^\top y. Write down the ridge estimate and the lasso estimate in terms of \hat w. Which one can produce exact zeros, and what is the bias and variance of ridge?"

Concept tested: the closed forms of ridge and lasso.

• Answer: ridge: \hat w = \hat w_{\text{OLS}} / (1 + \lambda), biased toward zero, variance cut by (1 + \lambda)^2. Lasso: \hat w = \operatorname{sign}(\hat w_{\text{OLS}}) \cdot \max(\lvert \hat w_{\text{OLS}} \rvert - \lambda/2, 0).
• Intuition: ridge scales every weight down by the same factor; lasso subtracts a fixed amount and stops at zero.
• Why: with X^\top X = I the problem separates by coordinate (Card 11).

Drill 12 · R^2 from nothing

A junior analyst adds 20 random columns to a 101-row dataset as a "sanity check" and is alarmed when R^2 comes out around 0.2. "If y is unrelated to all k features and you have n rows, what R^2 do you expect? What should they look at instead?"

Concept tested: R^2 inflation from noise features.

• Answer: E[R^2] = k / (n - 1). 20 noise features, 101 rows → about 0.20 (simulation: 0.198).
• Intuition: every feature gets to bend the fit a little toward the noise.
• Why: each fitted parameter absorbs about 1/(n - 1) of the variance by chance. That's why adjusted R^2 and held-out evaluation exist.

\mathbb E[R^2 \mid \text{no signal}]

the average R^2 when y is unrelated to every feature

k

the number of noise features (20)

n

the number of rows (101)

Causal designs — Drills 13–18

Drill 13 · Fixed effects, and their hidden cost

You have 5 years of data per customer and want the effect of app usage X on spend Y. Each customer has a fixed trait a_i (affluence) that drives both. The interviewer writes the model below. "What does pooled OLS give? What does a fixed-effects (within-customer) regression give? Now suppose usage is logged with noise of variance 0.5 — which estimator suffers more, and why?"

Concept tested: fixed effects remove time-invariant confounders but amplify attenuation.

Y_{it}

customer i's spend in year t

X_{it}

customer i's app usage in year t

a_i

customer i's fixed affluence: the same every year

\varepsilon_{it}

spend noise for customer i in year t

u_{it}

the part of usage not driven by affluence

a

the affluence terms a_i, across customers

u

the usage shocks u_{it}

\varepsilon

the spend noise \varepsilon_{it}

\sim

"is distributed as"

N(0,1)

a standard normal distribution: mean 0, variance 1

• Answer: pooled OLS → 1.5. Fixed effects (demean within each person) → 1. Now add measurement noise of variance 0.5 to X: pooled → 1.20, fixed effects → 0.67.
• Intuition: comparing each person with themselves removes everything constant about them — but it also removes most of the real variation in X, while the noise stays.
• Why: demeaning cancels a_i. The noise survives demeaning but the signal shrinks, so the reliability ratio falls from 2/2.5 = 0.8 (pooled) to 1/1.5 = 0.67 (within).

\bar Y_i

customer i's average spend over the 5 years (bar = mean)

\beta

the true effect of usage on spend (1)

\bar X_i

customer i's average usage

\bar\varepsilon_i

customer i's average noise

\lambda_{\text{within}}

reliability after demeaning: the share of within-customer usage spread that is real

\text{Var}(\tilde u)

the spread of real within-customer usage changes (tilde = demeaned)

\text{Var}(\tilde v)

the spread of the demeaned measurement noise

\lambda_{\text{pooled}}

reliability without demeaning (0.8)

Drill 14 · The launch in one region

A new checkout flow launched in one region (treated) but not another (control). You have conversion rates for both regions before and after. The interviewer writes the model below. "What does \hat\tau estimate, how do you compute it from four averages, and what must be true for it to be the causal effect?"

Concept tested: difference-in-differences and parallel trends.

Y

the conversion rate

\alpha

the control region's conversion before launch (the baseline)

\gamma

how much the treated region differs before launch

\delta

the before-to-after change in the control region

\tau\,(\text{Treated}\times\text{Post})

the treatment effect \tau, switched on only for the treated region after launch

\varepsilon

noise

• Answer: \hat\tau is the change in the treated group minus the change in the control group. It's unbiased if, without treatment, both groups would have followed parallel trends.
• Intuition: the control group's change shows what would have happened to the treated group anyway.
• Why: the first difference removes each group's fixed level; the second removes shocks common to both periods.

\hat\tau

the estimated treatment effect (difference-in-differences)

\bar Y_{T,\text{post}}

the treated region's average conversion after launch

\bar Y_{T,\text{pre}}

the treated region's average conversion before launch

\bar Y_{C,\text{post}}

the control region's average conversion after launch

\bar Y_{C,\text{pre}}

the control region's average conversion before launch

• Next they'll ask: "How do you check parallel trends?" → Plot pre-period trends and run an event study. "Staggered rollouts?" → The two-way fixed-effects regression can put negative weights on some groups' effects (Goodman-Bacon, 2021); use estimators built for staggered timing.

Drill 15 · Ten weak instruments

To fix confounding, a colleague uses ten instruments, each only weakly related to X, with n = 500. The model is below, with confounder U and each \pi_j = 0.05. "OLS is biased. Is 2SLS fine now? Which way is it biased, roughly by how much, and how would you diagnose the problem?"

Concept tested: weak-instrument bias.

X

the treatment whose effect you want

Z^\top\boldsymbol\pi

the ten instruments Z, each times its weak strength \pi_j = 0.05, added up

U

the confounder: moves both X and Y

Y

the outcome

• Answer: OLS → about 1.98. 2SLS → about 1.41: pulled toward OLS. Its relative bias (0.42) is close to 1/F (0.46), with first-stage F \approx 2.2.
• Intuition: with weak instruments the first stage mostly fits noise — including the confounder — so 2SLS starts to act like OLS.
• Why: Staiger and Stock's approximation:

\mathbb E[\hat\beta_{\text{2SLS}}]

the 2SLS estimate averaged over repeated samples

\beta

the true effect (1)

\text{plim}\,\hat\beta_{\text{OLS}}

what OLS settles to with unlimited data (about 1.98)

F

the first-stage F-statistic: how strongly the instruments move X

• Next they'll ask: "What F is enough?" → The old rule is F > 10; for a valid 5% t-test with one instrument, recent work puts it near 105. "With one weak instrument?" → The estimate has no finite mean and very heavy tails (in simulation its median was 1.02, almost unbiased); use Anderson–Rubin tests.

Drill 16 · Controlling for a blurry confounder

You can't measure customer intent C, but you have a survey score P = C + v that tracks it noisily (\text{Var}(v) = 1). "Y = X + C + \varepsilon with \text{Corr}(X, C) = 0.5, all unit variance. What's X's coefficient with no control, controlling for P, and controlling for C itself? Does the proxy solve the problem?"

Concept tested: noisy proxy controls.

• Answer: no control: 1.5. Control for P: 1.29. Control for C itself: 1.0. The proxy removes only about 43% of the bias.
• Intuition: a blurry measurement of the confounder only partly blocks the leak.
• Why: the part of C that P misses is still correlated with X, so it keeps leaking into X's coefficient (Card 9 with C replaced by its unexplained part).

Drill 17 · Controlling for the path

A discount X raises spend Y directly, and also by increasing visits M, which raise spend. "Let M = X + v and Y = M + X + \varepsilon. What do you get regressing Y on X, and on X and M together? Which is 'right'?"

Concept tested: mediators, direct vs total effects.

• Answer: Y on X: 2 (the total effect). Y on X and M: 1 (the direct effect only).
• Intuition: controlling for the road an effect travels on blocks everything that travels on it.
• Why: total effect = direct (1) + indirect (1 \times 1). Neither number is wrong — they answer different questions.

Drill 18 · Uneven treatment across groups

A promo was rolled out unevenly: in market A half of stores got it (true effect 1); in market B only 10% did (true effect 3). The markets are the same size. "You regress sales on the promo indicator plus a market dummy. What number do you get, and is it the average effect of 2? Why or why not?"

Concept tested: what OLS averages over when effects differ.

• Answer: \approx 1.53, not the average effect of 2.
• Intuition: OLS learns most from groups where treatment varies most, so it leans on group A.
• Why: with group controls, OLS weights each group by n_g \cdot p_g(1 - p_g) (Angrist, 1998):

\hat\tau_{\text{OLS}}

what OLS reports as the promo effect

\sum_g

add up over the markets g (A and B)

n_g

the number of stores in market g

p_g

the share of market g's stores that got the promo

\tau_g

the true promo effect in market g

• Next they'll ask: "How do you get the average effect?" → Estimate the effect within each group (or interact D with the groups) and average with the weights you care about.

Experiments — Drills 19–22

Drill 19 · Making the A/B test faster

Your A/B tests take four weeks to reach significance. You have each user's spend from the month before the test, which correlates 0.6 with spend during the test. "How would you use it to reduce variance? By how much does variance fall, and does it bias the treatment effect?"

Concept tested: CUPED / regression adjustment.

Y^{\text{cuped}}

the adjusted outcome: test-period spend minus its predictable part

Y

each user's spend during the test

\theta\,(X - \bar X)

the part of test spend you could predict beforehand: \theta times how far pre-test spend sits from average

\theta = \frac{\text{Cov}(Y, X)}{\text{Var}(X)}

the adjustment slope: the coefficient from regressing test spend on pre-test spend

X

each user's spend in the month before the test

\bar X

the average pre-test spend (bar = mean)

\rho

the correlation between pre-test and test spend (0.6)

• Answer: variance falls to 64% of the original (simulation: 0.640), so the test needs about 36% fewer users for the same power. The treatment effect stays unbiased.
• Intuition: subtract the part of each user's outcome you could have predicted before the experiment started.
• Why: X is measured before assignment, so it's independent of treatment and E[X - \bar X] is the same in both arms. The residual variance is (1 - \rho^2)\,\text{Var}(Y).
• Next they'll ask: "Is this just regression?" → Yes: it's equivalent to regressing Y on treatment and X.

Drill 20 · Covariates in a randomized test

A skeptical reviewer says: "Your experiment is randomized. Adding covariates to the regression can only introduce bias — report the raw difference." Is the reviewer right? When could covariate adjustment actually hurt, and how do you do it safely?

Concept tested: regression adjustment in experiments (Freedman's critique, Lin's estimator).

• Answer: not asymptotically. In small samples, plain regression adjustment can be slightly biased and, with unequal arm sizes and heterogeneous effects, can even lose precision (Freedman, 2008). Interacting treatment with centered covariates fixes both and is never less precise in large samples (Lin, 2013); use robust standard errors.
• Intuition: randomization already balances covariates on average; adjusting just removes noise — as long as each arm gets its own adjustment.
• Why: the interacted model fits a separate covariate slope per arm, so a difference in slopes isn't forced into the treatment coefficient.

Drill 21 · Wins that shrink on re-test

Your team ships only experiments that are significant at 5%. On re-test, the shipped effects are consistently about half as large. "Suppose a true effect of 0.1 with standard error 0.1, and you report the estimate only when it's significant. What's the average reported effect? Why, and what would you change?"

Concept tested: the winner's curse.

• Answer: the average reported estimate is about 0.25 — two and a half times the truth.
• Intuition: with low power, only estimates that got lucky noise clear the bar.
• Why: you're reporting E[\hat\theta \mid \hat\theta > 1.96 \cdot \text{SE}], which sits far above \theta when \theta is small relative to SE.
• Next they'll ask: "How do you correct it?" → Replicate, shrink estimates toward a prior (empirical Bayes), use a conditional estimator, and run experiments with adequate power in the first place.

Drill 22 · How much data?

A PM asks how many observations you need to detect whether price per square foot differs by 0.1 units from zero. "Assume noise sd 1, feature sd 1, \alpha = 0.05 two-sided, 80% power. Derive n, and tell me what happens if the true effect is half as big."

Concept tested: power and sample size for a slope.

\text{SE}(\hat w)

the slope's standard error

\frac{\sigma}{s_x\sqrt n}

noise sd \sigma (1) over the feature's sd s_x (1) times \sqrt n: shrinks as data grow

\,\sigma^2

the noise variance (1)

s_x

the feature's standard deviation (1)

n

the number of observations needed

z_{1-\alpha/2}

the normal cutoff for a two-sided test at level \alpha: 1.96

z_{1-\beta_{\text{II}}}

the normal cutoff for the target power: 0.84 for 80%

\beta

the true effect to detect (0.1)

• Answer: about 784 rows.
• Intuition: halve the effect and you need four times the data.
• Why: the slope's SE shrinks like 1/\sqrt{n}, and you need the effect to sit about 2.8 SEs from zero for 80% power at 5%.

Dependence and time series — Drills 23–25

Drill 23 · Two trending series

An analyst regresses monthly company revenue on a competitor's stock price over 8 years, finds t = 6 and R^2 = 0.7, and claims the competitor drives your revenue. The interviewer: "Suppose x and y are two independent random walks of length 100, as below. How often would a regression of y on x look significant at 5%? Why, and what would you do instead?"

Concept tested: spurious regression with non-stationary series.

x_t

the competitor's stock price in month t, a random walk

x_{t-1}

last month's value

\eta_t

this month's random step for x

y_t

your revenue in month t, an independent random walk

y_{t-1}

last month's revenue

\nu_t

this month's random step for y

\eta

the steps \eta_t of x

\nu

the steps \nu_t of y

• Answer: the slope is "significant" (\lvert t \rvert > 1.96) in about 77% of runs instead of 5% (simulation: 0.77), often with a high R^2.
• Intuition: two series that both wander will drift together for a while, and drift looks like a relationship.
• Why: the residuals are themselves a random walk, so the usual standard errors are invalid and t grows with the sample length (Granger and Newbold, 1974). An R^2 larger than the Durbin–Watson statistic is a classic warning sign.
• Next they'll ask: "How do you fix it?" → Difference the series, add lags, or test whether they're genuinely linked (cointegration, e.g. Engle–Granger).

Drill 24 · A store-level policy, transaction-level data

A pricing policy was set per store (200 stores), and you analyze 20 transactions per store with ordinary standard errors. Transactions in the same store share shocks with within-store correlation 0.1. "By how much are your standard errors off? In which direction? How do you fix it?"

Concept tested: the Moulton factor and clustered standard errors.

m

transactions per store (20)

\rho_x

the within-store correlation of the feature (1 here: the policy is set per store)

\rho_\varepsilon

the within-store correlation of the errors (0.1)

• Answer: naive SEs are too small by about 1.7\times (simulation: 1.77\times), so p-values are far too optimistic.
• Intuition: 20 people who share the same x and the same shock are worth much less than 20 independent people.
• Why: the effective sample size is closer to the number of groups than to the number of people when x is set at the group level.
• Next they'll ask: "Fix?" → Cluster-robust SEs at the group level, or analyze group means. With fewer than about 40–50 clusters, use the wild cluster bootstrap.

Drill 25 · Yesterday's sales as a feature

Your demand forecast regresses today's sales on yesterday's sales to estimate persistence. Shocks to demand (weather streaks, promotions) also persist. "With the model below, what does the OLS coefficient converge to? Is it the true persistence of 0.5? And what goes wrong in a short panel with store fixed effects?"

Concept tested: lagged dependent variable with autocorrelated errors; Nickell bias.

y_t

today's sales

y_{t-1}

yesterday's sales

u_t

today's demand shock

u_{t-1}

yesterday's shock, half of which lingers

e_t

a fresh random shock

• Answer: regressing y_t on y_{t-1} gives 0.8, not 0.5 (simulation: 0.800). It doesn't go away with more data.
• Intuition: yesterday's y contains yesterday's shock, which is still lingering today, so y_{t-1} gets credit for the shocks' persistence.
• Why: \text{Cov}(y_{t-1}, u_t) \ne 0, which breaks exogeneity. Here \operatorname{plim} = (\beta + \rho)/(1 + \beta\rho) = 1/1.25 = 0.8.
• Next they'll ask: "Panels with fixed effects and a lagged y?" → Nickell bias, about -(1 + \beta)/(T - 1) for short panels; use Arellano–Bond estimation with older lags as instruments.

Beyond OLS — Drills 26–29

Drill 26 · The uncorrelated control that changed everything

In a default model, adding a control Z that is completely uncorrelated with X changes X's logistic coefficient from 0.61 to 1.0. A colleague insists that's impossible, because "omitting an independent variable causes no bias." The true model is below, with Z independent of X and sd 2. "Who's right, and why?"

Concept tested: non-collapsibility of logistic (and probit) coefficients.

\text{logit}\,P(Y = 1)

the log-odds of default

X

the feature of interest

Z

the independent control, sd 2

• Answer: logistic regression of Y on X alone gives about 0.61, not 1 (simulation). Add Z and it's back to 1.00. In OLS, leaving out an independent variable would cause no bias at all.
• Intuition: logistic regression works on a fixed noise scale. Leaving Z out adds noise on the hidden latent scale, and the only way the model can absorb it is to shrink every coefficient.
• Why: think of a latent Y^* = X + Z + \varepsilon with logistic \varepsilon (variance \pi^2/3 \approx 3.29). Omitting Z inflates the noise variance to about 7.29, and coefficients rescale by roughly \sqrt{3.29 / 7.29} \approx 0.67. This is non-collapsibility.

\hat\beta_{\text{omit } Z}

X's logistic coefficient when Z is left out

\beta

X's true coefficient (1)

\pi

the circle constant 3.14159…; \pi^2/3 \approx 3.29 is the logistic noise variance

• Next they'll ask: "So what?" → You can't compare logistic coefficients (or odds ratios) across models with different controls, even in a randomized experiment. Compare average marginal effects instead.

Drill 27 · Oversampling defaults

Defaults are rare, so you train on every default but only 10% of non-defaults. The true model is \operatorname{logit} P(Y = 1) = -3 + X. "After fitting logistic regression on this sample, are the slope and intercept still right? If not, how do you correct them before using the probabilities in production?"

Concept tested: case-control sampling in logistic regression.

• Answer: the slope is still about 1 (simulation: 0.98). Only the intercept moves: to -3 - \ln(0.1) \approx -0.70 (simulation: -0.70).
• Intuition: sampling on y changes how common the outcome looks, not how much X shifts the odds.
• Why: odds ratios are unchanged by outcome-dependent sampling; the log of the sampling-rate ratio lands entirely in the intercept.

\beta_0^{\text{sample}}

the intercept you get when fitting on the oversampled data

\beta_0

the true population intercept (−3)

• Next they'll ask: "How do you get calibrated probabilities back?" → Subtract that log ratio from the intercept. Common in default and fraud models, where defaults are oversampled. Contrast with linear regression, where selecting on y biases the slope (Drill 4).

Drill 28 · Sales capped by stock-outs

Recorded sales can't go below zero, and many days show exactly zero. "Let the latent demand be Y^* = X + \varepsilon and you observe Y = \max(Y^*, 0), with X, \varepsilon standard normal. What does OLS of Y on X give? How does this differ from dropping the zero days, and what model would you use?"

Concept tested: censoring vs truncation; Tobit.

• Answer: OLS gives about 0.50 — equal to the share of rows that aren't censored (simulation: 0.499).
• Intuition: below the cap, every value is squashed onto the floor, which drags the left end of the line up and flattens it.
• Why: with normal X, the OLS slope on censored data is about \beta \times P(Y^* > 0).
• Next they'll ask: "Fix?" → A Tobit model, which models both the chance of hitting the cap and the value above it. If capped rows are dropped instead (truncation), use truncated regression or Heckman. Examples: loan amounts capped at a limit, sales capped by stock-outs.

Drill 29 · Mislabeled treatment

A "used the new feature" flag is wrong for about 10% of users in each direction. Half of users truly used it, and its true effect is 2. "What does the regression on the flag estimate? Can a second noisy flag used as an instrument fix it, as it would for continuous measurement error?"

Concept tested: misclassification of a binary regressor.

• Answer: the estimate is about 1.60 — 80% of the truth (simulation: 0.800 \times 2).
• Intuition: some "treated" people are really untreated and vice versa, which dilutes the comparison between groups.
• Why: here the attenuation factor is 1 - 0.1 - 0.1 = 0.8. Unlike classical measurement error, the error is negatively correlated with the true value (a true 1 can only flip down), so a second noisy measurement used as an instrument no longer fixes it.
• Next they'll ask: "Then what?" → Validation data to estimate the flip rates, or report bounds.

Overfitting and estimation theory — Drills 30–34

Drill 30 · Training error looked great

A model with 10 parameters fit on 50 rows reports a training MSE of 0.80, and the team is surprised that error on fresh data is about 1.2. "Assume the model is correctly specified with noise variance 1. Derive the expected training and test MSE. How do AIC and Mallows' Cp use this?"

Concept tested: optimism of training error.

\sigma^2\Big(1 - \frac{p}{n}\Big)

expected training MSE: noise variance \sigma^2 (1) shrunk by the share p/n = 10/50 the fit used up

\sigma^2\Big(1 + \frac{p}{n}\Big)

expected test MSE: the noise variance inflated by the same share p/n

\frac{2p\,\sigma^2}{n}

the optimism gap between test and training MSE

• Answer: training \text{MSE} \approx 0.80, test \text{MSE} \approx 1.20 (simulation: 0.80 and 1.19).
• Intuition: the fit spends p degrees of freedom chasing the training noise. That makes the training data look better and new data look worse, by the same amount.
• Why: E[\text{RSS}] = (n - p)\sigma^2 (Card 7); test error = \sigma^2 + average prediction variance = \sigma^2 + \sigma^2 p/n (Card 12).
• Next they'll ask: "How do model-selection criteria use this?" → Mallows' C_p and AIC add exactly this optimism (2p\hat\sigma^2/n) back onto the training error.

Drill 31 · The lowest scorers improved most

A tutoring report shows that students with the lowest September scores improved the most by December, and recommends targeting them. Nothing actually changed for anyone. "Let each student have a true level T and two noisy tests y_1 = T + e_1, y_2 = T + e_2, with \text{Var}(T) = \text{Var}(e). What slope do you get regressing (y_2 - y_1) on y_1? What does that mean for the report?"

Concept tested: regression to the mean in change scores.

• Answer: slope \approx -0.5 (simulation: -0.499): the highest scorers appear to "decline" and the lowest to "improve".
• Intuition: a high first score was partly luck, and luck doesn't repeat.
• Why: \text{Cov}(e_2 - e_1, T + e_1) = -\text{Var}(e), so the slope is -\text{Var}(e) / (\text{Var}(T) + \text{Var}(e)).
• Next they'll ask: "The worst-performing stores improved after our program. Did it work?" → Not without a control group; regression to the mean predicts that improvement anyway.

Drill 32 · Per-capita everything

Across cities, crimes per capita correlate with doctors per capita. The interviewer: "Suppose crimes x, doctors y and population z are independent, each with mean 10 and sd 1. What correlation do you expect between x/z and y/z? Why, and how should you model it instead?"

Concept tested: spurious correlation of ratios.

• Answer: correlation \approx 0.5 (simulation: 0.51), from nothing.
• Intuition: dividing two unrelated numbers by the same noisy number makes them rise and fall together.
• Why: Pearson's (1897) approximation, using coefficients of variation c = \text{sd}/\text{mean}:

x

a city's crimes

y

a city's doctors

z

a city's population

c_z

population's coefficient of variation, sd / mean (0.1)

c_x

crimes' coefficient of variation

c_y

doctors' coefficient of variation

• Next they'll ask: "Per-capita variables, then?" → Prefer modeling y with z as a control or an offset rather than regressing one ratio on another.

Drill 33 · Estimating many things at once

You estimate conversion lift for 10 unrelated product pages, each from one noisy measurement with variance 1. "Is reporting each page's raw estimate the best you can do for total squared error? Show me an estimator that beats it, and explain why it works even though the pages are unrelated."

Concept tested: Stein's paradox and shrinkage.

\hat{\boldsymbol\mu}_{\text{JS}}

the James–Stein estimates for all 10 pages

k

the number of pages (10)

\lVert \mathbf x \rVert^2

the squared length of the raw estimates: the sum of their squares

\mathbf x

the 10 raw lift estimates, one per page

• Answer: total squared error \approx 10 for the obvious estimate vs \approx 6.3 for James–Stein (simulation: 9.98 and 6.25). For k \ge 3, shrinkage always wins on total error.
• Intuition: when you estimate many things at once, pulling them all toward a common center cancels more noise than the bias it adds.
• Why: the same bias–variance trade as ridge (Card 11). It's also the logic behind empirical Bayes, partial pooling of rare categories (Q47), and correcting the winner's curse (Drill 21).

Drill 34 · A prediction used as a feature

Your churn model uses a "predicted lifetime value" feature produced by another model, and reports tight standard errors on it. Separately, an analyst ran 2SLS by hand as two OLS regressions. "Are the coefficients and standard errors in each case trustworthy? How would you fix them?"

Concept tested: generated regressors and two-step standard errors.

• Answer: the coefficient can be fine, but the reported SEs are too small — they ignore the uncertainty in \hat X.
• Intuition: you're treating an estimate as if it were data.
• Why: the second-stage formula assumes its features are fixed and known.
• Next they'll ask: "Fix?" → Bootstrap the whole two-step pipeline, apply a Murphy–Topel correction, or estimate both steps jointly. Classic trap: running 2SLS by hand as two separate OLS fits gives the right coefficient but wrong SEs — use a proper IV routine.

Bias-direction cheat sheet

Standard-error traps (estimate fine, uncertainty wrong): heteroscedasticity (Drill 10), duplicated rows (Q56), group-level x with clustered errors (Drill 24), random walks (Drill 23), generated regressors and hand-run 2SLS (Drill 34).

Rapid fire — the day before

One line each. Cover the right column and answer out loud; anything you hesitate on, go back to its full question above.

Review schedule: go through this table the day after you study, then 3 days later, a week later, and 3 weeks later. Any row you miss goes back to the start.