The Potential Outcomes Framework

The problem

You've just run an A/B test on a new seller dashboard. Treatment group revenue is up 8%. Your PM asks: "Is that real?" You say yes — the p-value is 0.02. But what does that actually mean? And what are you estimating, exactly?

Most practitioners know the recipe (randomise, measure, t-test) without knowing what quantity the recipe is actually producing. That matters because once you leave the perfect randomised experiment — which is most of the time — you need the formal framework to know what you can and can't claim.

The intuition

Rubin's potential outcomes framework gives causal inference its mathematical backbone. The idea is simple: every unit i has two potential outcomes — Y_i(1), what would happen if they received treatment, and Y_i(0), what would happen if they didn't. The causal effect for that unit is τ_i = Y_i(1) − Y_i(0).

The problem: you can only ever observe one of these. A seller either got the dashboard or they didn't. This is called the Fundamental Problem of Causal Inference — individual causal effects are inherently unobservable.

So instead we estimate averages. The Average Treatment Effect (ATE) is E[Y(1) − Y(0)] — the expected effect across the whole population. The ATT is the same average, but only among those who actually received treatment. In many business contexts, ATT is what you care about: did the intervention work for the people we gave it to?

The counterfactual is what didn't happen. The whole enterprise of causal inference is building a credible approximation of it.

In practice

In a properly randomised experiment, treatment assignment W_i is independent of potential outcomes. This means E[Y(1)] = E[Y | W=1] and E[Y(0)] = E[Y | W=0] — the observed group averages are unbiased estimates of the counterfactual means. Your ATE is just the difference in observed means.

In observational data, this independence breaks down. Sellers who received the new dashboard might be newer, or in a growth category. You need to either control for those differences or find an instrument.

The notation forces discipline. When a colleague says "the intervention lifted revenue," ask: ATE or ATT? Across the whole population, or conditional on some segment? The framework makes these questions precise.

Going deeper (optional)

SUTVA — the Stable Unit Treatment Value Assumption — is a prerequisite most people skip. It has two parts: (1) no interference, meaning my treatment doesn't affect your outcome; (2) no hidden versions of treatment, meaning "training program" means the same thing for everyone.

Both are violated more often than people admit. Network effects violate interference. Different rollout quality violates consistency. When SUTVA fails, potential outcomes aren't well-defined and your estimate doesn't have a clean causal interpretation.

Individual effect:  τ_i = Y_i(1) - Y_i(0)
ATE:  E[τ_i] = E[Y(1)] - E[Y(0)]
ATT:  E[τ_i | W_i = 1] = E[Y(1) | W=1] - E[Y(0) | W=1]

In RCT:  E[Y | W=1] - E[Y | W=0]  is unbiased for ATE