Causal Inference Study Module

Columbia Engineering | COMS 4995 — Spring 2026
View Chat Session →

Course Architecture

The course builds a complete pipeline from defining causal effects to estimating them from data:

1
Define the causal question — Potential outcomes $Y_i^{(D=d)}$, individual effect $\delta_i = Y_i^{(D=1)} - Y_i^{(D=0)}$, ATE $\delta = E[\delta_i]$
2
Encode causal knowledge as a graph — DAGs, Bayesian networks, functional vs probabilistic DGPs
3
Identify the estimand — d-separation, back-door criterion, find adjustment set $Z$
4
Choose an estimator — Stratification, matching, IPW, OLS, ML, Double ML
5
Validate and assess — Support check, balance assessment, sensitivity analysis
6
Handle harder settings — IV (unobserved confounders), RD (natural cutoffs), DID & ITS (time), panel methods (MC-NNM)

Final Exam: What's on It

Per the Review 2 deck (Adam Kelleher, May 6, 2026): the test focuses on the second half, but you also need graphs and d-separation. Application-focused, like the midterm.

Methods tested:

  • Instrumental Variables (L7) — Wald/2SLS, exclusion restriction, weak instruments, LATE
  • Front-Door Criterion (L8) — three graph conditions, FDC adjustment, per-mechanism shortcut
  • Regression Discontinuity (L9) — sharp & fuzzy, local linear regression, density check
  • Difference-in-Differences (L9) — parallel trends, ATT, time-varying controls
  • Interrupted Time Series (L9) — no-confounding assumption, counterfactual model
  • Panel & pooled cross-sectional designs (L10–11) — ever-treated, MCNNM
  • Selection bias adjustment (L10–11) — post-stratification, IPSW, time-series s-bias

For each method, know:

  • The graph assumption required to use it.
  • How to recognize when a system satisfies it.
  • Each estimator's additional assumptions.
  • How to apply it (the code). Open-book won't help if you haven't practiced.
Format — similar to mid-term, with two specific differences:
  • Qualitative questions describing DGPs, which you'll need to connect to causal assumptions (similar to the last homework).
  • A True/False section testing your understanding of inference methods (especially when their assumptions are justified).

Key Assumptions Across All Methods

SUTVA Core
Stable Unit Treatment Value Assumption. No interference between units — one unit's treatment doesn't affect another's outcome. Also: no hidden treatment variations (one version of each treatment level).
Consistency Core
If unit $i$ receives treatment $D=d$, then the observed outcome $Y_i = Y_i^{(D=d)}$. The potential outcome framework is "well-defined" — observed outcomes match the corresponding potential outcomes.
Unconfoundedness / Correct Graph Core
$Y^{(D=1)}, Y^{(D=0)} \perp D \mid Z$ — conditional on the adjustment set, treatment is as-if random. Equivalently: your causal graph is correct, and $Z$ satisfies the back-door criterion.
Positivity / Support Core
$0 < P(D=d \mid Z=z) < 1$ for all $z$ with $P(Z=z)>0$. Every unit has a nonzero chance of receiving each treatment level. Without this, we can't estimate counterfactuals for some strata.

L1: Foundations of Causal Inference

Potential Outcomes Framework Core
For each unit $i$ and treatment level $d$, there exists a potential outcome $Y_i^{(D=d)}$.
$\delta_i = Y_i^{(D=1)} - Y_i^{(D=0)}$

The individual causal effect $\delta_i$ is the difference between what would happen under treatment vs. control for the same unit.

Fundamental Problem Exam
We can never observe both $Y_i^{(D=1)}$ and $Y_i^{(D=0)}$ for the same unit. We always see exactly one potential outcome — the one corresponding to the treatment actually received. The other is the counterfactual.
ATE, ATT, ATC Exam
$\text{ATE} = E[Y^{(D=1)} - Y^{(D=0)}]$
$\text{ATT} = E[Y^{(D=1)} - Y^{(D=0)} \mid D=1]$
$\text{ATC} = E[Y^{(D=1)} - Y^{(D=0)} \mid D=0]$

$\text{ATE} = P(D=1)\cdot\text{ATT} + P(D=0)\cdot\text{ATC}$

Naive Estimator Bias Warning
$\delta_{\text{naive}} = E[Y \mid D=1] - E[Y \mid D=0] \neq \delta$
The naive estimator equals $\delta$ only when $D \perp (Y^{(D=1)}, Y^{(D=0)})$ — i.e., when treatment is randomly assigned. Otherwise it's biased by confounding.
Randomized Experiments Method
Random assignment ensures $D \perp (Y^{(D=0)}, Y^{(D=1)})$, making the naive estimator unbiased. The switching equation: $Y_i = D_i \cdot Y_i^{(D=1)} + (1-D_i) \cdot Y_i^{(D=0)}$.

Randomization check: Verify treatment/control balance using binomial probability bounds.

Bootstrap Method
Standard: Resample $n$ rows with replacement, compute statistic, repeat. Assumes i.i.d.
Poisson: Assign Poisson(1) weight to each row. Equivalent to standard bootstrap for large $n$.
Dependent-samples: Assign Poisson(1) weights at the cluster level (e.g., per user). Correctly accounts for within-cluster correlation. SE can be 2-3x larger than naive.

The do() Operator

$P(Y \mid do(X=x)) \neq P(Y \mid X=x)$

$P(Y \mid do(X=x))$ is the interventional distribution — the distribution of $Y$ if we physically set $X=x$ for everyone. Graphically: delete all edges into $X$ and set $X=x$.

$P(Y \mid X=x)$ is the observational conditional — the distribution of $Y$ among units that happen to have $X=x$. This includes confounding.

Concrete Example: Job Training (NSW / LaLonde 1986)

The National Supported Work program offered job training to disadvantaged workers. For a single trainee Maria, $Y_i^{(D=1)}$ is her earnings 18 months after training; $Y_i^{(D=0)}$ is what she would have earned without training. We see one but never both — the fundamental problem of causal inference.

The naive estimator $E[Y \mid D{=}1] - E[Y \mid D{=}0]$ compares trainees to all non-trainees and reports a negative effect. But trainees were unemployed and disadvantaged when they enrolled — they weren't comparable to the general workforce. The selection-bias term $E[Y^{(0)} \mid D{=}1] - E[Y^{(0)} \mid D{=}0]$ is large and negative: trainees would have earned less anyway. Once we control for baseline earnings and demographics, the ATT is positive and economically meaningful.

What this illustrates: we never see both potential outcomes for the same person; the naive estimator includes selection bias; ATT $\neq$ ATE when treatment is self-selected.

L2: Data Generating Processes & Causal Graphs

Functional vs Probabilistic DGPs Core
Probabilistic: Draw from conditional distributions: $X_1 \sim P(X_1)$, $X_2 \sim P(X_2|X_1)$, ...
Functional (Structural Equations): $X_i = f(\text{Parents}(X_i), \epsilon_i)$ where $\epsilon_i$ are independent noise terms.

Both are statistically equivalent — they produce samples from the same joint distribution. The functional approach makes interventions easier to reason about (just replace an equation).

Bayesian Networks Core
A DAG + conditional distributions. The joint factorizes as:
$P(X_1, ..., X_n) = \prod_i P(X_i \mid \text{Parents}(X_i))$
This factorization implies specific conditional independence relationships that can be read off the graph.
Observational Equivalence Warning
Multiple different causal graphs can produce the same observational distribution. $X \to Y$ and $Y \to X$ are observationally equivalent with just two variables. You cannot determine causal direction from data alone without additional assumptions.
Confounding Example: CTR Exam
Scenario: Title length $X$ appears to affect click rate $Y$, but both are driven by genre preference $Q$ (horror fans prefer 13-word titles and get more clicks).
Graph: $Q \to X$, $Q \to Y$ (no edge $X \to Y$).
Naive: $E[Y|X=x]$ shows a peak at 13 words. Stratified: $E[Y|X=x, Q=q]$ shows NO relationship — the correlation was entirely spurious.

Three Building Blocks of Causal Graphs

Chain: $X \to Z \to Y$ Core
Unconditional: $X \not\perp Y$ (correlated via the path).
Conditioning on Z: $X \perp Y \mid Z$ (path is BLOCKED).

Light switch analogy: If you know the current is flowing ($Z$), knowing the switch position ($X$) tells you nothing new about the light ($Y$).

Caution: If $Z$ is a mediator of the causal effect of $X$ on $Y$, conditioning on $Z$ will make $X$ appear to have no effect — even though it does!

Numerical Example: Smoking → Tar → Cancer

$X$ = smoking, $Z$ = tar deposits, $Y$ = lung cancer.

cor(X, Y)cor(X, Y | Z)
Full population0.40 (X and Y correlated)0.01 (independent given Z)

Smoking affects cancer only through tar. Fixing tar, smoking tells you nothing new about cancer. But this also means: conditioning on a mediator kills the causal signal!

Fork: $X \leftarrow Z \to Y$ Core
Unconditional: $X \not\perp Y$ (spurious correlation via common cause).
Conditioning on Z: $X \perp Y \mid Z$ (path is BLOCKED).

Lemonade & crime: Temperature ($Z$) causes both lemonade sales ($X$) and crime ($Y$). They're correlated but neither causes the other.

Numerical Example: Loyalty → Discount & Purchases

$Z$ = loyalty tier, $X$ = received discount, $Y$ = purchase rate. From the BDA example:

PopulationP(Discount)Purchase RateNaive cor(X, Y)
All customers50%—0.44 (inflated!)
Low loyalty only20%0.20 / 0.400.20 (true effect)
High loyalty only80%0.60 / 0.800.20 (true effect)

Conditioning on loyalty (Z) removes the confounding. Within each stratum, the true causal effect of 0.20 emerges.

Collider: $X \to Z \leftarrow Y$ Warning
Unconditional: $X \perp Y$ (path is BLOCKED at collider).
Conditioning on Z: $X \not\perp Y \mid Z$ (path is OPENED!).

University admission: Math ($X$) and writing ($Y$) skills are independent. But among admitted students ($Z=1$), they become negatively correlated.

Numerical Example: Math & Writing → Admission

$X$ = math score, $Y$ = writing score, $Z$ = admitted (Z=1 if $X + Y > $ threshold).

Populationcor(Math, Writing)N
All applicants0.012 (independent)10,000
Admitted only (Z=1)-0.290 (negative!)2,400

Among admitted students, low math implies high writing. Conditioning on the collider created a spurious negative association that doesn't exist in the full population.

Key rule: Conditioning on a collider (or its descendants) OPENS a previously blocked path.

Interactive: Three Building Blocks

Click a structure to see how information flows. Toggle conditioning on Z to see the path open or close.

Summary Table

StructureUnconditionalConditioning on middle node
Chain: $X \to Z \to Y$$X \not\perp Y$$X \perp Y \mid Z$ (BLOCKS)
Fork: $X \leftarrow Z \to Y$$X \not\perp Y$$X \perp Y \mid Z$ (BLOCKS)
Collider: $X \to Z \leftarrow Y$$X \perp Y$$X \not\perp Y \mid Z$ (OPENS)

Concrete Example: Berkeley Graduate Admissions (1973)

Berkeley was sued because women were admitted at a lower aggregate rate than men (35% vs 44%). Looking within department, women were admitted at equal or higher rates than men. Same data, opposite conclusion.

The graph: Gender → Department → Admission. Women applied disproportionately to humanities; men to engineering. Humanities had lower admit rates than engineering for everyone. Department is a fork on the back-door path Gender ← Department → Admission. Without conditioning on Department, the spurious gender→admission association is exposed; conditioning closes the back-door and the discriminatory signal disappears.

Bonus collider: in hospital data, smoking and car accidents are independent in the population, but among hospitalized patients they correlate negatively — Smoking → Hospital ← Accidents is a collider, and selecting on hospitalization opens the path. This is selection bias dressed as a graph.

What this illustrates: the fork building block creates Simpson's paradox; the collider building block creates selection-induced association.

L3: d-Separation & Back-Door Criterion

d-Separation Definition Core
A path between $X$ and $Y$ is blocked by a set $Z$ if, along the path, there is either:
  1. A chain or fork node that is IN $Z$ (blocking), OR
  2. A collider node that is NOT in $Z$ and has no descendant in $Z$
$X$ and $Y$ are d-separated by $Z$ if ALL paths between them are blocked.
$X \perp_d Y \mid Z \implies X \perp Y \mid Z$
d-separation in the graph implies conditional independence in the distribution (under faithfulness).
Back-Door Criterion Exam
A set $Z$ satisfies the back-door criterion for the effect of $X$ on $Y$ if:
  1. No descendant of $X$ is in $Z$
  2. $Z$ blocks all back-door paths from $X$ to $Y$ (paths that start with an arrow INTO $X$)

A "back-door path" is any path from $X$ to $Y$ that has an arrow pointing into $X$ (i.e., $\leftarrow X$). These are non-causal paths that create confounding.

Back-Door Adjustment Formula Exam
If $Z$ satisfies the back-door criterion:
$P(Y \mid do(X=x)) = \sum_z P(Y \mid X=x, Z=z) \, P(Z=z)$
In expectation form:
$E[Y \mid do(X=x)] = \sum_z E[Y \mid X=x, Z=z] \, P(Z=z)$

This converts the interventional distribution into observational quantities we can estimate from data.

Finding Back-Door Sets: Step-by-Step

1
Identify all back-door paths from $X$ to $Y$: paths with an arrow INTO $X$ at the start
2
Check for colliders on each path. A collider blocks the path by default; don't include it (or its descendants) in $Z$
3
Find non-descendant nodes that block all remaining open back-door paths
4
Verify: No node in $Z$ is a descendant of $X$, and all back-door paths are blocked

Interactive: d-Separation Exercise

Click on nodes to add/remove them from the conditioning set Z. The visualization shows which paths are blocked or open, and whether X and Y are d-separated.

Click a node to condition on it (highlighted). Click again to remove. X and Y cannot be conditioned on.

Stratification & Coarsening

Stratification = Direct BDA Implementation

For each stratum $z$, compute $E[Y|X=x, Z=z]$, then weight by $P(Z=z)$.

$\hat{\delta} = \sum_z \left(E_N[Y \mid X=1, Z=z] - E_N[Y \mid X=0, Z=z]\right) P_N(Z=z)$

Coarsening trade-off: Fine strata = less bias, more variance (sparse data per cell). Coarse strata = more bias (residual confounding), less variance (more data per cell).

Voter outreach example: County-level stratification (3000+ strata) was too fine with $N=2000$. State-level (50 strata) gave good estimates despite some residual confounding.

Concrete Example: Elite College & Earnings (Dale–Krueger 2002)

Does attending an elite college cause higher lifetime earnings? Confounders include high-school GPA, SAT, family income, parents' education, and unobserved motivation/ambition. The back-door criterion says: find a $Z$ that blocks all paths $X \leftarrow \cdots \to Y$.

A naive controlled regression on observed covariates leaves an "elite premium" of ~7% — but motivation is still flowing through an unblocked back-door. Dale & Krueger's clever fix: condition on the set of colleges a student applied to and was admitted to. Two students with the same admission set were similarly motivated and prepared, so the admission set acts as a near-perfect proxy for the unobserved confounder. Within admission-set strata, the elite-vs-non-elite earnings gap shrinks to near zero.

What this illustrates: the back-door criterion is a graph claim, not a data claim — domain knowledge supplies the graph; there's no statistical test for it; creative researchers find proxies that block paths through unmeasured confounders.

L4: Conditioning-Based Methods

Exact Matching Method
For each unit, find a match with the same confounder values but opposite treatment.
No model assumptions Curse of dimensionality Requires discretization

Problem: With 5 confounders and 5 bins each: $5^5 = 3125$ strata. Most will be empty with moderate $N$.

App example: Naive estimate was $-0.04$ (wrong sign!). Exact matching with coarse bins recovered $+0.06$ (true = $0.10$). Residual bias from coarsening.

Nearest-Neighbor Matching Method
Match each unit to its closest counterpart in the opposite treatment group using a distance metric.
$d_{\text{Mahalanobis}}(z_i, z_j) = \sqrt{(z_i - z_j)^T \Sigma^{-1} (z_i - z_j)}$
Works with continuous Z Curse of dimensionality in high-D

Caliper: Reject matches where distance exceeds a threshold. Trades off coverage for match quality.

Propensity Score Matching Method
The propensity score $e(Z) = P(X=1|Z)$ is a balance score: it's sufficient for removing confounding.
$X \perp Z \mid e(Z)$
Match on the 1D propensity score instead of the full $Z$ vector. Avoids curse of dimensionality.
Reduces to 1D matching Requires correct propensity model

Balance assessment: After matching, check that the distribution of $Z$ is similar between treated and matched controls. Use standardized mean difference statistic.

Support Assumption Warning
$0 < P(X=x \mid Z=z) < 1$ for all $z$ with $P(Z=z) > 0$
Without overlap, we can't find matches. Check support visually: plot propensity score distributions for treated vs. control. If they don't overlap somewhere, matching can't work there.

Checking support: (1) Estimate propensity score $e(Z) = P(X{=}1|Z)$ via logistic regression. (2) Plot overlapping histograms for $X{=}1$ vs $X{=}0$. (3) Flag scores below 0.05 or above 0.95 — these create instability.

Danger of Discretization Warning
Coarsely discretizing a continuous confounder only partially blocks the back-door path. Residual variation within bins maintains the association.

Example: In the fork $X_1 \leftarrow X_2 \to X_3$, conditioning on continuous $X_2$ gives $\beta_{X_1}=0$ (correct). Discretizing $X_2$ into 5 bins gives $\beta_{X_1}=0.087$ (biased!).

Bias Amplification Warning
Adjusting for the wrong variables can make bias worse, not better. If a variable is an instrument (causes $X$ but not $Y$ directly), conditioning on it amplifies confounding from unmeasured variables.

Voter example: County contact rate was NOT a confounder (it affected $X$ but not $Y$ directly). Stratifying by it amplified bias: estimate went from 0.068 to 0.072 (true = 0.05).

Rule: Only condition on variables that satisfy the back-door criterion. Blindly "controlling for everything" is dangerous.

Propensity Score & Support: Visual Guide

Interactive Propensity Score Overlap

Adjust the slider to see how treatment assignment strength affects overlap. When treatment is nearly deterministic (high separation), support fails and causal estimation breaks down.

Moderate
Good overlap — Both groups span the same range. All estimation methods work.

What Propensity Score Regions Mean

$e(Z)$ regionTreated unit ($X=1$)Control unit ($X=0$)Implication
Near 0 Very unusual — almost nobody like them is treated Typical — most similar people are untreated High IPW weight for treated; informative but unstable
Near 0.5 Even odds, well-represented Even odds, well-represented Ideal — best overlap, most stable estimates
Near 1 Typical — most similar people are also treated Very unusual — almost nobody like them is untreated High IPW weight for control; informative but unstable

How Each Method Breaks Down Without Support

MethodWhat happens
StratificationEmpty cell — can't compute stratum effect
MatchingNo match found — can't construct counterfactual
IPWWeight $= 1/P(X|Z) \to \infty$ — one unit dominates the estimate
Regression / MLExtrapolates — fills in the gap, but trusts model where there's no data to verify

Three Overlap Scenarios

Full Overlap

Both distributions cover the same propensity score range. Support holds everywhere. All methods work.

Partial Overlap

Overlap in the middle, not at tails. Options: trim to overlap region (changes estimand) or extrapolate (trust model).

No Overlap

Groups completely separated. Treatment is essentially deterministic given Z. No method can help.

Matching Estimator Formulas

After matching, construct potential outcomes for each unit:

$\hat{Y}_i^{(D=1)} = \begin{cases} Y_i & \text{if } D_i = 1 \\ Y_{\text{match}(i)} & \text{if } D_i = 0 \end{cases}$
$\hat{\delta}_{\text{ATE}} = \frac{1}{N}\sum_i \left(\hat{Y}_i^{(D=1)} - \hat{Y}_i^{(D=0)}\right)$

For ATT: only match the treated units. For ATC: only match the control units.

$\text{ATE} = P(D=1) \cdot \text{ATT} + P(D=0) \cdot \text{ATC}$

How Propensity Score Matching Works

Click Step Through to walk through the matching process one treated unit at a time. Each treated unit (orange) gets paired with the nearest control unit (blue) by propensity score.

Press Step Through to begin matching treated units to their nearest control.

Key Observations

Unequal group sizes are fine

At $e(Z)=0.3$, there may be 10 controls but only 2 treated. That's OK — we match from treated to control, not group-average. Controls can be reused (with replacement).

What $e(Z)$ means for controls

A control unit with $e(Z) = 0.8$ had an 80% predicted chance of being treated — but wasn't. They're rare and valuable: they look like treated units but didn't get treatment.

Density ≠ count (but related)

Density is a smoothed histogram — it shows concentration of units, not raw count. A tall peak at 0.7 means many units cluster there. The area under the curve integrates to 1.

Why near-1 scores are mostly treated

$e(Z)=0.95$ means 95% chance of treatment given covariates. So most units there are treated. Few controls exist — matching gets hard. This is where support problems live.

Why Matching on e(Z) Is Sufficient

The Rosenbaum-Rubin theorem says: if conditioning on all of Z removes confounding, then conditioning on the single scalar $e(Z) = P(X{=}1|Z)$ does too.

$$X \perp\!\!\!\perp (Y^{(0)}, Y^{(1)}) \mid Z \;\;\Longrightarrow\;\; X \perp\!\!\!\perp (Y^{(0)}, Y^{(1)}) \mid e(Z)$$

Click on any propensity score band to see the units inside it. Despite having very different covariate profiles, they share the same treatment probability — so within each band, treatment is as good as random.

Click a colored band to see the units inside that propensity score stratum.

The Compression Intuition

$$Z = (\text{age, income, education, region, ...}) \;\xrightarrow{\;e(\cdot)\;}\; e(Z) = 0.4$$

Many different covariate profiles map to the same propensity score. A young, high-income, urban person and an old, low-income, rural person can both have $e(Z) = 0.4$. What matters is not why they have that probability, but that they do. Within this group, treatment assignment carries no information about Z — confounding is gone.

Distant Propensity Scores → Systematically Different People

Drag the treated unit along the propensity score axis to see how the best available control match changes — and why matching across large PS distances produces biased comparisons.

Drag the orange treated unit left/right to explore what happens when you match across different PS distances.

Concrete Example: Right-Heart Catheterization (Connors et al. 1996)

5,735 ICU patients; some catheterized (RHC), some not. The naive comparison: catheterized patients died more (38% vs 31%). But sicker patients were catheterized — confounding by indication.

Adjustment set $Z$: ~50 covariates (age, APACHE severity, lab values, primary diagnosis). Direct stratification was infeasible. The team used propensity-score matching: estimate $\hat e(Z) = P(\text{RHC} \mid Z)$ via logistic regression, then match each catheterized patient to a non-catheterized patient with similar $\hat e(Z)$. Patients with $\hat e(Z) > 0.95$ had no comparable controls and were excluded — a hard positivity failure that the propensity score made visible.

After matching, RHC still increased mortality by ~7 percentage points. The result drove practice change in critical-care medicine.

What this illustrates: high-dim $Z$ requires PSM rather than direct stratification; propensity scores expose positivity violations as a side effect; clean back-door analysis can overturn naive correlations.

L5: Weighting, Regression & ML Methods

IPW Estimator — The Full Formula Method

The ATE is computed as two separate sums, one per group, then subtracted:

$\hat{\delta}_{\text{IPW}} = \underbrace{\frac{1}{N} \sum_{i:\, x_i=1} \frac{y_i}{e(Z_i)}}_{\text{estimate of } E[Y|do(X=1)]} \;-\; \underbrace{\frac{1}{N} \sum_{i:\, x_i=0} \frac{y_i}{1 - e(Z_i)}}_{\text{estimate of } E[Y|do(X=0)]}$

Reading the notation

SymbolMeaning
$i: x_i = 1$Sum only over treated units (a filter on the loop)
$i: x_i = 0$Sum only over control units
$N$Total sample size (both groups), NOT group size
$e(Z_i)$$P(X{=}1 \mid Z_i)$ — propensity score of unit $i$
$1 - e(Z_i)$$P(X{=}0 \mid Z_i)$ — probability of being untreated

The general weight rule

The denominator is always the probability of what actually happened:

$w_i = \frac{1}{P(X = x_i \mid Z_i)} = \begin{cases} 1/e(Z_i) & \text{if treated } (x_i = 1) \\ 1/(1 - e(Z_i)) & \text{if control } (x_i = 0) \end{cases}$

Units are "surprising" when their actual treatment status was unlikely given their covariates — the denominator is small, so the weight is large.

Concrete example: Control + High Loyalty

PropertyValue
Actual status$x_i = 0$ (control — not treated)
Covariates$Z_i = \text{High Loyalty}$
Propensity$e(Z_i) = P(X{=}1 \mid Z{=}\text{High}) = 0.80$
Weight denom.$P(X{=}0 \mid Z{=}\text{High}) = 1 - 0.80 = 0.20$
Weight$w_i = 1 / 0.20 = \mathbf{5.0}$
Interpretation80% of similar people got treated, but this unit didn't — surprising, so highly informative. Represents 5 similar people.

Works for any function: Replace $y_i$ with $f(y_i)$ to estimate $E[f(Y) \mid do(X=x)]$ — not just means, but variances, quantiles, etc.

No outcome model needed No stratification needed — works with continuous Z Weights explode when e(Z) near 0 or 1 Correct propensity score model

IPW vs BDA: Two Paths to the Same Answer

BDA and IPW are algebraically equivalent — they just rearrange the same adjustment formula. Toggle between them to see how.

OLS Regression — Why βX = ATE Method

For linear model $Y = \beta_0 + \beta_X X + \beta_Z Z + \epsilon$, claim: $\beta_X$ is the ATE.

Step-by-step: connecting regression to BDA

Write out the conditional expectation for each treatment value at a fixed $Z = z$:

$E[Y|X{=}1, Z{=}z] = \beta_0 + \beta_X(1) + \beta_Z z$

$E[Y|X{=}0, Z{=}z] = \beta_0 + \beta_X(0) + \beta_Z z$

Subtract (the effect of treatment at this specific $z$):

$E[Y|X{=}1, Z{=}z] - E[Y|X{=}0, Z{=}z] = \beta_X$

$\beta_0$ cancels. $\beta_Z z$ cancels. What's left is $\beta_X$ — and it's the same for every value of $z$. The linear model assumes the treatment effect doesn't depend on Z.

Plug into BDA: $\sum_z \beta_X \cdot P(Z{=}z) = \beta_X \cdot 1 = \beta_X$  ✓

The naive regression trap (omitting Z)

Regress Y on X without Z: $Y = \alpha_0 + \alpha_X X + \epsilon$

$\alpha_X = \underbrace{\beta_X}_{\text{true effect}} + \underbrace{\beta_Z \cdot (E[Z|X{=}1] - E[Z|X{=}0])}_{\text{selection bias in regression form}}$

Bias = how much Z affects Y ($\beta_Z$) × how different treated/control groups are on Z.

Simple, well-understood Assumes constant treatment effect Correct functional form (linearity)
Heterogeneous Treatment Effects Exam

"$\delta$ varies with Z" means the treatment works differently for different people:

Z (loyalty)Effect of discount
Low loyalty$\delta = +0.30$ (big boost — they needed the nudge)
High loyalty$\delta = +0.10$ (small boost — would have bought anyway)

The subtraction doesn't cancel cleanly: $E[Y|X{=}1, Z{=}z] - E[Y|X{=}0, Z{=}z] = \delta(z) \neq \text{constant}$

Constant-effect OLS gives a weighted average of effects — but weighted by Z's distribution within the treated group, not the population. So $\beta_X \neq$ ATE.

Two fixes

Fix 1 — Interaction term: $Y = \beta_0 + \beta_X X + \beta_Z Z + \beta_{XZ} X \cdot Z + \epsilon$. Then $\delta(z) = \beta_X + \beta_{XZ} z$.

Fix 2 — BDA predict-and-average: handles any functional form automatically.

BDA Predict-and-Average: Interactive

BDA works with any model. Toggle the effect type and walk through the 4 steps. The visualization shows the model's predictions under intervention.

Step: 1 / 4

Advanced Regression Methods

Model Extrapolation & Lack of Common Support Warning

The danger: imposing a functional form

All methods use data. The specific danger with regression is that it imposes a functional form (linear, polynomial, etc.) and extends it into regions with no data. Example:

"Y increases linearly with Z in [0.2, 0.8] where I have data, so it must continue linearly at Z = 0.95."
But maybe the relationship curves, saturates, or reverses there. There's no data to check.

Lack of common support

Where $P(X{=}1|Z) \approx 0$ or $\approx 1$, one group has no units. We have no empirical basis for the counterfactual. Regression manufactures one from assumptions; matching and IPW refuse.

Hierarchy of honesty

MethodNo-overlap regionFailure mode
Exact matchingDrops the unitHonest — no estimate
PSM with calipersDrops the unitHonest — no estimate
IPWWeight → ∞Breaks loudly
Regression / BDAExtrapolatesBreaks silently

The silent failure is uniquely dangerous — you get a confident estimate and a confidence interval, with no indication that the answer rests entirely on functional form assumptions rather than data.

IPW-Weighted Causal Regression Method
Train any model on $X$ alone (no $Z$!) with IPW-weighted loss:
$L_{\text{causal}} \propto \sum_i \frac{(y_i - \hat{f}(x_i))^2}{P(X=x_i|Z=z_i)}$

The model learns $E[Y \mid do(X)]$ directly.

Doubly-robust: Include $Z$ AND use IPW weights. Consistent if EITHER model is correct.

Why ML Models Break Causal Inference

ML models are biased by design. Regularization deliberately introduces systematic error to reduce variance. This is fine for prediction — but fatal for causal inference. Toggle to see why.

Double ML: Residualization

Double ML — The Partially Linear Model Method
$Y = \delta X + g(Z) + \epsilon$

Division of labor

PartWhatWho handles itRequirement
$\delta X$Treatment effect (constant)OLS on residualsValid inference, CIs
$g(Z)$Confounding (can be complex)ML (any model)Only needs good prediction

The model is "partially linear" — linear in X (one coefficient $\delta$), but $g(Z)$ can be arbitrarily nonlinear. ML handles the hard part; OLS handles the easy part.

The residualization procedure

  1. Model $\hat{E}[Y|Z]$ — predict Y from confounders alone (any ML model)
  2. Model $\hat{E}[X|Z]$ — predict treatment from confounders (this is the propensity score!)
  3. Compute residuals: $\tilde{Y}_i = Y_i - \hat{E}[Y|Z_i]$ and $\tilde{X}_i = X_i - \hat{E}[X|Z_i]$
  4. Regress: $\hat{\delta} = \sum_i \tilde{X}_i \tilde{Y}_i \;/\; \sum_i \tilde{X}_i^2$

Why the residual formula = OLS slope

The formula $\hat{\delta} = \sum \tilde{X}_i \tilde{Y}_i / \sum \tilde{X}_i^2$ is the OLS coefficient for regressing $\tilde{Y}$ on $\tilde{X}$ (no intercept). Since residuals are mean-zero by construction, this equals $\text{cov}(\tilde{X}, \tilde{Y}) / \text{var}(\tilde{X})$ — the slope. One unit of "unexplained-by-Z treatment variation" causes $\delta$ units of "unexplained-by-Z outcome variation."

Cross-fitting: why it's needed

ML regularization introduces bias. If you predict residuals on the same data you trained on, that bias leaks into $\hat{\delta}$. Cross-fitting (like cross-validation) trains on K-1 folds, predicts on the held-out fold. Held-out predictions are unbiased.

ML handles complex g(Z) Valid CIs via cross-fitting Assumes constant treatment effect
CATE & R-Learner — Heterogeneous Effects Exam

CATE: Conditional Average Treatment Effect

$\tau(z) = E[Y^{(1)} - Y^{(0)} \mid Z = z]$

"The treatment effect for people with covariates $Z = z$." Double ML gives one number ($\delta$) for everyone. CATE gives a function — the effect varies with Z.

R-Learner procedure

  1. Compute residuals exactly as Double ML: $\tilde{Y}_i$, $\tilde{X}_i$
  2. Fit a model $\hat{\tau}(Z)$ that minimizes:
$\sum_i \left(\tilde{Y}_i - \tau(Z_i) \cdot \tilde{X}_i\right)^2$

Intuition: Double ML asks "what's the single slope between $\tilde{X}$ and $\tilde{Y}$?" R-learner asks "what's the slope, and how does it change with Z?"

The model $\hat{\tau}(Z)$ can be a random forest, neural net, etc. — it learns which subgroups benefit most from treatment.

Double MLR-Learner
Effect$\delta$ (one number)$\tau(Z)$ (function of Z)
ResidualsSameSame
Final stepOLS: single slopeML: slope varies with Z
Use case"Does treatment work?""For whom does it work most?"

Double ML: Interactive Residualization

Walk through the residualization steps. Watch how removing Z's influence from both Y and X isolates the causal effect.

Step: 1 / 4

Concrete Example: Targeted Email Lift Estimation

A retailer wants to know: did the personalized recommendation email increase purchases? Population: 2M users; 200K received the email. Treatment was assigned by an existing ML personalization model on rich features $Z$ — last-purchase recency, browsing behavior, demographics, lifetime value. Standard back-door conditioning on $Z$ is identified, but $Z$ is high-dimensional.

IPW route: fit $\hat e(Z) = P(\text{email} \mid Z)$ via gradient boosting. Reweight each treated user by $1/\hat e(Z_i)$ and each control by $1/(1-\hat e(Z_i))$. Trim weights at $\hat e \in (0.05, 0.95)$ to avoid blow-up. The weighted ATE estimates lift.

Double ML route: 5-fold split. On each held-out fold, predict $\hat g(Z) = E[Y \mid Z]$ with XGBoost and $\hat m(Z) = P(D \mid Z)$ with another model. Compute residuals $\tilde Y = Y - \hat g(Z)$, $\tilde D = D - \hat m(Z)$, then OLS $\tilde Y$ on $\tilde D$. Coefficient gives valid $\sqrt n$ inference even with ML nuisances — Neyman orthogonality protects against ML's regularization bias.

What this illustrates: IPW handles high-dim $Z$ that stratification can't; Double ML lets you use ML for nuisance functions while still getting standard CIs for the causal parameter.

L6: Practical Limitations

Graph Sufficiency Exam
"Is your graph good enough?" The central question in applied causal inference. All methods assume the graph is correct. If you're missing a confounder or have the wrong structure, estimates will be biased.

There is no statistical test for whether your graph is correct. Domain knowledge is essential.

Causal Discovery (PC Algorithm) Method
Learns graph structure from data using conditional independence tests:
  1. Start with a fully connected graph
  2. Remove edges where conditional independence is found
  3. Orient edges using collider detection and propagation rules

Independence tests:

TestData TypeLimitation
Fisher-ZContinuous (+ binary)Assumes linear associations
Chi-squareDiscreteRequires discretizing continuous vars
KCIAny (nonparametric)$O(n^3)$, impractical at scale

Key limitation: Discretizing continuous mediators causes information loss and spurious edges.

Sensitivity Analysis: Stress-Testing Your Estimate

The Core Question Exam

Every method assumes the DAG is correct — that all confounders are measured and adjusted for. Sensitivity analysis asks: what if we're wrong?

"How strong would an unmeasured confounder $U$ have to be to explain away our estimated treatment effect?"

Suppose you've estimated $\hat{\delta} = 0.20$ after adjusting for all observed $Z$. But an unmeasured $U$ exists with $U \to X$ and $U \to Y$, creating an unblocked back-door path. The bias follows the same structure as the naive estimator:

$\text{Bias} = \beta_U \cdot \left(E[U \mid X{=}1] - E[U \mid X{=}0]\right)$

How much $U$ affects $Y$ ($\beta_U$) × how different the groups are on $U$ (imbalance). Your true effect is $\delta_{\text{true}} = \hat{\delta} - \text{Bias}$.

What sensitivity analysis is NOT

  • Does not tell you whether an unmeasured confounder exists
  • Does not fix the bias
  • Quantifies your vulnerability — how much hidden confounding your conclusion can tolerate

Analogy: "My bridge holds 10 tons. Sensitivity analysis says it survives up to 25 tons. I don't know if a 25-ton truck will ever cross — but I know my safety margin."

The Breakdown Point Exam

The key output: the minimum confounder strength that would zero out the effect.

Set $\hat{\delta} - \alpha = 0$ and solve. The breakdown value $\alpha^*$ is the confounding magnitude needed to make the estimated ATE vanish.

The shifted-outcome procedure (from the course)

  1. Start with your estimated ATE (e.g., $\hat{\delta} = 0.20$)
  2. Simulate an unmeasured confounder by shifting outcomes: $Y^{(\alpha)} = Y - \alpha \cdot (2D - 1) \cdot P(D = 1{-}d \mid Z)$
  3. Re-estimate the treatment effect with the shifted $Y^{(\alpha)}$
  4. Increase $\alpha$ until $\hat{\delta}^{(\alpha)} = 0$
  5. That $\alpha^*$ is the breakdown point

Interpreting the breakdown point

Breakdown valueInterpretationVerdict
Very largeNeed a confounder stronger than anything observedRobust
SmallA mild unmeasured confounder would kill the effectFragile
NegativeAdding confounding strengthens the effectVery robust

Connection to the hierarchy of assumptions

All causal methods rest on: (1) Graph structure is correct, (2) No unmeasured confounders, (3) Support/overlap. Sensitivity analysis stress-tests assumption #2. There's no statistical test for it — only this "how bad would it have to be?" reasoning.

Breakdown Point: Interactive

Drag the slider to increase the strength of a hypothetical unmeasured confounder. Watch the estimated ATE shrink toward zero.

α = 0.00

When to Use Which Method

1
Can you run an experiment?
→ Yes: Randomize. Naive estimator + bootstrap CIs. No DAG needed. Done.
2
Do you have a valid back-door set $Z$?
→ No: Need IV, RDD, or diff-in-diff (not on this exam). Stop.
3
What does $Z$ look like?
→ Discrete, low-dim (≤ 3 vars): Stratification (direct BDA sum)
→ Continuous, low-dim: Matching (NN/Mahalanobis or PSM). Check overlap first.
→ High-dim or mixed: go to step 4.
4
Which modeling approach?
→ Trust linear $E[Y|X,Z]$, constant effect: OLS
→ Don't trust linearity, trust your outcome model: BDA predict-and-average
→ Don't trust outcome model, trust propensity model: IPW
→ Don't trust either alone: Doubly-robust (IPW + regression)
→ Complex $g(Z)$, need valid CIs: Double ML
→ Want treatment effect heterogeneity $\tau(Z)$: R-Learner
5
Always do sensitivity analysis.
No matter which method: your result rests on the assumption that $Z$ is sufficient. Sensitivity analysis quantifies how much that assumption can be violated before your conclusion breaks.

One-Sentence Summary of Each Method

Stratification — Split the data by $Z$, compare within strata, weight by $P(Z)$
Matching — Find a twin with similar $Z$ (or propensity score), compare outcomes
IPW — Reweight every unit by $1/P(X|Z)$ to simulate an experiment
OLS — Fit a line with $Z$ included, read off $\beta_X$ as the ATE
BDA predict-and-average — Fit any model, predict under $X{=}1$ and $X{=}0$ for everyone, average
Double ML — Let ML handle confounding ($g(Z)$), OLS estimates the causal parameter ($\delta$)
Doubly-robust — Belt and suspenders: consistent if EITHER model is correct
Sensitivity Analysis — Stress-test: how strong must a hidden confounder be to kill the effect?

Concrete Example: Hormone Replacement Therapy (1990s vs WHI 2002)

Throughout the 1990s, the Nurses' Health Study and other observational studies showed HRT users had ~50% lower cardiovascular events than non-users. Doctors prescribed HRT widely for heart-disease prevention.

In 2002, the Women's Health Initiative — a large RCT — found HRT increased cardiovascular events. The observational result was wrong by sign. What confounder did the studies miss? The healthy-user effect: women who started HRT were more health-conscious in many other ways (diet, exercise, screening, SES) than non-users.

A retrospective sensitivity analysis on the original studies asks: how strong would the unmeasured confounder need to be to explain away the observed correlation? Estimates suggest a confounder $U$ with effect ~0.3 on selection and ~0.4 on outcome — about the magnitude of "is this person a regular exerciser?" — was sufficient. The breakdown point was small, and the result should have been flagged as fragile before the RCT contradicted it.

What this illustrates: graphs can be wrong; sensitivity analysis quantifies how wrong they're allowed to be; a small breakdown point is a warning sign even when the point estimate is large.

L7: Instrumental Variables

When the back-door criterion fails — because you have an unobserved confounder $U$ between treatment $X$ and outcome $Y$ — you can sometimes exploit an instrumental variable $Z$ that affects $Y$ only through $X$.

The IV Setup Core

The graph: $Z \to X \to Y$ with a hidden confounder $U \to X$, $U \to Y$.

We cannot adjust for $U$ because it is unobserved. But $Z$ provides a "natural experiment" — variation in $X$ that is exogenous (not driven by $U$).

Classic example: Quarter of birth $(Z)$ → Years of education $(X)$ → Earnings $(Y)$
Hidden confounder $U$ = ability, motivation
Three IV Conditions Exam
  1. Relevance: $Z$ must affect $X$. $\text{Cov}(Z, X) \neq 0$. Testable from data.
  2. Exclusion restriction: $Z$ affects $Y$ only through $X$. No direct $Z \to Y$ edge. Not testable — domain knowledge only.
  3. Independence: $Z \perp U$. The instrument is not confounded with the outcome. Not testable.

Two of three conditions are untestable. IV validity ultimately rests on domain knowledge and argumentation, not on statistical tests.

Wald Estimator Method

When $Z$ is binary (0/1):

$\hat{\delta}_{IV} = \frac{E[Y \mid Z=1] - E[Y \mid Z=0]}{E[X \mid Z=1] - E[X \mid Z=0]} = \frac{\alpha_1 \cdot \delta + \gamma}{\alpha_1}$

The numerator is NOT zero when exclusion holds. $Z$ still affects $Y$ — through $X$. The reduced form $E[Y|Z\!=\!1] - E[Y|Z\!=\!0] = \alpha_1 \cdot \delta$ captures the indirect path $Z \to X \to Y$. The denominator $\alpha_1$ rescales this to a per-unit-of-$X$ effect.

Key decomposition: When exclusion holds ($\gamma = 0$): $\hat{\delta}_{IV} = \alpha_1 \delta / \alpha_1 = \delta$. When violated: $\hat{\delta}_{IV} = \delta + \gamma/\alpha_1$.

Caution: $\gamma$ is the direct $Z \to Y$ effect, NOT $\text{Cov}(Z,Y)$. The total $\text{Cov}(Z,Y) \propto \alpha_1 \delta + \gamma$ includes the indirect path. Even a valid instrument has nonzero $\text{Cov}(Z,Y)$.

Cov-Form Wald & Bias Decomposition Exam

For continuous $Z$, the equivalent form (under the SEM $Y = \delta D + U + \epsilon_Y$):

$\hat\delta_{IV} = \dfrac{\mathrm{Cov}(Y, Z)}{\mathrm{Cov}(D, Z)}$

Three additional assumptions on top of exclusion:

  1. $\delta \perp \{D, Z\}$ — the effect doesn't vary with $D$ or $Z$. Satisfied if $\delta$ is constant or independently random.
  2. The exclusion restriction holds.
  3. The $Z \to D$ relationship is not weak.

Why "not weak": the estimator decomposes as

$\hat\delta_{IV} = \delta + \dfrac{\mathrm{Cov}(U,Z) + \mathrm{Cov}(\epsilon_Y,Z)}{\mathrm{Cov}(D,Z)}$

The numerator is bias from any small violation of the exclusion restriction. As $\mathrm{Cov}(D,Z) \to 0$, that bias blows up. Weak instruments amplify any leakage.

Diagnostic: first-stage F-statistic. Rule of thumb: $F \geq 10$.

Exclusion Restriction (Graphical) Core

The course's formal definition:

$(Z \perp Y)_{G_{\overline{D}}}$

$Z$ is independent of $Y$ once you delete the edges into $D$ in the graph. Qualitatively: $Z$ causes $Y$ only through $D$.

Contaminated version (still works for TSLS): $(Z \perp Y \mid W)_{G_{\overline{D}}}$ for some set $W$. Include $W$ in both stages of TSLS.

Worked example — non-compliance: users assigned to see an ad ($Z$). Only those who open the app see it ($D$). Latent budget $U$ confounds $D$ and purchases $Y$. In $G_{\overline D}$ both $Z \to D$ and $U \to D$ are removed; only $U \to Y$ remains, $Z$ is isolated, $(Z \perp Y)_{G_{\overline D}}$ holds. ✓

LATE: Revised Assumptions Exam

When effects are heterogeneous and $\delta \not\perp \{D, Z\}$, the same Wald estimator instead identifies the LATE on a subgroup $G$:

$E[\delta \mid G = g]$

For binary $Z$ and $D$, the revised assumptions:

  1. The exclusion restriction (or its contaminated form).
  2. $\kappa \neq 0$ for some — the instrument isn't weak.
  3. Monotonicity: $\kappa \geq 0$ xor $\kappa \leq 0$ for all units. (Replaces $\delta \perp \{D, Z\}$.)

If monotonicity holds with $\kappa \geq 0$, $G$ is the compliers; if $\kappa \leq 0$, the defiers. The two groups are mutually exclusive once a direction is fixed. All TSLS code is unchanged — only the interpretation shifts.

Two-Stage Least Squares (2SLS) Method

Generalization of Wald to continuous/multi-valued instruments:

Stage 1: Regress $X$ on $Z$: $\hat{X} = \hat{\alpha}_0 + \hat{\alpha}_1 Z$
Stage 2: Regress $Y$ on $\hat{X}$: $Y = \beta_0 + \beta_1 \hat{X} + \epsilon$

$\hat{X}$ is the exogenous part of $X$ — the variation predicted by the instrument, purged of confounding. The Stage 2 coefficient $\beta_1$ is the causal effect.

Why two stages? Direct regression of $Y$ on $X$ is biased by $U$. By using only the $Z$-predicted part of $X$, we break the $U \to X$ path.

LATE: Local Average Treatment Effect Warning

With heterogeneous effects and imperfect compliance, IV estimates the LATE — the causal effect for compliers only:

$\text{LATE} = E[\delta_i \mid \text{complier}]$

Four subgroups defined by how units respond to the instrument:

Type$Z=0$$Z=1$Response
Compliers$X=0$$X=1$Follow the instrument
Always-takers$X=1$$X=1$Always treated
Never-takers$X=0$$X=0$Never treated
Defiers$X=1$$X=0$Do the opposite

Monotonicity assumption: No defiers exist. Required for LATE to be well-defined. Always-takers and never-takers contribute nothing — the IV effect is entirely driven by compliers.

Contaminated Instruments Warning

What if the exclusion restriction is slightly violated? The IV estimate becomes:

$\hat{\delta}_{IV} = \delta + \frac{\gamma}{\alpha_1}$

The bias $\gamma/\alpha_1$ depends on the ratio of violation size to first-stage strength. With a weak instrument (small $\alpha_1$), even a tiny $\gamma$ gets amplified massively.

Comparison to naive OLS: OLS gives $\hat{\beta} = \text{Cov}(X,Y)/\text{Var}(X) = \delta + \text{bias from }U$. OLS picks up ALL $X$-$Y$ association (causal + confounding). IV isolates the $Z$-driven variation but leaks any direct $Z \to Y$ path. When $\gamma/\alpha_1 >$ confounding bias, IV is worse than doing nothing.

Interactive: The IV Logic — Path Decomposition

The Wald estimator divides the reduced-form effect (how Z moves Y) by the first-stage effect (how Z moves X). Drag the sliders to see how paths combine and how violations propagate.

Concrete Example: Quarter of Birth & Returns to Schooling (Angrist–Krueger 1991)

Effect of years of schooling ($D$) on log earnings ($Y$). Standard OLS gives 7–9% return per year, but ability and motivation $U$ confound: more capable people get more schooling and earn more independently.

The instrument: quarter of birth ($Z$). US compulsory-schooling laws kicked in at age 16. A student born in Q1 starts school at ~6.75 years old and reaches 16 with fewer years of schooling than a Q4-born student who starts at ~6. Q1-borns end up with about 0.1 fewer years of schooling. Quarter of birth is plausibly random with respect to ability — exactly the exclusion-restriction story we need.

Wald: $\hat\delta = (\bar Y_{Q4} - \bar Y_{Q1}) / (\bar D_{Q4} - \bar D_{Q1}) \approx 0.01 / 0.1 = 0.10$. About 10% return per year — slightly higher than OLS, opposite of what you'd expect if ability bias inflated OLS upward.

Caveats: Bound, Jaeger & Baker (1995) showed the first stage was weak — small $\mathrm{Cov}(D, Z)$ amplifies any leakage of $Z$ directly into $Y$ (e.g., season-of-birth health effects). And the LATE applies only to compliers: students whose schooling was actually pushed by compulsory-schooling laws — typically lower-income students near the dropout margin. Generalizing to all students requires extra assumptions.

What this illustrates: the exclusion restriction (untestable, debated for QoB); weak-instrument bias amplification; LATE on a specific complier subpopulation, not the full ATE.

L8: Front-Door Criterion & Experiments

Two parallel themes for L8. First: another escape from unblockable confounding — the Front-Door Criterion (FDC), which identifies $P(Y \mid \mathrm{do}(X))$ via a fully observed mediator. Second: experiments and effect decomposition. The two are linked: FDC asks "what is the total $X \to Y$ effect through the mechanism?"; mediation asks "how much of that effect goes through which mechanism?"

Front-Door Criterion: Three Conditions

A set $Z$ satisfies the FDC relative to ordered pair $(X, Y)$ if:

  1. $Z$ intercepts all directed paths from $X$ to $Y$. Every causal arrow from $X$ to $Y$ must pass through some node in $Z$.
  2. No unblocked back-door path from $X$ to $Z$. The mediator's value, given $X$, is "as-if random."
  3. All back-door paths from $Z$ to $Y$ are blocked by $X$. Conditioning on $X$ closes any open path from $Z$ to $Y$.

Worked example: two mediators

Treatment $X$ acts on outcome $Y$ through two parallel mediators $Z_1$ and $Z_2$. An unobserved confounder $U$ affects both $X$ and $Y$ but does not reach $Z$ except through $X$. Take $Z = \{Z_1, Z_2\}$ — does it satisfy the FDC?

X Z₁ Z₂ Y U unobserved causal confounding
✓ 1. $Z$ intercepts all directed $X \to Y$ paths.

Trace forward: every solid arrow from $X$ to $Y$ passes through $Z_1$ or $Z_2$ — namely $X \to Z_1 \to Y$ and $X \to Z_2 \to Y$.

✓ 2. No open back-door from $X$ to $Z$.

Back-door paths from $X$ must start by going up the dashed $X \leftarrow U$ edge. The only routes that reach $Z$ are $X \leftarrow U \to Y \leftarrow Z_1$ and $X \leftarrow U \to Y \leftarrow Z_2$ — both blocked by the collider at $Y$. This works because there's no $U \to Z$ edge in the graph; if you added one, $X \leftarrow U \to Z$ would be an unblocked back-door and condition 2 would fail.

✓ 3. All back-doors from $Z$ to $Y$ are blocked by $X$.

From $Z_1$, back-door paths are $Z_1 \leftarrow X \to Z_2 \to Y$ and $Z_1 \leftarrow X \leftarrow U \to Y$ (and symmetrically for $Z_2$). $X$ sits as a fork or chain node on every one of them — conditioning on $X$ blocks them all.

All three conditions hold $\Rightarrow$ FDC adjustment via $\{Z_1, Z_2\}$ identifies $P(Y \mid \mathrm{do}(X))$ even though $U$ is never measured.

FDC: Formula, Variants, Applicability

Front-Door Adjustment Formula Core
$P(Y \mid \mathrm{do}(X{=}x)) = \displaystyle\sum_{z} P(Z{=}z \mid X{=}x) \sum_{x'} P(Y \mid X{=}x', Z{=}z)\, P(X{=}x')$

Read it as a two-stage chain rule:

  • $P(Z \mid X)$ — how the mediator responds to the treatment (this is causal because of condition 2).
  • $\sum_{x'} P(Y \mid X{=}x', Z{=}z)\, P(X{=}x')$ — the back-door-adjusted effect of $Z$ on $Y$, where $X$ blocks the back-door (condition 3).

No new assumptions beyond the graph + the usual modeling assumptions: sufficient capacity and the right loss function for whichever conditional distributions you fit.

Per-Mechanism Shortcut Trick

If the structural equation is additive across mediators:

$Y = Y_1(Z_1, X) + Y_2(Z_2, X) + \cdots + Y_k(Z_k, X)$

Then you can estimate the effect through each mediator independently — you don't need to carry the full $Z$. Useful when one mediator is observed but others aren't, or for attribution.

When FDC Beats Back-Door Exam

Use FDC when:

  • Confounder $U$ between $X$ and $Y$ is unmeasured or unmeasurable.
  • You can measure the mechanism (mediator) cleanly.
  • The mediator captures all of the $X \to Y$ effect (condition 1).

Classic example: price → funnel metrics → purchases. Customer budget $U$ confounds price and purchases (via personalization), but the funnel is fully measured.

Experiments & Decomposition

Randomized experiments are the gold standard for causal inference. When you control assignment, confounding disappears by design. But real experiments introduce complications: non-compliance, time-varying effects, and the challenge of decomposing how a treatment works.

Why Experiments Work Core

Random assignment ensures $D \perp (Y^{(D=0)}, Y^{(D=1)})$:

$E[Y \mid D=1] - E[Y \mid D=0] = \text{ATE}$

The naive estimator is the causal effect. No adjustment needed. No unobserved confounders to worry about — because the graph has no edges into $D$ except from the randomization mechanism you control.

A/B Testing Framework Core

Apply treatment at time $t$ to a test group ($D_{it}=1$) and not to control ($D_{it}=0$). Treatment stays on; control stays off.

Define $D_i^* = 1$ if ever treated, 0 otherwise. The effect of $D_i^*$ on aggregate outcome $Y$ is identified by the experiment.

But: The effect of being treated at a specific time $D_{it}$ is harder — because outcomes carry forward (cumulative effects).

Time-Varying Treatment Effects Warning

$\delta_t$ can change over time in several important ways:

  • Novelty effects: Users explore a new feature more at first, engagement decays to steady-state
  • Hawthorne effect: Behavior changes because subjects know they're being observed (factory lighting study)
  • Cold start: Recommendation systems need data to work well — early performance understates long-run effect

To detect these, measure the effect at each time point $t$ separately. Look for stabilization. Due to smaller per-period samples, error analysis is especially important.

Effect Decomposition Method

Once we know the ATE, we may want to understand how the treatment works — through which mechanisms.

$Y = f(X, M_1, M_2, \ldots) + \epsilon$

Linear decomposition: If $Y = \beta_X X + \beta_{M} M + \epsilon$ and $M = \alpha X + \eta$, the total effect decomposes into:

  • Direct effect: $\beta_X$ (treatment $\to$ outcome, not through mediators)
  • Indirect effect: $\beta_M \cdot \alpha$ (treatment $\to$ mediator $\to$ outcome)

Neural decomposition: Uses neural networks to model non-linear mechanisms. Allows latent mediators and complex interactions. Identifies structural components without imposing linearity.

M (Mediator) vs Z (Instrument) — Mirror Images

Instrument $Z$Mediator $M$
Graph$Z \to X \to Y$$X \to M \to Y$
PositionUpstream of treatmentDownstream of treatment
GoalEstimate the total effect of $X$ on $Y$Decompose the total effect into direct vs. indirect
Why neededBack-door fails (unobserved $U$)Back-door works, but we want how the effect operates
Key dangerConditioning on $Z$? No — $Z$ is exogenousConditioning on $M$ opens collider bias if $M$ has other causes

The graphs are structurally similar ($A \to B \to C$) but the question is different. IV asks "what is the total $X \to Y$ effect?" Decomposition asks "how much of $X \to Y$ goes through $M$?"

Intent-to-Treat vs Per-Protocol Exam

When subjects don't comply with their assignment:

AnalysisWhat it comparesWhat it estimates
Intent-to-Treat (ITT)Groups as assignedEffect of assignment — always unbiased
Per-ProtocolGroups by actual complianceEffect of treatment — potentially biased (compliers ≠ non-compliers)
IV / LATEAssignment as instrument for treatmentEffect for compliers — unbiased if monotonicity holds

Connection to L7: The assignment $Z$ is a natural instrument for actual treatment $X$. The Wald estimator (ITT effect ÷ compliance rate) = LATE for compliers. ITT is the safe analysis; IV/LATE recovers the per-unit treatment effect without the selection bias of per-protocol analysis.

Interactive: A/B Test with Time-Varying Effects

See how novelty effects, cold start, and Hawthorne effects distort A/B test results over time. Toggle each phenomenon on/off.

Concrete Example: Smoking → Tar → Cancer (Pearl's Front-Door)

In the 1950s, tobacco companies argued the smoking–cancer correlation was confounded by a hypothetical "smoking gene" $U$: $U \to$ smoking and $U \to$ cancer. We can't measure $U$. But suppose we can measure tar deposits in lung tissue ($Z$).

Verify the FDC for $X = $ smoking, $Y = $ cancer, $Z = $ tar:

  1. $Z$ intercepts $X \to Y$: tar deposits sit on every causal path from smoking to lung cancer (assumed mechanism — biologically plausible). ✓
  2. No back-door $X \to Z$: the smoking gene doesn't cause tar directly — only via smoking. ✓
  3. Back-doors $Z \to Y$ blocked by $X$: smoking is the only common cause of tar and cancer; conditioning on $X$ closes the path. ✓

FDC adjustment then identifies $P(\text{cancer} \mid \mathrm{do}(\text{smoking}))$ from purely observational data — no need to measure $U$. This was Pearl's killing argument: even if the smoking gene exists, the front-door identifies the causal effect.

Companion: novelty effect in A/B testing

A streaming service tested a new recommendation algorithm. Day 1: +12% engagement (test vs control). Day 30: +3%. The decision depends on which number you trust. The 12% is novelty (exploration); the 3% is steady-state. The honest A/B report uses the post-stabilization number and flags the trajectory.

What this illustrates: FDC identifies $P(Y \mid \mathrm{do}(X))$ via a fully-observed mediator when the back-door fails; A/B tests must run long enough for time-varying effects to stabilize.

L9: Discontinuity Designs & Time Series

When experiments aren't possible and instruments aren't available, we can exploit natural cutoffs (regression discontinuity) or temporal structure (ITS, DID) to identify causal effects. These methods trade generality for credibility — they estimate local effects near specific boundaries or time points.

Regression Discontinuity

Sharp RD Design Method

Treatment is completely determined by a running variable $Z$ crossing a cutoff $z_0$:

$D_i = \mathbf{1}[Z_i \geq z_0]$

Units just above $z_0$ are nearly identical to units just below — like a local experiment.

$\text{ATE}_{RD} = \lim_{z \downarrow z_0} E[Y \mid Z=z] - \lim_{z \uparrow z_0} E[Y \mid Z=z]$

Example: Scholarship eligibility at GPA = 3.0. Compare students at 3.01 vs 2.99.

The effect is local: it applies only at the cutoff, not to the full population.

Fuzzy RD Design Method

The cutoff increases the probability of treatment but doesn't determine it completely:

$P(D=1 \mid Z) \text{ jumps at } z_0 \text{ but not from 0 to 1}$

This is exactly an IV problem at the discontinuity. The cutoff indicator is the instrument:

$\hat{\delta}_{FRD} = \frac{\lim_{z \downarrow z_0} E[Y|Z=z] - \lim_{z \uparrow z_0} E[Y|Z=z]}{\lim_{z \downarrow z_0} E[D|Z=z] - \lim_{z \uparrow z_0} E[D|Z=z]}$

The numerator is the jump in outcomes; the denominator is the jump in treatment take-up. This is the Wald estimator at the cutoff.

Interpretation: LATE for compliers at the cutoff — units who are treated because of the cutoff.

RD Threats: Manipulation & Pileup Warning

If units can manipulate the running variable to cross the cutoff, the "as-if random" logic breaks:

  • Students retaking exams to cross the GPA threshold
  • Firms manipulating revenue to cross regulatory cutoffs

Diagnostic: Check for bunching (pileup) in the density of $Z$ at $z_0$. A smooth density supports the design; a spike suggests manipulation.

McCrary test: Formal test for discontinuity in the running variable's density.

Sharp RD: Full Assumption List Exam

From the Review 2 deck — what to verify on the exam:

  • Sharpness: $P(D \mid Z = z_0^+) - P(D \mid Z = z_0^-) = 1$. Treatment really flips at $z_0$.
  • Continuity of potential outcomes in the running variable at $z_0$ — if you didn't flip $D$, $Y$ would be continuous; nothing else changes at $z_0$.
  • $Z$ is a back-door set at the cutoff: $Y^{(D=d)} \perp D \mid Z = z_0$.
  • No manipulation of the running variable (density check / McCrary).

Estimator best practices: use a loss consistent with conditional expectations; avoid global polynomials (sensitive to data far from the cutoff); prefer local linear regression within a bandwidth.

Time Series Designs

Panel Data Framework Core

Generalize from cross-section to $N$ units observed over $T$ time periods:

Treatment: $D_{it}$   Outcome: $Y_{it}$   Potential outcomes: $Y_{it}^{(D=d)}$
Individual effect: $\delta_{it} = Y_{it}^{(D=1)} - Y_{it}^{(D=0)}$

Data types by structure:

TypeUnitsTimeStructure
Cross-sectional$N$ fixed$t=1$One snapshot
Time series$N=1$$T$ periodsOne unit over time
Panel$N$ fixed$T$ discreteDense matrix $N \times T$
Pooled cross-sectionVarying$T$ periodsDifferent units each period
Rolling cross-sectionVaryingContinuousUnits measured at arbitrary times
Interrupted Time Series (ITS) Method

A single unit, measured over time. Treatment turns on at $t^*$. Estimate $\delta_{it}$ by extrapolating the pre-treatment trend into the post-treatment period:

$\hat{\delta}_{it} = Y_{it}^{(D=1)} - \hat{Y}_{it}^{(D=0)}$

The counterfactual $\hat{Y}^{(D=0)}$ is what would have happened without treatment, estimated from pre-treatment data.

Key assumption: The pre-treatment trend would have continued. This can fail due to:

  • Anticipation: Unit changes behavior before treatment (e.g., switch products before a feature is removed)
  • Simultaneous changes: Other events happen at $t^*$ (e.g., policy bundled with others)
  • Confounding: Outside factors drive both the treatment timing and the outcome
ITS Robustness Checks Exam

Strategies to build confidence that the ITS effect is real:

  1. Examine other outcomes that should be affected — if marketing drives visits, do sign-ups also increase?
  2. Examine outcomes that should NOT be affected — if only new visits change, baseline user behavior should be stable
  3. Assess subgroup effects — do effects vary predictably? (e.g., targeted users respond more)
  4. Adjust for concurrent trends — control for other variables that changed around $t^*$
  5. Compare with other units — similar websites/products that didn't receive treatment
  6. Is removing the cause impactful? — when the campaign ends, does the effect go away?
Difference-in-Differences (DID) Method

Multiple units, some treated at $t^*$, some not. The DID regression:

$E[Y_{it} \mid D^*=d, T=t] = \beta_0 + \beta_t \cdot t + \beta_D \cdot d + \beta_{DT} \cdot td$

The causal effect is $\delta = \beta_{DT}$ — the interaction term.

DID formula:

$\hat{\delta}_{DID} = \big(E[Y|D^*\!=\!1,t\!=\!1] - E[Y|D^*\!=\!1,t\!=\!0]\big) - \big(E[Y|D^*\!=\!0,t\!=\!1] - E[Y|D^*\!=\!0,t\!=\!0]\big)$

What DID removes: Constant confounders $U$ (they cancel in the first difference). $\beta_D$ captures baseline group differences; $\beta_t$ captures time trends; $\beta_{DT}$ is what's left = the treatment effect.

DID: Parallel Trends Assumption Warning

DID assumes the change in the confounder is the same for treatment and control groups.

$E[U_{i1} - U_{i0} \mid D^*=1] = E[U_{i1} - U_{i0} \mid D^*=0]$

If this fails — e.g., the treatment group was already trending differently — DID is biased.

Time-varying confounders: If $U_{it}$ changes differently across groups, the DID estimator includes bias: $\gamma \big(E[\Delta X_i | D^*\!=\!1] - E[\Delta X_i | D^*\!=\!0]\big)$

Fix: Add time-varying covariates $X_{it}$ to the regression: $E[Y_{it}|D^*\!=\!d, T\!=\!t, X_{it}] = \beta_0 + \beta_t t + \beta_D d + \beta_{DT} td + \gamma X_{it}$

Valid as long as $X_{it}$ is not itself affected by the treatment (not a post-treatment variable).

Interactive: Regression Discontinuity

See the sharp RD design in action. Adjust the bandwidth to see how the local estimate changes near the cutoff.

Interactive: Difference-in-Differences

See how DID isolates the treatment effect by differencing away both group differences and time trends. Toggle violations to see how bias enters.

Concrete Examples: Card–Krueger DID, Thistlethwaite RDD, California Prop 13 ITS

DID — Card & Krueger (1994), NJ minimum wage

In April 1992, New Jersey raised its state minimum wage from $4.25 to $5.05; Pennsylvania kept $4.25. Card and Krueger surveyed fast-food restaurants in NJ and PA border counties before (Feb–Mar) and after (Nov–Dec) the change. DID:

$\hat\delta = (\bar Y_{NJ,\text{after}} - \bar Y_{NJ,\text{before}}) - (\bar Y_{PA,\text{after}} - \bar Y_{PA,\text{before}})$

Result: NJ employment increased by ~0.6 workers per restaurant relative to PA — contradicting the naive supply-demand prediction. Parallel trends was credible because NJ and PA border counties shared regional shocks.

RDD — Thistlethwaite & Campbell (1960), original RD paper

National Merit Scholarship awards were assigned by a sharp test-score cutoff. Students one point above received a Certificate of Merit; students one point below didn't. Compare college outcomes for students at $\text{score} = c+1$ vs $\text{score} = c-1$. Local linear regression on each side, evaluated at $c$, gives the CATE at the cutoff. The famous "as-if random" intuition: at the boundary, who got the certificate is essentially a coin flip.

ITS — California Proposition 13 (1978)

Prop 13 capped property taxes, slashing local school revenue. Pre-1978 trend in per-pupil spending: rising 4%/yr. Post-1978: flat for 3 years, then resumed at 2%/yr. ITS effect = observed post-period minus extrapolated pre-trend. Caveat: anything else statewide that changed in 1978 would confound this — and it's a single unit, no control group.

What this illustrates: DID isolates a treatment effect by differencing out time-invariant confounding (parallel trends); RDD exploits a deterministic threshold for "as-if random" assignment locally; ITS extrapolates the counterfactual from a single unit's history when no comparison group exists.

L10–11: Time Series, Panel Designs & Selection Bias

When we move from cross-sectional data to time series and panels, new phenomena arise: dependence over time, regime shifts, and latent factors. The fundamental problem evolves — now we need to account for when treatment happens, not just whether it happens.

Time-Series Causal Graphs

Time Dependence & Stationarity Core

In time series, each variable at time $t$ is a separate node in the causal graph. Dependence over time creates edges like $S_{t-1}^a \to S_t^a$ (autocorrelation).

Stationarity assumption: The structural function is the same at every time step: $S_t = f(S_{t-1})$ rather than $S_t = f_t(S_{t-1})$.

This is the time-series analog of i.i.d. — it lets us pool data across time to estimate a single causal function.

Markov assumptions simplify the graph: dependence only goes back $k$ steps, like an ARMA model: $Y_t = \beta_1 Y_{t-1} + \beta_2 Y_{t-2} + \beta_3 Y_{t-3} + \epsilon_t$.

Back-Door in Time Series Core

The graph: $X_t \to S_t \leftarrow S_{t-1} \to X_t$ (and more complex interconnections with $U_t$).

Past values of outcomes ($S_{t-1}$) lie along back-door paths. Past causal states ($X_{t-1}$) can confound future outcomes.

Good news: Under stationarity, the back-door adjustment set is the same at every $t$: $Z = \{S_{t-1}^a, U_t\}$. We can apply the familiar back-door criterion — we just need to include lagged variables.

Causal Forecasting

Statistical vs Causal Forecasting Exam

Pure prediction: estimate $P(Y_{t+1} | X_t)$ by fitting $\hat{Y}_{t+1} = \hat{\beta} X_t$.

With a confounder $U_{t-1}$:

$X_t = \beta U_{t-1} + \epsilon_{X,t}$
$Y_{t+1} = \beta_X X_t + \beta_U U_{t-1} + \epsilon_{Y,t+1}$

The naive OLS slope has omitted variable bias:

$\hat{\beta} = \beta_X + \beta_U \cdot \frac{\beta \sigma_U^2}{\beta^2 \sigma_U^2 + \sigma_{\epsilon_X}^2}$

The bias depends on $\sigma_{\epsilon_X}^2$ — the exogenous variance of $X$.

Regime Shifts Break Statistical Models Warning

If the exogenous noise $\sigma_{\epsilon_X}^2$ changes at time $t^*$ (a regime shift):

  • The causal mechanism ($\beta_X$, $\beta_U$, $\beta$) is unchanged
  • But the OLS bias term changes because it depends on $\sigma_{\epsilon_X}^2$
  • A model fit on pre-$t^*$ data gives wrong predictions after $t^*$
Key insight: Causal models are robust to regime shifts because structural equations ($Y = \beta_X X + \beta_U U + \epsilon$) are invariant to changes in exogenous noise distributions. The naive statistical model must be re-fit after every regime shift — and you won't have data from the new regime to fit it!

Matrix Completion (MC-NNM)

The Missing Data Insight Core

In panel data with treatment switching on/off, we observe two incomplete matrices:

$Y_{it}^{(D=0)}$: observed when $D_{it}=0$, missing when $D_{it}=1$
$Y_{it}^{(D=1)}$: observed when $D_{it}=1$, missing when $D_{it}=0$

They are exactly complementary: every cell is observed in one matrix and missing in the other.

Causal inference = matrix completion: Fill in the missing entries to construct counterfactuals.

MC-NNM Framework Method

Assume the outcome matrix is low-rank (driven by a few latent factors). Recover it by nuclear norm minimization:

$\min_{Y^c} \frac{1}{|M|} \|M \circ (Y^{(D=d)} - Y^c)\|_F^2 + \lambda \|Y^c\|_*$

where $\|Y^c\|_*$ is the nuclear norm (sum of singular values) and $M$ is the observation mask.

Key assumptions:

  1. Low-rank structure: Outcomes driven by a small number of latent factors
  2. Missing at random: Conditional on observed potential outcomes, missingness pattern is independent of unobserved potential outcomes

Solved via: Proximal gradient descent with singular value thresholding (SVT).

Latent Factors & Selection Bias Core

Latent factors $F_1, F_2, \ldots$ jointly determine both treatment assignment and outcomes. This creates selection bias.

MC-NNM handles this by exploiting shared low-rank structure: if the same factors drive both treatment and outcomes, the matrix structure encodes the confounding — and completing the matrix implicitly adjusts for it.

When does it work?

  • The latent factor must drive many observed variables — otherwise we can't infer it
  • Works naturally with constant confounders (factor varies across units but not time)
  • Flexible with staggered adoption — different units treated at different times
Treatment-Confounder Feedback Warning

In dynamic settings, past treatment $X_{i0}$ can influence a confounder $Z_{it}$, which then affects future treatment $X_{it}$ and outcome $Y_{it}$.

$X_{i0} \to Z_{it} \to X_{it} \to Y_{it}$ (with $U_{it} \to Z_{it}$)

This creates a feedback loop between treatment and confounders. Standard back-door adjustment fails because $Z_{it}$ is both a confounder (it affects $Y$) and a mediator (it's affected by past $X$).

Solution: Sequential adjustment methods — adjust for $Z_{it}$ at each time step separately, being careful about conditioning on post-treatment variables.

MC-NNM Advantages Method
  • Handles staggered adoption (different treatment times for different units)
  • Admits uncertainty quantification via bootstrapping or Bayesian extensions
  • Combines regression across units and across time in a single framework
  • Can incorporate control variables into the loss function
  • Connects matrix completion (ML/RecSys) with causal inference (econometrics)

Interactive: Matrix Completion for Causal Inference

See how the potential outcome matrices relate. Click cells to toggle treatment assignment and watch the missingness pattern update.

Interactive: Regime Shift — Statistical vs Causal Forecasting

Compare how statistical (OLS) and causal models respond when the noise regime changes at $t^*$. Drag the noise slider to induce a regime shift.

Selection Bias Adjustment (L11)

Your sample is drawn under selection $S=1$ that depends on observables. The joint $P(X, Y \mid S=1)$ in your data isn't the population joint $P(X, Y)$. Adjustment is possible iff there's a covariate set $C$ that mediates selection.

Theorem: s-Bias Adjustment Core

If $C$ is measured both in the biased sample (alongside $X$ and $Y$) and at the population level (alongside $X$), and if $Y \perp S \mid \{X, C\}$, then $P(Y \mid X)$ is s-recoverable:

$P(Y \mid X) = \displaystyle\sum_c \underbrace{P(Y \mid X, S=1, C=c)}_{\text{sample}} \cdot \underbrace{P(C=c \mid X)}_{\text{population}}$

Critical: the assumption $Y \perp S \mid \{X, C\}$ is untestable — you don't have population-level $Y$ data, otherwise you wouldn't need the adjustment. Justify it with domain knowledge about what drives selection.

Three Selection-Bias Estimators

Post-Stratification Weighting Estimator

When: $X = \emptyset$ (you want a population-level expectation, not a conditional). And you only need an average, not a distribution.

$\hat E[Y] = \displaystyle\sum_c E[Y \mid S=1, C=c]\, P(C=c)$

Per-row weights: $w_i = P(C=c_i)/P(C=c_i \mid S=1)$. Standard survey-research move: weight your sample to match population strata (Census).

Inverse Selection-Propensity Weighting Estimator

When: you have a model for $P(S=1 \mid X, C)$ — or you can recover it from joint measurements:

$P(S=1 \mid X, C) = \dfrac{P(X, C \mid S=1)}{P(X, C)} P(S=1)$

Optimize a weighted loss to fit any model without selection bias:

$E_P[\mathcal L] = \displaystyle\sum_{i \in S} \mathcal L(x_i, y_i) \cdot \dfrac{1}{P(S=1 \mid X=x_i, C=c_i)}$
Time-Series s-Bias Adjustment Estimator

When: the population composition is roughly constant in time (or varies slowly enough to ignore) — so $N$ and $P(C=c)$ can be treated as fixed. Fit OLS to estimate $NP(C=c)$:

$N E[Y_t] = \displaystyle\sum_c E[Y_t \mid S=1, C=c]\, NP(C=c)$

Use when you observe $Y_t$ over time in a biased panel and want population-level $E[Y_t]$ trajectory.

When adjustment fails: if selection $S$ is on a descendant of the outcome (e.g. a 30-second dwell-time filter where engaged users dwell longer and buy more), no reweighting on observed covariates can recover $P(Y \mid X)$. The fix is upstream — remove the filter at the data source — not a clever weighting scheme.

Concrete Examples: Prop 99 Synthetic Control + 2016 Polling Bias

Synthetic control — California Proposition 99 (1988)

California's 1988 cigarette tax raised prices and bundled anti-smoking ads. Did per-capita cigarette sales fall because of the policy? The clever construction: build a "synthetic California" as a weighted combination of other states (Utah, Nevada, Montana, Connecticut, Colorado) chosen so the synthetic series matches California's pre-1988 sales trajectory. After 1988, real and synthetic California diverge — by 2000, real CA sells ~26 packs/person/year vs synthetic CA's ~60. Estimated effect: −34 packs/person.

MC-NNM generalizes this: instead of restricting weights to convex combinations of donor states, let nuclear-norm minimization fill in the missing $Y^{(D=0)}$ entries directly. This handles staggered adoption (different states implement different policies at different times) without specifying donor pools by hand.

Selection bias — 2016 election polling

Most major polls in October 2016 had Clinton up by 3–4 points. Trump won the Electoral College. Why did the polls miss? Online polls and even RDD telephone polls over-represented college-educated, urban respondents. Many pollsters did weight by age and region — but missed the college / non-college divide within white voters, which had become the dominant predictor of vote choice. The right method (post-stratification) was applied to the wrong covariate set $C$.

Formula reminder: $\hat E[\text{vote share}] = \sum_c E[\text{vote} \mid C=c, S=1]\, P(C=c)$, with marginals $P(C)$ from a voter file. The result is only as good as $C$ — if $S$ depends on a variable not in $C$, you can't fix the bias.

What this illustrates: matrix completion handles staggered, partially-observed potential outcomes with a structural assumption (low rank); s-bias adjustment fixes selection only if $C$ separates $S$ from $Y$ — picking the wrong $C$ gives confidently wrong answers.

Estimand Selection: Statistical vs Causal

The question determines the estimand, not the dataset. Pick the estimand first, then choose a method that identifies it from the data you have. Conflating "what we want to know" with "what we can compute" is the most common failure mode in applied work.

Two Definitions

Statistical Estimand Core

A functional of the observational joint distribution:

$\theta = T(P(V))$

Examples: population means, conditional probabilities, quantiles. Lives at Pearl's Level 1 (seeing).

Causal Estimand Core

A functional of an interventional distribution:

$\psi = T(P(V \mid \mathrm{do}(X)))$

Examples: ATE, ATT, CATE, LATE, elasticities. Lives at Pearl's Level 2 (doing).

Three Red Flags: When Causal Is Required

You need a causal estimand (not a statistical one) when any of these hold:

  1. Feedback. The estimand will change how $X$ is chosen. Conditional $P(Y \mid X)$ depends on the old policy; $P(Y \mid \mathrm{do}(X))$ doesn't. Any rollout, pricing change, threshold shift, or recommendation policy update is here.
  2. Shift. Exogenous distributions drift but structural equations are stable. Structural parameters transfer across the shift; regression coefficients don't. (Forecasting under regime change. Google Flu Trends: media-variance spike in 2013 changed the fitted coefficient even though the causal effect was constant.)
  3. Causal target. The question is inherently about intervention — elasticity, ATE, lift, "what if we did X?"

The 2×2 Adjustment Grid

Two questions decide the cell:

  • Is the target $P(\cdot)$ or $P(\cdot \mid \mathrm{do}(X))$? (rows)
  • Is there an open path between $S$ (selection) and $Y$? (columns)
No selection biasSelection bias
Statistical target(none — clean random sample, no adjustment needed)s-bias adjustment (post-stratification, IPSW, time-series s-bias)
Causal targetBack-door / front-door / IV / RDD / DID / ITSs-backdoor (combine selection adjustment with causal identification)

Exam discipline: name the cell first; the method follows. The empty cell (statistical target, no selection bias) is rare in business — a clean random sample of the population needs none of this.

Estimand Fallback: When You Can't Identify the One You Want

Often you start with one estimand, realize you can't identify it from the data you have, and back off to a related one:

WantCan't identify because…Fall back to
ATEHeterogeneous effects + IVLATE on compliers
ATERDD only identifies at the cutoffCATE at $z_0$
ATEDID only valid for treated groupATT on the treated
ATT in observational dataCommon support fails for some treated unitsATT on the overlap region

Each fallback narrows the population the answer applies to. Be explicit about it when reporting — "the effect on compliers near the cutoff" is a much more honest claim than "the effect."

Concrete Example: Google Flu Trends (2008–2014)

Google built a model predicting CDC flu cases from search-query volume — type "flu symptoms" or "what to take for fever," and the model added you to its estimate. From 2008 to 2012 it tracked the CDC numbers within ~2%, faster than the CDC itself. Then in January 2013, GFT predicted 11% of the population had flu; the CDC measured 6%. GFT had nearly doubled the right answer.

What broke: the relationship between search volume and flu prevalence wasn't structural — it was confounded by media coverage. In 2013, heavy news coverage of a bad flu season inflated search queries far above what people's actual flu prevalence implied. The OLS coefficient absorbed both the structural signal (sick people search more) and the media-driven amplification.

A causal forecast would have separated $P(\text{search} \mid \text{flu}, \mathrm{do}(\text{media coverage}))$ from $P(\text{search} \mid \text{flu})$. The structural mechanism stayed stable; only the exogenous distribution of media changed. Statistical models are vulnerable to this kind of shift — structural models aren't.

What this illustrates: the second of the three red flags — when exogenous distributions drift but structural equations are stable, regression coefficients don't transfer; structural parameters do. Choose the causal estimand even for a forecasting problem if the regime can shift.

Situational Drills (Final Review)

Each drill is a real business situation. None of them name an estimand up front — that's the first thing you have to decide. Three slides per drill: business problem, inference problem (what are we estimating? statistical or causal? why?), identification & estimator. Three at Level 1 (statistical), four at Level 2 (causal).

L1.1: Credit-Card Nowcast

Business Problem

Your hedge fund team trades a major US retailer's stock around its quarterly earnings announcement. To position before the print, you've licensed a credit-card transaction panel from a card issuer, hoping to nowcast retailer revenue. Catch: the cardholder base over-represents high-income, urban consumers. Earnings drop in two weeks.

Inference Problem

Estimand: Statistical: $E[\text{revenue}_m \mid q]$ at the US population level.

Why statistical? No counterfactual, no feedback, no shift — just a current-state quantity, observed under biased sampling. The trade rests on the retailer's actual reported revenue, not on changing any policy.

Identification & Estimator

Identification: Selection bias from sampling. With demographics $C$ measured both in-panel and at the population level (Census), assume $\text{revenue} \perp S \mid C$. s-bias adjustment applies.

Estimator: Post-stratification weighted sum.

$\hat R_m = \displaystyle\sum_c E[\text{revenue}_m \mid C=c, S=1]\, P(C=c)$

with weights $w_i = P(C=c_i)/P(C=c_i \mid S=1)$.

L1.2: Foot-Traffic for Lease Renewal

Business Problem

You work at a REIT. The lease on a flagship retail property comes up for renewal. You want monthly visit counts for the last 12 months. You subscribe to a geolocation panel — anonymized SDK pings. Two issues: (1) which devices are in the panel ($S_1$) depends on apps installed; (2) ping cadence ($S_2$) varies by app and location.

Inference Problem

Estimand: Statistical: $E[V_{P,t}]$ — expected monthly visits at the population level.

Why statistical? Renewal needs an honest count of how busy the property has been. No decision changes how visits are generated; only how visits are valued.

Identification & Estimator

Identification: Two layers of selection. Adjust on demographics $Z$ for $S_1$; on $(Z, L)$ for $S_2$.

Estimator: Stage-wise inverse-propensity weighting:

$w = \dfrac{1}{P(S_1 \mid Z) \cdot P(S_2 \mid Z, L, S_1=1)}$

Calibrate to a known ground-truth count (e.g. a nearby stadium event with reported attendance) before trusting absolute levels.

L1.3: Banner-Ad Reporting

Business Problem

Your e-commerce growth team launched a banner-ad campaign last week. The VP wants to know: of users who saw the banner, what fraction purchased? Catch: analytics only records sessions with dwell-time $\geq 30$s (a long-standing bot/bounce filter).

Inference Problem

Estimand: Statistical conditional: $P(\text{purchase} \mid \text{banner shown})$ over all banner viewers.

Why statistical? No intervention — just describing what happened. But verify the VP doesn't actually want a causal estimand (lift) — they often do without realizing it.

Identification & Estimator

Identification fails. $S$ (dwell-time $\geq 30$s) is on a descendant of the outcome — engaged users dwell longer and buy more. No reweighting on observed covariates recovers $P(Y \mid X)$.

Estimator: None that fixes it. Right answer: remove the filter at the source. Otherwise, give the VP a bound:

$E[Y] - E[Y \mid S=1] = P(S=0)\bigl(E[Y \mid S=0] - E[Y \mid S=1]\bigr)$

L2.1: Ad Rollout (IV)

Business Problem

Marketing wants to know whether to spend $10M next quarter rolling out a video-ad campaign. Last month they ran a small randomized pilot: half of users were eligible to be served the ad, but only $\sim 30\%$ of the eligible group ever opened the app and saw it.

Inference Problem

Estimand: Causal: $E[Y^{(D=1)} - Y^{(D=0)}]$. Under heterogeneity, retreat to LATE on compliers.

Why causal? Two red flags: (1) rollout will change how often $D$ happens (feedback), (2) the question is inherently counterfactual.

Identification & Estimator

Identification: Instrumental variables. $Z =$ randomized eligibility, $D =$ ad view. Exclusion restriction holds by design.

$\hat\delta = \dfrac{\mathrm{Cov}(Y, Z)}{\mathrm{Cov}(D, Z)}$

Or 2SLS. Under monotonicity, the estimate is LATE on compliers (the $\sim 30\%$ who view iff eligible).

L2.2: Pricing Decision (FDC)

Business Problem

Pricing team is debating dropping a SKU price by 10% next quarter. They have a year of observational sales data: every transaction has a price plus per-session funnel metrics — items viewed ($Z_1$) and time-to-checkout ($Z_2$). Customer budget $U$ confounds: discount-sensitive shoppers buy more and get shown lower prices via personalization.

Inference Problem

Estimand: Causal: $E[Y \mid \mathrm{do}(X)]$.

Why causal? The price drop is a policy change. The relationship under the current personalization regime won't transfer to a regime where everyone gets the lower price.

Identification & Estimator

Identification: Back-door could work if recommendation scores are accessible. Otherwise, since price plausibly affects purchases only through the funnel, use the front-door criterion via $\{Z_1, Z_2\}$.

$P(Y \mid \mathrm{do}(X{=}x)) = \displaystyle\sum_z P(Z{=}z \mid X{=}x) \sum_{x'} P(Y \mid X{=}x', Z{=}z) P(X{=}x')$

Or, if the funnel is additive in $Z_1, Z_2$, the per-mechanism estimator.

L2.3: Scholarship Program (RDD)

Business Problem

A provost is reviewing the merit-scholarship program. Currently, GPA $\geq 3.5$ gets the scholarship automatically. The provost is considering lowering the cutoff to 3.3. A density check at 3.5 shows no pileup.

Inference Problem

Estimand: Ideally the ATE, but the identification strategy will only get us a CATE near the existing cutoff: $E[Y^{(D=1)} - Y^{(D=0)} \mid Z = 3.5]$.

Why causal? The decision is whether to extend treatment to a new group — counterfactual.

Identification & Estimator

Identification: Sharp RDD. Assignment is deterministic at GPA $= 3.5$; potential outcomes continuous in GPA at the cutoff; density check rules out manipulation.

Estimator: Local linear regression on each side of 3.5; difference of fitted values at the cutoff.

$\hat\delta_{3.5} = E_{\text{right}}[Y \mid Z=3.5] - E_{\text{left}}[Y \mid Z=3.5]$

Caveat: extrapolating from CATE at 3.5 down to 3.3 needs a smoothness assumption — flag this for the provost.

L2.4: Minimum-Wage Policy (DID)

Business Problem

A state legislature is considering a minimum-wage hike after several nearby states raised their minimums last summer. Complication: the same treated states ended the federal pandemic UI supplement ($300/week) six months before the wage hike. Control states kept it until federal expiration. The early expiration plausibly tightened the low-wage labor market in treated states ahead of $t^*$.

Inference Problem

Estimand: Causal ATT on the treated states: $E[Y^{(D=1)} - Y^{(D=0)} \mid D^* = 1, t > t^*]$.

Why ATT, not ATE? Most defensible target with this design. We only have unbiased $Y^{(D=1)}$ for treated states; parallel trends identifies the counterfactual specifically for them.

Identification & Estimator

Identification: DID with parallel trends. Naive DID is biased: the early UI-supplement expiration created a low-wage employment trend in treated states ahead of $t^*$. Restore identification by conditioning on the time-varying control $X_{it} =$ per-state UI-supplement status by month.

$Y_{it} \sim \beta_0 + \beta_t t + \beta_D D^* + \beta_{DT}(t \cdot D^*) + \gamma X_{it}$

The treatment effect is $\beta_{DT}$.

Drill → Cell Mapping

No selection biasSelection bias
Statistical—L1.1, L1.2 (s-bias); L1.3 (fails)
CausalL2.1 (IV), L2.2 (FDC), L2.3 (RDD), L2.4 (DID)(s-backdoor)

Exam discipline: name the cell first, the method follows.

Master Method Comparison

What Does Each Method Model?

Method Models outcome $E[Y|X,Z]$? Models treatment $P(X|Z)$? Key formula
Stratification No No $\sum_z E[Y|X,Z{=}z] \cdot P(Z{=}z)$
Matching (Exact/NN/PSM) No PSM: Yes Pair treated/control by $Z$ or $e(Z)$, compare outcomes
IPW No Yes $\frac{1}{N}\sum_{i: x_i=x} \frac{y_i}{P(X{=}x_i|Z_i)}$
OLS Yes (linear) No $Y = \beta_0 + \beta_X X + \beta_Z Z + \epsilon$; read off $\beta_X$
BDA predict-and-average Yes (any model) No $\frac{1}{N}\sum_i [\hat{f}(1, Z_i) - \hat{f}(0, Z_i)]$
Double ML Yes (nuisance) Yes (nuisance) $\hat{\delta} = \sum \tilde{X}_i \tilde{Y}_i / \sum \tilde{X}_i^2$
Doubly-robust (IPW + Reg) Yes Yes IPW-weighted MSE: $\sum_i \frac{(y_i - \hat{f}(x_i))^2}{P(X{=}x_i|Z_i)}$

Capabilities & Limitations

Stratification Matching IPW OLS BDA Double ML Doubly-robust
Continuous $Z$ No Via PS Yes Yes Yes Yes Yes
High-dim $Z$ No Via PS Yes Risky Risky Yes Yes
Heterogeneous effects Yes Implicitly Yes No (basic) Yes R-learner Yes
Extrapolates? No No No Yes (danger) Yes (danger) Partially Yes (danger)
Valid CIs with ML? N/A Bootstrap Bootstrap No No Yes Depends
Any $f(Y)$? Means only Means only Yes Means only Means only Means only Means only
Failure mode Empty cells No match Weights explode Extrapolation Extrapolation Nuisance failure Both wrong

When to Choose Each

Situation Best choice Why
$Z$ is binary/few levels Stratification Simple, transparent, no modeling assumptions
Low-dim $Z$, want non-parametric PSM + calipers No outcome model, drops bad matches honestly
Need $E[f(Y)|do(X)]$ for arbitrary $f$ IPW Only method that estimates beyond conditional means
Trust linearity, constant effect OLS Simplest, $\beta_X$ = ATE, standard CIs
Any model, heterogeneous effects BDA predict-and-average Works with RF, NN, etc. — handles any $\delta(Z)$
Complex $g(Z)$, need valid CIs Double ML ML flexibility + debiased inference via cross-fitting
Unsure which model is correct Doubly-robust Insurance: consistent if EITHER model right
Want CATE $\tau(Z)$ with ML R-Learner Extends Double ML residualization to heterogeneous effects

Estimating $\delta(Z)$ (CATE)

Method Approach Assumption
Stratified matching $\hat{\delta}(z) = E[Y|X{=}1,Z{=}z] - E[Y|X{=}0,Z{=}z]$ Discrete $Z$, enough data per stratum
OLS + interaction $\delta(z) = \beta_X + \beta_{XZ} z$ Linear heterogeneity
BDA point-wise $\hat{\delta}(z) = \hat{f}(1, z) - \hat{f}(0, z)$ Correct outcome model
R-Learner Fit $\hat{\tau}(Z)$ to minimize $\sum(\tilde{Y}_i - \tau(Z_i)\tilde{X}_i)^2$ Good nuisance models + cross-fitting

The Three Assumptions Everything Rests On

1. Graph Structure — The DAG is correct: no missing arrows, no wrong directions. There is no statistical test for this. Domain knowledge is essential. PC algorithm can help discover structure but has limitations (discretization artifacts, can't distinguish some orientations).
2. No Unmeasured Confounders — All back-door paths are blockable with observed $Z$. Untestable from data alone. Sensitivity analysis stress-tests this: "how strong would a hidden confounder need to be?"
3. Support / Overlap — $0 < P(X{=}1|Z{=}z) < 1$ for all $z$ where $P(Z{=}z) > 0$. Both treatment groups must exist at every confounder level. Checkable via propensity score distributions. Without overlap: matching drops units, IPW weights explode, regression extrapolates silently.

Practice Problems

Q1: Given the graph $Z_1 \to X$, $Z_1 \to Y$, $Z_2 \to X$, $Z_2 \to Y$, $X \to Y$, what is a valid back-door set for the effect of $X$ on $Y$?

Back-door paths from $X$ to $Y$: (1) $X \leftarrow Z_1 \to Y$ and (2) $X \leftarrow Z_2 \to Y$.

Valid back-door sets: $\{Z_1, Z_2\}$, $\{Z_1\}$ alone blocks path 1 but not path 2, so we need both. Actually, $\{Z_1\}$ only blocks path 1. $\{Z_2\}$ only blocks path 2. We need $\{Z_1, Z_2\}$.

Check: Neither $Z_1$ nor $Z_2$ is a descendant of $X$. Both back-door paths are blocked.

Q2: In the graph $X \to M \to Y$ with $U \to X$ and $U \to Y$ (U unobserved), is $\{M\}$ a valid back-door set?

No! $M$ is a descendant of $X$, violating condition 1 of the back-door criterion. Additionally, conditioning on $M$ would block the causal path $X \to M \to Y$, which we don't want.

The valid back-door set would be $\{U\}$, but $U$ is unobserved. Without observing $U$, we cannot use back-door adjustment here.

Q3: You observe the following data with binary $X$ and confounder $Z \in \{0,1\}$:
$E[Y|X{=}1, Z{=}0] = 5$, $E[Y|X{=}0, Z{=}0] = 3$, $P(Z{=}0) = 0.6$
$E[Y|X{=}1, Z{=}1] = 8$, $E[Y|X{=}0, Z{=}1] = 7$, $P(Z{=}1) = 0.4$
What is the ATE using the back-door adjustment formula?

$E[Y|do(X{=}1)] = E[Y|X{=}1,Z{=}0] \cdot P(Z{=}0) + E[Y|X{=}1,Z{=}1] \cdot P(Z{=}1)$

$= 5 \times 0.6 + 8 \times 0.4 = 3.0 + 3.2 = 6.2$

$E[Y|do(X{=}0)] = 3 \times 0.6 + 7 \times 0.4 = 1.8 + 2.8 = 4.6$

ATE $= 6.2 - 4.6 = 1.6$

Stratum-level effects: $\delta_{Z=0} = 5 - 3 = 2$, $\delta_{Z=1} = 8 - 7 = 1$.

$\text{ATE} = 2 \times 0.6 + 1 \times 0.4 = 1.6$. Same answer.

Q4: Why does conditioning on a collider introduce bias? Give an intuitive explanation.

A collider $X \to Z \leftarrow Y$ blocks the path between $X$ and $Y$ by default. But conditioning on $Z$ (or selecting on $Z$) opens the path.

Intuition: Knowing the collider's value constrains the relationship between its parents. If $Z = X + Y$ and you know $Z = 10$, then knowing $X = 8$ tells you $Y = 2$. The parents become dependent given the collider's value, even though they were independent unconditionally.

Real-world: Among admitted students (collider = admission), math and writing skills appear negatively correlated, even though they're independent in the general population.

Q5: You have a propensity score $P(X{=}1|Z{=}z_i) = 0.8$ for unit $i$ with $X_i = 1$. What is the IPW weight for this unit? What about a unit with $P(X{=}1|Z) = 0.8$ and $X = 0$?

For unit with $X=1$: $w_i = 1/P(X{=}1|Z) = 1/0.8 = 1.25$

For unit with $X=0$: $w_i = 1/P(X{=}0|Z) = 1/(1-0.8) = 1/0.2 = 5.0$

The control unit at $Z$ where treatment is likely gets a much higher weight (5x vs 1.25x). This makes sense: seeing a control unit where almost everyone is treated is very informative about the counterfactual.

Q6: Explain why $\beta_X$ in OLS ($Y = \beta_X X + \beta_Z Z + \epsilon$) equals the ATE when the treatment effect is constant, but not when it varies with $Z$.

Constant effect: The BDA formula gives $E[Y|do(X{=}x)] = \sum_z E[Y|X{=}x, Z{=}z]P(Z{=}z)$. Plugging in the linear model: $E[Y|X{=}1,Z{=}z] - E[Y|X{=}0,Z{=}z] = \beta_X$ for all $z$. So $\delta = \sum_z \beta_X P(Z{=}z) = \beta_X$.

Heterogeneous effect: If the true model is $Y = \delta(Z) \cdot X + g(Z) + \epsilon$, fitting a constant-effect model forces $\delta(Z) \to \beta_X$. OLS minimizes MSE and produces a variance-weighted average of $\delta(Z)$, which doesn't equal $E[\delta(Z)]$ in general. The weighting depends on the propensity score distribution, biasing toward values of $Z$ where treatment assignment is most variable.

Fix: Add interaction term ($\beta_{XZ} XZ$) or use BDA predict-and-average.

Q7: What are the three steps of Double ML, and why is cross-fitting necessary?
  1. Cross-fit: Split data into $K$ folds
  2. Residualize: On each held-out fold, compute $\tilde{Y}_i = Y_i - \hat{E}[Y|Z_i]$ and $\tilde{X}_i = X_i - \hat{E}[X|Z_i]$ using models trained on other folds
  3. Estimate: Regress $\tilde{Y}$ on $\tilde{X}$: $\hat{\delta} = \sum \tilde{X}_i \tilde{Y}_i / \sum \tilde{X}_i^2$

Why cross-fitting? ML models use regularization (early stopping, depth limits, L2 penalties), which introduces bias in predictions. If nuisance models predict on their own training data, this bias leaks into the causal estimate. Cross-fitting ensures nuisance models never predict on data they were trained on, preventing overfitting bias from contaminating $\hat{\delta}$.

Q8: In sensitivity analysis, the breakdown point $\alpha^*$ for an estimated ATE of 1.0 is $\alpha^* \approx 1.0$. What does this mean practically?

The breakdown point is the minimum violation of unconfoundedness needed to explain away the entire treatment effect (reduce ATE to zero).

$\alpha^* \approx 1.0$ means: an unmeasured confounder would need to create a 1.0 unit difference in expected potential outcomes between treated and control groups (within each stratum of observed confounders) to fully explain the observed effect.

Whether this is a lot or a little depends on domain knowledge. If $Y$ is measured in standard deviations, a 1.0 unit violation is quite large (the unmeasured confounder would need to shift expected outcomes by a full standard deviation). If $Y$ is in dollars, 1.0 might be trivially small.

Key insight: Sensitivity analysis doesn't tell you whether unconfoundedness holds. It tells you how robust your conclusion is to violations.

Q9: You want to estimate the causal effect of a study app on student pass rates. Your confounders are: household income (continuous), parent education (5 categories), GPA (continuous), commute time (continuous), hobbies (continuous). Which estimation method would you recommend and why?

Not exact matching: $5^5 = 3125$ strata (with 5 bins each) — too many for most sample sizes. The curse of dimensionality.

Options:

  • Propensity score matching: Reduces 5D matching to 1D. Fit logistic regression for $P(\text{app}|Z)$, match on the score. Check balance post-matching.
  • OLS with all confounders: If linear model is reasonable. $\beta_{\text{app}}$ = ATE if constant effect.
  • IPW: Weight by inverse propensity. Good if you trust the propensity model but not an outcome model.
  • Double ML: Best if relationships are nonlinear. Uses flexible ML for nuisance models while giving valid CIs for the causal effect.

Always: Check support (propensity score overlap between users and non-users), and run sensitivity analysis afterwards.