An extended study session preparing for the Causal Inference mid-term exam
Initiate the exam helper by guiding me through the key concepts step-by-step. Start with the foundational concepts of the subject matter at a high level. What's Causal Inference?
At its core, causal inference answers one question:
"What happens to Y if I change X?"
This is fundamentally different from "What is Y when I observe X?" The first is about intervention. The second is about correlation.
We can never observe what would have happened to the same person under a different treatment. For every unit $i$ and treatment $d$, there exists a potential outcome $Y_i^{(D=d)}$. The causal effect is $\delta_i = Y_i^{(D=1)} - Y_i^{(D=0)}$. But you only ever see one of these.
The naive estimator $E[Y|D=1] - E[Y|D=0]$ compares those who happened to get treated vs. not. But these groups may differ systematically. The course's discount example: discounts offered during low-sales weeks made the naive estimate negative (-0.5), even though the true effect was positive (+0.5).
The entire course is about path 2. Can you explain in your own words why the naive estimator can give you the wrong sign?
In the naive estimator, the difference between the treatment and control groups can be systematic. The assignment mechanism has to be random in order for this naive estimator to equal the true estimator.
Correct When assignment is random, $D$ is independent of potential outcomes — the groups are comparable. The key equation:
The naive estimator equals the true effect plus the difference in baseline outcomes between groups. Randomization makes that second term zero. Without it, you're measuring the causal effect plus how different the groups already were.
There are exactly three building blocks that make up every causal graph:
| Structure | Example | Key Property |
|---|---|---|
| Chain | X → Z → Y | Conditioning on Z blocks the path |
| Fork | X ← Z → Y | Conditioning on Z blocks the path |
| Collider | X → Z ← Y | Conditioning on Z opens the path |
The first two are intuitive — hold the middle variable fixed, the endpoints become independent. The collider is the trap: blocked by default, but conditioning on it creates spurious association.
Explain in more detail: "conditioning on." Is it equivalent to "adjusting" and "blocking the path"? Is the goal to block all paths from X to Y except the front door?
"Conditioning on," "adjusting for," and "controlling for" all mean the same thing — holding a variable fixed when analyzing data. Three equivalent ways to think about it:
Important: Conditioning is a tool — it can block or open depending on structure. It doesn't always block. The goal is to block all non-causal (back-door) paths while keeping causal paths open. The remaining association is the causal effect.
By only looking at admitted students, we are conditioning on Z=1. Knowing a student is admitted gives us information about the relationship between X and Y: if the student has a low X, then Y is likely to be high.
Exactly In the full population, math and writing are independent. But restricting to admitted students creates an informational constraint: they got in somehow. Low math implies high writing, high math implies low writing. A spurious negative correlation appears.
The notebooks confirmed numerically: full population correlation 0.012 (basically zero), admitted students only -0.290 (strongly negative).
d-Separation: A path between X and Y is blocked by set Z if somewhere along it there is:
Back-Door Criterion for choosing adjustment set Z:
Let's test your understanding. Consider this graph:
U causes both X and Y. X causes Y through M. What's a valid adjustment set?
Adjustment set: none. X → M → Y is a chain structure — conditioning on M will block the causal path from X to Y, closing the front-door path.
Half right Your chain reasoning is correct — conditioning on M would block the causal path. But "none" is wrong.
The back-door path $X \leftarrow U \rightarrow Y$ is a fork that's open by default. U is confounding. The correct answer: adjust for {U}.
| Adjustment set | Back-door X←U→Y | Causal X→M→Y | Verdict |
|---|---|---|---|
| None | Open (confounded!) | Open | Biased |
| {U} | Blocked | Open | Correct |
| {M} | Open | Blocked | Kills causal signal |
| {U, M} | Blocked | Blocked | Kills causal signal |
Key: "Do nothing" means confounding flows freely. You need to actively block back-door paths. And the goal is not d-separation of X and Y — that would mean no association at all. You want to block non-causal paths while keeping causal paths open.
Proceed with the next steps. Give me a numerical example.
E-commerce example: Does offering a discount (X) increase purchases (Y)? Loyalty tier (Z) confounds — high-loyalty customers both receive more discounts and buy more.
| Z (loyalty) | X (discount) | Count | Purchase rate |
|---|---|---|---|
| Low | No discount | 400 | 0.20 |
| Low | Discount | 100 | 0.40 |
| High | No discount | 100 | 0.60 |
| High | Discount | 400 | 0.80 |
Naive estimate: 0.44 — inflated because the discount group is dominated by high-loyalty customers.
BDA (correct):
Within each stratum, the discount effect is 0.20. The naive estimate was inflated by confounding.
Explain the support checking process in detail — propensity score distribution and distribution overlap.
The support assumption (positivity):
For every confounder value where people exist, there must be a nonzero probability of each treatment level. You check this by plotting propensity score distributions for both groups. The overlap region is where causal inference is possible. The tails where only one group exists are danger zones.
| Method | Without support |
|---|---|
| Stratification | Can't compute — cell is empty |
| Matching | Can't find a match |
| IPW | Weight = 1/0 = ∞ |
| Regression | Extrapolates silently |
Let's move on to continuous or high-dimensional Z problems.
The BDA formula $\sum_z E[Y|X{=}x, Z{=}z] \cdot P(Z{=}z)$ breaks in two ways:
Four solutions:
| Method | Strategy |
|---|---|
| Matching | Find twins via distance metric or propensity score (1-D) |
| IPW | Collapse Z into propensity score, reweight |
| Regression/BDA | Let the model learn the functional form |
| Double ML | ML models partial out Z from both sides |
Exact Matching: Match on identical Z. Problem: curse of dimensionality — most units have no exact match with continuous/many covariates.
Nearest-Neighbor: Use Mahalanobis distance. Calipers set a maximum distance — drop the unit rather than make a bad match.
Propensity Score Matching (Rosenbaum & Rubin, 1983): Matching on the single scalar $e(Z) = P(X{=}1|Z)$ is sufficient to remove confounding. 50 covariates → 1 number.
Can you explain: why is matching on the propensity score sufficient, rather than needing to match on all of Z?
What does the Control (X=0) group's propensity score P(X=1|Z) mean? Density is the number of observations for each propensity score? Most similar people are also treated — does it mean that among the near-one propensity score group, most are in group X=1? Is equivalent density a pre-requisite for propensity score matching? What about score 0.3, where the density for X=0 is much larger than X=1?
P(X=1|Z) for controls: Every unit gets a propensity score — it's the predicted probability of being treated given their covariates. Control units still have a score; it means that despite having, say, a 70% chance, they happened not to be treated.
Density: Relative frequency (smoothed histogram), not raw count.
Near-one = mostly treated: Yes. At $e(Z) = 0.95$, 95% of similar units are treated. Very few controls exist — matching becomes hard at the extremes.
Equal density not required. At score 0.3, 200 controls and 20 treated is fine. PSM works by pairing individuals, not averaging groups. The only problem is when one group has zero units — that's a support violation.
Why is matching on the propensity score sufficient, rather than matching on all of Z? And why can't we just match to a nearest unit when one group has zero at a given propensity score?
The Rosenbaum-Rubin theorem:
Intuition: The propensity score is a sufficient summary of Z. Among all units with $e(Z) = 0.4$ — regardless of their specific Z values — treatment is effectively a coin flip with 40% probability. Some are young, rich, urban. Others are old, poor, rural. But within this group, who got treated is as good as random.
As for matching when one group is empty: you can, but if the nearest control has $e(Z) = 0.55$ and your treated unit has $e(Z) = 0.95$, these are systematically different people. The "as-good-as-random" argument breaks down. Calipers prevent bad matches by dropping unmatched units — but this changes the estimand to ATT for the matchable subpopulation only.
Matching finds twins. IPW takes a completely different approach: reweight the entire sample so it looks like an experiment. Each unit gets weight $w_i = 1/P(X = x_i \mid Z_i)$.
A high-loyalty customer who didn't get a discount is unusual and highly informative — they look like the treated group but happened not to be treated. IPW upweights these rare units:
| Unit type | P(X|Z) | Weight | Why |
|---|---|---|---|
| High-loyalty, treated | 0.90 | 1.1 | Expected — low info |
| High-loyalty, untreated | 0.10 | 10.0 | Surprising — high info |
| Low-loyalty, untreated | 0.80 | 1.25 | Expected — low info |
| Low-loyalty, treated | 0.20 | 5.0 | Surprising — high info |
How does IPW relate to the back-door adjustment formula?
BDA uses P(Z=z) as a weight and calculates the weighted sum of Y given Z=z (back-door conditioned); a weighted sum across different z values. IPW uses a weight P(X|Z) in the denominator, boosting the lower-chance event outcomes; treating such events as having higher information value; sum across different x value groups while z is given.
Good intuition on both. One refinement: IPW doesn't condition on Z in the summation. It sums over all units in the treatment group, using the weight to undo confounding without ever stratifying. That's its power.
They're algebraically equivalent, just rearranged:
Same formula, different failure modes: BDA breaks when Z is high-dimensional (empty strata). IPW breaks when $e(Z)$ is extreme (weights explode).
IPW sums over all units in the treatment group only? What does $i: x_i = x$ mean in the summation? How does IPW account for the surprisingly untreated controls? How would you write out P(X=0|Z) for control + high loyalty?
The ATE formula has two separate sums:
$i: x_i = 1$ means "loop over treated units only" — a filter. The "surprisingly untreated" units appear in the second sum. $N$ = total sample size.
The general rule: the denominator is always the probability of what actually happened.
Matching and IPW model treatment assignment. Regression flips this: model the outcome directly. For $Y = \beta_0 + \beta_X X + \beta_Z Z + \epsilon$, plug into BDA: at any fixed $Z = z$, the difference is $\beta_X$ — the Z terms cancel identically. OLS is doing BDA implicitly.
How do the Z terms cancel? How can we ensure each corresponding term is the same between X=1 and X=0? What does it mean that the treatment effect delta varies with Z?
At fixed $Z = z$:
Subtract: $\beta_0$ cancels. $\beta_Z z$ cancels. What's left is $\beta_X$ — the same for every z. The model assumes Z's contribution to Y is identical regardless of treatment. Only $\beta_X X$ changes when X flips.
"δ varies with Z": The treatment works differently for different people. Discount boosts purchases by 0.30 for low-loyalty but only 0.10 for high-loyalty. A constant-effect OLS gives a weighted average — but weighted by Z's distribution within the treated group, not the population. So $\beta_X \neq$ ATE.
BDA predict-and-average fixes this: fit any model, predict under $X{=}1$ and $X{=}0$ for everyone, average the differences. Handles heterogeneity automatically.
Model-based extrapolation is dangerous because regression's prediction solely depends on the existing data, even though the data points of the group that is of interest don't exist, which may fail to capture unique characteristics of the group if it is fundamentally different. This applies the same from the support assumption, where stratification won't work without any data points in a segment to compare the treatment effect.
Strong answer. Two refinements:
The specific danger is that regression imposes a functional form and extends it into regions with no data. "Y increases linearly with Z in [0.2, 0.8], so it must continue at 0.95" — but maybe the relationship curves. Matching/IPW would refuse. Regression gives you a confidence interval as if the answer were reliable.
"Fundamentally different" has a name: lack of common support.
| Method | No-overlap region | Failure mode |
|---|---|---|
| Exact matching | Drops the unit | Honest |
| PSM with calipers | Drops the unit | Honest |
| IPW | Weight → ∞ | Breaks loudly |
| Regression | Extrapolates | Breaks silently |
BDA requires correctly specifying $E[Y|X,Z]$. With complex Z you'd use ML, but ML regularization introduces bias that contaminates $\hat{\delta}$ and invalidates confidence intervals.
Cross-fitting prevents ML bias from leaking: train on all data except fold k, predict residuals on held-out fold k.
The residual regression outcome delta is equivalent to the $\beta_1 = \text{cov}(x,y)/\text{var}(x)$ slope in a simple linear regression?
Does Double ML assume a linear model, given that delta is constant? How can it fit nonlinear models? Is the $g(Z)$ term only subject to ML while delta is done by linear regression?
What is CATE, how does it relate to R-Learner, and how is it implemented with varying treatment effects across Z?
Yes, exactly. Since residuals are mean-zero, $\sum \tilde{X}_i \tilde{Y}_i / \sum \tilde{X}_i^2 = \text{cov}(\tilde{X}, \tilde{Y}) / \text{var}(\tilde{X})$ — the OLS slope.
Partially linear, not fully linear. $Y = \delta X + g(Z) + \epsilon$ — linear in X (one coefficient), but $g(Z)$ can be anything. ML handles complex $g(Z)$ (only needs good prediction). OLS handles $\delta$ (valid inference). Division of labor.
CATE = Conditional Average Treatment Effect: $\tau(z) = E[Y^{(1)} - Y^{(0)} | Z = z]$. Double ML gives one number; CATE gives a function.
R-Learner: Compute residuals as in Double ML, then fit $\hat{\tau}(Z)$ minimizing $\sum_i (\tilde{Y}_i - \tau(Z_i) \cdot \tilde{X}_i)^2$. Double ML asks "what's the single slope?" R-Learner asks "how does the slope change with Z?"
Explain in detail why "ML models are biased by design" and why this matters for causal inference but not for prediction.
Every model's error: Bias² + Variance + Noise. ML deliberately introduces bias to reduce variance — LASSO shrinks coefficients, tree depth limits prevent memorization, dropout forces redundancy.
Fine for prediction: If adding bias reduces variance by more, total error decreases. A house price model that's systematically $10K low but stable beats one that's unbiased but swings $200K.
Fatal for causal inference: You need an unbiased $\hat{\delta}$ and valid CIs. LASSO shrinks $\hat{\beta}_X$ toward zero. Your CI might be [0.08, 0.16] when truth is 0.25 — narrow, precise, doesn't contain the truth. Regularization bias doesn't shrink with more data. The CI is permanently centered wrong.
Double ML's fix: ML for nuisance models only (bias OK). OLS for $\delta$ (one parameter, no regularization). Cross-fitting ensures the bias doesn't leak.
Every method assumes the DAG is correct. Sensitivity analysis asks: what if we're wrong?
"How strong would an unmeasured confounder have to be to explain away our estimated treatment effect?"
If $U \to X$ and $U \to Y$ exist unblocked, the bias is $\beta_U \cdot (E[U|X{=}1] - E[U|X{=}0])$. The breakdown point $\alpha^*$ is the minimum confounder strength that zeros out the effect.
| Breakdown | Interpretation | Verdict |
|---|---|---|
| Very large | Need a confounder stronger than anything observed | Robust |
| Small | A mild confounder kills the effect | Fragile |
| Negative | Adding confounding strengthens the effect | Very robust |
Analogy: "My bridge holds 10 tons. Sensitivity analysis says it survives up to 25 tons. I don't know if a 25-ton truck will ever cross — but I know my safety margin."
Give me a full method comparison — "when to use which" — to tie everything together for the exam.
This was incredibly helpful. Thank you.
Good luck on the exam. The chain you built — potential outcomes → graphs → matching → IPW → regression → Double ML → sensitivity — is exactly how the pieces fit together. You're well prepared.