Studying Causal Inference with Claude

An extended study session preparing for the Causal Inference mid-term exam

March 10-11, 2026 27 topics Claude Opus 4.6

Interactive Study Module →

Initiate the exam helper by guiding me through the key concepts step-by-step. Start with the foundational concepts of the subject matter at a high level. What's Causal Inference?

Mapping out causal inference foundations, building from first principles

At its core, causal inference answers one question:

"What happens to Y if I change X?"

This is fundamentally different from "What is Y when I observe X?" The first is about intervention. The second is about correlation.

The Fundamental Problem

We can never observe what would have happened to the same person under a different treatment. For every unit $i$ and treatment $d$, there exists a potential outcome $Y_i^{(D=d)}$. The causal effect is $\delta_i = Y_i^{(D=1)} - Y_i^{(D=0)}$. But you only ever see one of these.

Why Not Just Compare Groups?

The naive estimator $E[Y|D=1] - E[Y|D=0]$ compares those who happened to get treated vs. not. But these groups may differ systematically. The course's discount example: discounts offered during low-sales weeks made the naive estimate negative (-0.5), even though the true effect was positive (+0.5).

Two Paths Forward

  1. Run an experiment — randomize D so it's independent of everything else
  2. Use observational data carefully — encode causal knowledge as a graph, find the right variables to adjust for

The entire course is about path 2. Can you explain in your own words why the naive estimator can give you the wrong sign?

In the naive estimator, the difference between the treatment and control groups can be systematic. The assignment mechanism has to be random in order for this naive estimator to equal the true estimator.

Evaluating response on naive estimator bias

Correct When assignment is random, $D$ is independent of potential outcomes — the groups are comparable. The key equation:

$\delta_{\text{naive}} = \delta + \underbrace{E[Y^{(D=0)}|D=1] - E[Y^{(D=0)}|D=0]}_{\text{selection bias}}$

The naive estimator equals the true effect plus the difference in baseline outcomes between groups. Randomization makes that second term zero. Without it, you're measuring the causal effect plus how different the groups already were.

Causal Graphs
Building up the three graph structures systematically

There are exactly three building blocks that make up every causal graph:

StructureExampleKey Property
ChainX → Z → YConditioning on Z blocks the path
ForkX ← Z → YConditioning on Z blocks the path
ColliderX → Z ← YConditioning on Z opens the path

The first two are intuitive — hold the middle variable fixed, the endpoints become independent. The collider is the trap: blocked by default, but conditioning on it creates spurious association.

Explain in more detail: "conditioning on." Is it equivalent to "adjusting" and "blocking the path"? Is the goal to block all paths from X to Y except the front door?

Clarifying conditioning, adjusting, and the back-door criterion

"Conditioning on," "adjusting for," and "controlling for" all mean the same thing — holding a variable fixed when analyzing data. Three equivalent ways to think about it:

  1. Subsetting: Split data by Z, analyze each subset separately
  2. Including in regression: $\beta_X$ in $Y = \beta_X X + \beta_Z Z$ gives X's effect after accounting for Z
  3. Sample selection: If your dataset only contains admitted students, you are implicitly conditioning on admission — even without including it in a regression

Important: Conditioning is a tool — it can block or open depending on structure. It doesn't always block. The goal is to block all non-causal (back-door) paths while keeping causal paths open. The remaining association is the causal effect.

By only looking at admitted students, we are conditioning on Z=1. Knowing a student is admitted gives us information about the relationship between X and Y: if the student has a low X, then Y is likely to be high.

Exactly In the full population, math and writing are independent. But restricting to admitted students creates an informational constraint: they got in somehow. Low math implies high writing, high math implies low writing. A spurious negative correlation appears.

The notebooks confirmed numerically: full population correlation 0.012 (basically zero), admitted students only -0.290 (strongly negative).

d-Separation & Back-Door Criterion
Covering d-separation rules, then testing with a graph exercise

d-Separation: A path between X and Y is blocked by set Z if somewhere along it there is:

  1. A chain or fork node that is in Z (you blocked it), OR
  2. A collider node not in Z with no descendant in Z (naturally blocked)

Back-Door Criterion for choosing adjustment set Z:

  1. No descendant of X is in Z
  2. Z blocks all back-door paths (paths entering X from behind)

Let's test your understanding. Consider this graph:

U ──→ X ──→ M ──→ Y │ ↑ └─────────────────┘

U causes both X and Y. X causes Y through M. What's a valid adjustment set?

Adjustment set: none. X → M → Y is a chain structure — conditioning on M will block the causal path from X to Y, closing the front-door path.

Evaluating graph exercise response, identifying the error

Half right Your chain reasoning is correct — conditioning on M would block the causal path. But "none" is wrong.

The back-door path $X \leftarrow U \rightarrow Y$ is a fork that's open by default. U is confounding. The correct answer: adjust for {U}.

Adjustment setBack-door X←U→YCausal X→M→YVerdict
NoneOpen (confounded!)OpenBiased
{U}BlockedOpenCorrect
{M}OpenBlockedKills causal signal
{U, M}BlockedBlockedKills causal signal

Key: "Do nothing" means confounding flows freely. You need to actively block back-door paths. And the goal is not d-separation of X and Y — that would mean no association at all. You want to block non-causal paths while keeping causal paths open.

Back-Door Adjustment

Proceed with the next steps. Give me a numerical example.

Walking through the BDA with the e-commerce discount example

E-commerce example: Does offering a discount (X) increase purchases (Y)? Loyalty tier (Z) confounds — high-loyalty customers both receive more discounts and buy more.

Z (loyalty)X (discount)CountPurchase rate
LowNo discount4000.20
LowDiscount1000.40
HighNo discount1000.60
HighDiscount4000.80

Naive estimate: 0.44 — inflated because the discount group is dominated by high-loyalty customers.

BDA (correct):

$E[Y|do(X{=}1)] = 0.40 \times 0.5 + 0.80 \times 0.5 = 0.60$
$E[Y|do(X{=}0)] = 0.20 \times 0.5 + 0.60 \times 0.5 = 0.40$
$\delta_{\text{BDA}} = 0.60 - 0.40 = \mathbf{0.20}$

Within each stratum, the discount effect is 0.20. The naive estimate was inflated by confounding.

The Support Assumption

Explain the support checking process in detail — propensity score distribution and distribution overlap.

Explaining the positivity assumption and how to diagnose violations

The support assumption (positivity):

$0 < P(X=x \mid Z=z) < 1$ for all $z$ where $P(Z=z) > 0$

For every confounder value where people exist, there must be a nonzero probability of each treatment level. You check this by plotting propensity score distributions for both groups. The overlap region is where causal inference is possible. The tails where only one group exists are danger zones.

MethodWithout support
StratificationCan't compute — cell is empty
MatchingCan't find a match
IPWWeight = 1/0 = ∞
RegressionExtrapolates silently
Continuous & High-Dimensional Z

Let's move on to continuous or high-dimensional Z problems.

Explaining how BDA breaks with continuous/high-dim Z, introducing the four solutions

The BDA formula $\sum_z E[Y|X{=}x, Z{=}z] \cdot P(Z{=}z)$ breaks in two ways:

  1. Continuous Z → infinite strata. You can't compute $E[Y|X{=}1, Z{=}42{,}387.51]$. Binning creates residual confounding.
  2. High-dim Z → curse of dimensionality. 10 binary confounders = $2^{10} = 1{,}024$ strata. Most cells are empty.

Four solutions:

MethodStrategy
MatchingFind twins via distance metric or propensity score (1-D)
IPWCollapse Z into propensity score, reweight
Regression/BDALet the model learn the functional form
Double MLML models partial out Z from both sides
Matching Methods
Covering exact matching, NN matching, and propensity score matching

Exact Matching: Match on identical Z. Problem: curse of dimensionality — most units have no exact match with continuous/many covariates.

Nearest-Neighbor: Use Mahalanobis distance. Calipers set a maximum distance — drop the unit rather than make a bad match.

Propensity Score Matching (Rosenbaum & Rubin, 1983): Matching on the single scalar $e(Z) = P(X{=}1|Z)$ is sufficient to remove confounding. 50 covariates → 1 number.

Can you explain: why is matching on the propensity score sufficient, rather than needing to match on all of Z?

What does the Control (X=0) group's propensity score P(X=1|Z) mean? Density is the number of observations for each propensity score? Most similar people are also treated — does it mean that among the near-one propensity score group, most are in group X=1? Is equivalent density a pre-requisite for propensity score matching? What about score 0.3, where the density for X=0 is much larger than X=1?

Addressing each propensity score question in turn

P(X=1|Z) for controls: Every unit gets a propensity score — it's the predicted probability of being treated given their covariates. Control units still have a score; it means that despite having, say, a 70% chance, they happened not to be treated.

Density: Relative frequency (smoothed histogram), not raw count.

Near-one = mostly treated: Yes. At $e(Z) = 0.95$, 95% of similar units are treated. Very few controls exist — matching becomes hard at the extremes.

Equal density not required. At score 0.3, 200 controls and 20 treated is fine. PSM works by pairing individuals, not averaging groups. The only problem is when one group has zero units — that's a support violation.

Why is matching on the propensity score sufficient, rather than matching on all of Z? And why can't we just match to a nearest unit when one group has zero at a given propensity score?

Explaining Rosenbaum-Rubin theorem and the caliper tradeoff

The Rosenbaum-Rubin theorem:

If $X \perp\!\!\!\perp (Y^{(0)}, Y^{(1)}) \mid Z$, then $X \perp\!\!\!\perp (Y^{(0)}, Y^{(1)}) \mid e(Z)$

Intuition: The propensity score is a sufficient summary of Z. Among all units with $e(Z) = 0.4$ — regardless of their specific Z values — treatment is effectively a coin flip with 40% probability. Some are young, rich, urban. Others are old, poor, rural. But within this group, who got treated is as good as random.

As for matching when one group is empty: you can, but if the nearest control has $e(Z) = 0.55$ and your treated unit has $e(Z) = 0.95$, these are systematically different people. The "as-good-as-random" argument breaks down. Calipers prevent bad matches by dropping unmatched units — but this changes the estimand to ATT for the matchable subpopulation only.

Inverse Propensity Weighting
Introducing IPW with the surprising-units intuition

Matching finds twins. IPW takes a completely different approach: reweight the entire sample so it looks like an experiment. Each unit gets weight $w_i = 1/P(X = x_i \mid Z_i)$.

The "Surprising Units" Intuition

A high-loyalty customer who didn't get a discount is unusual and highly informative — they look like the treated group but happened not to be treated. IPW upweights these rare units:

Unit typeP(X|Z)WeightWhy
High-loyalty, treated0.901.1Expected — low info
High-loyalty, untreated0.1010.0Surprising — high info
Low-loyalty, untreated0.801.25Expected — low info
Low-loyalty, treated0.205.0Surprising — high info

How does IPW relate to the back-door adjustment formula?

BDA uses P(Z=z) as a weight and calculates the weighted sum of Y given Z=z (back-door conditioned); a weighted sum across different z values. IPW uses a weight P(X|Z) in the denominator, boosting the lower-chance event outcomes; treating such events as having higher information value; sum across different x value groups while z is given.

Evaluating BDA vs IPW comparison, noting the key refinement

Good intuition on both. One refinement: IPW doesn't condition on Z in the summation. It sums over all units in the treatment group, using the weight to undo confounding without ever stratifying. That's its power.

They're algebraically equivalent, just rearranged:

  • BDA: $\sum_z E[Y|X{=}x, Z{=}z] \cdot P(Z{=}z)$ — stratify, weight by population share
  • IPW: $\frac{1}{N}\sum_{i: x_i=x} \frac{y_i}{P(X{=}x_i|Z_i)}$ — weight units individually, no strata

Same formula, different failure modes: BDA breaks when Z is high-dimensional (empty strata). IPW breaks when $e(Z)$ is extreme (weights explode).

IPW sums over all units in the treatment group only? What does $i: x_i = x$ mean in the summation? How does IPW account for the surprisingly untreated controls? How would you write out P(X=0|Z) for control + high loyalty?

Breaking down the IPW formula notation precisely

The ATE formula has two separate sums:

$\hat{\delta}_{\text{IPW}} = \underbrace{\frac{1}{N} \sum_{i: x_i=1} \frac{y_i}{e(Z_i)}}_{\text{treated only}} \;-\; \underbrace{\frac{1}{N} \sum_{i: x_i=0} \frac{y_i}{1 - e(Z_i)}}_{\text{control only}}$

$i: x_i = 1$ means "loop over treated units only" — a filter. The "surprisingly untreated" units appear in the second sum. $N$ = total sample size.

Control + High Loyalty

  • $x_i = 0$, $Z_i = \text{High}$, $e(Z_i) = 0.80$
  • Denominator: $P(X{=}0|Z{=}\text{High}) = 1 - 0.80 = 0.20$
  • Weight: $1/0.20 = 5.0$ — 80% of similar people got treated, this one didn't. Represents 5 similar people.

The general rule: the denominator is always the probability of what actually happened.

Regression for Causal Inference
Connecting OLS regression to the BDA formula

Matching and IPW model treatment assignment. Regression flips this: model the outcome directly. For $Y = \beta_0 + \beta_X X + \beta_Z Z + \epsilon$, plug into BDA: at any fixed $Z = z$, the difference is $\beta_X$ — the Z terms cancel identically. OLS is doing BDA implicitly.

Covered: naive regression bias formula, heterogeneous effects, interaction terms

How do the Z terms cancel? How can we ensure each corresponding term is the same between X=1 and X=0? What does it mean that the treatment effect delta varies with Z?

Showing the algebra step-by-step, then explaining heterogeneous effects

At fixed $Z = z$:

$E[Y|X{=}1, Z{=}z] = \beta_0 + \beta_X + \beta_Z z$
$E[Y|X{=}0, Z{=}z] = \beta_0 + \beta_Z z$

Subtract: $\beta_0$ cancels. $\beta_Z z$ cancels. What's left is $\beta_X$ — the same for every z. The model assumes Z's contribution to Y is identical regardless of treatment. Only $\beta_X X$ changes when X flips.

"δ varies with Z": The treatment works differently for different people. Discount boosts purchases by 0.30 for low-loyalty but only 0.10 for high-loyalty. A constant-effect OLS gives a weighted average — but weighted by Z's distribution within the treated group, not the population. So $\beta_X \neq$ ATE.

BDA predict-and-average fixes this: fit any model, predict under $X{=}1$ and $X{=}0$ for everyone, average the differences. Handles heterogeneity automatically.

Model Extrapolation

Model-based extrapolation is dangerous because regression's prediction solely depends on the existing data, even though the data points of the group that is of interest don't exist, which may fail to capture unique characteristics of the group if it is fundamentally different. This applies the same from the support assumption, where stratification won't work without any data points in a segment to compare the treatment effect.

Evaluating the extrapolation answer, refining two points

Strong answer. Two refinements:

The specific danger is that regression imposes a functional form and extends it into regions with no data. "Y increases linearly with Z in [0.2, 0.8], so it must continue at 0.95" — but maybe the relationship curves. Matching/IPW would refuse. Regression gives you a confidence interval as if the answer were reliable.

"Fundamentally different" has a name: lack of common support.

MethodNo-overlap regionFailure mode
Exact matchingDrops the unitHonest
PSM with calipersDrops the unitHonest
IPWWeight → ∞Breaks loudly
RegressionExtrapolatesBreaks silently
Double ML
Explaining Double ML residualization and cross-fitting

BDA requires correctly specifying $E[Y|X,Z]$. With complex Z you'd use ML, but ML regularization introduces bias that contaminates $\hat{\delta}$ and invalidates confidence intervals.

The Solution: Residualize Both Sides

  1. Model $\hat{E}[Y|Z]$ and $\hat{E}[X|Z]$ (the propensity score!) with any ML
  2. Compute residuals: $\tilde{Y}_i = Y_i - \hat{E}[Y|Z_i]$, $\tilde{X}_i = X_i - \hat{E}[X|Z_i]$
  3. Regress: $\hat{\delta} = \sum \tilde{X}_i \tilde{Y}_i \;/\; \sum \tilde{X}_i^2$

Cross-fitting prevents ML bias from leaking: train on all data except fold k, predict residuals on held-out fold k.

The residual regression outcome delta is equivalent to the $\beta_1 = \text{cov}(x,y)/\text{var}(x)$ slope in a simple linear regression?

Does Double ML assume a linear model, given that delta is constant? How can it fit nonlinear models? Is the $g(Z)$ term only subject to ML while delta is done by linear regression?

What is CATE, how does it relate to R-Learner, and how is it implemented with varying treatment effects across Z?

Addressing three questions on Double ML internals, CATE, and R-Learner

Yes, exactly. Since residuals are mean-zero, $\sum \tilde{X}_i \tilde{Y}_i / \sum \tilde{X}_i^2 = \text{cov}(\tilde{X}, \tilde{Y}) / \text{var}(\tilde{X})$ — the OLS slope.

Partially linear, not fully linear. $Y = \delta X + g(Z) + \epsilon$ — linear in X (one coefficient), but $g(Z)$ can be anything. ML handles complex $g(Z)$ (only needs good prediction). OLS handles $\delta$ (valid inference). Division of labor.

CATE = Conditional Average Treatment Effect: $\tau(z) = E[Y^{(1)} - Y^{(0)} | Z = z]$. Double ML gives one number; CATE gives a function.

R-Learner: Compute residuals as in Double ML, then fit $\hat{\tau}(Z)$ minimizing $\sum_i (\tilde{Y}_i - \tau(Z_i) \cdot \tilde{X}_i)^2$. Double ML asks "what's the single slope?" R-Learner asks "how does the slope change with Z?"

Explain in detail why "ML models are biased by design" and why this matters for causal inference but not for prediction.

Explaining the bias-variance tradeoff and its implications for causal estimation

Every model's error: Bias² + Variance + Noise. ML deliberately introduces bias to reduce variance — LASSO shrinks coefficients, tree depth limits prevent memorization, dropout forces redundancy.

Fine for prediction: If adding bias reduces variance by more, total error decreases. A house price model that's systematically $10K low but stable beats one that's unbiased but swings $200K.

Fatal for causal inference: You need an unbiased $\hat{\delta}$ and valid CIs. LASSO shrinks $\hat{\beta}_X$ toward zero. Your CI might be [0.08, 0.16] when truth is 0.25 — narrow, precise, doesn't contain the truth. Regularization bias doesn't shrink with more data. The CI is permanently centered wrong.

Double ML's fix: ML for nuisance models only (bias OK). OLS for $\delta$ (one parameter, no regularization). Cross-fitting ensures the bias doesn't leak.

Sensitivity Analysis
Covering sensitivity analysis framework, breakdown point, and interpretation

Every method assumes the DAG is correct. Sensitivity analysis asks: what if we're wrong?

"How strong would an unmeasured confounder have to be to explain away our estimated treatment effect?"

If $U \to X$ and $U \to Y$ exist unblocked, the bias is $\beta_U \cdot (E[U|X{=}1] - E[U|X{=}0])$. The breakdown point $\alpha^*$ is the minimum confounder strength that zeros out the effect.

BreakdownInterpretationVerdict
Very largeNeed a confounder stronger than anything observedRobust
SmallA mild confounder kills the effectFragile
NegativeAdding confounding strengthens the effectVery robust

Analogy: "My bridge holds 10 tons. Sensitivity analysis says it survives up to 25 tons. I don't know if a 25-ton truck will ever cross — but I know my safety margin."

Putting It All Together

Give me a full method comparison — "when to use which" — to tie everything together for the exam.

Synthesizing the complete method comparison and decision framework

The Decision Tree

  1. Can you run an experiment? → Randomize. Naive estimator + bootstrap. Done.
  2. Valid back-door set Z? → No: need IV/RDD/diff-in-diff.
  3. What does Z look like? Discrete → Stratification. Continuous → Matching. High-dim → step 4.
  4. Which modeling approach?
    • Trust linear model → OLS
    • Trust outcome model → BDA predict-and-average
    • Trust propensity model → IPW
    • Don't trust either → Doubly-robust
    • Need valid CIs with ML → Double ML
    • Want $\tau(Z)$ → R-Learner
  5. Always do sensitivity analysis.

One-Sentence Summaries

  • Stratification: Split by Z, compare within strata, weight by P(Z)
  • Matching: Find a twin, compare outcomes
  • IPW: Reweight by 1/P(X|Z) to simulate an experiment
  • OLS: Fit a line, read off $\beta_X$
  • BDA: Fit any model, predict under X=1 and X=0, average
  • Double ML: ML handles confounding, OLS estimates $\delta$
  • Doubly-robust: Consistent if either model is correct

Three Assumptions Everything Rests On

  1. Graph structure — DAG is correct. No statistical test for this.
  2. No unmeasured confounders — Sensitivity analysis stress-tests this.
  3. Support/overlap — $0 < P(X{=}1|Z) < 1$. Checkable via propensity scores.

This was incredibly helpful. Thank you.

Good luck on the exam. The chain you built — potential outcomes → graphs → matching → IPW → regression → Double ML → sensitivity — is exactly how the pieces fit together. You're well prepared.