Hernán & Robins

Causal Inference: What If · Part I

About this guide

A chapter-by-chapter guide to Part I of the book, written so you can read it alongside the book. The book’s tables are reproduced; its figures are described in words, so keep the book open for the pictures.

  • Fine Point boxes: side topics from the book, no extra maths.
  • Technical Point boxes: more formal detail from the book.

What Part I is about

The book’s introduction asks for “less casual causal inferences”: state the causal question explicitly, and keep separate what the data contribute and what your assumptions contribute. The authors set metaphysics aside and focus on causal effects in populations, meaning numbers that describe how the distribution of an outcome would change under different interventions. Their running example is the risk of death in patients with heart failure if they received a heart transplant compared with if they did not.

The book has three parts. Part I (this guide) covers identification without models: when, and under which assumptions, a causal effect can in principle be computed from data if you had an infinitely large sample. Part II adds statistical models for estimation in real, finite data, and Part III covers treatments that change over time.

Part I builds up in this order:

Notation used throughout

A
Treatment, e.g. A = 1 heart transplant, A = 0 no transplant.
Y
Observed outcome, e.g. Y = 1 death by day 5.
Ya
Counterfactual (potential) outcome: the outcome that would have occurred under treatment value a. Ya=1 and Ya=0 exist for every individual, but at most one of them is observed.
L
Covariates measured before treatment, e.g. critical condition, severity, age.
U
Unmeasured variables.
Pr[·], E[·]
Probability and expectation (mean) in the population.
Pr[Y = 1 | A = 1]
Conditional probability: the risk among those who actually received A = 1 (an association).
Pr[Ya=1 = 1]
The risk if everyone had received a = 1 (a causal quantity).
X ⫫ Z | L
X is independent of Z within levels of L.

Chapter 1 · A definition of causal effect

Book pp. 3–12

In short

This chapter gives names and symbols to an intuition everyone already has. A treatment causes an outcome in a person if that person’s outcome would differ depending on whether they got the treatment. Because we only ever see one of those two outcomes per person, individual effects are out of reach. What we can aim for is an average causal effect in a population: the risk if everyone were treated compared with the risk if no one were. The chapter’s central lesson is that this comparison of two hypothetical worlds is different from comparing the people who happened to be treated with those who happened not to be. That second comparison measures association, and association is not causation. Every later chapter is about when, and how, observed data can stand in for the hypothetical comparison.

1.1 Individual causal effects

The book opens with two patients. Zeus receives a heart transplant on January 1 and is dead five days later. Suppose we somehow knew that, had he not received the heart, he would have been alive on day five. Then most people would say the transplant killed him. Hera also gets a transplant on January 1 and is alive on day five. Suppose we knew she would also have been alive without it. Then the transplant made no difference to her five-day survival.

In both cases we reason the same way. We compare the outcome when an action is taken with the outcome when it is withheld. If the two differ, the action has a causal effect (harmful or protective). If they are the same, it has none. The action is variously called a treatment, exposure, intervention or policy.

The notation: A is the treatment (1 = treated, 0 = untreated) and Y is the outcome (1 = death, 0 = survival). Capital letters are random variables, lower-case letters are particular values. The new object is the counterfactual outcome Ya, read “Y under treatment a”:

Zeus has Ya=1 = 1 and Ya=0 = 0. Hera has Ya=1 = 0 and Ya=0 = 0.

Individual causal effect

Treatment A has a causal effect on individual i’s outcome if Yia=1 ≠ Yia=0. For Zeus, 1 ≠ 0, so there is an effect. For Hera, 0 = 0, so there is none.

These quantities are called potential outcomes (emphasising that either could have been observed) or counterfactual outcomes (emphasising that at least one of them did not happen). The book uses “counterfactual”.

One of the two counterfactuals is always factual: the one matching the treatment the person actually got. Zeus was treated (A = 1), so his Ya=1 = 1 is just his observed outcome Y = 1. This link between the hypothetical and the observed is called consistency.

Consistency

If Ai = a, then Yia = Yi. Written compactly: Y = YA, the counterfactual evaluated at the treatment actually received equals the observed outcome.

The other counterfactual is never seen. Individual causal effects are therefore a missing-data problem: for every person, half of what we need is missing. In general individual effects cannot be identified, meaning they cannot be written as a function of the observed data. (Fine Point 2.1, on crossover trials, describes a narrow exception.)

1.2 Average causal effects

An individual effect needs three ingredients: an outcome, the two actions being compared, and the individual. An average causal effect swaps the individual for a well-defined population. The running example is Zeus’s extended family of 20. Table 1.1 is a god’s-eye view: it lists both counterfactual outcomes for every member, something no real dataset contains.

Ya=0Ya=1
Rheia01
Kronos10
Demeter00
Hades00
Hestia00
Poseidon10
Hera00
Zeus01
Artemis11
Apollo10
Leto01
Ares11
Athena11
Hephaestus01
Aphrodite01
Polyphemus01
Persephone11
Hermes10
Hebe10
Dionysus10
Table 1.1 Both counterfactual outcomes (1 = died within five days) for each of the 20 members of the population: under no transplant (a = 0) and under transplant (a = 1). Only possible “by divine revelation”.

Read down the Ya=1 column: 10 of the 20 would die if everyone got a transplant, so Pr[Ya=1 = 1] = 10/20 = 0.5. Read down the Ya=0 column: 10 of the 20 would also die if no one got a transplant, so Pr[Ya=0 = 1] = 10/20 = 0.5. For a 0/1 outcome, the proportion with the outcome is the same as the mean of the column, which is why risks and expectations can be used interchangeably.

Average causal effect

An average causal effect of A on Y is present in a population if Pr[Ya=1 = 1] ≠ Pr[Ya=0 = 1], or more generally (covering non-binary outcomes) if E[Ya=1] ≠ E[Ya=0].

Here both risks are 0.5, so there is no average causal effect: half the family dies whether everyone or no one is transplanted. The null hypothesis of no average causal effect is true.

For a treatment with more than two levels, “the causal effect” means nothing until you say which two actions are compared. “The effect of aspirin” is empty. “150 mg of aspirin by mouth daily for 5 years, versus no aspirin” is a well-defined contrast, even if some other version of the intervention (say, 500 mg through the skin) is not.

A null average does not mean nobody was affected. In Table 1.1, 12 people have Ya=1 ≠ Ya=0. Six were harmed (Ya=1 − Ya=0 = 1: Rheia, Zeus, Leto, Hephaestus, Aphrodite, Polyphemus) and six were helped (difference = −1: Kronos, Poseidon, Apollo, Hermes, Hebe, Dionysus). The harms and benefits cancel. This is no coincidence: a difference of averages always equals the average of the differences,

E[Ya=1] − E[Ya=0] = E[Ya=1 − Ya=0] = (6 × 1 + 6 × (−1) + 8 × 0)/20 = 0.

Sharp causal null hypothesis

Ya=1 = Ya=0 for every individual. It implies the (average) causal null, but not the other way round, as Table 1.1 shows.

From here on the book says “causal effect” to mean average causal effect, and “causal null” to mean no average effect. Averages are the target because, unlike individual effects, they can sometimes be identified from data.

Fine Point 1.1 · Interference

The notation Yia quietly assumes that a person’s outcome depends only on their own treatment. If Hera’s transplant so upset Zeus that it changed his own survival, Zeus’s counterfactual would depend on Hera’s treatment too, and “the effect of transplant on Zeus” would need to specify what happened to Hera. Interference is common with infectious diseases (vaccinating your neighbour protects you) and social programs. The no-interference assumption is part of Rubin’s SUTVA (stable unit treatment value assumption). The book assumes no interference unless stated otherwise.

Fine Point 1.2 · Multiple versions of treatment

The notation also assumes that “treated” means one thing. If Zeus would survive surgery by Asclepios but die under Hygieia, then Ya=1 is not well defined without saying which surgeon (and which procedure, hospital, and so on). This is the other half of SUTVA. A softer version, “treatment variation irrelevance” (VanderWeele), allows several versions as long as they all lead to the same outcome. Chapter 3 returns to this; until then it is assumed.

1.3 Measures of causal effect

The causal null can be expressed on several scales. In the heart transplant population, where both counterfactual risks are 0.5:

(i) causal risk difference: Pr[Ya=1 = 1] − Pr[Ya=0 = 1] = 0.5 − 0.5 = 0
(ii) causal risk ratio: Pr[Ya=1 = 1] / Pr[Ya=0 = 1] = 0.5/0.5 = 1
(iii) causal odds ratio: {Pr[Ya=1 = 1] / Pr[Ya=1 = 0]} / {Pr[Ya=0 = 1] / Pr[Ya=0 = 0]} = 1

When there is a causal effect (say, smoking on lung cancer), these are no longer 0, 1 and 1. They quantify the same effect on different scales, and are collectively called effect measures.

Which scale to use depends on the question. Suppose 3 in a million people would develop the outcome if treated and 1 in a million if untreated. The causal risk ratio is 3: treatment triples the risk. The causal risk difference is 0.000002: two extra cases per million treated. The ratio (multiplicative scale) tells you how many times treatment raises the risk; the difference (additive scale) tells you how many cases are attributable to treatment, which is what matters for public-health burden.

A margin note adds a subtle point. The causal risk difference is the average of the individual effects Ya=1 − Ya=0. The causal risk ratio is a population-level measure but is not the average of any individual ratios Ya=1/Ya=0.

Fine Point 1.3 · Number needed to treat

Take 100 million patients: 20 million would die within five years if treated, 30 million if untreated. The causal risk difference is 0.2 − 0.3 = −0.1. Treating everyone gives 10 million fewer deaths, so you need to treat 10 patients to save one life. In general, for a beneficial treatment, NNT = −1 / (Pr[Ya=1 = 1] − Pr[Ya=0 = 1]) = −1/(−0.1) = 10. For a harmful treatment the analogous quantity is the number needed to harm. Like the risk difference, NNT is tied to a specific population and time window.

Technical Point 1.1 · Causal effects beyond the mean

E[Ya] is the mean of the counterfactual distribution (a sum over values for discrete outcomes, an integral for continuous ones). A population causal effect can be defined on any summary of the counterfactual distributions: median, variance, hazard, the whole cdf. In Table 1.1 the two counterfactual distributions are identical (10 deaths out of 20 each), so every such summary shows no effect. A caution: for the mean, a difference of means equals the mean of differences, but this fails for nonlinear summaries. Var(Ya=1) − Var(Ya=0) = 0 here, while Var(Ya=1 − Ya=0) > 0 because individual effects are −1, 0 or 1. A randomized trial can estimate the first but never the second, because the joint distribution of the two counterfactuals (their covariance) is never observed.

1.4 Random variability

Table 1.1 is unrealistic in another way: it covers a whole population of 20. Real studies observe a sample from a larger population. That introduces the first source of random error, sampling variability. View the 20 family members as a random sample from a near-infinite super-population. The sample proportion P̂r[Ya=0 = 1] = 10/20 = 0.5 estimates the super-population risk Pr[Ya=0 = 1], which might really be 0.57. The hat marks an estimator.

Consistent estimator

An estimator θ̂ of θ is consistent if, with probability approaching 1, θ̂ − θ approaches 0 as the sample size grows. Sample proportions are consistent for population probabilities by the law of large numbers. Note that “consistency” of an estimator is unrelated to the consistency assumption Y = YA from Section 1.1.

Because we only have estimates, we cannot be certain whether there is a causal effect; a statistical procedure is needed to weigh the evidence (Chapter 10).

The second source of randomness is nondeterministic counterfactuals. So far Zeus’s counterfactuals are fixed: he dies for sure if treated and survives for sure if not. One could instead imagine that Zeus has a 90% chance of dying if treated and a 10% chance if untreated, and that the 1s and 0s in Table 1.1 are just one roll of these “mortality coins”. Those probabilities would vary from person to person. Quantum mechanics suggests some outcomes really are irreducibly random in this sense.

Technical Point 1.2 · Nondeterministic counterfactuals

With stochastic counterfactuals each person carries a personal probability of the outcome under each treatment (for Zeus, 0.9 under treatment), or more generally a personal distribution of Ya. The population mean E[Ya] is the average of these personal means, so the population-level definitions carry over unchanged. For binary outcomes, the population causal risk ratio becomes a weighted average of individual risk ratios, with weights proportional to each person’s untreated risk (provided no one’s untreated risk is exactly 0).

Until Chapter 10, the book sets random error aside. It assumes deterministic counterfactuals and data on a huge super-population: think of the 20 family members as 20 billion people, a billion copies of each. All computations are then done “with Olympian certainty”. Chapter 10 shows that estimates and confidence intervals for super-population effects are the same whether the world is deterministic or stochastic, and super-population effects are usually the ones of interest.

1.5 Causation versus association

Real data do not look like Table 1.1. Each person is either treated or not, so we see only the treatment received, A, and the observed outcome, Y. Table 1.2 shows what an investigator would actually have for the same 20 people.

AY
Rheia00
Kronos01
Demeter00
Hades00
Hestia10
Poseidon10
Hera10
Zeus11
Artemis01
Apollo01
Leto00
Ares11
Athena11
Hephaestus11
Aphrodite11
Polyphemus11
Persephone11
Hermes10
Hebe10
Dionysus10
Table 1.2 The observed data for the same population: treatment actually received (A, 1 = transplant) and observed outcome (Y, 1 = died). Each Y is the entry of Table 1.1 matching that person’s A.

From these data we can compute risks conditional on treatment received. Of the 13 treated, 7 died, so Pr[Y = 1 | A = 1] = 7/13 ≈ 0.54. Of the 7 untreated, 3 died, so Pr[Y = 1 | A = 0] = 3/7 ≈ 0.43.

Independence / association

A and Y are independent, written Y ⫫ A, when Pr[Y = 1 | A = 1] = Pr[Y = 1 | A = 0]. Otherwise they are associated (dependent). For a continuous outcome, “mean independence” E[Y | A = 1] = E[Y | A = 0] plays this role; for binary outcomes the two notions coincide, and for binary A and Y, “not associated” is the same as “uncorrelated”.

The association measures mirror the effect measures: associational risk difference, risk ratio and odds ratio, which equal 0, 1 and 1 under independence. Here they are:

risk difference = 7/13 − 3/7 ≈ 0.54 − 0.43 = 0.11
risk ratio = (7/13)/(3/7) ≈ 1.26
odds ratio = (7/6)/(3/4) ≈ 1.56

So in the same population we have found (i) no causal effect: the risk if all 20 were treated (10/20) equals the risk if all 20 were untreated (10/20); and (ii) an association: the 13 who happened to be treated died more often (7/13) than the 7 who happened not to be (3/7). Figure 1.1 draws the distinction.

A diamond labelled ‘Population of interest’ is split into a white left part (Treated) and a smaller grey right part (Untreated). Two arrows lead down. Under ‘Causation’ an all-white diamond, labelled E[Y^(a=1)], is compared (‘vs.’) with an all-grey diamond, labelled E[Y^(a=0)]. Under ‘Association’ the white left piece alone, labelled E[Y|A=1], is compared with the grey right piece alone, labelled E[Y|A=0].

Figure 1.1 Causation compares the whole population under two hypothetical scenarios: everyone treated (all-white diamond) versus everyone untreated (all-grey diamond). Association compares two different subsets of the actual population: the people who were treated (white piece) versus those who were not (grey piece).

Put in symbols, Pr[Y = 1 | A = a] is a conditional probability: the risk in the subset who actually received a. Pr[Ya = 1] is an unconditional (marginal) probability: the risk in the whole population under a. Association compares two disjoint groups defined by what actually happened. Causation compares the same group under two different “what if” treatments. Causal questions live in counterfactual worlds; associational questions live in the actual one. The book often writes “causal effect” (redundantly) to keep clear of the loose use of “effect” for any association.

Why would the two disagree? If the transplant recipients were on average sicker than the non-recipients, they would die more often even if the transplant did nothing. Chapter 7 names this discrepancy confounding.

A margin example shows the stakes. Suppose aspirin halves 5-year mortality (causal risk ratio 0.5), but because doctors preferentially give aspirin to patients at high cardiovascular risk, the associational risk ratio is 1.5. A physician who reads only the association and stops prescribing aspirin will harm her patients.

The closing question of the chapter: causal inference needs data like Table 1.1, but we only ever get data like Table 1.2. Under what conditions can the latter answer causal questions? Chapter 2 gives the first answer: run a randomized experiment.

Key takeaways

  • A counterfactual outcome Ya is the outcome a person would have under treatment a. Treatment has an individual causal effect if Ya=1 ≠ Ya=0.
  • Consistency (Y = YA) links counterfactuals to observed data, but only one counterfactual per person is ever observed, so individual effects are generally not identifiable.
  • The average causal effect compares E[Ya=1] with E[Ya=0] in a whole population. A null average can hide many individual effects that cancel (6 harmed, 6 helped in Table 1.1).
  • Counterfactuals are only well defined under no interference and a single version of treatment (SUTVA); the treatment contrast must be stated precisely.
  • Risk difference, risk ratio, odds ratio and NNT are different scales for the same effect; choose by the question.
  • Random error comes from sampling and possibly from stochastic counterfactuals; the book ignores it until Chapter 10.
  • Association compares treated with untreated people; causation compares everyone treated with everyone untreated. In Table 1.2 the risk ratio is 1.26 while the causal risk ratio is 1.

Chapter 2 · Randomized experiments

Book pp. 13–26

In short

Randomization makes the missing counterfactuals missing by chance. When treatment is assigned by a coin, the treated and the untreated are exchangeable: each group would have had the same risk under the same treatment. Then the risk in the treated equals the risk if everyone were treated, and association is causation. When the coin’s bias depends on a measured covariate L (conditional randomization), exchangeability only holds within levels of L. The population effect can still be recovered, by standardization (average the stratum-specific risks over the distribution of L) or by inverse probability weighting (weight each person by 1 over the probability of the treatment they got). The two methods give the same answer, and both reappear in Part II as the main tools for observational data.

The chapter opens with a thought experiment. Does your looking up at the sky make passing pedestrians look up? You stand on a Madrid sidewalk and flip a coin for each approaching pedestrian: heads, you look up; tails, you look ahead. Over a few thousand trials, 55% of pedestrians look up when you do and 1% when you don’t. This is a randomized experiment: an experiment because you control the action, randomized because a coin decides it. Had you instead looked up only when a man approached, critics could say that men and women differ in how often they look up, so you compared noncomparable groups. The chapter explains why the coin makes the result convincing.

2.1 Randomization

In a real study we see only one counterfactual per person, the one matching their treatment. Table 2.1 is the heart transplant population from Chapter 1 as an investigator would see it, with the unobserved counterfactuals marked “?”.

AYY0Y1
Rheia000?
Kronos011?
Demeter000?
Hades000?
Hestia10?0
Poseidon10?0
Hera10?0
Zeus11?1
Artemis011?
Apollo011?
Leto000?
Ares11?1
Athena11?1
Hephaestus11?1
Aphrodite11?1
Polyphemus11?1
Persephone11?1
Hermes10?0
Hebe10?0
Dionysus10?0
Table 2.1 Observed treatment A and outcome Y, and the two counterfactual outcomes Y0 = Ya=0 and Y1 = Ya=1. By consistency, the counterfactual matching the received treatment equals Y; the other one is missing (“?”).

Randomized experiments produce the same kind of holes. What randomization adds is that the holes occur by chance. That is enough for effect measures to be consistently estimated despite the missing data.

The argument uses the diamond of Figure 1.1. Imagine a near-infinite population and a loaded coin flipped for each person: tails puts them in the white group, heads in the smaller grey group. The white group gets the treatment (A = 1), the grey group a placebo (A = 0). After five days, Pr[Y = 1 | A = 1] = 0.3 and Pr[Y = 1 | A = 0] = 0.6, so the associational risk ratio is 0.3/0.6 = 0.5 and the risk difference is −0.3. Assume an ideal randomized experiment: no loss to follow-up, full adherence, a single version of treatment, double-blind assignment.

Now suppose the assistants mixed up the instructions and treated the grey group instead. Does it change the conclusions? No. Because groups were formed by a coin, they are alike in everything except chance, so whichever group is treated would show a risk of 0.3, and whichever is untreated would show 0.6. Which group received the treatment is irrelevant. The groups are exchangeable.

Exchangeability

The treated and the untreated would have had the same risk had they received the same treatment: Pr[Ya = 1 | A = 1] = Pr[Ya = 1 | A = 0] for both a = 0 and a = 1. Equivalently, the counterfactual outcome is independent of the treatment actually received: Ya ⫫ A for all a. Treatment is then sometimes called exogenous.

If the counterfactual risk is the same in both groups, it must equal the risk in the whole population: Pr[Ya = 1 | A = 1] = Pr[Ya = 1 | A = 0] = Pr[Ya = 1]. And the risk under treatment in the white group is not hypothetical at all: they were treated. So, chaining exchangeability with consistency,

Pr[Ya=1 = 1] = Pr[Ya=1 = 1 | A = 1] = Pr[Y = 1 | A = 1] = 0.3
Pr[Ya=0 = 1] = Pr[Ya=0 = 1 | A = 0] = Pr[Y = 1 | A = 0] = 0.6

The causal risk ratio is 0.5 and the causal risk difference is −0.3, exactly the associational values. In ideal randomized experiments, association is causation.

Another way to see it: think of Ya as a fixed trait of the person, like their genes, that exists before treatment is assigned. A coin flip cannot depend on your genes, and it cannot depend on Ya either. The only difference is that you can learn Ya only after receiving treatment a.

Caution: Ya ⫫ A is not Y ⫫ A. Exchangeability is about the counterfactual outcome. If the treatment works, the observed outcome Y = YA depends on A, so Y and A are associated. That association is precisely the effect we want to measure.

Technical Point 2.1 · Full exchangeability and mean exchangeability

Randomization actually gives more than Ya ⫫ A for each a separately: it makes the whole set of counterfactuals jointly independent of A, (Ya=1, Ya=0) ⫫ A, called full exchangeability. At the other end, mean exchangeability E[Ya | A = 1] = E[Ya | A = 0] only requires equal means. For continuous outcomes, mean exchangeability is weaker (variances could still differ). It is also all you need: E[Y | A = a] = E[Ya | A = a] by consistency, and = E[Ya] by mean exchangeability. For binary outcomes the distinction vanishes.

Does exchangeability hold in the heart transplant study? With god’s-eye access to Table 1.1 we can check. Among the 13 treated, the risk under no treatment is Pr[Ya=0 = 1 | A = 1] = 7/13 (the seven treated with Ya=0 = 1 are Poseidon, Ares, Athena, Persephone, Hermes, Hebe, Dionysus). Among the 7 untreated, Pr[Ya=0 = 1 | A = 0] = 3/7. Since 7/13 > 3/7, the treated had a worse prognosis: they would have died more often even without treatment. Exchangeability fails (for a = 1 too, as you can check).

In practice we only have Table 2.1, which cannot reveal Pr[Ya=0 = 1 | A = 1], so we usually cannot check exchangeability directly. But even if we could, failure would not prove the study was not randomized, for two reasons. First, 20 people is too few; chance imbalance could explain almost anything (until Chapter 10, imagine each person stands for a billion identical copies). Second, a study can be randomized with more than one coin, and then exchangeability need not hold in the whole population. That is the next section.

Fine Point 2.1 · Crossover experiments

Can we ever see both counterfactuals for one person? Suppose Zeus agrees to hurl a lightning bolt only when told to. Yesterday he threw one (a = 1) and his blood pressure rose; today he did not (a = 0) and it did not. It looks as if we observed Ya=1 = 1 and Ya=0 = 0. This crossover logic identifies Zeus’s individual effect only under three strong assumptions: (i) no carryover, yesterday’s treatment does not affect today’s outcome; (ii) the individual effect is the same in both periods; (iii) the untreated outcome is the same in both periods. Then the difference between the two periods’ outcomes equals the individual effect. Condition (i) requires an outcome that starts abruptly and resolves fully, so crossover designs are useless for irreversible treatments and outcomes such as heart transplant and death.

2.2 Conditional randomization

Table 2.2 adds a covariate to the heart transplant data: L = 1 if the person was in critical condition before treatment was assigned, 0 otherwise.

LAY
Rheia000
Kronos001
Demeter000
Hades000
Hestia010
Poseidon010
Hera010
Zeus011
Artemis101
Apollo101
Leto100
Ares111
Athena111
Hephaestus111
Aphrodite111
Polyphemus111
Persephone111
Hermes110
Hebe110
Dionysus110
Table 2.2 The heart transplant study with the pre-treatment prognostic factor L (1 = critical condition). Eight people are noncritical (4 treated) and twelve are critical (9 treated).

Two randomized designs could have produced 13 treated out of 20:

A marginally randomized experiment is expected to give exchangeability, Ya ⫫ A. A conditionally randomized one generally does not, because by design one group gets more of the sicker patients. In Table 2.2, 9/13 = 69% of the treated were critical but only 3/7 = 43% of the untreated. So the treated would have done worse than the untreated even without treatment: A predicts Ya=0, and marginal exchangeability fails. Since the study was randomized, it must have been Design 2.

Design 2 is just two marginally randomized experiments run side by side, one among critical patients and one among noncritical ones. Within each, a single coin was used, so the treated and untreated are exchangeable within that stratum.

Conditional exchangeability

Pr[Ya = 1 | A = 1, L = l] = Pr[Ya = 1 | A = 0, L = l] for every l; written Ya ⫫ A | L for all a.

Summary: marginal randomization gives both marginal and conditional exchangeability. Conditional randomization gives only conditional exchangeability.

Under marginal randomization, the causal risk ratio would be read straight off the data: (7/13)/(3/7) = 1.26. Under conditional randomization, that number is wrong. There are two ways to proceed:

  1. Stratification. Within each stratum of L, association is causation, so the stratum-specific causal risk ratio Pr[Ya=1 = 1 | L = l] / Pr[Ya=0 = 1 | L = l] equals the associational ratio Pr[Y = 1 | L = l, A = 1] / Pr[Y = 1 | L = l, A = 0]. If these differ between strata, there is effect modification by L (treatment-effect heterogeneity), covered in Chapter 4.
  2. The population average effect Pr[Ya=1 = 1] / Pr[Ya=0 = 1] over everyone. This is useful, for instance, if L will not be available when future treatment decisions are made. Sections 2.3 and 2.4 give two ways to compute it.

A margin note links this to missing-data language. Treated people are missing Ya=0; untreated people are missing Ya=1. In a marginally randomized experiment the counterfactuals are missing completely at random (MCAR): Pr[A = a | L, Ya=1, Ya=0] = Pr[A = a]. In a conditionally randomized experiment they are missing at random (MAR): Pr[A = a | L, Ya=1, Ya=0] = Pr[A = a | L].

Fine Point 2.2 · Risk periods

A risk is always the proportion developing the outcome over a specific period, such as 5-day mortality. The period matters. In a trial of antibiotics for plague in elderly people, one analyst reports a causal risk ratio of 0.05 for 1-year mortality, another reports 1 for 100-year mortality. Both are right: over 100 years everyone dies regardless. A treatment that “reduces mortality” delays death; it does not prevent it.

2.3 Standardization

The heart transplant study treated noncritical patients with probability 0.5 and critical patients with probability 0.75. From Table 2.2, the stratum-specific risks are:

StratumSizeUntreated: deaths / nTreated: deaths / n
L = 0 (noncritical)8 (40%)1/4 = 0.251/4 = 0.25
L = 1 (critical)12 (60%)2/3 ≈ 0.676/9 = 2/3 ≈ 0.67
Summary of Table 2.2 by stratum (this summary is ours, not a numbered table in the book). Within each stratum the risk is the same in treated and untreated.

Because treatment was randomized within each stratum (Ya ⫫ A | L), these observed risks are the counterfactual risks for that stratum. For example, Pr[Y = 1 | L = 0, A = 1] = Pr[Ya=1 = 1 | L = 0] = 1/4, and Pr[Y = 1 | L = 1, A = 1] = Pr[Ya=1 = 1 | L = 1] = 2/3.

The risk if all 20 were treated is the average of the stratum risks under treatment, weighted by the size of each stratum:

Pr[Ya=1 = 1] = 1/4 × 0.4 + 2/3 × 0.6 = 0.1 + 0.4 = 0.5
Pr[Ya=0 = 1] = 1/4 × 0.4 + 2/3 × 0.6 = 0.5
causal risk ratio = 0.5 / 0.5 = 1

This recovers the truth from Table 1.1 (no causal effect), whereas the crude associational ratio was 1.26. In general,

Pr[Ya = 1] = Σl Pr[Ya = 1 | L = l] Pr[L = l] = Σl Pr[Y = 1 | L = l, A = a] Pr[L = l]

The first equality is just the law of total probability. The second swaps the counterfactual risk for an observed one, using conditional exchangeability (and consistency). The left side is unobservable; the right side uses only data on L, A and Y.

Identification

A counterfactual quantity is identified if it can be written as a function of the distribution of the observed data. Under conditional exchangeability, Pr[Ya = 1] is identified by the formula above.

Standardization

The standardized mean for treatment level a is Σl E[Y | L = l, A = a] × Pr[L = l]: the outcome means within strata of L among people who got a, averaged using the whole population’s distribution of L as the standard. The causal risk ratio by standardization is

Σl Pr[Y = 1 | L = l, A = 1] Pr[L = l] / Σl Pr[Y = 1 | L = l, A = 0] Pr[L = l].

2.4 Inverse probability weighting

The same causal risk ratio can be computed a second way. Draw Table 2.2 as a tree (Figure 2.1). Everyone starts at the left. The first circle splits by L: 8 noncritical (probability 0.4) and 12 critical (0.6). Each branch then splits by treatment, with the conditional probability of treatment in parentheses, and the circles on the right split by outcome.

Tree diagram. A left circle splits into L=0 (.4) and L=1 (.6). From L=0: branch A=0 (.5) with 4 people, ending in a circle Y=0 (.75) with 3 and Y=1 (.25) with 1; branch A=1 (.5) with 4 people, ending in Y=0 (.75) 3 and Y=1 (.25) 1. From L=1: branch A=0 (.25) with 3 people, ending in Y=0 (1/3) 1 and Y=1 (2/3) 2; branch A=1 (.75) with 9 people, ending in Y=0 (1/3) 3 and Y=1 (2/3) 6.

Figure 2.1 Table 2.2 as a tree. Numbers below each treatment branch are counts; numbers in parentheses are conditional probabilities; numbers at the far right are counts of survivors (Y = 0) and deaths (Y = 1). For example, among the 4 untreated noncritical patients, 3 survived and 1 died.

Risk if nobody were treated. Among the 8 noncritical patients, 4 were untreated and 1 of them died. If all 8 had been untreated, we would expect twice as many deaths: 2. Among the 12 critical patients, 3 were untreated and 2 died. If all 12 had been untreated (4 times as many people), we would expect 2 × 4 = 8 deaths. So had all 20 been untreated, 2 + 8 = 10 would have died: Pr[Ya=0 = 1] = 10/20 = 0.5. This scaling-up relies on the treated noncritical patients having, if untreated, the same risk as the untreated noncritical patients, which is exchangeability given L.

Risk if everybody were treated. The same logic: 4 treated noncritical with 1 death scale to 8 with 2 deaths; 9 treated critical with 6 deaths scale to 12 with 8 deaths. Total 10/20 = 0.5. The causal risk ratio is 0.5/0.5 = 1, matching standardization. Figure 2.2 shows the two simulated populations.

Two trees side by side, each starting from L=0 (.4) and L=1 (.6). Left tree (everyone untreated): Untreated (1) branches carry 8 people for L=0, ending in 6 survivors and 2 deaths, and 12 people for L=1, ending in 4 survivors and 8 deaths; Treated (0) branches carry 0 people. Right tree (everyone treated): Treated (1) branches carry 8 people for L=0 (6 survive, 2 die) and 12 for L=1 (4 survive, 8 die); Untreated (0) branches carry 0.

Figure 2.2 Two simulated versions of the population. Left: everyone untreated (each stratum’s untreated patients scaled up to the full stratum). Right: everyone treated. Each yields 10 deaths among 20.

Stacking the two simulated populations gives a pseudo-population twice the original size, in which every person appears once treated and once untreated (Figure 2.3). In the pseudo-population, L is independent of A (both arms have 8 noncritical and 12 critical), so the treated and untreated are exchangeable without conditioning. The crude associational risk ratio in the pseudo-population therefore equals the causal risk ratio, in the pseudo-population and in the original population.

Tree for the pseudo-population. L=0 (.4) splits into Untreated (1) with 8 people (6 survive, 2 die) and Treated (1) with 8 people (6 survive, 2 die). L=1 (.6) splits into Untreated (1) with 12 people (4 survive, 8 die) and Treated (1) with 12 people (4 survive, 8 die). A right-hand column headed W^A = 1/f(A|L) lists weights 1/.5=2 for both L=0 branches, 1/.25=4 for L=1 untreated, and 1/.75=1.33 for L=1 treated.

Figure 2.3 The pseudo-population created by IP weighting, with each person’s weight WA = 1/f(A|L) shown on the right. Each branch now contains the whole stratum, so treatment no longer depends on L.

Why “inverse probability”? The 4 untreated noncritical patients stand in for 8 pseudo-population members, so each gets weight 2 = 1/0.5, and 0.5 is their probability of being untreated given L = 0. The 9 treated critical patients stand in for 12, so each gets weight 1.33 = 1/0.75, the inverse of their probability of treatment given L = 1. The 3 untreated critical patients each get 1/0.25 = 4. Every person is weighted by the inverse of the probability of the treatment they actually received, given their L.

IP weight

WA = 1/f[A | L], where f[a | l] = Pr[A = a | L = l] for discrete variables. A treated person with L = l gets 1/Pr[A = 1 | L = l]; an untreated person gets 1/Pr[A = 0 | L = l].

The worked numbers, using the weights:

treated: (1 × 2 + 6 × 1.33) / (4 × 2 + 9 × 1.33) = (2 + 8) / (8 + 12) = 10/20 = 0.5
untreated: (1 × 2 + 2 × 4) / (4 × 2 + 3 × 4) = (2 + 8) / (8 + 12) = 10/20 = 0.5

Technical Point 2.2 · Formal definition of IP weights

The notation f[A | L] means the conditional probability (or density) of treatment, evaluated at each person’s own values of A and L, which gives one formula for everyone’s weight. The mean outcome among those with A = a in the pseudo-population equals the IP-weighted mean in the original population, E[I(A = a) Y / Pr(A = a | L)], where I(A = a) is 1 if A = a and 0 otherwise. The proof uses the fact that E[I(A = a) / Pr(A = a | L) | L] = 1: within each stratum, the weights of the people who got a add up to the stratum size.

IP weighting and standardization agreeing is not luck: they are mathematically equivalent (Technical Point 2.3). Both build a counterfactual tree in which everyone receives treatment a. They use different pieces of the observed tree to do it. IP weighting uses the probability of treatment given L, Pr[A | L]. Standardization uses the distribution of L, Pr[L], and the outcome risk given A and L, Pr[Y | A, L]. Both simulate a world in which L no longer influenced who got treated, so both are said to adjust for L. People sometimes say “control for”, but this analytic control is quite different from the physical control that randomization provides.

Technical Point 2.3 · Equivalence of IP weighting and standardization

Assume positivity: f[a | l] > 0 for every l that occurs, i.e. everyone has some chance of each treatment level. (Conditionally randomized experiments guarantee this.) Then

E[I(A = a) Y / f[A | L]] = Σl (1/f[a | l]) × E[Y | A = a, L = l] × f[a | l] × Pr[L = l] = Σl E[Y | A = a, L = l] Pr[L = l]

The probability of treatment cancels, leaving the standardized mean. This identity needs no counterfactuals. Adding conditional exchangeability (and consistency) makes both equal to E[Ya]. With a continuous treatment, the simple IP-weighted mean breaks down (positivity fails for any exact value a); Section 12.4 shows how to extend IP weighting to that case, and Technical Point 3.1 discusses what happens without positivity.

Why not stop here? Ideal randomized experiments plus standardization or IP weighting give average causal effects. But randomized experiments are often unethical, impractical or too slow. An ethics committee would hardly let hearts, which are scarce, be allocated by coin rather than to those most likely to benefit. Blinding a transplant is impossible, patients assigned to no transplant may seek one anyway, compatible hearts may not be available, and the trial would take years while decisions must be made now. Frequently an observational study is the least bad option, and the next chapter asks when such a study can be treated like a conditionally randomized experiment.

Key takeaways

  • Randomization does not remove missing counterfactuals; it makes them missing by chance.
  • Exchangeability, Ya ⫫ A, means the treated and untreated would have the same risk under the same treatment. With consistency it gives Pr[Ya = 1] = Pr[Y = 1 | A = a]: in an ideal marginally randomized trial, association is causation.
  • Ya ⫫ A is different from Y ⫫ A; an effective randomized treatment is associated with the observed outcome.
  • When the probability of treatment depends on L, only conditional exchangeability Ya ⫫ A | L holds. In the transplant data the crude risk ratio is 1.26 but the causal risk ratio is 1.
  • Standardization: Pr[Ya = 1] = Σl Pr[Y = 1 | L = l, A = a] Pr[L = l].
  • IP weighting: weight each person by 1/Pr[A = their treatment | L], creating a pseudo-population where L does not predict treatment. It is mathematically equivalent to standardization under positivity.
  • Both methods carry over to observational data, but only if conditional exchangeability holds there, which is no longer guaranteed by design.

Chapter 3 · Observational studies

Book pp. 27–42

In short

An observational study can give the same answer as a randomized experiment only if it can be treated as a conditionally randomized experiment. That requires three identifiability conditions: exchangeability given the measured covariates L, positivity, and consistency. In a real trial these hold by design. In observational data they are assumptions, and exchangeability in particular can never be checked from the data. Consistency has two parts: the treatment must be defined precisely enough that the counterfactual outcome means something, and the treatment version you defined must actually appear in the data. The chapter ends with the target trial: write down the protocol of the randomized trial you would like to run, then use the observational data to emulate it.

Why observational studies are harder

The chapter opens with the pedestrian example again. Instead of running an experiment where you look up at the sky, you watch pairs of strangers on the street and record whether the second one looks up after the first. A critic can point out that both may have looked up because of a thunderclap or the first raindrops. The association between the two people’s behavior then has a common cause, and you cannot tell whether one person’s looking up made the other look up. Randomization would have ruled this out.

Most human knowledge, though, comes from observation: evolution, plate tectonics, the fact that hot coffee burns. The chapter asks under which conditions observational data support causal conclusions.

3.1 Identifiability conditions

In an ideal randomized trial of heart transplant, the transplanted and non-transplanted groups would have had the same risk of death if both had gone without a transplant. So an associational risk ratio of, say, 0.7 can be read as the causal risk ratio.

In an observational study, doctors may give transplants to sicker patients. Then the transplanted group would have had a higher death risk even without the transplant. An observed risk ratio of 1.1 is a mix of two forces: the real benefit of transplant (pushing the ratio below 1) and the worse baseline prognosis of those who got it (pushing it above 1). The association no longer equals the effect.

The usual strategy is to analyze the data as if treatment had been randomly assigned within levels of measured covariates L. The observational study is treated as a conditionally randomized experiment. This analogy holds if three conditions are met:

  1. The treatment values being compared correspond to well-defined interventions, and those interventions match the versions of treatment that appear in the data. This is consistency (Chapter 1).
  2. The probability of receiving each treatment value depends only on the measured covariates L. This is conditional exchangeability (Chapter 2).
  3. The probability of receiving each treatment value, given L, is greater than zero. This is positivity (Technical Point 2.3).

Identifiability conditions

Consistency, exchangeability and positivity. When they hold, IP weighting or standardization applied to observational data recover the causal effect, exactly as in a conditionally randomized experiment.

Table 3.1 is the same data as Table 2.2 from the previous chapter. There it came from a conditionally randomized experiment, and standardization gave a causal risk ratio of 1. If the same numbers came from an observational study and the three conditions held, we would get the same answer, 1. The difference is where the conditions come from. In the trial they hold by design. In the observational study we must assume them. Causal inference from observational data therefore needs two inputs: the data and the identifiability assumptions.

LAY
Rheia000
Kronos001
Demeter000
Hades000
Hestia010
Poseidon010
Hera010
Zeus011
Artemis101
Apollo101
Leto100
Ares111
Athena111
Hephaestus111
Aphrodite111
Polyphemus111
Persephone111
Hermes110
Hebe110
Dionysus110
Table 3.1 The 20 members of Zeus’s family: L = 1 if in critical condition at baseline, A = 1 if they received a heart transplant, Y = 1 if they died. Identical to Table 2.2. Within L = 0 the death risk is 1/4 in both the treated and untreated; within L = 1 it is 2/3 in both. The standardized causal risk ratio is 1.

When any of the three conditions fails, the analogy with a conditionally randomized experiment breaks. Other approaches with different identifying assumptions exist, such as instrumental variables (Chapter 16), where some predictor of treatment is assumed to behave as if randomized. Epidemiology has traditionally relied on the conditional-randomization analogy; economics has leaned on instrumental variables. Until Chapter 16 the book stays with the first approach.

Fine Point 3.1 · Identifiability

An effect is identifiable under some assumptions if those assumptions, together with the distribution of the observed data, are compatible with only one value of the effect. If the data in Table 3.1 came from a trial where treatment probability depended on L, the causal risk ratio is identified as 1 with no further assumptions. If the data came from an observational study, the same data are compatible with a ratio below 1 (other risk factors more common among the treated), above 1 (more common among the untreated), or exactly 1 (other risk factors balanced, that is Ya ⫫ A | L). Only the external assumption picks out 1.

A margin note adds terminology you will meet in other papers. Rosenbaum and Rubin (1983) called exchangeability plus positivity weak ignorability, and full exchangeability plus positivity strong ignorability.

3.2 Exchangeability

In a marginally randomized trial, the treated and untreated are exchangeable because randomization spreads every independent predictor of the outcome equally across the two groups. (An independent predictor of the outcome is a covariate associated with Y within levels of treatment; for a binary outcome these are called risk factors.)

In Table 3.1, 69% of the treated (9 of 13) are in critical condition compared with 43% of the untreated (3 of 7). That imbalance rules out marginal exchangeability. But it is exactly what we expect in a conditionally randomized experiment where the chance of treatment depends on L. Within L = 1, and separately within L = 0, the treated and untreated are exchangeable. That is conditional exchangeability, Ya ⫫ A | L.

Now imagine the same table came from an observational study in which doctors give scarce hearts to those who need them most. Treatment probability is 9/12 = 0.75 when L = 1 and 4/8 = 0.50 when L = 0. If L is the only outcome predictor that differs between treated and untreated, this observational study is logically equivalent to a trial that randomized transplant with probability 0.75 to critical patients and 0.50 to the rest. Standardization or IP weighting then gives the causal effect.

The question that matters is whether L really is the only imbalanced outcome predictor. That question cannot be answered from the data. Two points follow.

First, not every variable that differs between groups needs to be in L. Hearts are matched to recipients by HLA genes. HLA type affects who gets transplanted, but it does not predict mortality given L and A, so it does not need adjusting for.

Second, unmeasured predictors can break exchangeability. Suppose doctors, without telling the investigators, prefer to transplant nonsmokers. Among patients with L = 1, smokers (U = 1) are less likely to be treated. Smoking predicts death. So within L = 1 the treated have fewer smokers than the untreated, and conditional exchangeability given L fails. Measuring smoking would fix that one variable, but there could be other unmeasured predictors U that you never thought of.

When conditional exchangeability fails

Ya ⫫ A | L does not hold if some unmeasured independent outcome predictor U affects the probability of treatment within strata of L. Because U is unmeasured, it cannot be used in standardization or IP weighting.

Worse, even when the assumption is true, you cannot verify it. Checking it would require comparing Pr[Ya = 1 | A = a, L = l] with Pr[Ya = 1 | A ≠ a, L = l]. The second quantity involves the outcome under treatment a for people who did not get a, which is never observed. Investigators can make the assumption more plausible by measuring many variables that drive treatment decisions and also predict the outcome, but no number of measured variables makes it testable. The validity of the analysis rests on expert knowledge being right.

Fine Point 3.2 · Crossover randomized experiments

Fine Point 2.1 showed that a crossover design (each person gets both treatments in two periods) can identify individual effects under three strong conditions: no carryover, an effect that is constant over time, and an untreated outcome that is constant over time. If the order of treatments is randomized, the third condition can be dropped. Individual effects are then no longer identified, but the average of the within-person differences still estimates the average causal effect, provided there is no carryover. For the treatments and outcomes in this book, no carryover is rarely plausible.

3.3 Positivity

An experiment that gave everyone the same treatment would be useless: with nobody in the comparison group you cannot compute an effect. So every participant must have a positive probability of receiving each treatment level. In a marginally randomized trial, Pr[A = 1] and Pr[A = 0] are both positive by design. In a conditionally randomized trial, Pr[A = 1 | L = l] and Pr[A = 0 | L = l] are positive for every eligible level of L. For Table 3.1 these are 0.75 and 0.50 for treatment; neither is 0 or 1.

Positivity

Pr[A = a | L = l] > 0 for every treatment value a in the causal contrast and every value l with Pr[L = l] ≠ 0 in the population of interest. Also called the experimental treatment assumption.

Two refinements. Positivity is needed only for values of L present in the population you are studying: if the study is restricted to critical patients, nothing is required of L = 0. And positivity is needed only for the variables required for exchangeability. Nobody asks whether blue-eyed patients have a positive chance of transplant, because eye color is not needed to make groups exchangeable.

In observational studies positivity is not guaranteed. If doctors always transplant critical patients, then Pr[A = 0 | L = 1] = 0. Figure 3.1 shows this case as a tree.

Tree diagram. A root circle splits into L=0 (probability .4) and L=1 (probability .6). L=0 branches to A=0 (.5, 4 people) and A=1 (.5, 4 people); each of those ends in a circle with Y=0 (.75) giving 3 people and Y=1 (.25) giving 1 person. L=1 branches to A=0 (probability 0, 0 people), whose circle shows Y=0 and Y=1 with 0 people each, and A=1 (probability 1, 12 people), whose circle shows Y=0 (1/3) with 4 people and Y=1 (2/3) with 8 people.

Figure 3.1 A positivity violation. Of the 20 people, 8 have L = 0 and are split 4/4 between treatments. All 12 with L = 1 are treated; no one with L = 1 is untreated, so the data contain no information about what would happen to critical patients without a transplant.

Unlike exchangeability, positivity can sometimes be checked in the data (Chapter 12). In Table 3.1 there are treated and untreated people at both levels of L, so positivity holds there. In Figure 3.1 the empty branch shows that it fails. The reason it matters: standardization needs Pr[Y = 1 | A = 0, L = 1], and with nobody in that cell, nothing in the data can stand in for what the treated critical patients would have experienced untreated.

Technical Point 3.1 · Positivity for standardization and IP weighting

The standardized mean Σl E[Y | A = a, L = l] Pr[L = l] is defined only when every l in the population has some people with A = a; otherwise a conditional mean is undefined. For IP weighting, the version E[I(A = a)Y / f[a|L]] hits 0/0 and is undefined. The version E[I(A = a)Y / f[A|L]] can always be computed, but under a positivity violation it equals E[Ya | L ∈ Q(a)] Pr[L ∈ Q(a)], where Q(a) is the set of l values where A = a can occur. With a binary treatment and a violation, Q(1) ≠ Q(0), so the difference between the two IP weighted means compares two different groups and has no causal interpretation, even under exchangeability. The software will still give you a number.

3.4 Consistency: First, define the counterfactual outcome

Consistency says Ya = Y for every individual with A = a. It sounds trivially true: if I take aspirin and die, my outcome under aspirin is death. The book splits it into two parts. The first part, covered here, is that Ya must be well defined, which means the intervention a must be specified in enough detail. The second part (Section 3.5) is that the observed data must contain people who received that intervention.

In a trial, the protocol defines the interventions. For a heart transplant trial it would specify pre-operative procedures, anesthesia, surgical technique, post-operative care and immunosuppression, so that each patient receives the same version of “transplant”. Ya=1 is then the outcome if the protocol’s instructions for a = 1 were followed. An observational study needs the same: you must say what intervention a stands for. For exercise, that means duration, frequency, intensity, type, and what the person would otherwise do with that time.

This is hard for biological or social states such as blood pressure, body weight or socioeconomic status. The book’s example: on Zeus’s 35th birthday we make him gain 20 kg by age 40 by cutting his exercise, and he dies of a heart attack at 49. In a parallel universe we make him gain the same 20 kg by feeding him more, and he is alive at 50. Both universes have “weight gain of 20 kg”, but the outcomes differ. So “gain 20 kg” is too vague to define Ya. Weight can only be changed through interventions like diet or exercise, and those interventions affect the outcome through other pathways too. A weight-loss pill does not have this problem: “take the pill” is itself an intervention.

Fine Point 3.3 · States and interventions

Variables like obesity or poverty describe states. “The effect of becoming obese on heart attack” has no clear meaning because there are many ways to become obese. If you specify the timing and the method of weight change, you are no longer asking about obesity; you are asking about the specific intervention you described. A margin note adds that well-definedness can depend on the outcome. For job discrimination, the treatment is “obesity as perceived by the employer” from a photo, and the biology behind the weight is irrelevant.

Absolute precision is neither possible nor needed. No one specifies whether you run clockwise or counterclockwise around the park, because experts agree it does not matter. We need interventions that are sufficiently well defined, with no meaningful vagueness left. Whether that point has been reached is a matter of expert consensus, and consensus changes. Future research could show that leaning right while running is harmful, and then the old definition would be too vague.

Even trial protocols leave gaps. The transplant protocol did not specify surgeon experience. Inexperienced surgeons raise post-transplant mortality, so Pr[Ya=1 = 1] depends on the mix of surgeons in the trial. Applying the result to a community with different surgeons would give a different effect. (Pragmatic trials leave some components unspecified on purpose, so that the versions used reflect real practice.)

Fine Point 3.4 · Protocols open to interpretation

Two randomized trials with identical populations and identical protocols can find different values of Pr[Ya=1 = 1]. Suppose the protocol does not say how to assign surgeons. In trial 1, high-risk patients go to experienced surgeons and almost nobody dies. In trial 2, surgeons are assigned without regard to risk, and many high-risk patients die. Randomization rules out a difference in exchangeability; the difference comes from different versions of treatment.

Fine Point 3.5 · Possible worlds

Philosophers (Stalnaker, Lewis) define Ya as the outcome in the “closest possible world” where the person received a. Under that definition consistency always holds, since the closest world to the actual one is itself. Robins and Greenland (2000) argued that in observational research the idea of a well-defined intervention should replace the closest possible world, because counterfactuals are only as clear as the intervention you specify.

The practical upshot: start from a vague question such as “the effect of obesity” and refine it with colleagues until everyone agrees no meaningful vagueness remains. The more precisely the treatment is stated, the less room there is for researchers and decision makers to misunderstand each other. A margin note quotes Holland’s “no causation without manipulation”, with the caveat that a well-defined intervention need not be feasible today; the effect of a genetic variant was considered well defined before gene editing existed.

3.5 Consistency: Second, link counterfactuals to the observed data

Suppose the transplant intervention a = 1 is fully specified, including that the surgeon has done at least 10 transplants in the past five years. Experts agree it is well defined. Now suppose every surgeon in your observational data has done only 5 to 9. Then nobody in the data received a = 1 as defined, and Ya=1 cannot be linked to any observed Y. This is a failure of unconditional positivity for the treatment version of interest.

So consistency can fail in two ways: the treatment is too vague (weight gain), or it is well defined but absent from the data (transplant by experienced surgeons). To link Ya=1 to Y, only people who received version a = 1 should count as treated in the analysis, and likewise for the untreated.

One way around a mismatch is to assume treatment variation irrelevance: the versions present in the data have the same effect as the version of interest. Sometimes that is reasonable. It may be fair to say that after 5 transplants a surgeon has learned everything that matters, so data on surgeons with 5–9 transplants can stand in for the “at least 10” protocol. It would be hard to argue the same for surgeons who had done only one. A margin note gives a second example: for the effect of lowering blood pressure with drugs, investigators may accept that the specific drug used does not matter.

Judging any of this requires detailed data on treatment versions. Without data on surgeon experience you cannot even tell whether there is a mismatch. Observational data often lack such detail, so consistency is often compromised. The book’s advice is to make the reasoning explicit so others can challenge it. A margin note warns about a common source of confusion: using the same letter for the hypothetical intervention a and the recorded variable A before checking that they actually match.

3.6 The target trial

An average causal effect compares mean counterfactual outcomes under different interventions. Since the counterfactuals must be well defined, we can always imagine a randomized experiment that would estimate the effect. The book calls it the target trial. When that trial is not feasible, ethical or timely, we analyze observational data, and that analysis can be viewed as an attempt to emulate the target trial. If the emulation succeeds, the observational result equals what the target trial would have found.

Target trial

The hypothetical randomized experiment that would answer your causal question. An observational analysis is framed as an emulation of it: (i) specify the protocol of the trial you would like to run, then (ii) describe how the observational data will be used to mimic each part of that protocol.

The three identifiability conditions are exactly what is needed to equate an observational study with a conditionally randomized target trial. When they hold, IP weighting or standardization give the causal effect. The guiding question becomes: what randomized experiment are you trying to emulate?

Specifying the target trial means writing down the key components of its protocol:

A margin note mentions PICO (Population, Intervention, Comparator, Outcome), which covers some of these. A valid emulation requires observational data rich enough to identify eligible people, assign them to arms according to the treatment they actually started, and ascertain their outcomes over follow-up. Writing the protocol down makes the interventions, and therefore the question, well defined. Once that is done, investigators can turn to the separate question of whether conditional exchangeability is achievable with the measured covariates. If it is doubtful, alternatives are other identifying assumptions (Chapter 16) or sensitivity analyses. Chapter 22 returns to the framework in depth.

When the question cannot be translated into a trial, the fix is to replace it with one that can. “The effect of weight loss on mortality in nonsmoking obese 40-year-olds” is vague; a trial of weight loss through a specific diet or exercise program is not.

Some authors object that insisting on a target trial restricts the questions we can ask. Their argument: we may not be able to pin down “the effect of weight loss”, but it is still valuable to know that many deaths could be prevented if obese people somehow lost weight. The book gives two reasons why this is a problem.

First, unspecified interventions can be unrealistic. If investigators simply compare death risk in obese and non-obese 40-year-olds, they are implicitly emulating a trial in which obese people are instantly turned into people with a BMI of 25 at baseline (by massive liposuction?). Almost nobody undergoes such a change in real life, so the counterfactual cannot be linked to observed outcomes, and consistency fails. Had the intervention been stated, everyone would have seen that. Explicit emulation stops investigators from making implicit consistency assumptions they would themselves reject. Even when the method of weight loss cannot be specified, a trial can still keep other components realistic. With longitudinal weight data, one can emulate a strategy of losing 5% of BMI per year from age 40 for as long as BMI stays above 25. This is still not fully defined (how is the weight lost?), but it avoids an impossible instantaneous change. Danaei et al. (2016) did this kind of analysis. Being unable to make counterfactuals perfectly defined is no excuse for leaving them less defined than you can.

Second, unspecified interventions make exchangeability and positivity harder. The covariates L needed for exchangeability can differ by intervention: what confounds the effect of diet-induced weight loss may differ from what confounds exercise-induced weight loss. If the intervention is unspecified, you cannot know which L to adjust for. Nor can you tell which combinations of L make the intervention impossible, which invites hidden positivity violations.

When a target trial truly cannot be specified, observational data are still useful for prediction. Obesity predicts mortality, which helps identify high-risk people. Saying “predicts” keeps you causally agnostic: obesity could predict death the way smoking predicts lung cancer, or the way carrying a lighter does. The association can generate hypotheses, but its size need not equal any causal effect.

Fine Point 3.6 · Attributable fraction

Risk ratios and differences compare two counterfactual risks. One can also compare the observed risk with a counterfactual one. Example: all 20 family members ate ambrosia (A = 1, 10 people) or nectar (A = 0, 10 people). Next day 7 of 10 ambrosia eaters and 1 of 10 nectar drinkers were sick. Assuming exchangeability, the causal risk ratio is 0.7/0.1 = 7 and the risk difference 0.6. (Doves had contaminated the ambrosia.) Observed risk is 8/20 = 0.4. Risk if everyone had nectar is 0.1. The excess is 0.3, and 0.3/0.4 = 0.75: 75% of cases are attributable to ambrosia, since only 2 of the 8 cases would have happened if everyone had drunk nectar.

excess fraction = (Pr[Y = 1] − Pr[Ya=0 = 1]) / Pr[Y = 1]

This excess fraction differs in general from the etiologic fraction, the share of cases actually caused by exposure. If treatment causes some cases and prevents others, the excess fraction is only a lower bound on the etiologic fraction, which can be computed only under strong assumptions.

Key takeaways

  • An observational study yields causal effects only if it can be treated as a conditionally randomized experiment. That needs exchangeability given L, positivity and consistency.
  • In a trial these conditions hold by design. In observational data they are assumptions. Exchangeability cannot be tested; positivity can sometimes be checked.
  • Conditional exchangeability fails when an unmeasured outcome predictor U affects treatment within strata of L. Choose L using knowledge of how treatment decisions are made.
  • Positivity requires people at every treatment level in every relevant stratum of L. Without it, standardization is undefined and IP weighting compares different subpopulations.
  • Consistency has two parts: a sufficiently well-defined intervention, and that intervention actually occurring in the data. Vague treatments (weight gain) and absent versions (experienced surgeons) both break it.
  • Frame every observational causal question as emulation of a target trial with explicit eligibility, strategies, assignment, outcome, time zero and follow-up, and contrast. This exposes impossible interventions and clarifies what to adjust for.
  • When no target trial can be specified, the data still support prediction, but not a causal reading.

Chapter 4 · Effect modification

Book pp. 43–56

In short

There is no single “causal effect of treatment”. The average effect depends on who is in the population, because individual effects vary and different populations contain different mixes of people. A variable V is an effect modifier when the average causal effect differs across levels of V, and whether it does depends on the scale (risk difference or risk ratio). You detect effect modification by stratifying on V while still adjusting for the confounders L. The chapter then shows that the four adjustment methods (standardization, IP weighting, stratification and matching) answer questions about different populations. When effects are heterogeneous they give different, equally correct numbers.

Why look at subgroups

Whether you need effects in subgroups depends on what decision you face. A policy maker deciding on nationwide water fluoridation cares about the average effect in the whole population, because everyone will get it. Subgroup effects matter when an intervention can be targeted to some people and not others, or when results from one population must be applied to another. The chapter’s message is that the causal effect always belongs to a particular population.

4.1 Heterogeneity of treatment effects

Chapter 1 used the counterfactual outcomes of Zeus’s 20 relatives to show that heart transplant had no average effect: 10 of 20 would die if all were treated, 10 of 20 if none were, so the causal risk ratio was 0.5/0.5 = 1 and the risk difference 0. Table 4.1 shows the same counterfactuals, reordered, with a new column V for sex (1 = woman, 0 = man).

VY0Y1
Rheia101
Demeter100
Hestia100
Hera100
Artemis111
Leto101
Athena111
Aphrodite101
Persephone111
Hebe110
Kronos010
Hades000
Poseidon010
Zeus001
Apollo010
Ares011
Hephaestus001
Polyphemus001
Hermes010
Dionysus010
Table 4.1 Counterfactual outcomes under no transplant (Y0) and transplant (Y1) for the 20 relatives, with sex V (1 = woman, first 10 rows; 0 = man, last 10 rows). Same data as Table 1.1.

Women (first 10 rows): 6 would die if treated and 4 if untreated. Pr[Ya=1 = 1 | V = 1] = 0.6, Pr[Ya=0 = 1 | V = 1] = 0.4. Risk ratio 0.6/0.4 = 1.5, risk difference 0.2. Transplant is harmful in women.

Men (last 10 rows): 4 would die if treated and 6 if untreated. Risk ratio 0.4/0.6 = 2/3, risk difference −0.2. Transplant is beneficial in men.

The effects are equal in size and opposite in sign, and the population is half women, so they cancel to exactly zero overall. Exact cancellation is rare, but heterogeneity of individual effects is common, because people differ in how they respond. The only case where there is no heterogeneity at all is the sharp null: if treatment affects nobody, the average effect is null in every subgroup.

Effect modifier

V modifies the effect of A on Y when the average causal effect of A on Y varies across levels of V. The book only considers modifiers V that are not affected by A.

Because effects can be measured on different scales, so can effect modification:

Additive: E[Ya=1 − Ya=0 | V = 1] ≠ E[Ya=1 − Ya=0 | V = 0]
Multiplicative: E[Ya=1 | V = 1] / E[Ya=0 | V = 1] ≠ E[Ya=1 | V = 0] / E[Ya=0 | V = 0]

In Table 4.1, sex modifies the effect on both scales. The effect also changes direction (harmful in women, protective in men). This is qualitative effect modification. When modification is qualitative, modification on one scale implies modification on the other. Without it, you can have modification on one scale only. The book’s second example:

Pr[Ya=0 = 1 | V]Pr[Ya=1 = 1 | V]Risk differenceRisk ratio
V = 10.80.90.11.1
V = 00.10.20.12
Counterfactual risks from the book’s margin example (no table number). The risk difference is the same in both strata, so there is no additive modification; the risk ratio differs (0.9/0.8 ≈ 1.1 vs 2), so there is multiplicative modification.

Since you cannot say whether there is effect modification without naming the measure, some authors prefer the term effect-measure modification. The book does not consider the odds ratio scale, because the odds ratio is rarely the parameter of interest in causal inference (see Fine Point 4.3).

4.2 Stratification to identify effect modification

To check whether V modifies the effect, compute the effect separately in each stratum of V. With counterfactuals in hand, as in Table 4.1, that is direct. Real data instead have the treatment received A and the observed outcome Y. What you need to do depends on the design.

Stratification

Computing the causal effect of A on Y within each stratum of V. For binary V: Pr[Ya=1 = 1 | V = 1] − Pr[Ya=0 = 1 | V = 1] and Pr[Ya=1 = 1 | V = 0] − Pr[Ya=0 = 1 | V = 0].

Marginally randomized trial. Randomization makes groups exchangeable in every subset of the population. So in women, the causal risk difference equals the associational one, Pr[Y = 1 | A = 1, V = 1] − Pr[Y = 1 | A = 0, V = 1], and likewise in men. A simple stratified analysis is enough.

Conditionally randomized trial (or observational study with exchangeability given L). Take 40 people: 20 Greeks (V = 1) and 20 Romans (V = 0). Transplant was given with probability 0.75 to those in critical condition (L = 1) and 0.50 to the others. The Greeks are the people in Table 3.1 (= Table 2.2). The Romans are in Table 4.2.

Stratum V = 0LAY
Cybele000
Saturn001
Ceres000
Pluto000
Vesta010
Neptune010
Juno011
Jupiter011
Diana100
Phoebus101
Latona100
Mars111
Minerva111
Vulcan111
Venus111
Seneca111
Proserpina111
Mercury110
Juventas110
Bacchus110
Table 4.2 The 20 Romans (V = 0): critical condition L, transplant A, death Y. The 20 Greeks (V = 1) are the people of Table 3.1.

Across all 40 people, standardizing by L gives Pr[Ya=1 = 1] = 0.55 and Pr[Ya=0 = 1] = 0.40, a risk difference of 0.15 and risk ratio of 1.375. Transplant is harmful on average in this population.

To look for modification by nationality, use two stages: (1) stratify by V, then (2) within each stratum, standardize (or IP weight) by L. For the Greeks, Chapter 2 already found a risk difference of 0 and ratio of 1. For the Romans, the book leaves the arithmetic to the reader. Here it is:

RomansPr[L = l]Risk if treated, Pr[Y = 1 | A = 1, L = l]Risk if untreated, Pr[Y = 1 | A = 0, L = l]
L = 0 (8 people)8/20 = 0.42/4 = 0.51/4 = 0.25
L = 1 (12 people)12/20 = 0.66/9 = 2/31/3
Standardized0.4×0.5 + 0.6×2/3 = 0.60.4×0.25 + 0.6×1/3 = 0.3
Worked standardization for the Roman stratum (computed from Table 4.2; not a numbered table in the book).

So among Romans, risk difference = 0.3 and risk ratio = 2. Among Greeks, 0 and 1. Nationality modifies the effect on both scales. The modification is not qualitative, because the effect is harmful or null in both groups, never protective. (Check: averaging over the two equal-sized groups gives (0.6 + 0.5)/2 = 0.55 and (0.3 + 0.5)/2 = 0.40, matching the population figures.) A margin note says that step 2 is unnecessary when V is itself the set of variables L needed for exchangeability.

None of this says nationality causes the difference. Passport nationality may just be a marker. If heart surgery is better in Greece than in Rome, improving surgery in Rome could erase the difference. The book calls nationality a surrogate effect modifier and quality of care a causal effect modifier. Some authors prefer the neutral phrase “effect heterogeneity across strata of V”. Chapter 5′s concept of interaction is the one that assigns a causal role.

Fine Point 4.1 · Effect in the treated

One subset of special interest is the treated themselves. The causal effect in the treated compares their observed risk, Pr[Y = 1 | A = 1] (equal to Pr[Ya=1 = 1 | A = 1] by consistency), with what their risk would have been untreated, Pr[Ya=0 = 1 | A = 1]. The ratio of these is the standardized morbidity ratio (SMR). The effect in the untreated is defined the same way with A = 0. These differ from the population effect when the distribution of individual effects differs between treated and untreated. Treatment group then acts as a marker for unidentified effect modifiers. The book mostly focuses on population effects because effects in the treated do not generalize well to time-varying treatments (Part III).

Diagram: a diamond labelled Population of interest is split into a white left half (Treated) and a grey right half (Untreated). Arrows lead to Effect in the treated, which compares the white left half E[Y|A=1] with a grey left half E[Y^(a=0)|A=1], and Effect in the untreated, which compares a white right half E[Y^(a=1)|A=0] with the grey right half E[Y|A=0].

Figure 4.1 The population splits into treated (left half) and untreated (right half). The effect in the treated compares the treated’s observed mean, E[Y | A = 1], with their counterfactual mean had they been untreated, E[Ya=0 | A = 1]. The effect in the untreated compares the untreated’s counterfactual mean had they been treated, E[Ya=1 | A = 0], with their observed mean, E[Y | A = 0]. White = treated condition, grey = untreated condition.

4.3 Why care about effect modification

The book gives three reasons, which is also why you should record pre-treatment characteristics V even in randomized trials.

  1. The average effect depends on the population’s mix of modifiers.
In Table 4.1 the population effect is null only because the population is half women. In a population that was mostly women, such as graduating college students, the same treatment would be harmful on average. Under non-qualitative modification the direction stays the same but the size changes; examples are asbestos (whose effect differs between smokers and nonsmokers) and universal health care (whose effect differs by family income). There is no “average causal effect of A on Y, period”. There is only the average effect in a population with a particular mix of effect modifiers.

Transportability

Using a causal effect estimated in one population for another population. Lack of transportability is sometimes called lack of external validity.

Effects within strata of known modifiers may transport better than population averages, but there is no guarantee. Other modifiers, unmeasured or unknown, may be distributed differently in the two populations. These are not the variables needed for exchangeability; they are any causes of the outcome that interact with treatment. You can learn the confounders by asking how treatment is allocated, but you have much less information about all the possible modifiers. So transportability is harder than identification in a single population, and rests on subject-matter judgment. Most experts would not transport the health effect of adding $100 to household income from Niger to the Netherlands, but would transport the effect of cholesterol-lowering drugs from Europeans to Canadians. A margin note jokes that Smith and Pell (2003) found no major modifiers of the effect of parachutes on death after “gravitational challenge”, so a parachute trial in any one group would generalize.

Fine Point 4.2 · Transportability

Transporting an effect from a study population to a target population can be justified if three things are similar between them. Effect modifiers: their distribution determines the average effect. Versions of treatment: a different mix of versions gives a different effect (Section 3.4). Interference: if treating one person affects others (a socially active person brings friends to exercise), different contact patterns change the effect. One can reconstruct the effect for a new population by reweighting stratum-specific effects. In the running example, the four strata Roman women, Greek women, Roman men and Greek men can be averaged with weights equal to their share in the new population. Unmeasured modifiers, interference and treatment versions can still make this reconstruction wrong.

Technical Point 4.1 · Computing the effect in the treated

The effect in the treated needs only partial exchangeability, Ya=0 ⫫ A | L: we only need to know what would have happened to the treated without treatment. The effect in the untreated needs Ya=1 ⫫ A | L. Under these, E[Ya | A = a′] can be computed by standardization, Σl E[Y | A = a, L = l] Pr[L = l | A = a′] (standardize to the covariate distribution of group a′), or by IP weighting with weights Pr[A = a′ | L] / f(A | L), taking the weighted mean of Y among those with A = a.

  1. Finding who benefits most.
In Table 4.1 the population effect is null, but men benefit and women are harmed. With no other information, a physician who knew this would treat the next patient only if he is a man. The case of multiplicative-only modification is subtler. In the second example (risks 0.8→0.9 and 0.1→0.2), treatment changes risk by 0.1 in both strata, so treating everyone has the same absolute impact in each stratum, even though the risk ratios differ. (Note: in that example treatment raises risk; read “benefit” as “change”.) For deciding whom to treat, additive effect modification is the relevant scale. Multiplicative modification without additive modification is of little use for decisions.

The multiplicative modification there was nearly forced. With an untreated risk of 0.8 in V = 1, the risk ratio cannot exceed 1/0.8 = 1.25, so it could never match the ratio of 2 in V = 0. More generally, if treatment has an effect in at least one stratum and the untreated risk Pr[Ya=0 = 1 | V = v] varies across strata, there must be effect modification on at least one scale. The book’s advice is to report both absolute counterfactual risks, Pr[Ya=1 = 1 | V = v] and Pr[Ya=0 = 1 | V = v], in every stratum rather than only a ratio or difference.

  1. Understanding mechanisms.
A higher HIV risk in uncircumcised men gives clues about how infection happens. Effect modification can also be a first step toward studying interaction between two treatments (Chapter 5).

4.4 Stratification as a form of adjustment

Two goals now sit side by side. To get the population effect, adjust for the confounders L (by standardization or IP weighting). To find effect modification, also stratify by V. In Section 4.2 we did both: stratified by nationality and standardized by L within each nationality.

In practice, stratification is also widely used as a way to adjust for L. Many epidemiologists treat “stratify” and “adjust” as synonyms. Asked to adjust Table 2.2 for L, they split it into the L = 0 and L = 1 subtables and report a risk ratio in each: Pr[Y = 1 | A = 1, L = l] / Pr[Y = 1 | A = 0, L = l] = 1 for both l (1/4 vs 1/4 and 2/3 vs 2/3). Under conditional exchangeability these are causal effects within each stratum, because Pr[Y = 1 | A = a, L = l] = Pr[Ya = 1 | L = l].

Conditional vs marginal effect

A conditional effect measure refers to a subset defined by covariate values (e.g., L = 1). A marginal effect measure refers to the whole population. Stratification gives conditional effects; standardization and IP weighting can give marginal ones.

Here the two conditional ratios and the marginal ratio are all 1, but only because L does not modify the effect. In general stratification gives one number per stratum, and none of them is the population effect. That is why Chapter 2 used standardization and IP weighting.

Stratification has another drawback. It forces you to stratify on every variable needed for exchangeability. In the 40-person population you would need four strata: Romans with L = 1, Greeks with L = 1, Romans with L = 0, Greeks with L = 0. You cannot get the effect in Romans just by looking at the V = 0 stratum, because nationality alone does not give exchangeability. So stratification makes you examine effect modification by all confounders whether or not you care. Stratifying on V and then standardizing or IP weighting for L keeps the two jobs separate.

Further problems: some measures, like the odds ratio, are noncollapsible (Fine Point 4.3); and with time-varying treatments, stratifying on time-varying confounders affected by earlier treatment introduces bias (Part III; Chapter 20). Stratification also needs positivity: no effect can be computed in a stratum with only treated or only untreated people.

Restriction

Computing the effect in only some strata of L. Stratification is restriction applied to a complete set of non-overlapping strata. Restriction is the remedy when positivity fails in some strata: you limit inference to the strata where it holds.

4.5 Matching as another form of adjustment

Matching builds a subset of the population in which L has the same distribution in the treated and the untreated. Using Table 3.1 (where L suffices for exchangeability): for each untreated person, randomly pick a treated person with the same L. One possible set of 7 matched pairs (untreated–treated):

LUntreated (Y)Matched treated (Y)
0Rheia (0)Hestia (0)
0Kronos (1)Poseidon (0)
0Demeter (0)Hera (0)
0Hades (0)Zeus (1)
1Artemis (1)Ares (1)
1Apollo (1)Aphrodite (1)
1Leto (0)Hermes (0)
Risk 3/7Risk 3/7
The book’s 7 matched pairs from Table 3.1, with outcomes looked up from that table (arranged here as a table for clarity). L is the matching factor; 3 of 7 pairs have L = 1 in both groups.

In the matched population the proportion with L = 1 is 3/7 in both groups by design. Under conditional exchangeability given L, the matched treated and untreated are unconditionally exchangeable, so you can compare them directly: risk 3/7 vs 3/7, risk ratio 1. Matching also guarantees positivity, because strata with only treated or only untreated people drop out.

All untreated people were kept and only a sample of the treated, so this is the effect in the untreated. Usually you take the smaller group and find matches for it in the larger group; that smaller group defines whose effect you are estimating. With fewer treated than untreated, you get the effect in the treated. Matching can be one-to-many as well as one-to-one. With several matching variables, matches are made within combinations of all of them; as the number grows, exact matches become scarce and approximate matching methods are needed (Stuart 2010). You can also target any chosen distribution of L, by individual matching or by frequency matching (e.g., sample treated people so that 70% have L = 1, then do the same for the untreated).

A margin note restricts this to cohort studies. In case-control studies one matches cases to controls, which does not make treated and untreated exchangeable, and the matching factors must still be adjusted for by stratification.

Because the matched population is a subset, its mix of effect modifiers usually differs from the original population’s. That is the subject of the next section.

4.6 Effect modification and adjustment methods

Standardization, IP weighting, stratification/restriction and matching all estimate average causal effects, but of different groups. Standardization and IP weighting can estimate marginal or conditional effects. Stratification/restriction and matching estimate only conditional effects in particular subsets. All four need exchangeability and positivity, but only in the subset whose effect is targeted. For the effect in L = l, the conditions need to hold only there; for the population effect, they must hold at every level of L.

Without effect modification all four give the same number. For mortality in Table 2.2 they all gave a risk ratio of 1. To see what happens with modification, take a new outcome: high blood pressure Z, in Table 4.3. Assume Za ⫫ A | L and positivity.

LAZ
Rheia000
Kronos001
Demeter000
Hades000
Hestia010
Poseidon010
Hera011
Zeus011
Artemis101
Apollo101
Leto100
Ares111
Athena111
Hephaestus111
Aphrodite110
Polyphemus110
Persephone110
Hermes110
Hebe110
Dionysus110
Table 4.3 Same people, L and A as Table 3.1, with outcome Z = 1 for high blood pressure.

The book leaves the calculations to the reader. Stratum-specific risks from the table:

Then each method:

MethodPopulation targetedCalculationRisk ratio
Standardization / IP weightingWhole population (60% have L = 1)Pr[Za=1 = 1] = 0.4×0.5 + 0.6×1/3 = 0.4; Pr[Za=0 = 1] = 0.4×0.25 + 0.6×2/3 = 0.50.8
StratificationL = 00.5 / 0.252.0
StratificationL = 1(1/3) / (2/3)0.5
Matching (pairs above)The untreated (3/7 have L = 1)Matched treated: Hestia 0, Poseidon 0, Hera 1, Zeus 1, Ares 1, Aphrodite 0, Hermes 0 = 3/7; untreated: 3/71.0
Four correct causal risk ratios for the effect of transplant on high blood pressure in Table 4.3, each for a different population (calculations worked out here; the book states only the results).

Four answers, 0.8, 2.0, 0.5 and 1.0, and all are correct. The explanation is qualitative effect modification by L: transplant doubles the risk of high blood pressure in non-critical patients and halves it in critical ones. The population effect is protective (0.8) because critical patients make up 60% of the population and have a much higher untreated risk. The effect in the untreated is null (1.0) because the untreated have a larger share of non-critical patients (4 of 7) than the population does (8 of 20).

Technical Point 4.3 · When is the marginal risk ratio below 1?

The marginal risk ratio is a weighted average of the stratum-specific ratios, with weights w(l) = Pr[Ya=0 = 1 | L = l] Pr[L = l] / Pr[Ya=0 = 1]. With stratum ratios of 0.5 (L = 1) and 2.0 (L = 0), the marginal ratio is below 1 exactly when

Pr[Ya=0 = 1 | L = 1] / Pr[Ya=0 = 1 | L = 0] > 2 × Pr[L = 0] / Pr[L = 1]

For Table 4.3: (2/3)/(1/4) ≈ 2.67, which exceeds 2 × 0.4/0.6 ≈ 1.33. So the population ratio is below 1.

The point is that you must state the population, or subset, to which an effect estimate refers. Chapter 3 argued that a well-defined intervention is a prerequisite for causal inference; this chapter adds a well-characterized target population. Trials have both built in (defined arms, explicit eligibility). Observational studies must state both. An investigator who used IP weighting (population effect) and one who used matching (effect in the untreated) do not need to argue over which method is better. They asked different questions. The second could have used IP weighting to get the effect in the untreated too (Technical Point 4.1).

A margin note anticipates Part II: an ordinary regression model is a form of stratification. The treatment coefficient estimates the association within levels of all the other covariates in the model, a conditional effect.

Finally, stratification gives average effects in subsets, never individual effects. Individual effects are identified only under extreme assumptions (Fine Points 2.1 and 3.2).

Technical Point 4.2 · Pooling stratum-specific estimates

With real samples, stratum estimates are noisy. Pooling methods (Woolf, Mantel–Haenszel, maximum likelihood, or a regression with all product terms among the L but none between A and L) combine them into one more precise estimate. This makes sense only if the strata share a common effect. For Z in Table 4.3, the Mantel–Haenszel risk ratio is 0.88. If the true stratum ratios were both 0.9 and 0.5 and 2.0 were sampling noise, 0.88 would be a good estimate. If the true ratios really are 0.5 and 2.0, pooling them makes little sense and 0.88 is hard to interpret. A pooled estimate is still a conditional effect, not the marginal one (which here is 0.8). In practice it is hard to tell sampling noise from real modification, and finer strata make it harder. The same issue arises in meta-analyses of heterogeneous studies.

Fine Point 4.3 · Collapsibility and the odds ratio

The population risk ratio is a weighted average of the stratum risk ratios, so it always lies between them. This property is called collapsibility; the risk ratio and risk difference have it. The odds ratio does not. Table 4.4 is a randomized study of moving to a high-altitude home on Mount Olympus (A) and later depression (Y), with sex V.

VAY
Rheia100
Demeter100
Hestia100
Hera100
Artemis101
Leto110
Athena111
Aphrodite111
Persephone110
Hebe111
Kronos000
Hades000
Poseidon001
Zeus001
Apollo000
Ares011
Hephaestus011
Polyphemus011
Hermes010
Dionysus011
Table 4.4 Sex V (1 = woman), randomized move to high altitude A, and subsequent depression Y, in 20 people not depressed at baseline.

Women: risk 3/5 = 0.6 if moved vs 1/5 = 0.2 if not; risk ratio 3, odds ratio (0.6/0.4)/(0.2/0.8) = 6. Men: 4/5 = 0.8 vs 2/5 = 0.4; risk ratio 2, odds ratio (0.8/0.2)/(0.4/0.6) = 6. Everyone: 7/10 vs 3/10; risk ratio 2.3 (between 2 and 3), but odds ratio (0.7/0.3)/(0.3/0.7) ≈ 5.4, smaller than the odds ratio of 6 in each half. The effect looks bigger in each half than in the whole. This happens when V is a risk factor for Y and independent of A. The odds ratio collapses under the sharp null, and is approximately collapsible (and close to the risk ratio) when the outcome is rare (under about 10%) in every stratum. A practical consequence: a change in the odds ratio after adjusting for a variable does not prove confounding. Here it changed by about 10% (1 − 6/5.4) even though treatment was randomized.

Key takeaways

  • There is no single causal effect of a treatment. The average effect depends on the population’s mix of effect modifiers.
  • V is an effect modifier if the average effect differs across levels of V. Whether it does depends on the scale. Qualitative modification (opposite directions) shows up on every scale.
  • A null population effect can hide real effects in opposite directions in subgroups (Table 4.1: harmful in women, protective in men).
  • To find modification by V in non-randomized data, stratify by V and still adjust for L within each stratum.
  • An effect modifier may be a surrogate (nationality) for the real causal modifier (quality of surgery).
  • For deciding whom to treat, additive modification is what matters. Report absolute risks under each treatment in each subgroup.
  • Effects transport to other populations only if modifiers, treatment versions and interference patterns are similar, and this cannot be verified from the data.
  • Standardization/IP weighting (any population), stratification (each stratum of L) and matching (usually the treated or untreated) estimate effects for different groups. With modification, they give different correct answers (0.8, 2.0, 0.5, 1.0 in Table 4.3). Always name the target population.
  • The odds ratio is noncollapsible: the population odds ratio can lie outside the range of stratum odds ratios even with no confounding.

Chapter 5 · Interaction

Book pp. 57–70

In short

So far the book studied one treatment at a time. This chapter asks what happens when two treatments are applied together, and whether the effect of one depends on the level of the other. That is interaction. It is defined through joint counterfactuals Ya,e, which makes it different from effect modification: an effect modifier is a variable we merely stratify on, whereas both members of an interacting pair are things we could intervene on. To identify interaction you need exchangeability, positivity and consistency for both treatments. The chapter then gives two lenses on interaction: counterfactual response types (16 of them for two binary treatments) and sufficient-component causes (“causal pies”), which describe mechanisms. The counterfactual lens is the one the rest of the book uses.

5.1 Interaction requires a joint intervention

Return to the heart transplant study and add a second treatment. Before the transplant decision, each patient is randomly assigned to a multivitamin complex (E = 1) or no vitamins (E = 0). Transplant is still A. Every patient now falls into one of four treatment groups, and has four potential outcomes, one for each combination:

Ya=1,e=1, Ya=1,e=0, Ya=0,e=1, Ya=0,e=0

Joint intervention

An intervention that sets two or more treatments at once. Ya,e is the outcome a person would have had if we had set A to a and E to e.

A margin note links this to earlier chapters. The single-treatment counterfactual Ya is just the joint counterfactual with E left at whatever value it actually took: Ya = Ya,E. Consistency is a special case of this substitution: the observed outcome is Y = YA = YA,E.

Interaction (counterfactual definition)

Treatments A and E interact if the causal effect of A on Y after a joint intervention that sets E = 1 differs from the causal effect of A after a joint intervention that sets E = 0.

On the risk difference scale, there is interaction on the additive scale if

Pr[Ya=1,e=1 = 1] − Pr[Ya=0,e=1 = 1] ≠ Pr[Ya=1,e=0 = 1] − Pr[Ya=0,e=0 = 1]

The book’s example: the risk difference for transplant is 0.1 if everybody takes vitamins and 0.2 if nobody does. Those differ, so transplant and vitamins interact additively. A bit of algebra (move terms across the inequality) shows the same statement can be written with the roles swapped:

Pr[Ya=1,e=1 = 1] − Pr[Ya=1,e=0 = 1] ≠ Pr[Ya=0,e=1 = 1] − Pr[Ya=0,e=0 = 1]

So “the effect of vitamins differs by transplant status” holds exactly when “the effect of transplant differs by vitamin status” holds. The two treatments have equal standing.

To make the numbers concrete (these four risks are illustrative, chosen to match the book’s 0.1 and 0.2): suppose Pr[Y1,1 = 1] = 0.3, Pr[Y0,1 = 1] = 0.2, Pr[Y1,0 = 1] = 0.4 and Pr[Y0,0 = 1] = 0.2. Transplant’s effect is 0.3 − 0.2 = 0.1 with vitamins and 0.4 − 0.2 = 0.2 without. Vitamins’ effect is 0.3 − 0.4 = −0.1 among the transplanted and 0.2 − 0.2 = 0 among the non-transplanted. Both comparisons show a gap of 0.1, as the algebra promises.

Interaction versus effect modification.

Chapter 4 defined an effect modifier V as a variable across whose levels the effect of A varies. Sex modified the transplant effect in Table 4.1, but the book never asked about the effect of sex. Effect modification uses counterfactuals Ya only; V is something we condition on and never imagine setting. Interaction uses Ya,e: both A and E are treatments we could, at least hypothetically, assign.

Technical Point 5.1 · Additive and multiplicative scales

No additive interaction can be rewritten as: the joint effect equals the sum of the separate effects,

Pr[Y1,1=1] − Pr[Y0,0=1] = (Pr[Y1,0=1] − Pr[Y0,0=1]) + (Pr[Y0,1=1] − Pr[Y0,0=1])

If the left side is larger, the interaction is superadditive; if smaller, subadditive. On the risk ratio scale, there is multiplicative interaction when

Pr[Y1,1=1] / Pr[Y0,0=1] ≠ (Pr[Y1,0=1] / Pr[Y0,0=1]) × (Pr[Y0,1=1] / Pr[Y0,0=1])

with “greater than” meaning supermultiplicative and “less than” submultiplicative. In the illustrative numbers above, the joint risk difference is 0.1 and the sum of separate effects is 0.2 + 0 = 0.2 (subadditive); the joint risk ratio is 1.5 and the product of separate ratios is 2 × 1 = 2 (submultiplicative). The presence and direction of interaction can depend on the scale you pick.

5.2 Identifying interaction

Interaction is a statement about the joint effect of A and E, so identifying it requires exchangeability, positivity and consistency for both treatments.

When E is randomized. If vitamins were assigned unconditionally at random, the group that got E = 1 is exchangeable with everyone. The risk if everyone got transplant and vitamins equals the risk if those who actually got vitamins were all transplanted: Pr[Ya=1,e=1 = 1] = Pr[Ya=1 = 1 | E = 1]. Substituting, the interaction condition becomes

Pr[Ya=1=1 | E=1] − Pr[Ya=0=1 | E=1] ≠ Pr[Ya=1=1 | E=0] − Pr[Ya=0=1 | E=0]

which is exactly the definition of additive effect modification of A by E. With E randomized, interaction and effect modification coincide, and the stratification methods of Chapter 4 apply directly.

When E is not randomized. You still need the four marginal risks Pr[Ya,e = 1]. Under the usual identifying assumptions you get them by standardization or IP weighting on measured covariates, now adjusting for confounders of both treatments. The cleanest way to think about it: treat the pair as one combined treatment AE with four levels (11, 01, 10, 00). Then nothing is new. It is the single-treatment problem from earlier chapters with a longer list of treatment values and more counterfactuals.

When you trust exchangeability only for A. For example, A was randomized and you look at its effect within subgroups of an unrandomized E. You cannot claim interaction, but you can still report effect modification by E, because computing the effect of A within strata of E requires no assumptions about E. This is why Chapter 4 used the letter V: nationality modified the transplant effect, but nobody claimed to estimate the effect of nationality. As argued in Section 4.2, nationality is plausibly a surrogate modifier, a proxy for something that does act on Y. In the book’s phrase, “no action, no interaction”. So you can have effect modification without interaction. The reverse (interaction without effect modification) is logically possible but rare, since it needs exact cancellations.

The sufficient condition, then, is exchangeability, positivity and consistency for the joint treatment (A, E) at all four values. Part III shows it is not necessary when the two treatments happen at different times. For the rest of Part I (outside this chapter) the book returns to a single treatment.

5.3 Counterfactual response types and interaction

Everything so far works for population averages and does not need deterministic counterfactuals. This section does assume deterministic counterfactuals and binary treatments and outcome.

With one binary treatment, each person has one of four response types (also called response patterns). In Table 4.1 (the 20 Greek gods): the doomed die whatever happens (Artemis, Athena, Persephone, Ares); the immune survive whatever happens (Demeter, Hestia, Hera, Hades); the helped die only if untreated (Hebe, Kronos, Poseidon, Apollo, Hermes, Dionysus); the hurt die only if treated (Rheia, Leto, Aphrodite, Zeus, Hephaestus, Polyphemus).

TypeYa=0Ya=1
Doomed11
Helped10
Hurt01
Immune00
Table 5.1 The four response types for one binary treatment and a binary outcome.

With two binary treatments, each person has four counterfactual outcomes, each 0 or 1, giving 24 = 16 response types (Table 5.2, following Miettinen 1982).

TypeYa,e for each (a, e)
1, 10, 11, 00, 0
11111
21110
31101
41100
51011
61010
71001
81000
90111
100110
110101
120100
130011
140010
150001
160000
Table 5.2 The 16 response types for two binary treatments A (transplant) and E (vitamins) and a binary outcome (death). Each row lists the four counterfactual outcomes under (a, e) = (1,1), (0,1), (1,0), (0,0).

Reading the table:

For types 1, 4, 6, 11, 13 and 16, the effect of A is the same whatever E is, and vice versa. A population made only of these six types has no additive interaction. Additive interaction therefore requires some people whose outcome under A = a cannot be predicted without knowing E. Greenland and Poole (1988) grouped these into three classes that do not depend on how you code 0 and 1:

  1. people who develop the outcome under only one of the four combinations (types 8, 12, 14, 15);
  2. people who develop it under two combinations where each treatment’s effect flips with the other’s level (types 7 and 10);
  3. people who develop it under three of the four combinations (types 2, 3, 5, 9).

No additive interaction means either nobody belongs to these classes, or their deviations cancel exactly, for example equal shares of types 7 and 10, or of types 8 and 12.

Technical Point 5.2 · Monotonicity

With one treatment, only the “helped” have Ya=0 > Ya=1. If nobody is helped (the treatment can never prevent the outcome), every person’s outcome is non-decreasing in a, and we call the effect monotonic. For two treatments, the effects are monotonic if every Ya,e is non-decreasing in both a and e, which rules out anyone with (Y1,1=0, Y0,1=1), (Y1,1=0, Y1,0=1), (Y1,0=0, Y0,0=1) or (Y0,1=0, Y0,0=1).

Fine Point 5.1 · Testing for synergism from data

Sometimes the question is whether anyone develops the outcome only when given both treatments (Y1,1 = 1 and Y0,1 = Y1,0 = 0, i.e. types 7 and 8). VanderWeele and Robins showed a sufficient condition you can check in a randomized experiment:

Pr[Y1,1=1] − Pr[Y0,1=1] > Pr[Y1,0=1]

It is strong, so it misses many cases where such people exist. If you are willing to assume monotonicity, a weaker condition suffices: superadditive interaction,

Pr[Y1,1=1] − Pr[Y0,1=1] > Pr[Y1,0=1] − Pr[Y0,0=1]

implies that type 8 people exist (monotonicity rules out type 7). In genetics, type 8 is called compositional epistasis.

5.4 Sufficient causes

Response types say what happens under each treatment; sufficient causes try to say how. Go back to a single treatment. Some treated people die and some do not, so transplant alone is not enough to cause death. As an oversimplified story, suppose transplant kills only people allergic to anaesthesia. Call the smallest set of background factors that, together with A = 1, inevitably produces death U1. Then {A = 1, U1 = 1} is a minimal sufficient cause.

Likewise, some untreated people die. Suppose no transplant kills only people with ejection fraction under 20%; call that background set U2, so {A = 0, U2 = 1} is another sufficient cause. Finally, some people with neither U1 nor U2 die regardless (the doomed), say because they have pancreatic cancer at baseline; call that U0, a sufficient cause on its own. By definition, background factors U cannot be intervened on and are not affected by A.

Three circles (causal pies). First circle split in half: A=1 on top, U1=1 below. Second circle split in half: A=0 on top, U2=1 below. Third circle whole: U0=1.

Figure 5.1 The three sufficient causes of death with one treatment: transplant plus U1, no transplant plus U2, and U0 alone. Each pie is a sufficient cause; its slices are the component causes.

The pies make one consequence of effect modification visible: the size of a causal effect depends on how common the background factors are. Take two populations identical in U2 and U0, but with 1% versus 10% of people having U1 = 1 (anaesthesia allergy). Randomize half of each to transplant. The harmful effect of transplant will be larger in the second population, because the pie {A = 1, U1 = 1} can be completed ten times as often there.

With two treatments there are more ways to die. Greenland and Poole (1988) listed nine:

  1. by A (E irrelevant)
  2. by absence of A (E irrelevant)
  3. by E (A irrelevant)
  4. by absence of E (A irrelevant)
  5. by both A and E
  6. by A and absence of E
  7. by E and absence of A
  8. by absence of both
  9. by other mechanisms (neither matters)

Each comes with its own background set U1, …, U8 or U0.

Nine causal pies in a 3 by 3 grid. Row 1: U0=1 alone; A=1 with U1=1; A=0 with U2=1. Row 2: E=1 with U3=1; E=0 with U4=1; A=1, E=1 and U5=1. Row 3: A=1, E=0, U6=1; A=0, E=1, U7=1; A=0, E=0, U8=1.

Figure 5.2 The nine possible sufficient-component causes for a binary outcome with two binary treatments A and E. Pies with both an A slice and an E slice (bottom-right four) are where the treatments act together.

Not all nine need exist. If vitamins never kill anyone, the three pies containing E = 1 are absent. If they did exist, some people (say those with U3 = 1) would be killed by vitamins, i.e. their death would be prevented by withholding vitamins. This picture is often called “the causal pies”.

5.5 Sufficient cause interaction

In everyday speech, “interaction” suggests the two treatments work together through some mechanism. The counterfactual definition from 5.1 needs no such mechanism; it is just a contrast of counterfactual risks, which an ideal randomized experiment on both treatments can identify. This section gives a second, more mechanistic concept.

Sufficient cause interaction

A and E have a sufficient cause interaction in a population if they appear together in some sufficient cause (some pie) that is completed for at least one person.

For example, if some people have U5 = 1, they die when given both vitamins and transplant but not under either alone. So if anyone has Y1,1 = 1 and Y0,1 = Y1,0 = 0, a sufficient cause interaction is present.

Although this concept talks about mechanisms, you do not always need to know them to detect it. If the inequalities of Fine Point 5.1 hold in data, synergism exists, without any thought about the biology. This follows from the correspondence between response types and pies (Fine Point 5.2). Since the inequality is sufficient but not necessary, synergism can exist even when the test fails.

Fine Point 5.2 · From response types to pies and back

For one treatment the mapping is:

TypeYa=0Ya=1Component causes
Doomed11U0 = 1, or {U1 = 1 and U2 = 1}
Helped10U0 = 0 and U1 = 0 and U2 = 1
Hurt01U0 = 0 and U1 = 1 and U2 = 0
Immune00U0 = 0 and U1 = 0 and U2 = 0
Mapping between response types and sufficient-component causes for one treatment (table inside Fine Point 5.2).

Each combination of background factors gives exactly one response type, but one type can arise from several combinations. Pies also give a mechanistic reading of exchangeability: those who would die if treated are the doomed plus the hurt (U0 = 1 or U1 = 1); those who would die if untreated are the doomed plus the helped (U0 = 1 or U2 = 1). Exchangeability holds when Pr[U0=1 or U1=1 | A=1] = Pr[U0=1 or U1=1 | A=0] and the same for U0 or U2.

Fine Point 5.3 · “Biologic” interaction

Epidemiologists often call sufficient cause interaction “biologic interaction”. The label can mislead. VanderWeele and Robins’ example: A and E are deleterious mutations in the two alleles of a gene for an essential protein. Babies with both mutations lack the protein and die; one or zero mutations is fine. There is synergism (a pie containing A = 1 and E = 1), yet the two alleles do not physically act on each other.

5.6 Counterfactuals or sufficient-component causes?

The two frameworks answer different questions. The sufficient-component-cause model starts from an effect and asks which combinations of events could have caused it (“how does it happen?”). It goes back to Mackie’s (1965) INUS condition: an Insufficient but Necessary part of a condition that is itself Unnecessary but Sufficient. The counterfactual model starts from one cause or intervention and asks what would happen if it were set differently (“what happens?”), without needing the mechanism.

For this book’s goal, estimating average effects of hypothetical interventions, the counterfactual framework is the natural one. Pies are useful for teaching: they show why effect sizes depend on the distribution of background factors, and how effect modification, interaction and synergism relate. Their use in real analysis is limited. In classical form they are deterministic, depend on how the outcome is coded, and only handle binary variables, which rules out continuous factors and limits you to a few binary ones. Extensions to stochastic settings and three-level treatments exist, but the data needed to make such fine distinctions are rarely available. Causal diagrams and decision theory, by contrast, are essentially equivalent to the counterfactual framework (next chapter).

Fine Point 5.4 · Attributable fractions don’t add up

Suppose the excess fraction for A is 75% (75% of cases would not occur had everyone been untreated) and for E is 50%. A joint intervention cannot prevent 125% of cases. The pies explain why: if Zeus has only U5 = 1 and received both treatments, removing either one saves him, so he is counted in both fractions. When two factors share a pie, asking what fraction is “due to” each one separately makes little sense. Phenylketonuria is the classic case: removing certain foods prevents all cases, and so would replacing the susceptibility gene, so the condition is both 100% environmental and 100% genetic.

Technical Point 5.3 · Monotonicity rules out pies

If smoking (A = 1) never prevents heart disease and inactivity (E = 1) never prevents it, then no pie can contain A = 0 or E = 0. If one did, someone (say with U2 = 1) would get the disease when unexposed, meaning exposure would prevent it for them. Figure 5.3 crosses out the pies that cannot exist.

The nine causal pies of Figure 5.2, with red X marks over the five pies containing A=0 or E=0: A=0 with U2; E=0 with U4; A=1, E=0, U6; A=0, E=1, U7; A=0, E=0, U8. Remaining: U0, A=1 with U1, E=1 with U3, and A=1, E=1, U5.

Figure 5.3 Under monotonic effects of A and E, the five sufficient causes that contain A = 0 or E = 0 cannot exist (crossed out). Only U0, {A=1, U1}, {E=1, U3} and {A=1, E=1, U5} remain.

Key takeaways

  • Interaction between A and E means the effect of one depends on the level at which the other is set; it is defined with joint counterfactuals Ya,e, and the two treatments are symmetric.
  • Effect modification only stratifies on V; interaction treats E as a second intervention. They coincide when E is randomized.
  • Identifying interaction needs exchangeability, positivity and consistency for both treatments. Treat (A, E) as one four-level treatment and use standardization or IP weighting.
  • Whether interaction is present, and its direction, depends on the scale (additive vs multiplicative).
  • With two binary treatments there are 16 response types; additive interaction requires people in types other than 1, 4, 6, 11, 13, 16 (unless effects cancel).
  • Sufficient-component causes (pies) describe mechanisms; synergism means both treatments sit in one pie. It can sometimes be detected from counterfactual risks alone.
  • Pies are good for intuition; counterfactuals are the working framework for estimation.

Chapter 6 · Graphical representation of causal effects

Book pp. 71–84

In short

Real problems involve many variables, so you need a compact way to write down what you know and assume about how they cause each other. This chapter introduces causal directed acyclic graphs (DAGs). An arrow means a direct causal effect for at least one person; a missing arrow is a strong claim of no effect for anyone. A DAG also tells you which variables should be associated in data. Association flows along paths: through causal chains, through common causes, and through common effects only if you condition on them (or their descendants). These rules, called d-separation, explain why adjusting for a common cause removes bias while conditioning on a common effect creates it. Bias then splits into structural types: confounding (common causes), selection bias (conditioning on common effects), and measurement bias. DAGs are good at showing bias and weak at showing effect modification.

6.1 Causal diagrams

The book’s advice for this chapter: draw your assumptions before your conclusions. The modern theory of causal diagrams came from computer science and artificial intelligence (Pearl 2009; Spirtes, Glymour and Scheines 2000).

L arrow to A, A arrow to Y, and a curved arrow from L to Y.

Figure 6.1 Disease severity L affects both heart transplant A and death Y; A affects Y. A conditionally randomized experiment, or an observational study where L is the only common cause.

Figure 6.1 has three nodes (random variables L, A, Y) and three edges. Time flows left to right, so L precedes A, which precedes Y. As before, L is disease severity, A heart transplant, Y death.

Directed acyclic graph (DAG)

Directed: each edge has a direction, so L → A means L may cause A and not the reverse. Acyclic: no variable can cause itself, directly or through other variables.

Causal Markov assumption

Conditional on its direct causes, any variable is independent of every variable it does not cause. One consequence: every common cause of any two variables on the graph must also be on the graph, even if unmeasured.

So if transplant is assigned at random with a probability that depends on severity, L is a common cause of A and Y and must appear (Figure 6.1). If everyone has the same probability of transplant, L is no longer a cause of A and can be left off (Figure 6.2).

A single arrow from A to Y.

Figure 6.2 A causes Y, and nothing else is shown. A marginally (unconditionally) randomized experiment.

Figure 6.1 can also describe an observational study, provided you are willing to assume the only parent of treatment is L and there are no other common causes of A and Y. Chapter 7 shows that drawing Figure 6.1 for an observational study is the graphical form of conditional exchangeability, Ya ⫫ A | L.

Technical Point 6.1 · Formal definition of a causal DAG

A DAG has nodes V = (V1, …, VM), directed edges and no directed cycles. The parents PAm of Vm are the nodes with arrows into it; descendants and ancestors follow arrows forward and backward. A distribution is Markov with respect to the DAG if each variable is independent of its non-descendants given its parents, equivalently if the density factorizes as

f(v) = ∏j=1M f(vj | paj)

A causal DAG adds: a missing arrow means no direct effect; all common causes of any pair are on the graph; and every variable causes its descendants. The causal Markov assumption is what links the causal graph to the data.

Many people find graphs more intuitive than counterfactual notation, but the two are tightly linked: every causal DAG stands for an underlying counterfactual model (Technical Points 6.2 and 6.3). Ordinary DAGs do not show the counterfactual variables themselves. Single World Intervention Graphs (SWIGs, Richardson and Robins 2013) put them on the graph; they arrive in Chapter 7.

Technical Points 6.2 and 6.3 · The counterfactual model behind a DAG

A DAG represents a nonparametric structural equation model (NPSEM): each variable is generated as Vm = fm(pam, εm) for unknown functions fm and unobserved errors εm. Only parents have direct effects. Counterfactuals for any intervention are built recursively; for example, setting V1 = v1 gives V3v1 = V3v1, V2v1. Robins (1986) introduced this as the FCISTG; Pearl showed how to draw it as a DAG. The model alone does not imply the causal Markov property; you need independence assumptions on the errors. Pearl’s NPSEM-IE assumes fully independent errors. Robins’ weaker FFRCISTG assumes only that the one-step-ahead counterfactuals are jointly independent. NPSEM-IE implies more independences: under Figure 6.2 it gives full exchangeability (Ya=0, Ya=1) ⫫ A, whereas FFRCISTG gives only Ya ⫫ A for each a. The book assumes FFRCISTG unless it says otherwise.

6.2 Causal diagrams and marginal independence

DAGs encode causal knowledge, but they also tell you which associations to expect. Three examples, each with no conditioning.

Path

A route between two variables along edges that visits no variable twice, regardless of arrow direction. A path is causal if all its arrows point the same way, otherwise noncausal.

  1. Cause and effect (Figure 6.2).
Aspirin A, randomly assigned, prevents heart disease Y. You expect A and Y to be associated, and in an ideal randomized trial causation, Pr[Ya=1=1] ≠ Pr[Ya=0=1], goes with association, Pr[Y=1 | A=1] ≠ Pr[Y=1 | A=0]. A helpful picture: paths are pipes through which association flows. Association is symmetric, so it flows either way along an arrow.

L arrow to A, and curved arrow from L to Y. No arrow between A and Y.

Figure 6.3 Smoking L causes both carrying a lighter A and lung cancer Y. Carrying a lighter has no effect on cancer. L is a common cause.

  1. Common cause (Figure 6.3).
Carrying a lighter has no effect on lung cancer, but smoking causes both. A naive investigator asks many people whether they carry a lighter and follows them for five years. Hera carries a lighter, so she is more likely to smoke, so her cancer risk is above average. Therefore Pr[Y=1 | A=1] ≠ Pr[Y=1 | A=0] even though there is no causal effect. Association flows from A to Y through the open path A ← L → Y. Concluding that lighters cause cancer would be a mistake.

Y arrow to L, and curved arrow from A to L. No arrow between A and Y.

Figure 6.4 A genetic haplotype A and smoking Y both cause heart disease L. The haplotype has no effect on smoking. L is a common effect (a collider).

  1. Common effect (Figure 6.4).
A haplotype A has no effect on becoming a smoker Y, and both cause heart disease L. On the path A → L ← Y, two arrowheads meet at L.

Collider

A variable on a path where two arrowheads collide (→ L ←). Colliders block the flow of association along that path.

An investigator genotypes many children and records who becomes a smoker. Learning that Apollo lacks the haplotype tells you nothing about whether he will smoke. That both cause heart disease is irrelevant. So A ⫫ Y: the only path is blocked by the collider L.

Summary: two variables are marginally associated if one causes the other or they share a common cause. Otherwise they are marginally independent.

6.3 Causal diagrams and conditional independence

Now revisit the three structures and ask what happens within levels of the third variable. A square box around a node means we condition on it (restrict or stratify by it).

A arrow to B (in a box), B arrow to Y.

Figure 6.5 Aspirin A lowers platelet aggregation B, which affects heart disease Y. The box around B means the analysis is restricted to one level of B.

Conditioning on a mediator (Figure 6.5). Suppose aspirin affects heart disease only by reducing platelet aggregation B. Restrict to people with low aggregation (B = 0). Their risk is already known to be low; since aspirin works only through B, knowing whether they took aspirin adds nothing. So within levels of B, Pr[Y=1 | A=1, B=b] = Pr[Y=1 | A=0, B=b], i.e. A ⫫ Y | B, even though A and Y are marginally associated. The box blocks A → B → Y.

A margin note: complete DAGs (every possible arrow present) imply no conditional independences at all, which is why people say the information is in the missing arrows.

L in a box, arrow from L to A, curved arrow from L to Y. No arrow from A to Y.

Figure 6.6 The lighter example with smoking L conditioned on (boxed).

Conditioning on a common cause (Figure 6.6). Restrict to non-smokers (L = 0). Now carrying a lighter does not predict cancer, because the only reason it did was its link to smoking. So A ⫫ Y | L: Pr[Y=1 | A=1, L=l] = Pr[Y=1 | A=0, L=l] for all l. The box blocks A ← L → Y. This is the graphical justification for stratifying on confounders to get exchangeability.

Y arrow to L (in a box), curved arrow from A to L. No arrow between A and Y.

Figure 6.7 The haplotype example with the common effect heart disease L conditioned on (boxed).

Conditioning on a collider (Figure 6.7). Now the investigators restrict to people with heart disease (L = 1). Among them, someone who lacks the haplotype more likely has the other cause, smoking. So among people with heart disease, smoking is more common in those without the haplotype: A and Y are inversely associated given L = 1. In the extreme, if A and Y were the only causes of L, absence of one would perfectly predict presence of the other among cases. Conditioning on a collider opens a path that was blocked. Two causes are independent, but they become associated once you stratify on their shared effect (more in Chapter 8).

Curved arrow from A to L, Y arrow to L, L arrow to C (in a box).

Figure 6.8 As Figure 6.7, plus a diuretic C prescribed because of heart disease. Conditioning on C, a descendant of the collider L, also opens the path A → L ← Y.

Conditioning on a descendant of a collider (Figure 6.8). Diuretic use C follows from a heart disease diagnosis. Restricting to diuretic users also makes A and Y associated. The path stays blocked only if you condition on neither L nor C.

So there are three structural reasons for an association: one variable causes the other; they share a common cause; or they share a common effect and you have conditioned on it (or on a descendant of it). The formal theory is d-separation (Pearl 1995). A fourth source, chance, shrinks as the sample grows; until Chapter 10 the book assumes an effectively infinite population so only structural associations matter.

Fine Point 6.1 · d-separation rules

  1. With no conditioning, a path is blocked if and only if two arrowheads collide at some variable on it. In Figure 6.1, L → A → Y is open; A → Y ← L is blocked by the collider Y.
  2. A path with a conditioned-on non-collider is blocked (Figure 6.5, conditioning on B).
  3. A conditioned-on collider does not block (Figure 6.7).
  4. A collider with a conditioned-on descendant does not block (Figure 6.8).

In one line: a path is blocked if and only if it contains a non-collider that has been conditioned on, or a collider that has not been conditioned on and has no conditioned-on descendants. Two variables are d-separated if every path between them is blocked; otherwise they are d-connected. In Figure 6.1, A and L are d-connected via L → A, even though the other path is blocked. In Figure 6.4, A and Y are d-separated.

Pearl (1988) proved that under the causal Markov assumption, d-separation implies statistical independence. The converse, that independence implies d-separation, is a separate assumption: faithfulness.

Fine Point 6.2 · Faithfulness

An arrow A → Y says A affects at least one person’s outcome, so you usually expect a non-null average effect and association. But recall Table 4.1: transplant raised risk in women and lowered it equally in men, so the average effect was exactly zero. Figure 6.2 is still the right DAG, yet the data show no association. The data are then unfaithful to the DAG. Exact cancellation is rare, so the book assumes faithfulness: lack of d-separation almost always means non-zero association.

L arrow to A, A arrow to Y, L arrow to S (in a box), A arrow to S.

Figure 6.9 A matched study: selection into the matched cohort S depends on both L and A, and the analysis is restricted to S = 1 (boxed).

Faithfulness can also fail by design. In a matched study (Section 4.5), selection S depends on L and A. d-separation says L and A should be associated given S = 1, via L → A and via L → S ← A. But matching makes L balanced across treatment groups. The association created through S is equal and opposite to the one through L → A, and they cancel. Matching produces unfaithfulness. Deterministic relations between variables can also break faithfulness.

6.4 Positivity and consistency in causal diagrams

Standardization and IP weighting can also be derived from graph theory (the “do-calculus”), so choosing counterfactual notation in Chapters 1–5 was only a choice of notation. In either notation, exchangeability, positivity and consistency are required. Exchangeability gets its graphical treatment in Chapters 7 and 8. Here: positivity and consistency.

In the book’s diagrams positivity is implicit and consistency is built in, because treatment nodes are only drawn for well-defined interventions. So treatment nodes have a special status. Some authors make this explicit with decision nodes (influence diagrams, Dawid 2000, 2002); the book does not, because it always states the intervention, but SWIGs in later chapters will mark treatment nodes differently. Earlier tools also marked this status: causal trees in Chapter 2 circled non-treatment branches, and the pies of Chapter 5 separated treatments from background factors U.

A DAG drawn as an NPSEM with independent errors treats every node alike, as if each could be intervened on. That can mislead. “Obesity” is fine as an outcome or covariate, but generally not as a treatment, for the reasons in Chapter 3.

Three nodes on the left: L, Z, U. Arrows from L, Z and U into A. Arrow from A to Y. Additional arrows from L to Y and from U to Y.

Figure 6.10 Weight loss A and mortality Y, with three causes of weight loss: caloric intake Z (acts on Y only through A), exercise L and genetic traits U (both also affect Y through other pathways).

Example: the effect of “weight loss” A on mortality Y. In Figure 6.10, weight can be lost through caloric restriction Z, exercise L or genetic changes U, and the last two also affect mortality by other routes. Losing weight by dieting, by exercising or by gene manipulation would give different mortality, so “the effect of weight loss” is not well defined. Even ignoring that, choosing the adjustment set would be very hard. Naming the intervention you mean is a prerequisite for a well-defined effect, for finding the right data, and for picking adjustment variables.

Fine Point 6.3 · Causal discovery

Can you learn the DAG from data instead of assuming it? Sometimes, if you assume faithfulness, and often not. A strong association between B and C fits B → C, C → B, a hidden common cause, conditioning on a common effect, or combinations. Knowing the time order only rules out one direction.

Z arrow to A, A arrow to Y, curved arrow from Z to Y. U1 points to Z and A. U3 points to A and Y. U2 points to Z and Y via curved arrows.

Figure 6.11 With time order Z, A, Y and no other knowledge, every arrow and every unmeasured common cause (U1, U2, U3) is possible.

Now suppose infinite data show all three pairs associated and the only conditional independence is Z ⫫ Y | A. Under faithfulness, the only compatible DAG is Z → A → Y, possibly with U1 as well. A direct Z → Y arrow, U2 or U3 would each break that independence. So A must cause Y, there is no unmeasured A–Y confounding, and E[Y | A=1] − E[Y | A=0] identifies the average causal effect. Finite samples complicate this (Technical Point 10.7).

6.5 A structural classification of bias

Systematic bias

The data are insufficient to identify the causal effect even with an infinite sample. Informally: any structural association between treatment and outcome that does not come from the effect of treatment on outcome in the population of interest. With systematic bias, no estimator is consistent.

For the whole population, there is (unconditional) bias when

Pr[Ya=1=1] − Pr[Ya=0=1] ≠ Pr[Y=1 | A=1] − Pr[Y=1 | A=0]

which happens when Ya ⫫ A fails. Without bias, the association measure consistently estimates the effect measure. Conditional bias is the same within some stratum L = l, which generally happens when Ya ⫫ A | L = l fails.

Lack of exchangeability causes bias under the null: treatment and outcome are associated even though treatment has no effect. In Table 3.1 the causal risk ratio was 1 and the associational risk ratio 1.26. Any structure causing bias under the null also causes bias under the alternative, but not the reverse (some selection can bias a non-null effect without creating one under the null).

With DAGs, lack of exchangeability comes from two structures:

  1. Common causes of treatment and outcome: confounding (Chapter 7).
  2. Conditioning on common effects: selection bias under the null (Chapter 8).

A third structural source is measurement bias (information bias) from mismeasured treatment, outcome or covariates (Chapter 9); some forms also produce bias under the null. Random variability is a separate, non-structural issue (Chapter 10). All three biases can affect randomized trials too: the idealized trials of earlier chapters had no loss to follow-up, perfect adherence and full blinding, which real trials rarely achieve.

6.6 The structure of effect modification

DAGs are good for spotting bias, less good for showing effect modification. Suppose transplant A is randomized, so Figure 6.2 applies and the associational risk difference is the causal one. The investigators suspect the effect depends on quality of care at the hospital, classify patients as high (V = 1) or normal (V = 0) quality, and find additive effect modification by V.

A arrow to Y, and a curved arrow from V to Y. No arrow from V to A.

Figure 6.12 Quality of care V affects death Y; randomized transplant A affects Y. No arrow from V to A.

Two caveats about Figure 6.12:

V arrow to N, N arrow to Y, curved arrow from V to Y, and A arrow to Y.

Figure 6.13 Therapy complications N partly mediate the effect of quality of care V on Y; N can also show up as an effect modifier.

Many effect modifiers have no effect on the outcome themselves; they are surrogate effect modifiers associated with the real (causal) modifier. Section 4.2 introduced the idea. Because the two are hard to tell apart in practice, “effect modification” covers both, and some prefer the neutral term “heterogeneity of causal effects”. The association between surrogate and causal modifier can come from any of the three structures:

V arrow to S, curved arrow from V to Y, and A arrow to Y. S has no arrow to Y.

Figure 6.14 Cost of treatment S is caused by quality of care V but has no effect on Y. S is a surrogate modifier through cause and effect.

Cause and effect (Figure 6.14). Cost S depends on quality V but does not affect mortality. Stratifying by cost will show modification even though quality is the true modifier. Reading “the effect is more beneficial when cost is higher” as a reason to raise prices without raising quality would be a mistake.

U arrow to P, U arrow to V, curved arrow from V to Y, A arrow to Y.

Figure 6.15 Place of residence U causes both quality of care V and passport nationality P. P is a surrogate modifier through a common cause.

Common cause (Figure 6.15). Residence U (Greece or Rome) affects both quality of care and passport nationality P, so P is associated with V and behaves like a modifier.

W arrow to S (in a box), V arrow to S, curved arrow from V to Y, A arrow to Y.

Figure 6.16 Bottled mineral water W and quality of care V both raise cost S; restricting to low-cost hospitals (boxed S) makes W a surrogate modifier through conditioning on a common effect.

Conditioning on a common effect (Figure 6.16). Using bottled water W raises hospital cost S but does not affect mortality. Among low-cost hospitals (S = 0), those using bottled water must save money elsewhere, likely on care that reduces mortality. So within low-cost hospitals, W is inversely associated with quality V and appears to modify the effect of A.

DAGs are also agnostic about interaction between two treatments. They can encode it only if augmented with nodes for sufficient-component causes, with deterministic arrows from treatments into them. Such graphs return in Chapter 8, because interaction affects the size and direction of bias from conditioning on common effects.

Key takeaways

  • A causal DAG encodes assumptions: an arrow means a direct effect for at least one person; a missing arrow means none for anyone. All common causes of any two nodes must be drawn, measured or not.
  • Association flows along open paths in either direction. Chains and forks are open unless you condition on the middle node; colliders are closed unless you condition on them or a descendant.
  • Conditioning on a common cause blocks a biasing path (the basis for adjustment). Conditioning on a common effect, or its descendant, opens one (the basis for selection bias).
  • d-separation plus the causal Markov assumption gives independence; faithfulness lets you read dependence. Faithfulness can fail through exact cancellation, as in matching.
  • Treatment nodes must be well-defined interventions; positivity is usually invisible on a DAG.
  • Structural bias comes from common causes (confounding), conditioning on common effects (selection bias) and measurement error (measurement bias).
  • DAGs do not show whether or how effects are modified; surrogate modifiers can arise from cause-and-effect, common-cause or collider structures.

Chapter 7 · Confounding

Book pp. 85–102

In short

Confounding is the bias created when treatment and outcome share a cause. On a causal diagram it shows up as an open backdoor path: a non-causal path from treatment to outcome that starts with an arrow pointing into the treatment. The chapter gives a graphical test, the backdoor criterion, for whether a set of measured covariates L is enough to make the treated and untreated exchangeable (Ya ⫫ A | L). It also shows why the old, association-based definition of a “confounder” can tell you to adjust for variables that create bias, introduces single-world intervention graphs (SWIGs) that put counterfactuals directly on the graph, and surveys the adjustment methods, including ones (difference-in-differences, front door, proximal inference) that work without conditional exchangeability.

7.1 The structure of confounding

The chapter opens with a pedestrian example. A researcher sees that when one pedestrian looks up, a second one tends to look up too. But a thunderous noise overhead makes both of them look up. The association between the two pedestrians is real, but part of it comes from the shared cause, so it cannot all be read as an effect of the first person’s looking up.

Figure 7.1 is the basic picture. Treatment A and outcome Y share a common cause L. There are two routes by which A and Y become associated: the causal path A → Y, and the path A ← L → Y, which carries association but no causation.

Causal DAG with L pointing to A, A pointing to Y, and a curved arrow from L to Y.

Figure 7.1 The simplest confounding structure: L causes both treatment A and outcome Y (same as Figure 6.1).

Backdoor path

A non-causal path between treatment and outcome that remains even after you delete every arrow going out of the treatment. In other words, a path that begins with an arrow pointing into A.

Confounding

The bias due to common causes of treatment and outcome, i.e. due to an open backdoor path between them. It is one form of lack of exchangeability between the treated and the untreated.

If L did not exist, the only path would be A → Y and the associational risk ratio Pr[Y = 1 | A = 1] / Pr[Y = 1 | A = 0] would equal the causal risk ratio Pr[Ya=1 = 1] / Pr[Ya=0 = 1]. With L present, the two ratios differ, and this does not go away with a larger sample: confounding is a structural bias, not a random one.

The book lists familiar named biases that all share this structure:

Causal DAG: U points to L and to Y; L points to A; A points to Y.

Figure 7.2 Confounding by indication with an unmeasured cause. U (e.g. atherosclerosis) causes the indication L (heart disease) and the outcome Y (stroke); L drives prescribing of A.

Causal DAG: U points to L and to A; L has a curved arrow to Y; A points to Y.

Figure 7.3 An unmeasured U causes both the treatment A and a measured variable L, which in turn causes the outcome Y.

A margin note makes a useful point about language. Yule called confounded associations “fictitious” and Pearson called them “spurious”. They are neither. They are real associations; they simply should not be interpreted as treatment effects. Some authors also replace an unmeasured U and its two arrows with a bidirected edge between the variables it causes.

For the rest of the chapter, the book assumes positivity and consistency, perfect measurement, no selection (no boxed nodes), and no random variability. Those other issues come in Chapters 8–10.

7.2 Confounding and exchangeability

This section links the graph-based idea of confounding to the counterfactual idea of exchangeability from earlier chapters.

If marginal exchangeability Ya ⫫ A holds (as in a marginally randomized trial), the average causal effect is just the difference of observed means, E[Y | A = 1] − E[Y | A = 0]. If only conditional exchangeability Ya ⫫ A | L holds (as in a conditionally randomized trial), the effect is still identified, but you must adjust for L by standardization or IP weighting:

E[Ya=1] − E[Ya=0] = Σl E[Y | L = l, A = 1] Pr[L = l] − Σl E[Y | L = l, A = 0] Pr[L = l]

Stratum-specific effects E[Ya=1 | L = l] − E[Ya=0 | L = l] are then identified by stratification.

The practical question is: is there a set of measured covariates L for which conditional exchangeability holds? Thinking directly about independence of counterfactuals is hard. If you know the causal DAG, though, you can answer it with a graphical rule.

Backdoor criterion

(Pearl 1995). A set of covariates L satisfies the backdoor criterion if (1) conditioning on L blocks every backdoor path between A and Y, and (2) L contains no descendants of A.

Under faithfulness (and the causal model the book uses throughout), Ya ⫫ A | L holds if and only if L satisfies the backdoor criterion. So by checking subsets of measured non-descendants of treatment against the DAG, you can find out which ones, if any, give you exchangeability. Efficient algorithms exist so that you do not need to try every subset.

Technical Point 7.1 · Does exchangeability imply the backdoor criterion?

The direction “backdoor criterion ⇒ conditional exchangeability” always holds, even without faithfulness. The reverse direction holds under faithfulness in Robins’ FFRCISTG model. Under Pearl’s NPSEM-IE model, which also assumes cross-world independences between counterfactuals, exchangeability can hold without the backdoor criterion (example: DAG with A → L and A → Y). Cross-world independences can never be tested even by an experiment, which is why the book uses the FFRCISTG model.

There are two situations in which the backdoor criterion is met:

  1. No common causes of treatment and outcome (e.g. Figure 6.2, a marginally randomized trial). There are no backdoor paths; the empty set satisfies the criterion. This is no confounding. Marginal exchangeability is equivalent to the absence of common causes.
  2. Common causes exist, but a measured set L of non-descendants of A blocks all backdoor paths (Figure 7.1). There is confounding, but no unmeasured confounding (no residual confounding that would need unmeasured variables). This is what a conditionally randomized trial guarantees and what observational studies hope for.

Sufficient set for confounding adjustment

A set L of measured non-descendants of A that blocks all backdoor paths, so that the treated and untreated are exchangeable within levels of L.

The heart transplant study of earlier chapters is the second case. The transplanted patients (A = 1) had more severe heart disease L, so had they not been transplanted, their risk of death would have been higher than that of the actually untreated. They are not marginally exchangeable with the untreated, but they are exchangeable within levels of L.

The backdoor criterion is all-or-nothing. It does not say how big the bias from an unblocked path is or which way it goes. Weak paths may cause little bias; strong paths in opposite directions may partly cancel.

Fine Point 7.1 · Strength and direction of confounding

Suppose your transplant study finds a risk ratio of 0.6, and a critic suggests confounding by smoking L. Smokers are less likely to get a transplant and more likely to die. So the transplant group has fewer smokers and would look better even if transplant did nothing. Adjusting for smoking will move the estimate toward the null (say from 0.6 to 0.7): the unadjusted estimate exaggerated the benefit. Signed diagrams formalize this for binary L, A, Y as in Figure 7.1: put a + or − on the L → A and L → Y arrows. Same signs give positive confounding (unadjusted estimate biased upward); opposite signs give negative confounding (biased downward). The rule can fail in more complex graphs. Large bias needs both a strong confounder–treatment and a strong confounder–outcome association, and for discrete confounders it also depends on how common the confounder is. When the confounders are unknown, sensitivity analyses can bound the plausible bias.

7.3 Confounding and the backdoor criterion

The book now works through a series of DAGs. All nodes are perfectly measured and none are conditioned on unless stated.

Causal DAG: U1 points to L and to Y; U2 points to L and to A; A points to Y. L is a collider between U1 and U2.

Figure 7.4 M-bias. L is a common effect (collider) of two unmeasured causes, U1 (a cause of Y) and U2 (a cause of A). There is no common cause of A and Y.

Figure 7.4 is the key counterexample. The only backdoor path is A ← U2 → L ← U1 → Y, and it is already blocked because L is a collider on it. There is no confounding: the crude association is causal. The book’s example: A is physical activity, Y cervical cancer, U1 a pre-cancer lesion, L a Pap smear, U2 a health-conscious personality (more active, more doctor visits). You should not adjust for L. Conditioning on the collider opens the path and creates bias. Pr[Ya = 1] is identified by the crude Pr[Y = 1 | A = a], while standardizing over L gives a biased answer. This structure is called M-bias because U2, L, U1 look like an M on its side.

Figure 7.4 is the first example where unconditional exchangeability holds but conditional exchangeability given L does not. The average effect is identified, but the effects within strata of L are not. The book calls the resulting bias selection bias (Chapter 8), because it comes from conditioning on a common effect. A margin note adds: if U1 caused U2, or vice versa, or some U3 caused both, there would be a common cause of A and Y, and neither unconditional nor conditional exchangeability given L would hold.

Confounder (structural definition)

For graphs where data on L, A, Y suffice for identification, L is a confounder if A and Y alone do not suffice (there is structural confounding), and a non-confounder if they do. In Figures 7.1–7.3, L is a confounder: Pr[Ya = 1] = Σl Pr[Y = 1 | A = a, L = l] Pr[L = l]. In Figures 7.2 and 7.3 L is not itself a common cause, but it is still a confounder because it blocks the path through U. In Figure 7.4, L is a non-confounder. Informally: a confounder is any variable that can be used to adjust for confounding.

Causal DAG: U1 points to L and to Y; U2 points to L and to A; L points to A; A points to Y.

Figure 7.5 Figure 7.4 plus an arrow L → A. Now there is a confounding path through L, and L is also a collider on another path.

Figure 7.5 adds L → A. That creates an open backdoor path A ← L ← U1 → Y: there is confounding. Conditioning on L blocks it, but also opens A ← U2 → L ← U1 → Y, on which L is a collider. (Being a collider is a property of a node on a given path: L is a collider on one path and not on the other.) The bias is intractable with only L. The fix is to measure another variable: either L1, sitting between U1 and A or Y (then L1 alone suffices), or L2, sitting between U2 and A or L (then {L, L2} suffices). Figure 7.6 shows both.

Causal DAG: U1 points to L and to L1; L1 points to Y; U2 points to L and to L2; L2 points to A; L points to A; A points to Y.

Figure 7.6 Figure 7.5 with two extra measured variables: L1 between U1 and Y, and L2 between U2 and A.

Fine Point 7.2 · Which effects are identified depends on what you measure

In Figure 7.6: measuring only L2 gives no exchangeability and identifies nothing. Measuring L2 and L gives exchangeability given both (but not given either alone). Then you can identify the effect within joint strata of (L, L2), using E[Y | A = a, L = l, L2 = l2]; the overall effect, by averaging that over Pr[L = l, L2 = l2]; the effect within strata of L, by averaging over Pr[L2 = l2 | L = l]; and the effect within strata of L2, by averaging over Pr[L = l | L2 = l2]. Measuring only L1 identifies the overall effect and effects within L1; adding L also gives effects within L and within (L, L1).

The section closes with a terminology choice. Two structures produce open backdoor paths: common causes (which the book calls confounding) and conditioning on a common effect (which it calls selection bias). An alternative definition calls confounding any “bias due to an open backdoor path”, or equivalently “any systematic bias that would be eliminated by randomizing A”. Under that definition, the bias from adjusting for L in Figure 7.4 is confounding, since randomizing A would remove U2 → A. One difference: the structural definition makes confounding a fact about the world, while the alternative makes it depend on the analysis (in Figure 7.4 you create confounding by adjusting). The choice has no practical consequence, since identification depends only on which exchangeability holds.

7.4 Confounding and confounders

The structural approach needs prior knowledge of the DAG, including unmeasured common causes, and then applies the backdoor criterion. The traditional approach instead labels variables as confounders by associations and adjusts for them. Its definition of a confounder:

  1. it is associated with the treatment;
  2. it is associated with the outcome conditional on the treatment (often “among the untreated”);
  3. it does not lie on a causal pathway between treatment and outcome.

In Figures 7.1–7.3 L meets all three conditions, and the traditional approach and the backdoor criterion agree that L should be adjusted for. In Figure 7.4, L also meets all three (it shares U2 with A and U1 with Y, and it is not a mediator). The traditional rule says adjust; the correct answer is that adjusting introduces bias. Figure 7.7 is another case.

Causal DAG: A points to L; U points to L and to Y. No arrow from A to Y.

Figure 7.7 L is a common effect of the treatment A and of an unmeasured cause U of the outcome. Adjusting for L opens A → L ← U → Y.

Associational criteria cannot characterize confounding. If you patch condition 2 to read “it is a cause of the outcome”, you avoid the mistake in Figure 7.4, but you now fail to call L a confounder in Figure 7.2, where it must be adjusted for. The traditional practice of declaring confounding when adjusted and unadjusted estimates differ (“change in estimate”) is also unreliable: the estimates can differ because of selection bias from adjusting for a non-confounder or because the effect measure is non-collapsible (Fine Point 4.3).

Technical Point 7.2 · Fixing the traditional definition

Robins (1997) gave conditions that repair it. Replace condition 3 with: there exist variables L and U such that Ya ⫫ A | L, U. Replace conditions 1 and 2 with: U can be split into disjoint parts U1 and U2 such that U1 is unassociated with A within strata of L, and U2 is unassociated with Y within joint strata of A, L, U1. If both hold, U is a non-confounder given data on L. These conditions correctly declare no confounding in Figure 7.4.

Causal DAG: U points to L, to A and to Y; A points to Y. L is a child of U only.

Figure 7.8 A surrogate confounder. Unmeasured U confounds A → Y; measured L is caused by U but is not on any backdoor path.

Fine Point 7.3 · Surrogate confounders

In Figure 7.8, socioeconomic status U confounds the effect of physical activity A on heart disease Y, and income L is a proxy for it. L is not on a backdoor path, so it cannot remove the confounding. But because it is correlated with U, adjusting for it usually removes part of the bias (if L were a perfect copy of U, it would remove all of it; a nondifferentially misclassified binary version partially blocks the path under weak conditions). So you should usually adjust for such surrogate confounders, and a reasonable strategy is to measure and adjust for as many as possible (Chapter 18).

The structural approach starts from the sources of confounding (the common causes) and then picks a sufficient adjustment set. It makes clear that whether a variable is a confounder depends on what else is in the set. In Figures 7.2 and 7.3, L is a confounder only because U is unmeasured; if U were measured and used, L would not be needed. In the book’s words: given a DAG, confounding is an absolute concept, whereas confounder is a relative one.

A structural approach can still be wrong, because the DAG can be wrong. Its two advantages: your actions match your beliefs (if you believe Figure 7.4, you do not adjust for L, whatever an associational rule says), and your assumptions are explicit, so others can criticize them.

7.5 Single-world intervention graphs

The equivalence between exchangeability and the backdoor criterion seems surprising because ordinary DAGs do not contain counterfactuals. Single-world intervention graphs (SWIGs), from Richardson and Robins (2013), add the counterfactual variables to the graph so that exchangeability can be checked directly by d-separation.

SWIG

A graph of the variables and causal relations in a hypothetical world where everyone received treatment level a. It is a transformation of the original causal DAG under one intervention.

How to build one from a DAG:

The book notes that the natural value A is usually unobservable once you intervene, but cites EEG studies that can detect a choice up to half a second before people are aware of it, which in principle would let you record A and still intervene. Arrows out of a are kept as bookkeeping, even though a constant cannot really cause anything.

SWIG: U points to L and to Y^a; L points to the natural treatment A; split node A | a; a points to Y^a.

Figure 7.9 SWIG for Figure 7.2 under intervention a. All paths between Ya and A pass through L, so Ya ⫫ A | L.

In Figure 7.9 the only path between Ya and the natural value A is A ← L ← U → Ya. Conditioning on L blocks it, so Ya and A are d-separated given L: conditional exchangeability can be read straight off the graph. (Under an FFRCISTG model, d-separation on the SWIG implies statistical independence.)

SWIG: U1 points to L and to Y^a; U2 points to L and to the natural treatment A; split node A | a; a points to Y^a.

Figure 7.10 SWIG for Figure 7.4 (M-bias). Ya and A are d-separated without conditioning, but conditioning on the collider L connects them.

In Figure 7.10, the path Ya ← U1 → L ← U2 → A is blocked at the collider L, so Ya ⫫ A (marginal exchangeability). Condition on L and the path opens, so Ya ⫫ A | L fails. This shows directly why the crude association is causal while any adjustment for L is biased. In general, on the SWIG, Ya is d-separated from A given L exactly when L is a set of non-descendants of A that blocks all backdoor paths. That is the proof of the backdoor criterion.

Causal DAG: U points to A and to L; A points to L; L points to Y.

Figure 7.11 L is a descendant of A (a mediator) that also blocks the backdoor path A ← U → L → Y.

SWIG: U points to the natural treatment A and to L^a; split node A | a; a points to L^a; L^a points to Y^a.

Figure 7.12 SWIG for Figure 7.11. The mediator becomes the counterfactual La; the factual L does not appear on the graph.

Fine Point 7.4 · Confounders cannot be descendants of treatment, but can be in its future

In Figure 7.11, L blocks the backdoor path, but it is also the only route by which A affects Y, so conditioning on it removes the effect itself. The SWIG (Figure 7.12) shows Ya ⫫ A | La, but nothing about the factual L, which is not even on the graph. So exchangeability given L cannot be assumed and the average effect is not identified from (L, A, Y). The problem comes from L being a descendant of A; being measured later is harmless on its own. If the arrow A → L were absent, L could be measured after treatment and still be a valid adjustment variable. What matters is which variables cause which, not the time order.

7.6 Confounding adjustment

Without randomization, causal inference rests on the untestable assumption that the measured L is a sufficient set. Given that assumption, the methods fall into two families:

The graph-level difference: IP weighting builds a pseudo-population in which A is independent of L, which amounts to deleting the arrow L → A. Stratification leaves the arrow in place and conditions on L, which you would draw as a box around L. With time-varying treatments and confounders (Part III), that conditioning can create selection bias, which is why g-methods are preferred there; g-estimation is the only generally valid stratification-based method.

Some methods handle confounding without assuming conditional exchangeability: difference-in-differences and negative controls, proximal inference (Technical Point 7.3), instrumental variables (Chapter 16), and the front door criterion (Technical Point 7.4). Each replaces exchangeability with other untestable assumptions. You pick the method whose assumptions you find most plausible in your setting.

Causal DAG: U points to C, to A and to Y; A points to Y; C has a curved arrow to Y.

Figure 7.13 Negative outcome control. C is the outcome measured before treatment; it shares the unmeasured confounder U with A and Y but cannot be affected by A.

Technical Point 7.3 · Difference-in-differences, negative controls, proximal inference

Say you want the effect of aspirin A on blood pressure Y, with unmeasured confounding by history of heart disease U. If you also have the pre-treatment blood pressure C (a negative outcome control, Figure 7.13), then E[C | A = 1] − E[C | A = 0] cannot reflect any effect of A, so it measures the confounding on the additive scale. Under additive equi-confounding, E[Y0 | A = 1] − E[Y0 | A = 0] = E[C | A = 1] − E[C | A = 0], the effect in the treated is

E[Y1 − Y0 | A = 1] = (E[Y | A = 1] − E[Y | A = 0]) − (E[C | A = 1] − E[C | A = 0])

This is difference-in-differences. It needs Y and C on the same scale and equal additive confounding; more general negative-control methods relax these. The arrow C → Y is not required. If you have both a negative outcome control C and a negative treatment control Z (Figure 7.15), the effect can be identified nonparametrically even with unmeasured U, provided (for discrete variables) C and Z have at least as many levels as U. This is proximal causal inference.

Causal DAG: U points to A and to Y; A points to M; M points to Y.

Figure 7.14 Front door setting. U confounds A and Y; the mediator M carries the whole effect of A on Y and shares no unmeasured causes with A or Y.

Causal DAG: U points to C, Z, A and Y; Z points to A; A points to Y; C has a curved arrow to Y.

Figure 7.15 Proximal inference setting: a negative outcome control C and a negative treatment control Z, both caused by the unmeasured confounder U.

Technical Point 7.4 · The front door criterion

In Figure 7.14 you cannot block the backdoor path because U is unmeasured. Pearl (1995) showed Pr[Ya = 1] is still identified by the front door formula:

Pr[Ya = 1] = Σm Pr[M = m | A = a] Σa′ Pr[Y = 1 | M = m, A = a′] Pr[A = a′]

Logic of the proof: the effect of A on M is unconfounded, so Pr[Ma = m] = Pr[M = m | A = a]. The effect of M on Y is confounded only through A, so it is identified by standardizing over A. Chain the two. (Pearl’s “backdoor formula” is what this book calls standardization or the g-formula.)

Because conditional exchangeability can never be guaranteed, observational studies measure many pre-treatment variables in the hope of blocking all backdoor paths. Expert knowledge helps in both directions: to find the variables you need, and to avoid ones that cause bias. At minimum, avoid adjusting for variables affected by the treatment or by the outcome. If experts consider several DAGs plausible, run the analysis under each and state the assumptions behind each.

The chapter ends with a point about criticism. Saying “your study may be confounded” is true of every observational study, so it is not informative. A useful criticism names a specific common cause, e.g. “smoking may leave a backdoor path open”. That is a testable challenge, and the investigator can respond by adjusting for smoking or running a sensitivity analysis. Selection and measurement bias also matter, and unlike confounding they can affect randomized trials too, which is the subject of the next chapter.

Key takeaways

  • Confounding is bias from common causes of treatment and outcome; on a DAG it is an open backdoor path into A.
  • The backdoor criterion: L is a sufficient adjustment set if it blocks all backdoor paths and contains no descendants of A. Under the book’s assumptions this is equivalent to Ya ⫫ A | L.
  • Confounding by indication (Figures 7.1, 7.2) is the clinical form: the reason for prescribing also predicts the outcome. A measured indication can block a path even when the deeper cause is unmeasured.
  • The associational definition of “confounder” fails: in M-bias (Figure 7.4) and Figure 7.7, adjusting for a variable that looks like a confounder creates bias.
  • Confounding is absolute given a DAG; whether a variable is a “confounder” depends on what else you adjust for. Proxies of unmeasured confounders (surrogate confounders) are usually worth adjusting for.
  • SWIGs split treatment into natural value A and intervened value a, so exchangeability can be read off by d-separation between Ya and A.
  • Adjustment methods: g-methods (standardization, IP weighting, g-estimation) and stratification/matching/regression. Difference-in-differences, negative controls, proximal inference, IVs and the front door formula trade exchangeability for other untestable assumptions.

Chapter 8 · Selection bias

Book pp. 103–118

In short

Selection bias is the bias created by conditioning on a common effect (a collider) of two variables, one of which is the treatment or associated with it and the other the outcome or associated with it. It typically arises from how people end up in the analysis: loss to follow-up, missing data, restricting to survivors, workers, volunteers, or people selected after treatment. Unlike confounding, randomization does not protect against it when selection happens after treatment is assigned. The fix is to treat selection (e.g. censoring C) as a second treatment, define the effect as “had everyone been treated and remained uncensored”, and adjust for confounding of the selection by IP weighting, which works in cases where stratification fails.

8.1 The structure of selection bias

The chapter reruns the pedestrian story as a randomized experiment. Now the investigator randomizes whether she looks up, so there is no confounding. But only pedestrians who later consent to data use are analyzed. Shy people (who are less likely to look up anyway) decline more often, and so do people in front of whom she looked up (they felt tricked). Among those who consented, the ones she looked up in front of must have overcome one reason to decline, so they are less likely to be shy, and hence more likely to look up. Selection alone produces an association between her looking up and theirs.

Selection bias

(structural definition) Bias from conditioning on a common effect of two variables, one of which is the treatment or a cause of it, and the other the outcome or a cause of it. Like confounding, it is a form of lack of exchangeability, and it exists even under the null. Unlike confounding, it can occur in randomized experiments.

The chapter focuses on selection bias under the null, i.e. bias present even if treatment has no effect.

Causal DAG: A points to Y; A has a curved arrow to C; Y points to C; C is boxed (conditioned on).

Figure 8.1 Selection on a common effect of treatment and outcome. C (fetal death) is caused by both folic acid A and cardiac malformation Y; the box shows the analysis is restricted to C = 0.

Figure 8.1: the effect of folic acid supplements A in early pregnancy on cardiac malformation Y. C is death before birth. Malformations raise mortality (Y → C); folic acid lowers mortality by preventing other malformations (A → C). The study includes only fetuses that survived to birth (C = 0, the box). Two paths now connect A and Y: the causal A → Y and the path A → C ← Y, which is opened by conditioning on the collider C. So Pr[Y = 1 | A = 1, C = 0] / Pr[Y = 1 | A = 0, C = 0] differs from the causal risk ratio. Without the restriction, the crude risk ratio would be causal.

Causal DAG: A points to Y; A has a curved arrow to C; Y points to C; C points to S; S is boxed.

Figure 8.2 Conditioning on a descendant of the collider. S (parental grief) is caused by fetal death C; restricting to non-grieving parents (S = 0) also opens A → C ← Y.

Figure 8.2 adds parental grief S, caused by C. If only non-grieving parents (S = 0) agree to take part, conditioning on S (a descendant of the collider) opens the same path. In both figures the bias comes from conditioning on a common effect of treatment and outcome, and it would appear even with no arrow A → Y.

The structure is more general than that. In Figures 8.3–8.6 the conditioned-on variable is a common effect of treatment (or a cause of it) and a cause of the outcome.

Causal DAG: U points to L and to Y; L has a curved arrow to C; A points to C; C is boxed. No arrow from A to Y.

Figure 8.3 Differential loss to follow-up. Unmeasured immunosuppression U causes symptoms L and death Y; both L and treatment A (via side effects) cause censoring C. The analysis is restricted to the uncensored.

Figure 8.3: an HIV cohort studying antiretroviral treatment A and 3-year death Y (no arrow A → Y, to keep things simple). U is severe immunosuppression, which raises the risk of death. Sicker people drop out (C = 1) more, through symptoms, CD4 count and viral load, collectively L (taken as unmeasured for now). Treated people drop out more because of side effects (A → C). Outcomes can only be seen in the uncensored, so the analysis conditions on C = 0. That opens A → C ← L ← U → Y. Intuition: a treated person with side effects who nevertheless stayed in the study is less likely to have the other reason for dropping out (high U). So among the uncensored, treatment is inversely associated with U, and therefore with death. The treatment looks protective when it has no effect.

Causal DAG: A points to L; U points to L and to Y; L points to C; C is boxed.

Figure 8.4 As Figure 8.3, but treatment affects censoring through symptoms: A → L → C.

Causal DAG: W points to A and to C; U points to L and to Y; L has a curved arrow to C; C is boxed.

Figure 8.5 Variation of Figure 8.3: an unmeasured common cause W of treatment and censoring replaces the arrow A → C.

Causal DAG: W points to A and to L; U points to L and to Y; L points to C; C is boxed.

Figure 8.6 Variation of Figure 8.4: an unmeasured common cause W of treatment and symptoms L.

In Figure 8.4 treatment affects symptoms L, which affect dropout. In Figures 8.5 and 8.6 there is no arrow from A into the selection structure; instead an unmeasured W (lifestyle, personality, education) causes both treatment and either willingness to attend visits (W → C) or the threshold for reporting symptoms (W → L). These last two are M-bias structures, like Figure 7.4. In all four, conditioning on C = 0 conditions on a collider of something linked to A and something linked to Y.

8.2 Examples of selection bias

Many named biases share this structure:

Fine Point 8.1 · Selection bias in case-control studies

This uses the structure of Figure 8.1, with C = 0 meaning selected into the case-control study. An investigator studies postmenopausal estrogen A and coronary heart disease Y, and picks controls preferentially among women with hip fracture. Estrogen prevents hip fracture, so A now affects selection (A → C), while case status affects selection by design (Y → C). The odds ratio is computed among the selected (C = 0), a collider. Controls are enriched for hip fracture, hence less likely to be on estrogen than cases, so the conditional odds ratio is pushed above the causal one. Berkson’s bias and incidence–prevalence bias have the same or similar structure.

Selection bias can occur in retrospective and prospective studies, and in both observational studies and randomized experiments. Figures 8.3 and 8.4 have no common causes of A and anything else, so they could depict a trial. If trial participants drop out before their outcome is measured, only Pr[Y = 1 | A = a, C = 0] is computable, and the uncensored may not be exchangeable with those lost.

This gives a key contrast: randomization protects against confounding but not against selection that happens after randomization. Selection before randomization is harmless. Trials enroll only volunteers, but there is no volunteer bias for the treatment comparison, because treatment is assigned after people agree (C = 0). Figures 8.3 and 8.4 cannot apply (treatment cannot cause agreement that came before it), and neither can 8.5 and 8.6 (randomization rules out a common cause of treatment and anything else).

8.3 Selection bias and confounding

The two chapters give two structural reasons for non-exchangeability: common causes (confounding) and conditioning on common effects (selection bias). Other fields use different words: statisticians and econometricians often call both “selection bias” (selection into the analysis versus selection into treatment), social scientists call unmeasured confounding “selection on unobservables”, and conditional exchangeability goes by “ignorability”, “selection on observables”, or “exogeneity”. The book’s point is that there are two distinct structures, whatever names you prefer.

Causal DAG: L points to C and to Y; U points to C and to A; C is boxed. No arrow from A to Y.

Figure 8.7 A study restricted to firefighters (C = 0). Parental SES L affects becoming a firefighter and heart disease Y; attraction to physical activity U affects becoming a firefighter and being physically active A.

Figure 8.7: a study of physical activity A and heart disease Y, restricted to firefighters. Suppose A has no effect on Y. Parental socioeconomic status L affects becoming a firefighter C and (through childhood diet) heart disease. An unmeasured liking for physical activity U affects becoming a firefighter and being active. There is no common cause of A and Y, so no confounding, and in the general population the crude risk ratio would be 1. Restricted to firefighters, conditioning on C opens A ← U → C ← L → Y. For the investigator, the label does not matter: in either case they must adjust for L. That is why many epidemiologists call any variable that needs adjusting a “confounder”.

The structural classification still pays off:

A margin note adds that Simpson’s paradox came from ignoring the difference between common causes and common effects, and that Blyth (1972) misread Simpson’s own example as extreme confounding.

Causal DAG: A points to Y1; U points to Y1 and to Y2; Y1 points to Y2; Y1 is boxed. No arrow from A to Y2.

Figure 8.8 Randomized heart transplant A, death at time 1 (Y1) and time 2 (Y2), protective haplotype U. No direct effect of A on Y2. The box on Y1 marks the conditioning built into the time-2 hazard.

Causal DAG: A points to Y1; Y1 points to Y2; A has a curved arrow to Y2; Y1 is boxed.

Figure 8.9 Alternative structure with a direct effect A → Y2. Without data on U, the data cannot tell Figure 8.8 from Figure 8.9.

Technical Point 8.1 · The built-in selection bias of hazard ratios

In the randomized trial of Figure 8.8, transplant lowers death at time 1 and has no direct effect at time 2. The risk ratios Pr[Y1 = 1 | A = 1]/Pr[Y1 = 1 | A = 0] and Pr[Y2 = 1 | A = 1]/Pr[Y2 = 1 | A = 0] are unbiased (the second is below 1 because it counts total mortality through time 2). But the hazard at time 2 is the risk of death among those who survived time 1, so the hazard ratio Pr[Y2 = 1 | A = 1, Y1 = 0] / Pr[Y2 = 1 | A = 0, Y1 = 0] conditions on the collider Y1, opening A → Y1 ← U → Y2. Treated survivors are less likely than untreated survivors to carry the protective haplotype (treatment helps explain their survival), so they die more at time 2. The hazard ratio is below 1 at time 1 and above 1 at time 2: the hazards cross even though the drug has no harmful effect. Within levels of U (a “frailty”) the time-2 hazard ratio is 1, but U is unmeasured. This applies to randomized trials too.

Fine Point 8.2 · Strength and direction of selection bias

Conditioning on a collider induces an association between its causes, but the sign depends on how the causes combine. If, given some background factor U, either A = 1 or E = 1 is enough to cause death (an “or” mechanism), then among the dead, A and E are negatively associated: a death without A is more likely explained by E. If both are needed (an “and” mechanism), they are positively associated among the dead. A plain DAG like Figure 8.12 cannot tell these apart; DAGs with sufficient-cause structure can. Large selection bias needs strong associations between the collider and both treatment and outcome (Greenland 2003 calls it collider-stratification bias).

8.4 Selection bias and censoring

The worked example is a marginally randomized trial of wasabi intake A on one-year death Y. Of 60 participants, 30 are assigned to wasabi and 30 to no wasabi. In truth, 17 die in each group, so the true risk ratio is 1. Censoring is more common in people with heart disease L = 1 at baseline and in the wasabi group. Only 9 wasabi and 22 non-wasabi participants stay uncensored, with 4 and 11 observed deaths:

Pr[Y = 1 | A = 1, C = 0] / Pr[Y = 1 | A = 0, C = 0] = (4/9) / (11/22) = 0.444 / 0.5 = 0.89

The 0.89 among the uncensored differs from the true 1: selection bias from conditioning on C. The structure is Figure 8.3, with U = atherosclerosis causing heart disease L and death Y. There are no common causes of A and Y. But C and Y do share a cause, through C ← L ← U → Y. If we wanted the effect of censoring on death, that path would be confounding, and the measured L blocks it.

Why talk about the effect of censoring when we care about A? Because with selection, the causal question itself should be redefined. Selection bias would vanish if nobody were censored, so the target becomes what would happen under treatment and no censoring:

Joint counterfactual

Ya=1,c=0 is the outcome had the person received treatment and remained uncensored; Ya=0,c=0 likewise under no treatment. The target contrast is, e.g., Pr[Ya=1,c=0 = 1] / Pr[Ya=0,c=0 = 1] or E[Ya=1,c=0] − E[Ya=0,c=0].

Even if censoring itself does not affect the outcome (usually reasonable, unless dropping out stops people getting other treatment), keeping c = 0 in the notation is useful. It makes clear that censoring is just another treatment: we want the effect of a joint intervention on A and C, and to remove selection bias for A we must adjust for confounding of C. That means exchangeability, positivity and consistency must hold for C as well as for A, and we use the same methods we would use to estimate an effect of C.

8.5 How to adjust for selection bias

Design can sometimes avoid selection bias, but loss to follow-up, self-selection and missing data often cannot be prevented. Then you correct in the analysis, e.g. by IP weighting: each uncensored person gets a weight

WC = 1 / Pr[C = 0 | L, A]

so that she stands for herself and for the censored people with the same L and A. (When there is also confounding, the product WAWC, with WA = 1/f(A | L), handles both; see Chapter 12 and Part III.)

Tree diagram: 60 people split by L (20 with L=0, 40 with L=1), then by A (half each), then by censoring C, then by death Y, with counts and probabilities on each branch.

Figure 8.10 The wasabi trial as a tree: heart disease L, treatment A, censoring C, death Y. Probabilities in parentheses, counts below. Deaths among the censored are shown for illustration only; a real investigator would not see them. The same numbers are in the table below.
LAnPr[C = 0 | L, A]Uncensored (deaths / n)Censored (deaths / n, unseen)Pr[Y = 1 | L, A]IP weight 1/Pr[C = 0]Pseudo-population (deaths / n)
001012 / 100 / 00.212 / 10
01100.51 / 51 / 50.222 / 10
10200.69 / 126 / 80.755/315 / 20
11200.23 / 412 / 160.75515 / 20
Table (from Figures 8.10 and 8.11) Wasabi trial. Pr[L = 1] = 2/3; Pr[A = 1] = 0.5 in each stratum of L. Within each (L, A) cell the death risk is the same among censored and uncensored, which is what lets weighting work. Totals among the uncensored: A = 1, 4/9; A = 0, 11/22 (RR 0.89). Totals in the pseudo-population: 17/30 in each arm (RR 1).

Reading the tree: 40 of the 60 had heart disease. Everyone had a 50% chance of wasabi regardless of L (no L → A arrow). The chance of staying uncensored varies, e.g. 50% for (L = 0, A = 1) and 60% for (L = 1, A = 0), which corresponds to the arrows A → C and L → C. Within each branch, the death risk is the same whether censored or not: C ⫫ Y | L, A.

Take the bottom branch: 20 people with L = 1, A = 1. Only 4 stay uncensored, so Pr[C = 0 | L = 1, A = 1] = 4/20 = 0.2. Each of the 4 gets weight 1/0.2 = 5; the 16 censored get weight 0. The 4 are copied 5 times to stand in for all 20. Doing this in every branch (weights 1, 2, 5/3, 5) gives a pseudo-population of 60 people with no censoring (Figure 8.11). In it, 17 of 30 die in each arm, so the risk ratio is 1, matching Pr[Ya=1,c=0 = 1] / Pr[Ya=0,c=0 = 1].

Tree diagram of the IP-weighted pseudo-population: same L and A splits, but every branch has C=0 with probability 1 and the full counts 10, 10, 20, 20 uncensored.

Figure 8.11 The pseudo-population after IP weighting for censoring. Every branch is fully uncensored (C = 0 with probability 1), with 2/10, 2/10, 15/20 and 15/20 deaths.

The pseudo-population association equals the causal effect if three conditions hold:

  1. Exchangeability for censoring. The uncensored must have the same average outcome as the censored with the same A and L. This holds if A and L block all backdoor paths between C and Y, i.e. include every factor that independently predicts both dropout and outcome. This cannot be tested.
  2. Positivity for being uncensored: Pr[C = 0 | A, L] > 0 in every cell. It is not needed for being censored, since we never want a world where everyone is censored. In the tree, Pr[C = 1 | L = 0, A = 0] = 0, and that causes no problem.
  3. Consistency, including a well-defined intervention on C. “Preventing loss to follow-up” or nonresponse is reasonably well defined. Treating a competing event as censoring is not. In a study of a drug and Alzheimer’s disease, death from other causes prevents the outcome. Censoring at death asks about a world where all other causes of death were abolished, which corresponds to no conceivable intervention.

Why not just stratify on L? In Figure 8.3, stratifying on L works: L blocks the backdoor path from C to Y, so Pr[Y = 1 | A = 1, C = 0, L = l] / Pr[Y = 1 | A = 0, C = 0, L = l] is the effect among the uncensored with L = l. It also works under the null in Figure 8.5. It fails in Figures 8.4 and 8.6. In Figure 8.4, L is a collider on A → L ← U → Y; conditioning on it blocks the C–Y path but opens a new A–Y path, so the conditional risk ratio differs from 1 even under the null. IP weighting works in all of Figures 8.3–8.6 because it reweights by L without conditioning on it. This is the first case in the book where stratification fails even though exchangeability, positivity and consistency all hold. The same issue returns with time-varying treatments in Part III.

8.6 Selection without bias

Conditioning on a collider always induces an association between its causes in at least one level of the collider, but not necessarily in every level.

Causal DAG: A points to Y; E points to Y; Y is boxed.

Figure 8.12 Surgery A and haplotype E are independent causes of death Y; the analysis conditions on Y.

In Figure 8.12, surgery A and a genetic haplotype E are marginally independent and both affect death Y. By d-separation they are associated given Y. Now suppose they act through completely separate mechanisms: surgery removes a tumor, the haplotype raises LDL cholesterol and the risk of heart attack. Split death into three cause-specific indicators: YA (death from tumor), YE (heart attack), YO (other). Y = 1 if any of them is 1 and Y = 0 only if all are 0. Figure 8.13 is this augmented DAG.

Augmented DAG: A points to Y_A; E points to Y_E; Y_O, Y_A and Y_E all point to Y. Y_O, Y_A, Y_E and Y are boxed.

Figure 8.13 Augmented DAG. A affects only tumor death YA, E only heart-attack death YE, and YO is death from other causes. Y is a deterministic function of the three.

Because Y is determined by the three components, conditioning on survival Y = 0 also fixes YA = YE = YO = 0. Those are non-colliders on the path from A to E, so the path is blocked and A ⫫ E | Y = 0. Conditioning on death Y = 1 does not fix the components (seven combinations are compatible with death), so A and E are associated among the dead. Restricting to survivors produces no bias here.

This breaks down if the mechanisms are linked:

In each case A and E are associated in both strata of Y.

Augmented DAG as in 8.13 plus crossing arrows A to Y_E and E to Y_A.

Figure 8.14 Shared mechanism: A also affects YE, and E also affects YA.

Augmented DAG as in 8.13 plus a node V pointing to both Y_A and Y_E.

Figure 8.15 A common cause V of tumor death and heart-attack death.

Augmented DAG as in 8.13 plus W1 pointing to Y_O and to A, and W2 pointing to Y_O and to E.

Figure 8.16 W1 is a common cause of A and other-cause death YO; W2 is a common cause of E and YO.

Technical Point 8.2 · Multiplicative survival model

The data follow a multiplicative survival model if Pr[Y = 0 | E = e, A = a] = g(e) h(a), i.e. no interaction between A and E on the multiplicative scale for survival. Figure 8.13 implies this: Pr[Y = 0 | e, a] = Pr[YO = 0] Pr[YA = 0 | a] Pr[YE = 0 | e]. Then the mortality 1 − g(e)h(a) is not multiplicative, which is why independence given Y = 0 comes with dependence given Y = 1.

The lesson: by adding unmeasured variables that determine Y, the augmented DAG represents both the independence given Y = 0 and the dependence given Y = 1, which the plain DAG cannot. Collider stratification is not always a source of selection bias, though in practice you need strong reasons (like separate mechanisms) to rely on this.

Key takeaways

  • Selection bias comes from conditioning on a collider (or its descendant) of something linked to treatment and something linked to outcome. It exists under the null.
  • Common forms: informative censoring, missing data / complete-case analysis, healthy worker bias, volunteer bias, selection after pre-study treatment, case-control control selection, and the built-in bias of hazard ratios.
  • Randomization prevents confounding but not selection bias arising after treatment assignment.
  • Treat censoring as a second treatment: the target is E[Ya,c=0], and you need exchangeability, positivity (for being uncensored) and consistency for C. Competing events such as death are poor candidates for “censoring”.
  • IP weighting by 1/Pr[C = 0 | A, L] removes selection bias (wasabi example: 0.89 among the uncensored, back to 1 after weighting). Stratifying on L fails when L is affected by treatment (Figures 8.4, 8.6).
  • Confounding and selection bias are distinct structures even when the analysis is the same; the distinction helps with design, method choice and communication.
  • Collider stratification does not always bias every stratum (Figure 8.13), but that requires special structure such as fully separate mechanisms.

Chapter 9 · Measurement bias and “noncausal” diagrams

Book pp. 119–130

In short

Every earlier chapter quietly assumed that treatment, outcome and confounders were recorded without error. This chapter drops that assumption. When we only have the measured versions A*, Y* and L*, the association between them can differ from the causal effect even with no confounding and no selection bias. This is measurement bias. The chapter shows how to draw measurement error on causal diagrams, classifies it as independent or dependent and as nondifferential or differential, and explains why a mismeasured confounder usually fails to block a backdoor path (with an important exception: if decision makers only saw the recorded value, the recorded value is exactly what you need). The second half asks a deeper question about the diagrams themselves: an arrow only has a causal meaning if there is a well-defined intervention on its source. Many published DAGs contain arrows out of things like obesity that fail this test. Such “noncausal” DAGs can still give the right adjustment set, but they can also hide a surrogate confounder and give false confidence.

Intro Opening example: the careless note-taker

The chapter reuses the pedestrian experiment. An investigator randomly decides whether to look up at the sky and records whether other pedestrians look up. Randomization rules out confounding, and every pedestrian’s response was recorded, so there is no selection bias. But the collaborator who wrote down the responses missed half of the pedestrians who looked up and recorded them as “did not look up”. Even if the true effect is strong, the association between treatment and the recorded outcome is diluted.

Measurement bias

(also called information bias): the association between treatment and outcome is weakened or strengthened because of the process by which the data were measured. It can occur in any design, randomized trials included.

9.1 Measurement error

Take an observational study of a cholesterol-lowering drug A and liver disease Y. If drug use is abstracted from medical records, several things can go wrong: the abstractor mistypes, the physician forgets to note the prescription, or the patient never takes the prescribed pills. The variable in the analysis file is then the measured treatment A* (“A-star”), which need not equal the true treatment A for a given person. Psychologists call A the “construct” and A* the “measure” or “indicator”.

Causal diagram: A points to Y and to A-star; U_A also points to A-star.

Figure 9.1 True treatment A causes the outcome Y and also causes its own recorded value A*. UA stands for everything else that determines the recorded value, i.e. the sources of measurement error. There is no confounding or selection here.

The measurement error for a person is the gap between A* and A; its size and direction are governed by UA. Strictly, UA need not be drawn, since it is neither a common cause nor conditioned on, but drawing it makes the error mechanism explicit and lets us compare structures below. Measurement error in a discrete variable is called misclassification.

The outcome can be mismeasured too. Figure 9.2 adds Y* and its error sources UY. This is the typical real situation: we want the effect of A on Y but only have A* and Y*.

Causal diagram: A points to Y; A points to A-star; Y points to Y-star; U_A points to A-star; U_Y points to Y-star.

Figure 9.2 Both treatment and outcome are recorded with error, each with its own separate error source. This is the “independent, nondifferential” structure.

With perfectly measured A and Y in Figure 9.2, association would be causation: Pr[Y = 1 | A = 1] / Pr[Y = 1 | A = 0] would equal Pr[Ya=1 = 1] / Pr[Ya=0 = 1]. With only the starred versions, there is no such guarantee:

Pr[Y* = 1 | A* = 1] / Pr[Y* = 1 | A* = 0] ≠ Pr[Ya=1 = 1] / Pr[Ya=0 = 1] (in general)

The key consequence: exchangeability, positivity and consistency are no longer enough to identify the effect once measurement bias is present. Measurement is a third, separate requirement.

Technical Point 9.1 · Formal definitions

Define the errors eA = A* − A and eY = Y* − Y. They are independent if their joint density factorizes, f(eY, eA) = f(eY) f(eA). The treatment error is nondifferential if its distribution does not depend on the true outcome, f(eA | Y) = f(eA); the outcome error is nondifferential if f(eY | A) = f(eY).

9.2 The structure of measurement error

Confounding has one structure (common causes) and selection bias has one structure (conditioning on common effects). Measurement error has several. The book sorts them along two axes.

Independence. In Figure 9.2 the error sources UA and UY are d-separated: any path between them passes through a collider (A* or Y*). That fits, for example, haphazard data-entry errors in electronic records. If instead both variables come from a retrospective phone interview, a person’s ability to recall their medical history (UAY) affects the recorded drug use and the recorded outcome together. The errors are then dependent (Figure 9.3).

Causal diagram as in Figure 9.2, plus a common parent U_AY pointing to both U_A and U_Y.

Figure 9.3 Dependent but nondifferential errors: a shared factor UAY (e.g. recall ability) drives the errors in both the recorded treatment and the recorded outcome.

Differentiality. In Figures 9.2 and 9.3 the error in treatment does not depend on the true outcome and vice versa, so the errors are nondifferential. If the true outcome affects how treatment gets recorded (an arrow Y → UA), the treatment error is differential (Figure 9.4). Examples:

Causal diagram as in Figure 9.2, plus an arrow from Y to U_A.

Figure 9.4 Independent but differential error in treatment: the true outcome Y influences the error in the recorded treatment (recall bias, reverse causation).

The reverse direction, an arrow A → UY, makes the outcome error differential (Figure 9.5). This happens if doctors who suspect the drug causes liver toxicity watch treated patients more closely, so liver problems get detected more often among the treated.

Causal diagram as in Figure 9.2, plus an arrow from A to U_Y.

Figure 9.5 Independent but differential error in the outcome: true treatment influences how well the outcome is ascertained (e.g. closer monitoring of treated patients).

Combining a shared error source with either differential arrow gives errors that are both dependent and differential (Figures 9.6 and 9.7).

Causal diagram: U_AY points to U_A and U_Y; U_A to A-star; U_Y to Y-star; A to A-star; Y to Y-star; A to Y; Y to U_A.

Figure 9.6 Dependent and differential: shared error source UAY plus an arrow from the true outcome Y into the treatment error UA.

Causal diagram: U_AY points to U_A and U_Y; U_A to A-star; U_Y to Y-star; A to A-star; Y to Y-star; A to Y; A to U_Y.

Figure 9.7 Dependent and differential: shared error source plus an arrow from the true treatment A into the outcome error UY.

Four types of measurement error

independent nondifferential (Fig. 9.2); dependent nondifferential (Fig. 9.3); independent differential (Figs. 9.4, 9.5); dependent differential (Figs. 9.6, 9.7).

The structure matters because it determines which correction methods apply. Most of the correction literature handles the independent nondifferential case and relies on modeling assumptions plus a validation sample, a subset where the key variables are measured well. The book does not cover these methods. Its message is that measuring variables, like selecting people, can introduce bias, and that a realistic diagram must show confounding, selection and measurement at once. The best remedy is better measurement.

Fine Point 9.1 · Strength and direction of measurement bias

A common belief is that nondifferential error always biases toward the null. The book says this is false in general. The only safe case is when A truly has no effect and the error is independent and nondifferential: with no A → Y arrow in Figure 9.2, both the true and the measured associations are null. Otherwise the measured association can be further from or closer to the null than the true one. For ordinal or continuous treatments, even independent nondifferential error can reverse the direction of a trend, when the mean of A* given A is not monotone in A. The size of the bias grows with the size of the error (the strength of UA → A* and UY → Y*), which a DAG cannot show because DAGs are qualitative.

9.3 Mismeasured confounders and colliders

Confounders can be mismeasured too, and that can create bias even when treatment and outcome are measured perfectly. In Figure 9.8, history of hepatitis L makes people less likely to get the drug A and more likely to develop liver disease Y. The backdoor path A ← L → Y is blocked by conditioning on L, so IP weighting or standardization on L would recover the causal risk ratio.

Causal diagram: L points to A and to Y (curved arrow); A points to Y; L points to L-star.

Figure 9.8 L (true hepatitis history) confounds A → Y. The analyst only has L*, a recorded version of L.

Now suppose hepatitis history comes from a questionnaire. Some people hide it, some forget, some make mistakes. The analyst has L* instead of L, and conditioning on L* does not in general block A ← L → Y. Standardizing on L* gives a biased risk ratio. The same happens in Figure 9.9, where L is not itself a common cause but lies on a backdoor path A ← L ← U → Y.

Causal diagram: U points to L and to Y; L points to A and to L-star; A points to Y.

Figure 9.9 The backdoor path runs A ← L ← U → Y. Conditioning on the true L would block it; conditioning on the mismeasured L* generally does not.

You can call this measurement bias or you can call it unmeasured confounding, treating L as unmeasured and L* as a surrogate confounder (Figure 9.8 is the same structure as Figure 7.8). The label makes no practical difference.

Spurious effect modification. Mismeasured confounders can also manufacture apparent effect modification. Suppose everyone who reports hepatitis (L* = 1) truly had it, but half of those who report no hepatitis (L* = 0) actually had it too. Suppose the drug has no effect on anyone (sharp null). In the stratum L* = 1 everyone has L = 1, so there is no confounding and the analyst finds no association. In the stratum L* = 0 there is a mix of L = 1 and L = 0, so residual confounding produces a non-null association. An analyst who reads both stratum estimates as causal effects would conclude that L* modifies the drug’s effect, although no effect exists in either stratum.

Mismeasured colliders. Conditioning on a mismeasured collider C* also causes selection bias, because C* is a descendant of the collider and therefore itself a common effect of A and Y (Figure 9.10, the same structure as Figure 8.2).

Causal diagram: A points to Y and to C (curved); Y points to C; C points to C-star, which is boxed.

Figure 9.10 C is a collider of A and Y. Conditioning (box) on its recorded version C* opens the path just as conditioning on C would.

Fine Point 9.2 · When mismeasured confounders are not a problem

Suppose high blood pressure L affects both antihypertensive therapy A and stroke Y. Doctors decide treatment from the blood pressure reading L* taken in the office, not from the patient’s true blood pressure. Then any part of L missed by L* could not have influenced treatment, and L* fully mediates L → A (Figure 9.11). The backdoor path is blocked by conditioning on either L or L*, so the measurement error is harmless.

Figure 9.12 is more extreme: there, data on the true L would not suffice but L* would. Conditioning on the true L blocks A ← L* ← L ← U1 → Y, but L is a collider on A ← L* ← U2 → L ← U1 → Y, so conditioning on it opens that path. Conditioning on L*, the only parent of A, blocks everything. The general rule: effects are identifiable whenever the data hold as much information as the decision makers used, whether or not that information was accurate.

Causal diagram: L points to L-star and to Y; L-star points to A; A points to Y.

Figure 9.11 The recorded value L* is what drives treatment. Adjusting for either L or L* blocks the backdoor path (same structure as Figure 7.2).

Causal diagram: U1 points to L and to Y; U2 points to L and to L-star; L points to L-star; L-star points to A; A points to Y.

Figure 9.12 Adjusting for the recorded L* is sufficient; adjusting for the true L is not.

9.4 Causal diagrams without mismeasured variables?

Earlier chapters made two simplifying assumptions when drawing diagrams. The first was that all variables are perfectly measured. That is unrealistic, and this chapter has shown what mismeasurement does: it can create an association between A* and Y* under a null effect with no confounding or selection, and it can prevent a measured confounder from blocking a backdoor path.

One could argue every diagram should then show both true and measured versions of every variable. The book instead recommends a two-step approach. First draw the diagram as if everything were measured perfectly, and study confounding and selection there. If those biases exist under perfect measurement they will usually exist under mismeasurement too (Fine Point 9.2 shows exceptions). Then add measurement error as a further layer. The rest of the book follows this practice: when the focus is confounding or selection, true and measured values are not distinguished.

The second assumption is more fundamental and is the subject of the next section.

9.5 Many proposed causal diagrams include noncausal arrows

Consider Figure 9.13 (the same as Figure 7.1). A is an antiviral for COVID-19 patients, Y is death, and L is obesity (BMI > 30). Obese patients are more likely to be treated and more likely to die without treatment, so experts draw L → A and L → Y. Assume treatment depends only on L and the physician’s preference, and that nothing is mismeasured.

Causal diagram: L points to A; A points to Y; L points to Y (curved arrow).

Figure 9.13 The usual confounding triangle: obesity L affects antiviral treatment A and death Y.

Treatment nodes have a special status (Section 6.4): quantitative causal inference needs a well-defined intervention on A, or the counterfactuals Ya are undefined and consistency fails. Giving or withholding an antiviral is well defined, so the arrow A → Y is meaningful. But Chapter 3 argued that “intervening on obesity” is not well defined (diet? surgery? genes?), so the counterfactuals Yl are undefined. Then what does the arrow L → Y mean?

Look at the two arrows out of L separately:

Causal DAG (book’s restricted sense)

a DAG in which every arrow has a well-defined causal interpretation. Under this definition Figure 9.13 is a “noncausal” diagram. The quotes are a reminder that much of the literature still calls such DAGs causal.

Fine Point 9.3 · Well-definedness depends on the outcome

How can an intervention on L be well defined for A but not for Y? Because the symbol L was used for two things: the physical body weight and its recorded value. Write L for the body weight and L* for the record. The doctor only reacts to L*. Changing the record while leaving the body unchanged changes the doctor’s behaviour just as changing the body would, so Al is well defined. No such trick works for death, so Yl is not.

A second group of experts offers another account of why obese patients die more: obesity is a surrogate for a hidden set of factors H (genetic, metabolic, microbiome, others not yet known) that cause death. This gives Figure 9.14 (same structure as Figure 7.2).

Causal diagram: H points to L and to Y; L points to A; A points to Y.

Figure 9.14 Obesity L is a recorded, lower-dimensional effect of hidden factors H. H affects death directly; L affects death only through treatment.

In Figure 9.14 there is no arrow H → A: H influences treatment only through L, which is plausible if BMI is the only information the doctor uses. The diagram’s counterfactuals are Al and Ya, and from these an intervention on L does get a meaning for Y: Yl = YAl, the outcome under whatever treatment the doctor would give at BMI l (this is “recursive substitution”, Technical Point 6.2). There is no contradiction with the claim that Yl is ill-defined under Figure 9.13: if Figure 9.14 is the true causal DAG, Figure 9.13 is simply not causal, because it omits the common cause H.

Figure 9.14 also claims there are well-defined interventions on H (arrows H → L, H → Y), even though we cannot list what H contains. The book’s justification: whenever a node L has an ill-defined effect, introduce a high-dimensional parent H collecting all unmeasured causes of L. L is then a lower-dimensional summary of H (sometimes a deterministic many-to-one function of it). Current knowledge does not rule out that interventions on the components of H exist, so the arrow H → Y is tentatively justified and Figure 9.14 counts as causal. The book reserves H for hidden common causes where the effect of one variable on the other is ill-defined, and U for the case where it is well defined; elsewhere it just uses U.

Fine Points 9.4 and 9.5 · Statistical DAGs and the front door

9.4. Some authors interpret DAGs purely statistically: arrows encode conditional independencies via d-separation, and only treatment has counterfactuals. Under that reading the arrow L → Y need not be removed. Richardson and Robins (2013) point out the weakness: if arrows carry no causal meaning, there is no reason the data should factor according to any incomplete DAG, and with unmeasured variables the implied independencies cannot even be checked. A researcher who asserts the front-door graph (Figure 7.14) presumably believes A and Y share a cause and that M fully mediates the effect, which is a causal belief. A milder version treats the graph as a prediction about what a future randomized trial of A would show.

9.5. Figure 9.14 is the front-door diagram with L → A in place of A → M, so E[Yl] is identified by the front-door formula. If someone adds a direct L → Y arrow (e.g. dietitians give unrecorded extra care to high-BMI patients), then Yl means the total effect through both paths and is no longer identified, though the part through L → A → Y still is (Chapter 23).

9.6 Does it matter that many proposed diagrams include noncausal arrows?

Sometimes it does not. Figure 9.13 is causally wrong, yet it leads to the right analysis: under both Figure 9.13 and Figure 9.14, conditioning on L blocks every backdoor path from A to Y, so standardization or IP weighting on L gives the average causal effect of A. This works because in both graphs no unmeasured variable points into A (compare Fine Point 9.2). Many DAGs in health and social science are noncausal in this sense, since they include variables like obesity without the hidden H that would make them causal. The identifying formula from the noncausal DAG may still coincide with the one from the causal DAG.

But it can also mislead. Figure 9.15 has U → L, U → A and L → Y. Read as a causal DAG, it says there is a well-defined intervention on L affecting Y, and conditioning on L blocks the backdoor path A ← U → L → Y. If L is really a surrogate of a hidden H (Figure 9.16), the path A ← U → H → Y runs through H, and conditioning on L does not block it. An investigator who drew Figure 9.15 without noticing that L is only a surrogate would wrongly conclude that adjusting for L is enough. This is the surrogate-confounder problem of Section 9.3 again.

Causal diagram: U points to L and to A; L points to Y (curved); A points to Y.

Figure 9.15 Drawn as if L itself causes Y. Under this diagram, adjusting for L blocks the backdoor path.

Causal diagram: U points to H and to A; H points to L and to Y (curved); A points to Y.

Figure 9.16 The same situation with L recognised as a surrogate of hidden factors H. The backdoor path A ← U → H → Y stays open after adjusting for L.

Fine Point 9.6 · From noncausal to causal diagrams

Investigators propose Figure 9.17, based on time order plus two facts: L is associated with Y, and known but unmeasured factors U affect A and are associated with L. Since L is a descendant of A and also on a path from U, the effect of A on Y is not identifiable under this graph (Fine Point 7.4). Then they realise L → Y has no well-defined intervention. They add a hidden H with H → Y, make L a surrogate of H, and redirect the arrows from U and A into H (Figure 9.18). The effect is still not identifiable, because A ← U → H → Y cannot be blocked by any measured variable. Even if L were a deterministic function of H, conditioning on L would not block paths through H, because H is richer than L.

Moving all arrows from L onto H is a substantive choice. If U affects L directly instead (Figure 9.19), there is no open backdoor path and the effect is identifiable. Example: U is a physician’s decision to order a diagnostic test, L the test result, and H the biology that determines the result.

Causal diagram: U points to A and to L; A points to L and to Y (curved); L points to Y.

Figure 9.17 Initial (noncausal) proposal: L is affected by A and U and is drawn as a cause of Y.

Causal diagram: U points to A and to H; A points to H and to Y (curved); H points to Y and to L.

Figure 9.18 Revised causal version: hidden H inherits all arrows that pointed into L, and L is just its recorded surrogate. The backdoor path through U and H cannot be blocked.

Causal diagram: U points to A and to L; A points to H and to Y (curved); H points to Y and to L.

Figure 9.19 Alternative causal version: U affects the surrogate L directly, not H. Now there is no open backdoor path from A to Y.

The broader lesson: an arrow X → Y cannot be interpreted without a well-defined intervention on X. When proposing a DAG, ask of each node whether the variable you happen to have measured is the one that has well-defined effects, or a stand-in for something else. Blind acceptance of noncausal arrows can produce false confidence. This scrutiny is pointless for an electrical circuit but essential in health and social sciences. Since no intervention is perfectly defined, the book settles on “sufficiently well-defined” interventions, specified with no meaningful vagueness given current knowledge, and from here on assumes (with flagged exceptions) that every arrow in its DAGs meets that bar.

Key takeaways

  • Analyses use measured variables A*, Y*, L*. Their associations can differ from causal effects even in a perfect randomized trial; this is measurement (information) bias, a third source of systematic bias alongside confounding and selection.
  • Measurement error is classified as independent or dependent, and as nondifferential or differential. Recall bias, reverse causation bias and closer monitoring of treated patients are differential errors.
  • Nondifferential error does not guarantee bias toward the null. Only under a true null with independent nondifferential error is the measured association guaranteed to be null too.
  • Adjusting for a mismeasured confounder usually leaves residual confounding and can create spurious effect modification. Conditioning on a mismeasured collider still causes selection bias.
  • If decision makers acted on the recorded value L*, then adjusting for L* is sufficient, and sometimes better than adjusting for the true L.
  • An arrow is causal only if its source has a well-defined intervention. Arrows out of variables like obesity are “noncausal”; making the diagram causal means adding hidden factors H of which the recorded variable is a surrogate.
  • Noncausal DAGs sometimes give the right adjustment set, but they can hide surrogate confounding and make you believe a backdoor path is blocked when it is not.

Chapter 10 · Random variability

Book pp. 131–144

In short

Chapters 1–9 pretended every study was effectively infinite, so the only obstacles were systematic biases (confounding, selection, measurement). That is identification. Real studies are finite, so we also need estimation: point estimates, standard errors and confidence intervals. This chapter defines those ideas using the heart-transplant data, explains what a 95% confidence interval does and does not mean, and points out that the usual binomial intervals quietly assume the study is a random sample from an imaginary infinite “super-population”. Its central example is a randomized trial where, by bad luck, smoking ended up associated with treatment: the book argues you should adjust for such chance imbalances (the conditionality principle). It then shows that this principle collapses when there are many covariates, because almost no stratum contains both treated and untreated people. This is the curse of dimensionality, and it is the reason the following chapters turn to models.

Intro Opening example: four pedestrians

The pedestrian experiment again, this time with perfect randomization, complete follow-up and perfect measurement. The problem is size: only 4 pedestrians, 2 per group. By chance, 1 of the 2 in the “looking up” group and neither in the “looking straight” group was blind. Even if looking up has a strong average effect, half the treated group could not respond to it, and the estimate is diluted. Causal inferences can go wrong for two qualitatively different reasons: systematic bias (Chapters 7–9) and random variability (this chapter).

10.1 Identification versus estimation

In Chapter 2 the heart-transplant study had 20 people, but each was treated as standing for a billion identical copies. That let the book ignore random fluctuation. Problems in which the population can be treated as effectively infinite are identification problems, and the identifying conditions (exchangeability, positivity, consistency) are what you need even with unlimited data. From now on the book also deals with statistical estimation from finite samples.

Back to the heart-transplant data: 7 of the 13 treated people died. Imagine the 20 people are a random sample from a huge super-population, and we want to learn about that super-population.

Estimand

the super-population quantity we want, e.g. Pr[Y = 1 | A = a].
Estimator a rule that turns any sample into a number for the estimand, e.g. the sample proportion P̂r[Y = 1 | A = a].
Estimate the number the rule gives for this particular sample, here 7/13 (a point estimate).

An estimator is consistent if its estimates get arbitrarily close to the estimand as the sample size n grows. The sample proportion is consistent for Pr[Y = 1 | A = a]. But a consistent estimator can still be far off in a small sample, so we want a measure of how much to trust it. In the absence of systematic bias, that measure is the confidence interval; larger samples give narrower intervals.

The common recipe is the 95% Wald interval: point estimate ± 1.96 × estimated standard error, where 1.96 is the 97.5% quantile of a standard normal. For a proportion p̂ the standard error is √[p(1 − p)/n] (the binomial standard error). For the treated heart-transplant patients:

SE = √[(7/13)(6/13)/13] = √(0.2485/13) = 0.138
95% CI = 0.538 ± 1.96 × 0.138 = 0.538 ± 0.271 = (0.27, 0.81)

(A Wald interval is only guaranteed in large samples; the book assumes 13 is enough for illustration.) The interval’s centre and width change from sample to sample. Its quality is judged by coverage:

Calibrated

the interval contains the estimand in 95% of random samples. Conservative in more than 95%. Anticonservative in fewer. Valid calibrated or conservative for every possible value of the true parameter. Among valid intervals we want the narrowest.

Why care about samples you never see? Because if you and your colleagues compute valid 95% intervals in every study for a lifetime, at least 95% of those intervals will contain their true parameter. You just cannot tell which ones are in the unlucky 5%.

A 95% interval does not mean there is a 95% probability that the estimand lies in (0.27, 0.81). The estimand is fixed, so it is either in that interval or not. The 95% describes how often the procedure traps the truth across repetitions. (A Bayesian 95% credible interval does carry the “95% probability” reading, but there probability means degree of belief. The book uses the frequency definition; see Fine Point 11.2.)

Intervals are small-sample valid if valid at every sample size (calibrated ones are called exact), or large-sample (asymptotically) valid if valid only as n grows. The Wald interval for a proportion is large-sample calibrated but can be badly anticonservative in small samples (Brown et al., 2001), and other large-sample intervals are often preferred. In the book, “95% confidence interval” means a large-sample valid one unless stated otherwise.

Finally, not every consistent estimator can centre a valid Wald interval. Most applied researchers call an estimator unbiased if it can and biased if it cannot, and the book adopts that working definition.

Fine Point 10.1 · Honest confidence intervals

For a large-sample interval, the sample size needed to reach 95% coverage can depend on the unknown true parameter. An interval is uniform or honest if one sample size works whatever the true value. Without uniformity, at any fixed n some data-generating distributions could have coverage far below 95%. The required n is usually unknown and hard to find even by simulation. From here on “valid” means large-sample honest.

Technical Point 10.1 · Bias and consistency

With n i.i.d. observations from a distribution P in a model M, an estimator θ̂n is consistent if PrP[|θ̂n − θ(P)| > ε] → 0 for every ε > 0 and every P in M. It is exactly unbiased if EP[θ̂n] = θ(P) for every P; for many parameters, such as a risk ratio, no exactly unbiased estimator exists. A systematically biased estimator is neither. Robins and Morgenstern (1987) show that the estimators able to centre valid Wald intervals are exactly the uniformly asymptotically normal and unbiased (UANU) ones: (θ̂n − θ)/σn converges to a standard normal uniformly over M. In the book, “unbiased” without qualification means UANU. Some consistent estimators fail this (examples in Chapter 18).

Fine Point 10.2 · Uncertainty from systematic biases

The width of a Wald interval reflects only random error. Systematic bias from confounding, selection or measurement does not shrink with sample size, so in a large study the interval captures a smaller and smaller share of the real uncertainty and the stated “95% confidence” becomes overconfidence. Some authors therefore call them compatibility intervals: the effect sizes most compatible with the data given the adjustment assumptions. Quantitative bias analysis and Bayesian methods try to fold systematic uncertainty into the interval.

10.2 Estimation of causal effects

Now let the heart-transplant study be a marginally randomized experiment, with the 20 people a random sample from a near-infinite super-population in which everyone was randomized and adhered. Exchangeability then holds in the super-population, Pr[Ya = 1] = Pr[Y = 1 | A = a], so the causal risk difference equals the associational one there. The sample proportions are unbiased for Pr[Y = 1 | A = a] and hence for Pr[Ya = 1]. Testing the causal null reduces to comparing 7/13 = 0.54 with 3/7 = 0.43. The causal risk difference is estimated by 7/13 − 3/7 = 0.11 and the risk ratio by (7/13)/(3/7) = 1.26, each with standard confidence intervals. In observational studies, similar but more involved procedures give intervals for standardized, IP-weighted or stratified estimates.

A different picture: only the 20 study participants are randomized. Then exchangeability will not hold exactly in the sample. If each person has a good or bad prognosis, chance might put 2 of the 13 treated and 3 of the 7 untreated in the bad-prognosis group. Larger samples make such imbalances proportionally smaller, but they are always possible.

This picture allows two targets of inference:

Usually the first target is not what we want. A trial comparing two first-line cancer treatments is run to guide treatment of future patients like the participants; the participants themselves have already been treated by the time the answer arrives. That requires some larger group to generalize to, the super-population.

10.3 The myth of the super-population

Chapter 1 named two sources of randomness: sampling variability and nondeterministic counterfactuals. Nearly everyone would report the binomial interval 7/13 ± 1.96 √[(7/13)(6/13)/13] “to account for random variability”. That interval is valid only if the count of deaths among the 13 treated is binomial. When is that true?

Scenario 2 is untenable: risk almost certainly differs between people, at least through genetics. If the pi vary, the natural estimand is their average, and the count is no longer binomial; the binomial interval is then conservative (too wide). So anyone who reports a binomial interval and accepts that people differ in risk is implicitly assuming Scenario 1. Under Scenario 1 the count is binomial whether counterfactuals are deterministic or stochastic, which is one attraction of the super-population view.

But in most studies nobody was sampled at random from anything. The super-population survives for two reasons: it gives simple methods, and it offers a simple story for generalizing from the study (20 patients) to a target population (say, all immortals in the Greek pantheon). Because the story is usually fiction, the book says an interval computed under Scenario 1 should be read as a what-if statement: it would cover the parameter had the participants been randomly sampled from a near-infinite population. Investigators may reasonably reject Scenario 1 if the pool of potential recipients is not much larger than the study, or if the target population differs from the study population in ways that sampling variability cannot explain. The rest of the book nevertheless accepts the super-population assumption.

10.4 The conditionality “principle”

A randomized trial of 240 people, 120 per arm, estimates the effect of treatment A on 1-year death Y (Table 10.1).

Y = 1Y = 0
A = 12496
A = 04278
Table 10.1 Randomized trial, 240 participants: deaths (Y = 1) and survivors (Y = 0) by treatment arm.

The risk difference is 24/120 − 42/120 = 0.20 − 0.35 = −0.15. Its estimated variance is

(0.20 × 0.80)/120 + (0.35 × 0.65)/120 = 31/9600 ≈ 0.00323
95% CI = −0.15 ± 1.96 × √(31/9600) = −0.15 ± 0.11 = (−0.26, −0.04)

The investigators publish: treatment cuts the risk of death by 15 percentage points. With 240 people, though, randomization only guarantees that any lack of exchangeability is due to chance. One can think of the uncertainty about chance associations between unmeasured risk factors and treatment as part of what the interval’s width of 0.22 reflects.

Months later they find that smoking L was recorded and stratify on it (Table 10.2).

StratumY = 1Y = 0Risk
L = 1 (smokers)A = 14764/80 = 0.05
A = 02382/40 = 0.05
L = 0 (nonsmokers)A = 1202020/40 = 0.50
A = 0404040/80 = 0.50
Table 10.2 The same trial stratified by smoking L. The “Risk” column is computed here for convenience and is not in the book. Within each stratum treated and untreated have identical risk.

Two-thirds of smokers (80/120) were treated, versus one-third of nonsmokers (40/120). Within smokers the risk difference is 0.05 − 0.05 = 0; within nonsmokers it is 0.50 − 0.50 = 0. In these data smokers happen to have low risk and were mostly treated, which made the treated group look better overall. The adjusted analysis says the treatment has no effect, with 95% CI (−0.076, 0.076) (variance from Technical Point 10.5).

Either someone tampered with the randomization (malfeasance), or randomization produced a badly imbalanced sample (very bad luck). Quality assurance rules out malfeasance, so it was luck. Should they retract?

The facts settle it. Assume the true risk difference is constant across L, and imagine rerunning the trial trillions of times. Keep only the runs where L and A are as strongly associated as observed. Within each level of L, any other risk factor U is positively associated with A in about as many of these runs as it is negatively associated, even when U and L are strongly correlated. So across these runs the adjusted estimate is unbiased and the unadjusted estimate is badly biased. Across all runs, both are unbiased, but the adjusted one has smaller variance (it is the maximum likelihood estimator when L is available). The adjusted estimator is conditionally unbiased and unconditionally more efficient. Investigator 2 is right; retract.

Conditionality principle

inference about a parameter should be conditional on ancillary statistics, quantities like the observed L–A association whose distribution carries no information about the parameter of interest. Researchers who follow it call an estimator unbiased only if it can centre a valid Wald interval conditional on the ancillaries.

Faced with the argument “adjustment is unnecessary because, over all possible runs, the unadjusted estimate is unbiased”, such a researcher answers: why should the L–A associations in hypothetical other studies govern what I do in mine? In my study, L is a confounder. This reply is convincing in trials and observational studies alike, as long as the number of measured confounders is small. When it is large, strict conditionality stops being wise, which is the topic of 10.5.

Technical Points 10.2–10.5 · The formal machinery (brief)

10.2. The likelihood factors into f(Y | L, A) × f(A | L) × f(L). If the stratum-specific risk difference is constant and is the target, the last two factors share no parameters with the first, so the data {Ai, Li} are S-ancillary, and the conditionality principle says to condition on them. An exact ancillary is an S-ancillary whose distribution is fully known.

10.3. If the risk difference varies by L, the target is the standardized risk difference, which depends on the distribution of L, so no exact ancillary exists. In a trial the A–L odds ratio is known to be 1, so a standardized measure of the observed A–L association (observed odds ratio minus 1, divided by its standard error) is an approximate ancillary; conditional on it, the adjusted estimate is unbiased and the unadjusted one is not. Adding a continuity principle (conclusions should not jump when the setting changes by an arbitrarily small amount) gives the extended conditionality principle: condition on approximate ancillaries too.

10.4. The unadjusted estimator is conditionally biased exactly when it is unconditionally inefficient, and both happen exactly when L is associated with Y given A. So when a measured risk factor for Y is available, the adjusted (MLE) estimator is preferred.

10.5. Most researchers already follow the extended principle without knowing it. For the smoker stratum of Table 10.2, the variance of the stratum risk difference can be computed with the observed group sizes, (0.05 × 0.95)/80 + (0.05 × 0.95)/40 = 1.78 × 10−3, or with the sizes expected under randomization (60 and 60), giving 1.58 × 10−3. Nearly everyone picks the first (observed information), and that choice is mathematically equivalent to conditioning on the approximate ancillary. Combining the two strata’s variances as V0V1/(V0 + V1) gives the adjusted interval (−0.076, 0.076) quoted above.

10.5 The curse of dimensionality

Everything above assumed the number of strata of L is small relative to the sample size. Now suppose the investigators had measured 100 binary pre-treatment variables, L = (L1, …, L100). Their combinations form 2100 ≈ 1030 strata. Data with this many possible covariate combinations are called high-dimensional. For simplicity, assume the risk difference is the same in every stratum and equals 0.

The investigators now agree to follow the conditionality principle, since the unadjusted −0.15 is conditionally biased. But the adjusted interval turns out much wider than the unadjusted one, the opposite of what happened with one binary L. With 240 people and 2100 strata, almost every person sits alone in their own stratum. At most a few strata contain both a treated and an untreated person. Suppose exactly one stratum holds one treated and one untreated person, and no other stratum has both. That single pair is the only within-stratum comparison, and its risk difference can only be −1, 0 or 1. The adjusted 95% interval is (−1, 1), which tells you nothing. The unadjusted interval stays at (−0.26, −0.04), because measuring more covariates does not change it.

The adjusted estimator is only guaranteed to beat the unadjusted one when there are many individuals per unknown parameter (a common rule of thumb is at least 10, though the right ratio depends on the data).

Curse of dimensionality

conditional on all 100 covariates, the unadjusted estimator is still biased, yet the fully stratified estimator is uninformative. Conditionality is compelling in simple examples but cannot be carried through in high-dimensional problems, so it should not be treated as an absolute principle.

This applies equally to observational studies. The book calls finding a way out a difficult, active research area; Chapter 18 reviews it, and Chapters 11–17 supply the needed background on models.

Technical Point 10.6 · Can the curse be reversed?

Conditional on the exact ancillary {Ai, Li}, no estimator can give informative inference with this many strata. Unconditionally, the unadjusted estimator is unaffected by the dimension of L: it stays unbiased with interval width proportional to 1/√n. It achieves this by using outside information the stratified MLE does not use, namely the knowledge that A ⫫ L because treatment was randomized. The MLE’s intervals remain uninformative even unconditionally. Whether L can be used to build an estimator that is unconditionally unbiased and more efficient than the unadjusted one is taken up in Chapter 18; sometimes it can.

Technical Point 10.7 · Random variability and causal discovery

Fine Point 6.3 said that with infinite data and faithfulness, finding Z ⫫ Y | A (with time order Z, A, Y) implies the structure Z → A → Y, so A and Y share no common cause and the crude risk difference is causal. With finite samples, an empirical conditional odds ratio of exactly 1 is also consistent with faithful distributions where the true odds ratio is only very close to 1 and the causal effect of A is zero. Robins et al. (2003) showed that, as a result, no honest frequentist 95% interval for the effect can exclude 0, even if the estimated risk difference is 0.2 and 30 standard errors from zero. A Bayesian who puts real prior weight on such graphs can get a narrow interval that excludes 0, but the answer is very sensitive to the prior. The authors argue that the prior probability that A and Y have no common cause at all in an observational study should be essentially zero. Causal discovery from observational data therefore leans heavily on subject-matter knowledge.

Key takeaways

  • Identification asks what could be learned with infinite data; estimation deals with the finite sample you actually have. Both are needed.
  • Estimand, estimator and estimate are three different things. A 95% Wald interval is estimate ± 1.96 × SE; for 7/13 treated deaths it is (0.27, 0.81).
  • A 95% interval is a statement about the procedure’s coverage over repeated studies, not a 95% probability for the single interval. It reflects random error only, so in large studies it understates total uncertainty.
  • The usual binomial interval assumes random sampling from an infinite super-population. That is generally a fiction, so the interval is best read as a what-if statement.
  • Randomization removes confounding only on average. If a measured prognostic factor is by chance imbalanced, the adjusted estimate is both conditionally unbiased and more efficient: adjust (conditionality principle).
  • With many covariates, fully stratified adjustment becomes uninformative because strata contain only treated or only untreated people. This curse of dimensionality motivates the modelling approaches of Chapters 11–18.

Source

Study notes on Part I of Hernán MA, Robins JM. Causal Inference: What If. Boca Raton: Chapman & Hall/CRC (2020, 2024 edition). The book is free to read at miguelhernan.org/whatifbook. Page numbers refer to the book’s own numbering.