What is Simpson's paradox?

Simpson's paradox is a phenomenon in probability and statistics in which a trend appears in several groups of data but disappears or reverses when the groups are combined. It is often encountered in social science and medical statistics. The Stanford Encyclopedia of Philosophy adds that the cases are unproblematic mathematically, yet they strike many people as surprising. [1] [2]

The Stanford Encyclopedia uses numbers taken from Simpson's original 1951 example, a medical treatment tested on 52 people. Overall, the treatment and control groups both have a 50 percent success rate, so the treatment looks ineffective. Yet among men, treatment succeeds about 61 percent of the time against 57 percent for control, and among women about 44 percent against 40 percent. The treatment helps in each subgroup but shows no benefit in the combined data. [2]

The reason is the mix of people in each group. Men recover more often than women whether treated or not, but the treatment group is mostly female (27 versus 13) while the control group is mostly male (7 versus 5). The overall rate for each group is a weighted average of its subgroup rates, and these different weights cancel out the real advantage. The same source notes that if treatment were not correlated with gender, for instance by balancing the sexes across both conditions, the paradox could not arise. [2]

A nursing-research paper describes the paradox as an extreme condition of confounding, in which an apparent association between two variables reverses when the data are analyzed within each stratum of a confounding variable. It names two required conditions: an overlooked confounder with a strong effect on the outcome, and a disproportionate distribution of that confounder among the groups compared. Unequal group sizes alone are generally not a problem without such other issues. [3]

The same paper warns that in experimental research a spurious association can lead to the false conclusion that an intervention works, or does not, and waste further effort. A course page from the University of Illinois makes a similar point: without stratifying, you can see the opposite of what is true. [3] [4]

The Illinois course page describes a Berkeley admissions dataset. Most individual majors favored women or had very similar acceptance rates. However, more men applied to easier majors (A and B, with acceptance rates over 50 percent), while more women applied to harder majors (C to F, all below 50 percent). The choice of major was the confounder, so the combined figures made the university look biased when comparing within each major showed otherwise. [4]

The paradox covers more than outright reversals. The Stanford Encyclopedia says an association can emerge, disappear or reverse between a population and its subpopulations. The standard variety, which it calls association reversal following Samuels (1993), is the one most studied by psychologists of reasoning and by philosophers. The encyclopedia also says the paradox has implications for decision theory, causal inference and evolutionary biology. [2]