The problem of external validity (the "lab vs. world" gap). Controlled studies, especially randomized controlled trials (RCTs), are the gold standard for establishing causality. But to achieve control, you must isolate variables in an artificial, simplified environment. The hard problem is that this very act of control often strips away the real-world context, complexity, and interactions that determine how a treatment or phenomenon actually functions in the wild. What works perfectly in a double-blind RCT might fail or cause harm in a messy society because people aren't lab rats and the world isn't a sterile cage.
Example: A prestigious RCT proves a new antidepressant is highly effective. But the study excluded people with substance abuse issues, chronic pain, or more than two other medications—a large portion of real-world patients. When prescribed widely, the drug shows severe side effects and lower efficacy because it interacts with dozens of variables absent from the lab. The hard problem: The more perfectly you control a study to prove internal causality, the less it can tell you about external applicability. The quest for purity in evidence can render the evidence irrelevant to complex reality. Hard Problem of Controlled Studies.
by Enkigal January 24, 2026
Get the Hard Problem of Controlled Studies mug.